2023-07-14T10:40:39.444 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T10:40:39.445 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411 -- /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/orig.config.yaml 2023-07-14T10:40:39.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T10:40:39.612 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411 branch: wip-rishabh-2023Jul13 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7337411' kernel: client: branch: testing flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi175.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi182.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: aa6a80e6cc655fff48452a7eca02070e61d074fc timestamp: 2023-07-14_10:26:42 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-14T10:40:39.612 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa; will attempt to use it 2023-07-14T10:40:39.613 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks 2023-07-14T10:40:39.613 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-14T10:40:39.615 INFO:teuthology.task.internal:Checking packages... 2023-07-14T10:40:39.648 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '3fc0089cc1082313a98853d48b892acdc1385b8b' 2023-07-14T10:40:39.648 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-14T10:40:39.649 INFO:teuthology.packaging:ref: None 2023-07-14T10:40:39.649 INFO:teuthology.packaging:tag: None 2023-07-14T10:40:39.649 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13 2023-07-14T10:40:39.649 INFO:teuthology.packaging:sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T10:40:39.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13 2023-07-14T10:40:39.976 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4757.g3fc0089c 2023-07-14T10:40:39.977 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-14T10:40:39.983 INFO:teuthology.task.internal:no buildpackages task found 2023-07-14T10:40:39.983 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-14T10:40:40.023 INFO:teuthology.task.internal:Saving configuration 2023-07-14T10:40:40.044 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-14T10:40:40.049 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T10:40:40.079 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 10:27:35.757815', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T10:40:40.106 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 10:27:35.756098', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T10:40:40.130 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi182.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 10:27:35.759162', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T10:40:40.130 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-14T10:40:40.136 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-14T10:40:40.136 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-14T10:40:40.136 INFO:teuthology.task.internal:roles: ubuntu@smithi182.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-14T10:40:40.136 INFO:teuthology.run_tasks:Running task console_log... 2023-07-14T10:40:40.276 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f146579c550>, signals=[15]) 2023-07-14T10:40:40.277 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-14T10:40:40.285 INFO:teuthology.task.internal:Opening connections... 2023-07-14T10:40:40.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T10:40:40.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T10:40:40.357 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2023-07-14T10:40:40.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T10:40:40.427 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi182.front.sepia.ceph.com 2023-07-14T10:40:40.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T10:40:40.497 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-14T10:40:40.505 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2023-07-14T10:40:40.526 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2023-07-14T10:40:40.526 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2023-07-14T10:40:40.584 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T10:40:40.584 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T10:40:40.584 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2023-07-14T10:40:40.584 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T10:40:40.585 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T10:40:40.586 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T10:40:40.586 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T10:40:40.586 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T10:40:40.586 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T10:40:40.586 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T10:40:40.587 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2023-07-14T10:40:40.721 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2023-07-14T10:40:40.743 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2023-07-14T10:40:40.743 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2023-07-14T10:40:40.802 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T10:40:40.803 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T10:40:40.803 INFO:teuthology.orchestra.run.smithi175.stdout:ID="rhel" 2023-07-14T10:40:40.803 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="fedora" 2023-07-14T10:40:40.803 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="8.6" 2023-07-14T10:40:40.803 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el8" 2023-07-14T10:40:40.803 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T10:40:40.804 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2023-07-14T10:40:40.804 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T10:40:40.804 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T10:40:40.804 INFO:teuthology.orchestra.run.smithi175.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T10:40:40.804 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T10:40:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T10:40:40.805 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T10:40:40.805 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T10:40:40.805 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T10:40:40.805 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T10:40:40.806 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2023-07-14T10:40:40.833 DEBUG:teuthology.orchestra.run.smithi182:> uname -m 2023-07-14T10:40:40.854 INFO:teuthology.orchestra.run.smithi182.stdout:x86_64 2023-07-14T10:40:40.854 DEBUG:teuthology.orchestra.run.smithi182:> cat /etc/os-release 2023-07-14T10:40:40.913 INFO:teuthology.orchestra.run.smithi182.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T10:40:40.913 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T10:40:40.914 INFO:teuthology.orchestra.run.smithi182.stdout:ID="rhel" 2023-07-14T10:40:40.914 INFO:teuthology.orchestra.run.smithi182.stdout:ID_LIKE="fedora" 2023-07-14T10:40:40.914 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION_ID="8.6" 2023-07-14T10:40:40.914 INFO:teuthology.orchestra.run.smithi182.stdout:PLATFORM_ID="platform:el8" 2023-07-14T10:40:40.914 INFO:teuthology.orchestra.run.smithi182.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T10:40:40.914 INFO:teuthology.orchestra.run.smithi182.stdout:ANSI_COLOR="0;31" 2023-07-14T10:40:40.915 INFO:teuthology.orchestra.run.smithi182.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T10:40:40.915 INFO:teuthology.orchestra.run.smithi182.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T10:40:40.915 INFO:teuthology.orchestra.run.smithi182.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T10:40:40.915 INFO:teuthology.orchestra.run.smithi182.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T10:40:40.915 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T10:40:40.915 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T10:40:40.915 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T10:40:40.916 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T10:40:40.916 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T10:40:40.916 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server 2023-07-14T10:40:40.933 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-14T10:40:40.946 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-14T10:40:41.034 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-14T10:40:41.035 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2023-07-14T10:40:41.037 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2023-07-14T10:40:41.040 DEBUG:teuthology.orchestra.run.smithi182:> test '!' -e /home/ubuntu/cephtest 2023-07-14T10:40:41.058 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-14T10:40:41.064 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-14T10:40:41.065 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2023-07-14T10:40:41.095 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2023-07-14T10:40:41.097 DEBUG:teuthology.orchestra.run.smithi182:> test -z $(ls -A /var/lib/ceph) 2023-07-14T10:40:41.131 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-14T10:40:41.212 INFO:teuthology.run_tasks:Running task kernel... 2023-07-14T10:40:41.234 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-14T10:40:41.235 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-07-14T10:40:41.235 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-07-14T10:40:41.236 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T10:40:41.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T10:40:41.255 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-14T10:40:41.656 DEBUG:teuthology.task.kernel:sha1 for client.0 is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T10:40:41.656 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T10:40:41.656 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-07-14T10:40:41.676 INFO:teuthology.orchestra.run.smithi161.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-14T10:40:41.676 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T10:40:41.677 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T10:40:41.677 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum upgrade -y linux-firmware 2023-07-14T10:40:42.857 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T10:40:42.857 INFO:teuthology.orchestra.run.smithi161.stdout:Unable to read consumer identity 2023-07-14T10:40:44.289 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-07-14T10:40:47.173 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 5.4 MB/s | 16 MB 00:02 2023-07-14T10:40:50.648 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 435 kB/s | 24 kB 00:00 2023-07-14T10:40:51.346 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T10:40:51.347 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-07-14T10:40:51.348 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T10:40:51.479 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-14T10:40:51.479 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T10:40:51.884 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-14T10:40:51.884 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-14T10:40:51.886 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi161.front.sepia.ceph.com to complete... 2023-07-14T10:46:27.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14 10:46:27 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-14T10:46:27.781 INFO:teuthology.orchestra.run.smithi161.stderr:FINISHED --2023-07-14 10:46:27-- 2023-07-14T10:46:27.781 INFO:teuthology.orchestra.run.smithi161.stderr:Total wall clock time: 5m 36s 2023-07-14T10:46:27.781 INFO:teuthology.orchestra.run.smithi161.stderr:Downloaded: 1 files, 363M in 5m 35s (1.08 MB/s) 2023-07-14T10:46:27.783 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-14T10:46:27.783 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-14T10:46:29.392 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying... ######################################## 2023-07-14T10:46:29.448 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing... ######################################## 2023-07-14T10:46:29.537 INFO:teuthology.orchestra.run.smithi161.stdout:Updating / installing... 2023-07-14T10:47:07.554 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-14T10:47:31.304 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-14T10:47:31.304 DEBUG:teuthology.orchestra.run.smithi161:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-14T10:47:31.355 INFO:teuthology.orchestra.run.smithi161.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.355 INFO:teuthology.orchestra.run.smithi161.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.355 INFO:teuthology.orchestra.run.smithi161.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.355 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.355 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-14T10:47:31.356 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-14T10:47:31.356 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-14T10:47:31.356 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-14T10:47:31.356 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T10:47:31.356 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T10:47:31.356 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T10:47:31.356 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T10:47:31.357 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T10:47:31.357 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T10:47:31.357 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T10:47:31.357 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-14T10:47:31.357 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T10:47:31.357 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T10:47:31.357 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T10:47:31.358 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T10:47:31.359 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T10:47:31.359 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-14T10:47:31.359 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-14T10:47:31.359 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-14T10:47:31.359 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T10:47:31.359 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-14T10:47:31.359 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T10:47:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T10:47:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T10:47:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-14T10:47:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T10:47:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T10:47:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T10:47:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T10:47:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T10:47:31.362 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T10:47:31.362 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T10:47:31.362 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T10:47:31.362 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T10:47:31.362 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T10:47:31.362 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T10:47:31.362 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T10:47:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T10:47:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T10:47:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T10:47:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T10:47:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T10:47:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T10:47:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T10:47:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T10:47:31.365 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T10:47:31.365 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T10:47:31.365 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T10:47:31.365 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T10:47:31.365 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T10:47:31.365 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T10:47:31.365 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T10:47:31.366 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T10:47:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-14T10:47:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-14T10:47:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T10:47:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T10:47:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-14T10:47:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T10:47:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-14T10:47:31.368 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T10:47:31.369 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T10:47:31.369 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T10:47:31.369 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T10:47:31.369 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T10:47:31.369 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T10:47:31.369 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T10:47:31.369 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T10:47:31.370 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T10:47:31.371 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T10:47:31.371 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T10:47:31.371 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T10:47:31.371 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T10:47:31.371 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T10:47:31.371 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T10:47:31.371 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T10:47:31.372 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T10:47:31.372 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T10:47:31.372 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T10:47:31.372 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T10:47:31.372 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T10:47:31.372 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T10:47:31.372 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T10:47:31.373 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T10:47:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T10:47:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T10:47:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T10:47:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T10:47:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T10:47:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T10:47:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T10:47:31.375 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T10:47:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-14T10:47:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T10:47:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-14T10:47:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-14T10:47:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-14T10:47:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-14T10:47:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T10:47:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T10:47:31.378 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-14T10:47:31.379 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T10:47:31.380 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-14T10:47:31.380 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T10:47:31.380 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T10:47:31.380 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T10:47:31.380 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T10:47:31.380 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T10:47:31.380 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T10:47:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-14T10:47:31.382 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T10:47:31.382 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T10:47:31.382 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T10:47:31.382 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-14T10:47:31.382 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-14T10:47:31.382 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T10:47:31.382 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-14T10:47:31.383 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T10:47:31.384 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T10:47:31.384 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-14T10:47:31.384 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-14T10:47:31.384 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T10:47:31.384 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-14T10:47:31.384 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-14T10:47:31.384 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T10:47:31.385 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T10:47:31.385 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-14T10:47:31.385 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-14T10:47:31.385 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-14T10:47:31.385 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T10:47:31.385 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-14T10:47:31.386 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T10:47:31.386 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T10:47:31.386 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T10:47:31.386 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-14T10:47:31.386 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T10:47:31.386 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-14T10:47:31.386 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T10:47:31.387 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-14T10:47:31.387 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-14T10:47:31.387 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-14T10:47:31.387 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-14T10:47:31.387 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T10:47:31.387 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-14T10:47:31.387 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T10:47:31.388 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T10:47:31.389 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T10:47:31.389 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T10:47:31.389 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T10:47:31.389 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T10:47:31.389 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T10:47:31.389 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T10:47:31.389 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T10:47:31.390 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T10:47:31.391 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T10:47:31.392 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T10:47:31.392 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T10:47:31.392 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T10:47:31.392 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T10:47:31.392 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T10:47:31.392 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T10:47:31.392 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T10:47:31.393 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T10:47:31.394 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T10:47:31.395 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T10:47:31.395 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T10:47:31.395 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T10:47:31.395 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T10:47:31.395 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T10:47:31.396 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T10:47:31.396 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T10:47:31.396 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T10:47:31.396 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T10:47:31.396 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T10:47:31.396 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T10:47:31.396 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T10:47:31.397 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T10:47:31.397 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T10:47:31.397 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T10:47:31.397 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T10:47:31.397 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T10:47:31.397 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T10:47:31.397 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T10:47:31.398 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T10:47:31.399 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T10:47:31.400 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-14T10:47:31.400 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T10:47:31.400 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T10:47:31.400 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T10:47:31.400 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T10:47:31.400 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T10:47:31.400 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T10:47:31.401 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T10:47:31.402 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T10:47:31.402 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T10:47:31.402 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T10:47:31.402 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T10:47:31.402 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T10:47:31.402 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T10:47:31.402 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T10:47:31.403 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T10:47:31.403 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T10:47:31.403 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T10:47:31.403 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T10:47:31.403 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T10:47:31.403 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T10:47:31.403 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T10:47:31.404 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T10:47:31.405 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T10:47:31.405 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T10:47:31.405 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T10:47:31.405 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T10:47:31.405 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T10:47:31.405 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T10:47:31.405 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T10:47:31.406 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T10:47:31.407 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T10:47:31.407 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T10:47:31.407 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T10:47:31.407 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T10:47:31.407 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T10:47:31.407 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T10:47:31.407 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T10:47:31.408 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T10:47:31.409 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T10:47:31.409 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T10:47:31.409 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T10:47:31.409 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T10:47:31.409 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T10:47:31.409 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T10:47:31.409 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T10:47:31.410 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T10:47:31.411 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T10:47:31.411 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T10:47:31.411 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T10:47:31.411 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T10:47:31.411 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T10:47:31.411 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T10:47:31.411 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T10:47:31.412 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T10:47:31.413 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T10:47:31.413 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T10:47:31.413 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T10:47:31.413 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T10:47:31.413 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T10:47:31.413 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T10:47:31.413 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T10:47:31.414 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T10:47:31.414 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T10:47:31.414 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T10:47:31.414 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T10:47:31.414 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T10:47:31.414 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T10:47:31.414 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T10:47:31.415 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T10:47:31.416 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T10:47:31.416 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T10:47:31.416 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T10:47:31.416 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T10:47:31.416 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T10:47:31.416 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T10:47:31.416 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-14T10:47:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T10:47:31.418 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T10:47:31.419 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T10:47:31.419 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T10:47:31.419 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T10:47:31.419 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T10:47:31.419 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T10:47:31.419 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T10:47:31.419 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T10:47:31.420 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T10:47:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T10:47:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T10:47:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-14T10:47:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T10:47:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-14T10:47:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T10:47:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T10:47:31.422 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T10:47:31.423 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T10:47:31.423 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-14T10:47:31.423 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-14T10:47:31.423 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T10:47:31.423 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-14T10:47:31.423 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T10:47:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-14T10:47:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T10:47:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-14T10:47:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T10:47:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-14T10:47:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-14T10:47:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-14T10:47:31.425 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T10:47:31.426 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T10:47:31.427 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-14T10:47:31.427 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T10:47:31.427 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-14T10:47:31.427 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T10:47:31.427 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T10:47:31.428 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T10:47:31.428 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T10:47:31.428 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T10:47:31.428 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T10:47:31.428 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T10:47:31.428 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T10:47:31.428 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-14T10:47:31.429 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T10:47:31.429 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-14T10:47:31.429 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T10:47:31.429 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T10:47:31.429 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T10:47:31.429 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T10:47:31.429 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T10:47:31.430 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T10:47:31.431 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T10:47:31.432 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T10:47:31.432 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-14T10:47:31.432 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-14T10:47:31.432 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T10:47:31.432 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T10:47:31.432 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T10:47:31.432 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T10:47:31.433 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T10:47:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T10:47:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T10:47:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T10:47:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T10:47:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T10:47:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T10:47:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-14T10:47:31.435 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-14T10:47:31.436 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T10:47:31.436 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T10:47:31.436 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T10:47:31.436 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T10:47:31.436 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T10:47:31.436 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T10:47:31.436 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T10:47:31.437 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T10:47:31.438 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-14T10:47:31.438 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-14T10:47:31.438 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T10:47:31.438 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T10:47:31.438 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T10:47:31.438 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T10:47:31.438 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T10:47:31.439 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T10:47:31.440 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T10:47:31.440 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T10:47:31.440 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-14T10:47:31.440 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T10:47:31.440 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T10:47:31.440 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-14T10:47:31.440 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T10:47:31.441 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T10:47:31.441 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T10:47:31.441 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-14T10:47:31.441 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-14T10:47:31.441 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-14T10:47:31.441 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-14T10:47:31.441 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T10:47:31.442 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-14T10:47:31.443 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-14T10:47:31.443 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T10:47:31.443 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-14T10:47:31.444 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-14T10:47:31.444 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T10:47:31.444 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T10:47:31.444 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T10:47:31.444 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-14T10:47:31.444 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T10:47:31.444 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T10:47:31.445 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-14T10:47:31.446 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T10:47:31.447 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-14T10:47:31.447 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T10:47:31.447 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-14T10:47:31.447 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T10:47:31.447 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-14T10:47:31.447 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T10:47:31.447 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-14T10:47:31.448 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T10:47:31.449 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-14T10:47:31.449 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T10:47:31.449 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T10:47:31.449 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-14T10:47:31.449 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T10:47:31.449 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T10:47:31.449 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-14T10:47:31.450 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T10:47:31.450 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T10:47:31.450 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-14T10:47:31.450 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T10:47:31.450 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T10:47:31.450 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-14T10:47:31.450 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-14T10:47:31.451 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T10:47:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-14T10:47:31.453 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-14T10:47:31.453 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T10:47:31.453 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T10:47:31.453 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T10:47:31.453 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T10:47:31.453 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T10:47:31.453 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T10:47:31.454 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T10:47:31.455 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-14T10:47:31.456 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T10:47:31.456 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-14T10:47:31.456 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T10:47:31.456 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-14T10:47:31.456 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T10:47:31.456 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-14T10:47:31.456 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T10:47:31.457 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-14T10:47:31.458 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T10:47:31.458 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T10:47:31.458 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T10:47:31.458 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-14T10:47:31.458 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-14T10:47:31.458 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T10:47:31.458 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-14T10:47:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T10:47:31.460 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-14T10:47:31.460 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T10:47:31.460 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T10:47:31.460 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-14T10:47:31.460 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T10:47:31.460 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-14T10:47:31.460 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T10:47:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T10:47:31.462 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T10:47:31.463 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-14T10:47:31.463 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-14T10:47:31.463 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-14T10:47:31.463 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T10:47:31.463 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-14T10:47:31.463 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T10:47:31.463 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T10:47:31.464 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-14T10:47:31.465 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-14T10:47:31.465 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T10:47:31.465 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T10:47:31.465 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-14T10:47:31.465 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T10:47:31.465 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T10:47:31.465 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T10:47:31.466 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T10:47:31.467 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-14T10:47:31.467 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T10:47:31.467 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T10:47:31.467 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-14T10:47:31.467 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T10:47:31.467 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T10:47:31.467 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T10:47:31.468 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T10:47:31.469 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-14T10:47:31.469 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T10:47:31.469 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T10:47:31.469 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T10:47:31.469 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T10:47:31.469 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T10:47:31.469 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T10:47:31.470 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T10:47:31.471 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-14T10:47:31.471 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T10:47:31.471 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T10:47:31.471 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T10:47:31.471 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T10:47:31.471 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T10:47:31.471 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T10:47:31.472 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T10:47:31.473 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T10:47:31.473 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T10:47:31.473 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T10:47:31.473 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T10:47:31.473 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-14T10:47:31.473 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T10:47:31.473 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T10:47:31.474 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-14T10:47:31.475 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-14T10:47:31.475 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T10:47:31.475 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T10:47:31.475 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T10:47:31.475 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-14T10:47:31.475 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T10:47:31.475 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-14T10:47:31.476 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-14T10:47:31.477 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T10:47:31.478 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T10:47:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T10:47:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-14T10:47:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-14T10:47:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T10:47:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-14T10:47:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-14T10:47:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T10:47:31.480 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T10:47:31.481 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T10:47:31.482 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T10:47:31.483 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T10:47:31.484 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T10:47:31.484 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T10:47:31.484 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-14T10:47:31.484 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T10:47:31.484 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T10:47:31.484 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-14T10:47:31.484 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-14T10:47:31.485 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T10:47:31.485 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T10:47:31.485 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T10:47:31.485 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T10:47:31.485 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-14T10:47:31.485 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T10:47:31.486 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T10:47:31.486 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-14T10:47:31.486 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T10:47:31.486 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T10:47:31.486 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T10:47:31.486 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T10:47:31.486 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T10:47:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T10:47:31.488 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T10:47:31.488 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T10:47:31.488 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T10:47:31.488 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T10:47:31.488 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T10:47:31.488 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T10:47:31.488 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T10:47:31.489 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T10:47:31.490 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T10:47:31.491 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T10:47:31.492 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T10:47:31.492 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-14T10:47:31.492 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T10:47:31.492 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T10:47:31.492 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-14T10:47:31.492 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T10:47:31.492 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-14T10:47:31.493 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T10:47:31.494 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-14T10:47:31.495 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-14T10:47:31.495 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-14T10:47:31.495 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-14T10:47:31.495 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-14T10:47:31.495 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T10:47:31.495 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-14T10:47:31.496 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T10:47:31.496 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T10:47:31.496 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-14T10:47:31.496 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T10:47:31.496 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T10:47:31.496 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-14T10:47:31.496 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T10:47:31.497 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T10:47:31.497 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T10:47:31.497 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T10:47:31.497 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T10:47:31.497 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T10:47:31.497 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T10:47:31.497 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T10:47:31.498 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T10:47:31.499 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T10:47:31.500 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-14T10:47:31.500 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T10:47:31.500 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-14T10:47:31.500 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T10:47:31.500 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T10:47:31.500 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-14T10:47:31.500 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T10:47:31.501 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T10:47:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-14T10:47:31.503 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T10:47:31.504 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-14T10:47:31.504 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-14T10:47:31.504 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T10:47:31.504 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-14T10:47:31.504 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T10:47:31.504 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T10:47:31.504 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-14T10:47:31.505 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T10:47:31.506 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T10:47:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T10:47:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T10:47:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T10:47:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T10:47:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T10:47:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T10:47:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-14T10:47:31.508 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-14T10:47:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T10:47:31.510 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-14T10:47:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T10:47:31.512 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-14T10:47:31.512 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T10:47:31.512 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T10:47:31.512 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-14T10:47:31.512 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T10:47:31.512 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T10:47:31.512 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T10:47:31.513 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T10:47:31.514 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T10:47:31.515 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T10:47:31.516 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T10:47:31.516 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T10:47:31.516 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T10:47:31.516 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T10:47:31.516 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T10:47:31.516 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T10:47:31.516 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T10:47:31.517 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T10:47:31.518 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T10:47:31.519 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T10:47:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T10:47:31.521 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-14T10:47:31.521 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T10:47:31.521 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T10:47:31.521 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T10:47:31.521 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T10:47:31.521 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T10:47:31.521 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T10:47:31.522 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T10:47:31.523 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T10:47:31.523 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-14T10:47:31.523 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T10:47:31.523 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T10:47:31.523 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-14T10:47:31.524 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T10:47:31.525 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T10:47:31.525 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T10:47:31.525 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T10:47:31.525 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-14T10:47:31.525 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-14T10:47:31.525 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T10:47:31.525 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-14T10:47:31.526 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-14T10:47:31.527 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-14T10:47:31.527 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-14T10:47:31.527 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-14T10:47:31.527 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T10:47:31.527 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T10:47:31.527 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-14T10:47:31.528 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T10:47:31.528 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T10:47:31.528 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T10:47:31.528 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T10:47:31.528 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T10:47:31.528 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T10:47:31.528 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T10:47:31.529 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T10:47:31.530 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T10:47:31.531 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T10:47:31.531 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T10:47:31.531 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T10:47:31.531 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T10:47:31.531 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T10:47:31.531 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T10:47:31.531 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T10:47:31.532 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-14T10:47:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-14T10:47:31.534 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T10:47:31.535 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-14T10:47:31.536 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T10:47:31.536 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-14T10:47:31.536 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T10:47:31.536 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-14T10:47:31.536 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T10:47:31.536 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-14T10:47:31.536 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T10:47:31.537 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T10:47:31.538 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-14T10:47:31.539 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T10:47:31.539 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-14T10:47:31.539 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T10:47:31.539 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-14T10:47:31.539 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T10:47:31.539 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-14T10:47:31.539 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-14T10:47:31.540 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T10:47:31.541 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T10:47:31.542 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-14T10:47:31.543 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T10:47:31.543 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T10:47:31.543 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T10:47:31.543 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T10:47:31.543 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T10:47:31.543 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T10:47:31.543 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T10:47:31.544 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T10:47:31.545 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T10:47:31.546 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T10:47:31.547 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T10:47:31.547 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T10:47:31.547 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T10:47:31.547 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T10:47:31.547 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T10:47:31.547 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T10:47:31.547 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T10:47:31.548 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-14T10:47:31.549 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-14T10:47:31.550 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-14T10:47:31.551 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-14T10:47:31.552 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T10:47:31.553 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-14T10:47:31.553 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T10:47:31.553 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-14T10:47:31.553 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T10:47:31.553 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-14T10:47:31.553 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T10:47:31.553 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-14T10:47:31.554 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-14T10:47:31.555 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T10:47:31.556 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T10:47:31.557 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T10:47:31.557 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T10:47:31.557 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-14T10:47:31.557 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-14T10:47:31.557 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-14T10:47:31.557 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T10:47:31.557 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T10:47:31.558 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T10:47:31.559 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-14T10:47:31.560 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T10:47:31.561 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-14T10:47:31.561 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T10:47:31.561 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T10:47:31.561 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-14T10:47:31.561 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T10:47:31.561 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T10:47:31.561 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T10:47:31.562 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T10:47:31.563 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T10:47:31.564 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T10:47:31.564 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T10:47:31.564 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-14T10:47:31.564 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T10:47:31.564 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T10:47:31.564 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-14T10:47:31.564 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T10:47:31.565 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T10:47:31.566 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T10:47:31.567 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-14T10:47:31.567 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-14T10:47:31.567 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T10:47:31.567 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T10:47:31.567 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T10:47:31.567 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-14T10:47:31.567 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T10:47:31.568 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T10:47:31.569 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T10:47:31.570 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T10:47:31.571 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T10:47:31.572 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T10:47:31.572 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T10:47:31.572 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T10:47:31.572 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T10:47:31.572 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T10:47:31.572 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T10:47:31.572 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T10:47:31.573 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T10:47:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T10:47:31.575 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T10:47:31.576 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T10:47:31.576 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T10:47:31.576 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T10:47:31.576 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T10:47:31.576 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T10:47:31.576 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T10:47:31.576 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T10:47:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-14T10:47:31.578 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T10:47:31.579 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T10:47:31.580 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-14T10:47:31.580 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T10:47:31.580 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-14T10:47:31.580 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T10:47:31.580 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-14T10:47:31.580 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T10:47:31.580 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T10:47:31.581 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T10:47:31.582 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T10:47:31.583 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T10:47:31.583 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-14T10:47:31.583 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T10:47:31.583 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T10:47:31.583 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T10:47:31.583 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T10:47:31.583 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T10:47:31.584 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T10:47:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T10:47:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T10:47:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T10:47:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T10:47:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T10:47:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T10:47:31.586 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T10:47:31.586 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T10:47:31.586 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T10:47:31.586 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T10:47:31.587 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T10:47:31.588 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T10:47:31.588 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T10:47:31.588 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T10:47:31.588 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T10:47:31.588 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T10:47:31.588 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T10:47:31.588 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T10:47:31.589 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T10:47:31.590 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T10:47:31.591 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T10:47:31.591 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T10:47:31.591 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T10:47:31.591 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T10:47:31.591 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-14T10:47:31.591 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T10:47:31.591 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T10:47:31.592 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T10:47:31.593 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T10:47:31.594 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T10:47:31.595 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T10:47:31.595 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T10:47:31.595 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T10:47:31.595 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T10:47:31.595 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T10:47:31.595 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T10:47:31.595 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T10:47:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T10:47:31.597 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T10:47:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T10:47:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T10:47:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T10:47:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T10:47:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T10:47:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T10:47:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T10:47:31.599 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T10:47:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-14T10:47:31.601 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T10:47:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-14T10:47:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T10:47:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T10:47:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T10:47:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T10:47:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-14T10:47:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T10:47:31.603 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T10:47:31.604 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T10:47:31.605 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T10:47:31.606 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T10:47:31.606 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T10:47:31.606 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T10:47:31.606 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T10:47:31.606 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T10:47:31.606 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T10:47:31.606 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T10:47:31.607 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T10:47:31.608 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-14T10:47:31.609 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T10:47:31.610 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T10:47:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-14T10:47:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T10:47:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-14T10:47:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-14T10:47:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-14T10:47:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-14T10:47:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-14T10:47:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T10:47:31.613 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T10:47:31.614 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T10:47:31.614 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T10:47:31.614 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T10:47:31.614 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T10:47:31.614 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T10:47:31.614 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T10:47:31.614 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T10:47:31.615 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.615 DEBUG:teuthology.orchestra.run.smithi161:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-14T10:47:31.642 INFO:teuthology.orchestra.run.smithi161.stdout:postinstall scriptlet (using /bin/sh): 2023-07-14T10:47:31.642 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-14T10:47:31.642 INFO:teuthology.orchestra.run.smithi161.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T10:47:31.643 INFO:teuthology.orchestra.run.smithi161.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T10:47:31.643 INFO:teuthology.orchestra.run.smithi161.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.643 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T10:47:31.643 INFO:teuthology.orchestra.run.smithi161.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T10:47:31.643 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2023-07-14T10:47:31.643 INFO:teuthology.orchestra.run.smithi161.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-14T10:47:31.643 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.644 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2023-07-14T10:47:31.645 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2023-07-14T10:47:31.723 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2023-07-14T10:47:31.723 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2023-07-14T10:47:31.723 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.02 2023-07-14T10:47:31.723 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 123.el8 2023-07-14T10:47:31.724 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2023-07-14T10:47:31.724 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-14T10:47:31.724 INFO:teuthology.orchestra.run.smithi161.stdout:Group : System Environment/Base 2023-07-14T10:47:31.724 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 9265450 2023-07-14T10:47:31.724 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2023-07-14T10:47:31.724 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-14T10:47:31.724 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Relocations : (not relocatable) 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : Red Hat, Inc. 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : Red Hat, Inc. 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2023-07-14T10:47:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2023-07-14T10:47:31.726 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T10:47:31.726 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-14T10:47:31.726 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-14T10:47:31.726 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-14T10:47:31.726 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2023-07-14T10:47:31.726 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T10:47:31.726 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2023-07-14T10:47:31.727 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-14T10:47:31.728 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:31.728 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-14T10:47:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2023-07-14T10:47:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:done 2023-07-14T10:47:37.895 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-07-14T10:47:37.912 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.EhY4YPU3HT 2023-07-14T10:47:37.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EhY4YPU3HT 2023-07-14T10:47:38.000 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.EhY4YPU3HT 2023-07-14T10:47:38.124 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.EhY4YPU3HT is 6KB 2023-07-14T10:47:38.145 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.EhY4YPU3HT 2023-07-14T10:47:38.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2023-07-14T10:47:38.225 INFO:teuthology.orchestra.run.smithi161.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-14T10:47:38.225 INFO:teuthology.orchestra.run.smithi161.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-14T10:47:38.226 INFO:teuthology.orchestra.run.smithi161.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-14T10:47:38.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-14T10:47:38.422 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2023-07-14T10:48:08.425 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-14T10:48:08.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T10:48:08.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T10:48:26.992 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-07-14T10:48:35.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T10:48:35.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T10:48:39.056 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-07-14T10:48:51.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T10:48:51.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T10:49:09.492 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-07-14T10:49:24.493 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T10:49:24.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T10:49:24.863 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-07-14T10:49:25.431 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T10:49:25.431 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T10:49:25.432 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-07-14T10:49:25.473 INFO:teuthology.orchestra.run.smithi161.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T10:49:25.474 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T10:49:25.474 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T10:49:25.474 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-14T10:49:26.475 DEBUG:teuthology.parallel:result is None 2023-07-14T10:49:26.476 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-14T10:49:26.485 INFO:teuthology.task.internal:Creating test directory... 2023-07-14T10:49:26.486 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T10:49:26.491 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T10:49:26.494 DEBUG:teuthology.orchestra.run.smithi182:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T10:49:26.530 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-14T10:49:26.536 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-14T10:49:26.560 INFO:teuthology.task.internal:Creating archive directory... 2023-07-14T10:49:26.560 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T10:49:26.578 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T10:49:26.582 DEBUG:teuthology.orchestra.run.smithi182:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T10:49:26.684 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-14T10:49:26.694 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-14T10:49:26.694 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T10:49:26.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T10:49:26.713 DEBUG:teuthology.orchestra.run.smithi182:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T10:49:26.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T10:49:26.732 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T10:49:26.737 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T10:49:26.755 DEBUG:teuthology.orchestra.run.smithi182:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T10:49:26.802 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T10:49:26.822 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T10:49:26.842 INFO:teuthology.orchestra.run.smithi182.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T10:49:26.857 INFO:teuthology.orchestra.run.smithi182.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T10:49:26.935 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T10:49:26.987 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T10:49:26.990 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-14T10:49:27.003 INFO:teuthology.task.internal:Configuring sudo... 2023-07-14T10:49:27.004 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T10:49:27.008 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T10:49:27.011 DEBUG:teuthology.orchestra.run.smithi182:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T10:49:27.075 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-14T10:49:27.086 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-14T10:49:27.086 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T10:49:27.091 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T10:49:27.093 DEBUG:teuthology.orchestra.run.smithi182:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T10:49:27.129 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T10:49:27.246 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T10:49:27.326 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T10:49:27.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T10:49:27.467 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T10:49:27.468 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T10:49:27.552 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T10:49:27.603 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T10:49:27.657 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T10:49:27.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T10:49:27.788 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T10:49:27.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T10:49:27.900 DEBUG:teuthology.orchestra.run.smithi182:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T10:49:27.952 DEBUG:teuthology.orchestra.run.smithi182:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T10:49:28.009 DEBUG:teuthology.orchestra.run.smithi182:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T10:49:28.058 DEBUG:teuthology.orchestra.run.smithi182:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T10:49:28.099 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T10:49:28.100 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T10:49:28.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2023-07-14T10:49:28.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2023-07-14T10:49:28.176 DEBUG:teuthology.orchestra.run.smithi182:> sudo service rsyslog restart 2023-07-14T10:49:28.252 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T10:49:28.304 INFO:teuthology.orchestra.run.smithi182.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T10:49:28.310 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T10:49:28.703 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-14T10:49:28.712 INFO:teuthology.task.internal:Starting timer... 2023-07-14T10:49:28.713 INFO:teuthology.run_tasks:Running task pcp... 2023-07-14T10:49:28.827 INFO:teuthology.run_tasks:Running task selinux... 2023-07-14T10:49:28.862 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-14T10:49:28.863 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2023-07-14T10:49:28.986 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2023-07-14T10:49:28.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2023-07-14T10:49:29.057 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2023-07-14T10:49:29.059 DEBUG:teuthology.orchestra.run.smithi182:> sudo service auditd rotate 2023-07-14T10:49:29.128 INFO:teuthology.orchestra.run.smithi182.stdout:Rotating logs: 2023-07-14T10:49:29.130 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-14T10:49:29.130 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2023-07-14T10:49:29.181 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2023-07-14T10:49:29.182 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2023-07-14T10:49:29.203 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2023-07-14T10:49:29.204 DEBUG:teuthology.orchestra.run.smithi182:> /usr/sbin/getenforce 2023-07-14T10:49:29.223 INFO:teuthology.orchestra.run.smithi182.stdout:Permissive 2023-07-14T10:49:29.224 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi182.front.sepia.ceph.com': 'permissive'} 2023-07-14T10:49:29.224 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T10:49:29.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T10:49:29.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T10:49:29.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T10:49:29.324 DEBUG:teuthology.orchestra.run.smithi182:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T10:49:29.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T10:49:29.353 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-14T10:49:29.353 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2023-07-14T10:49:29.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2023-07-14T10:49:29.449 DEBUG:teuthology.orchestra.run.smithi182:> sudo /usr/sbin/setenforce permissive 2023-07-14T10:49:29.477 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-14T10:49:29.486 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-14T10:49:29.487 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-14T10:49:29.509 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-14T10:49:29.510 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi161.front.sepia.ceph.com,smithi175.front.sepia.ceph.com,smithi182.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-14T11:03:01.852 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com'), Remote(name='ubuntu@smithi182.front.sepia.ceph.com')] 2023-07-14T11:03:01.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T11:03:01.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:03:01.944 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-07-14T11:03:02.057 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T11:03:02.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-07-14T11:03:02.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:03:02.126 DEBUG:teuthology.orchestra.run.smithi175:> true 2023-07-14T11:03:02.195 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-07-14T11:03:02.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com' 2023-07-14T11:03:02.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:03:02.266 DEBUG:teuthology.orchestra.run.smithi182:> true 2023-07-14T11:03:02.294 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi182.front.sepia.ceph.com' 2023-07-14T11:03:02.296 INFO:teuthology.run_tasks:Running task clock... 2023-07-14T11:03:02.308 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-14T11:03:02.309 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:03:02.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:03:02.314 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:03:02.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:03:02.317 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:03:02.317 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:03:02.348 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T11:03:02.364 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T11:03:02.367 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T11:03:02.383 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T11:03:02.386 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T11:03:02.421 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T11:03:02.448 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2023-07-14T11:03:02.460 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2023-07-14T11:03:02.468 INFO:teuthology.orchestra.run.smithi182.stderr:sudo: ntpd: command not found 2023-07-14T11:03:02.474 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T11:03:02.480 INFO:teuthology.orchestra.run.smithi182.stdout:506 Cannot talk to daemon 2023-07-14T11:03:02.488 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T11:03:02.495 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T11:03:02.509 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T11:03:02.513 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2023-07-14T11:03:02.542 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2023-07-14T11:03:02.552 INFO:teuthology.orchestra.run.smithi175.stderr:bash: ntpq: command not found 2023-07-14T11:03:02.555 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T11:03:02.556 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-07-14T11:03:02.556 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.556 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.556 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.556 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.576 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T11:03:02.581 INFO:teuthology.orchestra.run.smithi182.stderr:bash: ntpq: command not found 2023-07-14T11:03:02.584 INFO:teuthology.orchestra.run.smithi182.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T11:03:02.584 INFO:teuthology.orchestra.run.smithi182.stdout:=============================================================================== 2023-07-14T11:03:02.585 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.585 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.585 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.585 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.610 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T11:03:02.696 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-07-14T11:03:02.703 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T11:03:02.704 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-07-14T11:03:02.704 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.704 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.704 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.704 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:03:02.705 INFO:teuthology.run_tasks:Running task pexec... 2023-07-14T11:03:02.715 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-14T11:03:02.716 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T11:03:02.716 DEBUG:teuthology.orchestra.run.smithi161:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T11:03:02.717 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi175.front.sepia.ceph.com 2023-07-14T11:03:02.717 DEBUG:teuthology.orchestra.run.smithi175:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T11:03:02.717 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi182.front.sepia.ceph.com 2023-07-14T11:03:02.718 DEBUG:teuthology.orchestra.run.smithi182:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T11:03:03.089 INFO:teuthology.orchestra.run.smithi175.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:03.089 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:03.201 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:04.575 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:05:02 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:04.637 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:05:02 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:04.919 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:12 ago on Fri 14 Jul 2023 11:00:52 AM UTC. 2023-07-14T11:03:06.446 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-07-14T11:03:06.447 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-07-14T11:03:06.448 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-07-14T11:03:06.448 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-07-14T11:03:06.448 INFO:teuthology.orchestra.run.smithi175.stdout:Resetting modules: 2023-07-14T11:03:06.448 INFO:teuthology.orchestra.run.smithi175.stdout: container-tools 2023-07-14T11:03:06.449 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:03:06.449 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-07-14T11:03:06.449 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-07-14T11:03:06.449 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:03:06.516 INFO:teuthology.orchestra.run.smithi182.stdout:Dependencies resolved. 2023-07-14T11:03:06.517 INFO:teuthology.orchestra.run.smithi182.stdout:================================================================================ 2023-07-14T11:03:06.517 INFO:teuthology.orchestra.run.smithi182.stdout: Package Architecture Version Repository Size 2023-07-14T11:03:06.517 INFO:teuthology.orchestra.run.smithi182.stdout:================================================================================ 2023-07-14T11:03:06.518 INFO:teuthology.orchestra.run.smithi182.stdout:Resetting modules: 2023-07-14T11:03:06.518 INFO:teuthology.orchestra.run.smithi182.stdout: container-tools 2023-07-14T11:03:06.518 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:03:06.518 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction Summary 2023-07-14T11:03:06.518 INFO:teuthology.orchestra.run.smithi182.stdout:================================================================================ 2023-07-14T11:03:06.518 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:03:06.713 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T11:03:06.714 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T11:03:06.715 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2023-07-14T11:03:06.715 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T11:03:06.715 INFO:teuthology.orchestra.run.smithi161.stdout:Resetting modules: 2023-07-14T11:03:06.715 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools 2023-07-14T11:03:06.715 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:03:06.715 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T11:03:06.716 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T11:03:06.716 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:03:07.424 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:05:04 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:07.469 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:05:05 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:07.961 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:15 ago on Fri 14 Jul 2023 11:00:52 AM UTC. 2023-07-14T11:03:15.561 INFO:teuthology.orchestra.run.smithi182.stdout:Complete! 2023-07-14T11:03:16.032 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:17.669 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-07-14T11:03:17.690 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:05:15 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:18.117 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T11:03:18.141 INFO:teuthology.orchestra.run.smithi175.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:18.713 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:19.404 INFO:teuthology.orchestra.run.smithi182.stdout:Dependencies resolved. 2023-07-14T11:03:19.497 INFO:teuthology.orchestra.run.smithi182.stdout:============================================================================================================ 2023-07-14T11:03:19.497 INFO:teuthology.orchestra.run.smithi182.stdout: Package Arch Version Repository Size 2023-07-14T11:03:19.497 INFO:teuthology.orchestra.run.smithi182.stdout:============================================================================================================ 2023-07-14T11:03:19.497 INFO:teuthology.orchestra.run.smithi182.stdout:Installing group/module packages: 2023-07-14T11:03:19.497 INFO:teuthology.orchestra.run.smithi182.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T11:03:19.497 INFO:teuthology.orchestra.run.smithi182.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T11:03:19.498 INFO:teuthology.orchestra.run.smithi182.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T11:03:19.498 INFO:teuthology.orchestra.run.smithi182.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T11:03:19.498 INFO:teuthology.orchestra.run.smithi182.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T11:03:19.498 INFO:teuthology.orchestra.run.smithi182.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T11:03:19.498 INFO:teuthology.orchestra.run.smithi182.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T11:03:19.498 INFO:teuthology.orchestra.run.smithi182.stdout:Installing dependencies: 2023-07-14T11:03:19.498 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout:Installing module profiles: 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout: container-tools/common 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout:Enabling module streams: 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout: container-tools rhel8 2023-07-14T11:03:19.499 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:03:19.500 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction Summary 2023-07-14T11:03:19.500 INFO:teuthology.orchestra.run.smithi182.stdout:============================================================================================================ 2023-07-14T11:03:19.500 INFO:teuthology.orchestra.run.smithi182.stdout:Install 10 Packages 2023-07-14T11:03:19.500 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:03:19.501 INFO:teuthology.orchestra.run.smithi182.stdout:Total download size: 18 M 2023-07-14T11:03:19.501 INFO:teuthology.orchestra.run.smithi182.stdout:Installed size: 65 M 2023-07-14T11:03:19.501 INFO:teuthology.orchestra.run.smithi182.stdout:Downloading Packages: 2023-07-14T11:03:19.659 INFO:teuthology.orchestra.run.smithi182.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 590 kB/s | 94 kB 00:00 2023-07-14T11:03:19.713 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:05:17 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:19.817 INFO:teuthology.orchestra.run.smithi182.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 78 kB/s | 25 kB 00:00 2023-07-14T11:03:19.834 INFO:teuthology.orchestra.run.smithi182.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 122 kB/s | 41 kB 00:00 2023-07-14T11:03:19.876 INFO:teuthology.orchestra.run.smithi182.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 227 kB/s | 49 kB 00:00 2023-07-14T11:03:20.034 INFO:teuthology.orchestra.run.smithi182.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.4 MB/s | 524 kB 00:00 2023-07-14T11:03:20.077 INFO:teuthology.orchestra.run.smithi182.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 746 kB/s | 149 kB 00:00 2023-07-14T11:03:20.310 INFO:teuthology.orchestra.run.smithi182.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 17 MB/s | 8.0 MB 00:00 2023-07-14T11:03:20.452 INFO:teuthology.orchestra.run.smithi182.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 510 kB/s | 213 kB 00:00 2023-07-14T11:03:20.512 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:27 ago on Fri 14 Jul 2023 11:00:52 AM UTC. 2023-07-14T11:03:20.736 INFO:teuthology.orchestra.run.smithi182.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 11 MB/s | 7.0 MB 00:00 2023-07-14T11:03:20.786 INFO:teuthology.orchestra.run.smithi182.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 4.6 MB/s | 2.2 MB 00:00 2023-07-14T11:03:20.787 INFO:teuthology.orchestra.run.smithi182.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:03:20.787 INFO:teuthology.orchestra.run.smithi182.stdout:Total 14 MB/s | 18 MB 00:01 2023-07-14T11:03:20.987 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction check 2023-07-14T11:03:21.006 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction check succeeded. 2023-07-14T11:03:21.006 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction test 2023-07-14T11:03:21.188 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction test succeeded. 2023-07-14T11:03:21.191 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction 2023-07-14T11:03:21.562 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-07-14T11:03:21.654 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================================ 2023-07-14T11:03:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-07-14T11:03:21.654 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================================ 2023-07-14T11:03:21.655 INFO:teuthology.orchestra.run.smithi175.stdout:Installing group/module packages: 2023-07-14T11:03:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T11:03:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T11:03:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T11:03:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T11:03:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T11:03:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T11:03:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T11:03:21.656 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-07-14T11:03:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T11:03:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T11:03:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T11:03:21.656 INFO:teuthology.orchestra.run.smithi175.stdout:Installing module profiles: 2023-07-14T11:03:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: container-tools/common 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling module streams: 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: container-tools rhel8 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================================ 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout:Install 10 Packages 2023-07-14T11:03:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:03:21.658 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 18 M 2023-07-14T11:03:21.658 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 65 M 2023-07-14T11:03:21.658 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-07-14T11:03:21.710 INFO:teuthology.orchestra.run.smithi182.stdout: Preparing : 1/1 2023-07-14T11:03:21.797 INFO:teuthology.orchestra.run.smithi175.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 669 kB/s | 94 kB 00:00 2023-07-14T11:03:21.822 INFO:teuthology.orchestra.run.smithi175.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 246 kB/s | 41 kB 00:00 2023-07-14T11:03:21.839 INFO:teuthology.orchestra.run.smithi175.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 136 kB/s | 25 kB 00:00 2023-07-14T11:03:21.912 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T11:03:21.948 INFO:teuthology.orchestra.run.smithi175.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 328 kB/s | 49 kB 00:00 2023-07-14T11:03:22.023 INFO:teuthology.orchestra.run.smithi175.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.5 MB/s | 524 kB 00:00 2023-07-14T11:03:22.049 INFO:teuthology.orchestra.run.smithi175.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.5 MB/s | 149 kB 00:00 2023-07-14T11:03:22.078 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:03:22.099 INFO:teuthology.orchestra.run.smithi175.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-07-14T11:03:22.241 INFO:teuthology.orchestra.run.smithi175.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 20 MB/s | 8.0 MB 00:00 2023-07-14T11:03:22.257 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T11:03:22.316 INFO:teuthology.orchestra.run.smithi175.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 10 MB/s | 2.2 MB 00:00 2023-07-14T11:03:22.325 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T11:03:22.408 INFO:teuthology.orchestra.run.smithi175.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-07-14T11:03:22.409 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:03:22.409 INFO:teuthology.orchestra.run.smithi175.stdout:Total 24 MB/s | 18 MB 00:00 2023-07-14T11:03:22.424 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T11:03:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-07-14T11:03:22.425 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T11:03:22.425 INFO:teuthology.orchestra.run.smithi161.stdout:Installing group/module packages: 2023-07-14T11:03:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T11:03:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T11:03:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T11:03:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T11:03:22.426 INFO:teuthology.orchestra.run.smithi161.stdout:Installing module profiles: 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools/common 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools rhel8 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T11:03:22.427 INFO:teuthology.orchestra.run.smithi161.stdout:Install 10 Packages 2023-07-14T11:03:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:03:22.428 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 18 M 2023-07-14T11:03:22.428 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 65 M 2023-07-14T11:03:22.428 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-07-14T11:03:22.458 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T11:03:22.581 INFO:teuthology.orchestra.run.smithi161.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 266 kB/s | 41 kB 00:00 2023-07-14T11:03:22.606 INFO:teuthology.orchestra.run.smithi161.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 526 kB/s | 94 kB 00:00 2023-07-14T11:03:22.609 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-07-14T11:03:22.623 INFO:teuthology.orchestra.run.smithi161.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 127 kB/s | 25 kB 00:00 2023-07-14T11:03:22.628 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-07-14T11:03:22.628 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-07-14T11:03:22.706 INFO:teuthology.orchestra.run.smithi161.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.1 MB/s | 524 kB 00:00 2023-07-14T11:03:22.723 INFO:teuthology.orchestra.run.smithi161.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 347 kB/s | 49 kB 00:00 2023-07-14T11:03:22.811 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-07-14T11:03:22.814 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-07-14T11:03:22.823 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T11:03:22.849 INFO:teuthology.orchestra.run.smithi161.stdout:(6/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-07-14T11:03:22.891 INFO:teuthology.orchestra.run.smithi161.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 813 kB/s | 149 kB 00:00 2023-07-14T11:03:22.991 INFO:teuthology.orchestra.run.smithi161.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-07-14T11:03:23.074 INFO:teuthology.orchestra.run.smithi161.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 12 MB/s | 2.2 MB 00:00 2023-07-14T11:03:23.216 INFO:teuthology.orchestra.run.smithi161.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 19 MB/s | 7.0 MB 00:00 2023-07-14T11:03:23.217 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:03:23.217 INFO:teuthology.orchestra.run.smithi161.stdout:Total 23 MB/s | 18 MB 00:00 2023-07-14T11:03:23.349 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-07-14T11:03:23.523 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T11:03:23.551 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-07-14T11:03:23.574 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-07-14T11:03:23.575 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-07-14T11:03:23.590 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T11:03:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:03:23.909 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T11:03:24.086 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T11:03:24.105 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-07-14T11:03:24.110 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-07-14T11:03:24.427 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T11:03:24.450 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T11:03:24.662 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T11:03:24.668 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-07-14T11:03:24.843 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T11:03:24.885 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T11:03:25.012 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:03:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:03:25.193 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T11:03:25.279 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T11:03:25.284 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:03:25.284 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T11:03:25.284 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:03:25.284 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T11:03:25.284 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T11:03:25.284 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T11:03:25.285 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T11:03:25.285 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T11:03:25.285 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T11:03:25.285 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T11:03:25.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T11:03:25.883 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T11:03:26.071 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T11:03:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T11:03:26.429 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T11:03:26.430 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:05:23 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:26.539 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T11:03:26.703 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T11:03:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:03:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:03:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T11:03:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:03:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T11:03:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T11:03:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T11:03:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T11:03:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T11:03:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T11:03:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T11:03:27.619 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T11:03:27.901 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T11:03:28.091 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T11:03:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T11:03:28.173 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:05:25 ago on Fri 14 Jul 2023 10:58:02 AM UTC. 2023-07-14T11:03:28.314 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:03:28.725 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:03:28.726 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T11:03:28.726 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:03:28.726 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T11:03:28.726 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T11:03:28.726 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T11:03:28.726 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T11:03:28.727 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T11:03:28.727 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T11:03:28.727 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T11:03:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T11:03:30.182 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:37 ago on Fri 14 Jul 2023 11:00:52 AM UTC. 2023-07-14T11:03:34.598 INFO:teuthology.orchestra.run.smithi182.stdout:Installed products updated. 2023-07-14T11:03:34.835 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:03:34.836 INFO:teuthology.orchestra.run.smithi182.stdout:Installed: 2023-07-14T11:03:34.836 INFO:teuthology.orchestra.run.smithi182.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:34.836 INFO:teuthology.orchestra.run.smithi182.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:03:34.836 INFO:teuthology.orchestra.run.smithi182.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:34.836 INFO:teuthology.orchestra.run.smithi182.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T11:03:34.836 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T11:03:34.836 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T11:03:34.837 INFO:teuthology.orchestra.run.smithi182.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:34.837 INFO:teuthology.orchestra.run.smithi182.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:34.837 INFO:teuthology.orchestra.run.smithi182.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:03:34.837 INFO:teuthology.orchestra.run.smithi182.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T11:03:34.837 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:03:34.837 INFO:teuthology.orchestra.run.smithi182.stdout:Complete! 2023-07-14T11:03:34.992 DEBUG:teuthology.parallel:result is None 2023-07-14T11:03:37.423 INFO:teuthology.orchestra.run.smithi175.stdout:Installed products updated. 2023-07-14T11:03:37.627 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:03:37.628 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-07-14T11:03:37.628 INFO:teuthology.orchestra.run.smithi175.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:37.628 INFO:teuthology.orchestra.run.smithi175.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:03:37.628 INFO:teuthology.orchestra.run.smithi175.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:37.628 INFO:teuthology.orchestra.run.smithi175.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T11:03:37.628 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T11:03:37.629 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T11:03:37.629 INFO:teuthology.orchestra.run.smithi175.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:37.629 INFO:teuthology.orchestra.run.smithi175.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:37.629 INFO:teuthology.orchestra.run.smithi175.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:03:37.629 INFO:teuthology.orchestra.run.smithi175.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T11:03:37.629 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:03:37.629 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-07-14T11:03:37.797 DEBUG:teuthology.parallel:result is None 2023-07-14T11:03:39.550 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2023-07-14T11:03:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:03:39.761 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-07-14T11:03:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:03:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:03:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T11:03:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:03:39.763 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T11:03:39.999 DEBUG:teuthology.parallel:result is None 2023-07-14T11:03:40.000 INFO:teuthology.run_tasks:Running task install... 2023-07-14T11:03:40.009 DEBUG:teuthology.task.install:project ceph 2023-07-14T11:03:40.010 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}} 2023-07-14T11:03:40.010 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'} 2023-07-14T11:03:40.010 INFO:teuthology.task.install:Using flavor: default 2023-07-14T11:03:40.016 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-14T11:03:40.017 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-14T11:03:40.017 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T11:03:40.018 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:03:40.021 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T11:03:40.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:03:40.022 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T11:03:40.023 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:03:40.279 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T11:03:40.280 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T11:03:40.292 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T11:03:40.293 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T11:03:40.300 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T11:03:40.301 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T11:03:40.519 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T11:03:40.520 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:03:40.521 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T11:03:40.534 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T11:03:40.535 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:03:40.535 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T11:03:40.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T11:03:40.538 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:03:40.538 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T11:03:40.561 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T11:03:40.562 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T11:03:40.571 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T11:03:40.571 DEBUG:teuthology.orchestra.run.smithi182:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T11:03:40.593 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T11:03:40.593 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T11:03:40.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T11:03:40.643 DEBUG:teuthology.orchestra.run.smithi182:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T11:03:40.704 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T11:03:40.707 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T11:03:40.724 DEBUG:teuthology.orchestra.run.smithi182:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T11:03:40.772 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2023-07-14T11:03:40.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2023-07-14T11:03:40.793 INFO:teuthology.orchestra.run.smithi182.stdout:check_obsoletes = 1 2023-07-14T11:03:40.794 DEBUG:teuthology.orchestra.run.smithi182:> sudo yum clean all 2023-07-14T11:03:40.800 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T11:03:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2023-07-14T11:03:40.872 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2023-07-14T11:03:41.158 INFO:teuthology.orchestra.run.smithi175.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:41.170 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:41.321 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:42.204 INFO:teuthology.orchestra.run.smithi175.stdout:44 files removed 2023-07-14T11:03:42.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T11:03:42.270 INFO:teuthology.orchestra.run.smithi182.stdout:44 files removed 2023-07-14T11:03:42.324 DEBUG:teuthology.orchestra.run.smithi182:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T11:03:42.437 INFO:teuthology.orchestra.run.smithi161.stdout:44 files removed 2023-07-14T11:03:42.500 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T11:03:42.599 INFO:teuthology.orchestra.run.smithi175.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:42.658 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:42.947 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T11:03:44.587 INFO:teuthology.orchestra.run.smithi182.stdout:ceph packages for x86_64 122 kB/s | 81 kB 00:00 2023-07-14T11:03:44.649 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 125 kB/s | 81 kB 00:00 2023-07-14T11:03:44.902 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 117 kB/s | 81 kB 00:00 2023-07-14T11:03:45.188 INFO:teuthology.orchestra.run.smithi182.stdout:ceph noarch packages 17 kB/s | 9.4 kB 00:00 2023-07-14T11:03:45.267 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 16 kB/s | 9.4 kB 00:00 2023-07-14T11:03:45.502 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 17 kB/s | 9.4 kB 00:00 2023-07-14T11:03:45.781 INFO:teuthology.orchestra.run.smithi182.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-07-14T11:03:45.851 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-07-14T11:03:46.081 INFO:teuthology.orchestra.run.smithi182.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-07-14T11:03:46.103 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-07-14T11:03:46.144 INFO:teuthology.orchestra.run.smithi175.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-07-14T11:03:46.395 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-07-14T11:03:47.432 INFO:teuthology.orchestra.run.smithi182.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-07-14T11:03:47.644 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-07-14T11:03:50.998 INFO:teuthology.orchestra.run.smithi182.stdout:lab-extras 161 kB/s | 24 kB 00:00 2023-07-14T11:03:51.202 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 497 kB/s | 24 kB 00:00 2023-07-14T11:03:52.167 INFO:teuthology.orchestra.run.smithi182.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-07-14T11:03:52.273 INFO:teuthology.orchestra.run.smithi175.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-07-14T11:04:00.286 INFO:teuthology.orchestra.run.smithi182.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-07-14T11:04:00.508 INFO:teuthology.orchestra.run.smithi175.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 MB/s | 53 MB 00:01 2023-07-14T11:04:11.438 INFO:teuthology.orchestra.run.smithi182.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:04:11.439 INFO:teuthology.orchestra.run.smithi182.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:04:11.453 INFO:teuthology.orchestra.run.smithi182.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T11:04:11.454 INFO:teuthology.orchestra.run.smithi182.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T11:04:11.455 INFO:teuthology.orchestra.run.smithi182.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T11:04:11.456 INFO:teuthology.orchestra.run.smithi182.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T11:04:11.458 INFO:teuthology.orchestra.run.smithi182.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T11:04:11.461 INFO:teuthology.orchestra.run.smithi182.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T11:04:11.462 INFO:teuthology.orchestra.run.smithi182.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T11:04:11.597 INFO:teuthology.orchestra.run.smithi182.stdout:Dependencies resolved. 2023-07-14T11:04:11.609 INFO:teuthology.orchestra.run.smithi182.stdout:=========================================================================================================================== 2023-07-14T11:04:11.609 INFO:teuthology.orchestra.run.smithi182.stdout: Package Arch Version Repository Size 2023-07-14T11:04:11.609 INFO:teuthology.orchestra.run.smithi182.stdout:=========================================================================================================================== 2023-07-14T11:04:11.610 INFO:teuthology.orchestra.run.smithi182.stdout:Installing: 2023-07-14T11:04:11.610 INFO:teuthology.orchestra.run.smithi182.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T11:04:11.610 INFO:teuthology.orchestra.run.smithi182.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T11:04:11.610 INFO:teuthology.orchestra.run.smithi182.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T11:04:11.610 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T11:04:11.610 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T11:04:11.610 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T11:04:11.611 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T11:04:11.611 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T11:04:11.611 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T11:04:11.611 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T11:04:11.611 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T11:04:11.612 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T11:04:11.612 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T11:04:11.612 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T11:04:11.612 INFO:teuthology.orchestra.run.smithi182.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T11:04:11.612 INFO:teuthology.orchestra.run.smithi182.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T11:04:11.612 INFO:teuthology.orchestra.run.smithi182.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T11:04:11.612 INFO:teuthology.orchestra.run.smithi182.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T11:04:11.613 INFO:teuthology.orchestra.run.smithi182.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T11:04:11.613 INFO:teuthology.orchestra.run.smithi182.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T11:04:11.613 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T11:04:11.613 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T11:04:11.613 INFO:teuthology.orchestra.run.smithi182.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T11:04:11.613 INFO:teuthology.orchestra.run.smithi182.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T11:04:11.613 INFO:teuthology.orchestra.run.smithi182.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T11:04:11.614 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T11:04:11.615 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T11:04:11.615 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T11:04:11.615 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T11:04:11.615 INFO:teuthology.orchestra.run.smithi182.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T11:04:11.615 INFO:teuthology.orchestra.run.smithi182.stdout:Upgrading: 2023-07-14T11:04:11.615 INFO:teuthology.orchestra.run.smithi182.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T11:04:11.615 INFO:teuthology.orchestra.run.smithi182.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T11:04:11.616 INFO:teuthology.orchestra.run.smithi182.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T11:04:11.616 INFO:teuthology.orchestra.run.smithi182.stdout:Installing dependencies: 2023-07-14T11:04:11.616 INFO:teuthology.orchestra.run.smithi182.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T11:04:11.616 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T11:04:11.616 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T11:04:11.617 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T11:04:11.617 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T11:04:11.617 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T11:04:11.617 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T11:04:11.617 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T11:04:11.617 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T11:04:11.618 INFO:teuthology.orchestra.run.smithi182.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T11:04:11.618 INFO:teuthology.orchestra.run.smithi182.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T11:04:11.618 INFO:teuthology.orchestra.run.smithi182.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T11:04:11.618 INFO:teuthology.orchestra.run.smithi182.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T11:04:11.618 INFO:teuthology.orchestra.run.smithi182.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T11:04:11.618 INFO:teuthology.orchestra.run.smithi182.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T11:04:11.618 INFO:teuthology.orchestra.run.smithi182.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T11:04:11.619 INFO:teuthology.orchestra.run.smithi182.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T11:04:11.619 INFO:teuthology.orchestra.run.smithi182.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T11:04:11.619 INFO:teuthology.orchestra.run.smithi182.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T11:04:11.619 INFO:teuthology.orchestra.run.smithi182.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T11:04:11.619 INFO:teuthology.orchestra.run.smithi182.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T11:04:11.619 INFO:teuthology.orchestra.run.smithi182.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T11:04:11.619 INFO:teuthology.orchestra.run.smithi182.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T11:04:11.620 INFO:teuthology.orchestra.run.smithi182.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T11:04:11.621 INFO:teuthology.orchestra.run.smithi182.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T11:04:11.621 INFO:teuthology.orchestra.run.smithi182.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T11:04:11.621 INFO:teuthology.orchestra.run.smithi182.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T11:04:11.621 INFO:teuthology.orchestra.run.smithi182.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T11:04:11.621 INFO:teuthology.orchestra.run.smithi182.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T11:04:11.621 INFO:teuthology.orchestra.run.smithi182.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T11:04:11.621 INFO:teuthology.orchestra.run.smithi182.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T11:04:11.622 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T11:04:11.623 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:04:11.624 INFO:teuthology.orchestra.run.smithi182.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T11:04:11.624 INFO:teuthology.orchestra.run.smithi182.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T11:04:11.624 INFO:teuthology.orchestra.run.smithi182.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T11:04:11.624 INFO:teuthology.orchestra.run.smithi182.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T11:04:11.624 INFO:teuthology.orchestra.run.smithi182.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T11:04:11.624 INFO:teuthology.orchestra.run.smithi182.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T11:04:11.624 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T11:04:11.625 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T11:04:11.626 INFO:teuthology.orchestra.run.smithi182.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T11:04:11.627 INFO:teuthology.orchestra.run.smithi182.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T11:04:11.628 INFO:teuthology.orchestra.run.smithi182.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T11:04:11.628 INFO:teuthology.orchestra.run.smithi182.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T11:04:11.628 INFO:teuthology.orchestra.run.smithi182.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T11:04:11.628 INFO:teuthology.orchestra.run.smithi182.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T11:04:11.628 INFO:teuthology.orchestra.run.smithi182.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T11:04:11.628 INFO:teuthology.orchestra.run.smithi182.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T11:04:11.628 INFO:teuthology.orchestra.run.smithi182.stdout:Enabling module streams: 2023-07-14T11:04:11.629 INFO:teuthology.orchestra.run.smithi182.stdout: postgresql 10 2023-07-14T11:04:11.629 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:04:11.629 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction Summary 2023-07-14T11:04:11.629 INFO:teuthology.orchestra.run.smithi182.stdout:=========================================================================================================================== 2023-07-14T11:04:11.629 INFO:teuthology.orchestra.run.smithi182.stdout:Install 128 Packages 2023-07-14T11:04:11.629 INFO:teuthology.orchestra.run.smithi182.stdout:Upgrade 3 Packages 2023-07-14T11:04:11.629 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:04:11.630 INFO:teuthology.orchestra.run.smithi182.stdout:Total download size: 205 M 2023-07-14T11:04:11.630 INFO:teuthology.orchestra.run.smithi182.stdout:Downloading Packages: 2023-07-14T11:04:11.675 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:04:11.676 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:04:11.691 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T11:04:11.692 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T11:04:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T11:04:11.694 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T11:04:11.696 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T11:04:11.700 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T11:04:11.701 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T11:04:11.832 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-07-14T11:04:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:=========================================================================================================================== 2023-07-14T11:04:11.845 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-07-14T11:04:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:=========================================================================================================================== 2023-07-14T11:04:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-07-14T11:04:11.845 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T11:04:11.845 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T11:04:11.845 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T11:04:11.845 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T11:04:11.846 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T11:04:11.846 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T11:04:11.846 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T11:04:11.846 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T11:04:11.846 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T11:04:11.846 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T11:04:11.847 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T11:04:11.847 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T11:04:11.847 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T11:04:11.847 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T11:04:11.847 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T11:04:11.847 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T11:04:11.847 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T11:04:11.848 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T11:04:11.849 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T11:04:11.849 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T11:04:11.849 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T11:04:11.849 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T11:04:11.849 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T11:04:11.849 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T11:04:11.850 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T11:04:11.850 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T11:04:11.850 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T11:04:11.850 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T11:04:11.851 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T11:04:11.851 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T11:04:11.851 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2023-07-14T11:04:11.851 INFO:teuthology.orchestra.run.smithi175.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T11:04:11.852 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T11:04:11.852 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T11:04:11.852 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-07-14T11:04:11.852 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T11:04:11.852 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T11:04:11.852 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T11:04:11.852 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T11:04:11.853 INFO:teuthology.orchestra.run.smithi175.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T11:04:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T11:04:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T11:04:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T11:04:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T11:04:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T11:04:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T11:04:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T11:04:11.855 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T11:04:11.856 INFO:teuthology.orchestra.run.smithi175.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T11:04:11.856 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T11:04:11.856 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T11:04:11.856 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T11:04:11.856 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T11:04:11.856 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T11:04:11.856 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T11:04:11.857 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T11:04:11.857 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T11:04:11.857 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T11:04:11.857 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T11:04:11.857 INFO:teuthology.orchestra.run.smithi175.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T11:04:11.857 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T11:04:11.858 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T11:04:11.858 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T11:04:11.858 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T11:04:11.858 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T11:04:11.858 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T11:04:11.859 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T11:04:11.859 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T11:04:11.859 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T11:04:11.859 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T11:04:11.859 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T11:04:11.859 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T11:04:11.859 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T11:04:11.860 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T11:04:11.860 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:04:11.860 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T11:04:11.860 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T11:04:11.861 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T11:04:11.861 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T11:04:11.861 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T11:04:11.861 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T11:04:11.862 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T11:04:11.862 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T11:04:11.862 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T11:04:11.862 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T11:04:11.863 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T11:04:11.863 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T11:04:11.863 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T11:04:11.863 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T11:04:11.864 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T11:04:11.864 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T11:04:11.864 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T11:04:11.865 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T11:04:11.865 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:04:11.865 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T11:04:11.865 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T11:04:11.866 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T11:04:11.866 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T11:04:11.866 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T11:04:11.867 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T11:04:11.867 INFO:teuthology.orchestra.run.smithi175.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T11:04:11.867 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T11:04:11.867 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T11:04:11.868 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T11:04:11.868 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T11:04:11.868 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T11:04:11.869 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T11:04:11.869 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T11:04:11.869 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T11:04:11.870 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T11:04:11.870 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T11:04:11.870 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T11:04:11.871 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling module streams: 2023-07-14T11:04:11.871 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql 10 2023-07-14T11:04:11.871 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:04:11.871 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-07-14T11:04:11.872 INFO:teuthology.orchestra.run.smithi175.stdout:=========================================================================================================================== 2023-07-14T11:04:11.872 INFO:teuthology.orchestra.run.smithi175.stdout:Install 128 Packages 2023-07-14T11:04:11.872 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 3 Packages 2023-07-14T11:04:11.873 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:04:11.874 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 205 M 2023-07-14T11:04:11.874 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-07-14T11:04:12.307 INFO:teuthology.orchestra.run.smithi175.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-07-14T11:04:13.182 INFO:teuthology.orchestra.run.smithi175.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 970 kB/s | 848 kB 00:00 2023-07-14T11:04:13.357 INFO:teuthology.orchestra.run.smithi175.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 946 kB/s | 165 kB 00:00 2023-07-14T11:04:15.274 INFO:teuthology.orchestra.run.smithi175.stdout:(4/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-07-14T11:04:16.074 INFO:teuthology.orchestra.run.smithi182.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 13 kB/s | 6.4 kB 00:00 2023-07-14T11:04:16.324 INFO:teuthology.orchestra.run.smithi175.stdout:(5/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 1.5 MB/s | 1.5 MB 00:01 2023-07-14T11:04:17.052 INFO:teuthology.orchestra.run.smithi182.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 868 kB/s | 848 kB 00:00 2023-07-14T11:04:17.227 INFO:teuthology.orchestra.run.smithi182.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 945 kB/s | 165 kB 00:00 2023-07-14T11:04:18.557 INFO:teuthology.orchestra.run.smithi175.stdout:(6/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 844 kB/s | 5.4 MB 00:06 2023-07-14T11:04:18.741 INFO:teuthology.orchestra.run.smithi175.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 1.9 MB/s | 4.6 MB 00:02 2023-07-14T11:04:18.752 INFO:teuthology.orchestra.run.smithi182.stdout:(4/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 1.5 MB/s | 2.3 MB 00:01 2023-07-14T11:04:19.886 INFO:teuthology.orchestra.run.smithi182.stdout:(5/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 1.4 MB/s | 1.5 MB 00:01 2023-07-14T11:04:20.094 INFO:teuthology.orchestra.run.smithi182.stdout:(6/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-07-14T11:04:22.586 INFO:teuthology.orchestra.run.smithi182.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 1.7 MB/s | 4.6 MB 00:02 2023-07-14T11:04:24.549 INFO:teuthology.orchestra.run.smithi175.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 1.8 MB/s | 22 MB 00:12 2023-07-14T11:04:24.666 INFO:teuthology.orchestra.run.smithi175.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 209 kB/s | 24 kB 00:00 2023-07-14T11:04:25.508 INFO:teuthology.orchestra.run.smithi175.stdout:(10/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 2.5 MB/s | 18 MB 00:06 2023-07-14T11:04:25.700 INFO:teuthology.orchestra.run.smithi175.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 1.3 MB/s | 258 kB 00:00 2023-07-14T11:04:25.825 INFO:teuthology.orchestra.run.smithi175.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 248 kB/s | 31 kB 00:00 2023-07-14T11:04:26.242 INFO:teuthology.orchestra.run.smithi175.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 1.8 MB/s | 771 kB 00:00 2023-07-14T11:04:26.359 INFO:teuthology.orchestra.run.smithi175.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.1 MB/s | 137 kB 00:00 2023-07-14T11:04:26.476 INFO:teuthology.orchestra.run.smithi175.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 1.2 MB/s | 139 kB 00:00 2023-07-14T11:04:26.784 INFO:teuthology.orchestra.run.smithi175.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 1.5 MB/s | 473 kB 00:00 2023-07-14T11:04:28.568 INFO:teuthology.orchestra.run.smithi175.stdout:(17/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 1.4 MB/s | 13 MB 00:09 2023-07-14T11:04:28.685 INFO:teuthology.orchestra.run.smithi175.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 384 kB/s | 45 kB 00:00 2023-07-14T11:04:28.835 INFO:teuthology.orchestra.run.smithi175.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 792 kB/s | 119 kB 00:00 2023-07-14T11:04:29.168 INFO:teuthology.orchestra.run.smithi175.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 590 kB/s | 197 kB 00:00 2023-07-14T11:04:29.319 INFO:teuthology.orchestra.run.smithi175.stdout:(21/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 2.7 MB/s | 6.7 MB 00:02 2023-07-14T11:04:29.469 INFO:teuthology.orchestra.run.smithi175.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 2.4 MB/s | 364 kB 00:00 2023-07-14T11:04:29.510 INFO:teuthology.orchestra.run.smithi182.stdout:(8/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x86 1.9 MB/s | 18 MB 00:09 2023-07-14T11:04:29.569 INFO:teuthology.orchestra.run.smithi175.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 1.0 MB/s | 107 kB 00:00 2023-07-14T11:04:29.611 INFO:teuthology.orchestra.run.smithi182.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 244 kB/s | 24 kB 00:00 2023-07-14T11:04:29.628 INFO:teuthology.orchestra.run.smithi175.stdout:(24/131): python3-rados-18.0.0-4757.g3fc0089c.e 809 kB/s | 371 kB 00:00 2023-07-14T11:04:29.670 INFO:teuthology.orchestra.run.smithi175.stdout:(25/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 942 kB/s | 94 kB 00:00 2023-07-14T11:04:29.778 INFO:teuthology.orchestra.run.smithi175.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.6 MB/s | 179 kB 00:00 2023-07-14T11:04:29.862 INFO:teuthology.orchestra.run.smithi175.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 277 kB/s | 23 kB 00:00 2023-07-14T11:04:29.970 INFO:teuthology.orchestra.run.smithi175.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 1.2 MB/s | 129 kB 00:00 2023-07-14T11:04:30.662 INFO:teuthology.orchestra.run.smithi175.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 2.5 MB/s | 1.7 MB 00:00 2023-07-14T11:04:30.919 INFO:teuthology.orchestra.run.smithi182.stdout:(10/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 1.6 MB/s | 13 MB 00:08 2023-07-14T11:04:31.120 INFO:teuthology.orchestra.run.smithi182.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 1.3 MB/s | 258 kB 00:00 2023-07-14T11:04:31.361 INFO:teuthology.orchestra.run.smithi182.stdout:(12/131): ceph-common-18.0.0-4757.g3fc0089c.el8 1.4 MB/s | 22 MB 00:15 2023-07-14T11:04:31.378 INFO:teuthology.orchestra.run.smithi182.stdout:(13/131): libcephfs-devel-18.0.0-4757.g3fc0089c 120 kB/s | 31 kB 00:00 2023-07-14T11:04:31.487 INFO:teuthology.orchestra.run.smithi182.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.2 MB/s | 137 kB 00:00 2023-07-14T11:04:31.720 INFO:teuthology.orchestra.run.smithi182.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 596 kB/s | 139 kB 00:00 2023-07-14T11:04:32.221 INFO:teuthology.orchestra.run.smithi182.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 946 kB/s | 473 kB 00:00 2023-07-14T11:04:32.246 INFO:teuthology.orchestra.run.smithi182.stdout:(17/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 873 kB/s | 771 kB 00:00 2023-07-14T11:04:32.346 INFO:teuthology.orchestra.run.smithi182.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 447 kB/s | 45 kB 00:00 2023-07-14T11:04:32.546 INFO:teuthology.orchestra.run.smithi182.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 594 kB/s | 119 kB 00:00 2023-07-14T11:04:32.764 INFO:teuthology.orchestra.run.smithi182.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 906 kB/s | 197 kB 00:00 2023-07-14T11:04:33.172 INFO:teuthology.orchestra.run.smithi182.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 909 kB/s | 371 kB 00:00 2023-07-14T11:04:33.572 INFO:teuthology.orchestra.run.smithi182.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 911 kB/s | 364 kB 00:00 2023-07-14T11:04:34.031 INFO:teuthology.orchestra.run.smithi182.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 233 kB/s | 107 kB 00:00 2023-07-14T11:04:34.248 INFO:teuthology.orchestra.run.smithi182.stdout:(24/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 436 kB/s | 94 kB 00:00 2023-07-14T11:04:34.262 INFO:teuthology.orchestra.run.smithi175.stdout:(30/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 762 kB/s | 3.4 MB 00:04 2023-07-14T11:04:34.604 INFO:teuthology.orchestra.run.smithi175.stdout:(31/131): ceph-mgr-diskprediction-local-18.0.0- 1.9 MB/s | 7.4 MB 00:03 2023-07-14T11:04:34.671 INFO:teuthology.orchestra.run.smithi175.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 713 kB/s | 47 kB 00:00 2023-07-14T11:04:34.730 INFO:teuthology.orchestra.run.smithi175.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 249 kB/s | 15 kB 00:00 2023-07-14T11:04:34.830 INFO:teuthology.orchestra.run.smithi175.stdout:(34/131): ceph-mgr-modules-core-18.0.0-4757.g3f 424 kB/s | 240 kB 00:00 2023-07-14T11:04:35.014 INFO:teuthology.orchestra.run.smithi175.stdout:(35/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 910 kB/s | 258 kB 00:00 2023-07-14T11:04:35.106 INFO:teuthology.orchestra.run.smithi175.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 196 kB/s | 18 kB 00:00 2023-07-14T11:04:35.189 INFO:teuthology.orchestra.run.smithi175.stdout:(37/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 464 kB/s | 166 kB 00:00 2023-07-14T11:04:35.265 INFO:teuthology.orchestra.run.smithi175.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2023-07-14T11:04:35.298 INFO:teuthology.orchestra.run.smithi175.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 542 kB/s | 59 kB 00:00 2023-07-14T11:04:35.323 INFO:teuthology.orchestra.run.smithi175.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-07-14T11:04:35.365 INFO:teuthology.orchestra.run.smithi175.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-07-14T11:04:35.416 INFO:teuthology.orchestra.run.smithi175.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-07-14T11:04:35.499 INFO:teuthology.orchestra.run.smithi175.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.3 MB/s | 476 kB 00:00 2023-07-14T11:04:35.516 INFO:teuthology.orchestra.run.smithi175.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-07-14T11:04:35.550 INFO:teuthology.orchestra.run.smithi175.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-07-14T11:04:35.575 INFO:teuthology.orchestra.run.smithi175.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-07-14T11:04:35.600 INFO:teuthology.orchestra.run.smithi175.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-07-14T11:04:35.625 INFO:teuthology.orchestra.run.smithi175.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-07-14T11:04:35.659 INFO:teuthology.orchestra.run.smithi175.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-07-14T11:04:35.684 INFO:teuthology.orchestra.run.smithi175.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 301 kB/s | 18 kB 00:00 2023-07-14T11:04:35.735 INFO:teuthology.orchestra.run.smithi175.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 972 kB/s | 49 kB 00:00 2023-07-14T11:04:35.768 INFO:teuthology.orchestra.run.smithi175.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-07-14T11:04:35.868 INFO:teuthology.orchestra.run.smithi175.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 440 kB/s | 59 kB 00:00 2023-07-14T11:04:36.028 INFO:teuthology.orchestra.run.smithi175.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 232 kB/s | 60 kB 00:00 2023-07-14T11:04:36.061 INFO:teuthology.orchestra.run.smithi175.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-07-14T11:04:36.086 INFO:teuthology.orchestra.run.smithi175.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 270 kB/s | 16 kB 00:00 2023-07-14T11:04:36.111 INFO:teuthology.orchestra.run.smithi175.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-07-14T11:04:36.136 INFO:teuthology.orchestra.run.smithi175.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-07-14T11:04:36.162 INFO:teuthology.orchestra.run.smithi175.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-07-14T11:04:36.212 INFO:teuthology.orchestra.run.smithi175.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-07-14T11:04:36.237 INFO:teuthology.orchestra.run.smithi175.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2023-07-14T11:04:36.263 INFO:teuthology.orchestra.run.smithi175.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-07-14T11:04:36.288 INFO:teuthology.orchestra.run.smithi175.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-07-14T11:04:36.313 INFO:teuthology.orchestra.run.smithi175.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-07-14T11:04:36.339 INFO:teuthology.orchestra.run.smithi175.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-07-14T11:04:36.348 INFO:teuthology.orchestra.run.smithi182.stdout:(25/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 1.6 MB/s | 6.7 MB 00:04 2023-07-14T11:04:36.364 INFO:teuthology.orchestra.run.smithi175.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-07-14T11:04:36.389 INFO:teuthology.orchestra.run.smithi175.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-07-14T11:04:36.406 INFO:teuthology.orchestra.run.smithi175.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 550 kB/s | 23 kB 00:00 2023-07-14T11:04:36.432 INFO:teuthology.orchestra.run.smithi175.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T11:04:36.540 INFO:teuthology.orchestra.run.smithi182.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 937 kB/s | 179 kB 00:00 2023-07-14T11:04:36.548 INFO:teuthology.orchestra.run.smithi175.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-07-14T11:04:36.648 INFO:teuthology.orchestra.run.smithi182.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 213 kB/s | 23 kB 00:00 2023-07-14T11:04:36.682 INFO:teuthology.orchestra.run.smithi175.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 827 kB/s | 110 kB 00:00 2023-07-14T11:04:36.774 INFO:teuthology.orchestra.run.smithi175.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 591 kB/s | 54 kB 00:00 2023-07-14T11:04:36.824 INFO:teuthology.orchestra.run.smithi182.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 737 kB/s | 129 kB 00:00 2023-07-14T11:04:36.959 INFO:teuthology.orchestra.run.smithi175.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-07-14T11:04:36.991 INFO:teuthology.orchestra.run.smithi175.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2023-07-14T11:04:37.100 INFO:teuthology.orchestra.run.smithi175.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 359 kB/s | 39 kB 00:00 2023-07-14T11:04:37.125 INFO:teuthology.orchestra.run.smithi175.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.9 MB/s | 323 kB 00:00 2023-07-14T11:04:37.250 INFO:teuthology.orchestra.run.smithi175.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.2 MB/s | 538 kB 00:00 2023-07-14T11:04:37.276 INFO:teuthology.orchestra.run.smithi175.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2023-07-14T11:04:37.384 INFO:teuthology.orchestra.run.smithi175.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-07-14T11:04:37.443 INFO:teuthology.orchestra.run.smithi175.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 817 kB/s | 157 kB 00:00 2023-07-14T11:04:37.568 INFO:teuthology.orchestra.run.smithi175.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 3.8 MB/s | 713 kB 00:00 2023-07-14T11:04:37.693 INFO:teuthology.orchestra.run.smithi175.stdout:(82/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-07-14T11:04:37.719 INFO:teuthology.orchestra.run.smithi175.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 177 kB/s | 27 kB 00:00 2023-07-14T11:04:37.827 INFO:teuthology.orchestra.run.smithi175.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-07-14T11:04:37.903 INFO:teuthology.orchestra.run.smithi175.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 239 kB/s | 44 kB 00:00 2023-07-14T11:04:38.019 INFO:teuthology.orchestra.run.smithi175.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2023-07-14T11:04:38.086 INFO:teuthology.orchestra.run.smithi175.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-07-14T11:04:38.120 INFO:teuthology.orchestra.run.smithi175.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-07-14T11:04:38.224 INFO:teuthology.orchestra.run.smithi182.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 1.2 MB/s | 1.7 MB 00:01 2023-07-14T11:04:38.228 INFO:teuthology.orchestra.run.smithi175.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 451 kB/s | 64 kB 00:00 2023-07-14T11:04:38.262 INFO:teuthology.orchestra.run.smithi175.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 891 kB/s | 126 kB 00:00 2023-07-14T11:04:38.274 INFO:teuthology.orchestra.run.smithi182.stdout:(30/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 877 kB/s | 3.4 MB 00:04 2023-07-14T11:04:38.371 INFO:teuthology.orchestra.run.smithi175.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2023-07-14T11:04:38.413 INFO:teuthology.orchestra.run.smithi175.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 688 kB/s | 103 kB 00:00 2023-07-14T11:04:38.546 INFO:teuthology.orchestra.run.smithi175.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 4.0 MB/s | 709 kB 00:00 2023-07-14T11:04:38.591 INFO:teuthology.orchestra.run.smithi182.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 760 kB/s | 240 kB 00:00 2023-07-14T11:04:38.714 INFO:teuthology.orchestra.run.smithi175.stdout:(94/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-07-14T11:04:38.725 INFO:teuthology.orchestra.run.smithi182.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 356 kB/s | 47 kB 00:00 2023-07-14T11:04:38.801 INFO:teuthology.orchestra.run.smithi175.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 777 kB/s | 198 kB 00:00 2023-07-14T11:04:38.833 INFO:teuthology.orchestra.run.smithi182.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 134 kB/s | 15 kB 00:00 2023-07-14T11:04:38.981 INFO:teuthology.orchestra.run.smithi175.stdout:(96/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-14T11:04:39.006 INFO:teuthology.orchestra.run.smithi175.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 987 kB/s | 202 kB 00:00 2023-07-14T11:04:39.139 INFO:teuthology.orchestra.run.smithi175.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-07-14T11:04:39.217 INFO:teuthology.orchestra.run.smithi182.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 673 kB/s | 258 kB 00:00 2023-07-14T11:04:39.290 INFO:teuthology.orchestra.run.smithi175.stdout:(99/131): postgresql-server-10.21-2.module+el8. 18 MB/s | 5.1 MB 00:00 2023-07-14T11:04:39.340 INFO:teuthology.orchestra.run.smithi175.stdout:(100/131): postgresql-contrib-10.21-2.module+el 3.9 MB/s | 808 kB 00:00 2023-07-14T11:04:39.434 INFO:teuthology.orchestra.run.smithi182.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 769 kB/s | 166 kB 00:00 2023-07-14T11:04:39.490 INFO:teuthology.orchestra.run.smithi175.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 7.5 MB/s | 1.5 MB 00:00 2023-07-14T11:04:39.516 INFO:teuthology.orchestra.run.smithi175.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 267 kB/s | 47 kB 00:00 2023-07-14T11:04:39.542 INFO:teuthology.orchestra.run.smithi182.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 166 kB/s | 18 kB 00:00 2023-07-14T11:04:39.632 INFO:teuthology.orchestra.run.smithi175.stdout:(103/131): python3-jwt-1.6.1-2.el8.noarch.rpm 365 kB/s | 43 kB 00:00 2023-07-14T11:04:39.658 INFO:teuthology.orchestra.run.smithi175.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2023-07-14T11:04:39.751 INFO:teuthology.orchestra.run.smithi182.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-07-14T11:04:39.792 INFO:teuthology.orchestra.run.smithi175.stdout:(105/131): keyutils-libs-devel-1.5.10-9.el8.x86 360 kB/s | 48 kB 00:00 2023-07-14T11:04:39.810 INFO:teuthology.orchestra.run.smithi182.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-07-14T11:04:39.817 INFO:teuthology.orchestra.run.smithi175.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-07-14T11:04:39.868 INFO:teuthology.orchestra.run.smithi182.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-07-14T11:04:39.959 INFO:teuthology.orchestra.run.smithi175.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 522 kB/s | 87 kB 00:00 2023-07-14T11:04:39.960 INFO:teuthology.orchestra.run.smithi182.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 5.1 MB/s | 476 kB 00:00 2023-07-14T11:04:39.984 INFO:teuthology.orchestra.run.smithi175.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 255 kB/s | 43 kB 00:00 2023-07-14T11:04:40.011 INFO:teuthology.orchestra.run.smithi182.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 875 kB/s | 44 kB 00:00 2023-07-14T11:04:40.051 INFO:teuthology.orchestra.run.smithi175.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.2 MB/s | 109 kB 00:00 2023-07-14T11:04:40.069 INFO:teuthology.orchestra.run.smithi182.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-07-14T11:04:40.101 INFO:teuthology.orchestra.run.smithi175.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-07-14T11:04:40.111 INFO:teuthology.orchestra.run.smithi182.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 793 kB/s | 33 kB 00:00 2023-07-14T11:04:40.153 INFO:teuthology.orchestra.run.smithi182.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 392 kB/s | 16 kB 00:00 2023-07-14T11:04:40.180 INFO:teuthology.orchestra.run.smithi175.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.2 MB/s | 560 kB 00:00 2023-07-14T11:04:40.212 INFO:teuthology.orchestra.run.smithi182.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-07-14T11:04:40.236 INFO:teuthology.orchestra.run.smithi175.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 140 kB/s | 19 kB 00:00 2023-07-14T11:04:40.285 INFO:teuthology.orchestra.run.smithi175.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.4 MB/s | 155 kB 00:00 2023-07-14T11:04:40.287 INFO:teuthology.orchestra.run.smithi182.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2023-07-14T11:04:40.327 INFO:teuthology.orchestra.run.smithi175.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 5.9 MB/s | 559 kB 00:00 2023-07-14T11:04:40.346 INFO:teuthology.orchestra.run.smithi182.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-07-14T11:04:40.396 INFO:teuthology.orchestra.run.smithi182.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2023-07-14T11:04:40.436 INFO:teuthology.orchestra.run.smithi175.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 119 kB/s | 18 kB 00:00 2023-07-14T11:04:40.447 INFO:teuthology.orchestra.run.smithi182.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 352 kB/s | 18 kB 00:00 2023-07-14T11:04:40.469 INFO:teuthology.orchestra.run.smithi175.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.2 MB/s | 605 kB 00:00 2023-07-14T11:04:40.547 INFO:teuthology.orchestra.run.smithi182.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-07-14T11:04:40.570 INFO:teuthology.orchestra.run.smithi175.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.7 MB/s | 238 kB 00:00 2023-07-14T11:04:40.597 INFO:teuthology.orchestra.run.smithi182.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 974 kB/s | 49 kB 00:00 2023-07-14T11:04:40.656 INFO:teuthology.orchestra.run.smithi182.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-07-14T11:04:40.670 INFO:teuthology.orchestra.run.smithi175.stdout:(118/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2023-07-14T11:04:40.704 INFO:teuthology.orchestra.run.smithi175.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 542 kB/s | 72 kB 00:00 2023-07-14T11:04:40.715 INFO:teuthology.orchestra.run.smithi182.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-07-14T11:04:40.774 INFO:teuthology.orchestra.run.smithi182.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-07-14T11:04:40.813 INFO:teuthology.orchestra.run.smithi175.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 583 kB/s | 83 kB 00:00 2023-07-14T11:04:40.824 INFO:teuthology.orchestra.run.smithi182.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 313 kB/s | 16 kB 00:00 2023-07-14T11:04:40.874 INFO:teuthology.orchestra.run.smithi182.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 674 kB/s | 34 kB 00:00 2023-07-14T11:04:40.888 INFO:teuthology.orchestra.run.smithi175.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 326 kB/s | 60 kB 00:00 2023-07-14T11:04:40.924 INFO:teuthology.orchestra.run.smithi182.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-07-14T11:04:40.975 INFO:teuthology.orchestra.run.smithi182.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-07-14T11:04:41.025 INFO:teuthology.orchestra.run.smithi182.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 376 kB/s | 19 kB 00:00 2023-07-14T11:04:41.147 INFO:teuthology.orchestra.run.smithi175.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 293 kB/s | 98 kB 00:00 2023-07-14T11:04:41.159 INFO:teuthology.orchestra.run.smithi182.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 179 kB/s | 24 kB 00:00 2023-07-14T11:04:41.172 INFO:teuthology.orchestra.run.smithi175.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 203 kB/s | 58 kB 00:00 2023-07-14T11:04:41.255 INFO:teuthology.orchestra.run.smithi175.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 357 kB/s | 39 kB 00:00 2023-07-14T11:04:41.292 INFO:teuthology.orchestra.run.smithi182.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 219 kB/s | 29 kB 00:00 2023-07-14T11:04:41.323 INFO:teuthology.orchestra.run.smithi175.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.2 MB/s | 644 kB 00:00 2023-07-14T11:04:41.359 INFO:teuthology.orchestra.run.smithi182.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 414 kB/s | 27 kB 00:00 2023-07-14T11:04:41.364 INFO:teuthology.orchestra.run.smithi175.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.5 MB/s | 171 kB 00:00 2023-07-14T11:04:41.426 INFO:teuthology.orchestra.run.smithi182.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 3.8 MB/s | 255 kB 00:00 2023-07-14T11:04:41.440 INFO:teuthology.orchestra.run.smithi175.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.2 MB/s | 138 kB 00:00 2023-07-14T11:04:41.477 INFO:teuthology.orchestra.run.smithi182.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-07-14T11:04:41.535 INFO:teuthology.orchestra.run.smithi182.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-07-14T11:04:41.586 INFO:teuthology.orchestra.run.smithi182.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-07-14T11:04:41.636 INFO:teuthology.orchestra.run.smithi182.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2023-07-14T11:04:41.728 INFO:teuthology.orchestra.run.smithi182.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2023-07-14T11:04:41.744 INFO:teuthology.orchestra.run.smithi182.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-07-14T11:04:42.516 INFO:teuthology.orchestra.run.smithi182.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 18 MB/s | 14 MB 00:00 2023-07-14T11:04:42.670 INFO:teuthology.orchestra.run.smithi182.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 716 kB/s | 110 kB 00:00 2023-07-14T11:04:42.812 INFO:teuthology.orchestra.run.smithi182.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 382 kB/s | 54 kB 00:00 2023-07-14T11:04:42.954 INFO:teuthology.orchestra.run.smithi182.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2023-07-14T11:04:43.096 INFO:teuthology.orchestra.run.smithi182.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2023-07-14T11:04:43.238 INFO:teuthology.orchestra.run.smithi182.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 274 kB/s | 39 kB 00:00 2023-07-14T11:04:43.380 INFO:teuthology.orchestra.run.smithi182.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2023-07-14T11:04:43.555 INFO:teuthology.orchestra.run.smithi182.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.0 MB/s | 538 kB 00:00 2023-07-14T11:04:43.681 INFO:teuthology.orchestra.run.smithi182.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-07-14T11:04:43.856 INFO:teuthology.orchestra.run.smithi182.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-07-14T11:04:44.023 INFO:teuthology.orchestra.run.smithi182.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 4.2 MB/s | 713 kB 00:00 2023-07-14T11:04:44.232 INFO:teuthology.orchestra.run.smithi182.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-07-14T11:04:44.349 INFO:teuthology.orchestra.run.smithi182.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 227 kB/s | 27 kB 00:00 2023-07-14T11:04:44.490 INFO:teuthology.orchestra.run.smithi182.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2023-07-14T11:04:44.658 INFO:teuthology.orchestra.run.smithi182.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 264 kB/s | 44 kB 00:00 2023-07-14T11:04:44.816 INFO:teuthology.orchestra.run.smithi182.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 24 MB/s | 3.7 MB 00:00 2023-07-14T11:04:44.950 INFO:teuthology.orchestra.run.smithi182.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.4 MB/s | 457 kB 00:00 2023-07-14T11:04:45.092 INFO:teuthology.orchestra.run.smithi182.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.6 MB/s | 225 kB 00:00 2023-07-14T11:04:45.217 INFO:teuthology.orchestra.run.smithi182.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 512 kB/s | 64 kB 00:00 2023-07-14T11:04:45.367 INFO:teuthology.orchestra.run.smithi182.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 842 kB/s | 126 kB 00:00 2023-07-14T11:04:45.534 INFO:teuthology.orchestra.run.smithi182.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2023-07-14T11:04:45.660 INFO:teuthology.orchestra.run.smithi182.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 828 kB/s | 103 kB 00:00 2023-07-14T11:04:45.785 INFO:teuthology.orchestra.run.smithi182.stdout:(92/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-07-14T11:04:45.985 INFO:teuthology.orchestra.run.smithi182.stdout:(93/131): openblas-threads-0.3.15-3.el8.x86_64. 24 MB/s | 4.9 MB 00:00 2023-07-14T11:04:46.098 INFO:teuthology.orchestra.run.smithi175.stdout:(128/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 782 kB/s | 3.6 MB 00:04 2023-07-14T11:04:46.161 INFO:teuthology.orchestra.run.smithi182.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-07-14T11:04:46.265 INFO:teuthology.orchestra.run.smithi175.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.7 MB/s | 798 kB 00:00 2023-07-14T11:04:46.411 INFO:teuthology.orchestra.run.smithi182.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-07-14T11:04:46.503 INFO:teuthology.orchestra.run.smithi182.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 2.2 MB/s | 202 kB 00:00 2023-07-14T11:04:46.611 INFO:teuthology.orchestra.run.smithi182.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-07-14T11:04:46.829 INFO:teuthology.orchestra.run.smithi182.stdout:(98/131): postgresql-server-10.21-2.module+el8. 23 MB/s | 5.1 MB 00:00 2023-07-14T11:04:46.979 INFO:teuthology.orchestra.run.smithi182.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 5.3 MB/s | 808 kB 00:00 2023-07-14T11:04:47.154 INFO:teuthology.orchestra.run.smithi182.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 8.6 MB/s | 1.5 MB 00:00 2023-07-14T11:04:47.213 INFO:teuthology.orchestra.run.smithi182.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 807 kB/s | 47 kB 00:00 2023-07-14T11:04:47.389 INFO:teuthology.orchestra.run.smithi182.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.1 MB/s | 200 kB 00:00 2023-07-14T11:04:47.480 INFO:teuthology.orchestra.run.smithi182.stdout:(103/131): python3-jwt-1.6.1-2.el8.noarch.rpm 466 kB/s | 43 kB 00:00 2023-07-14T11:04:47.639 INFO:teuthology.orchestra.run.smithi182.stdout:(104/131): ceph-mgr-diskprediction-local-18.0.0 806 kB/s | 7.4 MB 00:09 2023-07-14T11:04:47.664 INFO:teuthology.orchestra.run.smithi182.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-07-14T11:04:47.799 INFO:teuthology.orchestra.run.smithi182.stdout:(106/131): libsepol-devel-2.9-3.el8.x86_64.rpm 681 kB/s | 87 kB 00:00 2023-07-14T11:04:47.823 INFO:teuthology.orchestra.run.smithi182.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 262 kB/s | 48 kB 00:00 2023-07-14T11:04:47.915 INFO:teuthology.orchestra.run.smithi182.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 364 kB/s | 43 kB 00:00 2023-07-14T11:04:47.982 INFO:teuthology.orchestra.run.smithi182.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 688 kB/s | 109 kB 00:00 2023-07-14T11:04:48.066 INFO:teuthology.orchestra.run.smithi182.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-07-14T11:04:48.133 INFO:teuthology.orchestra.run.smithi182.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.6 MB/s | 560 kB 00:00 2023-07-14T11:04:48.183 INFO:teuthology.orchestra.run.smithi182.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 160 kB/s | 19 kB 00:00 2023-07-14T11:04:48.241 INFO:teuthology.orchestra.run.smithi182.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.4 MB/s | 155 kB 00:00 2023-07-14T11:04:48.275 INFO:teuthology.orchestra.run.smithi182.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 5.9 MB/s | 559 kB 00:00 2023-07-14T11:04:48.425 INFO:teuthology.orchestra.run.smithi182.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 97 kB/s | 18 kB 00:00 2023-07-14T11:04:48.459 INFO:teuthology.orchestra.run.smithi182.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-07-14T11:04:48.601 INFO:teuthology.orchestra.run.smithi182.stdout:(117/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 16 MB/s | 2.3 MB 00:00 2023-07-14T11:04:48.626 INFO:teuthology.orchestra.run.smithi182.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-07-14T11:04:48.710 INFO:teuthology.orchestra.run.smithi182.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 666 kB/s | 72 kB 00:00 2023-07-14T11:04:48.793 INFO:teuthology.orchestra.run.smithi182.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 495 kB/s | 83 kB 00:00 2023-07-14T11:04:48.836 INFO:teuthology.orchestra.run.smithi182.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 477 kB/s | 60 kB 00:00 2023-07-14T11:04:48.853 INFO:teuthology.orchestra.run.smithi182.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.6 MB/s | 98 kB 00:00 2023-07-14T11:04:48.953 INFO:teuthology.orchestra.run.smithi182.stdout:(123/131): libcom_err-devel-1.45.6-4.el8.x86_64 386 kB/s | 39 kB 00:00 2023-07-14T11:04:48.995 INFO:teuthology.orchestra.run.smithi182.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 363 kB/s | 58 kB 00:00 2023-07-14T11:04:49.114 INFO:teuthology.orchestra.run.smithi182.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.9 MB/s | 644 kB 00:00 2023-07-14T11:04:49.139 INFO:teuthology.orchestra.run.smithi182.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.2 MB/s | 171 kB 00:00 2023-07-14T11:04:49.248 INFO:teuthology.orchestra.run.smithi182.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.0 MB/s | 138 kB 00:00 2023-07-14T11:04:55.463 INFO:teuthology.orchestra.run.smithi175.stdout:(130/131): librados2-18.0.0-4757.g3fc0089c.el8. 266 kB/s | 3.7 MB 00:14 2023-07-14T11:04:57.130 INFO:teuthology.orchestra.run.smithi182.stdout:(128/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 462 kB/s | 3.6 MB 00:07 2023-07-14T11:04:57.289 INFO:teuthology.orchestra.run.smithi182.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.9 MB/s | 798 kB 00:00 2023-07-14T11:05:00.389 INFO:teuthology.orchestra.run.smithi182.stdout:(130/131): librados2-18.0.0-4757.g3fc0089c.el8. 334 kB/s | 3.7 MB 00:11 2023-07-14T11:05:28.318 INFO:teuthology.orchestra.run.smithi182.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 834 kB/s | 48 MB 00:58 2023-07-14T11:05:28.321 INFO:teuthology.orchestra.run.smithi182.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:05:28.322 INFO:teuthology.orchestra.run.smithi182.stdout:Total 2.7 MB/s | 205 MB 01:16 2023-07-14T11:05:29.131 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction check 2023-07-14T11:05:29.250 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction check succeeded. 2023-07-14T11:05:29.250 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction test 2023-07-14T11:05:30.468 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction test succeeded. 2023-07-14T11:05:30.472 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction 2023-07-14T11:05:35.191 INFO:teuthology.orchestra.run.smithi182.stdout: Preparing : 1/1 2023-07-14T11:05:35.358 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T11:05:35.529 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T11:05:35.758 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T11:05:35.917 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T11:05:36.041 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T11:05:36.293 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T11:05:36.689 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T11:05:36.884 INFO:teuthology.orchestra.run.smithi182.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:05:37.277 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:05:37.442 INFO:teuthology.orchestra.run.smithi182.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:05:37.551 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:05:37.717 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:05:37.815 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:05:38.018 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:05:38.253 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T11:05:38.453 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T11:05:38.830 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T11:05:39.062 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T11:05:39.192 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:05:39.295 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:05:39.417 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:05:39.498 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:05:39.650 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T11:05:39.792 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T11:05:39.925 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T11:05:40.525 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T11:05:40.692 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T11:05:41.292 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T11:05:41.483 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T11:05:42.172 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T11:05:42.456 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T11:05:42.684 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T11:05:42.895 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T11:05:43.072 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T11:05:43.280 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T11:05:43.534 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T11:05:43.714 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T11:05:43.861 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T11:05:44.764 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T11:05:45.071 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T11:05:45.342 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T11:05:45.588 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T11:05:45.782 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T11:05:47.537 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T11:05:47.858 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T11:05:48.126 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T11:05:48.317 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T11:05:48.505 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T11:05:48.669 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T11:05:48.822 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T11:05:49.025 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T11:05:49.201 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T11:05:49.334 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T11:05:49.536 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T11:05:49.726 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T11:05:50.081 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T11:05:50.360 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T11:05:50.550 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T11:05:50.760 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T11:05:50.984 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T11:05:51.339 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T11:05:51.523 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T11:05:51.716 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T11:05:51.927 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T11:05:52.093 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T11:05:52.268 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T11:05:52.563 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T11:05:52.818 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T11:05:53.060 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T11:05:53.296 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T11:05:53.505 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T11:05:53.658 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T11:05:53.839 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T11:05:53.913 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T11:05:54.151 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T11:05:54.334 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T11:05:54.481 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T11:05:54.558 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T11:05:54.762 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T11:05:54.940 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T11:05:55.139 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T11:05:55.253 INFO:teuthology.orchestra.run.smithi175.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 540 kB/s | 48 MB 01:30 2023-07-14T11:05:55.255 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:05:55.256 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.0 MB/s | 205 MB 01:43 2023-07-14T11:05:55.325 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T11:05:55.606 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T11:05:55.854 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T11:05:56.002 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T11:05:56.046 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-07-14T11:05:56.157 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T11:05:56.166 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-07-14T11:05:56.166 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-07-14T11:05:56.354 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T11:05:56.558 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T11:05:56.740 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T11:05:56.908 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T11:05:57.053 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T11:05:57.281 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T11:05:57.295 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-07-14T11:05:57.300 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-07-14T11:05:57.525 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T11:05:57.727 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T11:05:58.015 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T11:05:58.196 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T11:05:58.353 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T11:05:58.552 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T11:05:59.621 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T11:05:59.892 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T11:06:00.163 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T11:06:00.480 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T11:06:00.764 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T11:06:00.963 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T11:06:01.720 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T11:06:01.888 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:06:01.962 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:06:02.122 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T11:06:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-07-14T11:06:02.411 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T11:06:02.572 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T11:06:02.812 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T11:06:02.980 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T11:06:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T11:06:03.355 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T11:06:03.763 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T11:06:03.955 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:06:04.390 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:06:04.613 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:06:04.744 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:06:04.913 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:06:05.038 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:06:05.197 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:06:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T11:06:05.666 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T11:06:05.709 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:06:05.938 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:06:06.065 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T11:06:06.291 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T11:06:06.454 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:06:06.520 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:06:06.554 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:06:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:06:06.766 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:06:06.806 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:06:06.889 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:06:06.937 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T11:06:06.995 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:06:07.085 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T11:06:07.238 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T11:06:07.886 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T11:06:08.071 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T11:06:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T11:06:08.879 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T11:06:09.563 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T11:06:09.844 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T11:06:10.055 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T11:06:10.316 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T11:06:10.485 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T11:06:10.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T11:06:10.971 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T11:06:11.160 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T11:06:11.290 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T11:06:12.174 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T11:06:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T11:06:12.746 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T11:06:13.018 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T11:06:13.203 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T11:06:14.960 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T11:06:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T11:06:15.514 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T11:06:15.738 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T11:06:15.925 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T11:06:16.132 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T11:06:16.260 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T11:06:16.437 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T11:06:16.638 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T11:06:16.789 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T11:06:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T11:06:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T11:06:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T11:06:17.847 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T11:06:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T11:06:18.271 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T11:06:18.540 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T11:06:19.225 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T11:06:19.476 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T11:06:19.673 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T11:06:19.906 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T11:06:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T11:06:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T11:06:20.567 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T11:06:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T11:06:21.073 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T11:06:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T11:06:21.484 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T11:06:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T11:06:21.843 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T11:06:21.900 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T11:06:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T11:06:22.355 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T11:06:22.501 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T11:06:22.578 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T11:06:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T11:06:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T11:06:23.201 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T11:06:23.380 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T11:06:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T11:06:23.883 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T11:06:24.056 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T11:06:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T11:06:24.395 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T11:06:24.623 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T11:06:24.797 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T11:06:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T11:06:25.141 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T11:06:25.364 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T11:06:25.639 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T11:06:25.824 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T11:06:25.836 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:06:25.836 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /sys 2023-07-14T11:06:25.836 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /proc 2023-07-14T11:06:25.836 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /dev 2023-07-14T11:06:25.836 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /run 2023-07-14T11:06:25.837 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /mnt 2023-07-14T11:06:25.837 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /var/tmp 2023-07-14T11:06:25.837 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /home 2023-07-14T11:06:25.837 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /tmp 2023-07-14T11:06:25.837 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /dev 2023-07-14T11:06:25.837 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:06:26.090 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T11:06:26.159 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:06:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T11:06:26.507 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T11:06:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T11:06:27.721 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:06:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T11:06:27.869 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:06:28.087 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:06:28.112 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T11:06:28.225 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:06:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T11:06:28.679 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:06:28.710 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T11:06:28.834 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:06:28.851 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:06:28.984 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T11:06:29.156 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T11:06:29.947 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T11:06:30.133 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:06:30.214 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:06:30.367 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T11:06:30.570 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:06:30.763 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:06:30.889 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T11:06:31.081 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T11:06:31.241 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:06:31.455 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:06:31.701 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:06:32.659 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:06:32.816 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:06:32.907 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:06:33.262 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T11:06:33.501 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:06:33.536 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:06:33.691 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:06:33.702 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:06:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:06:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:06:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:06:34.983 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T11:06:35.187 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:06:35.221 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:06:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:06:35.333 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:06:35.374 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:06:35.416 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:06:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:06:35.583 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:06:35.927 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:06:36.066 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:06:40.643 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:06:40.951 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T11:06:41.115 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T11:06:41.304 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T11:06:42.177 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T11:06:42.521 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T11:06:42.684 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/134 2023-07-14T11:06:42.869 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T11:06:43.035 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T11:06:43.174 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:06:45.325 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:06:45.573 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:06:45.591 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:06:45.739 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T11:06:46.014 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T11:06:46.342 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T11:06:46.599 INFO:teuthology.orchestra.run.smithi182.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T11:06:46.739 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:06:46.747 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:06:46.926 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T11:06:47.114 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T11:06:47.200 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T11:06:47.414 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T11:06:47.426 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T11:06:47.427 INFO:teuthology.orchestra.run.smithi182.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:06:47.617 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:06:47.617 INFO:teuthology.orchestra.run.smithi182.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:06:47.852 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:06:48.037 INFO:teuthology.orchestra.run.smithi182.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:06:48.542 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:06:48.542 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T11:06:48.542 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T11:06:48.542 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T11:06:48.542 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T11:06:48.543 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T11:06:48.544 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T11:06:48.544 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T11:06:48.544 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T11:06:48.544 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T11:06:48.544 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T11:06:48.544 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T11:06:48.544 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T11:06:48.545 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T11:06:48.546 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T11:06:48.546 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T11:06:48.546 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T11:06:48.552 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T11:06:48.552 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T11:06:48.553 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T11:06:48.553 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T11:06:48.553 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T11:06:48.553 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T11:06:48.553 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T11:06:48.553 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T11:06:48.553 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T11:06:48.554 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T11:06:48.555 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T11:06:48.556 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T11:06:48.556 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T11:06:48.556 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T11:06:48.556 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T11:06:48.556 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T11:06:48.556 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T11:06:48.556 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T11:06:48.557 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T11:06:48.558 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T11:06:48.559 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T11:06:48.560 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T11:06:48.560 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T11:06:48.560 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T11:06:48.560 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T11:06:48.560 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T11:06:48.560 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T11:06:48.560 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T11:06:48.561 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T11:06:48.562 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T11:06:48.563 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T11:06:48.564 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T11:06:48.564 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T11:06:48.564 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T11:06:48.564 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T11:06:48.564 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:06:48.564 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T11:06:48.564 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T11:06:48.565 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:06:48.566 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T11:06:51.072 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:06:51.073 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:02:52 ago on Fri 14 Jul 2023 11:03:59 AM UTC. 2023-07-14T11:06:54.448 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:06:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2023-07-14T11:06:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2023-07-14T11:06:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /dev 2023-07-14T11:06:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /run 2023-07-14T11:06:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2023-07-14T11:06:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2023-07-14T11:06:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2023-07-14T11:06:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2023-07-14T11:06:54.450 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /dev 2023-07-14T11:06:54.450 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:06:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:06:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:06:56.556 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:06:56.774 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:06:56.979 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:06:57.433 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:06:57.578 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:06:57.595 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:06:59.307 INFO:teuthology.orchestra.run.smithi182.stdout:Installed products updated. 2023-07-14T11:06:59.309 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:06:59.508 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:06:59.522 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:06:59.523 INFO:teuthology.orchestra.run.smithi182.stdout:Upgraded: 2023-07-14T11:06:59.523 INFO:teuthology.orchestra.run.smithi182.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T11:06:59.523 INFO:teuthology.orchestra.run.smithi182.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.523 INFO:teuthology.orchestra.run.smithi182.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.523 INFO:teuthology.orchestra.run.smithi182.stdout:Installed: 2023-07-14T11:06:59.524 INFO:teuthology.orchestra.run.smithi182.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T11:06:59.524 INFO:teuthology.orchestra.run.smithi182.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T11:06:59.524 INFO:teuthology.orchestra.run.smithi182.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T11:06:59.524 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.524 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.524 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.524 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.525 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.525 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.525 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.525 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.525 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.525 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.525 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.526 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.526 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.526 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.526 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.526 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.526 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.526 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.527 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.527 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.527 INFO:teuthology.orchestra.run.smithi182.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.527 INFO:teuthology.orchestra.run.smithi182.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.527 INFO:teuthology.orchestra.run.smithi182.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:06:59.527 INFO:teuthology.orchestra.run.smithi182.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T11:06:59.527 INFO:teuthology.orchestra.run.smithi182.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T11:06:59.528 INFO:teuthology.orchestra.run.smithi182.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T11:06:59.528 INFO:teuthology.orchestra.run.smithi182.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T11:06:59.528 INFO:teuthology.orchestra.run.smithi182.stdout: jq-1.6-3.el8.x86_64 2023-07-14T11:06:59.528 INFO:teuthology.orchestra.run.smithi182.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T11:06:59.528 INFO:teuthology.orchestra.run.smithi182.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T11:06:59.528 INFO:teuthology.orchestra.run.smithi182.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T11:06:59.528 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.529 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.529 INFO:teuthology.orchestra.run.smithi182.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.529 INFO:teuthology.orchestra.run.smithi182.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T11:06:59.529 INFO:teuthology.orchestra.run.smithi182.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:06:59.529 INFO:teuthology.orchestra.run.smithi182.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T11:06:59.529 INFO:teuthology.orchestra.run.smithi182.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T11:06:59.529 INFO:teuthology.orchestra.run.smithi182.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:06:59.530 INFO:teuthology.orchestra.run.smithi182.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T11:06:59.530 INFO:teuthology.orchestra.run.smithi182.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.530 INFO:teuthology.orchestra.run.smithi182.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.530 INFO:teuthology.orchestra.run.smithi182.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T11:06:59.530 INFO:teuthology.orchestra.run.smithi182.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.530 INFO:teuthology.orchestra.run.smithi182.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T11:06:59.530 INFO:teuthology.orchestra.run.smithi182.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T11:06:59.531 INFO:teuthology.orchestra.run.smithi182.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T11:06:59.531 INFO:teuthology.orchestra.run.smithi182.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T11:06:59.531 INFO:teuthology.orchestra.run.smithi182.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T11:06:59.531 INFO:teuthology.orchestra.run.smithi182.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T11:06:59.531 INFO:teuthology.orchestra.run.smithi182.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T11:06:59.531 INFO:teuthology.orchestra.run.smithi182.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T11:06:59.531 INFO:teuthology.orchestra.run.smithi182.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T11:06:59.532 INFO:teuthology.orchestra.run.smithi182.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T11:06:59.532 INFO:teuthology.orchestra.run.smithi182.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T11:06:59.532 INFO:teuthology.orchestra.run.smithi182.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T11:06:59.532 INFO:teuthology.orchestra.run.smithi182.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T11:06:59.532 INFO:teuthology.orchestra.run.smithi182.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T11:06:59.532 INFO:teuthology.orchestra.run.smithi182.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:06:59.532 INFO:teuthology.orchestra.run.smithi182.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:06:59.533 INFO:teuthology.orchestra.run.smithi182.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:06:59.533 INFO:teuthology.orchestra.run.smithi182.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T11:06:59.533 INFO:teuthology.orchestra.run.smithi182.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T11:06:59.533 INFO:teuthology.orchestra.run.smithi182.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T11:06:59.533 INFO:teuthology.orchestra.run.smithi182.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T11:06:59.533 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T11:06:59.533 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.534 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.534 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.534 INFO:teuthology.orchestra.run.smithi182.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T11:06:59.534 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T11:06:59.534 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T11:06:59.534 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T11:06:59.534 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T11:06:59.535 INFO:teuthology.orchestra.run.smithi182.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T11:06:59.535 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T11:06:59.535 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T11:06:59.535 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T11:06:59.535 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T11:06:59.535 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T11:06:59.535 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T11:06:59.536 INFO:teuthology.orchestra.run.smithi182.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T11:06:59.536 INFO:teuthology.orchestra.run.smithi182.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T11:06:59.536 INFO:teuthology.orchestra.run.smithi182.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T11:06:59.536 INFO:teuthology.orchestra.run.smithi182.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T11:06:59.536 INFO:teuthology.orchestra.run.smithi182.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T11:06:59.536 INFO:teuthology.orchestra.run.smithi182.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T11:06:59.536 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T11:06:59.537 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T11:06:59.537 INFO:teuthology.orchestra.run.smithi182.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T11:06:59.537 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T11:06:59.537 INFO:teuthology.orchestra.run.smithi182.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T11:06:59.537 INFO:teuthology.orchestra.run.smithi182.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T11:06:59.537 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T11:06:59.538 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T11:06:59.538 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T11:06:59.538 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T11:06:59.538 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T11:06:59.538 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.538 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.538 INFO:teuthology.orchestra.run.smithi182.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T11:06:59.539 INFO:teuthology.orchestra.run.smithi182.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T11:06:59.539 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.539 INFO:teuthology.orchestra.run.smithi182.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T11:06:59.539 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T11:06:59.539 INFO:teuthology.orchestra.run.smithi182.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T11:06:59.539 INFO:teuthology.orchestra.run.smithi182.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T11:06:59.539 INFO:teuthology.orchestra.run.smithi182.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T11:06:59.540 INFO:teuthology.orchestra.run.smithi182.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T11:06:59.540 INFO:teuthology.orchestra.run.smithi182.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T11:06:59.540 INFO:teuthology.orchestra.run.smithi182.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T11:06:59.540 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T11:06:59.540 INFO:teuthology.orchestra.run.smithi182.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T11:06:59.540 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T11:06:59.540 INFO:teuthology.orchestra.run.smithi182.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T11:06:59.541 INFO:teuthology.orchestra.run.smithi182.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T11:06:59.542 INFO:teuthology.orchestra.run.smithi182.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T11:06:59.542 INFO:teuthology.orchestra.run.smithi182.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T11:06:59.542 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:06:59.542 INFO:teuthology.orchestra.run.smithi182.stdout:Complete! 2023-07-14T11:06:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T11:06:59.704 DEBUG:teuthology.parallel:result is None 2023-07-14T11:06:59.833 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T11:07:00.028 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:07:00.243 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:07:00.497 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:07:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:07:01.653 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:07:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:07:02.166 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T11:07:02.372 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:07:02.407 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:07:02.577 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:07:02.588 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:07:03.874 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T11:07:04.062 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:07:04.095 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:07:04.294 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:07:04.335 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:07:04.493 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:07:04.836 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:07:04.994 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:07:09.563 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:07:09.868 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T11:07:10.026 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T11:07:10.214 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T11:07:11.107 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T11:07:11.457 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T11:07:11.628 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/134 2023-07-14T11:07:11.854 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T11:07:12.004 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T11:07:12.185 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:07:14.367 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:07:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:07:14.676 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:07:14.833 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T11:07:15.149 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T11:07:15.449 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T11:07:15.785 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T11:07:15.958 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:07:15.966 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:07:16.123 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T11:07:16.357 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T11:07:16.435 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T11:07:16.641 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T11:07:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T11:07:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:07:16.878 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:07:16.878 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:07:17.096 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:07:17.332 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:07:17.849 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:07:17.849 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T11:07:17.849 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T11:07:17.850 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T11:07:17.850 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T11:07:17.850 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T11:07:17.850 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:07:17.850 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:07:17.850 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:07:17.850 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T11:07:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T11:07:17.852 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T11:07:17.852 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T11:07:17.852 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T11:07:17.852 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T11:07:17.852 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T11:07:17.852 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T11:07:17.852 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T11:07:17.853 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T11:07:17.853 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T11:07:17.859 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T11:07:17.860 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T11:07:17.860 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T11:07:17.860 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T11:07:17.860 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T11:07:17.860 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T11:07:17.860 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T11:07:17.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T11:07:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T11:07:17.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T11:07:17.864 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T11:07:17.864 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T11:07:17.864 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T11:07:17.864 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T11:07:17.864 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T11:07:17.864 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T11:07:17.864 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T11:07:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T11:07:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T11:07:17.867 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T11:07:17.867 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T11:07:17.867 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T11:07:17.867 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T11:07:17.867 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T11:07:17.867 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T11:07:17.867 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T11:07:17.868 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T11:07:17.868 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T11:07:17.868 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T11:07:17.868 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T11:07:17.868 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T11:07:17.868 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T11:07:17.868 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T11:07:17.869 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T11:07:17.870 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T11:07:17.871 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T11:07:17.872 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T11:07:17.872 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T11:07:17.872 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T11:07:17.872 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:07:17.872 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T11:07:17.872 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T11:07:17.872 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:07:17.873 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T11:07:20.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:07:20.344 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:03:20 ago on Fri 14 Jul 2023 11:04:00 AM UTC. 2023-07-14T11:07:29.454 INFO:teuthology.orchestra.run.smithi175.stdout:Installed products updated. 2023-07-14T11:07:29.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:07:29.668 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2023-07-14T11:07:29.668 INFO:teuthology.orchestra.run.smithi175.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T11:07:29.669 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.669 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.669 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-07-14T11:07:29.669 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T11:07:29.669 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T11:07:29.669 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T11:07:29.669 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.670 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.671 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:07:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: jq-1.6-3.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T11:07:29.673 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.674 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.674 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.674 INFO:teuthology.orchestra.run.smithi175.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T11:07:29.674 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:07:29.674 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T11:07:29.674 INFO:teuthology.orchestra.run.smithi175.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T11:07:29.674 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T11:07:29.675 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T11:07:29.676 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T11:07:29.677 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T11:07:29.677 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T11:07:29.677 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T11:07:29.677 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:07:29.677 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:07:29.677 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:07:29.677 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.678 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T11:07:29.679 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T11:07:29.680 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T11:07:29.681 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T11:07:29.682 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T11:07:29.682 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T11:07:29.682 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T11:07:29.682 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T11:07:29.682 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.682 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.682 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T11:07:29.683 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T11:07:29.684 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.685 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.685 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:07:29.685 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T11:07:29.685 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T11:07:29.685 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T11:07:29.685 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T11:07:29.685 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T11:07:29.686 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T11:07:29.686 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-07-14T11:07:29.686 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-07-14T11:07:29.864 DEBUG:teuthology.parallel:result is None 2023-07-14T11:08:22.888 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 58 kB/s | 16 MB 04:36 2023-07-14T11:08:26.960 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 521 kB/s | 24 kB 00:00 2023-07-14T11:08:28.104 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-07-14T11:08:36.954 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-07-14T11:08:48.532 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:08:48.533 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:08:48.548 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T11:08:48.549 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T11:08:48.551 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T11:08:48.552 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T11:08:48.554 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T11:08:48.558 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T11:08:48.559 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T11:08:48.697 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T11:08:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T11:08:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-07-14T11:08:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T11:08:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-07-14T11:08:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T11:08:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T11:08:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T11:08:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T11:08:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T11:08:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T11:08:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T11:08:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T11:08:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T11:08:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T11:08:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T11:08:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T11:08:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T11:08:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T11:08:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T11:08:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T11:08:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T11:08:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T11:08:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T11:08:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T11:08:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T11:08:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T11:08:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T11:08:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T11:08:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T11:08:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T11:08:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T11:08:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T11:08:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T11:08:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T11:08:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T11:08:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T11:08:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T11:08:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T11:08:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T11:08:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T11:08:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T11:08:48.716 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2023-07-14T11:08:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T11:08:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T11:08:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T11:08:48.717 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-07-14T11:08:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T11:08:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T11:08:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T11:08:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T11:08:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T11:08:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T11:08:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T11:08:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T11:08:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T11:08:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T11:08:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T11:08:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T11:08:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T11:08:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T11:08:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T11:08:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T11:08:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T11:08:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T11:08:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T11:08:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T11:08:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T11:08:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T11:08:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T11:08:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T11:08:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T11:08:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T11:08:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T11:08:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T11:08:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T11:08:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T11:08:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T11:08:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T11:08:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T11:08:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T11:08:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T11:08:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T11:08:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T11:08:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T11:08:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T11:08:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T11:08:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T11:08:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T11:08:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T11:08:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T11:08:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T11:08:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T11:08:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T11:08:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T11:08:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T11:08:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T11:08:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T11:08:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T11:08:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T11:08:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:08:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T11:08:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T11:08:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T11:08:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T11:08:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T11:08:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T11:08:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T11:08:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T11:08:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T11:08:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T11:08:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T11:08:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T11:08:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T11:08:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T11:08:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T11:08:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T11:08:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T11:08:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T11:08:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:08:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T11:08:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T11:08:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T11:08:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T11:08:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T11:08:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T11:08:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T11:08:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T11:08:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T11:08:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T11:08:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T11:08:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T11:08:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T11:08:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T11:08:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T11:08:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T11:08:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T11:08:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T11:08:48.730 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2023-07-14T11:08:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql 10 2023-07-14T11:08:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:08:48.730 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T11:08:48.731 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T11:08:48.731 INFO:teuthology.orchestra.run.smithi161.stdout:Install 128 Packages 2023-07-14T11:08:48.731 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 3 Packages 2023-07-14T11:08:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:08:48.731 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 205 M 2023-07-14T11:08:48.732 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-07-14T11:08:49.421 INFO:teuthology.orchestra.run.smithi161.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 11 kB/s | 6.4 kB 00:00 2023-07-14T11:08:51.130 INFO:teuthology.orchestra.run.smithi161.stdout:(2/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 2.3 MB/s | 5.4 MB 00:02 2023-07-14T11:08:51.196 INFO:teuthology.orchestra.run.smithi161.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 2.4 MB/s | 165 kB 00:00 2023-07-14T11:08:51.947 INFO:teuthology.orchestra.run.smithi161.stdout:(4/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 3.1 MB/s | 2.3 MB 00:00 2023-07-14T11:08:52.030 INFO:teuthology.orchestra.run.smithi161.stdout:(5/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 325 kB/s | 848 kB 00:02 2023-07-14T11:08:52.505 INFO:teuthology.orchestra.run.smithi161.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 2.8 MB/s | 1.5 MB 00:00 2023-07-14T11:08:56.572 INFO:teuthology.orchestra.run.smithi161.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 1.0 MB/s | 4.6 MB 00:04 2023-07-14T11:08:57.839 INFO:teuthology.orchestra.run.smithi161.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 2.5 MB/s | 22 MB 00:09 2023-07-14T11:08:57.914 INFO:teuthology.orchestra.run.smithi161.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 328 kB/s | 24 kB 00:00 2023-07-14T11:08:59.656 INFO:teuthology.orchestra.run.smithi161.stdout:(10/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 2.5 MB/s | 18 MB 00:07 2023-07-14T11:08:59.789 INFO:teuthology.orchestra.run.smithi161.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 1.9 MB/s | 258 kB 00:00 2023-07-14T11:08:59.881 INFO:teuthology.orchestra.run.smithi161.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 339 kB/s | 31 kB 00:00 2023-07-14T11:09:00.218 INFO:teuthology.orchestra.run.smithi161.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 2.2 MB/s | 771 kB 00:00 2023-07-14T11:09:00.299 INFO:teuthology.orchestra.run.smithi161.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.7 MB/s | 137 kB 00:00 2023-07-14T11:09:00.390 INFO:teuthology.orchestra.run.smithi161.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 1.5 MB/s | 139 kB 00:00 2023-07-14T11:09:00.657 INFO:teuthology.orchestra.run.smithi161.stdout:(16/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 3.3 MB/s | 13 MB 00:04 2023-07-14T11:09:00.683 INFO:teuthology.orchestra.run.smithi161.stdout:(17/131): libradosstriper1-18.0.0-4757.g3fc0089 1.6 MB/s | 473 kB 00:00 2023-07-14T11:09:00.783 INFO:teuthology.orchestra.run.smithi161.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 448 kB/s | 45 kB 00:00 2023-07-14T11:09:00.875 INFO:teuthology.orchestra.run.smithi161.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 1.3 MB/s | 119 kB 00:00 2023-07-14T11:09:01.000 INFO:teuthology.orchestra.run.smithi161.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 1.5 MB/s | 197 kB 00:00 2023-07-14T11:09:01.176 INFO:teuthology.orchestra.run.smithi161.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 2.1 MB/s | 371 kB 00:00 2023-07-14T11:09:01.317 INFO:teuthology.orchestra.run.smithi161.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 2.5 MB/s | 364 kB 00:00 2023-07-14T11:09:01.393 INFO:teuthology.orchestra.run.smithi161.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 1.4 MB/s | 107 kB 00:00 2023-07-14T11:09:01.476 INFO:teuthology.orchestra.run.smithi161.stdout:(24/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.1 MB/s | 94 kB 00:00 2023-07-14T11:09:01.760 INFO:teuthology.orchestra.run.smithi161.stdout:(25/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 6.1 MB/s | 6.7 MB 00:01 2023-07-14T11:09:01.852 INFO:teuthology.orchestra.run.smithi161.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.9 MB/s | 179 kB 00:00 2023-07-14T11:09:01.961 INFO:teuthology.orchestra.run.smithi161.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 212 kB/s | 23 kB 00:00 2023-07-14T11:09:02.061 INFO:teuthology.orchestra.run.smithi161.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 1.3 MB/s | 129 kB 00:00 2023-07-14T11:09:02.478 INFO:teuthology.orchestra.run.smithi161.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 4.1 MB/s | 1.7 MB 00:00 2023-07-14T11:09:09.927 INFO:teuthology.orchestra.run.smithi161.stdout:(30/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 418 kB/s | 3.4 MB 00:08 2023-07-14T11:09:10.769 INFO:teuthology.orchestra.run.smithi161.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 286 kB/s | 240 kB 00:00 2023-07-14T11:09:11.030 INFO:teuthology.orchestra.run.smithi161.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 181 kB/s | 47 kB 00:00 2023-07-14T11:09:11.136 INFO:teuthology.orchestra.run.smithi161.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 138 kB/s | 15 kB 00:00 2023-07-14T11:09:11.836 INFO:teuthology.orchestra.run.smithi161.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 369 kB/s | 258 kB 00:00 2023-07-14T11:09:12.237 INFO:teuthology.orchestra.run.smithi161.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 416 kB/s | 166 kB 00:00 2023-07-14T11:09:12.354 INFO:teuthology.orchestra.run.smithi161.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 154 kB/s | 18 kB 00:00 2023-07-14T11:09:13.197 INFO:teuthology.orchestra.run.smithi161.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 363 kB/s | 306 kB 00:00 2023-07-14T11:09:13.725 INFO:teuthology.orchestra.run.smithi161.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 111 kB/s | 59 kB 00:00 2023-07-14T11:09:14.705 INFO:teuthology.orchestra.run.smithi161.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 77 kB/s | 75 kB 00:00 2023-07-14T11:09:20.213 INFO:teuthology.orchestra.run.smithi161.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 86 kB/s | 476 kB 00:05 2023-07-14T11:09:20.463 INFO:teuthology.orchestra.run.smithi161.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 175 kB/s | 44 kB 00:00 2023-07-14T11:09:20.939 INFO:teuthology.orchestra.run.smithi161.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 390 kB/s | 185 kB 00:00 2023-07-14T11:09:21.056 INFO:teuthology.orchestra.run.smithi161.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 283 kB/s | 33 kB 00:00 2023-07-14T11:09:21.123 INFO:teuthology.orchestra.run.smithi161.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2023-07-14T11:09:21.456 INFO:teuthology.orchestra.run.smithi161.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 518 kB/s | 173 kB 00:00 2023-07-14T11:09:22.040 INFO:teuthology.orchestra.run.smithi161.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 660 kB/s | 384 kB 00:00 2023-07-14T11:09:22.207 INFO:teuthology.orchestra.run.smithi161.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 488 kB/s | 81 kB 00:00 2023-07-14T11:09:22.266 INFO:teuthology.orchestra.run.smithi161.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 191 kB/s | 11 kB 00:00 2023-07-14T11:09:22.341 INFO:teuthology.orchestra.run.smithi161.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-07-14T11:09:24.724 INFO:teuthology.orchestra.run.smithi161.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 501 kB/s | 1.2 MB 00:02 2023-07-14T11:09:24.900 INFO:teuthology.orchestra.run.smithi161.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 279 kB/s | 49 kB 00:00 2023-07-14T11:09:25.083 INFO:teuthology.orchestra.run.smithi161.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 320 kB/s | 59 kB 00:00 2023-07-14T11:09:25.234 INFO:teuthology.orchestra.run.smithi161.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 403 kB/s | 60 kB 00:00 2023-07-14T11:09:25.351 INFO:teuthology.orchestra.run.smithi161.stdout:(54/131): ceph-mgr-diskprediction-local-18.0.0- 332 kB/s | 7.4 MB 00:22 2023-07-14T11:09:25.592 INFO:teuthology.orchestra.run.smithi161.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 65 kB/s | 16 kB 00:00 2023-07-14T11:09:25.726 INFO:teuthology.orchestra.run.smithi161.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 253 kB/s | 34 kB 00:00 2023-07-14T11:09:25.793 INFO:teuthology.orchestra.run.smithi161.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 506 kB/s | 283 kB 00:00 2023-07-14T11:09:25.885 INFO:teuthology.orchestra.run.smithi161.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 673 kB/s | 61 kB 00:00 2023-07-14T11:09:25.968 INFO:teuthology.orchestra.run.smithi161.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 225 kB/s | 19 kB 00:00 2023-07-14T11:09:26.044 INFO:teuthology.orchestra.run.smithi161.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 319 kB/s | 24 kB 00:00 2023-07-14T11:09:26.069 INFO:teuthology.orchestra.run.smithi161.stdout:(61/131): python3-routes-2.4.1-12.el8.noarch.rp 573 kB/s | 196 kB 00:00 2023-07-14T11:09:26.136 INFO:teuthology.orchestra.run.smithi161.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 317 kB/s | 29 kB 00:00 2023-07-14T11:09:26.162 INFO:teuthology.orchestra.run.smithi161.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 298 kB/s | 27 kB 00:00 2023-07-14T11:09:26.345 INFO:teuthology.orchestra.run.smithi161.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-07-14T11:09:26.437 INFO:teuthology.orchestra.run.smithi161.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 672 kB/s | 61 kB 00:00 2023-07-14T11:09:26.529 INFO:teuthology.orchestra.run.smithi161.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 651 kB/s | 255 kB 00:00 2023-07-14T11:09:26.554 INFO:teuthology.orchestra.run.smithi161.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 730 kB/s | 85 kB 00:00 2023-07-14T11:09:26.646 INFO:teuthology.orchestra.run.smithi161.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-07-14T11:09:26.672 INFO:teuthology.orchestra.run.smithi161.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-14T11:09:26.855 INFO:teuthology.orchestra.run.smithi161.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 255 kB/s | 47 kB 00:00 2023-07-14T11:09:26.997 INFO:teuthology.orchestra.run.smithi161.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2023-07-14T11:09:27.189 INFO:teuthology.orchestra.run.smithi161.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 223 kB/s | 43 kB 00:00 2023-07-14T11:09:27.290 INFO:teuthology.orchestra.run.smithi161.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.2 MB/s | 220 kB 00:00 2023-07-14T11:09:27.449 INFO:teuthology.orchestra.run.smithi161.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-07-14T11:09:27.498 INFO:teuthology.orchestra.run.smithi161.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 231 kB/s | 48 kB 00:00 2023-07-14T11:09:27.657 INFO:teuthology.orchestra.run.smithi161.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 279 kB/s | 43 kB 00:00 2023-07-14T11:09:27.682 INFO:teuthology.orchestra.run.smithi161.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 373 kB/s | 87 kB 00:00 2023-07-14T11:09:27.791 INFO:teuthology.orchestra.run.smithi161.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-07-14T11:09:27.825 INFO:teuthology.orchestra.run.smithi161.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 653 kB/s | 109 kB 00:00 2023-07-14T11:09:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.5 MB/s | 560 kB 00:00 2023-07-14T11:09:27.975 INFO:teuthology.orchestra.run.smithi161.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 124 kB/s | 19 kB 00:00 2023-07-14T11:09:28.000 INFO:teuthology.orchestra.run.smithi161.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-07-14T11:09:28.051 INFO:teuthology.orchestra.run.smithi161.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 7.3 MB/s | 559 kB 00:00 2023-07-14T11:09:28.076 INFO:teuthology.orchestra.run.smithi161.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 238 kB/s | 18 kB 00:00 2023-07-14T11:09:28.143 INFO:teuthology.orchestra.run.smithi161.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-07-14T11:09:28.177 INFO:teuthology.orchestra.run.smithi161.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-07-14T11:09:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2023-07-14T11:09:28.302 INFO:teuthology.orchestra.run.smithi161.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-07-14T11:09:28.328 INFO:teuthology.orchestra.run.smithi161.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-07-14T11:09:28.386 INFO:teuthology.orchestra.run.smithi161.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 714 kB/s | 60 kB 00:00 2023-07-14T11:09:28.412 INFO:teuthology.orchestra.run.smithi161.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.1 MB/s | 98 kB 00:00 2023-07-14T11:09:28.445 INFO:teuthology.orchestra.run.smithi161.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 987 kB/s | 58 kB 00:00 2023-07-14T11:09:28.471 INFO:teuthology.orchestra.run.smithi161.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 661 kB/s | 39 kB 00:00 2023-07-14T11:09:28.513 INFO:teuthology.orchestra.run.smithi161.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-07-14T11:09:28.538 INFO:teuthology.orchestra.run.smithi161.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-14T11:09:28.571 INFO:teuthology.orchestra.run.smithi161.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-07-14T11:09:28.614 INFO:teuthology.orchestra.run.smithi161.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-14T11:09:28.680 INFO:teuthology.orchestra.run.smithi161.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 816 kB/s | 54 kB 00:00 2023-07-14T11:09:28.935 INFO:teuthology.orchestra.run.smithi161.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2023-07-14T11:09:28.989 INFO:teuthology.orchestra.run.smithi161.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.9 MB/s | 323 kB 00:00 2023-07-14T11:09:29.031 INFO:teuthology.orchestra.run.smithi161.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 938 kB/s | 39 kB 00:00 2023-07-14T11:09:29.098 INFO:teuthology.orchestra.run.smithi161.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-14T11:09:29.148 INFO:teuthology.orchestra.run.smithi161.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-14T11:09:29.316 INFO:teuthology.orchestra.run.smithi161.stdout:(104/131): python3-scipy-1.0.0-21.module+el8.5. 18 MB/s | 14 MB 00:00 2023-07-14T11:09:29.341 INFO:teuthology.orchestra.run.smithi161.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 818 kB/s | 157 kB 00:00 2023-07-14T11:09:29.366 INFO:teuthology.orchestra.run.smithi161.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-14T11:09:29.408 INFO:teuthology.orchestra.run.smithi161.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-07-14T11:09:29.450 INFO:teuthology.orchestra.run.smithi161.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 638 kB/s | 27 kB 00:00 2023-07-14T11:09:29.500 INFO:teuthology.orchestra.run.smithi161.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-07-14T11:09:29.542 INFO:teuthology.orchestra.run.smithi161.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-07-14T11:09:29.634 INFO:teuthology.orchestra.run.smithi161.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-07-14T11:09:29.684 INFO:teuthology.orchestra.run.smithi161.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-07-14T11:09:29.751 INFO:teuthology.orchestra.run.smithi161.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 3.3 MB/s | 225 kB 00:00 2023-07-14T11:09:29.810 INFO:teuthology.orchestra.run.smithi161.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-07-14T11:09:29.835 INFO:teuthology.orchestra.run.smithi161.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 764 kB/s | 64 kB 00:00 2023-07-14T11:09:29.861 INFO:teuthology.orchestra.run.smithi161.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-14T11:09:29.886 INFO:teuthology.orchestra.run.smithi161.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-07-14T11:09:29.903 INFO:teuthology.orchestra.run.smithi161.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-07-14T11:09:29.937 INFO:teuthology.orchestra.run.smithi161.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-07-14T11:09:29.979 INFO:teuthology.orchestra.run.smithi161.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-14T11:09:30.096 INFO:teuthology.orchestra.run.smithi161.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 25 MB/s | 4.9 MB 00:00 2023-07-14T11:09:30.181 INFO:teuthology.orchestra.run.smithi161.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 2.3 MB/s | 202 kB 00:00 2023-07-14T11:09:30.263 INFO:teuthology.orchestra.run.smithi161.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-07-14T11:09:30.297 INFO:teuthology.orchestra.run.smithi161.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-07-14T11:09:30.364 INFO:teuthology.orchestra.run.smithi161.stdout:(125/131): postgresql-contrib-10.21-2.module+el 12 MB/s | 808 kB 00:00 2023-07-14T11:09:30.447 INFO:teuthology.orchestra.run.smithi161.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-07-14T11:09:30.514 INFO:teuthology.orchestra.run.smithi161.stdout:(127/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-07-14T11:09:32.406 INFO:teuthology.orchestra.run.smithi161.stdout:(128/131): librados2-18.0.0-4757.g3fc0089c.el8. 1.9 MB/s | 3.7 MB 00:01 2023-07-14T11:09:32.481 INFO:teuthology.orchestra.run.smithi161.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 11 MB/s | 798 kB 00:00 2023-07-14T11:09:34.206 INFO:teuthology.orchestra.run.smithi161.stdout:(130/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 987 kB/s | 3.6 MB 00:03 2023-07-14T11:09:41.264 INFO:teuthology.orchestra.run.smithi161.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 1.1 MB/s | 48 MB 00:43 2023-07-14T11:09:41.267 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:09:41.267 INFO:teuthology.orchestra.run.smithi161.stdout:Total 3.9 MB/s | 205 MB 00:52 2023-07-14T11:09:42.918 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-07-14T11:09:43.053 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-07-14T11:09:43.053 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-07-14T11:09:44.651 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-07-14T11:09:44.656 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-07-14T11:09:49.725 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-07-14T11:09:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T11:09:50.175 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T11:09:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T11:09:50.488 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T11:09:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T11:09:51.209 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T11:09:51.457 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T11:09:51.908 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T11:09:52.090 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:09:52.237 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:09:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:09:52.582 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:09:52.788 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:09:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-14T11:09:53.248 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T11:09:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T11:09:53.938 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T11:09:54.186 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T11:09:54.548 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:09:54.686 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:09:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:09:55.005 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:09:55.246 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-14T11:09:55.460 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-14T11:09:55.672 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-14T11:09:55.823 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T11:09:56.022 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T11:09:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T11:09:56.936 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T11:09:57.114 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T11:09:57.826 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T11:09:58.006 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T11:09:58.860 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T11:09:59.147 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-14T11:09:59.365 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-14T11:09:59.625 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T11:09:59.830 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T11:09:59.969 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T11:10:00.200 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T11:10:00.439 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T11:10:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T11:10:03.679 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-14T11:10:04.052 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-14T11:10:04.354 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-14T11:10:04.589 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-14T11:10:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T11:10:04.995 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T11:10:05.222 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T11:10:05.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T11:10:05.630 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-14T11:10:05.801 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-14T11:10:06.038 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T11:10:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T11:10:06.404 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-14T11:10:06.627 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-14T11:10:06.801 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T11:10:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T11:10:07.116 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-14T11:10:07.309 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-14T11:10:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-14T11:10:07.699 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-14T11:10:07.882 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-14T11:10:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-14T11:10:08.282 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-14T11:10:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-14T11:10:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-14T11:10:09.286 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-14T11:10:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-14T11:10:09.790 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-14T11:10:10.064 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-14T11:10:10.397 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-14T11:10:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-14T11:10:10.874 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-14T11:10:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-14T11:10:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-14T11:10:11.460 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-14T11:10:11.820 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-14T11:10:12.099 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-14T11:10:12.360 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-14T11:10:12.607 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-14T11:10:12.797 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T11:10:13.012 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T11:10:13.262 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T11:10:13.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T11:10:13.684 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T11:10:13.891 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T11:10:14.093 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T11:10:14.336 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T11:10:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T11:10:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T11:10:14.945 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T11:10:15.188 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T11:10:15.410 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T11:10:15.653 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T11:10:15.920 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T11:10:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T11:10:16.331 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T11:10:16.607 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T11:10:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T11:10:18.282 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T11:10:18.534 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T11:10:18.927 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T11:10:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T11:10:19.451 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T11:10:20.357 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T11:10:20.569 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:10:20.662 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:10:20.894 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T11:10:25.889 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:10:26.111 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:10:26.812 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:10:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:10:27.241 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:10:27.376 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:10:47.937 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:10:47.938 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2023-07-14T11:10:47.938 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2023-07-14T11:10:47.938 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-07-14T11:10:47.938 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2023-07-14T11:10:47.938 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2023-07-14T11:10:47.939 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2023-07-14T11:10:47.939 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2023-07-14T11:10:47.939 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2023-07-14T11:10:47.939 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-07-14T11:10:47.939 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:10:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:10:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:10:50.265 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:10:50.535 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:10:50.731 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:10:51.261 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:10:51.456 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:10:51.498 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:10:54.180 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:10:54.401 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:10:54.530 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T11:10:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T11:10:54.988 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:10:55.248 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:10:55.464 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:10:56.581 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:10:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:10:56.894 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:10:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T11:10:57.558 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:10:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:10:57.838 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:10:57.867 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:10:59.277 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T11:10:59.522 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:10:59.593 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:10:59.747 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:10:59.826 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:11:00.063 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:11:00.473 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:11:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:11:05.674 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:11:06.901 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T11:11:07.224 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-14T11:11:07.401 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 117/134 2023-07-14T11:11:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-14T11:11:07.820 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T11:11:08.072 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T11:11:08.317 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-14T11:11:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T11:11:08.680 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:11:11.955 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:11:12.254 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:11:12.291 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:11:12.562 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T11:11:12.754 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T11:11:13.095 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T11:11:13.471 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T11:11:13.661 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T11:11:13.777 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T11:11:13.986 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T11:11:14.014 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T11:11:14.227 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T11:11:14.250 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T11:11:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-14T11:11:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:11:14.676 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:11:14.676 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:11:14.925 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:11:15.169 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:11:15.430 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:11:15.430 INFO:teuthology.orchestra.run.smithi161.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-14T11:11:15.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:11:16.094 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T11:11:16.095 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T11:11:16.095 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T11:11:16.095 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T11:11:16.095 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T11:11:16.095 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:11:16.095 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:11:16.095 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:11:16.096 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:11:16.096 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T11:11:16.096 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T11:11:16.096 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T11:11:16.096 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T11:11:16.096 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T11:11:16.096 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T11:11:16.097 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T11:11:16.098 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T11:11:16.098 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T11:11:16.098 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T11:11:16.098 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T11:11:16.098 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T11:11:16.098 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T11:11:16.105 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T11:11:16.105 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T11:11:16.105 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T11:11:16.105 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T11:11:16.105 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T11:11:16.106 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T11:11:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T11:11:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T11:11:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T11:11:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T11:11:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T11:11:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T11:11:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T11:11:16.108 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T11:11:16.109 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T11:11:16.110 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T11:11:16.110 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T11:11:16.110 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T11:11:16.110 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T11:11:16.110 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T11:11:16.110 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T11:11:16.110 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-14T11:11:16.111 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-14T11:11:16.112 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-14T11:11:16.112 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-14T11:11:16.112 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-14T11:11:16.112 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-14T11:11:16.112 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-14T11:11:16.112 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-14T11:11:16.112 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-14T11:11:16.113 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-14T11:11:16.114 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-14T11:11:16.115 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-14T11:11:16.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-14T11:11:16.117 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-14T11:11:16.117 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-14T11:11:16.117 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-14T11:11:16.117 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-14T11:11:16.117 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-14T11:11:16.117 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-14T11:11:16.117 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-14T11:11:16.118 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-14T11:11:16.118 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-14T11:11:16.118 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T11:11:16.118 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T11:11:16.118 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T11:11:16.118 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:11:16.118 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T11:11:18.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:11:18.882 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:42 ago on Fri 14 Jul 2023 11:08:36 AM UTC. 2023-07-14T11:11:21.535 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2023-07-14T11:11:21.775 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:11:21.776 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2023-07-14T11:11:21.776 INFO:teuthology.orchestra.run.smithi161.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T11:11:21.776 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.776 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.776 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-07-14T11:11:21.776 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T11:11:21.777 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T11:11:21.777 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T11:11:21.777 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.777 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.777 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.777 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.777 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.779 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T11:11:21.780 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-3.el8.x86_64 2023-07-14T11:11:21.781 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T11:11:21.781 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T11:11:21.781 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T11:11:21.781 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.781 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.781 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.781 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.782 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T11:11:21.783 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.783 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T11:11:21.783 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T11:11:21.783 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T11:11:21.783 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T11:11:21.783 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T11:11:21.783 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T11:11:21.784 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T11:11:21.785 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:11:21.785 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:11:21.785 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:11:21.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T11:11:21.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T11:11:21.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T11:11:21.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T11:11:21.786 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T11:11:21.787 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T11:11:21.787 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T11:11:21.787 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T11:11:21.787 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T11:11:21.787 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T11:11:21.787 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T11:11:21.787 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T11:11:21.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T11:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T11:11:21.790 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T11:11:21.790 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T11:11:21.790 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.790 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.790 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T11:11:21.790 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T11:11:21.790 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T11:11:21.791 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T11:11:21.792 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T11:11:21.792 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T11:11:21.792 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T11:11:21.792 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T11:11:21.792 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T11:11:21.792 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.792 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.793 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:11:21.793 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T11:11:21.793 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T11:11:21.793 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T11:11:21.793 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T11:11:21.793 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T11:11:21.794 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T11:11:21.794 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:11:21.794 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T11:11:22.751 DEBUG:teuthology.parallel:result is None 2023-07-14T11:11:22.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:11:23.098 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T11:11:23.152 INFO:teuthology.orchestra.run.smithi161.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T11:11:23.153 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T11:11:23.154 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T11:11:23.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:11:23.413 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T11:11:23.442 INFO:teuthology.orchestra.run.smithi175.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T11:11:23.442 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T11:11:23.443 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T11:11:23.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:11:23.688 DEBUG:teuthology.orchestra.run.smithi182:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T11:11:23.717 INFO:teuthology.orchestra.run.smithi182.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T11:11:23.718 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T11:11:23.718 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T11:11:23.718 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-14T11:11:23.719 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:11:23.719 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:11:23.808 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:11:23.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:11:23.855 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:11:23.855 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:11:23.902 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-14T11:11:23.902 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:11:23.902 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:11:23.970 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:11:24.078 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:11:24.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:11:24.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:11:24.180 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:11:24.180 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:11:24.212 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:11:24.282 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-14T11:11:24.282 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:11:24.283 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:11:24.349 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:11:24.456 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:11:24.456 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:11:24.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:11:24.557 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:11:24.558 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:11:24.588 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:11:24.657 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-14T11:11:24.658 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:11:24.658 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:11:24.725 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:11:24.832 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:11:24.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:11:24.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:11:24.934 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:11:24.934 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:11:24.965 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:11:25.035 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-14T11:11:25.118 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-07-14T11:11:25.118 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:11:25.128 INFO:tasks.cephadm:Cluster fsid is 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:11:25.129 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-14T11:11:25.129 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.161', 'mon.b': '172.21.15.175', 'mon.c': '172.21.15.182'} 2023-07-14T11:11:25.129 INFO:tasks.cephadm:First mon is mon.a on smithi161 2023-07-14T11:11:25.130 INFO:tasks.cephadm:First mgr is x 2023-07-14T11:11:25.130 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-14T11:11:25.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2023-07-14T11:11:25.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2023-07-14T11:11:25.233 DEBUG:teuthology.orchestra.run.smithi182:> sudo hostname $(hostname -s) 2023-07-14T11:11:25.265 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-14T11:11:25.266 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:11:25.704 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71399/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4757-g3fc0089c', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4757.g3fc0089c'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-13 19:47:46.444052', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-14T11:11:26.076 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13, sha1 3fc0089cc1082313a98853d48b892acdc1385b8b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:11:26.077 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm 2023-07-14T11:11:26.077 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:27.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 11:11 /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:27.326 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:29.365 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 11:11 /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:29.366 DEBUG:teuthology.orchestra.run.smithi182:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:30.951 INFO:teuthology.orchestra.run.smithi182.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 11:11 /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:30.952 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:31.003 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:31.024 DEBUG:teuthology.orchestra.run.smithi182:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T11:11:31.059 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b on all hosts... 2023-07-14T11:11:31.059 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T11:11:31.064 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T11:11:31.066 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T11:11:32.345 INFO:teuthology.orchestra.run.smithi182.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:11:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:11:35.312 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:12:00.042 INFO:teuthology.orchestra.run.smithi182.stdout:{ 2023-07-14T11:12:00.042 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:12:00.043 INFO:teuthology.orchestra.run.smithi182.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T11:12:00.043 INFO:teuthology.orchestra.run.smithi182.stdout: "repo_digests": [ 2023-07-14T11:12:00.043 INFO:teuthology.orchestra.run.smithi182.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T11:12:00.043 INFO:teuthology.orchestra.run.smithi182.stdout: ] 2023-07-14T11:12:00.043 INFO:teuthology.orchestra.run.smithi182.stdout:} 2023-07-14T11:12:02.574 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:12:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:12:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T11:12:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2023-07-14T11:12:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T11:12:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:12:02.575 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:12:06.166 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2023-07-14T11:12:06.167 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:12:06.167 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T11:12:06.167 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2023-07-14T11:12:06.167 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T11:12:06.167 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2023-07-14T11:12:06.167 INFO:teuthology.orchestra.run.smithi175.stdout:} 2023-07-14T11:12:06.187 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2023-07-14T11:12:06.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2023-07-14T11:12:06.295 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /etc/ceph 2023-07-14T11:12:06.332 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2023-07-14T11:12:06.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2023-07-14T11:12:06.428 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod 777 /etc/ceph 2023-07-14T11:12:06.458 INFO:tasks.cephadm:Writing seed config... 2023-07-14T11:12:06.459 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-14T11:12:06.460 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-14T11:12:06.460 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-14T11:12:06.460 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-14T11:12:06.460 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-14T11:12:06.460 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-14T11:12:06.461 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-14T11:12:06.461 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-14T11:12:06.461 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-14T11:12:06.461 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-14T11:12:06.461 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-14T11:12:06.462 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-14T11:12:06.462 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-14T11:12:06.462 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-14T11:12:06.462 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-14T11:12:06.462 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-14T11:12:06.463 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-14T11:12:06.463 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-14T11:12:06.463 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-14T11:12:06.463 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-14T11:12:06.463 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-14T11:12:06.464 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-14T11:12:06.464 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-14T11:12:06.464 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-14T11:12:06.464 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-14T11:12:06.464 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-14T11:12:06.465 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-14T11:12:06.465 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-14T11:12:06.465 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-14T11:12:06.465 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-14T11:12:06.465 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-14T11:12:06.466 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-14T11:12:06.466 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-14T11:12:06.466 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-14T11:12:06.466 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-14T11:12:06.466 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-14T11:12:06.467 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-14T11:12:06.467 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-14T11:12:06.467 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-14T11:12:06.467 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-14T11:12:06.467 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-14T11:12:06.467 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-14T11:12:06.468 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-14T11:12:06.468 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-07-14T11:12:06.469 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:12:06.469 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-14T11:12:06.512 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2bf7fc3c-2237-11ee-9b32-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-14T11:12:06.512 DEBUG:teuthology.orchestra.run.smithi161:mon.a> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a.service 2023-07-14T11:12:06.555 DEBUG:teuthology.orchestra.run.smithi161:mgr.x> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.x.service 2023-07-14T11:12:06.601 INFO:tasks.cephadm:Bootstrapping... 2023-07-14T11:12:06.602 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b -v bootstrap --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.161 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:12:06.623 INFO:journalctl@ceph.mon.a.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:12:06.670 INFO:journalctl@ceph.mgr.x.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:12:06.825 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:12:06.825 INFO:teuthology.orchestra.run.smithi161.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b', '-v', 'bootstrap', '--fsid', '2bf7fc3c-2237-11ee-9b32-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.161', '--skip-admin-label'] 2023-07-14T11:12:06.997 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stdout 4.1.1 2023-07-14T11:12:06.998 INFO:teuthology.orchestra.run.smithi161.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-14T11:12:06.998 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying podman|docker is present... 2023-07-14T11:12:07.168 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stdout 4.1.1 2023-07-14T11:12:07.168 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying lvm2 is present... 2023-07-14T11:12:07.169 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying time synchronization is in place... 2023-07-14T11:12:07.182 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T11:12:07.183 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T11:12:07.195 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T11:12:07.195 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout inactive 2023-07-14T11:12:07.208 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout enabled 2023-07-14T11:12:07.221 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout active 2023-07-14T11:12:07.221 INFO:teuthology.orchestra.run.smithi161.stdout:Unit chronyd.service is enabled and running 2023-07-14T11:12:07.222 INFO:teuthology.orchestra.run.smithi161.stdout:Repeating the final host check... 2023-07-14T11:12:07.392 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stdout 4.1.1 2023-07-14T11:12:07.393 INFO:teuthology.orchestra.run.smithi161.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-14T11:12:07.393 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl is present 2023-07-14T11:12:07.393 INFO:teuthology.orchestra.run.smithi161.stdout:lvcreate is present 2023-07-14T11:12:07.406 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T11:12:07.406 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T11:12:07.418 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T11:12:07.418 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout inactive 2023-07-14T11:12:07.432 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout enabled 2023-07-14T11:12:07.445 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout active 2023-07-14T11:12:07.445 INFO:teuthology.orchestra.run.smithi161.stdout:Unit chronyd.service is enabled and running 2023-07-14T11:12:07.445 INFO:teuthology.orchestra.run.smithi161.stdout:Host looks OK 2023-07-14T11:12:07.445 INFO:teuthology.orchestra.run.smithi161.stdout:Cluster fsid: 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:07.446 INFO:teuthology.orchestra.run.smithi161.stdout:Acquiring lock 140718657612264 on /run/cephadm/2bf7fc3c-2237-11ee-9b32-001a4aab830c.lock 2023-07-14T11:12:07.446 INFO:teuthology.orchestra.run.smithi161.stdout:Lock 140718657612264 acquired on /run/cephadm/2bf7fc3c-2237-11ee-9b32-001a4aab830c.lock 2023-07-14T11:12:07.446 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying IP 172.21.15.161 port 3300 ... 2023-07-14T11:12:07.446 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying IP 172.21.15.161 port 6789 ... 2023-07-14T11:12:07.447 INFO:teuthology.orchestra.run.smithi161.stdout:Base mon IP(s) is [172.21.15.161:3300, 172.21.15.161:6789], mon addrv is [v2:172.21.15.161:3300,v1:172.21.15.161:6789] 2023-07-14T11:12:07.455 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-14T11:12:07.455 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.161 metric 100 2023-07-14T11:12:07.461 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-14T11:12:07.462 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-14T11:12:07.462 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-14T11:12:07.468 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-14T11:12:07.468 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-14T11:12:07.468 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T11:12:07.468 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-14T11:12:07.468 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout inet6 fe80::7941:741e:f751:64dd/64 scope link noprefixroute 2023-07-14T11:12:07.469 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T11:12:07.469 INFO:teuthology.orchestra.run.smithi161.stdout:Mon IP `172.21.15.161` is in CIDR network `172.21.0.0/20` 2023-07-14T11:12:07.469 INFO:teuthology.orchestra.run.smithi161.stdout:Mon IP `172.21.15.161` is in CIDR network `172.21.0.0/20` 2023-07-14T11:12:07.469 INFO:teuthology.orchestra.run.smithi161.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-14T11:12:07.470 INFO:teuthology.orchestra.run.smithi161.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-14T11:12:07.471 INFO:teuthology.orchestra.run.smithi161.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:12:07.945 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stdout afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-14T11:12:07.946 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:12:07.946 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stderr Getting image source signatures 2023-07-14T11:12:07.946 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stderr Copying blob sha256:d8eac6e0ff3c58bd5607bffacb1c2214d20cdd3ff8cfaf23cfeb0cc60b2d0d6b 2023-07-14T11:12:07.946 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-14T11:12:07.946 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stderr Copying config sha256:afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-14T11:12:07.947 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-14T11:12:07.947 INFO:teuthology.orchestra.run.smithi161.stdout:/bin/podman: stderr Storing signatures 2023-07-14T11:12:08.921 INFO:teuthology.orchestra.run.smithi161.stdout:ceph: stdout ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-14T11:12:08.922 INFO:teuthology.orchestra.run.smithi161.stdout:Ceph version: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-14T11:12:08.922 INFO:teuthology.orchestra.run.smithi161.stdout:Extracting ceph user uid/gid from container image... 2023-07-14T11:12:09.696 INFO:teuthology.orchestra.run.smithi161.stdout:stat: stdout 167 167 2023-07-14T11:12:09.697 INFO:teuthology.orchestra.run.smithi161.stdout:Creating initial keys... 2023-07-14T11:12:10.472 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph-authtool: stdout AQCKLbFkMCz8AxAADIKsOOXkiaCp8k+ot8TxcA== 2023-07-14T11:12:11.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph-authtool: stdout AQCKLbFkZ/4lMhAAGcnCNDFoQ1c5Gb6IDi+S1w== 2023-07-14T11:12:12.424 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph-authtool: stdout AQCMLbFkXZW5BBAAn78xpQ4bkgRJiQ1+5GCBSQ== 2023-07-14T11:12:12.425 INFO:teuthology.orchestra.run.smithi161.stdout:Creating initial monmap... 2023-07-14T11:12:13.305 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T11:12:13.306 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-14T11:12:13.306 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:13.306 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T11:12:13.306 INFO:teuthology.orchestra.run.smithi161.stdout:monmaptool for a [v2:172.21.15.161:3300,v1:172.21.15.161:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T11:12:13.306 INFO:teuthology.orchestra.run.smithi161.stdout:setting min_mon_release = pacific 2023-07-14T11:12:13.306 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/monmaptool: set fsid to 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:13.306 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T11:12:13.307 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:12:13.307 INFO:teuthology.orchestra.run.smithi161.stdout:Creating mon... 2023-07-14T11:12:14.079 INFO:teuthology.orchestra.run.smithi161.stdout:create mon.a on 2023-07-14T11:12:14.321 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-14T11:12:14.518 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-14T11:12:14.765 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c.target → /etc/systemd/system/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c.target. 2023-07-14T11:12:14.766 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c.target → /etc/systemd/system/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c.target. 2023-07-14T11:12:15.111 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a 2023-07-14T11:12:15.111 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a.service: Unit ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a.service not loaded. 2023-07-14T11:12:15.319 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c.target.wants/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@.service. 2023-07-14T11:12:15.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:15 smithi161 systemd[1]: Starting Ceph mon.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:12:16.078 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:15 smithi161 podman[94972]: 2023-07-14T11:12:16.079 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:16 smithi161 bash[94972]: c62f55a5a4d7eca4c3ada00c8fe4632eeaa685371d21e6aa880257965ece5c72 2023-07-14T11:12:16.129 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:12:16.129 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout disabled 2023-07-14T11:12:16.145 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:12:16.145 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout inactive 2023-07-14T11:12:16.145 INFO:teuthology.orchestra.run.smithi161.stdout:firewalld.service is not enabled 2023-07-14T11:12:16.145 INFO:teuthology.orchestra.run.smithi161.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T11:12:16.146 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for mon to start... 2023-07-14T11:12:16.146 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for mon... 2023-07-14T11:12:16.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:16 smithi161 ceph-mon[95012]: mkfs 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:16.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:16 smithi161 ceph-mon[95012]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T11:12:16.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:16 smithi161 systemd[1]: Started Ceph mon.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:12:17.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:17 smithi161 ceph-mon[95012]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T11:12:17.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:17 smithi161 ceph-mon[95012]: monmap e1: 1 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T11:12:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:17 smithi161 ceph-mon[95012]: fsmap 2023-07-14T11:12:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:17 smithi161 ceph-mon[95012]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T11:12:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:17 smithi161 ceph-mon[95012]: mgrmap e1: no daemons active 2023-07-14T11:12:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:17 smithi161 ceph-mon[95012]: from='client.? 172.21.15.161:0/2953617432' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-14T11:12:18.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout cluster: 2023-07-14T11:12:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout id: 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-14T11:12:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout services: 2023-07-14T11:12:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.709922s) 2023-07-14T11:12:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-14T11:12:18.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-14T11:12:18.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:18.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout data: 2023-07-14T11:12:18.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-14T11:12:18.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-14T11:12:18.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-14T11:12:18.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout pgs: 2023-07-14T11:12:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.697+0000 7f15d22c8700 1 Processor -- start 2023-07-14T11:12:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.697+0000 7f15d22c8700 1 -- start start 2023-07-14T11:12:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.697+0000 7f15d22c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.697+0000 7f15d22c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc07a840 con 0x7f15cc07be00 2023-07-14T11:12:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.698+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.698+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45084/0 (socket says 172.21.15.161:45084) 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.698+0000 7f15cbfff700 1 -- 172.21.15.161:0/1583839176 learned_addr learned my addr 172.21.15.161:0/1583839176 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.699+0000 7f15cbfff700 1 -- 172.21.15.161:0/1583839176 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc07a980 con 0x7f15cc07be00 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.699+0000 7f15cbfff700 1 --2- 172.21.15.161:0/1583839176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0098a0 tx=0x7f15bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ccf2c1cf7adacc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.699+0000 7f15caffd700 1 -- 172.21.15.161:0/1583839176 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15bc004160 con 0x7f15cc07be00 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.699+0000 7f15caffd700 1 -- 172.21.15.161:0/1583839176 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc0042c0 con 0x7f15cc07be00 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.699+0000 7f15caffd700 1 -- 172.21.15.161:0/1583839176 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15bc0045a0 con 0x7f15cc07be00 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.700+0000 7f15d22c8700 1 -- 172.21.15.161:0/1583839176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 msgr2=0x7f15cc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.700+0000 7f15d22c8700 1 --2- 172.21.15.161:0/1583839176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0098a0 tx=0x7f15bc009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:18.440 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.700+0000 7f15d22c8700 1 -- 172.21.15.161:0/1583839176 shutdown_connections 2023-07-14T11:12:18.440 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.700+0000 7f15d22c8700 1 --2- 172.21.15.161:0/1583839176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc07a300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:18.440 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.700+0000 7f15d22c8700 1 -- 172.21.15.161:0/1583839176 >> 172.21.15.161:0/1583839176 conn(0x7f15cc100970 msgr2=0x7f15cc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:18.440 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.700+0000 7f15d22c8700 1 -- 172.21.15.161:0/1583839176 shutdown_connections 2023-07-14T11:12:18.440 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.700+0000 7f15d22c8700 1 -- 172.21.15.161:0/1583839176 wait complete. 2023-07-14T11:12:18.440 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.702+0000 7f15d22c8700 1 Processor -- start 2023-07-14T11:12:18.440 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.702+0000 7f15d22c8700 1 -- start start 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.702+0000 7f15d22c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc108750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.702+0000 7f15d22c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc108c90 con 0x7f15cc07be00 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.702+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.703+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45100/0 (socket says 172.21.15.161:45100) 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.703+0000 7f15cbfff700 1 -- 172.21.15.161:0/2953617432 learned_addr learned my addr 172.21.15.161:0/2953617432 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.703+0000 7f15cbfff700 1 -- 172.21.15.161:0/2953617432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc009550 con 0x7f15cc07be00 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.703+0000 7f15cbfff700 1 --2- 172.21.15.161:0/2953617432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc108750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f15bc004000 tx=0x7f15bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:18.441 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.703+0000 7f15c97fa700 1 -- 172.21.15.161:0/2953617432 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15bc011630 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.703+0000 7f15c97fa700 1 -- 172.21.15.161:0/2953617432 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc011790 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.703+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc108e90 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.704+0000 7f15c97fa700 1 -- 172.21.15.161:0/2953617432 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15bc011a30 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.704+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc105380 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.704+0000 7f15c97fa700 1 -- 172.21.15.161:0/2953617432 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f15bc022070 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.704+0000 7f15c97fa700 1 -- 172.21.15.161:0/2953617432 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f15bc01c450 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.705+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc061b40 con 0x7f15cc07be00 2023-07-14T11:12:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.708+0000 7f15c97fa700 1 -- 172.21.15.161:0/2953617432 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f15bc01b450 con 0x7f15cc07be00 2023-07-14T11:12:18.443 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.781+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f15cc105d90 con 0x7f15cc07be00 2023-07-14T11:12:18.443 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.781+0000 7f15c97fa700 1 -- 172.21.15.161:0/2953617432 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f15bc01b5f0 con 0x7f15cc07be00 2023-07-14T11:12:18.443 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.782+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 msgr2=0x7f15cc108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:18.443 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.782+0000 7f15d22c8700 1 --2- 172.21.15.161:0/2953617432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc108750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f15bc004000 tx=0x7f15bc004750 comp rx=0 tx=0).stop 2023-07-14T11:12:18.443 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.782+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 shutdown_connections 2023-07-14T11:12:18.443 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.782+0000 7f15d22c8700 1 --2- 172.21.15.161:0/2953617432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15cc07be00 0x7f15cc108750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:18.443 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.782+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 >> 172.21.15.161:0/2953617432 conn(0x7f15cc100970 msgr2=0x7f15cc1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:18.444 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.782+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 shutdown_connections 2023-07-14T11:12:18.444 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:16.782+0000 7f15d22c8700 1 -- 172.21.15.161:0/2953617432 wait complete. 2023-07-14T11:12:18.444 INFO:teuthology.orchestra.run.smithi161.stdout:mon is available 2023-07-14T11:12:18.444 INFO:teuthology.orchestra.run.smithi161.stdout:Assimilating anything we can from ceph.conf... 2023-07-14T11:12:19.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:19 smithi161 ceph-mon[95012]: from='client.? 172.21.15.161:0/2422424638' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T11:12:19.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:19 smithi161 ceph-mon[95012]: from='client.? 172.21.15.161:0/2422424638' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-14T11:12:19.518 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout fsid = 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.161:3300,v1:172.21.15.161:6789] 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T11:12:19.519 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:19.520 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T11:12:19.520 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T11:12:19.520 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:19.520 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T11:12:19.520 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T11:12:19.520 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T11:12:19.520 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T11:12:19.521 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T11:12:19.521 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T11:12:19.521 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.945+0000 7f577f8f1700 1 Processor -- start 2023-07-14T11:12:19.521 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.946+0000 7f577f8f1700 1 -- start start 2023-07-14T11:12:19.521 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.946+0000 7f577f8f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f5778108210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:19.521 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.946+0000 7f577f8f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778079da0 con 0x7f5778105e30 2023-07-14T11:12:19.521 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.947+0000 7f577d68d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f5778108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.947+0000 7f577d68d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f5778108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42724/0 (socket says 172.21.15.161:42724) 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.947+0000 7f577d68d700 1 -- 172.21.15.161:0/1510127982 learned_addr learned my addr 172.21.15.161:0/1510127982 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.947+0000 7f577d68d700 1 -- 172.21.15.161:0/1510127982 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778079ee0 con 0x7f5778105e30 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.947+0000 7f577d68d700 1 --2- 172.21.15.161:0/1510127982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f5778108210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f57740098a0 tx=0x7f5774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db9bb5942906b85 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.947+0000 7f576ffff700 1 -- 172.21.15.161:0/1510127982 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5774004160 con 0x7f5778105e30 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f576ffff700 1 -- 172.21.15.161:0/1510127982 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57740042c0 con 0x7f5778105e30 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f576ffff700 1 -- 172.21.15.161:0/1510127982 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57740045a0 con 0x7f5778105e30 2023-07-14T11:12:19.522 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f577f8f1700 1 -- 172.21.15.161:0/1510127982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 msgr2=0x7f5778108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f577f8f1700 1 --2- 172.21.15.161:0/1510127982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f5778108210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f57740098a0 tx=0x7f5774009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f577f8f1700 1 -- 172.21.15.161:0/1510127982 shutdown_connections 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f577f8f1700 1 --2- 172.21.15.161:0/1510127982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f5778108210 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f577f8f1700 1 -- 172.21.15.161:0/1510127982 >> 172.21.15.161:0/1510127982 conn(0x7f57780ff890 msgr2=0x7f5778101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f577f8f1700 1 -- 172.21.15.161:0/1510127982 shutdown_connections 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.948+0000 7f577f8f1700 1 -- 172.21.15.161:0/1510127982 wait complete. 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.949+0000 7f577f8f1700 1 Processor -- start 2023-07-14T11:12:19.523 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.950+0000 7f577f8f1700 1 -- start start 2023-07-14T11:12:19.524 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.950+0000 7f577f8f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f577819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:19.524 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.950+0000 7f577f8f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577819b000 con 0x7f5778105e30 2023-07-14T11:12:19.524 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.950+0000 7f577d68d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f577819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:19.524 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.950+0000 7f577d68d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f577819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42734/0 (socket says 172.21.15.161:42734) 2023-07-14T11:12:19.524 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.950+0000 7f577d68d700 1 -- 172.21.15.161:0/2422424638 learned_addr learned my addr 172.21.15.161:0/2422424638 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:19.524 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.950+0000 7f577d68d700 1 -- 172.21.15.161:0/2422424638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774009550 con 0x7f5778105e30 2023-07-14T11:12:19.524 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f577d68d700 1 --2- 172.21.15.161:0/2422424638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f577819aac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5774004000 tx=0x7f5774004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:19.525 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5774011630 con 0x7f5778105e30 2023-07-14T11:12:19.525 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5774011790 con 0x7f5778105e30 2023-07-14T11:12:19.525 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577819b200 con 0x7f5778105e30 2023-07-14T11:12:19.525 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5774011a30 con 0x7f5778105e30 2023-07-14T11:12:19.525 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577819b620 con 0x7f5778105e30 2023-07-14T11:12:19.525 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5774022070 con 0x7f5778105e30 2023-07-14T11:12:19.525 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.951+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f577401c790 con 0x7f5778105e30 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.952+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5778194a00 con 0x7f5778105e30 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:18.955+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f577401bcf0 con 0x7f5778105e30 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.023+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f577802cad0 con 0x7f5778105e30 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.028+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f577401be90 con 0x7f5778105e30 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.028+0000 7f576e7fc700 1 -- 172.21.15.161:0/2422424638 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f57740245c0 con 0x7f5778105e30 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.029+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 msgr2=0x7f577819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.029+0000 7f577f8f1700 1 --2- 172.21.15.161:0/2422424638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f577819aac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5774004000 tx=0x7f5774004750 comp rx=0 tx=0).stop 2023-07-14T11:12:19.526 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.029+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 shutdown_connections 2023-07-14T11:12:19.527 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.029+0000 7f577f8f1700 1 --2- 172.21.15.161:0/2422424638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5778105e30 0x7f577819aac0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:19.527 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.029+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 >> 172.21.15.161:0/2422424638 conn(0x7f57780ff890 msgr2=0x7f5778101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:19.527 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.029+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 shutdown_connections 2023-07-14T11:12:19.527 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:19.029+0000 7f577f8f1700 1 -- 172.21.15.161:0/2422424638 wait complete. 2023-07-14T11:12:19.527 INFO:teuthology.orchestra.run.smithi161.stdout:Generating new minimal ceph.conf... 2023-07-14T11:12:20.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:20 smithi161 ceph-mon[95012]: from='client.? 172.21.15.161:0/2666952929' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:12:20.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.059+0000 7f215bc79700 1 Processor -- start 2023-07-14T11:12:20.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.059+0000 7f215bc79700 1 -- start start 2023-07-14T11:12:20.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.060+0000 7f215bc79700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215407a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:20.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.060+0000 7f215bc79700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215407a8a0 con 0x7f215407be60 2023-07-14T11:12:20.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.060+0000 7f2159a15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215407a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:20.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.060+0000 7f2159a15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215407a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42738/0 (socket says 172.21.15.161:42738) 2023-07-14T11:12:20.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.060+0000 7f2159a15700 1 -- 172.21.15.161:0/2687086934 learned_addr learned my addr 172.21.15.161:0/2687086934 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:20.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.060+0000 7f2159a15700 1 -- 172.21.15.161:0/2687086934 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215407a9e0 con 0x7f215407be60 2023-07-14T11:12:20.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f2159a15700 1 --2- 172.21.15.161:0/2687086934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215407a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21500098a0 tx=0x7f2150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44674f8ba392ad96 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:20.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f2158a13700 1 -- 172.21.15.161:0/2687086934 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2150004160 con 0x7f215407be60 2023-07-14T11:12:20.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f2158a13700 1 -- 172.21.15.161:0/2687086934 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21500042c0 con 0x7f215407be60 2023-07-14T11:12:20.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f2158a13700 1 -- 172.21.15.161:0/2687086934 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2150011a00 con 0x7f215407be60 2023-07-14T11:12:20.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f215bc79700 1 -- 172.21.15.161:0/2687086934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 msgr2=0x7f215407a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f215bc79700 1 --2- 172.21.15.161:0/2687086934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215407a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21500098a0 tx=0x7f2150009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f215bc79700 1 -- 172.21.15.161:0/2687086934 shutdown_connections 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f215bc79700 1 --2- 172.21.15.161:0/2687086934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215407a360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.061+0000 7f215bc79700 1 -- 172.21.15.161:0/2687086934 >> 172.21.15.161:0/2687086934 conn(0x7f2154100970 msgr2=0x7f2154102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.062+0000 7f215bc79700 1 -- 172.21.15.161:0/2687086934 shutdown_connections 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.062+0000 7f215bc79700 1 -- 172.21.15.161:0/2687086934 wait complete. 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.063+0000 7f215bc79700 1 Processor -- start 2023-07-14T11:12:20.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.063+0000 7f215bc79700 1 -- start start 2023-07-14T11:12:20.599 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.063+0000 7f215bc79700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:20.599 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.063+0000 7f215bc79700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215419b850 con 0x7f215407be60 2023-07-14T11:12:20.599 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.063+0000 7f2159a15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:20.599 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.063+0000 7f2159a15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42752/0 (socket says 172.21.15.161:42752) 2023-07-14T11:12:20.599 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.063+0000 7f2159a15700 1 -- 172.21.15.161:0/2666952929 learned_addr learned my addr 172.21.15.161:0/2666952929 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:20.599 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.064+0000 7f2159a15700 1 -- 172.21.15.161:0/2666952929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2150009550 con 0x7f215407be60 2023-07-14T11:12:20.599 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.064+0000 7f2159a15700 1 --2- 172.21.15.161:0/2666952929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215419b310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2150009130 tx=0x7f2150011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.064+0000 7f214affd700 1 -- 172.21.15.161:0/2666952929 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2150003ab0 con 0x7f215407be60 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.064+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215419ba50 con 0x7f215407be60 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.064+0000 7f214affd700 1 -- 172.21.15.161:0/2666952929 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2150003c10 con 0x7f215407be60 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.064+0000 7f214affd700 1 -- 172.21.15.161:0/2666952929 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21500247d0 con 0x7f215407be60 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.064+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215419be70 con 0x7f215407be60 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.065+0000 7f214affd700 1 -- 172.21.15.161:0/2666952929 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2150024a50 con 0x7f215407be60 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.065+0000 7f214affd700 1 -- 172.21.15.161:0/2666952929 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f215002db60 con 0x7f215407be60 2023-07-14T11:12:20.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.065+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21541952f0 con 0x7f215407be60 2023-07-14T11:12:20.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.068+0000 7f214affd700 1 -- 172.21.15.161:0/2666952929 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f215002ddc0 con 0x7f215407be60 2023-07-14T11:12:20.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.137+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f215402c850 con 0x7f215407be60 2023-07-14T11:12:20.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.138+0000 7f214affd700 1 -- 172.21.15.161:0/2666952929 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f215001a8d0 con 0x7f215407be60 2023-07-14T11:12:20.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.139+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 msgr2=0x7f215419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:20.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.139+0000 7f215bc79700 1 --2- 172.21.15.161:0/2666952929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215419b310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2150009130 tx=0x7f2150011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:20.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.139+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 shutdown_connections 2023-07-14T11:12:20.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.139+0000 7f215bc79700 1 --2- 172.21.15.161:0/2666952929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215407be60 0x7f215419b310 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:20.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.139+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 >> 172.21.15.161:0/2666952929 conn(0x7f2154100970 msgr2=0x7f2154102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:20.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.139+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 shutdown_connections 2023-07-14T11:12:20.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:20.139+0000 7f215bc79700 1 -- 172.21.15.161:0/2666952929 wait complete. 2023-07-14T11:12:20.602 INFO:teuthology.orchestra.run.smithi161.stdout:Restarting the monitor... 2023-07-14T11:12:20.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:20 smithi161 systemd[1]: Stopping Ceph mon.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:12:21.413 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:21 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-a[94988]: 2023-07-14T11:12:21.107+0000 7f6b74272700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:12:21.413 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:21 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-a[94988]: 2023-07-14T11:12:21.107+0000 7f6b74272700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-14T11:12:21.694 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:21 smithi161 bash[95359]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-a 2023-07-14T11:12:21.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:21 smithi161 systemd[1]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a.service: Succeeded. 2023-07-14T11:12:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:21 smithi161 systemd[1]: Stopped Ceph mon.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:12:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:21 smithi161 systemd[1]: Starting Ceph mon.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:12:22.363 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 podman[95486]: 2023-07-14T11:12:22.364 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 bash[95486]: 0ecdc1bf6ec746e664798bbbb5dcf0db47b059b807692ece353eff1bdbf5bae2 2023-07-14T11:12:22.364 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T11:12:22.379 INFO:teuthology.orchestra.run.smithi161.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-14T11:12:22.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T11:12:22.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: pidfile_write: ignore empty --pid-file 2023-07-14T11:12:22.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: load: jerasure load: lrc 2023-07-14T11:12:22.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: RocksDB version: 7.9.2 2023-07-14T11:12:22.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Git sha 0 2023-07-14T11:12:22.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T11:12:22.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: DB SUMMARY 2023-07-14T11:12:22.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: DB Session ID: ZUX57RLI6D845ZEKONR7 2023-07-14T11:12:22.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: CURRENT file: CURRENT 2023-07-14T11:12:22.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T11:12:22.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-14T11:12:22.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-14T11:12:22.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99274 ; 2023-07-14T11:12:22.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.error_if_exists: 0 2023-07-14T11:12:22.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.create_if_missing: 0 2023-07-14T11:12:22.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.paranoid_checks: 1 2023-07-14T11:12:22.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T11:12:22.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T11:12:22.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T11:12:22.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.env: 0x55583bb5a720 2023-07-14T11:12:22.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.info_log: 0x55583cfb1280 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.statistics: (nil) 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.use_fsync: 0 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_log_file_size: 0 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T11:12:22.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T11:12:22.449 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T11:12:22.449 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.allow_fallocate: 1 2023-07-14T11:12:22.449 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T11:12:22.449 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T11:12:22.449 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.use_direct_reads: 0 2023-07-14T11:12:22.449 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T11:12:22.449 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.db_log_dir: 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.wal_dir: 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T11:12:22.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.write_buffer_manager: 0x55583ced23c0 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T11:12:22.451 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T11:12:22.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T11:12:22.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T11:12:22.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T11:12:22.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.unordered_write: 0 2023-07-14T11:12:22.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T11:12:22.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T11:12:22.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.row_cache: None 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.wal_filter: None 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.two_write_queues: 0 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T11:12:22.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.wal_compression: 0 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.atomic_flush: 0 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.log_readahead_size: 0 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T11:12:22.454 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T11:12:22.455 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T11:12:22.455 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T11:12:22.455 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T11:12:22.455 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T11:12:22.456 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_background_jobs: 2 2023-07-14T11:12:22.456 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_background_compactions: -1 2023-07-14T11:12:22.456 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_subcompactions: 1 2023-07-14T11:12:22.456 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T11:12:22.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T11:12:22.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T11:12:22.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T11:12:22.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T11:12:22.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T11:12:22.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T11:12:22.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_open_files: -1 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_background_flushes: -1 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Compression algorithms supported: 2023-07-14T11:12:22.458 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kZSTD supported: 0 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kXpressCompression supported: 0 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kZlibCompression supported: 1 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kSnappyCompression supported: 1 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kLZ4Compression supported: 1 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: kBZip2Compression supported: 0 2023-07-14T11:12:22.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T11:12:22.460 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T11:12:22.460 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-14T11:12:22.460 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T11:12:22.460 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T11:12:22.460 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.merge_operator: 2023-07-14T11:12:22.460 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_filter: None 2023-07-14T11:12:22.460 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55583cfb1380) 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout: cache_index_and_filter_blocks: 1 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T11:12:22.461 INFO:journalctl@ceph.mon.a.smithi161.stdout: pin_top_level_index_and_filter: 1 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: index_type: 0 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: data_block_index_type: 0 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: index_shortening: 1 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: checksum: 4 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: no_block_cache: 0 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_cache: 0x55583cf71350 2023-07-14T11:12:22.462 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_cache_name: BinnedLRUCache 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_cache_options: 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: capacity : 536870912 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: num_shard_bits : 4 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: strict_capacity_limit : 0 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: high_pri_pool_ratio: 0.000 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_cache_compressed: (nil) 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: persistent_cache: (nil) 2023-07-14T11:12:22.463 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_size: 4096 2023-07-14T11:12:22.464 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_size_deviation: 10 2023-07-14T11:12:22.464 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_restart_interval: 16 2023-07-14T11:12:22.464 INFO:journalctl@ceph.mon.a.smithi161.stdout: index_block_restart_interval: 1 2023-07-14T11:12:22.464 INFO:journalctl@ceph.mon.a.smithi161.stdout: metadata_block_size: 4096 2023-07-14T11:12:22.464 INFO:journalctl@ceph.mon.a.smithi161.stdout: partition_filters: 0 2023-07-14T11:12:22.464 INFO:journalctl@ceph.mon.a.smithi161.stdout: use_delta_encoding: 1 2023-07-14T11:12:22.464 INFO:journalctl@ceph.mon.a.smithi161.stdout: filter_policy: bloomfilter 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: whole_key_filtering: 1 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: verify_compression: 0 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: read_amp_bytes_per_bit: 0 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: format_version: 5 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: enable_index_compression: 1 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: block_align: 0 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: max_auto_readahead_size: 262144 2023-07-14T11:12:22.465 INFO:journalctl@ceph.mon.a.smithi161.stdout: prepopulate_block_cache: 0 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout: initial_auto_readahead_size: 8192 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression: NoCompression 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T11:12:22.466 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.num_levels: 7 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T11:12:22.467 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T11:12:22.468 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T11:12:22.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T11:12:22.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T11:12:22.471 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T11:12:22.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T11:12:22.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T11:12:22.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T11:12:22.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T11:12:22.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T11:12:22.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T11:12:22.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T11:12:22.473 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.table_properties_collectors: 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.inplace_update_support: 0 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.bloom_locality: 0 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.max_successive_merges: 0 2023-07-14T11:12:22.474 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.ttl: 2592000 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T11:12:22.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.enable_blob_files: false 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.min_blob_size: 0 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T11:12:22.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T11:12:22.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T11:12:22.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 systemd[1]: Started Ceph mon.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:12:22.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-14T11:12:22.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-14T11:12:22.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b5d10c58-7ce6-461a-ad24-1efd15cb7548 2023-07-14T11:12:22.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689333142374571, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-14T11:12:22.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689333142377203, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94271, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 323, "table_properties": {"data_size": 92276, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13656, "raw_average_key_size": 49, "raw_value_size": 84828, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 274, "num_filter_entries": 274, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689333142, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b5d10c58-7ce6-461a-ad24-1efd15cb7548", "db_session_id": "ZUX57RLI6D845ZEKONR7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689333142377353, "job": 1, "event": "recovery_finished"} 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55583d00a000 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: DB pointer 0x55583cff6000 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T11:12:22.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T11:12:22.479 INFO:journalctl@ceph.mon.a.smithi161.stdout: ** DB Stats ** 2023-07-14T11:12:22.479 INFO:journalctl@ceph.mon.a.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:12:22.479 INFO:journalctl@ceph.mon.a.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T11:12:22.479 INFO:journalctl@ceph.mon.a.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:12:22.479 INFO:journalctl@ceph.mon.a.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:12:22.479 INFO:journalctl@ceph.mon.a.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T11:12:22.479 INFO:journalctl@ceph.mon.a.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: ** Compaction Stats [default] ** 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: L0 2/0 93.90 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: Sum 2/0 93.90 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-07-14T11:12:22.480 INFO:journalctl@ceph.mon.a.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: ** Compaction Stats [default] ** 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 58.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T11:12:22.481 INFO:journalctl@ceph.mon.a.smithi161.stdout: 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: Cumulative compaction: 0.00 GB write, 10.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:12:22.482 INFO:journalctl@ceph.mon.a.smithi161.stdout: Interval compaction: 0.00 GB write, 10.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout: Block cache BinnedLRUCache@0x55583cf71350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout: 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: starting mon.a rank 0 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???) e1 preinit fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:22.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???).mds e1 new map 2023-07-14T11:12:22.484 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???).mds e1 print_map 2023-07-14T11:12:22.484 INFO:journalctl@ceph.mon.a.smithi161.stdout: e1 2023-07-14T11:12:22.484 INFO:journalctl@ceph.mon.a.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T11:12:22.484 INFO:journalctl@ceph.mon.a.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:12:22.484 INFO:journalctl@ceph.mon.a.smithi161.stdout: legacy client fscid: -1 2023-07-14T11:12:22.484 INFO:journalctl@ceph.mon.a.smithi161.stdout: 2023-07-14T11:12:22.484 INFO:journalctl@ceph.mon.a.smithi161.stdout: No filesystems configured 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expand map: {default=false} 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta from 'false' to 'false' 2023-07-14T11:12:22.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expanded map: {default=false} 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expand map: {default=info} 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta from 'info' to 'info' 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expanded map: {default=info} 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expand map: {default=daemon} 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expand map: {default=debug} 2023-07-14T11:12:22.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T11:12:22.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: expand_channel_meta expanded map: {default=debug} 2023-07-14T11:12:22.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T11:12:22.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: monmap e1: 1 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T11:12:22.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: fsmap 2023-07-14T11:12:22.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T11:12:22.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:22 smithi161 ceph-mon[95526]: mgrmap e1: no daemons active 2023-07-14T11:12:24.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:24 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3379804640' entity='client.admin' 2023-07-14T11:12:24.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.918+0000 7f06fe52f700 1 Processor -- start 2023-07-14T11:12:24.973 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.918+0000 7f06fe52f700 1 -- start start 2023-07-14T11:12:24.973 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.918+0000 7f06fe52f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:24.974 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.918+0000 7f06fe52f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f807a820 con 0x7f06f807bde0 2023-07-14T11:12:24.974 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:24.974 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42756/0 (socket says 172.21.15.161:42756) 2023-07-14T11:12:24.974 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f7fff700 1 -- 172.21.15.161:0/3091077846 learned_addr learned my addr 172.21.15.161:0/3091077846 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:24.975 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f7fff700 1 -- 172.21.15.161:0/3091077846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f807a960 con 0x7f06f807bde0 2023-07-14T11:12:24.975 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f7fff700 1 --2- 172.21.15.161:0/3091077846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f807a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06e80098a0 tx=0x7f06e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5314a11192fcf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:24.975 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f6ffd700 1 -- 172.21.15.161:0/3091077846 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06e8004160 con 0x7f06f807bde0 2023-07-14T11:12:24.975 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f6ffd700 1 -- 172.21.15.161:0/3091077846 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06e80042c0 con 0x7f06f807bde0 2023-07-14T11:12:24.976 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.919+0000 7f06f6ffd700 1 -- 172.21.15.161:0/3091077846 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06e8011a00 con 0x7f06f807bde0 2023-07-14T11:12:24.976 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.920+0000 7f06fe52f700 1 -- 172.21.15.161:0/3091077846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 msgr2=0x7f06f807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:24.976 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.920+0000 7f06fe52f700 1 --2- 172.21.15.161:0/3091077846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f807a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06e80098a0 tx=0x7f06e8009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:24.976 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.920+0000 7f06fe52f700 1 -- 172.21.15.161:0/3091077846 shutdown_connections 2023-07-14T11:12:24.977 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.920+0000 7f06fe52f700 1 --2- 172.21.15.161:0/3091077846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f807a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:24.977 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.920+0000 7f06fe52f700 1 -- 172.21.15.161:0/3091077846 >> 172.21.15.161:0/3091077846 conn(0x7f06f8100970 msgr2=0x7f06f8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:24.977 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.920+0000 7f06fe52f700 1 -- 172.21.15.161:0/3091077846 shutdown_connections 2023-07-14T11:12:24.977 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.920+0000 7f06fe52f700 1 -- 172.21.15.161:0/3091077846 wait complete. 2023-07-14T11:12:24.977 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.922+0000 7f06fe52f700 1 Processor -- start 2023-07-14T11:12:24.977 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.922+0000 7f06fe52f700 1 -- start start 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.922+0000 7f06fe52f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.922+0000 7f06fe52f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f819b850 con 0x7f06f807bde0 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.923+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.923+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42758/0 (socket says 172.21.15.161:42758) 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.923+0000 7f06f7fff700 1 -- 172.21.15.161:0/3379804640 learned_addr learned my addr 172.21.15.161:0/3379804640 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.923+0000 7f06f7fff700 1 -- 172.21.15.161:0/3379804640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8009550 con 0x7f06f807bde0 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.923+0000 7f06f7fff700 1 --2- 172.21.15.161:0/3379804640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f819b310 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009130 tx=0x7f06e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:24.978 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.924+0000 7f06f57fa700 1 -- 172.21.15.161:0/3379804640 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06e8003ab0 con 0x7f06f807bde0 2023-07-14T11:12:24.979 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.924+0000 7f06f57fa700 1 -- 172.21.15.161:0/3379804640 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06e8003c10 con 0x7f06f807bde0 2023-07-14T11:12:24.979 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.924+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f819ba50 con 0x7f06f807bde0 2023-07-14T11:12:24.979 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.924+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f819be70 con 0x7f06f807bde0 2023-07-14T11:12:24.979 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.924+0000 7f06f57fa700 1 -- 172.21.15.161:0/3379804640 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06e80247c0 con 0x7f06f807bde0 2023-07-14T11:12:24.979 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.924+0000 7f06f57fa700 1 -- 172.21.15.161:0/3379804640 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f06e8024ce0 con 0x7f06f807bde0 2023-07-14T11:12:24.979 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.924+0000 7f06f57fa700 1 -- 172.21.15.161:0/3379804640 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f06e80193c0 con 0x7f06f807bde0 2023-07-14T11:12:24.979 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.925+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f81952f0 con 0x7f06f807bde0 2023-07-14T11:12:24.980 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.928+0000 7f06f57fa700 1 -- 172.21.15.161:0/3379804640 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f06e801abb0 con 0x7f06f807bde0 2023-07-14T11:12:24.980 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:22.998+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f06f802c850 con 0x7f06f807bde0 2023-07-14T11:12:24.980 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.001+0000 7f06f57fa700 1 -- 172.21.15.161:0/3379804640 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f06e801ad50 con 0x7f06f807bde0 2023-07-14T11:12:24.980 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.001+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 msgr2=0x7f06f819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:24.980 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.001+0000 7f06fe52f700 1 --2- 172.21.15.161:0/3379804640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f819b310 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009130 tx=0x7f06e8011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:24.980 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.001+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 shutdown_connections 2023-07-14T11:12:24.981 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.001+0000 7f06fe52f700 1 --2- 172.21.15.161:0/3379804640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f807bde0 0x7f06f819b310 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:24.981 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.001+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 >> 172.21.15.161:0/3379804640 conn(0x7f06f8100970 msgr2=0x7f06f8102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:24.981 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.001+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 shutdown_connections 2023-07-14T11:12:24.981 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:23.002+0000 7f06fe52f700 1 -- 172.21.15.161:0/3379804640 wait complete. 2023-07-14T11:12:24.981 INFO:teuthology.orchestra.run.smithi161.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-14T11:12:24.999 INFO:teuthology.orchestra.run.smithi161.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:12:25.000 INFO:teuthology.orchestra.run.smithi161.stdout:Creating mgr... 2023-07-14T11:12:25.000 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying port 9283 ... 2023-07-14T11:12:25.001 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying port 8765 ... 2023-07-14T11:12:25.372 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.x 2023-07-14T11:12:25.373 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.x.service: Unit ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.x.service not loaded. 2023-07-14T11:12:25.583 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c.target.wants/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@.service. 2023-07-14T11:12:25.746 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:25 smithi161 systemd[1]: Starting Ceph mgr.x for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:12:26.369 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:26 smithi161 podman[95754]: 2023-07-14T11:12:26.369 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:26 smithi161 bash[95754]: c3a8ad3e3e77af6ce391b8a4d0b07807ab1c4dc65afdb8066a849ef0717baa0d 2023-07-14T11:12:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:12:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout disabled 2023-07-14T11:12:26.410 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:12:26.410 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout inactive 2023-07-14T11:12:26.410 INFO:teuthology.orchestra.run.smithi161.stdout:firewalld.service is not enabled 2023-07-14T11:12:26.411 INFO:teuthology.orchestra.run.smithi161.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T11:12:26.430 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:12:26.430 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout disabled 2023-07-14T11:12:26.448 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:12:26.448 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout inactive 2023-07-14T11:12:26.449 INFO:teuthology.orchestra.run.smithi161.stdout:firewalld.service is not enabled 2023-07-14T11:12:26.449 INFO:teuthology.orchestra.run.smithi161.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-14T11:12:26.449 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for mgr to start... 2023-07-14T11:12:26.449 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for mgr... 2023-07-14T11:12:26.657 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:26 smithi161 systemd[1]: Started Ceph mgr.x for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:12:26.657 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:26 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:26.549+0000 7fe41bce0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:12:26.657 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:26 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:26.655+0000 7fe41bce0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:12:26.910 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:26 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:26.779+0000 7fe41bce0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:12:26.911 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:26 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:26.907+0000 7fe41bce0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:12:27.165 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:27 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:27.139+0000 7fe41bce0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:12:27.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:27 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2218409300' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T11:12:27.549 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:27.549 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:12:27.549 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T11:12:27.550 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 0 2023-07-14T11:12:27.551 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:12:27.551 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T11:12:27.551 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T11:12:27.551 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:12:27.551 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2023-07-14T11:12:27.551 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T11:12:27.551 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T11:12:27.552 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T11:12:27.558 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T11:12:27.558 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T11:12:27.558 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T11:12:27.558 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:27.558 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T11:12:27.559 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T11:12:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T11:12:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T11:12:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T11:12:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T11:12:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T11:12:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T11:12:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T11:12:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T11:12:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T11:12:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T11:12:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T11:12:16.074288+0000", 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:27.562 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.056+0000 7f88dbadd700 1 Processor -- start 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.056+0000 7f88dbadd700 1 -- start start 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.056+0000 7f88dbadd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.056+0000 7f88dbadd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d407a840 con 0x7f88d407be00 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.057+0000 7f88d9879700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.057+0000 7f88d9879700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42778/0 (socket says 172.21.15.161:42778) 2023-07-14T11:12:27.563 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.057+0000 7f88d9879700 1 -- 172.21.15.161:0/140461113 learned_addr learned my addr 172.21.15.161:0/140461113 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:27.564 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.057+0000 7f88d9879700 1 -- 172.21.15.161:0/140461113 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d407a980 con 0x7f88d407be00 2023-07-14T11:12:27.564 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.057+0000 7f88d9879700 1 --2- 172.21.15.161:0/140461113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d407a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88d00098a0 tx=0x7f88d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122ead51242ee0b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:27.564 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.057+0000 7f88d8877700 1 -- 172.21.15.161:0/140461113 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d0004160 con 0x7f88d407be00 2023-07-14T11:12:27.564 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88d8877700 1 -- 172.21.15.161:0/140461113 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88d00042c0 con 0x7f88d407be00 2023-07-14T11:12:27.564 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88d8877700 1 -- 172.21.15.161:0/140461113 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d0011a00 con 0x7f88d407be00 2023-07-14T11:12:27.564 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88dbadd700 1 -- 172.21.15.161:0/140461113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 msgr2=0x7f88d407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:27.564 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88dbadd700 1 --2- 172.21.15.161:0/140461113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d407a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88d00098a0 tx=0x7f88d0009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88dbadd700 1 -- 172.21.15.161:0/140461113 shutdown_connections 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88dbadd700 1 --2- 172.21.15.161:0/140461113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d407a300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88dbadd700 1 -- 172.21.15.161:0/140461113 >> 172.21.15.161:0/140461113 conn(0x7f88d4100970 msgr2=0x7f88d4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88dbadd700 1 -- 172.21.15.161:0/140461113 shutdown_connections 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.058+0000 7f88dbadd700 1 -- 172.21.15.161:0/140461113 wait complete. 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.059+0000 7f88dbadd700 1 Processor -- start 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.060+0000 7f88dbadd700 1 -- start start 2023-07-14T11:12:27.565 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.060+0000 7f88dbadd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.060+0000 7f88dbadd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d419b880 con 0x7f88d407be00 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.060+0000 7f88d9879700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.060+0000 7f88d9879700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42790/0 (socket says 172.21.15.161:42790) 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.060+0000 7f88d9879700 1 -- 172.21.15.161:0/2218409300 learned_addr learned my addr 172.21.15.161:0/2218409300 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.060+0000 7f88d9879700 1 -- 172.21.15.161:0/2218409300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0009550 con 0x7f88d407be00 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.061+0000 7f88d9879700 1 --2- 172.21.15.161:0/2218409300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d419b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009130 tx=0x7f88d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.061+0000 7f88caffd700 1 -- 172.21.15.161:0/2218409300 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d0003ab0 con 0x7f88d407be00 2023-07-14T11:12:27.566 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.061+0000 7f88caffd700 1 -- 172.21.15.161:0/2218409300 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88d0003c10 con 0x7f88d407be00 2023-07-14T11:12:27.567 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.061+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d419ba80 con 0x7f88d407be00 2023-07-14T11:12:27.567 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.061+0000 7f88caffd700 1 -- 172.21.15.161:0/2218409300 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d00247d0 con 0x7f88d407be00 2023-07-14T11:12:27.567 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.061+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d419bea0 con 0x7f88d407be00 2023-07-14T11:12:27.567 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.062+0000 7f88caffd700 1 -- 172.21.15.161:0/2218409300 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f88d0024a50 con 0x7f88d407be00 2023-07-14T11:12:27.567 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.062+0000 7f88caffd700 1 -- 172.21.15.161:0/2218409300 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f88d002db70 con 0x7f88d407be00 2023-07-14T11:12:27.567 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.062+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d41952f0 con 0x7f88d407be00 2023-07-14T11:12:27.567 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.065+0000 7f88caffd700 1 -- 172.21.15.161:0/2218409300 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f88d001a8d0 con 0x7f88d407be00 2023-07-14T11:12:27.568 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.141+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f88d402c850 con 0x7f88d407be00 2023-07-14T11:12:27.568 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.141+0000 7f88caffd700 1 -- 172.21.15.161:0/2218409300 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f88d0031070 con 0x7f88d407be00 2023-07-14T11:12:27.568 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.142+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 msgr2=0x7f88d419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:27.568 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.142+0000 7f88dbadd700 1 --2- 172.21.15.161:0/2218409300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d419b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009130 tx=0x7f88d0011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:27.568 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.142+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 shutdown_connections 2023-07-14T11:12:27.568 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.142+0000 7f88dbadd700 1 --2- 172.21.15.161:0/2218409300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d407be00 0x7f88d419b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:27.568 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.142+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 >> 172.21.15.161:0/2218409300 conn(0x7f88d4100970 msgr2=0x7f88d41020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:27.569 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.142+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 shutdown_connections 2023-07-14T11:12:27.569 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:27.142+0000 7f88dbadd700 1 -- 172.21.15.161:0/2218409300 wait complete. 2023-07-14T11:12:27.569 INFO:teuthology.orchestra.run.smithi161.stdout:mgr not available, waiting (1/15)... 2023-07-14T11:12:27.839 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:27 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:27.560+0000 7fe41bce0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:12:27.840 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:27 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:27.677+0000 7fe41bce0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:12:28.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:27 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:27.837+0000 7fe41bce0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:12:28.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:27 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:27.960+0000 7fe41bce0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:12:28.995 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:28 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:28.501+0000 7fe41bce0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:12:29.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:29 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:29.273+0000 7fe41bce0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:12:29.746 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:29 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:29.482+0000 7fe41bce0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:12:30.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:30.456+0000 7fe41bce0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:12:30.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:30.576+0000 7fe41bce0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:12:31.164 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:30.800+0000 7fe41bce0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:12:31.443 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:31 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:31.162+0000 7fe41bce0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:12:31.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:31 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:31.441+0000 7fe41bce0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:12:31.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:31 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:31.635+0000 7fe41bce0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:12:32.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:31 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:31.984+0000 7fe41bce0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:12:33.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:32 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:32.929+0000 7fe41bce0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:12:33.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:33.100+0000 7fe41bce0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:12:33.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:33.341+0000 7fe41bce0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:12:33.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:33.490+0000 7fe41bce0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:12:34.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:33.766+0000 7fe41bce0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:12:35.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:34.834+0000 7fe41bce0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:12:35.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: Activating manager daemon x 2023-07-14T11:12:35.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: mgrmap e2: x(active, starting, since 0.00406678s) 2023-07-14T11:12:35.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:12:35.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:12:35.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:12:35.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:12:35.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:12:35.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: Manager daemon x is now available 2023-07-14T11:12:35.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:12:35.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' 2023-07-14T11:12:35.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:12:35.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' 2023-07-14T11:12:35.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:34 smithi161 ceph-mon[95526]: from='mgr.14100 172.21.15.161:0/4266608482' entity='mgr.x' 2023-07-14T11:12:36.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:36.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:12:36.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:12:36.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T11:12:36.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T11:12:36.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T11:12:36.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T11:12:36.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 0 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:12:36.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:36.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T11:12:36.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T11:12:36.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T11:12:36.762 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T11:12:36.763 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T11:12:36.763 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T11:12:36.763 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:36.763 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T11:12:36.763 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T11:12:36.763 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T11:12:36.764 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T11:12:36.764 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T11:12:36.764 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T11:12:36.764 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T11:12:36.764 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T11:12:36.764 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T11:12:36.765 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:12:36.766 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T11:12:16.074288+0000", 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:12:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.983+0000 7f6371db6700 1 Processor -- start 2023-07-14T11:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.983+0000 7f6371db6700 1 -- start start 2023-07-14T11:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.983+0000 7f6371db6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.983+0000 7f6371db6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c07a860 con 0x7f636c07be20 2023-07-14T11:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.984+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.984+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50826/0 (socket says 172.21.15.161:50826) 2023-07-14T11:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.984+0000 7f636b7fe700 1 -- 172.21.15.161:0/737531062 learned_addr learned my addr 172.21.15.161:0/737531062 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.984+0000 7f636b7fe700 1 -- 172.21.15.161:0/737531062 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c07a9a0 con 0x7f636c07be20 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.984+0000 7f636b7fe700 1 --2- 172.21.15.161:0/737531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c07a320 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098a0 tx=0x7f635c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f713d6041243b8bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f636a7fc700 1 -- 172.21.15.161:0/737531062 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c004160 con 0x7f636c07be20 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f636a7fc700 1 -- 172.21.15.161:0/737531062 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f635c0042c0 con 0x7f636c07be20 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f636a7fc700 1 -- 172.21.15.161:0/737531062 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c011a00 con 0x7f636c07be20 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f6371db6700 1 -- 172.21.15.161:0/737531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 msgr2=0x7f636c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f6371db6700 1 --2- 172.21.15.161:0/737531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c07a320 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098a0 tx=0x7f635c009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f6371db6700 1 -- 172.21.15.161:0/737531062 shutdown_connections 2023-07-14T11:12:36.769 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f6371db6700 1 --2- 172.21.15.161:0/737531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c07a320 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f6371db6700 1 -- 172.21.15.161:0/737531062 >> 172.21.15.161:0/737531062 conn(0x7f636c100970 msgr2=0x7f636c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.985+0000 7f6371db6700 1 -- 172.21.15.161:0/737531062 shutdown_connections 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.986+0000 7f6371db6700 1 -- 172.21.15.161:0/737531062 wait complete. 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.986+0000 7f6371db6700 1 Processor -- start 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.987+0000 7f6371db6700 1 -- start start 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.987+0000 7f6371db6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.987+0000 7f6371db6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19b880 con 0x7f636c07be20 2023-07-14T11:12:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.987+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.987+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50828/0 (socket says 172.21.15.161:50828) 2023-07-14T11:12:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.987+0000 7f636b7fe700 1 -- 172.21.15.161:0/472756471 learned_addr learned my addr 172.21.15.161:0/472756471 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.988+0000 7f636b7fe700 1 -- 172.21.15.161:0/472756471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c009550 con 0x7f636c07be20 2023-07-14T11:12:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.988+0000 7f636b7fe700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c19b340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f635c003f50 tx=0x7f635c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.988+0000 7f6368ff9700 1 -- 172.21.15.161:0/472756471 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c003aa0 con 0x7f636c07be20 2023-07-14T11:12:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.988+0000 7f6368ff9700 1 -- 172.21.15.161:0/472756471 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f635c003c00 con 0x7f636c07be20 2023-07-14T11:12:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.988+0000 7f6368ff9700 1 -- 172.21.15.161:0/472756471 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635c0247c0 con 0x7f636c07be20 2023-07-14T11:12:36.772 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.988+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c19ba80 con 0x7f636c07be20 2023-07-14T11:12:36.772 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.988+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c19bea0 con 0x7f636c07be20 2023-07-14T11:12:36.772 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.989+0000 7f6368ff9700 1 -- 172.21.15.161:0/472756471 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f635c01ab60 con 0x7f636c07be20 2023-07-14T11:12:36.772 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.989+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c195320 con 0x7f636c07be20 2023-07-14T11:12:36.772 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.990+0000 7f6368ff9700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f63540385e0 0x7f635403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:36.772 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.990+0000 7f6368ff9700 1 -- 172.21.15.161:0/472756471 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f635c04c0b0 con 0x7f636c07be20 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.990+0000 7f636affd700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f63540385e0 0x7f635403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.991+0000 7f636affd700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f63540385e0 0x7f635403aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:35.995+0000 7f6368ff9700 1 -- 172.21.15.161:0/472756471 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f635c07e790 con 0x7f636c07be20 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.262+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f636c061b40 con 0x7f636c07be20 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.262+0000 7f6368ff9700 1 -- 172.21.15.161:0/472756471 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f635c024920 con 0x7f636c07be20 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f63540385e0 msgr2=0x7f635403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f63540385e0 0x7f635403aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:36.773 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 msgr2=0x7f636c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c19b340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f635c003f50 tx=0x7f635c011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 shutdown_connections 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f63540385e0 0x7f635403aa90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 --2- 172.21.15.161:0/472756471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c07be20 0x7f636c19b340 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 >> 172.21.15.161:0/472756471 conn(0x7f636c100970 msgr2=0x7f636c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 shutdown_connections 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:36.265+0000 7f6371db6700 1 -- 172.21.15.161:0/472756471 wait complete. 2023-07-14T11:12:36.774 INFO:teuthology.orchestra.run.smithi161.stdout:mgr is available 2023-07-14T11:12:37.058 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:36 smithi161 ceph-mon[95526]: mgrmap e3: x(active, since 1.00943s) 2023-07-14T11:12:37.058 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:36 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/472756471' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T11:12:37.941 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:37.941 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T11:12:37.941 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout fsid = 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:37.942 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T11:12:37.942 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T11:12:37.942 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T11:12:37.942 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T11:12:37.942 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:37.942 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T11:12:37.943 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T11:12:37.943 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 2023-07-14T11:12:37.943 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T11:12:37.943 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T11:12:37.943 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T11:12:37.943 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T11:12:37.943 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T11:12:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T11:12:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.314+0000 7f98fe0c5700 1 Processor -- start 2023-07-14T11:12:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.314+0000 7f98fe0c5700 1 -- start start 2023-07-14T11:12:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.315+0000 7f98fe0c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f8105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.315+0000 7f98fe0c5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8105c70 con 0x7f98f8105320 2023-07-14T11:12:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.315+0000 7f98f77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f8105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.315+0000 7f98f77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f8105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50830/0 (socket says 172.21.15.161:50830) 2023-07-14T11:12:37.945 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.315+0000 7f98f77fe700 1 -- 172.21.15.161:0/3454657340 learned_addr learned my addr 172.21.15.161:0/3454657340 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:37.945 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.315+0000 7f98f77fe700 1 -- 172.21.15.161:0/3454657340 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8105db0 con 0x7f98f8105320 2023-07-14T11:12:37.945 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.316+0000 7f98f77fe700 1 --2- 172.21.15.161:0/3454657340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f8105730 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098a0 tx=0x7f98e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a055d8e046ce0267 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:37.945 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.316+0000 7f98f67fc700 1 -- 172.21.15.161:0/3454657340 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e8004160 con 0x7f98f8105320 2023-07-14T11:12:37.945 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.316+0000 7f98f67fc700 1 -- 172.21.15.161:0/3454657340 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98e80042c0 con 0x7f98f8105320 2023-07-14T11:12:37.945 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.316+0000 7f98f67fc700 1 -- 172.21.15.161:0/3454657340 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e8011a00 con 0x7f98f8105320 2023-07-14T11:12:37.946 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.316+0000 7f98fe0c5700 1 -- 172.21.15.161:0/3454657340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 msgr2=0x7f98f8105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:37.946 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.316+0000 7f98fe0c5700 1 --2- 172.21.15.161:0/3454657340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f8105730 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098a0 tx=0x7f98e8009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:37.946 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.317+0000 7f98fe0c5700 1 -- 172.21.15.161:0/3454657340 shutdown_connections 2023-07-14T11:12:37.946 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.317+0000 7f98fe0c5700 1 --2- 172.21.15.161:0/3454657340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f8105730 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:37.946 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.317+0000 7f98fe0c5700 1 -- 172.21.15.161:0/3454657340 >> 172.21.15.161:0/3454657340 conn(0x7f98f8100970 msgr2=0x7f98f8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:37.946 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.317+0000 7f98fe0c5700 1 -- 172.21.15.161:0/3454657340 shutdown_connections 2023-07-14T11:12:37.946 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.317+0000 7f98fe0c5700 1 -- 172.21.15.161:0/3454657340 wait complete. 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.318+0000 7f98fe0c5700 1 Processor -- start 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.318+0000 7f98fe0c5700 1 -- start start 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.318+0000 7f98fe0c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.318+0000 7f98fe0c5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f819b820 con 0x7f98f8105320 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.318+0000 7f98f77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.318+0000 7f98f77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50838/0 (socket says 172.21.15.161:50838) 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.318+0000 7f98f77fe700 1 -- 172.21.15.161:0/2675688266 learned_addr learned my addr 172.21.15.161:0/2675688266 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:37.947 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.319+0000 7f98f77fe700 1 -- 172.21.15.161:0/2675688266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8009550 con 0x7f98f8105320 2023-07-14T11:12:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.319+0000 7f98f77fe700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f819b2e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009130 tx=0x7f98e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.319+0000 7f98f4ff9700 1 -- 172.21.15.161:0/2675688266 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e8003b70 con 0x7f98f8105320 2023-07-14T11:12:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.319+0000 7f98f4ff9700 1 -- 172.21.15.161:0/2675688266 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98e8003cd0 con 0x7f98f8105320 2023-07-14T11:12:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.319+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f819ba20 con 0x7f98f8105320 2023-07-14T11:12:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.319+0000 7f98f4ff9700 1 -- 172.21.15.161:0/2675688266 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e80197d0 con 0x7f98f8105320 2023-07-14T11:12:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.319+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f819be40 con 0x7f98f8105320 2023-07-14T11:12:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.320+0000 7f98f4ff9700 1 -- 172.21.15.161:0/2675688266 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f98e801abb0 con 0x7f98f8105320 2023-07-14T11:12:37.949 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.320+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f81097b0 con 0x7f98f8105320 2023-07-14T11:12:37.949 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.321+0000 7f98f4ff9700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f98e0038710 0x7f98e003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:37.949 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.321+0000 7f98f4ff9700 1 -- 172.21.15.161:0/2675688266 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f98e804c840 con 0x7f98f8105320 2023-07-14T11:12:37.949 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.321+0000 7f98f6ffd700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f98e0038710 0x7f98e003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:37.949 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.322+0000 7f98f6ffd700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f98e0038710 0x7f98e003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98ec006fd0 tx=0x7f98ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:37.949 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.326+0000 7f98f4ff9700 1 -- 172.21.15.161:0/2675688266 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f98e801a430 con 0x7f98f8105320 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.519+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f98f8061b40 con 0x7f98f8105320 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.520+0000 7f98f4ff9700 1 -- 172.21.15.161:0/2675688266 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f98e802db00 con 0x7f98f8105320 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f98e0038710 msgr2=0x7f98e003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f98e0038710 0x7f98e003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98ec006fd0 tx=0x7f98ec006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 msgr2=0x7f98f819b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f819b2e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009130 tx=0x7f98e8011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 shutdown_connections 2023-07-14T11:12:37.950 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f98e0038710 0x7f98e003abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:37.951 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 --2- 172.21.15.161:0/2675688266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f8105320 0x7f98f819b2e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:37.951 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 >> 172.21.15.161:0/2675688266 conn(0x7f98f8100970 msgr2=0x7f98f81020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:37.951 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 shutdown_connections 2023-07-14T11:12:37.951 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:37.523+0000 7f98fe0c5700 1 -- 172.21.15.161:0/2675688266 wait complete. 2023-07-14T11:12:37.951 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling cephadm module... 2023-07-14T11:12:38.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:37 smithi161 ceph-mon[95526]: mgrmap e4: x(active, since 2s) 2023-07-14T11:12:38.201 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2675688266' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T11:12:39.123 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:38 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/843856473' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-14T11:12:39.123 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:38 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: ignoring --setuser ceph since I am not root 2023-07-14T11:12:39.123 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:38 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: ignoring --setgroup ceph since I am not root 2023-07-14T11:12:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.457+0000 7fa4b8185700 1 Processor -- start 2023-07-14T11:12:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.457+0000 7fa4b8185700 1 -- start start 2023-07-14T11:12:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.457+0000 7fa4b8185700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.457+0000 7fa4b8185700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b007a860 con 0x7fa4b007be20 2023-07-14T11:12:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.458+0000 7fa4b5f21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.458+0000 7fa4b5f21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32850/0 (socket says 172.21.15.161:32850) 2023-07-14T11:12:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.458+0000 7fa4b5f21700 1 -- 172.21.15.161:0/1204468481 learned_addr learned my addr 172.21.15.161:0/1204468481 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.458+0000 7fa4b5f21700 1 -- 172.21.15.161:0/1204468481 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b007a9a0 con 0x7fa4b007be20 2023-07-14T11:12:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.458+0000 7fa4b5f21700 1 --2- 172.21.15.161:0/1204468481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b007a320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0098a0 tx=0x7fa4ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0958840101f34d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b4f1f700 1 -- 172.21.15.161:0/1204468481 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac004160 con 0x7fa4b007be20 2023-07-14T11:12:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b4f1f700 1 -- 172.21.15.161:0/1204468481 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4ac0042c0 con 0x7fa4b007be20 2023-07-14T11:12:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b4f1f700 1 -- 172.21.15.161:0/1204468481 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac011a00 con 0x7fa4b007be20 2023-07-14T11:12:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b8185700 1 -- 172.21.15.161:0/1204468481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 msgr2=0x7fa4b007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b8185700 1 --2- 172.21.15.161:0/1204468481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b007a320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0098a0 tx=0x7fa4ac009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b8185700 1 -- 172.21.15.161:0/1204468481 shutdown_connections 2023-07-14T11:12:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b8185700 1 --2- 172.21.15.161:0/1204468481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b007a320 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b8185700 1 -- 172.21.15.161:0/1204468481 >> 172.21.15.161:0/1204468481 conn(0x7fa4b0100970 msgr2=0x7fa4b0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b8185700 1 -- 172.21.15.161:0/1204468481 shutdown_connections 2023-07-14T11:12:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.459+0000 7fa4b8185700 1 -- 172.21.15.161:0/1204468481 wait complete. 2023-07-14T11:12:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.461+0000 7fa4b8185700 1 Processor -- start 2023-07-14T11:12:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.461+0000 7fa4b8185700 1 -- start start 2023-07-14T11:12:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.461+0000 7fa4b8185700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.461+0000 7fa4b8185700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b019b880 con 0x7fa4b007be20 2023-07-14T11:12:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.462+0000 7fa4b5f21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.462+0000 7fa4b5f21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32864/0 (socket says 172.21.15.161:32864) 2023-07-14T11:12:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.462+0000 7fa4b5f21700 1 -- 172.21.15.161:0/843856473 learned_addr learned my addr 172.21.15.161:0/843856473 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.462+0000 7fa4b5f21700 1 -- 172.21.15.161:0/843856473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac009550 con 0x7fa4b007be20 2023-07-14T11:12:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.462+0000 7fa4b5f21700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b019b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac003f50 tx=0x7fa4ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.462+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac003aa0 con 0x7fa4b007be20 2023-07-14T11:12:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.463+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4ac003c00 con 0x7fa4b007be20 2023-07-14T11:12:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.463+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac0247c0 con 0x7fa4b007be20 2023-07-14T11:12:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.463+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b019ba80 con 0x7fa4b007be20 2023-07-14T11:12:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.463+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b019bea0 con 0x7fa4b007be20 2023-07-14T11:12:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.464+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fa4ac01ab60 con 0x7fa4b007be20 2023-07-14T11:12:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.464+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b0195320 con 0x7fa4b007be20 2023-07-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.464+0000 7fa4a6ffd700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fa49c038710 0x7fa49c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.464+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa4ac04c2a0 con 0x7fa4b007be20 2023-07-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.464+0000 7fa4b5720700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fa49c038710 0x7fa49c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.465+0000 7fa4b5720700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fa49c038710 0x7fa49c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0006fd0 tx=0x7fa4a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.469+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa4ac024920 con 0x7fa4b007be20 2023-07-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.713+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa4b0061b40 con 0x7fa4b007be20 2023-07-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.852+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa4ac02c350 con 0x7fa4b007be20 2023-07-14T11:12:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.853+0000 7fa4a6ffd700 1 -- 172.21.15.161:0/843856473 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa4ac04e680 con 0x7fa4b007be20 2023-07-14T11:12:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fa49c038710 msgr2=0x7fa49c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fa49c038710 0x7fa49c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0006fd0 tx=0x7fa4a0006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 msgr2=0x7fa4b019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b019b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac003f50 tx=0x7fa4ac011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 shutdown_connections 2023-07-14T11:12:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fa49c038710 0x7fa49c03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 --2- 172.21.15.161:0/843856473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4b007be20 0x7fa4b019b340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.856+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 >> 172.21.15.161:0/843856473 conn(0x7fa4b0100970 msgr2=0x7fa4b0102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.857+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 shutdown_connections 2023-07-14T11:12:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:38.857+0000 7fa4b8185700 1 -- 172.21.15.161:0/843856473 wait complete. 2023-07-14T11:12:39.475 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:39.119+0000 7ff4a2fd9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:12:39.475 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:39.224+0000 7ff4a2fd9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:12:39.475 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:39.344+0000 7ff4a2fd9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:12:39.475 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:39.472+0000 7ff4a2fd9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:12:39.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:39.707+0000 7ff4a2fd9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:12:40.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:39 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/843856473' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-14T11:12:40.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:39 smithi161 ceph-mon[95526]: mgrmap e5: x(active, since 4s) 2023-07-14T11:12:40.174 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:40 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:40.101+0000 7ff4a2fd9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:12:40.495 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:40 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:40.220+0000 7ff4a2fd9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:12:40.495 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:40 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:40.389+0000 7ff4a2fd9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:12:40.600 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:12:40.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-14T11:12:40.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T11:12:40.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T11:12:40.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T11:12:40.601 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:12:40.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.879+0000 7f8e96464700 1 Processor -- start 2023-07-14T11:12:40.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.879+0000 7f8e96464700 1 -- start start 2023-07-14T11:12:40.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.879+0000 7f8e96464700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e9007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:40.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.879+0000 7f8e96464700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9007a840 con 0x7f8e9007be00 2023-07-14T11:12:40.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.879+0000 7f8e8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e9007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:40.602 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.880+0000 7f8e8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e9007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32888/0 (socket says 172.21.15.161:32888) 2023-07-14T11:12:40.603 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.880+0000 7f8e8ffff700 1 -- 172.21.15.161:0/2600085551 learned_addr learned my addr 172.21.15.161:0/2600085551 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:40.603 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.880+0000 7f8e8ffff700 1 -- 172.21.15.161:0/2600085551 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9007a980 con 0x7f8e9007be00 2023-07-14T11:12:40.603 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.880+0000 7f8e8ffff700 1 --2- 172.21.15.161:0/2600085551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e9007a300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e800098a0 tx=0x7f8e80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d0e4b5db6ca156 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:40.603 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.880+0000 7f8e8effd700 1 -- 172.21.15.161:0/2600085551 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e80004160 con 0x7f8e9007be00 2023-07-14T11:12:40.603 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.880+0000 7f8e8effd700 1 -- 172.21.15.161:0/2600085551 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e800042c0 con 0x7f8e9007be00 2023-07-14T11:12:40.603 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.880+0000 7f8e8effd700 1 -- 172.21.15.161:0/2600085551 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e80011a00 con 0x7f8e9007be00 2023-07-14T11:12:40.603 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.881+0000 7f8e96464700 1 -- 172.21.15.161:0/2600085551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 msgr2=0x7f8e9007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:40.604 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.881+0000 7f8e96464700 1 --2- 172.21.15.161:0/2600085551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e9007a300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e800098a0 tx=0x7f8e80009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:40.604 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.881+0000 7f8e96464700 1 -- 172.21.15.161:0/2600085551 shutdown_connections 2023-07-14T11:12:40.604 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.881+0000 7f8e96464700 1 --2- 172.21.15.161:0/2600085551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e9007a300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:40.604 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.881+0000 7f8e96464700 1 -- 172.21.15.161:0/2600085551 >> 172.21.15.161:0/2600085551 conn(0x7f8e90100970 msgr2=0x7f8e90102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:40.604 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.881+0000 7f8e96464700 1 -- 172.21.15.161:0/2600085551 shutdown_connections 2023-07-14T11:12:40.604 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.881+0000 7f8e96464700 1 -- 172.21.15.161:0/2600085551 wait complete. 2023-07-14T11:12:40.604 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.883+0000 7f8e96464700 1 Processor -- start 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.883+0000 7f8e96464700 1 -- start start 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.883+0000 7f8e96464700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e90108750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.883+0000 7f8e96464700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e90108c90 con 0x7f8e9007be00 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.884+0000 7f8e8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e90108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.884+0000 7f8e8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e90108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32894/0 (socket says 172.21.15.161:32894) 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.884+0000 7f8e8ffff700 1 -- 172.21.15.161:0/4013739983 learned_addr learned my addr 172.21.15.161:0/4013739983 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.884+0000 7f8e8ffff700 1 -- 172.21.15.161:0/4013739983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80009550 con 0x7f8e9007be00 2023-07-14T11:12:40.605 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.884+0000 7f8e8ffff700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e90108750 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009130 tx=0x7f8e80011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.884+0000 7f8e8d7fa700 1 -- 172.21.15.161:0/4013739983 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e80003ab0 con 0x7f8e9007be00 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.884+0000 7f8e8d7fa700 1 -- 172.21.15.161:0/4013739983 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e80003c10 con 0x7f8e9007be00 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.885+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e90108e90 con 0x7f8e9007be00 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.885+0000 7f8e8d7fa700 1 -- 172.21.15.161:0/4013739983 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e800247d0 con 0x7f8e9007be00 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.885+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e90105380 con 0x7f8e9007be00 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.886+0000 7f8e8d7fa700 1 -- 172.21.15.161:0/4013739983 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8e8001a430 con 0x7f8e9007be00 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.886+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e90105e50 con 0x7f8e9007be00 2023-07-14T11:12:40.606 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.886+0000 7f8e8d7fa700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 0x7f8e7803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:40.607 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.886+0000 7f8e8d7fa700 1 -- 172.21.15.161:0/4013739983 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e8004c4a0 con 0x7f8e9007be00 2023-07-14T11:12:40.607 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.886+0000 7f8e8f7fe700 1 -- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 msgr2=0x7f8e7803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/4131434778 2023-07-14T11:12:40.607 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.886+0000 7f8e8f7fe700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 0x7f8e7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:12:40.607 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:39.891+0000 7f8e8d7fa700 1 -- 172.21.15.161:0/4013739983 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8e80018b40 con 0x7f8e9007be00 2023-07-14T11:12:40.607 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.087+0000 7f8e8f7fe700 1 -- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 msgr2=0x7f8e7803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/4131434778 2023-07-14T11:12:40.607 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.087+0000 7f8e8f7fe700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 0x7f8e7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:12:40.607 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.146+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8e90195110 con 0x7f8e9007be00 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.147+0000 7f8e8d7fa700 1 -- 172.21.15.161:0/4013739983 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8e80018b40 con 0x7f8e9007be00 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 msgr2=0x7f8e7803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 0x7f8e7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 msgr2=0x7f8e90108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e90108750 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009130 tx=0x7f8e80011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 shutdown_connections 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7f8e78038760 0x7f8e7803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:40.608 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 --2- 172.21.15.161:0/4013739983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9007be00 0x7f8e90108750 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:40.609 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.149+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 >> 172.21.15.161:0/4013739983 conn(0x7f8e90100970 msgr2=0x7f8e901020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:40.609 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.150+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 shutdown_connections 2023-07-14T11:12:40.609 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:40.150+0000 7f8e96464700 1 -- 172.21.15.161:0/4013739983 wait complete. 2023-07-14T11:12:40.609 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for the mgr to restart... 2023-07-14T11:12:40.609 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for mgr epoch 5... 2023-07-14T11:12:40.855 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:40 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:40.513+0000 7ff4a2fd9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:12:41.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:40 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4013739983' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T11:12:41.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:41.080+0000 7ff4a2fd9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:12:42.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:41.818+0000 7ff4a2fd9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:12:42.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:42 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:42.024+0000 7ff4a2fd9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:12:43.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:42 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:42.959+0000 7ff4a2fd9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:12:43.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:43.076+0000 7ff4a2fd9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:12:43.645 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:43.294+0000 7ff4a2fd9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:12:43.916 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:43.642+0000 7ff4a2fd9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:12:44.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:43.913+0000 7ff4a2fd9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:12:44.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:44.100+0000 7ff4a2fd9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:12:44.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:44.436+0000 7ff4a2fd9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:12:45.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:45.344+0000 7ff4a2fd9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:12:45.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:45.515+0000 7ff4a2fd9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:12:46.166 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:45.752+0000 7ff4a2fd9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:12:46.166 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:45.897+0000 7ff4a2fd9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:12:46.166 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:46 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:46.162+0000 7ff4a2fd9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:12:47.495 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:12:47.208+0000 7ff4a2fd9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:12:47.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: Active manager daemon x restarted 2023-07-14T11:12:47.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: Activating manager daemon x 2023-07-14T11:12:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: osdmap e2: 0 total, 0 up, 0 in 2023-07-14T11:12:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: mgrmap e6: x(active, starting, since 0.00376182s) 2023-07-14T11:12:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:12:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:12:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:12:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:12:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:12:47.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: Manager daemon x is now available 2023-07-14T11:12:47.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:47.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:47.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:12:47.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:12:47.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:47 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:12:48.656 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:48 smithi161 ceph-mon[95526]: Found migration_current of "None". Setting to last migration. 2023-07-14T11:12:48.656 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:48 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:12:48.656 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:48 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:12:48.656 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:48 smithi161 ceph-mon[95526]: mgrmap e7: x(active, since 1.00712s) 2023-07-14T11:12:48.697 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:12:48.697 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-14T11:12:48.697 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T11:12:48.698 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:12:48.698 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.163+0000 7fdd9ea53700 1 Processor -- start 2023-07-14T11:12:48.698 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.163+0000 7fdd9ea53700 1 -- start start 2023-07-14T11:12:48.699 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.164+0000 7fdd9ea53700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd98106040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:48.699 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.164+0000 7fdd9ea53700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd98079da0 con 0x7fdd98103c20 2023-07-14T11:12:48.699 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.164+0000 7fdd97fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd98106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:48.699 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.164+0000 7fdd97fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd98106040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32902/0 (socket says 172.21.15.161:32902) 2023-07-14T11:12:48.700 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.164+0000 7fdd97fff700 1 -- 172.21.15.161:0/3353304832 learned_addr learned my addr 172.21.15.161:0/3353304832 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:48.700 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.164+0000 7fdd97fff700 1 -- 172.21.15.161:0/3353304832 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd98079ee0 con 0x7fdd98103c20 2023-07-14T11:12:48.700 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.164+0000 7fdd97fff700 1 --2- 172.21.15.161:0/3353304832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd98106040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd880098a0 tx=0x7fdd88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b9a2bb47f10cb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:48.700 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.165+0000 7fdd96ffd700 1 -- 172.21.15.161:0/3353304832 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd88004160 con 0x7fdd98103c20 2023-07-14T11:12:48.701 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.165+0000 7fdd96ffd700 1 -- 172.21.15.161:0/3353304832 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd880042c0 con 0x7fdd98103c20 2023-07-14T11:12:48.701 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.165+0000 7fdd96ffd700 1 -- 172.21.15.161:0/3353304832 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd88011a00 con 0x7fdd98103c20 2023-07-14T11:12:48.701 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.165+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3353304832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 msgr2=0x7fdd98106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:48.702 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.165+0000 7fdd9ea53700 1 --2- 172.21.15.161:0/3353304832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd98106040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd880098a0 tx=0x7fdd88009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:48.702 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.166+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3353304832 shutdown_connections 2023-07-14T11:12:48.702 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.166+0000 7fdd9ea53700 1 --2- 172.21.15.161:0/3353304832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd98106040 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:48.702 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.166+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3353304832 >> 172.21.15.161:0/3353304832 conn(0x7fdd980ff890 msgr2=0x7fdd98101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:48.703 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.166+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3353304832 shutdown_connections 2023-07-14T11:12:48.703 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.166+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3353304832 wait complete. 2023-07-14T11:12:48.703 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.167+0000 7fdd9ea53700 1 Processor -- start 2023-07-14T11:12:48.703 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.167+0000 7fdd9ea53700 1 -- start start 2023-07-14T11:12:48.703 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.167+0000 7fdd9ea53700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd9819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:48.704 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.167+0000 7fdd9ea53700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9819fc30 con 0x7fdd98103c20 2023-07-14T11:12:48.704 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.168+0000 7fdd97fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd9819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:48.704 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.168+0000 7fdd97fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd9819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32906/0 (socket says 172.21.15.161:32906) 2023-07-14T11:12:48.704 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.168+0000 7fdd97fff700 1 -- 172.21.15.161:0/3337210340 learned_addr learned my addr 172.21.15.161:0/3337210340 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:48.704 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.168+0000 7fdd97fff700 1 -- 172.21.15.161:0/3337210340 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88009550 con 0x7fdd98103c20 2023-07-14T11:12:48.705 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.168+0000 7fdd97fff700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd9819f6f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009130 tx=0x7fdd88011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:48.705 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.168+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd88003ab0 con 0x7fdd98103c20 2023-07-14T11:12:48.705 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.169+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9819fe30 con 0x7fdd98103c20 2023-07-14T11:12:48.705 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.169+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd88003c10 con 0x7fdd98103c20 2023-07-14T11:12:48.705 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.169+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd880247d0 con 0x7fdd98103c20 2023-07-14T11:12:48.705 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.169+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd981a0250 con 0x7fdd98103c20 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.170+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fdd8802d3f0 con 0x7fdd98103c20 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.170+0000 7fdd957fa700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.170+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 --> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd8003b2d0 con 0x7fdd80038710 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.170+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd8804c6c0 con 0x7fdd98103c20 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.170+0000 7fdd977fe700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 msgr2=0x7fdd8003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/4131434778 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.170+0000 7fdd977fe700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.371+0000 7fdd977fe700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 msgr2=0x7fdd8003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/4131434778 2023-07-14T11:12:48.706 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.371+0000 7fdd977fe700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:12:48.707 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.772+0000 7fdd977fe700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 msgr2=0x7fdd8003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/4131434778 2023-07-14T11:12:48.707 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:41.772+0000 7fdd977fe700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T11:12:48.707 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:42.573+0000 7fdd977fe700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 msgr2=0x7fdd8003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/4131434778 2023-07-14T11:12:48.707 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:42.573+0000 7fdd977fe700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T11:12:48.707 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:44.175+0000 7fdd977fe700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 msgr2=0x7fdd8003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/4131434778 2023-07-14T11:12:48.707 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:44.175+0000 7fdd977fe700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T11:12:48.708 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:47.213+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7fdd8802dc10 con 0x7fdd98103c20 2023-07-14T11:12:48.708 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:47.213+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 msgr2=0x7fdd8003abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:12:48.708 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:47.213+0000 7fdd957fa700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:48.708 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.215+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fdd8804de10 con 0x7fdd98103c20 2023-07-14T11:12:48.708 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.215+0000 7fdd957fa700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fdd8003bfe0 0x7fdd8003e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:48.708 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.215+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd8003b2d0 con 0x7fdd8003bfe0 2023-07-14T11:12:48.708 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.215+0000 7fdd977fe700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fdd8003bfe0 0x7fdd8003e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.217+0000 7fdd977fe700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fdd8003bfe0 0x7fdd8003e3c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c003a10 tx=0x7fdd8c009290 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.218+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fdd8003b2d0 con 0x7fdd8003bfe0 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.224+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdd981a0a30 con 0x7fdd8003bfe0 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.224+0000 7fdd957fa700 1 -- 172.21.15.161:0/3337210340 <== mgr.14116 v2:172.21.15.161:6800/3292405426 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdd981a0a30 con 0x7fdd8003bfe0 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fdd8003bfe0 msgr2=0x7fdd8003e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fdd8003bfe0 0x7fdd8003e3c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c003a10 tx=0x7fdd8c009290 comp rx=0 tx=0).stop 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 msgr2=0x7fdd9819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:48.709 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd9819f6f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009130 tx=0x7fdd88011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 shutdown_connections 2023-07-14T11:12:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fdd8003bfe0 0x7fdd8003e3c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd98103c20 0x7fdd9819f6f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 --2- 172.21.15.161:0/3337210340 >> [v2:172.21.15.161:6800/4131434778,v1:172.21.15.161:6801/4131434778] conn(0x7fdd80038710 0x7fdd8003abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 >> 172.21.15.161:0/3337210340 conn(0x7fdd980ff890 msgr2=0x7fdd98100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 shutdown_connections 2023-07-14T11:12:48.710 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:48.225+0000 7fdd9ea53700 1 -- 172.21.15.161:0/3337210340 wait complete. 2023-07-14T11:12:48.711 INFO:teuthology.orchestra.run.smithi161.stdout:mgr epoch 5 is available 2023-07-14T11:12:48.711 INFO:teuthology.orchestra.run.smithi161.stdout:Setting orchestrator backend to cephadm... 2023-07-14T11:12:49.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T11:12:49.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T11:12:49.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:49.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:49.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: [14/Jul/2023:11:12:48] ENGINE Bus STARTING 2023-07-14T11:12:49.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: [14/Jul/2023:11:12:48] ENGINE Serving on http://172.21.15.161:8765 2023-07-14T11:12:49.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: [14/Jul/2023:11:12:48] ENGINE Serving on https://172.21.15.161:7150 2023-07-14T11:12:49.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: [14/Jul/2023:11:12:48] ENGINE Bus STARTED 2023-07-14T11:12:49.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:49 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:12:49.874 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.209+0000 7efc74bd7700 1 Processor -- start 2023-07-14T11:12:49.874 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.209+0000 7efc74bd7700 1 -- start start 2023-07-14T11:12:49.875 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.210+0000 7efc74bd7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:49.875 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.210+0000 7efc74bd7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7007a840 con 0x7efc7007be00 2023-07-14T11:12:49.875 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.210+0000 7efc6e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:49.875 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.210+0000 7efc6e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47184/0 (socket says 172.21.15.161:47184) 2023-07-14T11:12:49.875 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.210+0000 7efc6e59c700 1 -- 172.21.15.161:0/4277137497 learned_addr learned my addr 172.21.15.161:0/4277137497 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:49.875 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.211+0000 7efc6e59c700 1 -- 172.21.15.161:0/4277137497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7007a980 con 0x7efc7007be00 2023-07-14T11:12:49.875 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.211+0000 7efc6e59c700 1 --2- 172.21.15.161:0/4277137497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7007a300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efc600098a0 tx=0x7efc60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4362387abd857e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:49.876 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.211+0000 7efc6d59a700 1 -- 172.21.15.161:0/4277137497 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc60004160 con 0x7efc7007be00 2023-07-14T11:12:49.876 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.211+0000 7efc6d59a700 1 -- 172.21.15.161:0/4277137497 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc600042c0 con 0x7efc7007be00 2023-07-14T11:12:49.876 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.211+0000 7efc6d59a700 1 -- 172.21.15.161:0/4277137497 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc60011a00 con 0x7efc7007be00 2023-07-14T11:12:49.876 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.212+0000 7efc74bd7700 1 -- 172.21.15.161:0/4277137497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 msgr2=0x7efc7007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:49.876 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.212+0000 7efc74bd7700 1 --2- 172.21.15.161:0/4277137497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7007a300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efc600098a0 tx=0x7efc60009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:49.876 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.212+0000 7efc74bd7700 1 -- 172.21.15.161:0/4277137497 shutdown_connections 2023-07-14T11:12:49.879 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.212+0000 7efc74bd7700 1 --2- 172.21.15.161:0/4277137497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7007a300 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:49.879 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.212+0000 7efc74bd7700 1 -- 172.21.15.161:0/4277137497 >> 172.21.15.161:0/4277137497 conn(0x7efc70100970 msgr2=0x7efc70102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.212+0000 7efc74bd7700 1 -- 172.21.15.161:0/4277137497 shutdown_connections 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.212+0000 7efc74bd7700 1 -- 172.21.15.161:0/4277137497 wait complete. 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.213+0000 7efc74bd7700 1 Processor -- start 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.213+0000 7efc74bd7700 1 -- start start 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.213+0000 7efc74bd7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.213+0000 7efc74bd7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7019b880 con 0x7efc7007be00 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.213+0000 7efc6e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:49.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc6e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47186/0 (socket says 172.21.15.161:47186) 2023-07-14T11:12:49.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc6e59c700 1 -- 172.21.15.161:0/2828626082 learned_addr learned my addr 172.21.15.161:0/2828626082 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:49.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc6e59c700 1 -- 172.21.15.161:0/2828626082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc60009550 con 0x7efc7007be00 2023-07-14T11:12:49.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc6e59c700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7019b340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fb0 tx=0x7efc60011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:49.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc60003a40 con 0x7efc7007be00 2023-07-14T11:12:49.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7019ba80 con 0x7efc7007be00 2023-07-14T11:12:49.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc60003ba0 con 0x7efc7007be00 2023-07-14T11:12:49.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc6001a750 con 0x7efc7007be00 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.214+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7019bea0 con 0x7efc7007be00 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.215+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7efc60024430 con 0x7efc7007be00 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.216+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc701952f0 con 0x7efc7007be00 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.216+0000 7efc5f7fe700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7efc580385a0 0x7efc5803aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.216+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efc6004c680 con 0x7efc7007be00 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.216+0000 7efc6dd9b700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7efc580385a0 0x7efc5803aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.217+0000 7efc6dd9b700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7efc580385a0 0x7efc5803aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc64006fd0 tx=0x7efc64006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.222+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efc600189a0 con 0x7efc7007be00 2023-07-14T11:12:49.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.309+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7efc600246e0 con 0x7efc7007be00 2023-07-14T11:12:49.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.433+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7efc70060950 con 0x7efc580385a0 2023-07-14T11:12:49.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.439+0000 7efc5f7fe700 1 -- 172.21.15.161:0/2828626082 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efc70060950 con 0x7efc580385a0 2023-07-14T11:12:49.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7efc580385a0 msgr2=0x7efc5803aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:49.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7efc580385a0 0x7efc5803aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc64006fd0 tx=0x7efc64006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:49.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 msgr2=0x7efc7019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:49.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7019b340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fb0 tx=0x7efc60011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:49.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 shutdown_connections 2023-07-14T11:12:49.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7efc580385a0 0x7efc5803aa50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:49.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 --2- 172.21.15.161:0/2828626082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc7007be00 0x7efc7019b340 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:49.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 >> 172.21.15.161:0/2828626082 conn(0x7efc70100970 msgr2=0x7efc701020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:49.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 shutdown_connections 2023-07-14T11:12:49.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:49.442+0000 7efc74bd7700 1 -- 172.21.15.161:0/2828626082 wait complete. 2023-07-14T11:12:50.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:50 smithi161 ceph-mon[95526]: mgrmap e8: x(active, since 2s) 2023-07-14T11:12:50.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:50 smithi161 ceph-mon[95526]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:12:50.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:50 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:50 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:12:51.055 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-14T11:12:51.056 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.408+0000 7f5dccae5700 1 Processor -- start 2023-07-14T11:12:51.056 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.408+0000 7f5dccae5700 1 -- start start 2023-07-14T11:12:51.056 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.409+0000 7f5dccae5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:51.056 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.409+0000 7f5dccae5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc807a840 con 0x7f5dc807be00 2023-07-14T11:12:51.056 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.409+0000 7f5dc659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:51.056 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.409+0000 7f5dc659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47190/0 (socket says 172.21.15.161:47190) 2023-07-14T11:12:51.057 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.409+0000 7f5dc659c700 1 -- 172.21.15.161:0/1053699288 learned_addr learned my addr 172.21.15.161:0/1053699288 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:51.057 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.409+0000 7f5dc659c700 1 -- 172.21.15.161:0/1053699288 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc807a980 con 0x7f5dc807be00 2023-07-14T11:12:51.057 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.410+0000 7f5dc659c700 1 --2- 172.21.15.161:0/1053699288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc807a300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5db80098a0 tx=0x7f5db8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fa140c288d9f4a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:51.057 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.410+0000 7f5dc559a700 1 -- 172.21.15.161:0/1053699288 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db8004160 con 0x7f5dc807be00 2023-07-14T11:12:51.057 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.410+0000 7f5dc559a700 1 -- 172.21.15.161:0/1053699288 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5db80042c0 con 0x7f5dc807be00 2023-07-14T11:12:51.057 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.410+0000 7f5dc559a700 1 -- 172.21.15.161:0/1053699288 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db8011a00 con 0x7f5dc807be00 2023-07-14T11:12:51.057 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.410+0000 7f5dccae5700 1 -- 172.21.15.161:0/1053699288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 msgr2=0x7f5dc807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:51.058 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.410+0000 7f5dccae5700 1 --2- 172.21.15.161:0/1053699288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc807a300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5db80098a0 tx=0x7f5db8009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:51.058 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.411+0000 7f5dccae5700 1 -- 172.21.15.161:0/1053699288 shutdown_connections 2023-07-14T11:12:51.058 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.411+0000 7f5dccae5700 1 --2- 172.21.15.161:0/1053699288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc807a300 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:51.058 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.411+0000 7f5dccae5700 1 -- 172.21.15.161:0/1053699288 >> 172.21.15.161:0/1053699288 conn(0x7f5dc8100970 msgr2=0x7f5dc8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:51.058 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.411+0000 7f5dccae5700 1 -- 172.21.15.161:0/1053699288 shutdown_connections 2023-07-14T11:12:51.058 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.411+0000 7f5dccae5700 1 -- 172.21.15.161:0/1053699288 wait complete. 2023-07-14T11:12:51.058 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.412+0000 7f5dccae5700 1 Processor -- start 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.412+0000 7f5dccae5700 1 -- start start 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.412+0000 7f5dccae5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.412+0000 7f5dccae5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc819b880 con 0x7f5dc807be00 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.412+0000 7f5dc659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5dc659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47198/0 (socket says 172.21.15.161:47198) 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5dc659c700 1 -- 172.21.15.161:0/966075355 learned_addr learned my addr 172.21.15.161:0/966075355 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5dc659c700 1 -- 172.21.15.161:0/966075355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8009550 con 0x7f5dc807be00 2023-07-14T11:12:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5dc659c700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc819b340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009130 tx=0x7f5db8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5db77fe700 1 -- 172.21.15.161:0/966075355 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db8003ab0 con 0x7f5dc807be00 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5db77fe700 1 -- 172.21.15.161:0/966075355 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5db8003c10 con 0x7f5dc807be00 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5db77fe700 1 -- 172.21.15.161:0/966075355 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db80247d0 con 0x7f5dc807be00 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.413+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc819ba80 con 0x7f5dc807be00 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.414+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc819bea0 con 0x7f5dc807be00 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.415+0000 7f5db77fe700 1 -- 172.21.15.161:0/966075355 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5db801abb0 con 0x7f5dc807be00 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.415+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc81952f0 con 0x7f5dc807be00 2023-07-14T11:12:51.060 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.415+0000 7f5db77fe700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f5db0038760 0x7f5db003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:51.061 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.415+0000 7f5db77fe700 1 -- 172.21.15.161:0/966075355 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5db804c790 con 0x7f5dc807be00 2023-07-14T11:12:51.061 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.415+0000 7f5dc5d9b700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f5db0038760 0x7f5db003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:51.061 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.416+0000 7f5dc5d9b700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f5db0038760 0x7f5db003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc006fd0 tx=0x7f5dbc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:51.061 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.420+0000 7f5db77fe700 1 -- 172.21.15.161:0/966075355 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5db8018530 con 0x7f5dc807be00 2023-07-14T11:12:51.061 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.620+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dc8060950 con 0x7f5db0038760 2023-07-14T11:12:51.061 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.620+0000 7f5db77fe700 1 -- 172.21.15.161:0/966075355 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5dc8060950 con 0x7f5db0038760 2023-07-14T11:12:51.061 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f5db0038760 msgr2=0x7f5db003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f5db0038760 0x7f5db003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc006fd0 tx=0x7f5dbc006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 msgr2=0x7f5dc819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc819b340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009130 tx=0x7f5db8011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 shutdown_connections 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f5db0038760 0x7f5db003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 --2- 172.21.15.161:0/966075355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc807be00 0x7f5dc819b340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 >> 172.21.15.161:0/966075355 conn(0x7f5dc8100970 msgr2=0x7f5dc81020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:51.062 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 shutdown_connections 2023-07-14T11:12:51.063 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:50.623+0000 7f5dccae5700 1 -- 172.21.15.161:0/966075355 wait complete. 2023-07-14T11:12:51.063 INFO:teuthology.orchestra.run.smithi161.stdout:Generating ssh key... 2023-07-14T11:12:51.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-mon[95526]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:12:52.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: Generating public/private rsa key pair. 2023-07-14T11:12:52.246 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: Your identification has been saved in /tmp/tmp7mkr9i9j/key. 2023-07-14T11:12:52.246 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: Your public key has been saved in /tmp/tmp7mkr9i9j/key.pub. 2023-07-14T11:12:52.246 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: The key fingerprint is: 2023-07-14T11:12:52.246 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: SHA256:F4qyHfKamkwe1q7A6TrsgGOwEhgdz2ery/Qmet9duGg ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:52.246 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: The key's randomart image is: 2023-07-14T11:12:52.246 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: +---[RSA 3072]----+ 2023-07-14T11:12:52.246 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: | . | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: | . + | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: |. . o o . | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: |.. o o . . | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: |+ o + S . | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: |+o.. B . . . | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: |O++ = o . . | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: |*O *o=..Eo o | 2023-07-14T11:12:52.247 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: |++B=*+o.o o | 2023-07-14T11:12:52.248 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:12:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: +----[SHA256]-----+ 2023-07-14T11:12:52.431 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.606+0000 7fcc7af02700 1 Processor -- start 2023-07-14T11:12:52.432 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.606+0000 7fcc7af02700 1 -- start start 2023-07-14T11:12:52.432 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.606+0000 7fcc7af02700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:52.432 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.606+0000 7fcc7af02700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7407a880 con 0x7fcc7407be40 2023-07-14T11:12:52.432 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc78c9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:52.432 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc78c9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47214/0 (socket says 172.21.15.161:47214) 2023-07-14T11:12:52.432 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc78c9e700 1 -- 172.21.15.161:0/3124561375 learned_addr learned my addr 172.21.15.161:0/3124561375 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:52.433 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc78c9e700 1 -- 172.21.15.161:0/3124561375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7407a9c0 con 0x7fcc7407be40 2023-07-14T11:12:52.433 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc78c9e700 1 --2- 172.21.15.161:0/3124561375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7407a340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc680098a0 tx=0x7fcc68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb0d7ec099df65d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:52.433 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc737fe700 1 -- 172.21.15.161:0/3124561375 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc68004160 con 0x7fcc7407be40 2023-07-14T11:12:52.433 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc737fe700 1 -- 172.21.15.161:0/3124561375 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc680042c0 con 0x7fcc7407be40 2023-07-14T11:12:52.433 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.607+0000 7fcc737fe700 1 -- 172.21.15.161:0/3124561375 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc68011a00 con 0x7fcc7407be40 2023-07-14T11:12:52.433 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.608+0000 7fcc7af02700 1 -- 172.21.15.161:0/3124561375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 msgr2=0x7fcc7407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:52.433 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.608+0000 7fcc7af02700 1 --2- 172.21.15.161:0/3124561375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7407a340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc680098a0 tx=0x7fcc68009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.608+0000 7fcc7af02700 1 -- 172.21.15.161:0/3124561375 shutdown_connections 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.608+0000 7fcc7af02700 1 --2- 172.21.15.161:0/3124561375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7407a340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.608+0000 7fcc7af02700 1 -- 172.21.15.161:0/3124561375 >> 172.21.15.161:0/3124561375 conn(0x7fcc74100970 msgr2=0x7fcc74102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.608+0000 7fcc7af02700 1 -- 172.21.15.161:0/3124561375 shutdown_connections 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.608+0000 7fcc7af02700 1 -- 172.21.15.161:0/3124561375 wait complete. 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.610+0000 7fcc7af02700 1 Processor -- start 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.610+0000 7fcc7af02700 1 -- start start 2023-07-14T11:12:52.434 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.610+0000 7fcc7af02700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.610+0000 7fcc7af02700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419b860 con 0x7fcc7407be40 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc78c9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc78c9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47222/0 (socket says 172.21.15.161:47222) 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc78c9e700 1 -- 172.21.15.161:0/4169122839 learned_addr learned my addr 172.21.15.161:0/4169122839 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc78c9e700 1 -- 172.21.15.161:0/4169122839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68009550 con 0x7fcc7407be40 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc78c9e700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7419b320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009130 tx=0x7fcc68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc71ffb700 1 -- 172.21.15.161:0/4169122839 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc68003ab0 con 0x7fcc7407be40 2023-07-14T11:12:52.435 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc71ffb700 1 -- 172.21.15.161:0/4169122839 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc68003c10 con 0x7fcc7407be40 2023-07-14T11:12:52.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc71ffb700 1 -- 172.21.15.161:0/4169122839 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc680247d0 con 0x7fcc7407be40 2023-07-14T11:12:52.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.611+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7419ba60 con 0x7fcc7407be40 2023-07-14T11:12:52.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.612+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7419be80 con 0x7fcc7407be40 2023-07-14T11:12:52.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.613+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc74195300 con 0x7fcc7407be40 2023-07-14T11:12:52.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.613+0000 7fcc71ffb700 1 -- 172.21.15.161:0/4169122839 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcc6801abb0 con 0x7fcc7407be40 2023-07-14T11:12:52.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.613+0000 7fcc71ffb700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fcc64038760 0x7fcc6403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:52.436 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.613+0000 7fcc71ffb700 1 -- 172.21.15.161:0/4169122839 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc6804c850 con 0x7fcc7407be40 2023-07-14T11:12:52.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.614+0000 7fcc73fff700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fcc64038760 0x7fcc6403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:52.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.614+0000 7fcc73fff700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fcc64038760 0x7fcc6403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006fd0 tx=0x7fcc60006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:52.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.618+0000 7fcc71ffb700 1 -- 172.21.15.161:0/4169122839 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcc68018530 con 0x7fcc7407be40 2023-07-14T11:12:52.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.814+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc74060950 con 0x7fcc64038760 2023-07-14T11:12:52.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.925+0000 7fcc71ffb700 1 -- 172.21.15.161:0/4169122839 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc74060950 con 0x7fcc64038760 2023-07-14T11:12:52.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fcc64038760 msgr2=0x7fcc6403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:52.437 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fcc64038760 0x7fcc6403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006fd0 tx=0x7fcc60006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 msgr2=0x7fcc7419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7419b320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009130 tx=0x7fcc68011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 shutdown_connections 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fcc64038760 0x7fcc6403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 --2- 172.21.15.161:0/4169122839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc7407be40 0x7fcc7419b320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.927+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 >> 172.21.15.161:0/4169122839 conn(0x7fcc74100970 msgr2=0x7fcc74102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.928+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 shutdown_connections 2023-07-14T11:12:52.438 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:51.928+0000 7fcc7af02700 1 -- 172.21.15.161:0/4169122839 wait complete. 2023-07-14T11:12:52.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:52 smithi161 ceph-mon[95526]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:12:52.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:52 smithi161 ceph-mon[95526]: Generating ssh key... 2023-07-14T11:12:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:52 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:52 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:53.611 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe/8P46t5ZZ70sveVKxNngPuPw9t71vYYSPY+AoRmAewQJBczXIYMhm1IkeGuisfD7ShkQZGq2xlMwgkldCZ4f+e9lSUQBiJopEBPCQAhUOewN5HXlzdPepCHSRzB9o1OYH+wi7+i364LWU3JHCfoLkI1iWJ8eAK4/lsASn5bqlHxAgUYPAuIeXNY7TnF7VmajKjHO00+ApE5vPDsHxcBlD5mzQyc4chSa2MXdqZn7pfiAQqAFXOiXWmtvTeAmI5qUeouz1+6FcpprrzSUI0R3+4+UhcV6gemDqslhvIk11YxE1Cs1ekBi7AcjawneT8J/upEe9tUYb2u5D2Rwnd3fpSeY+OzBMkxprLfYMGIT0yz7H2XCtb0bd3Wm3dMz7x+4TSDLiqIgA6cMiS7rNyLYd9FHxBnz2cyOPqEDUk3w4P3xbsRUCWeKFZFWeL8pH6gocLn/dZsCeA+kUqdsw98j/UNyNnKOyaCGJ+mPUimgKGdxOtkNHcY9fDxo+Q0rizM= ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:12:53.611 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.957+0000 7fb4f5f77700 1 Processor -- start 2023-07-14T11:12:53.611 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.957+0000 7fb4f5f77700 1 -- start start 2023-07-14T11:12:53.611 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.957+0000 7fb4f5f77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:53.611 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.957+0000 7fb4f5f77700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f007a860 con 0x7fb4f007be20 2023-07-14T11:12:53.612 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.958+0000 7fb4ef7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:53.612 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.958+0000 7fb4ef7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47226/0 (socket says 172.21.15.161:47226) 2023-07-14T11:12:53.612 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.958+0000 7fb4ef7fe700 1 -- 172.21.15.161:0/3269708418 learned_addr learned my addr 172.21.15.161:0/3269708418 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:53.612 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.958+0000 7fb4ef7fe700 1 -- 172.21.15.161:0/3269708418 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f007a9a0 con 0x7fb4f007be20 2023-07-14T11:12:53.612 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.958+0000 7fb4ef7fe700 1 --2- 172.21.15.161:0/3269708418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f007a320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00098a0 tx=0x7fb4e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8f8189fcc87028 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:53.612 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.958+0000 7fb4ee7fc700 1 -- 172.21.15.161:0/3269708418 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4e0004160 con 0x7fb4f007be20 2023-07-14T11:12:53.612 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4ee7fc700 1 -- 172.21.15.161:0/3269708418 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4e00042c0 con 0x7fb4f007be20 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4ee7fc700 1 -- 172.21.15.161:0/3269708418 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4e0011a00 con 0x7fb4f007be20 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4f5f77700 1 -- 172.21.15.161:0/3269708418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 msgr2=0x7fb4f007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4f5f77700 1 --2- 172.21.15.161:0/3269708418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f007a320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00098a0 tx=0x7fb4e0009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4f5f77700 1 -- 172.21.15.161:0/3269708418 shutdown_connections 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4f5f77700 1 --2- 172.21.15.161:0/3269708418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f007a320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4f5f77700 1 -- 172.21.15.161:0/3269708418 >> 172.21.15.161:0/3269708418 conn(0x7fb4f0100970 msgr2=0x7fb4f0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4f5f77700 1 -- 172.21.15.161:0/3269708418 shutdown_connections 2023-07-14T11:12:53.613 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.959+0000 7fb4f5f77700 1 -- 172.21.15.161:0/3269708418 wait complete. 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.961+0000 7fb4f5f77700 1 Processor -- start 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.961+0000 7fb4f5f77700 1 -- start start 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.961+0000 7fb4f5f77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.961+0000 7fb4f5f77700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f019b880 con 0x7fb4f007be20 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.962+0000 7fb4ef7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.962+0000 7fb4ef7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47238/0 (socket says 172.21.15.161:47238) 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.962+0000 7fb4ef7fe700 1 -- 172.21.15.161:0/858749548 learned_addr learned my addr 172.21.15.161:0/858749548 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:53.614 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.962+0000 7fb4ef7fe700 1 -- 172.21.15.161:0/858749548 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009550 con 0x7fb4f007be20 2023-07-14T11:12:53.615 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.962+0000 7fb4ef7fe700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f019b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0003f50 tx=0x7fb4e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:53.615 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.962+0000 7fb4ecff9700 1 -- 172.21.15.161:0/858749548 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4e0003aa0 con 0x7fb4f007be20 2023-07-14T11:12:53.615 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.963+0000 7fb4ecff9700 1 -- 172.21.15.161:0/858749548 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4e0003c00 con 0x7fb4f007be20 2023-07-14T11:12:53.615 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.963+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f019ba80 con 0x7fb4f007be20 2023-07-14T11:12:53.615 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.963+0000 7fb4ecff9700 1 -- 172.21.15.161:0/858749548 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4e00247c0 con 0x7fb4f007be20 2023-07-14T11:12:53.615 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.963+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f019bea0 con 0x7fb4f007be20 2023-07-14T11:12:53.615 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.964+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f0195320 con 0x7fb4f007be20 2023-07-14T11:12:53.616 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.964+0000 7fb4ecff9700 1 -- 172.21.15.161:0/858749548 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fb4e001ab60 con 0x7fb4f007be20 2023-07-14T11:12:53.616 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.964+0000 7fb4ecff9700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fb4d8038760 0x7fb4d803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:53.616 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.964+0000 7fb4ecff9700 1 -- 172.21.15.161:0/858749548 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb4e004c680 con 0x7fb4f007be20 2023-07-14T11:12:53.616 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.964+0000 7fb4eeffd700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fb4d8038760 0x7fb4d803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:53.616 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.965+0000 7fb4eeffd700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fb4d8038760 0x7fb4d803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006fd0 tx=0x7fb4e4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:53.616 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:52.969+0000 7fb4ecff9700 1 -- 172.21.15.161:0/858749548 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb4e0024920 con 0x7fb4f007be20 2023-07-14T11:12:53.616 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.164+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4f0060950 con 0x7fb4d8038760 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.165+0000 7fb4ecff9700 1 -- 172.21.15.161:0/858749548 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb4f0060950 con 0x7fb4d8038760 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fb4d8038760 msgr2=0x7fb4d803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fb4d8038760 0x7fb4d803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006fd0 tx=0x7fb4e4006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 msgr2=0x7fb4f019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f019b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0003f50 tx=0x7fb4e0011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 shutdown_connections 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fb4d8038760 0x7fb4d803ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:53.617 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 --2- 172.21.15.161:0/858749548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f007be20 0x7fb4f019b340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:53.618 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 >> 172.21.15.161:0/858749548 conn(0x7fb4f0100970 msgr2=0x7fb4f0102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:53.618 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 shutdown_connections 2023-07-14T11:12:53.618 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:53.168+0000 7fb4f5f77700 1 -- 172.21.15.161:0/858749548 wait complete. 2023-07-14T11:12:53.618 INFO:teuthology.orchestra.run.smithi161.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-14T11:12:53.618 INFO:teuthology.orchestra.run.smithi161.stdout:Adding key to root@localhost authorized_keys... 2023-07-14T11:12:53.618 INFO:teuthology.orchestra.run.smithi161.stdout:Adding host smithi161... 2023-07-14T11:12:54.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:54 smithi161 ceph-mon[95526]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:12:55.074 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:55 smithi161 ceph-mon[95526]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "addr": "172.21.15.161", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:12:56.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:56 smithi161 ceph-mon[95526]: Deploying cephadm binary to smithi161 2023-07-14T11:12:58.005 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout Added host 'smithi161' with addr '172.21.15.161' 2023-07-14T11:12:58.005 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.168+0000 7ff83fd11700 1 Processor -- start 2023-07-14T11:12:58.005 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.168+0000 7ff83fd11700 1 -- start start 2023-07-14T11:12:58.005 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.168+0000 7ff83fd11700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:58.005 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.168+0000 7ff83fd11700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83807a840 con 0x7ff83807be00 2023-07-14T11:12:58.006 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.168+0000 7ff83daad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:58.006 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.168+0000 7ff83daad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47254/0 (socket says 172.21.15.161:47254) 2023-07-14T11:12:58.006 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.168+0000 7ff83daad700 1 -- 172.21.15.161:0/4239890390 learned_addr learned my addr 172.21.15.161:0/4239890390 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:58.006 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.169+0000 7ff83daad700 1 -- 172.21.15.161:0/4239890390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83807a980 con 0x7ff83807be00 2023-07-14T11:12:58.006 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.169+0000 7ff83daad700 1 --2- 172.21.15.161:0/4239890390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83807a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8340098a0 tx=0x7ff834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2415f059a889fd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:58.006 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.169+0000 7ff83caab700 1 -- 172.21.15.161:0/4239890390 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff834004160 con 0x7ff83807be00 2023-07-14T11:12:58.006 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.169+0000 7ff83caab700 1 -- 172.21.15.161:0/4239890390 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8340042c0 con 0x7ff83807be00 2023-07-14T11:12:58.007 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.169+0000 7ff83caab700 1 -- 172.21.15.161:0/4239890390 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff834011a00 con 0x7ff83807be00 2023-07-14T11:12:58.007 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.170+0000 7ff83fd11700 1 -- 172.21.15.161:0/4239890390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 msgr2=0x7ff83807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:58.007 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.170+0000 7ff83fd11700 1 --2- 172.21.15.161:0/4239890390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83807a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8340098a0 tx=0x7ff834009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:58.007 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.170+0000 7ff83fd11700 1 -- 172.21.15.161:0/4239890390 shutdown_connections 2023-07-14T11:12:58.007 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.170+0000 7ff83fd11700 1 --2- 172.21.15.161:0/4239890390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83807a300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:58.007 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.170+0000 7ff83fd11700 1 -- 172.21.15.161:0/4239890390 >> 172.21.15.161:0/4239890390 conn(0x7ff838100970 msgr2=0x7ff838102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:58.007 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.170+0000 7ff83fd11700 1 -- 172.21.15.161:0/4239890390 shutdown_connections 2023-07-14T11:12:58.008 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.170+0000 7ff83fd11700 1 -- 172.21.15.161:0/4239890390 wait complete. 2023-07-14T11:12:58.008 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.171+0000 7ff83fd11700 1 Processor -- start 2023-07-14T11:12:58.008 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.171+0000 7ff83fd11700 1 -- start start 2023-07-14T11:12:58.008 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.172+0000 7ff83fd11700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:58.008 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.172+0000 7ff83fd11700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83819b880 con 0x7ff83807be00 2023-07-14T11:12:58.008 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.172+0000 7ff83daad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:58.008 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.172+0000 7ff83daad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47270/0 (socket says 172.21.15.161:47270) 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.172+0000 7ff83daad700 1 -- 172.21.15.161:0/2448799711 learned_addr learned my addr 172.21.15.161:0/2448799711 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.172+0000 7ff83daad700 1 -- 172.21.15.161:0/2448799711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff834009550 con 0x7ff83807be00 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.172+0000 7ff83daad700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83819b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff834009130 tx=0x7ff834011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.173+0000 7ff82effd700 1 -- 172.21.15.161:0/2448799711 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff834003ab0 con 0x7ff83807be00 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.173+0000 7ff82effd700 1 -- 172.21.15.161:0/2448799711 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff834003c10 con 0x7ff83807be00 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.173+0000 7ff82effd700 1 -- 172.21.15.161:0/2448799711 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8340247d0 con 0x7ff83807be00 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.173+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83819ba80 con 0x7ff83807be00 2023-07-14T11:12:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.173+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83819bea0 con 0x7ff83807be00 2023-07-14T11:12:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.174+0000 7ff82effd700 1 -- 172.21.15.161:0/2448799711 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff83401a430 con 0x7ff83807be00 2023-07-14T11:12:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.174+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8381952f0 con 0x7ff83807be00 2023-07-14T11:12:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.174+0000 7ff82effd700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7ff824038760 0x7ff82403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.174+0000 7ff82effd700 1 -- 172.21.15.161:0/2448799711 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff83404c840 con 0x7ff83807be00 2023-07-14T11:12:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.175+0000 7ff83d2ac700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7ff824038760 0x7ff82403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.175+0000 7ff83d2ac700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7ff824038760 0x7ff82403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff828006fd0 tx=0x7ff828006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.179+0000 7ff82effd700 1 -- 172.21.15.161:0/2448799711 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff834018b40 con 0x7ff83807be00 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:54.379+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "addr": "172.21.15.161", "target": ["mon-mgr", ""]}) v1 -- 0x7ff838060950 con 0x7ff824038760 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.541+0000 7ff82effd700 1 -- 172.21.15.161:0/2448799711 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff838060950 con 0x7ff824038760 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7ff824038760 msgr2=0x7ff82403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7ff824038760 0x7ff82403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff828006fd0 tx=0x7ff828006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 msgr2=0x7ff83819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83819b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff834009130 tx=0x7ff834011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 shutdown_connections 2023-07-14T11:12:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7ff824038760 0x7ff82403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:58.012 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 --2- 172.21.15.161:0/2448799711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff83807be00 0x7ff83819b340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:58.012 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 >> 172.21.15.161:0/2448799711 conn(0x7ff838100970 msgr2=0x7ff8381020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:58.012 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 shutdown_connections 2023-07-14T11:12:58.012 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:57.544+0000 7ff83fd11700 1 -- 172.21.15.161:0/2448799711 wait complete. 2023-07-14T11:12:58.012 INFO:teuthology.orchestra.run.smithi161.stdout:Deploying unmanaged mon service... 2023-07-14T11:12:58.837 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:58 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:12:58.838 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:58 smithi161 ceph-mon[95526]: Added host smithi161 2023-07-14T11:12:58.838 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:58 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:12:59.296 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-14T11:12:59.296 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.578+0000 7f8f21ff8700 1 Processor -- start 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.579+0000 7f8f21ff8700 1 -- start start 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.579+0000 7f8f21ff8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c079d20 0x7f8f1c107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.579+0000 7f8f21ff8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c108400 con 0x7f8f1c079d20 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.579+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c079d20 0x7f8f1c107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.579+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c079d20 0x7f8f1c107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49252/0 (socket says 172.21.15.161:49252) 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.579+0000 7f8f1b7fe700 1 -- 172.21.15.161:0/1388191955 learned_addr learned my addr 172.21.15.161:0/1388191955 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.580+0000 7f8f1b7fe700 1 -- 172.21.15.161:0/1388191955 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c108540 con 0x7f8f1c079d20 2023-07-14T11:12:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.580+0000 7f8f1b7fe700 1 --2- 172.21.15.161:0/1388191955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c079d20 0x7f8f1c107ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0098a0 tx=0x7f8f0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48480f32c49c8cd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:59.298 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.580+0000 7f8f1a7fc700 1 -- 172.21.15.161:0/1388191955 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f0c004160 con 0x7f8f1c079d20 2023-07-14T11:12:59.298 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.580+0000 7f8f1a7fc700 1 -- 172.21.15.161:0/1388191955 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f0c0042c0 con 0x7f8f1c079d20 2023-07-14T11:12:59.298 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.580+0000 7f8f1a7fc700 1 -- 172.21.15.161:0/1388191955 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f0c011a00 con 0x7f8f1c079d20 2023-07-14T11:12:59.298 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.581+0000 7f8f21ff8700 1 -- 172.21.15.161:0/1388191955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c079d20 msgr2=0x7f8f1c107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:59.298 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.581+0000 7f8f21ff8700 1 --2- 172.21.15.161:0/1388191955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c079d20 0x7f8f1c107ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0098a0 tx=0x7f8f0c009bb0 comp rx=0 tx=0).stop 2023-07-14T11:12:59.298 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.581+0000 7f8f21ff8700 1 -- 172.21.15.161:0/1388191955 shutdown_connections 2023-07-14T11:12:59.298 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.581+0000 7f8f21ff8700 1 --2- 172.21.15.161:0/1388191955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c079d20 0x7f8f1c107ec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.581+0000 7f8f21ff8700 1 -- 172.21.15.161:0/1388191955 >> 172.21.15.161:0/1388191955 conn(0x7f8f1c0ff890 msgr2=0x7f8f1c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.581+0000 7f8f21ff8700 1 -- 172.21.15.161:0/1388191955 shutdown_connections 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.581+0000 7f8f21ff8700 1 -- 172.21.15.161:0/1388191955 wait complete. 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.582+0000 7f8f21ff8700 1 Processor -- start 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.582+0000 7f8f21ff8700 1 -- start start 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.582+0000 7f8f21ff8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c1057d0 0x7f8f1c103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.583+0000 7f8f21ff8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c105be0 con 0x7f8f1c1057d0 2023-07-14T11:12:59.299 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.583+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c1057d0 0x7f8f1c103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:59.300 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.583+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c1057d0 0x7f8f1c103e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49262/0 (socket says 172.21.15.161:49262) 2023-07-14T11:12:59.300 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.583+0000 7f8f1b7fe700 1 -- 172.21.15.161:0/4019315806 learned_addr learned my addr 172.21.15.161:0/4019315806 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:12:59.300 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.583+0000 7f8f1b7fe700 1 -- 172.21.15.161:0/4019315806 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c009550 con 0x7f8f1c1057d0 2023-07-14T11:12:59.300 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.583+0000 7f8f1b7fe700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c1057d0 0x7f8f1c103e60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009130 tx=0x7f8f0c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:59.300 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.584+0000 7f8f18ff9700 1 -- 172.21.15.161:0/4019315806 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f0c003ab0 con 0x7f8f1c1057d0 2023-07-14T11:12:59.300 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.584+0000 7f8f18ff9700 1 -- 172.21.15.161:0/4019315806 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f0c003c10 con 0x7f8f1c1057d0 2023-07-14T11:12:59.300 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.584+0000 7f8f18ff9700 1 -- 172.21.15.161:0/4019315806 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f0c0247d0 con 0x7f8f1c1057d0 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.584+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c1043a0 con 0x7f8f1c1057d0 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.584+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c1047c0 con 0x7f8f1c1057d0 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.585+0000 7f8f18ff9700 1 -- 172.21.15.161:0/4019315806 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8f0c01abb0 con 0x7f8f1c1057d0 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.585+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c061b40 con 0x7f8f1c1057d0 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.585+0000 7f8f18ff9700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f8f04038760 0x7f8f0403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.585+0000 7f8f18ff9700 1 -- 172.21.15.161:0/4019315806 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8f0c04c810 con 0x7f8f1c1057d0 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.586+0000 7f8f1affd700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f8f04038760 0x7f8f0403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:12:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.586+0000 7f8f1affd700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f8f04038760 0x7f8f0403ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f10006fd0 tx=0x7f8f10006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:12:59.302 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.590+0000 7f8f18ff9700 1 -- 172.21.15.161:0/4019315806 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8f0c018530 con 0x7f8f1c1057d0 2023-07-14T11:12:59.302 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.803+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8f1c101be0 con 0x7f8f04038760 2023-07-14T11:12:59.302 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.810+0000 7f8f18ff9700 1 -- 172.21.15.161:0/4019315806 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8f1c101be0 con 0x7f8f04038760 2023-07-14T11:12:59.302 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f8f04038760 msgr2=0x7f8f0403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:59.302 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f8f04038760 0x7f8f0403ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f10006fd0 tx=0x7f8f10006e40 comp rx=0 tx=0).stop 2023-07-14T11:12:59.302 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c1057d0 msgr2=0x7f8f1c103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:12:59.302 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c1057d0 0x7f8f1c103e60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009130 tx=0x7f8f0c011ad0 comp rx=0 tx=0).stop 2023-07-14T11:12:59.303 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 shutdown_connections 2023-07-14T11:12:59.303 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f8f04038760 0x7f8f0403ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:59.303 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 --2- 172.21.15.161:0/4019315806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f1c1057d0 0x7f8f1c103e60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:12:59.303 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 >> 172.21.15.161:0/4019315806 conn(0x7f8f1c0ff890 msgr2=0x7f8f1c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:12:59.303 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 shutdown_connections 2023-07-14T11:12:59.303 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:12:58.813+0000 7f8f21ff8700 1 -- 172.21.15.161:0/4019315806 wait complete. 2023-07-14T11:12:59.303 INFO:teuthology.orchestra.run.smithi161.stdout:Deploying unmanaged mgr service... 2023-07-14T11:12:59.968 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:59 smithi161 ceph-mon[95526]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:12:59.968 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:59 smithi161 ceph-mon[95526]: Saving service mon spec with placement count:5 2023-07-14T11:12:59.968 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:12:59 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:13:00.989 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-14T11:13:00.989 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.220+0000 7f99f63f2700 1 Processor -- start 2023-07-14T11:13:00.989 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.221+0000 7f99f63f2700 1 -- start start 2023-07-14T11:13:00.989 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.221+0000 7f99f63f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:00.989 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.221+0000 7f99f63f2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f007a840 con 0x7f99f007be00 2023-07-14T11:13:00.989 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.221+0000 7f99effff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.221+0000 7f99effff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49274/0 (socket says 172.21.15.161:49274) 2023-07-14T11:13:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.221+0000 7f99effff700 1 -- 172.21.15.161:0/3940660817 learned_addr learned my addr 172.21.15.161:0/3940660817 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.222+0000 7f99effff700 1 -- 172.21.15.161:0/3940660817 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f007a980 con 0x7f99f007be00 2023-07-14T11:13:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.222+0000 7f99effff700 1 --2- 172.21.15.161:0/3940660817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f007a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098a0 tx=0x7f99e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20853667918f70f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.222+0000 7f99eeffd700 1 -- 172.21.15.161:0/3940660817 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e0004160 con 0x7f99f007be00 2023-07-14T11:13:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.222+0000 7f99eeffd700 1 -- 172.21.15.161:0/3940660817 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99e00042c0 con 0x7f99f007be00 2023-07-14T11:13:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.222+0000 7f99eeffd700 1 -- 172.21.15.161:0/3940660817 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e0011a00 con 0x7f99f007be00 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.223+0000 7f99f63f2700 1 -- 172.21.15.161:0/3940660817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 msgr2=0x7f99f007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.223+0000 7f99f63f2700 1 --2- 172.21.15.161:0/3940660817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f007a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098a0 tx=0x7f99e0009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.223+0000 7f99f63f2700 1 -- 172.21.15.161:0/3940660817 shutdown_connections 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.223+0000 7f99f63f2700 1 --2- 172.21.15.161:0/3940660817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f007a300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.223+0000 7f99f63f2700 1 -- 172.21.15.161:0/3940660817 >> 172.21.15.161:0/3940660817 conn(0x7f99f0100970 msgr2=0x7f99f0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.223+0000 7f99f63f2700 1 -- 172.21.15.161:0/3940660817 shutdown_connections 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.223+0000 7f99f63f2700 1 -- 172.21.15.161:0/3940660817 wait complete. 2023-07-14T11:13:00.991 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.224+0000 7f99f63f2700 1 Processor -- start 2023-07-14T11:13:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.224+0000 7f99f63f2700 1 -- start start 2023-07-14T11:13:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.225+0000 7f99f63f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.225+0000 7f99f63f2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019b880 con 0x7f99f007be00 2023-07-14T11:13:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.225+0000 7f99effff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.225+0000 7f99effff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49286/0 (socket says 172.21.15.161:49286) 2023-07-14T11:13:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.225+0000 7f99effff700 1 -- 172.21.15.161:0/4201654927 learned_addr learned my addr 172.21.15.161:0/4201654927 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.225+0000 7f99effff700 1 -- 172.21.15.161:0/4201654927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009550 con 0x7f99f007be00 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.226+0000 7f99effff700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f019b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009130 tx=0x7f99e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.226+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e0003ab0 con 0x7f99f007be00 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.226+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99e0003c10 con 0x7f99f007be00 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.226+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f019ba80 con 0x7f99f007be00 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.226+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e00247d0 con 0x7f99f007be00 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.226+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f019bea0 con 0x7f99f007be00 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.227+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f01952f0 con 0x7f99f007be00 2023-07-14T11:13:00.993 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.228+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f99e001a430 con 0x7f99f007be00 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.228+0000 7f99ed7fa700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f99d8038710 0x7f99d803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.228+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f99e004c8e0 con 0x7f99f007be00 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.228+0000 7f99ef7fe700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f99d8038710 0x7f99d803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.229+0000 7f99ef7fe700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f99d8038710 0x7f99d803abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99e4006fd0 tx=0x7f99e4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.233+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f99e0018530 con 0x7f99f007be00 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.248+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mon.0 v2:172.21.15.161:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e0018530 con 0x7f99f007be00 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.465+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f99f0060950 con 0x7f99d8038710 2023-07-14T11:13:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.471+0000 7f99ed7fa700 1 -- 172.21.15.161:0/4201654927 <== mgr.14116 v2:172.21.15.161:6800/3292405426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f99f0060950 con 0x7f99d8038710 2023-07-14T11:13:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f99d8038710 msgr2=0x7f99d803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f99d8038710 0x7f99d803abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99e4006fd0 tx=0x7f99e4006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 msgr2=0x7f99f019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f019b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009130 tx=0x7f99e0011ad0 comp rx=0 tx=0).stop 2023-07-14T11:13:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 shutdown_connections 2023-07-14T11:13:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f99d8038710 0x7f99d803abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 --2- 172.21.15.161:0/4201654927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99f007be00 0x7f99f019b340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:00.996 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.473+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 >> 172.21.15.161:0/4201654927 conn(0x7f99f0100970 msgr2=0x7f99f01020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:00.996 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.474+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 shutdown_connections 2023-07-14T11:13:00.996 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:00.474+0000 7f99f63f2700 1 -- 172.21.15.161:0/4201654927 wait complete. 2023-07-14T11:13:01.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:01 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:13:01.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:01 smithi161 ceph-mon[95526]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:01.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:01 smithi161 ceph-mon[95526]: Saving service mgr spec with placement count:2 2023-07-14T11:13:01.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:01 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:13:01.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:01 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:13:02.367 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.570+0000 7f96293a8700 1 Processor -- start 2023-07-14T11:13:02.367 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.570+0000 7f96293a8700 1 -- start start 2023-07-14T11:13:02.367 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.570+0000 7f96293a8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:02.368 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.570+0000 7f96293a8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962407a840 con 0x7f962407be00 2023-07-14T11:13:02.368 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.571+0000 7f9622ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:02.368 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.571+0000 7f9622ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49300/0 (socket says 172.21.15.161:49300) 2023-07-14T11:13:02.368 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.571+0000 7f9622ffd700 1 -- 172.21.15.161:0/3007333424 learned_addr learned my addr 172.21.15.161:0/3007333424 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:02.368 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.571+0000 7f9622ffd700 1 -- 172.21.15.161:0/3007333424 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962407a980 con 0x7f962407be00 2023-07-14T11:13:02.368 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.571+0000 7f9622ffd700 1 --2- 172.21.15.161:0/3007333424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962407a300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f96140098a0 tx=0x7f9614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d869f8f5c90b4796 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:02.368 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.571+0000 7f9621ffb700 1 -- 172.21.15.161:0/3007333424 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9614004160 con 0x7f962407be00 2023-07-14T11:13:02.369 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f9621ffb700 1 -- 172.21.15.161:0/3007333424 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96140042c0 con 0x7f962407be00 2023-07-14T11:13:02.369 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f9621ffb700 1 -- 172.21.15.161:0/3007333424 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140119b0 con 0x7f962407be00 2023-07-14T11:13:02.369 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f96293a8700 1 -- 172.21.15.161:0/3007333424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 msgr2=0x7f962407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:02.369 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f96293a8700 1 --2- 172.21.15.161:0/3007333424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962407a300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f96140098a0 tx=0x7f9614009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:02.369 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f96293a8700 1 -- 172.21.15.161:0/3007333424 shutdown_connections 2023-07-14T11:13:02.369 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f96293a8700 1 --2- 172.21.15.161:0/3007333424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962407a300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:02.369 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f96293a8700 1 -- 172.21.15.161:0/3007333424 >> 172.21.15.161:0/3007333424 conn(0x7f9624100970 msgr2=0x7f9624102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f96293a8700 1 -- 172.21.15.161:0/3007333424 shutdown_connections 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.572+0000 7f96293a8700 1 -- 172.21.15.161:0/3007333424 wait complete. 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.574+0000 7f96293a8700 1 Processor -- start 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.574+0000 7f96293a8700 1 -- start start 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.574+0000 7f96293a8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.574+0000 7f96293a8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962419b880 con 0x7f962407be00 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.574+0000 7f9622ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:02.370 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.574+0000 7f9622ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49304/0 (socket says 172.21.15.161:49304) 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.574+0000 7f9622ffd700 1 -- 172.21.15.161:0/3269751110 learned_addr learned my addr 172.21.15.161:0/3269751110 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.575+0000 7f9622ffd700 1 -- 172.21.15.161:0/3269751110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614009550 con 0x7f962407be00 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.575+0000 7f9622ffd700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962419b340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9614003e60 tx=0x7f9614011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.575+0000 7f960bfff700 1 -- 172.21.15.161:0/3269751110 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9614003a60 con 0x7f962407be00 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.575+0000 7f960bfff700 1 -- 172.21.15.161:0/3269751110 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9614003bc0 con 0x7f962407be00 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.575+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962419ba80 con 0x7f962407be00 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.575+0000 7f960bfff700 1 -- 172.21.15.161:0/3269751110 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140196c0 con 0x7f962407be00 2023-07-14T11:13:02.371 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.575+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962419bea0 con 0x7f962407be00 2023-07-14T11:13:02.372 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.576+0000 7f960bfff700 1 -- 172.21.15.161:0/3269751110 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f961401a430 con 0x7f962407be00 2023-07-14T11:13:02.372 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.576+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9624061b40 con 0x7f962407be00 2023-07-14T11:13:02.372 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.577+0000 7f960bfff700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f960c038790 0x7f960c03ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:02.372 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.577+0000 7f960bfff700 1 -- 172.21.15.161:0/3269751110 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f961404c9d0 con 0x7f962407be00 2023-07-14T11:13:02.372 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.577+0000 7f96227fc700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f960c038790 0x7f960c03ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:02.372 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.578+0000 7f96227fc700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f960c038790 0x7f960c03ac40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9618006fd0 tx=0x7f9618006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:02.372 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.582+0000 7f960bfff700 1 -- 172.21.15.161:0/3269751110 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9614010380 con 0x7f962407be00 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.793+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f962419e8e0 con 0x7f962407be00 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.797+0000 7f960bfff700 1 -- 172.21.15.161:0/3269751110 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f9614010380 con 0x7f962407be00 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.799+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f960c038790 msgr2=0x7f960c03ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.799+0000 7f96293a8700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f960c038790 0x7f960c03ac40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9618006fd0 tx=0x7f9618006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.799+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 msgr2=0x7f962419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.799+0000 7f96293a8700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962419b340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9614003e60 tx=0x7f9614011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.800+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 shutdown_connections 2023-07-14T11:13:02.373 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.800+0000 7f96293a8700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f960c038790 0x7f960c03ac40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:02.374 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.800+0000 7f96293a8700 1 --2- 172.21.15.161:0/3269751110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962407be00 0x7f962419b340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:02.374 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.800+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 >> 172.21.15.161:0/3269751110 conn(0x7f9624100970 msgr2=0x7f9624102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:02.374 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.800+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 shutdown_connections 2023-07-14T11:13:02.374 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:01.800+0000 7f96293a8700 1 -- 172.21.15.161:0/3269751110 wait complete. 2023-07-14T11:13:03.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:02 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3269751110' entity='client.admin' 2023-07-14T11:13:03.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.027+0000 7f95373c8700 1 Processor -- start 2023-07-14T11:13:03.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.027+0000 7f95373c8700 1 -- start start 2023-07-14T11:13:03.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.028+0000 7f95373c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:03.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.028+0000 7f95373c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953007a840 con 0x7f953007be00 2023-07-14T11:13:03.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.028+0000 7f9535164700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:03.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.028+0000 7f9535164700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49310/0 (socket says 172.21.15.161:49310) 2023-07-14T11:13:03.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.028+0000 7f9535164700 1 -- 172.21.15.161:0/3702608933 learned_addr learned my addr 172.21.15.161:0/3702608933 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:03.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.029+0000 7f9535164700 1 -- 172.21.15.161:0/3702608933 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953007a980 con 0x7f953007be00 2023-07-14T11:13:03.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.029+0000 7f9535164700 1 --2- 172.21.15.161:0/3702608933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953007a300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098a0 tx=0x7f952c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f0a6a610d98561 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:03.754 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.029+0000 7f9527fff700 1 -- 172.21.15.161:0/3702608933 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f952c004160 con 0x7f953007be00 2023-07-14T11:13:03.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.029+0000 7f9527fff700 1 -- 172.21.15.161:0/3702608933 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f952c0042c0 con 0x7f953007be00 2023-07-14T11:13:03.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.029+0000 7f9527fff700 1 -- 172.21.15.161:0/3702608933 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f952c0119b0 con 0x7f953007be00 2023-07-14T11:13:03.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.030+0000 7f95373c8700 1 -- 172.21.15.161:0/3702608933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 msgr2=0x7f953007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:03.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.030+0000 7f95373c8700 1 --2- 172.21.15.161:0/3702608933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953007a300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098a0 tx=0x7f952c009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:03.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.030+0000 7f95373c8700 1 -- 172.21.15.161:0/3702608933 shutdown_connections 2023-07-14T11:13:03.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.030+0000 7f95373c8700 1 --2- 172.21.15.161:0/3702608933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953007a300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:03.755 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.030+0000 7f95373c8700 1 -- 172.21.15.161:0/3702608933 >> 172.21.15.161:0/3702608933 conn(0x7f9530100970 msgr2=0x7f9530102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:03.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.030+0000 7f95373c8700 1 -- 172.21.15.161:0/3702608933 shutdown_connections 2023-07-14T11:13:03.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.030+0000 7f95373c8700 1 -- 172.21.15.161:0/3702608933 wait complete. 2023-07-14T11:13:03.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.031+0000 7f95373c8700 1 Processor -- start 2023-07-14T11:13:03.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.031+0000 7f95373c8700 1 -- start start 2023-07-14T11:13:03.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.031+0000 7f95373c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:03.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.031+0000 7f95373c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953019b880 con 0x7f953007be00 2023-07-14T11:13:03.756 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.032+0000 7f9535164700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.032+0000 7f9535164700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49324/0 (socket says 172.21.15.161:49324) 2023-07-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.032+0000 7f9535164700 1 -- 172.21.15.161:0/1105647480 learned_addr learned my addr 172.21.15.161:0/1105647480 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.032+0000 7f9535164700 1 -- 172.21.15.161:0/1105647480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009550 con 0x7f953007be00 2023-07-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.032+0000 7f9535164700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953019b340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f952c006af0 tx=0x7f952c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.032+0000 7f95267fc700 1 -- 172.21.15.161:0/1105647480 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f952c003a40 con 0x7f953007be00 2023-07-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.033+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953019ba80 con 0x7f953007be00 2023-07-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.033+0000 7f95267fc700 1 -- 172.21.15.161:0/1105647480 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f952c003ba0 con 0x7f953007be00 2023-07-14T11:13:03.758 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.033+0000 7f95267fc700 1 -- 172.21.15.161:0/1105647480 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f952c019690 con 0x7f953007be00 2023-07-14T11:13:03.758 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.033+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953019bea0 con 0x7f953007be00 2023-07-14T11:13:03.758 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.034+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9530061b40 con 0x7f953007be00 2023-07-14T11:13:03.758 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.034+0000 7f95267fc700 1 -- 172.21.15.161:0/1105647480 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f952c01aa60 con 0x7f953007be00 2023-07-14T11:13:03.758 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.034+0000 7f95267fc700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f951c0386c0 0x7f951c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:03.758 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.034+0000 7f95267fc700 1 -- 172.21.15.161:0/1105647480 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f952c04c740 con 0x7f953007be00 2023-07-14T11:13:03.758 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.035+0000 7f9534963700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f951c0386c0 0x7f951c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:03.759 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.035+0000 7f9534963700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f951c0386c0 0x7f951c03ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9520006fd0 tx=0x7f9520006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:03.759 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.039+0000 7f95267fc700 1 -- 172.21.15.161:0/1105647480 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f952c023750 con 0x7f953007be00 2023-07-14T11:13:03.759 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.245+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f953019e8e0 con 0x7f953007be00 2023-07-14T11:13:03.759 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.249+0000 7f95267fc700 1 -- 172.21.15.161:0/1105647480 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f952c023390 con 0x7f953007be00 2023-07-14T11:13:03.759 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f951c0386c0 msgr2=0x7f951c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:03.759 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f951c0386c0 0x7f951c03ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9520006fd0 tx=0x7f9520006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:03.759 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 msgr2=0x7f953019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:03.760 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953019b340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f952c006af0 tx=0x7f952c011a10 comp rx=0 tx=0).stop 2023-07-14T11:13:03.760 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 shutdown_connections 2023-07-14T11:13:03.760 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f951c0386c0 0x7f951c03ab70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:03.760 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 --2- 172.21.15.161:0/1105647480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953007be00 0x7f953019b340 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:03.760 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 >> 172.21.15.161:0/1105647480 conn(0x7f9530100970 msgr2=0x7f95301020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:03.760 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.252+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 shutdown_connections 2023-07-14T11:13:03.760 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:03.253+0000 7f95373c8700 1 -- 172.21.15.161:0/1105647480 wait complete. 2023-07-14T11:13:03.761 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling the dashboard module... 2023-07-14T11:13:04.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:04 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:13:04.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:04 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1105647480' entity='client.admin' 2023-07-14T11:13:05.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:05 smithi161 ceph-mon[95526]: from='mgr.14116 172.21.15.161:0/605333364' entity='mgr.x' 2023-07-14T11:13:05.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:05 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1726026097' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-14T11:13:05.746 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: ignoring --setuser ceph since I am not root 2023-07-14T11:13:05.746 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: ignoring --setgroup ceph since I am not root 2023-07-14T11:13:06.045 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.344+0000 7fc493901700 1 Processor -- start 2023-07-14T11:13:06.045 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.344+0000 7fc493901700 1 -- start start 2023-07-14T11:13:06.045 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.345+0000 7fc493901700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.345+0000 7fc493901700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c105c70 con 0x7fc48c105320 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.345+0000 7fc49169d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.345+0000 7fc49169d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49334/0 (socket says 172.21.15.161:49334) 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.345+0000 7fc49169d700 1 -- 172.21.15.161:0/4048342995 learned_addr learned my addr 172.21.15.161:0/4048342995 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.346+0000 7fc49169d700 1 -- 172.21.15.161:0/4048342995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c105db0 con 0x7fc48c105320 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.346+0000 7fc49169d700 1 --2- 172.21.15.161:0/4048342995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c105730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4880098a0 tx=0x7fc488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7916ecd6bbd43f9a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.346+0000 7fc483fff700 1 -- 172.21.15.161:0/4048342995 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc488004160 con 0x7fc48c105320 2023-07-14T11:13:06.046 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.346+0000 7fc483fff700 1 -- 172.21.15.161:0/4048342995 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4880042c0 con 0x7fc48c105320 2023-07-14T11:13:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.346+0000 7fc483fff700 1 -- 172.21.15.161:0/4048342995 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4880119b0 con 0x7fc48c105320 2023-07-14T11:13:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.347+0000 7fc493901700 1 -- 172.21.15.161:0/4048342995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 msgr2=0x7fc48c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.347+0000 7fc493901700 1 --2- 172.21.15.161:0/4048342995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c105730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4880098a0 tx=0x7fc488009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.347+0000 7fc493901700 1 -- 172.21.15.161:0/4048342995 shutdown_connections 2023-07-14T11:13:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.347+0000 7fc493901700 1 --2- 172.21.15.161:0/4048342995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c105730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.347+0000 7fc493901700 1 -- 172.21.15.161:0/4048342995 >> 172.21.15.161:0/4048342995 conn(0x7fc48c100970 msgr2=0x7fc48c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.347+0000 7fc493901700 1 -- 172.21.15.161:0/4048342995 shutdown_connections 2023-07-14T11:13:06.048 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.347+0000 7fc493901700 1 -- 172.21.15.161:0/4048342995 wait complete. 2023-07-14T11:13:06.048 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.348+0000 7fc493901700 1 Processor -- start 2023-07-14T11:13:06.048 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.348+0000 7fc493901700 1 -- start start 2023-07-14T11:13:06.048 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.348+0000 7fc493901700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:06.048 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.348+0000 7fc493901700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c19b840 con 0x7fc48c105320 2023-07-14T11:13:06.048 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.349+0000 7fc49169d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:06.048 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.349+0000 7fc49169d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49350/0 (socket says 172.21.15.161:49350) 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.349+0000 7fc49169d700 1 -- 172.21.15.161:0/1726026097 learned_addr learned my addr 172.21.15.161:0/1726026097 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.349+0000 7fc49169d700 1 -- 172.21.15.161:0/1726026097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488009550 con 0x7fc48c105320 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.349+0000 7fc49169d700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c19b300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc488003f00 tx=0x7fc488011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.350+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc488003b20 con 0x7fc48c105320 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.350+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc488003c80 con 0x7fc48c105320 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.350+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c19ba40 con 0x7fc48c105320 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.350+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc488019770 con 0x7fc48c105320 2023-07-14T11:13:06.049 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.350+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c19be60 con 0x7fc48c105320 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.351+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c061b40 con 0x7fc48c105320 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.351+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc48801ab60 con 0x7fc48c105320 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.352+0000 7fc4827fc700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fc478038760 0x7fc47803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.352+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc48804ca30 con 0x7fc48c105320 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.352+0000 7fc490e9c700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fc478038760 0x7fc47803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.352+0000 7fc490e9c700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fc478038760 0x7fc47803ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006fd0 tx=0x7fc47c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.356+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc48801a430 con 0x7fc48c105320 2023-07-14T11:13:06.050 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:04.626+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc48c19ef10 con 0x7fc48c105320 2023-07-14T11:13:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.488+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fc48802d350 con 0x7fc48c105320 2023-07-14T11:13:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.490+0000 7fc4827fc700 1 -- 172.21.15.161:0/1726026097 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc48804c380 con 0x7fc48c105320 2023-07-14T11:13:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fc478038760 msgr2=0x7fc47803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fc478038760 0x7fc47803ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006fd0 tx=0x7fc47c006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 msgr2=0x7fc48c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c19b300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc488003f00 tx=0x7fc488011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 shutdown_connections 2023-07-14T11:13:06.052 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fc478038760 0x7fc47803ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:06.052 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 --2- 172.21.15.161:0/1726026097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc48c105320 0x7fc48c19b300 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc488003f00 tx=0x7fc488011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:06.052 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 >> 172.21.15.161:0/1726026097 conn(0x7fc48c100970 msgr2=0x7fc48c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:06.052 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 shutdown_connections 2023-07-14T11:13:06.052 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:05.493+0000 7fc493901700 1 -- 172.21.15.161:0/1726026097 wait complete. 2023-07-14T11:13:06.062 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:05.791+0000 7f6cef664200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:13:06.062 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:05.904+0000 7f6cef664200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:13:06.062 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:06.032+0000 7f6cef664200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:13:06.403 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:06.173+0000 7f6cef664200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:13:06.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:06 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1726026097' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-14T11:13:06.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:06 smithi161 ceph-mon[95526]: mgrmap e9: x(active, since 18s) 2023-07-14T11:13:06.746 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:06.401+0000 7f6cef664200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:13:07.114 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:06.824+0000 7f6cef664200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:13:07.115 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:06.946+0000 7f6cef664200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:13:07.115 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:07.112+0000 7f6cef664200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:13:07.494 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:07.244+0000 7f6cef664200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:13:07.590 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:13:07.590 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-14T11:13:07.590 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T11:13:07.590 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T11:13:07.591 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T11:13:07.591 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:13:07.591 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.862+0000 7f6aa4c48700 1 Processor -- start 2023-07-14T11:13:07.591 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.863+0000 7f6aa4c48700 1 -- start start 2023-07-14T11:13:07.591 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.863+0000 7f6aa4c48700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:07.591 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.863+0000 7f6aa4c48700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa007a880 con 0x7f6aa007be40 2023-07-14T11:13:07.591 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.863+0000 7f6a9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.863+0000 7f6a9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49374/0 (socket says 172.21.15.161:49374) 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.863+0000 7f6a9e59c700 1 -- 172.21.15.161:0/3774215040 learned_addr learned my addr 172.21.15.161:0/3774215040 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.864+0000 7f6a9e59c700 1 -- 172.21.15.161:0/3774215040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa007a9c0 con 0x7f6aa007be40 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.864+0000 7f6a9e59c700 1 --2- 172.21.15.161:0/3774215040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa007a340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a900098a0 tx=0x7f6a90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d0ae17d76a5c08 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.864+0000 7f6a9d59a700 1 -- 172.21.15.161:0/3774215040 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a90004160 con 0x7f6aa007be40 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.864+0000 7f6a9d59a700 1 -- 172.21.15.161:0/3774215040 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a900042c0 con 0x7f6aa007be40 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.864+0000 7f6a9d59a700 1 -- 172.21.15.161:0/3774215040 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a900119b0 con 0x7f6aa007be40 2023-07-14T11:13:07.592 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.865+0000 7f6aa4c48700 1 -- 172.21.15.161:0/3774215040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 msgr2=0x7f6aa007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:07.593 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.865+0000 7f6aa4c48700 1 --2- 172.21.15.161:0/3774215040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa007a340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a900098a0 tx=0x7f6a90009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:07.593 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.865+0000 7f6aa4c48700 1 -- 172.21.15.161:0/3774215040 shutdown_connections 2023-07-14T11:13:07.593 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.865+0000 7f6aa4c48700 1 --2- 172.21.15.161:0/3774215040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa007a340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:07.593 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.865+0000 7f6aa4c48700 1 -- 172.21.15.161:0/3774215040 >> 172.21.15.161:0/3774215040 conn(0x7f6aa0100970 msgr2=0x7f6aa0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:07.593 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.865+0000 7f6aa4c48700 1 -- 172.21.15.161:0/3774215040 shutdown_connections 2023-07-14T11:13:07.593 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.865+0000 7f6aa4c48700 1 -- 172.21.15.161:0/3774215040 wait complete. 2023-07-14T11:13:07.593 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.866+0000 7f6aa4c48700 1 Processor -- start 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.866+0000 7f6aa4c48700 1 -- start start 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.866+0000 7f6aa4c48700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.866+0000 7f6aa4c48700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019b870 con 0x7f6aa007be40 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.867+0000 7f6a9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.867+0000 7f6a9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49386/0 (socket says 172.21.15.161:49386) 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.867+0000 7f6a9e59c700 1 -- 172.21.15.161:0/92992409 learned_addr learned my addr 172.21.15.161:0/92992409 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.867+0000 7f6a9e59c700 1 -- 172.21.15.161:0/92992409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a90009550 con 0x7f6aa007be40 2023-07-14T11:13:07.594 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.867+0000 7f6a9e59c700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa019b330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a90003e60 tx=0x7f6a90011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.867+0000 7f6a8f7fe700 1 -- 172.21.15.161:0/92992409 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a90003af0 con 0x7f6aa007be40 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.868+0000 7f6a8f7fe700 1 -- 172.21.15.161:0/92992409 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a90003c50 con 0x7f6aa007be40 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.868+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa019ba70 con 0x7f6aa007be40 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.868+0000 7f6a8f7fe700 1 -- 172.21.15.161:0/92992409 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a90019790 con 0x7f6aa007be40 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.868+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa019be90 con 0x7f6aa007be40 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.869+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa0195310 con 0x7f6aa007be40 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.869+0000 7f6a8f7fe700 1 -- 172.21.15.161:0/92992409 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f6a9001a430 con 0x7f6aa007be40 2023-07-14T11:13:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.870+0000 7f6a8f7fe700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 0x7f6a8803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:07.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.870+0000 7f6a8f7fe700 1 -- 172.21.15.161:0/92992409 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6a9004caa0 con 0x7f6aa007be40 2023-07-14T11:13:07.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.870+0000 7f6a9dd9b700 1 -- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 msgr2=0x7f6a8803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/3292405426 2023-07-14T11:13:07.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.870+0000 7f6a9dd9b700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 0x7f6a8803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:13:07.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:06.874+0000 7f6a8f7fe700 1 -- 172.21.15.161:0/92992409 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6a90010560 con 0x7f6aa007be40 2023-07-14T11:13:07.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.071+0000 7f6a9dd9b700 1 -- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 msgr2=0x7f6a8803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/3292405426 2023-07-14T11:13:07.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.071+0000 7f6a9dd9b700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 0x7f6a8803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:13:07.596 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.135+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6aa0061b40 con 0x7f6aa007be40 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.135+0000 7f6a8f7fe700 1 -- 172.21.15.161:0/92992409 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6a90010380 con 0x7f6aa007be40 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 msgr2=0x7f6a8803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 0x7f6a8803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 msgr2=0x7f6aa019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa019b330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a90003e60 tx=0x7f6a90011aa0 comp rx=0 tx=0).stop 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 shutdown_connections 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7f6a880387b0 0x7f6a8803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:07.597 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 --2- 172.21.15.161:0/92992409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa007be40 0x7f6aa019b330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:07.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 >> 172.21.15.161:0/92992409 conn(0x7f6aa0100970 msgr2=0x7f6aa0102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:07.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 shutdown_connections 2023-07-14T11:13:07.598 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:07.138+0000 7f6aa4c48700 1 -- 172.21.15.161:0/92992409 wait complete. 2023-07-14T11:13:07.598 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for the mgr to restart... 2023-07-14T11:13:07.598 INFO:teuthology.orchestra.run.smithi161.stdout:Waiting for mgr epoch 9... 2023-07-14T11:13:07.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:07 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/92992409' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T11:13:08.245 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:07.798+0000 7f6cef664200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:13:08.995 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:08.557+0000 7f6cef664200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:13:08.995 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:08.763+0000 7f6cef664200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:13:09.995 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:09.698+0000 7f6cef664200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:13:09.995 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:09.816+0000 7f6cef664200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:13:10.397 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:10.039+0000 7f6cef664200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:13:10.398 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:10.395+0000 7f6cef664200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:13:10.995 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:10.670+0000 7f6cef664200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:13:10.995 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:10.859+0000 7f6cef664200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:13:11.495 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:11 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:11.200+0000 7f6cef664200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:13:12.495 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:12 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:12.120+0000 7f6cef664200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:13:12.495 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:12 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:12.290+0000 7f6cef664200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:13:12.954 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:12 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:12.533+0000 7f6cef664200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:13:12.954 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:12 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:12.679+0000 7f6cef664200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:13:12.954 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:12 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:12.951+0000 7f6cef664200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:13:14.495 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:13:14.002+0000 7f6cef664200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:13:14.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: Active manager daemon x restarted 2023-07-14T11:13:14.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: Activating manager daemon x 2023-07-14T11:13:14.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: osdmap e3: 0 total, 0 up, 0 in 2023-07-14T11:13:14.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: mgrmap e10: x(active, starting, since 0.00418374s) 2023-07-14T11:13:14.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:13:14.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:13:14.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:13:14.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:13:14.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:13:14.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: Manager daemon x is now available 2023-07-14T11:13:14.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:14.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:15.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:13:15.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:13:15.470 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:15 smithi161 ceph-mon[95526]: mgrmap e11: x(active, since 1.00917s) 2023-07-14T11:13:15.495 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:13:15.495 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-14T11:13:15.496 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T11:13:15.496 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:13:15.496 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.164+0000 7fa7a936c700 1 Processor -- start 2023-07-14T11:13:15.496 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.164+0000 7fa7a936c700 1 -- start start 2023-07-14T11:13:15.496 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.165+0000 7fa7a936c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:15.496 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.165+0000 7fa7a936c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a407a840 con 0x7fa7a407be00 2023-07-14T11:13:15.496 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.165+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:15.497 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.165+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37764/0 (socket says 172.21.15.161:37764) 2023-07-14T11:13:15.497 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.165+0000 7fa7a2ffd700 1 -- 172.21.15.161:0/3785123969 learned_addr learned my addr 172.21.15.161:0/3785123969 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:15.497 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.165+0000 7fa7a2ffd700 1 -- 172.21.15.161:0/3785123969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a407a980 con 0x7fa7a407be00 2023-07-14T11:13:15.497 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.166+0000 7fa7a2ffd700 1 --2- 172.21.15.161:0/3785123969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a407a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7940098a0 tx=0x7fa794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2529a67b0e3bb9f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:15.497 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.166+0000 7fa7a1ffb700 1 -- 172.21.15.161:0/3785123969 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa794004160 con 0x7fa7a407be00 2023-07-14T11:13:15.497 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.166+0000 7fa7a1ffb700 1 -- 172.21.15.161:0/3785123969 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7940042c0 con 0x7fa7a407be00 2023-07-14T11:13:15.497 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.166+0000 7fa7a1ffb700 1 -- 172.21.15.161:0/3785123969 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7940119b0 con 0x7fa7a407be00 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.167+0000 7fa7a936c700 1 -- 172.21.15.161:0/3785123969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 msgr2=0x7fa7a407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.167+0000 7fa7a936c700 1 --2- 172.21.15.161:0/3785123969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a407a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7940098a0 tx=0x7fa794009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.167+0000 7fa7a936c700 1 -- 172.21.15.161:0/3785123969 shutdown_connections 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.167+0000 7fa7a936c700 1 --2- 172.21.15.161:0/3785123969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a407a300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.167+0000 7fa7a936c700 1 -- 172.21.15.161:0/3785123969 >> 172.21.15.161:0/3785123969 conn(0x7fa7a4100970 msgr2=0x7fa7a4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.167+0000 7fa7a936c700 1 -- 172.21.15.161:0/3785123969 shutdown_connections 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.167+0000 7fa7a936c700 1 -- 172.21.15.161:0/3785123969 wait complete. 2023-07-14T11:13:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.168+0000 7fa7a936c700 1 Processor -- start 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.168+0000 7fa7a936c700 1 -- start start 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.168+0000 7fa7a936c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.168+0000 7fa7a936c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a419b880 con 0x7fa7a407be00 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.169+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.169+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37774/0 (socket says 172.21.15.161:37774) 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.169+0000 7fa7a2ffd700 1 -- 172.21.15.161:0/1788272808 learned_addr learned my addr 172.21.15.161:0/1788272808 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.169+0000 7fa7a2ffd700 1 -- 172.21.15.161:0/1788272808 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794009550 con 0x7fa7a407be00 2023-07-14T11:13:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.169+0000 7fa7a2ffd700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a419b340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa794003e60 tx=0x7fa794011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.170+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa794003a60 con 0x7fa7a407be00 2023-07-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.170+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a419ba80 con 0x7fa7a407be00 2023-07-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.170+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa794003bc0 con 0x7fa7a407be00 2023-07-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.170+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7940196c0 con 0x7fa7a407be00 2023-07-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.170+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a419bea0 con 0x7fa7a407be00 2023-07-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.171+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa79401a430 con 0x7fa7a407be00 2023-07-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.172+0000 7fa78bfff700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.172+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 --> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa78c03b300 con 0x7fa78c038740 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.172+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa79404c980 con 0x7fa7a407be00 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.172+0000 7fa7a27fc700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 msgr2=0x7fa78c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/3292405426 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.172+0000 7fa7a27fc700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.373+0000 7fa7a27fc700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 msgr2=0x7fa78c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/3292405426 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.373+0000 7fa7a27fc700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.774+0000 7fa7a27fc700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 msgr2=0x7fa78c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/3292405426 2023-07-14T11:13:15.501 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:08.774+0000 7fa7a27fc700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T11:13:15.502 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:09.575+0000 7fa7a27fc700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 msgr2=0x7fa78c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/3292405426 2023-07-14T11:13:15.502 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:09.575+0000 7fa7a27fc700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T11:13:15.502 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:11.176+0000 7fa7a27fc700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 msgr2=0x7fa78c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/3292405426 2023-07-14T11:13:15.502 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:11.177+0000 7fa7a27fc700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T11:13:15.502 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:14.007+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fa79401a780 con 0x7fa7a407be00 2023-07-14T11:13:15.502 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:14.007+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 msgr2=0x7fa78c03abf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:13:15.502 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:14.007+0000 7fa78bfff700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.010+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fa79404e4f0 con 0x7fa7a407be00 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.010+0000 7fa78bfff700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa78c03bfd0 0x7fa78c03e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.010+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa78c03b300 con 0x7fa78c03bfd0 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.012+0000 7fa7a27fc700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa78c03bfd0 0x7fa78c03e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.014+0000 7fa7a27fc700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa78c03bfd0 0x7fa78c03e3b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa798003a10 tx=0x7fa7980092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.014+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fa78c03b300 con 0x7fa78c03bfd0 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.026+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa7a419c680 con 0x7fa78c03bfd0 2023-07-14T11:13:15.503 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa78bfff700 1 -- 172.21.15.161:0/1788272808 <== mgr.14148 v2:172.21.15.161:6800/2004305703 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa7a419c680 con 0x7fa78c03bfd0 2023-07-14T11:13:15.504 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa78c03bfd0 msgr2=0x7fa78c03e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:15.504 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa78c03bfd0 0x7fa78c03e3b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa798003a10 tx=0x7fa7980092b0 comp rx=0 tx=0).stop 2023-07-14T11:13:15.504 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 msgr2=0x7fa7a419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:15.504 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a419b340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa794003e60 tx=0x7fa794011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:15.504 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 shutdown_connections 2023-07-14T11:13:15.504 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa78c03bfd0 0x7fa78c03e3b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a407be00 0x7fa7a419b340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 --2- 172.21.15.161:0/1788272808 >> [v2:172.21.15.161:6800/3292405426,v1:172.21.15.161:6801/3292405426] conn(0x7fa78c038740 0x7fa78c03abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 >> 172.21.15.161:0/1788272808 conn(0x7fa7a4100970 msgr2=0x7fa7a41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.027+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 shutdown_connections 2023-07-14T11:13:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:15.028+0000 7fa7a936c700 1 -- 172.21.15.161:0/1788272808 wait complete. 2023-07-14T11:13:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:mgr epoch 9 is available 2023-07-14T11:13:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:Generating a dashboard self-signed certificate... 2023-07-14T11:13:16.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:16 smithi161 ceph-mon[95526]: [14/Jul/2023:11:13:15] ENGINE Bus STARTING 2023-07-14T11:13:16.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:16 smithi161 ceph-mon[95526]: [14/Jul/2023:11:13:15] ENGINE Serving on http://172.21.15.161:8765 2023-07-14T11:13:16.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:16 smithi161 ceph-mon[95526]: [14/Jul/2023:11:13:15] ENGINE Serving on https://172.21.15.161:7150 2023-07-14T11:13:16.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:16 smithi161 ceph-mon[95526]: [14/Jul/2023:11:13:15] ENGINE Bus STARTED 2023-07-14T11:13:16.879 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-14T11:13:16.879 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.168+0000 7f4728e5c700 1 Processor -- start 2023-07-14T11:13:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.168+0000 7f4728e5c700 1 -- start start 2023-07-14T11:13:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.168+0000 7f4728e5c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f4724105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.168+0000 7f4728e5c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724105c70 con 0x7f4724105320 2023-07-14T11:13:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.169+0000 7f472259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f4724105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.169+0000 7f472259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f4724105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37876/0 (socket says 172.21.15.161:37876) 2023-07-14T11:13:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.169+0000 7f472259c700 1 -- 172.21.15.161:0/714836126 learned_addr learned my addr 172.21.15.161:0/714836126 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.169+0000 7f472259c700 1 -- 172.21.15.161:0/714836126 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724105db0 con 0x7f4724105320 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.170+0000 7f472259c700 1 --2- 172.21.15.161:0/714836126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f4724105730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f47140098a0 tx=0x7f4714009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a487c61e89a39646 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.170+0000 7f472159a700 1 -- 172.21.15.161:0/714836126 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4714004160 con 0x7f4724105320 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.170+0000 7f472159a700 1 -- 172.21.15.161:0/714836126 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47140042c0 con 0x7f4724105320 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.170+0000 7f472159a700 1 -- 172.21.15.161:0/714836126 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47140119b0 con 0x7f4724105320 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.170+0000 7f4728e5c700 1 -- 172.21.15.161:0/714836126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 msgr2=0x7f4724105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.170+0000 7f4728e5c700 1 --2- 172.21.15.161:0/714836126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f4724105730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f47140098a0 tx=0x7f4714009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.171+0000 7f4728e5c700 1 -- 172.21.15.161:0/714836126 shutdown_connections 2023-07-14T11:13:16.881 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.171+0000 7f4728e5c700 1 --2- 172.21.15.161:0/714836126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f4724105730 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:16.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.171+0000 7f4728e5c700 1 -- 172.21.15.161:0/714836126 >> 172.21.15.161:0/714836126 conn(0x7f4724100970 msgr2=0x7f4724102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:16.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.171+0000 7f4728e5c700 1 -- 172.21.15.161:0/714836126 shutdown_connections 2023-07-14T11:13:16.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.171+0000 7f4728e5c700 1 -- 172.21.15.161:0/714836126 wait complete. 2023-07-14T11:13:16.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.172+0000 7f4728e5c700 1 Processor -- start 2023-07-14T11:13:16.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.172+0000 7f4728e5c700 1 -- start start 2023-07-14T11:13:16.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.173+0000 7f4728e5c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f47241990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:16.882 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.173+0000 7f4728e5c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724199620 con 0x7f4724105320 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.173+0000 7f472259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f47241990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.173+0000 7f472259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f47241990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37892/0 (socket says 172.21.15.161:37892) 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.173+0000 7f472259c700 1 -- 172.21.15.161:0/3799429950 learned_addr learned my addr 172.21.15.161:0/3799429950 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.173+0000 7f472259c700 1 -- 172.21.15.161:0/3799429950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714009550 con 0x7f4724105320 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.174+0000 7f472259c700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f47241990e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4714003f00 tx=0x7f4714011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.174+0000 7f47137fe700 1 -- 172.21.15.161:0/3799429950 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4714003b20 con 0x7f4724105320 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.174+0000 7f47137fe700 1 -- 172.21.15.161:0/3799429950 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4714003c80 con 0x7f4724105320 2023-07-14T11:13:16.883 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.174+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4724199820 con 0x7f4724105320 2023-07-14T11:13:16.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.174+0000 7f47137fe700 1 -- 172.21.15.161:0/3799429950 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4714019770 con 0x7f4724105320 2023-07-14T11:13:16.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.174+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4724199c40 con 0x7f4724105320 2023-07-14T11:13:16.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.175+0000 7f47137fe700 1 -- 172.21.15.161:0/3799429950 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f47140199e0 con 0x7f4724105320 2023-07-14T11:13:16.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.175+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47241097b0 con 0x7f4724105320 2023-07-14T11:13:16.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.176+0000 7f47137fe700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f470c0387b0 0x7f470c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:16.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.176+0000 7f47137fe700 1 -- 172.21.15.161:0/3799429950 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f471404caf0 con 0x7f4724105320 2023-07-14T11:13:16.884 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.176+0000 7f4721d9b700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f470c0387b0 0x7f470c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.176+0000 7f4721d9b700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f470c0387b0 0x7f470c03ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.185+0000 7f47137fe700 1 -- 172.21.15.161:0/3799429950 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4714018db0 con 0x7f4724105320 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.409+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f4724102ce0 con 0x7f470c0387b0 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.470+0000 7f47137fe700 1 -- 172.21.15.161:0/3799429950 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f4724102ce0 con 0x7f470c0387b0 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f470c0387b0 msgr2=0x7f470c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f470c0387b0 0x7f470c03ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 msgr2=0x7f47241990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:16.885 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f47241990e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4714003f00 tx=0x7f4714011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:16.886 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 shutdown_connections 2023-07-14T11:13:16.886 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f470c0387b0 0x7f470c03ac60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:16.886 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 --2- 172.21.15.161:0/3799429950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4724105320 0x7f47241990e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:16.886 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 >> 172.21.15.161:0/3799429950 conn(0x7f4724100970 msgr2=0x7f47241020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:16.886 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 shutdown_connections 2023-07-14T11:13:16.886 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:16.472+0000 7f4728e5c700 1 -- 172.21.15.161:0/3799429950 wait complete. 2023-07-14T11:13:16.886 INFO:teuthology.orchestra.run.smithi161.stdout:Creating initial admin user... 2023-07-14T11:13:17.141 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:17 smithi161 ceph-mon[95526]: mgrmap e12: x(active, since 2s) 2023-07-14T11:13:17.142 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:17.142 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:17 smithi161 ceph-mon[95526]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:17.142 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:17.142 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:18.257 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$NAUmsX318HPP0rLepxeYNORloF2bleQcYuQMnHl2AxC1Qk/BnR2Ke", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689333197, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-14T11:13:18.258 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.405+0000 7f7836dfd700 1 Processor -- start 2023-07-14T11:13:18.258 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.405+0000 7f7836dfd700 1 -- start start 2023-07-14T11:13:18.258 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.406+0000 7f7836dfd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f7830106040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:18.258 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.406+0000 7f7836dfd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7830079da0 con 0x7f7830103c20 2023-07-14T11:13:18.258 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.406+0000 7f7834b99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f7830106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:18.258 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.406+0000 7f7834b99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f7830106040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37896/0 (socket says 172.21.15.161:37896) 2023-07-14T11:13:18.258 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.406+0000 7f7834b99700 1 -- 172.21.15.161:0/3757527024 learned_addr learned my addr 172.21.15.161:0/3757527024 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:18.262 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.406+0000 7f7834b99700 1 -- 172.21.15.161:0/3757527024 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7830079ee0 con 0x7f7830103c20 2023-07-14T11:13:18.262 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.407+0000 7f7834b99700 1 --2- 172.21.15.161:0/3757527024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f7830106040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78240098a0 tx=0x7f7824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46409ccf5ef026a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:18.262 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.407+0000 7f782f7fe700 1 -- 172.21.15.161:0/3757527024 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7824004160 con 0x7f7830103c20 2023-07-14T11:13:18.262 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.407+0000 7f782f7fe700 1 -- 172.21.15.161:0/3757527024 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78240042c0 con 0x7f7830103c20 2023-07-14T11:13:18.262 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.407+0000 7f782f7fe700 1 -- 172.21.15.161:0/3757527024 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78240119b0 con 0x7f7830103c20 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.408+0000 7f7836dfd700 1 -- 172.21.15.161:0/3757527024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 msgr2=0x7f7830106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.408+0000 7f7836dfd700 1 --2- 172.21.15.161:0/3757527024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f7830106040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78240098a0 tx=0x7f7824009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.408+0000 7f7836dfd700 1 -- 172.21.15.161:0/3757527024 shutdown_connections 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.408+0000 7f7836dfd700 1 --2- 172.21.15.161:0/3757527024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f7830106040 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.408+0000 7f7836dfd700 1 -- 172.21.15.161:0/3757527024 >> 172.21.15.161:0/3757527024 conn(0x7f78300ff890 msgr2=0x7f7830101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.408+0000 7f7836dfd700 1 -- 172.21.15.161:0/3757527024 shutdown_connections 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.408+0000 7f7836dfd700 1 -- 172.21.15.161:0/3757527024 wait complete. 2023-07-14T11:13:18.263 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.409+0000 7f7836dfd700 1 Processor -- start 2023-07-14T11:13:18.264 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.409+0000 7f7836dfd700 1 -- start start 2023-07-14T11:13:18.264 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.410+0000 7f7836dfd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f783019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:18.264 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.410+0000 7f7836dfd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783019fc80 con 0x7f7830103c20 2023-07-14T11:13:18.264 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.410+0000 7f7834b99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f783019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:18.264 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.410+0000 7f7834b99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f783019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37912/0 (socket says 172.21.15.161:37912) 2023-07-14T11:13:18.264 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.410+0000 7f7834b99700 1 -- 172.21.15.161:0/2918963690 learned_addr learned my addr 172.21.15.161:0/2918963690 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:18.264 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.410+0000 7f7834b99700 1 -- 172.21.15.161:0/2918963690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824009550 con 0x7f7830103c20 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.410+0000 7f7834b99700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f783019f740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7824003e60 tx=0x7f7824011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.411+0000 7f782dffb700 1 -- 172.21.15.161:0/2918963690 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7824003a60 con 0x7f7830103c20 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.411+0000 7f782dffb700 1 -- 172.21.15.161:0/2918963690 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7824003bc0 con 0x7f7830103c20 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.411+0000 7f782dffb700 1 -- 172.21.15.161:0/2918963690 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78240196c0 con 0x7f7830103c20 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.411+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783019fe80 con 0x7f7830103c20 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.411+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78301a02a0 con 0x7f7830103c20 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.412+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7830061b40 con 0x7f7830103c20 2023-07-14T11:13:18.265 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.412+0000 7f782dffb700 1 -- 172.21.15.161:0/2918963690 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f782401ab60 con 0x7f7830103c20 2023-07-14T11:13:18.266 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.413+0000 7f782dffb700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7820038790 0x7f782003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:18.266 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.413+0000 7f782dffb700 1 -- 172.21.15.161:0/2918963690 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f782404ca10 con 0x7f7830103c20 2023-07-14T11:13:18.266 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.413+0000 7f782ffff700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7820038790 0x7f782003ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:18.266 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.414+0000 7f782ffff700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7820038790 0x7f782003ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f781c006fd0 tx=0x7f781c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:18.266 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.423+0000 7f782dffb700 1 -- 172.21.15.161:0/2918963690 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7824018b00 con 0x7f7830103c20 2023-07-14T11:13:18.266 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.643+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7830061960 con 0x7f7820038790 2023-07-14T11:13:18.266 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.875+0000 7f782dffb700 1 -- 172.21.15.161:0/2918963690 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f7830061960 con 0x7f7820038790 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7820038790 msgr2=0x7f782003ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7820038790 0x7f782003ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f781c006fd0 tx=0x7f781c006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 msgr2=0x7f783019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f783019f740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7824003e60 tx=0x7f7824011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 shutdown_connections 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7820038790 0x7f782003ac40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 --2- 172.21.15.161:0/2918963690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7830103c20 0x7f783019f740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:18.267 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.878+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 >> 172.21.15.161:0/2918963690 conn(0x7f78300ff890 msgr2=0x7f7830101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:18.268 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.879+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 shutdown_connections 2023-07-14T11:13:18.268 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:17.879+0000 7f7836dfd700 1 -- 172.21.15.161:0/2918963690 wait complete. 2023-07-14T11:13:18.268 INFO:teuthology.orchestra.run.smithi161.stdout:Fetching dashboard port number... 2023-07-14T11:13:18.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:18 smithi161 ceph-mon[95526]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:18.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:18 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:18.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:18 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:19.746 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stdout 8443 2023-07-14T11:13:19.746 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.802+0000 7f1339953700 1 Processor -- start 2023-07-14T11:13:19.746 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.802+0000 7f1339953700 1 -- start start 2023-07-14T11:13:19.747 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.802+0000 7f1339953700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:19.747 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.802+0000 7f1339953700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133407a840 con 0x7f133407be00 2023-07-14T11:13:19.747 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.803+0000 7f1332ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:19.747 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.803+0000 7f1332ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33288/0 (socket says 172.21.15.161:33288) 2023-07-14T11:13:19.747 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.803+0000 7f1332ffd700 1 -- 172.21.15.161:0/2523623934 learned_addr learned my addr 172.21.15.161:0/2523623934 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:19.747 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.803+0000 7f1332ffd700 1 -- 172.21.15.161:0/2523623934 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133407a980 con 0x7f133407be00 2023-07-14T11:13:19.747 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.803+0000 7f1332ffd700 1 --2- 172.21.15.161:0/2523623934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133407a300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13240098a0 tx=0x7f1324009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86acf8e57196995 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1331ffb700 1 -- 172.21.15.161:0/2523623934 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1324004160 con 0x7f133407be00 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1331ffb700 1 -- 172.21.15.161:0/2523623934 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13240042c0 con 0x7f133407be00 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1331ffb700 1 -- 172.21.15.161:0/2523623934 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13240119b0 con 0x7f133407be00 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1339953700 1 -- 172.21.15.161:0/2523623934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 msgr2=0x7f133407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1339953700 1 --2- 172.21.15.161:0/2523623934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133407a300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13240098a0 tx=0x7f1324009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1339953700 1 -- 172.21.15.161:0/2523623934 shutdown_connections 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1339953700 1 --2- 172.21.15.161:0/2523623934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133407a300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.804+0000 7f1339953700 1 -- 172.21.15.161:0/2523623934 >> 172.21.15.161:0/2523623934 conn(0x7f1334100970 msgr2=0x7f1334102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.805+0000 7f1339953700 1 -- 172.21.15.161:0/2523623934 shutdown_connections 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.805+0000 7f1339953700 1 -- 172.21.15.161:0/2523623934 wait complete. 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.806+0000 7f1339953700 1 Processor -- start 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.806+0000 7f1339953700 1 -- start start 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.806+0000 7f1339953700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.806+0000 7f1339953700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419b880 con 0x7f133407be00 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.807+0000 7f1332ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:19.749 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.807+0000 7f1332ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33292/0 (socket says 172.21.15.161:33292) 2023-07-14T11:13:19.750 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.807+0000 7f1332ffd700 1 -- 172.21.15.161:0/2658491620 learned_addr learned my addr 172.21.15.161:0/2658491620 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:19.750 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.807+0000 7f1332ffd700 1 -- 172.21.15.161:0/2658491620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324009550 con 0x7f133407be00 2023-07-14T11:13:19.750 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.807+0000 7f1332ffd700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133419b340 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1324003e60 tx=0x7f1324011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:19.750 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.808+0000 7f1338951700 1 -- 172.21.15.161:0/2658491620 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1324003a60 con 0x7f133407be00 2023-07-14T11:13:19.750 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.808+0000 7f1338951700 1 -- 172.21.15.161:0/2658491620 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1324003bc0 con 0x7f133407be00 2023-07-14T11:13:19.750 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.808+0000 7f1338951700 1 -- 172.21.15.161:0/2658491620 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13240196c0 con 0x7f133407be00 2023-07-14T11:13:19.750 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.808+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133419ba80 con 0x7f133407be00 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.808+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133419bea0 con 0x7f133407be00 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.809+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133410b6d0 con 0x7f133407be00 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.809+0000 7f1338951700 1 -- 172.21.15.161:0/2658491620 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f132401ab60 con 0x7f133407be00 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.809+0000 7f1338951700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f131c038790 0x7f131c03ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.809+0000 7f1338951700 1 -- 172.21.15.161:0/2658491620 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f132404ca50 con 0x7f133407be00 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.810+0000 7f13327fc700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f131c038790 0x7f131c03ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.810+0000 7f13327fc700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f131c038790 0x7f131c03ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1328006fd0 tx=0x7f1328006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:18.814+0000 7f1338951700 1 -- 172.21.15.161:0/2658491620 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1324019930 con 0x7f133407be00 2023-07-14T11:13:19.752 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.022+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f133410b8c0 con 0x7f133407be00 2023-07-14T11:13:19.752 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.023+0000 7f1338951700 1 -- 172.21.15.161:0/2658491620 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1324018350 con 0x7f133407be00 2023-07-14T11:13:19.752 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f131c038790 msgr2=0x7f131c03ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:19.752 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f131c038790 0x7f131c03ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1328006fd0 tx=0x7f1328006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:19.752 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 msgr2=0x7f133419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:19.752 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133419b340 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1324003e60 tx=0x7f1324011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:19.752 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 shutdown_connections 2023-07-14T11:13:19.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f131c038790 0x7f131c03ac40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:19.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 --2- 172.21.15.161:0/2658491620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133407be00 0x7f133419b340 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:19.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 >> 172.21.15.161:0/2658491620 conn(0x7f1334100970 msgr2=0x7f13341020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:19.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 shutdown_connections 2023-07-14T11:13:19.753 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:19.025+0000 7f1339953700 1 -- 172.21.15.161:0/2658491620 wait complete. 2023-07-14T11:13:19.765 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:13:19.765 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout disabled 2023-07-14T11:13:19.781 INFO:teuthology.orchestra.run.smithi161.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:13:19.781 INFO:teuthology.orchestra.run.smithi161.stdout:systemctl: stdout inactive 2023-07-14T11:13:19.781 INFO:teuthology.orchestra.run.smithi161.stdout:firewalld.service is not enabled 2023-07-14T11:13:19.781 INFO:teuthology.orchestra.run.smithi161.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-14T11:13:19.783 INFO:teuthology.orchestra.run.smithi161.stdout:Ceph Dashboard is now available at: 2023-07-14T11:13:19.783 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:19.783 INFO:teuthology.orchestra.run.smithi161.stdout: URL: https://smithi161.front.sepia.ceph.com:8443/ 2023-07-14T11:13:19.784 INFO:teuthology.orchestra.run.smithi161.stdout: User: admin 2023-07-14T11:13:19.784 INFO:teuthology.orchestra.run.smithi161.stdout: Password: 6kxxmghn6i 2023-07-14T11:13:19.784 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:19.784 INFO:teuthology.orchestra.run.smithi161.stdout:Saving cluster configuration to /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config directory 2023-07-14T11:13:19.785 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling autotune for osd_memory_target 2023-07-14T11:13:19.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:19 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2658491620' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-14T11:13:20.965 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.329+0000 7f93a4d82700 1 Processor -- start 2023-07-14T11:13:20.965 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.329+0000 7f93a4d82700 1 -- start start 2023-07-14T11:13:20.966 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.329+0000 7f93a4d82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:20.966 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.329+0000 7f93a4d82700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a007a840 con 0x7f93a007be00 2023-07-14T11:13:20.966 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.329+0000 7f939e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:20.966 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.329+0000 7f939e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33304/0 (socket says 172.21.15.161:33304) 2023-07-14T11:13:20.966 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.329+0000 7f939e59c700 1 -- 172.21.15.161:0/1958797025 learned_addr learned my addr 172.21.15.161:0/1958797025 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:20.966 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.330+0000 7f939e59c700 1 -- 172.21.15.161:0/1958797025 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a007a980 con 0x7f93a007be00 2023-07-14T11:13:20.966 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.330+0000 7f939e59c700 1 --2- 172.21.15.161:0/1958797025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a007a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93900098a0 tx=0x7f9390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5407312ef840a160 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:20.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.330+0000 7f939d59a700 1 -- 172.21.15.161:0/1958797025 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9390004160 con 0x7f93a007be00 2023-07-14T11:13:20.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.330+0000 7f939d59a700 1 -- 172.21.15.161:0/1958797025 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93900042c0 con 0x7f93a007be00 2023-07-14T11:13:20.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.330+0000 7f939d59a700 1 -- 172.21.15.161:0/1958797025 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93900119b0 con 0x7f93a007be00 2023-07-14T11:13:20.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.331+0000 7f93a4d82700 1 -- 172.21.15.161:0/1958797025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 msgr2=0x7f93a007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:20.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.331+0000 7f93a4d82700 1 --2- 172.21.15.161:0/1958797025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a007a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93900098a0 tx=0x7f9390009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:20.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.331+0000 7f93a4d82700 1 -- 172.21.15.161:0/1958797025 shutdown_connections 2023-07-14T11:13:20.967 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.331+0000 7f93a4d82700 1 --2- 172.21.15.161:0/1958797025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a007a300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.331+0000 7f93a4d82700 1 -- 172.21.15.161:0/1958797025 >> 172.21.15.161:0/1958797025 conn(0x7f93a0100970 msgr2=0x7f93a0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.331+0000 7f93a4d82700 1 -- 172.21.15.161:0/1958797025 shutdown_connections 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.331+0000 7f93a4d82700 1 -- 172.21.15.161:0/1958797025 wait complete. 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.332+0000 7f93a4d82700 1 Processor -- start 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.332+0000 7f93a4d82700 1 -- start start 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.332+0000 7f93a4d82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.332+0000 7f93a4d82700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a019b880 con 0x7f93a007be00 2023-07-14T11:13:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.333+0000 7f939e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.333+0000 7f939e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33308/0 (socket says 172.21.15.161:33308) 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.333+0000 7f939e59c700 1 -- 172.21.15.161:0/3985123474 learned_addr learned my addr 172.21.15.161:0/3985123474 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.333+0000 7f939e59c700 1 -- 172.21.15.161:0/3985123474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9390009550 con 0x7f93a007be00 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.333+0000 7f939e59c700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a019b340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9390003e60 tx=0x7f9390011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.333+0000 7f938f7fe700 1 -- 172.21.15.161:0/3985123474 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9390003a60 con 0x7f93a007be00 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.334+0000 7f938f7fe700 1 -- 172.21.15.161:0/3985123474 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9390003bc0 con 0x7f93a007be00 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.334+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a019ba80 con 0x7f93a007be00 2023-07-14T11:13:20.969 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.334+0000 7f938f7fe700 1 -- 172.21.15.161:0/3985123474 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93900196c0 con 0x7f93a007be00 2023-07-14T11:13:20.970 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.334+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a019bea0 con 0x7f93a007be00 2023-07-14T11:13:20.970 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.335+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0061b40 con 0x7f93a007be00 2023-07-14T11:13:20.970 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.335+0000 7f938f7fe700 1 -- 172.21.15.161:0/3985123474 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f939001ab60 con 0x7f93a007be00 2023-07-14T11:13:20.970 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.335+0000 7f938f7fe700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9388038790 0x7f938803ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:20.970 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.335+0000 7f938f7fe700 1 -- 172.21.15.161:0/3985123474 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f939004cae0 con 0x7f93a007be00 2023-07-14T11:13:20.970 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.335+0000 7f939dd9b700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9388038790 0x7f938803ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:20.970 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.336+0000 7f939dd9b700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9388038790 0x7f938803ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9394006fd0 tx=0x7f9394006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.340+0000 7f938f7fe700 1 -- 172.21.15.161:0/3985123474 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9390018b00 con 0x7f93a007be00 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.533+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f93a019e8e0 con 0x7f93a007be00 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.534+0000 7f938f7fe700 1 -- 172.21.15.161:0/3985123474 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f9390018b00 con 0x7f93a007be00 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9388038790 msgr2=0x7f938803ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9388038790 0x7f938803ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9394006fd0 tx=0x7f9394006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 msgr2=0x7f93a019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a019b340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9390003e60 tx=0x7f9390011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:20.971 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 shutdown_connections 2023-07-14T11:13:20.972 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9388038790 0x7f938803ac40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:20.972 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 --2- 172.21.15.161:0/3985123474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a007be00 0x7f93a019b340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:20.972 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.536+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 >> 172.21.15.161:0/3985123474 conn(0x7f93a0100970 msgr2=0x7f93a01020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:20.972 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.537+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 shutdown_connections 2023-07-14T11:13:20.972 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:20.537+0000 7f93a4d82700 1 -- 172.21.15.161:0/3985123474 wait complete. 2023-07-14T11:13:21.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:21 smithi161 ceph-mon[95526]: mgrmap e13: x(active, since 6s) 2023-07-14T11:13:22.242 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.461+0000 7f377a592700 1 Processor -- start 2023-07-14T11:13:22.242 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.461+0000 7f377a592700 1 -- start start 2023-07-14T11:13:22.242 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.462+0000 7f377a592700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:22.243 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.462+0000 7f377a592700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377407a860 con 0x7f377407be20 2023-07-14T11:13:22.243 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.462+0000 7f3773fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:22.243 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.462+0000 7f3773fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33310/0 (socket says 172.21.15.161:33310) 2023-07-14T11:13:22.243 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.462+0000 7f3773fff700 1 -- 172.21.15.161:0/2608983156 learned_addr learned my addr 172.21.15.161:0/2608983156 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:22.246 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.462+0000 7f3773fff700 1 -- 172.21.15.161:0/2608983156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377407a9a0 con 0x7f377407be20 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f3773fff700 1 --2- 172.21.15.161:0/2608983156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377407a320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37640098a0 tx=0x7f3764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0eb6522f8cf9c51 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f3772ffd700 1 -- 172.21.15.161:0/2608983156 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3764004160 con 0x7f377407be20 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f3772ffd700 1 -- 172.21.15.161:0/2608983156 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37640042c0 con 0x7f377407be20 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f3772ffd700 1 -- 172.21.15.161:0/2608983156 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37640119b0 con 0x7f377407be20 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f377a592700 1 -- 172.21.15.161:0/2608983156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 msgr2=0x7f377407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f377a592700 1 --2- 172.21.15.161:0/2608983156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377407a320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37640098a0 tx=0x7f3764009bb0 comp rx=0 tx=0).stop 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f377a592700 1 -- 172.21.15.161:0/2608983156 shutdown_connections 2023-07-14T11:13:22.247 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f377a592700 1 --2- 172.21.15.161:0/2608983156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377407a320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:22.248 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.463+0000 7f377a592700 1 -- 172.21.15.161:0/2608983156 >> 172.21.15.161:0/2608983156 conn(0x7f3774100970 msgr2=0x7f3774102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:22.248 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.464+0000 7f377a592700 1 -- 172.21.15.161:0/2608983156 shutdown_connections 2023-07-14T11:13:22.248 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.464+0000 7f377a592700 1 -- 172.21.15.161:0/2608983156 wait complete. 2023-07-14T11:13:22.248 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.465+0000 7f377a592700 1 Processor -- start 2023-07-14T11:13:22.248 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.465+0000 7f377a592700 1 -- start start 2023-07-14T11:13:22.248 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.466+0000 7f377a592700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:22.248 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.466+0000 7f377a592700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419b890 con 0x7f377407be20 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.466+0000 7f3773fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.466+0000 7f3773fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33312/0 (socket says 172.21.15.161:33312) 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.466+0000 7f3773fff700 1 -- 172.21.15.161:0/1839781045 learned_addr learned my addr 172.21.15.161:0/1839781045 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.466+0000 7f3773fff700 1 -- 172.21.15.161:0/1839781045 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764009550 con 0x7f377407be20 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.466+0000 7f3773fff700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377419b350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3764000c00 tx=0x7f3764011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.467+0000 7f37717fa700 1 -- 172.21.15.161:0/1839781045 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3764003aa0 con 0x7f377407be20 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.467+0000 7f37717fa700 1 -- 172.21.15.161:0/1839781045 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3764003c00 con 0x7f377407be20 2023-07-14T11:13:22.249 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.467+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377419ba90 con 0x7f377407be20 2023-07-14T11:13:22.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.467+0000 7f37717fa700 1 -- 172.21.15.161:0/1839781045 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3764024800 con 0x7f377407be20 2023-07-14T11:13:22.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.467+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377419beb0 con 0x7f377407be20 2023-07-14T11:13:22.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.468+0000 7f37717fa700 1 -- 172.21.15.161:0/1839781045 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f376401ab60 con 0x7f377407be20 2023-07-14T11:13:22.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.468+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3774195330 con 0x7f377407be20 2023-07-14T11:13:22.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.468+0000 7f37717fa700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f375c038850 0x7f375c03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:22.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.468+0000 7f37717fa700 1 -- 172.21.15.161:0/1839781045 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f376404c9d0 con 0x7f377407be20 2023-07-14T11:13:22.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.469+0000 7f37737fe700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f375c038850 0x7f375c03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.469+0000 7f37737fe700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f375c038850 0x7f375c03ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3768006fd0 tx=0x7f3768006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.473+0000 7f37717fa700 1 -- 172.21.15.161:0/1839781045 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3764018530 con 0x7f377407be20 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.777+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f377402cce0 con 0x7f377407be20 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.780+0000 7f37717fa700 1 -- 172.21.15.161:0/1839781045 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3764018350 con 0x7f377407be20 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.782+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f375c038850 msgr2=0x7f375c03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.782+0000 7f377a592700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f375c038850 0x7f375c03ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3768006fd0 tx=0x7f3768006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.782+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 msgr2=0x7f377419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:22.251 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.782+0000 7f377a592700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377419b350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3764000c00 tx=0x7f3764011a30 comp rx=0 tx=0).stop 2023-07-14T11:13:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.783+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 shutdown_connections 2023-07-14T11:13:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.783+0000 7f377a592700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f375c038850 0x7f375c03ad00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.783+0000 7f377a592700 1 --2- 172.21.15.161:0/1839781045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377407be20 0x7f377419b350 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.783+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 >> 172.21.15.161:0/1839781045 conn(0x7f3774100970 msgr2=0x7f3774102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.783+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 shutdown_connections 2023-07-14T11:13:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr 2023-07-14T11:13:21.783+0000 7f377a592700 1 -- 172.21.15.161:0/1839781045 wait complete. 2023-07-14T11:13:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-14T11:13:22.253 INFO:teuthology.orchestra.run.smithi161.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-14T11:13:22.253 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.253 INFO:teuthology.orchestra.run.smithi161.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:22.253 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.253 INFO:teuthology.orchestra.run.smithi161.stdout:Or, if you are only running a single cluster on this host: 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout: ceph telemetry on 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.254 INFO:teuthology.orchestra.run.smithi161.stdout:For more information see: 2023-07-14T11:13:22.255 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.255 INFO:teuthology.orchestra.run.smithi161.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-14T11:13:22.255 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:22.255 INFO:teuthology.orchestra.run.smithi161.stdout:Bootstrap complete. 2023-07-14T11:13:22.307 INFO:tasks.cephadm:Fetching config... 2023-07-14T11:13:22.308 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:13:22.308 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-14T11:13:22.351 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-14T11:13:22.352 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:13:22.352 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-14T11:13:22.439 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-14T11:13:22.439 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:13:22.439 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-14T11:13:22.549 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-14T11:13:22.549 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:13:22.549 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-14T11:13:22.631 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-14T11:13:22.632 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe/8P46t5ZZ70sveVKxNngPuPw9t71vYYSPY+AoRmAewQJBczXIYMhm1IkeGuisfD7ShkQZGq2xlMwgkldCZ4f+e9lSUQBiJopEBPCQAhUOewN5HXlzdPepCHSRzB9o1OYH+wi7+i364LWU3JHCfoLkI1iWJ8eAK4/lsASn5bqlHxAgUYPAuIeXNY7TnF7VmajKjHO00+ApE5vPDsHxcBlD5mzQyc4chSa2MXdqZn7pfiAQqAFXOiXWmtvTeAmI5qUeouz1+6FcpprrzSUI0R3+4+UhcV6gemDqslhvIk11YxE1Cs1ekBi7AcjawneT8J/upEe9tUYb2u5D2Rwnd3fpSeY+OzBMkxprLfYMGIT0yz7H2XCtb0bd3Wm3dMz7x+4TSDLiqIgA6cMiS7rNyLYd9FHxBnz2cyOPqEDUk3w4P3xbsRUCWeKFZFWeL8pH6gocLn/dZsCeA+kUqdsw98j/UNyNnKOyaCGJ+mPUimgKGdxOtkNHcY9fDxo+Q0rizM= ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T11:13:22.773 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe/8P46t5ZZ70sveVKxNngPuPw9t71vYYSPY+AoRmAewQJBczXIYMhm1IkeGuisfD7ShkQZGq2xlMwgkldCZ4f+e9lSUQBiJopEBPCQAhUOewN5HXlzdPepCHSRzB9o1OYH+wi7+i364LWU3JHCfoLkI1iWJ8eAK4/lsASn5bqlHxAgUYPAuIeXNY7TnF7VmajKjHO00+ApE5vPDsHxcBlD5mzQyc4chSa2MXdqZn7pfiAQqAFXOiXWmtvTeAmI5qUeouz1+6FcpprrzSUI0R3+4+UhcV6gemDqslhvIk11YxE1Cs1ekBi7AcjawneT8J/upEe9tUYb2u5D2Rwnd3fpSeY+OzBMkxprLfYMGIT0yz7H2XCtb0bd3Wm3dMz7x+4TSDLiqIgA6cMiS7rNyLYd9FHxBnz2cyOPqEDUk3w4P3xbsRUCWeKFZFWeL8pH6gocLn/dZsCeA+kUqdsw98j/UNyNnKOyaCGJ+mPUimgKGdxOtkNHcY9fDxo+Q0rizM= ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:13:22.798 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:22 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1839781045' entity='client.admin' 2023-07-14T11:13:22.807 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe/8P46t5ZZ70sveVKxNngPuPw9t71vYYSPY+AoRmAewQJBczXIYMhm1IkeGuisfD7ShkQZGq2xlMwgkldCZ4f+e9lSUQBiJopEBPCQAhUOewN5HXlzdPepCHSRzB9o1OYH+wi7+i364LWU3JHCfoLkI1iWJ8eAK4/lsASn5bqlHxAgUYPAuIeXNY7TnF7VmajKjHO00+ApE5vPDsHxcBlD5mzQyc4chSa2MXdqZn7pfiAQqAFXOiXWmtvTeAmI5qUeouz1+6FcpprrzSUI0R3+4+UhcV6gemDqslhvIk11YxE1Cs1ekBi7AcjawneT8J/upEe9tUYb2u5D2Rwnd3fpSeY+OzBMkxprLfYMGIT0yz7H2XCtb0bd3Wm3dMz7x+4TSDLiqIgA6cMiS7rNyLYd9FHxBnz2cyOPqEDUk3w4P3xbsRUCWeKFZFWeL8pH6gocLn/dZsCeA+kUqdsw98j/UNyNnKOyaCGJ+mPUimgKGdxOtkNHcY9fDxo+Q0rizM= ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T11:13:22.851 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe/8P46t5ZZ70sveVKxNngPuPw9t71vYYSPY+AoRmAewQJBczXIYMhm1IkeGuisfD7ShkQZGq2xlMwgkldCZ4f+e9lSUQBiJopEBPCQAhUOewN5HXlzdPepCHSRzB9o1OYH+wi7+i364LWU3JHCfoLkI1iWJ8eAK4/lsASn5bqlHxAgUYPAuIeXNY7TnF7VmajKjHO00+ApE5vPDsHxcBlD5mzQyc4chSa2MXdqZn7pfiAQqAFXOiXWmtvTeAmI5qUeouz1+6FcpprrzSUI0R3+4+UhcV6gemDqslhvIk11YxE1Cs1ekBi7AcjawneT8J/upEe9tUYb2u5D2Rwnd3fpSeY+OzBMkxprLfYMGIT0yz7H2XCtb0bd3Wm3dMz7x+4TSDLiqIgA6cMiS7rNyLYd9FHxBnz2cyOPqEDUk3w4P3xbsRUCWeKFZFWeL8pH6gocLn/dZsCeA+kUqdsw98j/UNyNnKOyaCGJ+mPUimgKGdxOtkNHcY9fDxo+Q0rizM= ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:13:22.864 DEBUG:teuthology.orchestra.run.smithi182:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe/8P46t5ZZ70sveVKxNngPuPw9t71vYYSPY+AoRmAewQJBczXIYMhm1IkeGuisfD7ShkQZGq2xlMwgkldCZ4f+e9lSUQBiJopEBPCQAhUOewN5HXlzdPepCHSRzB9o1OYH+wi7+i364LWU3JHCfoLkI1iWJ8eAK4/lsASn5bqlHxAgUYPAuIeXNY7TnF7VmajKjHO00+ApE5vPDsHxcBlD5mzQyc4chSa2MXdqZn7pfiAQqAFXOiXWmtvTeAmI5qUeouz1+6FcpprrzSUI0R3+4+UhcV6gemDqslhvIk11YxE1Cs1ekBi7AcjawneT8J/upEe9tUYb2u5D2Rwnd3fpSeY+OzBMkxprLfYMGIT0yz7H2XCtb0bd3Wm3dMz7x+4TSDLiqIgA6cMiS7rNyLYd9FHxBnz2cyOPqEDUk3w4P3xbsRUCWeKFZFWeL8pH6gocLn/dZsCeA+kUqdsw98j/UNyNnKOyaCGJ+mPUimgKGdxOtkNHcY9fDxo+Q0rizM= ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T11:13:22.908 INFO:teuthology.orchestra.run.smithi182.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe/8P46t5ZZ70sveVKxNngPuPw9t71vYYSPY+AoRmAewQJBczXIYMhm1IkeGuisfD7ShkQZGq2xlMwgkldCZ4f+e9lSUQBiJopEBPCQAhUOewN5HXlzdPepCHSRzB9o1OYH+wi7+i364LWU3JHCfoLkI1iWJ8eAK4/lsASn5bqlHxAgUYPAuIeXNY7TnF7VmajKjHO00+ApE5vPDsHxcBlD5mzQyc4chSa2MXdqZn7pfiAQqAFXOiXWmtvTeAmI5qUeouz1+6FcpprrzSUI0R3+4+UhcV6gemDqslhvIk11YxE1Cs1ekBi7AcjawneT8J/upEe9tUYb2u5D2Rwnd3fpSeY+OzBMkxprLfYMGIT0yz7H2XCtb0bd3Wm3dMz7x+4TSDLiqIgA6cMiS7rNyLYd9FHxBnz2cyOPqEDUk3w4P3xbsRUCWeKFZFWeL8pH6gocLn/dZsCeA+kUqdsw98j/UNyNnKOyaCGJ+mPUimgKGdxOtkNHcY9fDxo+Q0rizM= ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:13:22.921 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-14T11:13:23.865 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:13:25.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.942+0000 7f5e6c6be700 1 -- 172.21.15.161:0/3935073546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 msgr2=0x7f5e640ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:25.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.942+0000 7f5e6c6be700 1 --2- 172.21.15.161:0/3935073546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e640ffa10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009910 tx=0x7f5e60009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:25.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.942+0000 7f5e6c6be700 1 -- 172.21.15.161:0/3935073546 shutdown_connections 2023-07-14T11:13:25.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.942+0000 7f5e6c6be700 1 --2- 172.21.15.161:0/3935073546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e640ffa10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:25.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.942+0000 7f5e6c6be700 1 -- 172.21.15.161:0/3935073546 >> 172.21.15.161:0/3935073546 conn(0x7f5e640fab90 msgr2=0x7f5e640fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:25.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.942+0000 7f5e6c6be700 1 -- 172.21.15.161:0/3935073546 shutdown_connections 2023-07-14T11:13:25.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.942+0000 7f5e6c6be700 1 -- 172.21.15.161:0/3935073546 wait complete. 2023-07-14T11:13:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.943+0000 7f5e6c6be700 1 Processor -- start 2023-07-14T11:13:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.944+0000 7f5e6c6be700 1 -- start start 2023-07-14T11:13:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.944+0000 7f5e6c6be700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e64194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.944+0000 7f5e6c6be700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64194c90 con 0x7f5e640ff600 2023-07-14T11:13:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.944+0000 7f5e6a45a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e64194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:25.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.944+0000 7f5e6a45a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e64194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33324/0 (socket says 172.21.15.161:33324) 2023-07-14T11:13:25.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.944+0000 7f5e6a45a700 1 -- 172.21.15.161:0/1066862898 learned_addr learned my addr 172.21.15.161:0/1066862898 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.944+0000 7f5e6a45a700 1 -- 172.21.15.161:0/1066862898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600095f0 con 0x7f5e640ff600 2023-07-14T11:13:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.945+0000 7f5e6a45a700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e64194750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e60004f40 tx=0x7f5e600057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.945+0000 7f5e5b7fe700 1 -- 172.21.15.161:0/1066862898 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e6001d070 con 0x7f5e640ff600 2023-07-14T11:13:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.945+0000 7f5e5b7fe700 1 -- 172.21.15.161:0/1066862898 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e6000bc10 con 0x7f5e640ff600 2023-07-14T11:13:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.945+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e64194e90 con 0x7f5e640ff600 2023-07-14T11:13:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.945+0000 7f5e5b7fe700 1 -- 172.21.15.161:0/1066862898 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e60003a70 con 0x7f5e640ff600 2023-07-14T11:13:25.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.945+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e64195330 con 0x7f5e640ff600 2023-07-14T11:13:25.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.946+0000 7f5e5b7fe700 1 -- 172.21.15.161:0/1066862898 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f5e60003d30 con 0x7f5e640ff600 2023-07-14T11:13:25.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.946+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6418e1d0 con 0x7f5e640ff600 2023-07-14T11:13:25.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.947+0000 7f5e5b7fe700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5e500387b0 0x7f5e5003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:25.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.947+0000 7f5e5b7fe700 1 -- 172.21.15.161:0/1066862898 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5e6004c5a0 con 0x7f5e640ff600 2023-07-14T11:13:25.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.947+0000 7f5e69c59700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5e500387b0 0x7f5e5003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:25.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.947+0000 7f5e69c59700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5e500387b0 0x7f5e5003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e54006fd0 tx=0x7f5e54006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:25.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:25.952+0000 7f5e5b7fe700 1 -- 172.21.15.161:0/1066862898 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5e6002ae20 con 0x7f5e640ff600 2023-07-14T11:13:26.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.149+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5e6402c850 con 0x7f5e640ff600 2023-07-14T11:13:26.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.152+0000 7f5e5b7fe700 1 -- 172.21.15.161:0/1066862898 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5e60026070 con 0x7f5e640ff600 2023-07-14T11:13:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5e500387b0 msgr2=0x7f5e5003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5e500387b0 0x7f5e5003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e54006fd0 tx=0x7f5e54006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 msgr2=0x7f5e64194750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e64194750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e60004f40 tx=0x7f5e600057c0 comp rx=0 tx=0).stop 2023-07-14T11:13:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 shutdown_connections 2023-07-14T11:13:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5e500387b0 0x7f5e5003ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 --2- 172.21.15.161:0/1066862898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e640ff600 0x7f5e64194750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 >> 172.21.15.161:0/1066862898 conn(0x7f5e640fab90 msgr2=0x7f5e640fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 shutdown_connections 2023-07-14T11:13:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:26.155+0000 7f5e6c6be700 1 -- 172.21.15.161:0/1066862898 wait complete. 2023-07-14T11:13:26.666 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-14T11:13:26.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-14T11:13:26.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:13:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1066862898' entity='client.admin' 2023-07-14T11:13:26.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:26.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:27.386 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:13:28.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.864+0000 7f91f88bd700 1 -- 172.21.15.161:0/1462205420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 msgr2=0x7f91f40ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:28.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.864+0000 7f91f88bd700 1 --2- 172.21.15.161:0/1462205420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f40ffa10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009910 tx=0x7f91e4009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:28.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.864+0000 7f91f88bd700 1 -- 172.21.15.161:0/1462205420 shutdown_connections 2023-07-14T11:13:28.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.864+0000 7f91f88bd700 1 --2- 172.21.15.161:0/1462205420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f40ffa10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:28.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.864+0000 7f91f88bd700 1 -- 172.21.15.161:0/1462205420 >> 172.21.15.161:0/1462205420 conn(0x7f91f40fab90 msgr2=0x7f91f40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:28.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.864+0000 7f91f88bd700 1 -- 172.21.15.161:0/1462205420 shutdown_connections 2023-07-14T11:13:28.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.864+0000 7f91f88bd700 1 -- 172.21.15.161:0/1462205420 wait complete. 2023-07-14T11:13:28.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.865+0000 7f91f88bd700 1 Processor -- start 2023-07-14T11:13:28.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.866+0000 7f91f88bd700 1 -- start start 2023-07-14T11:13:28.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.866+0000 7f91f88bd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f4196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:28.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.866+0000 7f91f88bd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4196e80 con 0x7f91f40ff600 2023-07-14T11:13:28.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.866+0000 7f91f259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f4196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:28.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.866+0000 7f91f259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f4196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54542/0 (socket says 172.21.15.161:54542) 2023-07-14T11:13:28.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.866+0000 7f91f259c700 1 -- 172.21.15.161:0/2111434360 learned_addr learned my addr 172.21.15.161:0/2111434360 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:28.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.866+0000 7f91f259c700 1 -- 172.21.15.161:0/2111434360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40095f0 con 0x7f91f40ff600 2023-07-14T11:13:28.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.867+0000 7f91f259c700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f4196940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f91e4004f40 tx=0x7f91e40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:28.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.867+0000 7f91e37fe700 1 -- 172.21.15.161:0/2111434360 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91e401d070 con 0x7f91f40ff600 2023-07-14T11:13:28.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.867+0000 7f91e37fe700 1 -- 172.21.15.161:0/2111434360 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e400bc10 con 0x7f91f40ff600 2023-07-14T11:13:28.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.867+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f4197080 con 0x7f91f40ff600 2023-07-14T11:13:28.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.867+0000 7f91e37fe700 1 -- 172.21.15.161:0/2111434360 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91e4003a70 con 0x7f91f40ff600 2023-07-14T11:13:28.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.867+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f4197520 con 0x7f91f40ff600 2023-07-14T11:13:28.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.868+0000 7f91e37fe700 1 -- 172.21.15.161:0/2111434360 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f91e4003d30 con 0x7f91f40ff600 2023-07-14T11:13:28.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.868+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f41905d0 con 0x7f91f40ff600 2023-07-14T11:13:28.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.869+0000 7f91e37fe700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91dc0387b0 0x7f91dc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:28.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.869+0000 7f91e37fe700 1 -- 172.21.15.161:0/2111434360 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91e404c5a0 con 0x7f91f40ff600 2023-07-14T11:13:28.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.869+0000 7f91f1d9b700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91dc0387b0 0x7f91dc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:28.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.869+0000 7f91f1d9b700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91dc0387b0 0x7f91dc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f91e8006fd0 tx=0x7f91e8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:28.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:28.874+0000 7f91e37fe700 1 -- 172.21.15.161:0/2111434360 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f91e402ae20 con 0x7f91f40ff600 2023-07-14T11:13:29.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.075+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f91f40609e0 con 0x7f91dc0387b0 2023-07-14T11:13:29.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.079+0000 7f91e37fe700 1 -- 172.21.15.161:0/2111434360 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f91f40609e0 con 0x7f91dc0387b0 2023-07-14T11:13:29.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.081+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91dc0387b0 msgr2=0x7f91dc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:29.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.081+0000 7f91f88bd700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91dc0387b0 0x7f91dc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f91e8006fd0 tx=0x7f91e8006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:29.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.081+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 msgr2=0x7f91f4196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:29.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.081+0000 7f91f88bd700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f4196940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f91e4004f40 tx=0x7f91e40057c0 comp rx=0 tx=0).stop 2023-07-14T11:13:29.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.082+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 shutdown_connections 2023-07-14T11:13:29.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.082+0000 7f91f88bd700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91dc0387b0 0x7f91dc03ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:29.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.082+0000 7f91f88bd700 1 --2- 172.21.15.161:0/2111434360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f40ff600 0x7f91f4196940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:29.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.082+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 >> 172.21.15.161:0/2111434360 conn(0x7f91f40fab90 msgr2=0x7f91f40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:29.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.082+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 shutdown_connections 2023-07-14T11:13:29.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:29.082+0000 7f91f88bd700 1 -- 172.21.15.161:0/2111434360 wait complete. 2023-07-14T11:13:29.559 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2023-07-14T11:13:29.559 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:13:29.559 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2023-07-14T11:13:29.579 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:13:29.579 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:29.638 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2023-07-14T11:13:29.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch host add smithi175 2023-07-14T11:13:30.377 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:13:30.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:30 smithi161 ceph-mon[95526]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:30.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:30 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:30 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:30 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:30 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:13:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:30 smithi161 ceph-mon[95526]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T11:13:31.366 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:31 smithi161 ceph-mon[95526]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:13:31.366 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:31 smithi161 ceph-mon[95526]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:31.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.977+0000 7fabc549f700 1 -- 172.21.15.161:0/354957485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 msgr2=0x7fabc0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:31.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.977+0000 7fabc549f700 1 --2- 172.21.15.161:0/354957485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc0072c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009910 tx=0x7fabb0009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:31.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.977+0000 7fabc549f700 1 -- 172.21.15.161:0/354957485 shutdown_connections 2023-07-14T11:13:31.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.977+0000 7fabc549f700 1 --2- 172.21.15.161:0/354957485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc0072c20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:31.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.977+0000 7fabc549f700 1 -- 172.21.15.161:0/354957485 >> 172.21.15.161:0/354957485 conn(0x7fabc00fb150 msgr2=0x7fabc00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:31.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.977+0000 7fabc549f700 1 -- 172.21.15.161:0/354957485 shutdown_connections 2023-07-14T11:13:31.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.977+0000 7fabc549f700 1 -- 172.21.15.161:0/354957485 wait complete. 2023-07-14T11:13:31.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.978+0000 7fabc549f700 1 Processor -- start 2023-07-14T11:13:31.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.979+0000 7fabc549f700 1 -- start start 2023-07-14T11:13:31.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.979+0000 7fabc549f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:31.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.979+0000 7fabc549f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc019cff0 con 0x7fabc0072850 2023-07-14T11:13:31.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.979+0000 7fabbeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:31.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.979+0000 7fabbeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54574/0 (socket says 172.21.15.161:54574) 2023-07-14T11:13:31.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.979+0000 7fabbeffd700 1 -- 172.21.15.161:0/2712079558 learned_addr learned my addr 172.21.15.161:0/2712079558 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:31.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.980+0000 7fabbeffd700 1 -- 172.21.15.161:0/2712079558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb00095f0 con 0x7fabc0072850 2023-07-14T11:13:31.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.980+0000 7fabbeffd700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc019cab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:31.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.980+0000 7faba7fff700 1 -- 172.21.15.161:0/2712079558 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabb001d070 con 0x7fabc0072850 2023-07-14T11:13:31.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.980+0000 7faba7fff700 1 -- 172.21.15.161:0/2712079558 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabb00044a0 con 0x7fabc0072850 2023-07-14T11:13:31.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.980+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc019d1f0 con 0x7fabc0072850 2023-07-14T11:13:31.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.980+0000 7faba7fff700 1 -- 172.21.15.161:0/2712079558 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabb0003a70 con 0x7fabc0072850 2023-07-14T11:13:31.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.980+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc019d610 con 0x7fabc0072850 2023-07-14T11:13:31.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.981+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc0061b40 con 0x7fabc0072850 2023-07-14T11:13:31.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.982+0000 7faba7fff700 1 -- 172.21.15.161:0/2712079558 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fabb0003cb0 con 0x7fabc0072850 2023-07-14T11:13:31.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.982+0000 7faba7fff700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faba80387b0 0x7faba803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:31.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.982+0000 7faba7fff700 1 -- 172.21.15.161:0/2712079558 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fabb004cae0 con 0x7fabc0072850 2023-07-14T11:13:31.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.983+0000 7fabbe7fc700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faba80387b0 0x7faba803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:31.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.983+0000 7fabbe7fc700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faba80387b0 0x7faba803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006fd0 tx=0x7fabb4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:31.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:31.987+0000 7faba7fff700 1 -- 172.21.15.161:0/2712079558 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fabb0005c00 con 0x7fabc0072850 2023-07-14T11:13:32.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:32.186+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}) v1 -- 0x7fabc0108c40 con 0x7faba80387b0 2023-07-14T11:13:32.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:32 smithi161 ceph-mon[95526]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:13:32.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:32.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:32.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:33.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:33 smithi161 ceph-mon[95526]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.469+0000 7faba7fff700 1 -- 172.21.15.161:0/2712079558 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fabc0108c40 con 0x7faba80387b0 2023-07-14T11:13:34.472 INFO:teuthology.orchestra.run.smithi161.stdout:Added host 'smithi175' with addr '172.21.15.175' 2023-07-14T11:13:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faba80387b0 msgr2=0x7faba803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faba80387b0 0x7faba803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006fd0 tx=0x7fabb4006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 msgr2=0x7fabc019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc019cab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb0004ff0 comp rx=0 tx=0).stop 2023-07-14T11:13:34.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 shutdown_connections 2023-07-14T11:13:34.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faba80387b0 0x7faba803ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:34.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 --2- 172.21.15.161:0/2712079558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabc0072850 0x7fabc019cab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:34.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 >> 172.21.15.161:0/2712079558 conn(0x7fabc00fb150 msgr2=0x7fabc01061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:34.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 shutdown_connections 2023-07-14T11:13:34.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:34.472+0000 7fabc549f700 1 -- 172.21.15.161:0/2712079558 wait complete. 2023-07-14T11:13:34.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:34 smithi161 ceph-mon[95526]: Deploying cephadm binary to smithi175 2023-07-14T11:13:34.964 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T11:13:35.686 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:13:35.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:35 smithi161 ceph-mon[95526]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:35.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:35 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:35.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:35 smithi161 ceph-mon[95526]: Added host smithi175 2023-07-14T11:13:35.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:35 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:35.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:35 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:36.960 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:37.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.194+0000 7ff354de4700 1 -- 172.21.15.161:0/2676954548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 msgr2=0x7ff350105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:37.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.194+0000 7ff354de4700 1 --2- 172.21.15.161:0/2676954548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350105a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff340009910 tx=0x7ff340009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:37.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.194+0000 7ff354de4700 1 -- 172.21.15.161:0/2676954548 shutdown_connections 2023-07-14T11:13:37.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.194+0000 7ff354de4700 1 --2- 172.21.15.161:0/2676954548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350105a20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:37.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.194+0000 7ff354de4700 1 -- 172.21.15.161:0/2676954548 >> 172.21.15.161:0/2676954548 conn(0x7ff3500fb150 msgr2=0x7ff3500fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:37.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.194+0000 7ff354de4700 1 -- 172.21.15.161:0/2676954548 shutdown_connections 2023-07-14T11:13:37.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.194+0000 7ff354de4700 1 -- 172.21.15.161:0/2676954548 wait complete. 2023-07-14T11:13:37.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.195+0000 7ff354de4700 1 Processor -- start 2023-07-14T11:13:37.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.195+0000 7ff354de4700 1 -- start start 2023-07-14T11:13:37.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.196+0000 7ff354de4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:37.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.196+0000 7ff354de4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff350194ec0 con 0x7ff350105650 2023-07-14T11:13:37.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.196+0000 7ff34e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:37.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.196+0000 7ff34e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54604/0 (socket says 172.21.15.161:54604) 2023-07-14T11:13:37.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.196+0000 7ff34e59c700 1 -- 172.21.15.161:0/3073781385 learned_addr learned my addr 172.21.15.161:0/3073781385 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:37.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.196+0000 7ff34e59c700 1 -- 172.21.15.161:0/3073781385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400095f0 con 0x7ff350105650 2023-07-14T11:13:37.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.197+0000 7ff34e59c700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350194980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff340004f40 tx=0x7ff340005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:37.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.197+0000 7ff33f7fe700 1 -- 172.21.15.161:0/3073781385 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff34001d070 con 0x7ff350105650 2023-07-14T11:13:37.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.197+0000 7ff33f7fe700 1 -- 172.21.15.161:0/3073781385 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff340004370 con 0x7ff350105650 2023-07-14T11:13:37.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.197+0000 7ff33f7fe700 1 -- 172.21.15.161:0/3073781385 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3400039b0 con 0x7ff350105650 2023-07-14T11:13:37.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.197+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff350195120 con 0x7ff350105650 2023-07-14T11:13:37.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.197+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff350195620 con 0x7ff350105650 2023-07-14T11:13:37.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.198+0000 7ff33f7fe700 1 -- 172.21.15.161:0/3073781385 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7ff340003c70 con 0x7ff350105650 2023-07-14T11:13:37.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.198+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff350061b40 con 0x7ff350105650 2023-07-14T11:13:37.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.198+0000 7ff33f7fe700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff3380387b0 0x7ff33803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:37.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.198+0000 7ff33f7fe700 1 -- 172.21.15.161:0/3073781385 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff34004c470 con 0x7ff350105650 2023-07-14T11:13:37.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.199+0000 7ff34dd9b700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff3380387b0 0x7ff33803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:37.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.199+0000 7ff34dd9b700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff3380387b0 0x7ff33803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff344006fd0 tx=0x7ff344006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:37.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.203+0000 7ff33f7fe700 1 -- 172.21.15.161:0/3073781385 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff34000b6f0 con 0x7ff350105650 2023-07-14T11:13:37.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.404+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff350195620 con 0x7ff3380387b0 2023-07-14T11:13:37.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.405+0000 7ff33f7fe700 1 -- 172.21.15.161:0/3073781385 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7ff350195620 con 0x7ff3380387b0 2023-07-14T11:13:37.408 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:37.408 INFO:teuthology.orchestra.run.smithi161.stdout:[{"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2023-07-14T11:13:37.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff3380387b0 msgr2=0x7ff33803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:37.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff3380387b0 0x7ff33803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff344006fd0 tx=0x7ff344006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:37.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 msgr2=0x7ff350194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:37.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350194980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff340004f40 tx=0x7ff340005020 comp rx=0 tx=0).stop 2023-07-14T11:13:37.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 shutdown_connections 2023-07-14T11:13:37.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff3380387b0 0x7ff33803ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:37.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 --2- 172.21.15.161:0/3073781385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350105650 0x7ff350194980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:37.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 >> 172.21.15.161:0/3073781385 conn(0x7ff3500fb150 msgr2=0x7ff3500fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:37.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 shutdown_connections 2023-07-14T11:13:37.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:37.408+0000 7ff354de4700 1 -- 172.21.15.161:0/3073781385 wait complete. 2023-07-14T11:13:37.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:37 smithi161 ceph-mon[95526]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:37.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:37.861 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi182 2023-07-14T11:13:37.862 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:13:37.862 DEBUG:teuthology.orchestra.run.smithi182:> dd of=/etc/ceph/ceph.conf 2023-07-14T11:13:37.882 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:13:37.882 DEBUG:teuthology.orchestra.run.smithi182:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:37.941 INFO:tasks.cephadm:Adding host smithi182 to orchestrator... 2023-07-14T11:13:37.942 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch host add smithi182 2023-07-14T11:13:38.660 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:13:38.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:38 smithi161 ceph-mon[95526]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:13:39.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:39 smithi161 ceph-mon[95526]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:40.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.072+0000 7f4fef8a3700 1 -- 172.21.15.161:0/3229099103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 msgr2=0x7f4fe8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:40.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.072+0000 7f4fef8a3700 1 --2- 172.21.15.161:0/3229099103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe8072c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009910 tx=0x7f4fe4009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.072+0000 7f4fef8a3700 1 -- 172.21.15.161:0/3229099103 shutdown_connections 2023-07-14T11:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.072+0000 7f4fef8a3700 1 --2- 172.21.15.161:0/3229099103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe8072c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.072+0000 7f4fef8a3700 1 -- 172.21.15.161:0/3229099103 >> 172.21.15.161:0/3229099103 conn(0x7f4fe80fb150 msgr2=0x7f4fe80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.072+0000 7f4fef8a3700 1 -- 172.21.15.161:0/3229099103 shutdown_connections 2023-07-14T11:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.072+0000 7f4fef8a3700 1 -- 172.21.15.161:0/3229099103 wait complete. 2023-07-14T11:13:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.073+0000 7f4fef8a3700 1 Processor -- start 2023-07-14T11:13:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.074+0000 7f4fef8a3700 1 -- start start 2023-07-14T11:13:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.074+0000 7f4fef8a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.074+0000 7f4fef8a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe819cf90 con 0x7f4fe80728b0 2023-07-14T11:13:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.074+0000 7f4fed63f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.074+0000 7f4fed63f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48888/0 (socket says 172.21.15.161:48888) 2023-07-14T11:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.074+0000 7f4fed63f700 1 -- 172.21.15.161:0/1797705720 learned_addr learned my addr 172.21.15.161:0/1797705720 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.075+0000 7f4fed63f700 1 -- 172.21.15.161:0/1797705720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40095f0 con 0x7f4fe80728b0 2023-07-14T11:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.075+0000 7f4fed63f700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe819ca50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4006010 tx=0x7f4fe4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.075+0000 7f4fde7fc700 1 -- 172.21.15.161:0/1797705720 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe401d070 con 0x7f4fe80728b0 2023-07-14T11:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.075+0000 7f4fde7fc700 1 -- 172.21.15.161:0/1797705720 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fe40044a0 con 0x7f4fe80728b0 2023-07-14T11:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.075+0000 7f4fde7fc700 1 -- 172.21.15.161:0/1797705720 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe4003a70 con 0x7f4fe80728b0 2023-07-14T11:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.075+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe819d190 con 0x7f4fe80728b0 2023-07-14T11:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.075+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe819d5b0 con 0x7f4fe80728b0 2023-07-14T11:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.076+0000 7f4fde7fc700 1 -- 172.21.15.161:0/1797705720 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f4fe4003c60 con 0x7f4fe80728b0 2023-07-14T11:13:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.076+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe8061b40 con 0x7f4fe80728b0 2023-07-14T11:13:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.077+0000 7f4fde7fc700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4fd4038800 0x7f4fd403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.077+0000 7f4fde7fc700 1 -- 172.21.15.161:0/1797705720 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4fe404c630 con 0x7f4fe80728b0 2023-07-14T11:13:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.077+0000 7f4fece3e700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4fd4038800 0x7f4fd403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.078+0000 7f4fece3e700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4fd4038800 0x7f4fd403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006fd0 tx=0x7f4fd8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.082+0000 7f4fde7fc700 1 -- 172.21.15.161:0/1797705720 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4fe4029360 con 0x7f4fe80728b0 2023-07-14T11:13:40.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:40.286+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi182", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fe8108c30 con 0x7f4fd4038800 2023-07-14T11:13:40.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:13:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:13:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: Updating smithi175:/etc/ceph/ceph.conf 2023-07-14T11:13:40.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:40.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:40.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:41.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:41 smithi161 ceph-mon[95526]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:13:41.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:41 smithi161 ceph-mon[95526]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:41.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:41 smithi161 ceph-mon[95526]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:41.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:41 smithi161 ceph-mon[95526]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:13:41.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:41 smithi161 ceph-mon[95526]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi182", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:42.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.590+0000 7f4fde7fc700 1 -- 172.21.15.161:0/1797705720 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4fe8108c30 con 0x7f4fd4038800 2023-07-14T11:13:42.592 INFO:teuthology.orchestra.run.smithi161.stdout:Added host 'smithi182' with addr '172.21.15.182' 2023-07-14T11:13:42.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4fd4038800 msgr2=0x7f4fd403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:42.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4fd4038800 0x7f4fd403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006fd0 tx=0x7f4fd8006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:42.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 msgr2=0x7f4fe819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:42.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe819ca50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4006010 tx=0x7f4fe4004ff0 comp rx=0 tx=0).stop 2023-07-14T11:13:42.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 shutdown_connections 2023-07-14T11:13:42.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4fd4038800 0x7f4fd403acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:42.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 --2- 172.21.15.161:0/1797705720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fe80728b0 0x7f4fe819ca50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:42.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 >> 172.21.15.161:0/1797705720 conn(0x7f4fe80fb150 msgr2=0x7f4fe8106150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:42.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 shutdown_connections 2023-07-14T11:13:42.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:42.593+0000 7f4fef8a3700 1 -- 172.21.15.161:0/1797705720 wait complete. 2023-07-14T11:13:42.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:42 smithi161 ceph-mon[95526]: Deploying cephadm binary to smithi182 2023-07-14T11:13:43.047 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T11:13:43.769 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:13:43.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:43 smithi161 ceph-mon[95526]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:43.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:43 smithi161 ceph-mon[95526]: Added host smithi182 2023-07-14T11:13:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:44.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:44.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:44 smithi161 ceph-mon[95526]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:45.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.285+0000 7f813bf7d700 1 -- 172.21.15.161:0/3944556575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 msgr2=0x7f8134105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:45.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.285+0000 7f813bf7d700 1 --2- 172.21.15.161:0/3944556575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134105a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8130009910 tx=0x7f8130009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:45.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.285+0000 7f813bf7d700 1 -- 172.21.15.161:0/3944556575 shutdown_connections 2023-07-14T11:13:45.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.285+0000 7f813bf7d700 1 --2- 172.21.15.161:0/3944556575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134105a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:45.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.285+0000 7f813bf7d700 1 -- 172.21.15.161:0/3944556575 >> 172.21.15.161:0/3944556575 conn(0x7f81340fb150 msgr2=0x7f81340fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:45.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.285+0000 7f813bf7d700 1 -- 172.21.15.161:0/3944556575 shutdown_connections 2023-07-14T11:13:45.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.285+0000 7f813bf7d700 1 -- 172.21.15.161:0/3944556575 wait complete. 2023-07-14T11:13:45.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.286+0000 7f813bf7d700 1 Processor -- start 2023-07-14T11:13:45.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.287+0000 7f813bf7d700 1 -- start start 2023-07-14T11:13:45.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.287+0000 7f813bf7d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:45.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.287+0000 7f813bf7d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8134194ec0 con 0x7f8134105650 2023-07-14T11:13:45.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.287+0000 7f8139d19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:45.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.287+0000 7f8139d19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48908/0 (socket says 172.21.15.161:48908) 2023-07-14T11:13:45.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.287+0000 7f8139d19700 1 -- 172.21.15.161:0/2764347311 learned_addr learned my addr 172.21.15.161:0/2764347311 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:45.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.287+0000 7f8139d19700 1 -- 172.21.15.161:0/2764347311 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300095f0 con 0x7f8134105650 2023-07-14T11:13:45.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.288+0000 7f8139d19700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134194980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8130006010 tx=0x7f8130004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:45.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.288+0000 7f812affd700 1 -- 172.21.15.161:0/2764347311 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f813001d070 con 0x7f8134105650 2023-07-14T11:13:45.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.288+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8134195120 con 0x7f8134105650 2023-07-14T11:13:45.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.288+0000 7f812affd700 1 -- 172.21.15.161:0/2764347311 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f813000bc10 con 0x7f8134105650 2023-07-14T11:13:45.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.288+0000 7f812affd700 1 -- 172.21.15.161:0/2764347311 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8130003910 con 0x7f8134105650 2023-07-14T11:13:45.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.288+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8134195620 con 0x7f8134105650 2023-07-14T11:13:45.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.289+0000 7f812affd700 1 -- 172.21.15.161:0/2764347311 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f813000bd80 con 0x7f8134105650 2023-07-14T11:13:45.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.289+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8134061b40 con 0x7f8134105650 2023-07-14T11:13:45.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.290+0000 7f812affd700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81200387b0 0x7f812003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:45.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.290+0000 7f812affd700 1 -- 172.21.15.161:0/2764347311 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f813004c410 con 0x7f8134105650 2023-07-14T11:13:45.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.290+0000 7f8139518700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81200387b0 0x7f812003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:45.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.290+0000 7f8139518700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81200387b0 0x7f812003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8124006fd0 tx=0x7f8124006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:45.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.295+0000 7f812affd700 1 -- 172.21.15.161:0/2764347311 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8130029360 con 0x7f8134105650 2023-07-14T11:13:45.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.494+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8134195620 con 0x7f81200387b0 2023-07-14T11:13:45.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.496+0000 7f812affd700 1 -- 172.21.15.161:0/2764347311 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f8134195620 con 0x7f81200387b0 2023-07-14T11:13:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:13:45.498 INFO:teuthology.orchestra.run.smithi161.stdout:[{"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}, {"addr": "172.21.15.182", "hostname": "smithi182", "labels": [], "status": ""}] 2023-07-14T11:13:45.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81200387b0 msgr2=0x7f812003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:45.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81200387b0 0x7f812003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8124006fd0 tx=0x7f8124006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:45.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 msgr2=0x7f8134194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:45.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134194980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8130006010 tx=0x7f8130004ff0 comp rx=0 tx=0).stop 2023-07-14T11:13:45.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 shutdown_connections 2023-07-14T11:13:45.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81200387b0 0x7f812003ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:45.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 --2- 172.21.15.161:0/2764347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8134105650 0x7f8134194980 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:45.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 >> 172.21.15.161:0/2764347311 conn(0x7f81340fb150 msgr2=0x7f81340fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:45.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 shutdown_connections 2023-07-14T11:13:45.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:45.498+0000 7f813bf7d700 1 -- 172.21.15.161:0/2764347311 wait complete. 2023-07-14T11:13:45.940 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-14T11:13:45.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd crush tunables default 2023-07-14T11:13:46.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:45 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:46.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:45 smithi161 ceph-mon[95526]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:13:46.664 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:13:47.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:47 smithi161 ceph-mon[95526]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:48.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.296+0000 7fd464e06700 1 -- 172.21.15.161:0/1960278966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 msgr2=0x7fd460107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:48.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.296+0000 7fd464e06700 1 --2- 172.21.15.161:0/1960278966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460107bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd450009910 tx=0x7fd450009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:48.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.296+0000 7fd464e06700 1 -- 172.21.15.161:0/1960278966 shutdown_connections 2023-07-14T11:13:48.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.296+0000 7fd464e06700 1 --2- 172.21.15.161:0/1960278966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460107bf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:48.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.296+0000 7fd464e06700 1 -- 172.21.15.161:0/1960278966 >> 172.21.15.161:0/1960278966 conn(0x7fd4600fd360 msgr2=0x7fd4600ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:48.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.296+0000 7fd464e06700 1 -- 172.21.15.161:0/1960278966 shutdown_connections 2023-07-14T11:13:48.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.296+0000 7fd464e06700 1 -- 172.21.15.161:0/1960278966 wait complete. 2023-07-14T11:13:48.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.297+0000 7fd464e06700 1 Processor -- start 2023-07-14T11:13:48.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.298+0000 7fd464e06700 1 -- start start 2023-07-14T11:13:48.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.298+0000 7fd464e06700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:48.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.298+0000 7fd464e06700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4601971e0 con 0x7fd460107820 2023-07-14T11:13:48.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.298+0000 7fd45e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:48.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.298+0000 7fd45e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35590/0 (socket says 172.21.15.161:35590) 2023-07-14T11:13:48.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.298+0000 7fd45e59c700 1 -- 172.21.15.161:0/2545553984 learned_addr learned my addr 172.21.15.161:0/2545553984 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:13:48.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.298+0000 7fd45e59c700 1 -- 172.21.15.161:0/2545553984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500095f0 con 0x7fd460107820 2023-07-14T11:13:48.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.299+0000 7fd45e59c700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460196ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd450004f40 tx=0x7fd450005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:48.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.299+0000 7fd44f7fe700 1 -- 172.21.15.161:0/2545553984 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd45001d070 con 0x7fd460107820 2023-07-14T11:13:48.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.299+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4601973e0 con 0x7fd460107820 2023-07-14T11:13:48.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.299+0000 7fd44f7fe700 1 -- 172.21.15.161:0/2545553984 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd45000be20 con 0x7fd460107820 2023-07-14T11:13:48.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.299+0000 7fd44f7fe700 1 -- 172.21.15.161:0/2545553984 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd450022d00 con 0x7fd460107820 2023-07-14T11:13:48.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.299+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd460197880 con 0x7fd460107820 2023-07-14T11:13:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.300+0000 7fd44f7fe700 1 -- 172.21.15.161:0/2545553984 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fd450022500 con 0x7fd460107820 2023-07-14T11:13:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.300+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460061b40 con 0x7fd460107820 2023-07-14T11:13:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.301+0000 7fd44f7fe700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd4480387b0 0x7fd44803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.301+0000 7fd44f7fe700 1 -- 172.21.15.161:0/2545553984 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd45004c890 con 0x7fd460107820 2023-07-14T11:13:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.301+0000 7fd45dd9b700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd4480387b0 0x7fd44803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.302+0000 7fd45dd9b700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd4480387b0 0x7fd44803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd454006fd0 tx=0x7fd454006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:48.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.306+0000 7fd44f7fe700 1 -- 172.21.15.161:0/2545553984 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd45002a540 con 0x7fd460107820 2023-07-14T11:13:48.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:48.519+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd460197fc0 con 0x7fd460107820 2023-07-14T11:13:48.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:48.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi182", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:13:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:13:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:13:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:48.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:48.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:48.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:48 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2545553984' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T11:13:49.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.511+0000 7fd44f7fe700 1 -- 172.21.15.161:0/2545553984 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd450027070 con 0x7fd460107820 2023-07-14T11:13:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd4480387b0 msgr2=0x7fd44803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd4480387b0 0x7fd44803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd454006fd0 tx=0x7fd454006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 msgr2=0x7fd460196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460196ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd450004f40 tx=0x7fd450005020 comp rx=0 tx=0).stop 2023-07-14T11:13:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 shutdown_connections 2023-07-14T11:13:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd4480387b0 0x7fd44803ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 --2- 172.21.15.161:0/2545553984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460107820 0x7fd460196ca0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 >> 172.21.15.161:0/2545553984 conn(0x7fd4600fd360 msgr2=0x7fd4600ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 shutdown_connections 2023-07-14T11:13:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:13:49.514+0000 7fd464e06700 1 -- 172.21.15.161:0/2545553984 wait complete. 2023-07-14T11:13:49.518 INFO:teuthology.orchestra.run.smithi161.stderr:adjusted tunables profile to default 2023-07-14T11:13:49.942 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:49 smithi161 ceph-mon[95526]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:13:49.942 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:49 smithi161 ceph-mon[95526]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:49.942 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:49 smithi161 ceph-mon[95526]: Updating smithi182:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:49.942 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:49 smithi161 ceph-mon[95526]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:13:49.942 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2545553984' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T11:13:49.943 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:49 smithi161 ceph-mon[95526]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:13:49.944 INFO:tasks.cephadm:Adding mon.a on smithi161 2023-07-14T11:13:49.945 INFO:tasks.cephadm:Adding mon.b on smithi175 2023-07-14T11:13:49.945 INFO:tasks.cephadm:Adding mon.c on smithi182 2023-07-14T11:13:49.945 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch apply mon '3;smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c' 2023-07-14T11:13:50.533 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:13:51.619 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.615+0000 7fb93d4b6700 1 -- 172.21.15.182:0/1377617771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 msgr2=0x7fb938100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:51.619 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.615+0000 7fb93d4b6700 1 --2- 172.21.15.182:0/1377617771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb938100a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb928009910 tx=0x7fb928009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:51.619 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.615+0000 7fb93d4b6700 1 -- 172.21.15.182:0/1377617771 shutdown_connections 2023-07-14T11:13:51.620 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.615+0000 7fb93d4b6700 1 --2- 172.21.15.182:0/1377617771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb938100a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:51.620 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.615+0000 7fb93d4b6700 1 -- 172.21.15.182:0/1377617771 >> 172.21.15.182:0/1377617771 conn(0x7fb9380fa020 msgr2=0x7fb9380fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:51.620 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.615+0000 7fb93d4b6700 1 -- 172.21.15.182:0/1377617771 shutdown_connections 2023-07-14T11:13:51.620 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.615+0000 7fb93d4b6700 1 -- 172.21.15.182:0/1377617771 wait complete. 2023-07-14T11:13:51.620 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.616+0000 7fb93d4b6700 1 Processor -- start 2023-07-14T11:13:51.620 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.616+0000 7fb93d4b6700 1 -- start start 2023-07-14T11:13:51.621 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.616+0000 7fb93d4b6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb9380fe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:51.621 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.616+0000 7fb93d4b6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb938100440 con 0x7fb938100690 2023-07-14T11:13:51.622 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.617+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb9380fe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:51.622 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.617+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb9380fe4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.182:52242/0 (socket says 172.21.15.182:52242) 2023-07-14T11:13:51.622 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.617+0000 7fb936ffd700 1 -- 172.21.15.182:0/4214336118 learned_addr learned my addr 172.21.15.182:0/4214336118 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:13:51.623 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.617+0000 7fb936ffd700 1 -- 172.21.15.182:0/4214336118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280095f0 con 0x7fb938100690 2023-07-14T11:13:51.623 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.617+0000 7fb936ffd700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb9380fe4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb928006010 tx=0x7fb928004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:51.623 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.618+0000 7fb91ffff700 1 -- 172.21.15.182:0/4214336118 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb92801d070 con 0x7fb938100690 2023-07-14T11:13:51.623 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.618+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9380fea00 con 0x7fb938100690 2023-07-14T11:13:51.623 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.618+0000 7fb91ffff700 1 -- 172.21.15.182:0/4214336118 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9280044a0 con 0x7fb938100690 2023-07-14T11:13:51.623 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.618+0000 7fb91ffff700 1 -- 172.21.15.182:0/4214336118 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb928003a70 con 0x7fb938100690 2023-07-14T11:13:51.623 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.618+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9380fee80 con 0x7fb938100690 2023-07-14T11:13:51.624 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.619+0000 7fb91ffff700 1 -- 172.21.15.182:0/4214336118 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fb928003cb0 con 0x7fb938100690 2023-07-14T11:13:51.624 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.619+0000 7fb91ffff700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb9200387b0 0x7fb92003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:51.624 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.619+0000 7fb91ffff700 1 -- 172.21.15.182:0/4214336118 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb92804c5d0 con 0x7fb938100690 2023-07-14T11:13:51.624 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.619+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb938106e10 con 0x7fb938100690 2023-07-14T11:13:51.624 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.619+0000 7fb9367fc700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb9200387b0 0x7fb92003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:51.624 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.620+0000 7fb9367fc700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb9200387b0 0x7fb92003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006fd0 tx=0x7fb92c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:51.626 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.624+0000 7fb91ffff700 1 -- 172.21.15.182:0/4214336118 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb928026020 con 0x7fb938100690 2023-07-14T11:13:51.819 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.816+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9381098c0 con 0x7fb9200387b0 2023-07-14T11:13:51.826 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.824+0000 7fb91ffff700 1 -- 172.21.15.182:0/4214336118 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb9381098c0 con 0x7fb9200387b0 2023-07-14T11:13:51.826 INFO:teuthology.orchestra.run.smithi182.stdout:Scheduled mon update... 2023-07-14T11:13:51.828 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb9200387b0 msgr2=0x7fb92003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:51.828 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb9200387b0 0x7fb92003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006fd0 tx=0x7fb92c006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:51.828 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 msgr2=0x7fb9380fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:51.829 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb9380fe4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb928006010 tx=0x7fb928004ff0 comp rx=0 tx=0).stop 2023-07-14T11:13:51.829 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 shutdown_connections 2023-07-14T11:13:51.829 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb9200387b0 0x7fb92003ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:51.829 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 --2- 172.21.15.182:0/4214336118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb938100690 0x7fb9380fe4c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:51.829 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.825+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 >> 172.21.15.182:0/4214336118 conn(0x7fb9380fa020 msgr2=0x7fb938103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:51.829 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.826+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 shutdown_connections 2023-07-14T11:13:51.829 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:51.826+0000 7fb93d4b6700 1 -- 172.21.15.182:0/4214336118 wait complete. 2023-07-14T11:13:51.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:51 smithi161 ceph-mon[95526]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:52.306 DEBUG:teuthology.orchestra.run.smithi175:mon.b> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.b.service 2023-07-14T11:13:52.310 DEBUG:teuthology.orchestra.run.smithi182:mon.c> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.c.service 2023-07-14T11:13:52.312 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T11:13:52.312 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph mon dump -f json 2023-07-14T11:13:52.340 INFO:journalctl@ceph.mon.c.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:13:52.345 INFO:journalctl@ceph.mon.b.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:13:53.038 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:13:53.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: Saving service mon spec with placement smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c;count:3 2023-07-14T11:13:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:13:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:13:53.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:53.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: Deploying daemon mon.c on smithi182 2023-07-14T11:13:53.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:52 smithi161 ceph-mon[95526]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.052 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.049+0000 7f60bb04b700 1 -- 172.21.15.182:0/261289189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 msgr2=0x7f60b4104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:55.053 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.049+0000 7f60bb04b700 1 --2- 172.21.15.182:0/261289189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4104910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009910 tx=0x7f60a8009c20 comp rx=0 tx=0).stop 2023-07-14T11:13:55.053 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.049+0000 7f60bb04b700 1 -- 172.21.15.182:0/261289189 shutdown_connections 2023-07-14T11:13:55.053 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.049+0000 7f60bb04b700 1 --2- 172.21.15.182:0/261289189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4104910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:55.053 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.049+0000 7f60bb04b700 1 -- 172.21.15.182:0/261289189 >> 172.21.15.182:0/261289189 conn(0x7f60b40fa020 msgr2=0x7f60b40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:55.054 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.049+0000 7f60bb04b700 1 -- 172.21.15.182:0/261289189 shutdown_connections 2023-07-14T11:13:55.054 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.049+0000 7f60bb04b700 1 -- 172.21.15.182:0/261289189 wait complete. 2023-07-14T11:13:55.056 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.050+0000 7f60bb04b700 1 Processor -- start 2023-07-14T11:13:55.057 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.050+0000 7f60bb04b700 1 -- start start 2023-07-14T11:13:55.057 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.050+0000 7f60bb04b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:55.057 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.050+0000 7f60bb04b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b4195cd0 con 0x7f60b4104540 2023-07-14T11:13:55.057 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.051+0000 7f60b8de7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:55.057 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.051+0000 7f60b8de7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.182:52248/0 (socket says 172.21.15.182:52248) 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.051+0000 7f60b8de7700 1 -- 172.21.15.182:0/37698530 learned_addr learned my addr 172.21.15.182:0/37698530 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.051+0000 7f60b8de7700 1 -- 172.21.15.182:0/37698530 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a80095f0 con 0x7f60b4104540 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.052+0000 7f60b8de7700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4195790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f60a8004f40 tx=0x7f60a8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.052+0000 7f60b1ffb700 1 -- 172.21.15.182:0/37698530 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60a801d070 con 0x7f60b4104540 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.052+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b4195f30 con 0x7f60b4104540 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.052+0000 7f60b1ffb700 1 -- 172.21.15.182:0/37698530 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a8004370 con 0x7f60b4104540 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.052+0000 7f60b1ffb700 1 -- 172.21.15.182:0/37698530 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60a80039b0 con 0x7f60b4104540 2023-07-14T11:13:55.058 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.052+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b41963b0 con 0x7f60b4104540 2023-07-14T11:13:55.059 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.053+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b4196600 con 0x7f60b4104540 2023-07-14T11:13:55.059 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.053+0000 7f60b1ffb700 1 -- 172.21.15.182:0/37698530 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f60a8003c70 con 0x7f60b4104540 2023-07-14T11:13:55.059 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.053+0000 7f60b1ffb700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f60a40387b0 0x7f60a403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:13:55.059 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.053+0000 7f60b1ffb700 1 -- 172.21.15.182:0/37698530 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60a804c520 con 0x7f60b4104540 2023-07-14T11:13:55.059 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.054+0000 7f60b3fff700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f60a40387b0 0x7f60a403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:13:55.060 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.054+0000 7f60b3fff700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f60a40387b0 0x7f60a403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006fd0 tx=0x7f60a0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:13:55.061 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.059+0000 7f60b1ffb700 1 -- 172.21.15.182:0/37698530 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f60b4196600 con 0x7f60b4104540 2023-07-14T11:13:55.339 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.335+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f60b4196600 con 0x7f60b4104540 2023-07-14T11:13:55.339 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.336+0000 7f60b1ffb700 1 -- 172.21.15.182:0/37698530 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f60b4196600 con 0x7f60b4104540 2023-07-14T11:13:55.339 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:13:55.340 INFO:teuthology.orchestra.run.smithi182.stdout:{"epoch":1,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","modified":"2023-07-14T11:12:12.829808Z","created":"2023-07-14T11:12:12.829808Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-14T11:13:55.341 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f60a40387b0 msgr2=0x7f60a403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:55.341 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f60a40387b0 0x7f60a403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006fd0 tx=0x7f60a0006e40 comp rx=0 tx=0).stop 2023-07-14T11:13:55.341 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 msgr2=0x7f60b4195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:13:55.342 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4195790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f60a8004f40 tx=0x7f60a8005020 comp rx=0 tx=0).stop 2023-07-14T11:13:55.342 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 shutdown_connections 2023-07-14T11:13:55.342 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f60a40387b0 0x7f60a403ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:55.342 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 --2- 172.21.15.182:0/37698530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60b4104540 0x7f60b4195790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:13:55.342 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.338+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 >> 172.21.15.182:0/37698530 conn(0x7f60b40fa020 msgr2=0x7f60b40fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:13:55.342 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.339+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 shutdown_connections 2023-07-14T11:13:55.342 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:13:55.339+0000 7f60bb04b700 1 -- 172.21.15.182:0/37698530 wait complete. 2023-07-14T11:13:55.343 INFO:teuthology.orchestra.run.smithi182.stderr:dumped monmap epoch 1 2023-07-14T11:13:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 systemd[1]: Started Ceph mon.c for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:13:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).mds e1 new map 2023-07-14T11:13:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-14T11:13:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout: e1 2023-07-14T11:13:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T11:13:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:13:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout: legacy client fscid: -1 2023-07-14T11:13:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout: 2023-07-14T11:13:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout: No filesystems configured 2023-07-14T11:13:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T11:13:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T11:13:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:13:55.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:13:55.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:55.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:13:55.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi182", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:55.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Deploying cephadm binary to smithi182 2023-07-14T11:13:55.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Added host smithi182 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:13:55.468 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.469 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.469 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.469 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi182", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.470 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.471 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2545553984' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T11:13:55.471 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:13:55.471 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.471 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Updating smithi182:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:13:55.471 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:13:55.471 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2545553984' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T11:13:55.471 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Saving service mon spec with placement smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c;count:3 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:13:55.472 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:13:55.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:13:55.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:13:55.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: Deploying daemon mon.c on smithi182 2023-07-14T11:13:55.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:55.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T11:13:55.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:13:55 smithi182 ceph-mon[129366]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-14T11:13:55.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:13:55 smithi161 ceph-mon[95526]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:13:56.736 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T11:13:56.736 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph mon dump -f json 2023-07-14T11:13:57.300 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:14:00.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:14:00.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: mon.a calling monitor election 2023-07-14T11:14:00.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:00.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: mon.c calling monitor election 2023-07-14T11:14:00.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:00.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.749 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:14:00.749 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: monmap e2: 2 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:00.749 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: fsmap 2023-07-14T11:14:00.749 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:00.750 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: mgrmap e13: x(active, since 46s) 2023-07-14T11:14:00.750 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: overall HEALTH_OK 2023-07-14T11:14:00.750 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:00.750 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:14:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: mon.a calling monitor election 2023-07-14T11:14:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: mon.c calling monitor election 2023-07-14T11:14:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:14:00.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: monmap e2: 2 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:00.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: fsmap 2023-07-14T11:14:00.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:00.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: mgrmap e13: x(active, since 46s) 2023-07-14T11:14:00.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: overall HEALTH_OK 2023-07-14T11:14:00.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:00.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:01.500 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.497+0000 7fbe84128700 1 -- 172.21.15.182:0/4260473471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680035c0 msgr2=0x7fbe68005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:01.500 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.497+0000 7fbe84128700 1 --2- 172.21.15.182:0/4260473471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680035c0 0x7fbe68005a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0778a0 tx=0x7fbe6c00feb0 comp rx=0 tx=0).stop 2023-07-14T11:14:01.501 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.497+0000 7fbe84128700 1 -- 172.21.15.182:0/4260473471 shutdown_connections 2023-07-14T11:14:01.501 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.497+0000 7fbe84128700 1 --2- 172.21.15.182:0/4260473471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680035c0 0x7fbe68005a40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:01.501 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.497+0000 7fbe84128700 1 -- 172.21.15.182:0/4260473471 >> 172.21.15.182:0/4260473471 conn(0x7fbe7c073d70 msgr2=0x7fbe7c074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:01.501 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.497+0000 7fbe84128700 1 -- 172.21.15.182:0/4260473471 shutdown_connections 2023-07-14T11:14:01.501 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.497+0000 7fbe84128700 1 -- 172.21.15.182:0/4260473471 wait complete. 2023-07-14T11:14:01.502 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.498+0000 7fbe84128700 1 Processor -- start 2023-07-14T11:14:01.502 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.498+0000 7fbe84128700 1 -- start start 2023-07-14T11:14:01.503 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.498+0000 7fbe84128700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 0x7fbe7c194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:01.503 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe84128700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe7c1952c0 0x7fbe7c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:01.503 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe84128700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c195840 con 0x7fbe7c1952c0 2023-07-14T11:14:01.503 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe84128700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1959b0 con 0x7fbe680035c0 2023-07-14T11:14:01.503 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe81ec4700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 0x7fbe7c194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:01.503 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe81ec4700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 0x7fbe7c194d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.182:36870/0 (socket says 172.21.15.182:36870) 2023-07-14T11:14:01.503 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe81ec4700 1 -- 172.21.15.182:0/1580387479 learned_addr learned my addr 172.21.15.182:0/1580387479 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:14:01.504 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe81ec4700 1 -- 172.21.15.182:0/1580387479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 msgr2=0x7fbe7c194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-14T11:14:01.504 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe81ec4700 1 -- 172.21.15.182:0/1580387479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 msgr2=0x7fbe7c194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T11:14:01.504 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe81ec4700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 0x7fbe7c194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:14:01.504 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe816c3700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe7c1952c0 0x7fbe7c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:01.504 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe81ec4700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 0x7fbe7c194d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:14:01.504 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe816c3700 1 -- 172.21.15.182:0/1580387479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 msgr2=0x7fbe7c194d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:14:01.504 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe816c3700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 0x7fbe7c194d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:01.505 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.499+0000 7fbe816c3700 1 -- 172.21.15.182:0/1580387479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c00f900 con 0x7fbe7c1952c0 2023-07-14T11:14:01.505 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.500+0000 7fbe816c3700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe7c1952c0 0x7fbe7c1996e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b4d0 tx=0x7fbe7800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:01.505 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.503+0000 7fbe72ffd700 1 -- 172.21.15.182:0/1580387479 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fbe78011630 con 0x7fbe7c1952c0 2023-07-14T11:14:01.506 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.503+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c199c80 con 0x7fbe7c1952c0 2023-07-14T11:14:01.506 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.503+0000 7fbe72ffd700 1 -- 172.21.15.182:0/1580387479 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe78011c50 con 0x7fbe7c1952c0 2023-07-14T11:14:01.506 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.503+0000 7fbe72ffd700 1 -- 172.21.15.182:0/1580387479 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fbe78010d90 con 0x7fbe7c1952c0 2023-07-14T11:14:01.506 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.503+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c19a180 con 0x7fbe7c1952c0 2023-07-14T11:14:01.507 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.503+0000 7fbe72ffd700 1 -- 172.21.15.182:0/1580387479 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fbe7801a400 con 0x7fbe7c1952c0 2023-07-14T11:14:01.507 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.504+0000 7fbe72ffd700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbe60038590 0x7fbe6003aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:01.507 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.504+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7c061b40 con 0x7fbe7c1952c0 2023-07-14T11:14:01.507 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.504+0000 7fbe72ffd700 1 -- 172.21.15.182:0/1580387479 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe7804aed0 con 0x7fbe7c1952c0 2023-07-14T11:14:01.507 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.504+0000 7fbe81ec4700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbe60038590 0x7fbe6003aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:01.508 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.505+0000 7fbe81ec4700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbe60038590 0x7fbe6003aa40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c001eb0 tx=0x7fbe6c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:01.512 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.510+0000 7fbe72ffd700 1 -- 172.21.15.182:0/1580387479 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbe780176d0 con 0x7fbe7c1952c0 2023-07-14T11:14:01.775 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.772+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbe7c1961d0 con 0x7fbe7c1952c0 2023-07-14T11:14:01.776 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.774+0000 7fbe72ffd700 1 -- 172.21.15.182:0/1580387479 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7fbe7800d1c0 con 0x7fbe7c1952c0 2023-07-14T11:14:01.776 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:14:01.776 INFO:teuthology.orchestra.run.smithi182.stdout:{"epoch":2,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","modified":"2023-07-14T11:13:55.409761Z","created":"2023-07-14T11:12:12.829808Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:3300","nonce":0},{"type":"v1","addr":"172.21.15.182:6789","nonce":0}]},"addr":"172.21.15.182:6789/0","public_addr":"172.21.15.182:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T11:14:01.778 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbe60038590 msgr2=0x7fbe6003aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:01.778 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbe60038590 0x7fbe6003aa40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c001eb0 tx=0x7fbe6c005960 comp rx=0 tx=0).stop 2023-07-14T11:14:01.778 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe7c1952c0 msgr2=0x7fbe7c1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:01.779 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe7c1952c0 0x7fbe7c1996e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b4d0 tx=0x7fbe7800b890 comp rx=0 tx=0).stop 2023-07-14T11:14:01.779 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 shutdown_connections 2023-07-14T11:14:01.779 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbe60038590 0x7fbe6003aa40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:01.779 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbe680035c0 0x7fbe7c194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:01.779 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 --2- 172.21.15.182:0/1580387479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe7c1952c0 0x7fbe7c1996e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:01.779 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 >> 172.21.15.182:0/1580387479 conn(0x7fbe7c073d70 msgr2=0x7fbe7c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:01.779 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 shutdown_connections 2023-07-14T11:14:01.780 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:01.775+0000 7fbe84128700 1 -- 172.21.15.182:0/1580387479 wait complete. 2023-07-14T11:14:01.780 INFO:teuthology.orchestra.run.smithi182.stderr:dumped monmap epoch 2 2023-07-14T11:14:01.801 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:01 smithi182 ceph-mon[129366]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:01.802 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:01 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:01.802 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:01 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:01.802 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:01 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:01.802 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:01 smithi182 ceph-mon[129366]: Deploying daemon mon.b on smithi175 2023-07-14T11:14:01.802 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:01 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:01.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:01 smithi161 ceph-mon[95526]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:01 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:01 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:01 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:01 smithi161 ceph-mon[95526]: Deploying daemon mon.b on smithi175 2023-07-14T11:14:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:01 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:02.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:02 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/1580387479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:14:02.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:02 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/1580387479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:14:03.159 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T11:14:03.159 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph mon dump -f json 2023-07-14T11:14:03.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 systemd[1]: Starting Ceph mon.b for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:14:03.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 podman[129041]: 2023-07-14T11:14:03.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T11:14:03.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T11:14:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pidfile_write: ignore empty --pid-file 2023-07-14T11:14:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 bash[129041]: fcb95897c17b49467f7b449e4b27d3adf5b85cc8de692ba57758d5587bc6429e 2023-07-14T11:14:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: load: jerasure load: lrc 2023-07-14T11:14:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: RocksDB version: 7.9.2 2023-07-14T11:14:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Git sha 0 2023-07-14T11:14:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T11:14:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: DB SUMMARY 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: DB Session ID: RWMCTA96A36D1AUNQU0J 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: CURRENT file: CURRENT 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.error_if_exists: 0 2023-07-14T11:14:03.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.create_if_missing: 0 2023-07-14T11:14:03.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.paranoid_checks: 1 2023-07-14T11:14:03.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T11:14:03.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T11:14:03.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T11:14:03.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.env: 0x5587e9819720 2023-07-14T11:14:03.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T11:14:03.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.info_log: 0x5587eb4cb280 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.statistics: (nil) 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.use_fsync: 0 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_log_file_size: 0 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T11:14:03.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.allow_fallocate: 1 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.use_direct_reads: 0 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.db_log_dir: 2023-07-14T11:14:03.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.wal_dir: 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T11:14:03.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T11:14:03.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.write_buffer_manager: 0x5587eb3ec3c0 2023-07-14T11:14:03.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T11:14:03.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T11:14:03.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T11:14:03.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T11:14:03.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T11:14:03.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.unordered_write: 0 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T11:14:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.row_cache: None 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.wal_filter: None 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.two_write_queues: 0 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.wal_compression: 0 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.atomic_flush: 0 2023-07-14T11:14:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T11:14:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T11:14:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T11:14:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.log_readahead_size: 0 2023-07-14T11:14:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T11:14:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T11:14:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T11:14:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T11:14:03.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T11:14:03.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T11:14:03.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T11:14:03.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_background_jobs: 2 2023-07-14T11:14:03.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_background_compactions: -1 2023-07-14T11:14:03.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_subcompactions: 1 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T11:14:03.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_open_files: -1 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_background_flushes: -1 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Compression algorithms supported: 2023-07-14T11:14:03.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T11:14:03.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kZSTD supported: 0 2023-07-14T11:14:03.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kXpressCompression supported: 0 2023-07-14T11:14:03.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T11:14:03.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kZlibCompression supported: 1 2023-07-14T11:14:03.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kSnappyCompression supported: 1 2023-07-14T11:14:03.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kLZ4Compression supported: 1 2023-07-14T11:14:03.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: kBZip2Compression supported: 0 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.merge_operator: 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_filter: None 2023-07-14T11:14:03.685 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5587eb4cb380) 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout: cache_index_and_filter_blocks: 1 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T11:14:03.686 INFO:journalctl@ceph.mon.b.smithi175.stdout: pin_top_level_index_and_filter: 1 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_type: 0 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: data_block_index_type: 0 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_shortening: 1 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: checksum: 4 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: no_block_cache: 0 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache: 0x5587eb48b1f0 2023-07-14T11:14:03.687 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_name: BinnedLRUCache 2023-07-14T11:14:03.688 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_options: 2023-07-14T11:14:03.688 INFO:journalctl@ceph.mon.b.smithi175.stdout: capacity : 536870912 2023-07-14T11:14:03.688 INFO:journalctl@ceph.mon.b.smithi175.stdout: num_shard_bits : 4 2023-07-14T11:14:03.688 INFO:journalctl@ceph.mon.b.smithi175.stdout: strict_capacity_limit : 0 2023-07-14T11:14:03.688 INFO:journalctl@ceph.mon.b.smithi175.stdout: high_pri_pool_ratio: 0.000 2023-07-14T11:14:03.688 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_compressed: (nil) 2023-07-14T11:14:03.688 INFO:journalctl@ceph.mon.b.smithi175.stdout: persistent_cache: (nil) 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_size: 4096 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_size_deviation: 10 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_restart_interval: 16 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_block_restart_interval: 1 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: metadata_block_size: 4096 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: partition_filters: 0 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: use_delta_encoding: 1 2023-07-14T11:14:03.689 INFO:journalctl@ceph.mon.b.smithi175.stdout: filter_policy: bloomfilter 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: whole_key_filtering: 1 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: verify_compression: 0 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: read_amp_bytes_per_bit: 0 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: format_version: 5 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: enable_index_compression: 1 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_align: 0 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: max_auto_readahead_size: 262144 2023-07-14T11:14:03.690 INFO:journalctl@ceph.mon.b.smithi175.stdout: prepopulate_block_cache: 0 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout: initial_auto_readahead_size: 8192 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression: NoCompression 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T11:14:03.691 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.num_levels: 7 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T11:14:03.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T11:14:03.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:14:03.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T11:14:03.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T11:14:03.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:14:03.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:14:03.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T11:14:03.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:14:03.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T11:14:03.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T11:14:03.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T11:14:03.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T11:14:03.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.table_properties_collectors: 2023-07-14T11:14:03.699 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.inplace_update_support: 0 2023-07-14T11:14:03.699 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T11:14:03.699 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T11:14:03.699 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T11:14:03.699 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T11:14:03.699 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.bloom_locality: 0 2023-07-14T11:14:03.699 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.max_successive_merges: 0 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.ttl: 2592000 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T11:14:03.700 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.enable_blob_files: false 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.min_blob_size: 0 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T11:14:03.701 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b16fc120-f665-4367-8194-fbc41f605e12 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689333243512139, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T11:14:03.702 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689333243512557, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689333243, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b16fc120-f665-4367-8194-fbc41f605e12", "db_session_id": "RWMCTA96A36D1AUNQU0J", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T11:14:03.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689333243512625, "job": 1, "event": "recovery_finished"} 2023-07-14T11:14:03.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T11:14:03.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 systemd[1]: Started Ceph mon.b for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:14:03.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T11:14:03.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5587eb524000 2023-07-14T11:14:03.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: DB pointer 0x5587eb510000 2023-07-14T11:14:03.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** DB Stats ** 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T11:14:03.704 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:14:03.705 INFO:journalctl@ceph.mon.b.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T11:14:03.706 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2023-07-14T11:14:03.707 INFO:journalctl@ceph.mon.b.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:14:03.707 INFO:journalctl@ceph.mon.b.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T11:14:03.707 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T11:14:03.707 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T11:14:03.707 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T11:14:03.707 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T11:14:03.707 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout: Block cache BinnedLRUCache@0x5587eb48b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-14T11:14:03.708 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: using public_addr v2:172.21.15.175:0/0 -> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: starting mon.b rank -1 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(???) e0 preinit fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).mds e1 new map 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout: e1 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:14:03.709 INFO:journalctl@ceph.mon.b.smithi175.stdout: legacy client fscid: -1 2023-07-14T11:14:03.710 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2023-07-14T11:14:03.710 INFO:journalctl@ceph.mon.b.smithi175.stdout: No filesystems configured 2023-07-14T11:14:03.710 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T11:14:03.710 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T11:14:03.710 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T11:14:03.710 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T11:14:03.710 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:14:03.711 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi182", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Deploying cephadm binary to smithi182 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Added host smithi182 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:03.712 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi182", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2545553984' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T11:14:03.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:03.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Updating smithi182:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:14:03.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:14:03.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2545553984' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T11:14:03.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Saving service mon spec with placement smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c;count:3 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:03.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Deploying daemon mon.c on smithi182 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:03.717 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.a calling monitor election 2023-07-14T11:14:03.718 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.718 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:03.718 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:03.718 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.c calling monitor election 2023-07-14T11:14:03.718 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.718 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:03.718 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: monmap e2: 2 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: fsmap 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mgrmap e13: x(active, since 46s) 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: overall HEALTH_OK 2023-07-14T11:14:03.719 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: Deploying daemon mon.b on smithi175 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:03.720 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/1580387479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expand map: {default=false} 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta from 'false' to 'false' 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expanded map: {default=false} 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expand map: {default=info} 2023-07-14T11:14:03.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta from 'info' to 'info' 2023-07-14T11:14:03.722 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expanded map: {default=info} 2023-07-14T11:14:03.722 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expand map: {default=daemon} 2023-07-14T11:14:03.722 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T11:14:03.722 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T11:14:03.722 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expand map: {default=debug} 2023-07-14T11:14:03.722 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T11:14:03.722 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:03 smithi175 ceph-mon[129081]: expand_channel_meta expanded map: {default=debug} 2023-07-14T11:14:03.723 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:14:03.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:03 smithi182 ceph-mon[129366]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:03.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:03 smithi161 ceph-mon[95526]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:14:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: mon.c calling monitor election 2023-07-14T11:14:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: mon.a calling monitor election 2023-07-14T11:14:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: monmap e3: 3 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: fsmap 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: mgrmap e13: x(active, since 54s) 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T11:14:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: mon.b (rank 2) addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] is down (out of quorum) 2023-07-14T11:14:08.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:08.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:08.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:08.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:14:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: mon.c calling monitor election 2023-07-14T11:14:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: mon.a calling monitor election 2023-07-14T11:14:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:14:08.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: monmap e3: 3 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: fsmap 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: mgrmap e13: x(active, since 54s) 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: mon.b (rank 2) addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] is down (out of quorum) 2023-07-14T11:14:08.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:08.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:08.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:08.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:09.304 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.302+0000 7f289d080700 1 -- 172.21.15.182:0/3154441420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 msgr2=0x7f2880005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:09.304 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.302+0000 7f289d080700 1 --2- 172.21.15.182:0/3154441420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 0x7f2880005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2898077900 tx=0x7f288c00fe60 comp rx=0 tx=0).stop 2023-07-14T11:14:09.304 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.302+0000 7f289d080700 1 -- 172.21.15.182:0/3154441420 shutdown_connections 2023-07-14T11:14:09.305 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.302+0000 7f289d080700 1 --2- 172.21.15.182:0/3154441420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 0x7f2880005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:09.305 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.302+0000 7f289d080700 1 -- 172.21.15.182:0/3154441420 >> 172.21.15.182:0/3154441420 conn(0x7f2898073d70 msgr2=0x7f2898074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:09.305 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.302+0000 7f289d080700 1 -- 172.21.15.182:0/3154441420 shutdown_connections 2023-07-14T11:14:09.305 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.302+0000 7f289d080700 1 -- 172.21.15.182:0/3154441420 wait complete. 2023-07-14T11:14:09.307 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 Processor -- start 2023-07-14T11:14:09.307 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 -- start start 2023-07-14T11:14:09.307 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 0x7f28981999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:09.308 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 0x7f2898193a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:09.308 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 0x7f2898194400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:09.308 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289819a4f0 con 0x7f28800035c0 2023-07-14T11:14:09.308 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980fe7c0 con 0x7f2898193f60 2023-07-14T11:14:09.308 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289d080700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980fe980 con 0x7f2898199ee0 2023-07-14T11:14:09.308 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 0x7f2898193a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:09.308 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 0x7f2898193a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.182:36896/0 (socket says 172.21.15.182:36896) 2023-07-14T11:14:09.309 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.303+0000 7f289659c700 1 -- 172.21.15.182:0/673490586 learned_addr learned my addr 172.21.15.182:0/673490586 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:14:09.309 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289659c700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 msgr2=0x7f2898193a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-14T11:14:09.309 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289659c700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 msgr2=0x7f2898193a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T11:14:09.309 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289659c700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 0x7f2898193a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:14:09.309 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f2896d9d700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 0x7f28981999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:09.309 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289659c700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 0x7f2898193a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:14:09.309 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289759e700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 0x7f2898194400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:09.310 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f2896d9d700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 msgr2=0x7f2898193a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:14:09.310 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289759e700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 msgr2=0x7f2898194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-14T11:14:09.310 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289759e700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 msgr2=0x7f2898194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T11:14:09.310 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f2896d9d700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 0x7f2898193a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:09.310 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289759e700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 0x7f2898194400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:14:09.310 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f2896d9d700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 msgr2=0x7f2898194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:09.310 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289759e700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 0x7f2898194400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:14:09.311 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f2896d9d700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 0x7f2898194400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:09.311 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f2896d9d700 1 -- 172.21.15.182:0/673490586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c00f8b0 con 0x7f28800035c0 2023-07-14T11:14:09.311 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f2896d9d700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 0x7f28981999a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f288c015040 tx=0x7f288c012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:09.311 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f287ffff700 1 -- 172.21.15.182:0/673490586 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c004130 con 0x7f28800035c0 2023-07-14T11:14:09.311 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f287ffff700 1 -- 172.21.15.182:0/673490586 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f288c004290 con 0x7f28800035c0 2023-07-14T11:14:09.311 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28980fec00 con 0x7f28800035c0 2023-07-14T11:14:09.311 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f287ffff700 1 -- 172.21.15.182:0/673490586 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c027870 con 0x7f28800035c0 2023-07-14T11:14:09.312 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.304+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28980ff150 con 0x7f28800035c0 2023-07-14T11:14:09.312 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.305+0000 7f287ffff700 1 -- 172.21.15.182:0/673490586 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f288c024050 con 0x7f28800035c0 2023-07-14T11:14:09.312 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.305+0000 7f287ffff700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2884038c80 0x7f288403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:09.312 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.305+0000 7f287ffff700 1 -- 172.21.15.182:0/673490586 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f288c000e40 con 0x7f28800035c0 2023-07-14T11:14:09.312 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.306+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2898061b40 con 0x7f28800035c0 2023-07-14T11:14:09.312 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.306+0000 7f289659c700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2884038c80 0x7f288403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:09.313 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.306+0000 7f289659c700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2884038c80 0x7f288403b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2888009180 tx=0x7f2888008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:09.313 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.311+0000 7f287ffff700 1 -- 172.21.15.182:0/673490586 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f288c022070 con 0x7f28800035c0 2023-07-14T11:14:09.585 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.584+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f28980ff410 con 0x7f28800035c0 2023-07-14T11:14:09.586 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.584+0000 7f287ffff700 1 -- 172.21.15.182:0/673490586 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f288c026730 con 0x7f28800035c0 2023-07-14T11:14:09.586 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-14T11:14:09.586 INFO:teuthology.orchestra.run.smithi182.stdout:{"epoch":3,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","modified":"2023-07-14T11:14:03.545838Z","created":"2023-07-14T11:12:12.829808Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:3300","nonce":0},{"type":"v1","addr":"172.21.15.182:6789","nonce":0}]},"addr":"172.21.15.182:6789/0","public_addr":"172.21.15.182:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T11:14:09.588 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2884038c80 msgr2=0x7f288403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:09.588 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2884038c80 0x7f288403b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2888009180 tx=0x7f2888008040 comp rx=0 tx=0).stop 2023-07-14T11:14:09.589 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 msgr2=0x7f28981999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:09.589 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 0x7f28981999a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f288c015040 tx=0x7f288c012d70 comp rx=0 tx=0).stop 2023-07-14T11:14:09.589 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 shutdown_connections 2023-07-14T11:14:09.589 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28800035c0 0x7f28981999a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:09.589 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2884038c80 0x7f288403b130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:09.589 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2898193f60 0x7f2898194400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:09.589 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 --2- 172.21.15.182:0/673490586 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2898199ee0 0x7f2898193a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:09.590 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 >> 172.21.15.182:0/673490586 conn(0x7f2898073d70 msgr2=0x7f289818caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:09.590 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 shutdown_connections 2023-07-14T11:14:09.590 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:09.587+0000 7f289d080700 1 -- 172.21.15.182:0/673490586 wait complete. 2023-07-14T11:14:09.590 INFO:teuthology.orchestra.run.smithi182.stderr:dumped monmap epoch 3 2023-07-14T11:14:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:09 smithi182 ceph-mon[129366]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:09 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/673490586' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:14:09.978 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-14T11:14:09.978 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph config generate-minimal-conf 2023-07-14T11:14:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:09 smithi161 ceph-mon[95526]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:09 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/673490586' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:14:10.730 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:14:10.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:14:10.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: mon.c calling monitor election 2023-07-14T11:14:10.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: mon.a calling monitor election 2023-07-14T11:14:10.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:10.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:14:10.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:10.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:10.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:10.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:10.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:10.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:10.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:10.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: monmap e3: 3 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: fsmap 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: mgrmap e13: x(active, since 54s) 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T11:14:10.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: mon.b (rank 2) addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] is down (out of quorum) 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:10.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:10.923 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:10 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/673490586' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:14:11.600 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: mon.b calling monitor election 2023-07-14T11:14:11.600 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:11.600 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T11:14:11.600 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: Updating smithi175:/etc/ceph/ceph.conf 2023-07-14T11:14:11.600 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:14:11.601 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:11.601 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:11.601 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: mon.b calling monitor election 2023-07-14T11:14:11.601 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: mon.a calling monitor election 2023-07-14T11:14:11.601 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: mon.c calling monitor election 2023-07-14T11:14:11.601 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T11:14:11.601 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: monmap e3: 3 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: fsmap 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: mgrmap e13: x(active, since 56s) 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: Cluster is now healthy 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: overall HEALTH_OK 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.602 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.603 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: mon.b calling monitor election 2023-07-14T11:14:11.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:11.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T11:14:11.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: Updating smithi175:/etc/ceph/ceph.conf 2023-07-14T11:14:11.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:14:11.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: mon.b calling monitor election 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: mon.a calling monitor election 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: mon.c calling monitor election 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: monmap e3: 3 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: fsmap 2023-07-14T11:14:11.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: mgrmap e13: x(active, since 56s) 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: Cluster is now healthy 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: overall HEALTH_OK 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: mon.b calling monitor election 2023-07-14T11:14:11.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:11.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T11:14:11.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: Updating smithi175:/etc/ceph/ceph.conf 2023-07-14T11:14:11.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: mon.b calling monitor election 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: mon.a calling monitor election 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: mon.c calling monitor election 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: monmap e3: 3 mons at {a=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0]} removed_ranks: {} 2023-07-14T11:14:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: fsmap 2023-07-14T11:14:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:14:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: mgrmap e13: x(active, since 56s) 2023-07-14T11:14:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T11:14:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: Cluster is now healthy 2023-07-14T11:14:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: overall HEALTH_OK 2023-07-14T11:14:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:11.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:12.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.121+0000 7ff8134cc700 1 -- 172.21.15.161:0/3392343519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 msgr2=0x7ff80c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.121+0000 7ff8134cc700 1 --2- 172.21.15.161:0/3392343519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c072c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff808009910 tx=0x7ff808009c20 comp rx=0 tx=0).stop 2023-07-14T11:14:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.121+0000 7ff8134cc700 1 -- 172.21.15.161:0/3392343519 shutdown_connections 2023-07-14T11:14:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.121+0000 7ff8134cc700 1 --2- 172.21.15.161:0/3392343519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c072c80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.121+0000 7ff8134cc700 1 -- 172.21.15.161:0/3392343519 >> 172.21.15.161:0/3392343519 conn(0x7ff80c0fb0d0 msgr2=0x7ff80c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:12.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.121+0000 7ff8134cc700 1 -- 172.21.15.161:0/3392343519 shutdown_connections 2023-07-14T11:14:12.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.121+0000 7ff8134cc700 1 -- 172.21.15.161:0/3392343519 wait complete. 2023-07-14T11:14:12.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.122+0000 7ff8134cc700 1 Processor -- start 2023-07-14T11:14:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.122+0000 7ff8134cc700 1 -- start start 2023-07-14T11:14:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff8134cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff8134cc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff80c19b710 0x7ff80c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff8134cc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff80c1a00c0 0x7ff80c1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff8134cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c19bde0 con 0x7ff80c0728b0 2023-07-14T11:14:12.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff8134cc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1a7e30 con 0x7ff80c1a00c0 2023-07-14T11:14:12.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff8134cc700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1a7f70 con 0x7ff80c19b710 2023-07-14T11:14:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff811268700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff811268700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c19b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41324/0 (socket says 172.21.15.161:41324) 2023-07-14T11:14:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.123+0000 7ff811268700 1 -- 172.21.15.161:0/2392671745 learned_addr learned my addr 172.21.15.161:0/2392671745 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:14:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff811a69700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff80c1a00c0 0x7ff80c1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff810a67700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff80c19b710 0x7ff80c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:12.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff811a69700 1 -- 172.21.15.161:0/2392671745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff80c19b710 msgr2=0x7ff80c19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:12.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff811a69700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff80c19b710 0x7ff80c19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff811a69700 1 -- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 msgr2=0x7ff80c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff811a69700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff811a69700 1 -- 172.21.15.161:0/2392671745 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080095f0 con 0x7ff80c1a00c0 2023-07-14T11:14:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff810a67700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff80c19b710 0x7ff80c19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:14:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.124+0000 7ff811268700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:14:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.125+0000 7ff811a69700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff80c1a00c0 0x7ff80c1a0560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff80400bd80 tx=0x7ff80400bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.125+0000 7ff8027fc700 1 -- 172.21.15.161:0/2392671745 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80400cc90 con 0x7ff80c1a00c0 2023-07-14T11:14:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.125+0000 7ff8027fc700 1 -- 172.21.15.161:0/2392671745 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff80400cdf0 con 0x7ff80c1a00c0 2023-07-14T11:14:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.125+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c1a8220 con 0x7ff80c1a00c0 2023-07-14T11:14:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.125+0000 7ff8027fc700 1 -- 172.21.15.161:0/2392671745 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040127e0 con 0x7ff80c1a00c0 2023-07-14T11:14:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.125+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c1a86d0 con 0x7ff80c1a00c0 2023-07-14T11:14:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.126+0000 7ff8027fc700 1 -- 172.21.15.161:0/2392671745 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7ff804012940 con 0x7ff80c1a00c0 2023-07-14T11:14:12.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.126+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c061b40 con 0x7ff80c1a00c0 2023-07-14T11:14:12.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.127+0000 7ff8027fc700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff7f8038c30 0x7ff7f803b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:12.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.127+0000 7ff8027fc700 1 -- 172.21.15.161:0/2392671745 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff80404ae30 con 0x7ff80c1a00c0 2023-07-14T11:14:12.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.127+0000 7ff811268700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff7f8038c30 0x7ff7f803b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:12.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.127+0000 7ff811268700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff7f8038c30 0x7ff7f803b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8080051a0 tx=0x7ff80800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:12.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.132+0000 7ff8027fc700 1 -- 172.21.15.161:0/2392671745 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff80401d3b0 con 0x7ff80c1a00c0 2023-07-14T11:14:12.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.343+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff80c19c530 con 0x7ff80c1a00c0 2023-07-14T11:14:12.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.344+0000 7ff8027fc700 1 -- 172.21.15.161:0/2392671745 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7ff804019070 con 0x7ff80c1a00c0 2023-07-14T11:14:12.348 INFO:teuthology.orchestra.run.smithi161.stdout:# minimal ceph.conf for 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:14:12.348 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2023-07-14T11:14:12.348 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:14:12.348 INFO:teuthology.orchestra.run.smithi161.stdout: mon_host = [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] 2023-07-14T11:14:12.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff7f8038c30 msgr2=0x7ff7f803b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:12.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff7f8038c30 0x7ff7f803b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8080051a0 tx=0x7ff80800b540 comp rx=0 tx=0).stop 2023-07-14T11:14:12.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff80c1a00c0 msgr2=0x7ff80c1a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:12.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff80c1a00c0 0x7ff80c1a0560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff80400bd80 tx=0x7ff80400bdb0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 shutdown_connections 2023-07-14T11:14:12.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c0728b0 0x7ff80c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff7f8038c30 0x7ff7f803b0e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff80c1a00c0 0x7ff80c1a0560 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 --2- 172.21.15.161:0/2392671745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff80c19b710 0x7ff80c19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:12.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 >> 172.21.15.161:0/2392671745 conn(0x7ff80c0fb0d0 msgr2=0x7ff80c0fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:12.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 shutdown_connections 2023-07-14T11:14:12.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:12.347+0000 7ff8134cc700 1 -- 172.21.15.161:0/2392671745 wait complete. 2023-07-14T11:14:12.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T11:14:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: Reconfiguring daemon mon.a on smithi161 2023-07-14T11:14:12.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:12 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2392671745' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:12.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T11:14:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: Reconfiguring daemon mon.a on smithi161 2023-07-14T11:14:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:12 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2392671745' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:12.745 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:14:12.544+0000 7f6ca8fed700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-14T11:14:12.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:14:12.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:14:12.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T11:14:12.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:12.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:12.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:12.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: Reconfiguring daemon mon.a on smithi161 2023-07-14T11:14:12.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:12 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2392671745' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:12.922 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-14T11:14:12.923 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:14:12.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T11:14:13.035 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:14:13.036 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:14:13.147 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:14:13.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T11:14:13.183 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:14:13.183 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:14:13.253 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:14:13.254 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T11:14:13.289 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:14:13.290 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:14:13.360 INFO:tasks.cephadm:Adding mgr.x on smithi161 2023-07-14T11:14:13.360 INFO:tasks.cephadm:Adding mgr.y on smithi175 2023-07-14T11:14:13.360 INFO:tasks.cephadm:Adding mgr.z on smithi182 2023-07-14T11:14:13.361 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch apply mgr '3;smithi161=x;smithi175=y;smithi182=z' 2023-07-14T11:14:13.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:13 smithi161 ceph-mon[95526]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:13.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:13.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:13 smithi175 ceph-mon[129081]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:13.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:13.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:13.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:13.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:13.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:13.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:13 smithi182 ceph-mon[129366]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:13.951 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:14:14.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:14 smithi182 ceph-mon[129366]: Reconfiguring mon.b (monmap changed)... 2023-07-14T11:14:14.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:14 smithi182 ceph-mon[129366]: Reconfiguring daemon mon.b on smithi175 2023-07-14T11:14:14.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:14 smithi161 ceph-mon[95526]: Reconfiguring mon.b (monmap changed)... 2023-07-14T11:14:14.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:14 smithi161 ceph-mon[95526]: Reconfiguring daemon mon.b on smithi175 2023-07-14T11:14:14.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:14 smithi175 ceph-mon[129081]: Reconfiguring mon.b (monmap changed)... 2023-07-14T11:14:14.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:14 smithi175 ceph-mon[129081]: Reconfiguring daemon mon.b on smithi175 2023-07-14T11:14:15.144 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.141+0000 7f48dd6a8700 1 -- 172.21.15.182:0/3816153511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d80feb30 msgr2=0x7f48d80fef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:15.144 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.141+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/3816153511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d80feb30 0x7f48d80fef40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009910 tx=0x7f48cc009c20 comp rx=0 tx=0).stop 2023-07-14T11:14:15.144 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.141+0000 7f48dd6a8700 1 -- 172.21.15.182:0/3816153511 shutdown_connections 2023-07-14T11:14:15.144 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.141+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/3816153511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d80feb30 0x7f48d80fef40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.145 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.141+0000 7f48dd6a8700 1 -- 172.21.15.182:0/3816153511 >> 172.21.15.182:0/3816153511 conn(0x7f48d80f9320 msgr2=0x7f48d80fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:15.145 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.141+0000 7f48dd6a8700 1 -- 172.21.15.182:0/3816153511 shutdown_connections 2023-07-14T11:14:15.145 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.141+0000 7f48dd6a8700 1 -- 172.21.15.182:0/3816153511 wait complete. 2023-07-14T11:14:15.145 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 Processor -- start 2023-07-14T11:14:15.145 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 -- start start 2023-07-14T11:14:15.145 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f48d80feb30 0x7f48d80fe210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:15.145 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d81977a0 0x7f48d8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:15.146 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d819a110 0x7f48d819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:15.146 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d819cd40 con 0x7f48d819a110 2023-07-14T11:14:15.146 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d819ceb0 con 0x7f48d81977a0 2023-07-14T11:14:15.146 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.142+0000 7f48dd6a8700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d819d070 con 0x7f48d80feb30 2023-07-14T11:14:15.147 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f48d80feb30 0x7f48d80fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:15.148 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f48d80feb30 0x7f48d80fe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.182:36904/0 (socket says 172.21.15.182:36904) 2023-07-14T11:14:15.148 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 -- 172.21.15.182:0/1298608210 learned_addr learned my addr 172.21.15.182:0/1298608210 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:14:15.148 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d67fc700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d81977a0 0x7f48d8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:15.148 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d77fe700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d819a110 0x7f48d819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:15.148 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 -- 172.21.15.182:0/1298608210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d81977a0 msgr2=0x7f48d8199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:15.148 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d81977a0 0x7f48d8199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.149 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 -- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d819a110 msgr2=0x7f48d819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:15.149 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d819a110 0x7f48d819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.149 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 -- 172.21.15.182:0/1298608210 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0095f0 con 0x7f48d80feb30 2023-07-14T11:14:15.149 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48d6ffd700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f48d80feb30 0x7f48d80fe210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f48cc004cb0 tx=0x7f48cc004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:15.149 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48bffff700 1 -- 172.21.15.182:0/1298608210 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc01d070 con 0x7f48d80feb30 2023-07-14T11:14:15.149 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48bffff700 1 -- 172.21.15.182:0/1298608210 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48cc00be20 con 0x7f48d80feb30 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48bffff700 1 -- 172.21.15.182:0/1298608210 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc022e50 con 0x7f48d80feb30 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d819d2f0 con 0x7f48d80feb30 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.143+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d819d7c0 con 0x7f48d80feb30 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.144+0000 7f48bffff700 1 -- 172.21.15.182:0/1298608210 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f48cc00f460 con 0x7f48d80feb30 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.144+0000 7f48bffff700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f48c0038c30 0x7f48c003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.144+0000 7f48bffff700 1 -- 172.21.15.182:0/1298608210 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f48cc04cad0 con 0x7f48d80feb30 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.144+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d818d190 con 0x7f48d80feb30 2023-07-14T11:14:15.150 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.145+0000 7f48d67fc700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f48c0038c30 0x7f48c003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:15.151 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.145+0000 7f48d67fc700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f48c0038c30 0x7f48c003b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f48d80fe770 tx=0x7f48c8009210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:15.151 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.149+0000 7f48bffff700 1 -- 172.21.15.182:0/1298608210 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f48cc027020 con 0x7f48d80feb30 2023-07-14T11:14:15.361 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.359+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi161=x;smithi175=y;smithi182=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f48d8060950 con 0x7f48c0038c30 2023-07-14T11:14:15.368 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.366+0000 7f48bffff700 1 -- 172.21.15.182:0/1298608210 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f48d8060950 con 0x7f48c0038c30 2023-07-14T11:14:15.368 INFO:teuthology.orchestra.run.smithi182.stdout:Scheduled mgr update... 2023-07-14T11:14:15.370 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f48c0038c30 msgr2=0x7f48c003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:15.370 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f48c0038c30 0x7f48c003b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f48d80fe770 tx=0x7f48c8009210 comp rx=0 tx=0).stop 2023-07-14T11:14:15.371 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f48d80feb30 msgr2=0x7f48d80fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:15.371 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f48d80feb30 0x7f48d80fe210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f48cc004cb0 tx=0x7f48cc004ce0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.371 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 shutdown_connections 2023-07-14T11:14:15.371 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d819a110 0x7f48d819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.371 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f48c0038c30 0x7f48c003b0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.371 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f48d80feb30 0x7f48d80fe210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.371 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 --2- 172.21.15.182:0/1298608210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d81977a0 0x7f48d8199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:15.372 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 >> 172.21.15.182:0/1298608210 conn(0x7f48d80f9320 msgr2=0x7f48d80fa070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:15.372 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 shutdown_connections 2023-07-14T11:14:15.372 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:14:15.369+0000 7f48dd6a8700 1 -- 172.21.15.182:0/1298608210 wait complete. 2023-07-14T11:14:15.553 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:15.553 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.553 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.553 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: Reconfiguring mon.c (monmap changed)... 2023-07-14T11:14:15.554 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:15.554 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:15.554 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:15.554 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: Reconfiguring daemon mon.c on smithi182 2023-07-14T11:14:15.554 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:15.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: Reconfiguring mon.c (monmap changed)... 2023-07-14T11:14:15.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:15.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:15.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:15.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: Reconfiguring daemon mon.c on smithi182 2023-07-14T11:14:15.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.790 DEBUG:teuthology.orchestra.run.smithi175:mgr.y> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.y.service 2023-07-14T11:14:15.793 DEBUG:teuthology.orchestra.run.smithi182:mgr.z> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.z.service 2023-07-14T11:14:15.795 INFO:tasks.cephadm:Deploying OSDs... 2023-07-14T11:14:15.795 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:14:15.795 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:14:15.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:15.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.822 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: Reconfiguring mon.c (monmap changed)... 2023-07-14T11:14:15.822 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:14:15.822 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:14:15.822 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:15.822 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: Reconfiguring daemon mon.c on smithi182 2023-07-14T11:14:15.822 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:15.823 INFO:journalctl@ceph.mgr.z.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:14:15.827 INFO:journalctl@ceph.mgr.y.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:14:15.843 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:14:15.844 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2023-07-14T11:14:15.890 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:14:15.890 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:15.890 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-07-14T11:14:15.890 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:15.890 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:15.891 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 11:13:24.944870916 +0000 2023-07-14T11:14:15.891 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 11:02:19.469088001 +0000 2023-07-14T11:14:15.891 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 11:02:19.469088001 +0000 2023-07-14T11:14:15.891 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2023-07-14 11:02:19.439088630 +0000 2023-07-14T11:14:15.891 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:14:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T11:14:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T11:14:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000295306 s, 1.7 MB/s 2023-07-14T11:14:15.961 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:14:16.007 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2023-07-14T11:14:16.091 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:14:16.091 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:16.091 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-07-14T11:14:16.091 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:16.091 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:16.091 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 11:13:24.944870916 +0000 2023-07-14T11:14:16.092 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 11:02:20.496066475 +0000 2023-07-14T11:14:16.092 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 11:02:20.496066475 +0000 2023-07-14T11:14:16.092 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2023-07-14 11:02:20.466067104 +0000 2023-07-14T11:14:16.092 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:14:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T11:14:16.200 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T11:14:16.200 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000315637 s, 1.6 MB/s 2023-07-14T11:14:16.202 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:14:16.247 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2023-07-14T11:14:16.330 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:14:16.330 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:16.331 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-07-14T11:14:16.331 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:16.331 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:16.331 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 11:13:24.944870916 +0000 2023-07-14T11:14:16.331 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 11:02:21.329049015 +0000 2023-07-14T11:14:16.331 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 11:02:21.329049015 +0000 2023-07-14T11:14:16.331 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2023-07-14 11:02:21.302049580 +0000 2023-07-14T11:14:16.332 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:14:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T11:14:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T11:14:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000279619 s, 1.8 MB/s 2023-07-14T11:14:16.438 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:14:16.484 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2023-07-14T11:14:16.567 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:14:16.568 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:16.568 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-07-14T11:14:16.568 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:16.568 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:16.568 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 11:13:24.944870916 +0000 2023-07-14T11:14:16.568 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 11:02:22.162031554 +0000 2023-07-14T11:14:16.568 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 11:02:22.162031554 +0000 2023-07-14T11:14:16.569 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2023-07-14 11:02:22.135032120 +0000 2023-07-14T11:14:16.569 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:14:16.673 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T11:14:16.673 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T11:14:16.673 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000347502 s, 1.5 MB/s 2023-07-14T11:14:16.675 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:14:16.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi161=x;smithi175=y;smithi182=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:16.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: Saving service mgr spec with placement smithi161=x;smithi175=y;smithi182=z;count:3 2023-07-14T11:14:16.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:16.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:16.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:14:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:16.720 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-07-14T11:14:16.720 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:14:16.740 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:14:16.741 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2023-07-14T11:14:16.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi161=x;smithi175=y;smithi182=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:16.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: Saving service mgr spec with placement smithi161=x;smithi175=y;smithi182=z;count:3 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:14:16.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:16.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:16.799 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:14:16.800 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:16.800 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 379251 Links: 1 2023-07-14T11:14:16.800 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:16.800 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:16.800 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-07-14 11:14:02.979876345 +0000 2023-07-14T11:14:16.800 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-07-14 10:59:23.820680201 +0000 2023-07-14T11:14:16.800 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-07-14 10:59:23.820680201 +0000 2023-07-14T11:14:16.801 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-07-14T11:14:16.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:14:16.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi161=x;smithi175=y;smithi182=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:16.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: Saving service mgr spec with placement smithi161=x;smithi175=y;smithi182=z;count:3 2023-07-14T11:14:16.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:16.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:14:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:16.868 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-07-14T11:14:16.868 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-07-14T11:14:16.868 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000234994 s, 2.2 MB/s 2023-07-14T11:14:16.869 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:14:16.929 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2023-07-14T11:14:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:14:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:16.987 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 379337 Links: 1 2023-07-14T11:14:16.987 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:16.987 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:16.988 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-07-14 11:14:02.979876345 +0000 2023-07-14T11:14:16.988 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-07-14 10:59:24.456670170 +0000 2023-07-14T11:14:16.988 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-07-14 10:59:24.456670170 +0000 2023-07-14T11:14:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-07-14T11:14:16.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:14:17.055 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-07-14T11:14:17.055 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-07-14T11:14:17.055 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000234749 s, 2.2 MB/s 2023-07-14T11:14:17.056 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:14:17.115 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2023-07-14T11:14:17.173 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:14:17.173 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:17.173 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 379390 Links: 1 2023-07-14T11:14:17.173 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:17.173 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:17.174 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-07-14 11:14:02.979876345 +0000 2023-07-14T11:14:17.174 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-07-14 10:59:25.076660392 +0000 2023-07-14T11:14:17.174 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-07-14 10:59:25.076660392 +0000 2023-07-14T11:14:17.174 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-07-14T11:14:17.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:14:17.239 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-07-14T11:14:17.240 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-07-14T11:14:17.240 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000213296 s, 2.4 MB/s 2023-07-14T11:14:17.240 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:14:17.298 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2023-07-14T11:14:17.356 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:14:17.356 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:17.356 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 381624 Links: 1 2023-07-14T11:14:17.356 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:17.356 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:17.357 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-07-14 11:14:02.979876345 +0000 2023-07-14T11:14:17.357 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-07-14 10:59:25.678650897 +0000 2023-07-14T11:14:17.357 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-07-14 10:59:25.678650897 +0000 2023-07-14T11:14:17.357 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-07-14T11:14:17.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:14:17.421 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-07-14T11:14:17.421 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-07-14T11:14:17.421 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.00023162 s, 2.2 MB/s 2023-07-14T11:14:17.422 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:14:17.480 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-07-14T11:14:17.480 DEBUG:teuthology.orchestra.run.smithi182:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:14:17.498 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:14:17.498 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_1 2023-07-14T11:14:17.557 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:14:17.557 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:17.557 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 379783 Links: 1 2023-07-14T11:14:17.558 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:17.558 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:17.558 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-07-14 11:14:17.499111967 +0000 2023-07-14T11:14:17.558 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-07-14 10:59:27.233164959 +0000 2023-07-14T11:14:17.558 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-07-14 10:59:27.233164959 +0000 2023-07-14T11:14:17.558 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-07-14T11:14:17.558 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:14:17.626 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:17 smithi182 ceph-mon[129366]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:17.626 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:17 smithi182 ceph-mon[129366]: Deploying daemon mgr.z on smithi182 2023-07-14T11:14:17.627 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:17 smithi182 systemd[1]: Starting Ceph mgr.z for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:14:17.629 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-07-14T11:14:17.630 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-07-14T11:14:17.630 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.000243353 s, 2.1 MB/s 2023-07-14T11:14:17.631 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:14:17.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:17 smithi175 ceph-mon[129081]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:17.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:17 smithi175 ceph-mon[129081]: Deploying daemon mgr.z on smithi182 2023-07-14T11:14:17.689 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_2 2023-07-14T11:14:17.745 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:14:17.746 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:17.746 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 373703 Links: 1 2023-07-14T11:14:17.746 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:17.746 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:17.746 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-07-14 11:14:17.499111967 +0000 2023-07-14T11:14:17.746 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-07-14 10:59:27.874150540 +0000 2023-07-14T11:14:17.746 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-07-14 10:59:27.874150540 +0000 2023-07-14T11:14:17.747 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-07-14T11:14:17.747 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:14:17.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:17 smithi161 ceph-mon[95526]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:17.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:17 smithi161 ceph-mon[95526]: Deploying daemon mgr.z on smithi182 2023-07-14T11:14:17.810 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-07-14T11:14:17.810 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-07-14T11:14:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.00019783 s, 2.6 MB/s 2023-07-14T11:14:17.811 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:14:17.868 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_3 2023-07-14T11:14:17.925 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:14:17.925 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:17.925 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 384083 Links: 1 2023-07-14T11:14:17.925 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:17.925 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:17.925 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-07-14 11:14:17.499111967 +0000 2023-07-14T11:14:17.926 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-07-14 10:59:28.474137044 +0000 2023-07-14T11:14:17.926 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-07-14 10:59:28.474137044 +0000 2023-07-14T11:14:17.926 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-07-14T11:14:17.926 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:14:17.963 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:17 smithi182 podman[131293]: 2023-07-14T11:14:17.989 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-07-14T11:14:17.989 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-07-14T11:14:17.989 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.000200166 s, 2.6 MB/s 2023-07-14T11:14:17.990 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:14:18.048 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_4 2023-07-14T11:14:18.105 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:14:18.105 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:14:18.105 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 378154 Links: 1 2023-07-14T11:14:18.106 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:14:18.106 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:14:18.106 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-07-14 11:14:17.499111967 +0000 2023-07-14T11:14:18.106 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-07-14 10:59:29.041124290 +0000 2023-07-14T11:14:18.106 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-07-14 10:59:29.041124290 +0000 2023-07-14T11:14:18.106 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-07-14T11:14:18.107 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:14:18.172 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-07-14T11:14:18.173 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-07-14T11:14:18.173 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.000213769 s, 2.4 MB/s 2023-07-14T11:14:18.173 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:14:18.230 INFO:tasks.cephadm:Deploying osd.0 on smithi161 with /dev/vg_nvme/lv_4... 2023-07-14T11:14:18.230 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T11:14:18.237 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 bash[131293]: 9e68936b6858a6e023597c80cf9d0144cb36e92cbfba24c602cb618544a78d24 2023-07-14T11:14:18.238 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 ceph-mgr[131476]: -- 172.21.15.182:0/830018630 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556296f711e0 con 0x556296f47000 2023-07-14T11:14:18.529 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 systemd[1]: Started Ceph mgr.z for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:14:18.529 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:18.264+0000 7feb70a6a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:14:18.529 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:18.347+0000 7feb70a6a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:14:18.529 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:18.435+0000 7feb70a6a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:14:18.529 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:18.527+0000 7feb70a6a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:14:18.963 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:14:18.964 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:18.692+0000 7feb70a6a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:14:19.299 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:18.996+0000 7feb70a6a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:14:19.299 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:19.083+0000 7feb70a6a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:14:19.300 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:19.210+0000 7feb70a6a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:14:19.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:14:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:19 smithi161 ceph-mon[95526]: Deploying daemon mgr.y on smithi175 2023-07-14T11:14:19.530 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:19.530 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.530 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.531 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.531 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:19.531 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:14:19.531 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:19.531 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:19.531 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:19 smithi175 ceph-mon[129081]: Deploying daemon mgr.y on smithi175 2023-07-14T11:14:19.713 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:19.303+0000 7feb70a6a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:14:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:19.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:14:19.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:19.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:19.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-mon[129366]: Deploying daemon mgr.y on smithi175 2023-07-14T11:14:19.843 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:19 smithi175 systemd[1]: Starting Ceph mgr.y for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:14:20.213 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:19.717+0000 7feb70a6a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:14:20.418 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 podman[130531]: 2023-07-14T11:14:20.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 bash[130531]: 6979a9212f0e84a8b32db61457b1a433b40a4e24b07829b88d343ed35af8330e 2023-07-14T11:14:20.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 ceph-mgr[130571]: -- 172.21.15.175:0/1629381390 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c5a0c111e0 con 0x55c5a0be7000 2023-07-14T11:14:20.713 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:20.282+0000 7feb70a6a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:14:20.714 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:20.453+0000 7feb70a6a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:14:20.746 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 systemd[1]: Started Ceph mgr.y for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:14:20.746 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:20.475+0000 7fe2ff8c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:14:20.746 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:20.557+0000 7fe2ff8c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:14:20.747 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:20.644+0000 7fe2ff8c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:14:20.747 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:20.743+0000 7fe2ff8c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:14:21.008 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T11:14:21.009 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T11:14:21.009 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T11:14:21.009 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T11:14:21.009 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0494945 s, 212 MB/s 2023-07-14T11:14:21.009 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T11:14:21.168 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:20 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:20.928+0000 7fe2ff8c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:14:21.432 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2023-07-14T11:14:21.524 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:21.254+0000 7feb70a6a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:14:21.524 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:21.345+0000 7feb70a6a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:14:21.524 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-mon[129366]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:21.525 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.525 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.525 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.525 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.525 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:21.585 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:21.260+0000 7fe2ff8c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:14:21.585 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:21.350+0000 7fe2ff8c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:14:21.586 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:21.482+0000 7fe2ff8c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:14:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-mon[129081]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.587 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:21.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:21 smithi161 ceph-mon[95526]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:21.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:21.818 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:21.522+0000 7feb70a6a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:14:22.160 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:14:22.207 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:21.816+0000 7feb70a6a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:14:22.207 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:22.053+0000 7feb70a6a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:14:22.207 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:22.205+0000 7feb70a6a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:14:22.418 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:21 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:21.582+0000 7fe2ff8c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:14:22.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:22 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:22.029+0000 7fe2ff8c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:14:22.918 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:22 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:22.613+0000 7fe2ff8c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:14:22.919 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:22 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:22.783+0000 7fe2ff8c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:14:22.963 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:22.472+0000 7feb70a6a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:14:23.368 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:23.368 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.368 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:23.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:23.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T11:14:23.369 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:23.370 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:23.370 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:23.370 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:23 smithi161 ceph-mon[95526]: Reconfiguring daemon mgr.x on smithi161 2023-07-14T11:14:23.463 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:23.170+0000 7feb70a6a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:14:23.464 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:23.299+0000 7feb70a6a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:14:23.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:23.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:23.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:23.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:23.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-mon[129366]: Reconfiguring daemon mgr.x on smithi161 2023-07-14T11:14:23.541 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:23.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:23.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:23.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:23.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T11:14:23.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:14:23.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:14:23.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:23.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-mon[129081]: Reconfiguring daemon mgr.x on smithi161 2023-07-14T11:14:23.799 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:23.540+0000 7fe2ff8c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:14:23.800 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:23.630+0000 7fe2ff8c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:14:23.814 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:23.488+0000 7feb70a6a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:14:23.814 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:23.599+0000 7feb70a6a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:14:24.071 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:23 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:23.798+0000 7fe2ff8c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:14:24.213 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:23.811+0000 7feb70a6a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:14:24.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/3979748351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4105850 msgr2=0x7fd7c4107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:24.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/3979748351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4105850 0x7fd7c4107cb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009f90 tx=0x7fd7bc00b330 comp rx=0 tx=0).stop 2023-07-14T11:14:24.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/3979748351 shutdown_connections 2023-07-14T11:14:24.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/3979748351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4105850 0x7fd7c4107cb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:24.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/3979748351 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4100610 0x7fd7c41029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:24.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/3979748351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7c4102f30 0x7fd7c4105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:24.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/3979748351 >> 172.21.15.161:0/3979748351 conn(0x7fd7c40fa0b0 msgr2=0x7fd7c40fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:24.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/3979748351 shutdown_connections 2023-07-14T11:14:24.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.272+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/3979748351 wait complete. 2023-07-14T11:14:24.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 Processor -- start 2023-07-14T11:14:24.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 -- start start 2023-07-14T11:14:24.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4100610 0x7fd7c4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:24.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4102f30 0x7fd7c41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:24.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7c4105850 0x7fd7c419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:24.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419f740 con 0x7fd7c4100610 2023-07-14T11:14:24.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419f880 con 0x7fd7c4105850 2023-07-14T11:14:24.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.273+0000 7fd7cbdc3700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419fa10 con 0x7fd7c4102f30 2023-07-14T11:14:24.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c935e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4102f30 0x7fd7c41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:24.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c9b5f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4100610 0x7fd7c4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:24.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c935e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4102f30 0x7fd7c41980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46000/0 (socket says 172.21.15.161:46000) 2023-07-14T11:14:24.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c935e700 1 -- 172.21.15.161:0/1779266021 learned_addr learned my addr 172.21.15.161:0/1779266021 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:14:24.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7ca360700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7c4105850 0x7fd7c419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:24.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c9b5f700 1 -- 172.21.15.161:0/1779266021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4102f30 msgr2=0x7fd7c41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:24.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c9b5f700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4102f30 0x7fd7c41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:24.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c9b5f700 1 -- 172.21.15.161:0/1779266021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7c4105850 msgr2=0x7fd7c419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:24.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c9b5f700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7c4105850 0x7fd7c419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:24.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c9b5f700 1 -- 172.21.15.161:0/1779266021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4009520 con 0x7fd7c4100610 2023-07-14T11:14:24.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c935e700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4102f30 0x7fd7c41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:14:24.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.274+0000 7fd7c9b5f700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4100610 0x7fd7c4197bb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000e9f0 tx=0x7fd7c000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:24.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.275+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000c9c0 con 0x7fd7c4100610 2023-07-14T11:14:24.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.275+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c0010430 con 0x7fd7c4100610 2023-07-14T11:14:24.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.275+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc009c40 con 0x7fd7c4100610 2023-07-14T11:14:24.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.275+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0018680 con 0x7fd7c4100610 2023-07-14T11:14:24.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.275+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c41a00b0 con 0x7fd7c4100610 2023-07-14T11:14:24.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.276+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c41073c0 con 0x7fd7c4100610 2023-07-14T11:14:24.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.276+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fd7c00105a0 con 0x7fd7c4100610 2023-07-14T11:14:24.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.277+0000 7fd7baffd700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd7b00389e0 0x7fd7b003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:24.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.277+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd7c0014070 con 0x7fd7c4100610 2023-07-14T11:14:24.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.277+0000 7fd7c935e700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd7b00389e0 0x7fd7b003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:24.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.278+0000 7fd7c935e700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd7b00389e0 0x7fd7b003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009fd0 tx=0x7fd7b4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:24.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.283+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd7c001f900 con 0x7fd7c4100610 2023-07-14T11:14:24.418 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:24 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:24.070+0000 7fe2ff8c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:14:24.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:24 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:24.297+0000 7fe2ff8c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:14:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:24.489+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7c4060950 con 0x7fd7b00389e0 2023-07-14T11:14:24.703 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:24 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:24.446+0000 7fe2ff8c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:14:24.963 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:14:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:14:24.667+0000 7feb70a6a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:14:25.072 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:24 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:24.702+0000 7fe2ff8c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:14:25.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:25.247+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 14) v1 ==== 90600+0+0 (secure 0 0 0) 0x7fd7c00187e0 con 0x7fd7c4100610 2023-07-14T11:14:25.395 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:25.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:25.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:25.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: Standby manager daemon z started 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:14:25.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:14:25.398 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:14:25.398 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:14:25.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:25.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:25.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:25.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:25.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:25.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:14:25.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:14:25.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:25.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: Standby manager daemon z started 2023-07-14T11:14:25.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:14:25.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:14:25.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:14:25.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:25 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:14:25.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:25.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:25.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:14:25.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:14:25.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:25.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: Standby manager daemon z started 2023-07-14T11:14:25.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:14:25.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:14:25.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:14:25.499 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:25 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/128404097' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:14:25.668 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:25.393+0000 7fe2ff8c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:14:25.669 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:25.522+0000 7fe2ff8c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:14:26.031 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:25.708+0000 7fe2ff8c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:14:26.031 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:25 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:25.819+0000 7fe2ff8c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:14:26.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:26 smithi175 ceph-mon[129081]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-14T11:14:26.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:26 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:14:26.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:26 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:26.029+0000 7fe2ff8c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:14:26.615 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:26 smithi161 ceph-mon[95526]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-14T11:14:26.616 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:14:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:26 smithi182 ceph-mon[129366]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-14T11:14:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:26 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:14:27.168 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:14:26 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:14:26.873+0000 7fe2ff8c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:14:27.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:27.261+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7fd7c001fb30 con 0x7fd7c4100610 2023-07-14T11:14:27.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:27 smithi175 ceph-mon[129081]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:27 smithi175 ceph-mon[129081]: Standby manager daemon y started 2023-07-14T11:14:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:27 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:14:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:27 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:14:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:27 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:14:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:27 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:14:27.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:27 smithi182 ceph-mon[129366]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:27.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:27 smithi182 ceph-mon[129366]: Standby manager daemon y started 2023-07-14T11:14:27.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:27 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:14:27.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:27 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:14:27.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:27 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:14:27.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:27 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:14:27.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:27 smithi161 ceph-mon[95526]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:27.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:27 smithi161 ceph-mon[95526]: Standby manager daemon y started 2023-07-14T11:14:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:27 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:14:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:27 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:14:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:27 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:14:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:27 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3941623046' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:14:28.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:28 smithi175 ceph-mon[129081]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-14T11:14:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:28 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:14:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:28 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3270678135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]: dispatch 2023-07-14T11:14:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:28 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]: dispatch 2023-07-14T11:14:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:28 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]': finished 2023-07-14T11:14:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:28 smithi175 ceph-mon[129081]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T11:14:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:28 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:28 smithi182 ceph-mon[129366]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-14T11:14:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:28 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:14:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:28 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3270678135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]: dispatch 2023-07-14T11:14:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:28 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]: dispatch 2023-07-14T11:14:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:28 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]': finished 2023-07-14T11:14:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:28 smithi182 ceph-mon[129366]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T11:14:28.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:28 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:28.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:28 smithi161 ceph-mon[95526]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-14T11:14:28.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:28 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:14:28.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:28 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3270678135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]: dispatch 2023-07-14T11:14:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:28 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]: dispatch 2023-07-14T11:14:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:28 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "606d353c-d690-4ba5-b3d5-2283accaf2e5"}]': finished 2023-07-14T11:14:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:28 smithi161 ceph-mon[95526]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T11:14:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:28 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:29.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:29 smithi175 ceph-mon[129081]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:29.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:29 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/703760449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:14:29.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:29 smithi182 ceph-mon[129366]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:29 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/703760449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:14:29.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:29 smithi161 ceph-mon[95526]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:29.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:29 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/703760449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:14:31.528 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:31 smithi161 ceph-mon[95526]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:31.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:31 smithi175 ceph-mon[129081]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:31 smithi182 ceph-mon[129366]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:33.629 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:33 smithi161 ceph-mon[95526]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:33.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:33 smithi175 ceph-mon[129081]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:33.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:33 smithi182 ceph-mon[129366]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:35.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:35 smithi175 ceph-mon[129081]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:35.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:35 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T11:14:35.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:35 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:35.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:35 smithi182 ceph-mon[129366]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:35.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:35 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T11:14:35.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:35 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:35.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:35 smithi161 ceph-mon[95526]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:35.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:35 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T11:14:35.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:35 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:36.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:36 smithi182 ceph-mon[129366]: Deploying daemon osd.0 on smithi161 2023-07-14T11:14:36.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:36 smithi161 ceph-mon[95526]: Deploying daemon osd.0 on smithi161 2023-07-14T11:14:36.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:36 smithi175 ceph-mon[129081]: Deploying daemon osd.0 on smithi161 2023-07-14T11:14:37.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:37 smithi182 ceph-mon[129366]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:37.725 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:37 smithi161 ceph-mon[95526]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:37.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:37 smithi175 ceph-mon[129081]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:39.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:39 smithi161 ceph-mon[95526]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:39.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:39 smithi182 ceph-mon[129366]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:39.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:39 smithi175 ceph-mon[129081]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:41.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:41 smithi182 ceph-mon[129366]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:41.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:41.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:41.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:41.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:41 smithi161 ceph-mon[95526]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:41.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:41.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:41.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:41.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:41 smithi175 ceph-mon[129081]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:43.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:43 smithi182 ceph-mon[129366]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:43.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:43 smithi161 ceph-mon[95526]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:43.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:43 smithi175 ceph-mon[129081]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:44.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.858+0000 7fd7baffd700 1 -- 172.21.15.161:0/1779266021 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd7c4060950 con 0x7fd7b00389e0 2023-07-14T11:14:44.861 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 0 on host 'smithi161' 2023-07-14T11:14:44.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd7b00389e0 msgr2=0x7fd7b003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:44.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd7b00389e0 0x7fd7b003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009fd0 tx=0x7fd7b4009260 comp rx=0 tx=0).stop 2023-07-14T11:14:44.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4100610 msgr2=0x7fd7c4197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:44.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4100610 0x7fd7c4197bb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000e9f0 tx=0x7fd7c000ed00 comp rx=0 tx=0).stop 2023-07-14T11:14:44.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 shutdown_connections 2023-07-14T11:14:44.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c4100610 0x7fd7c4197bb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:44.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd7b00389e0 0x7fd7b003ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:44.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7c4105850 0x7fd7c419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:44.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 --2- 172.21.15.161:0/1779266021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7c4102f30 0x7fd7c41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:44.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.862+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 >> 172.21.15.161:0/1779266021 conn(0x7fd7c40fa0b0 msgr2=0x7fd7c40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:44.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.863+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 shutdown_connections 2023-07-14T11:14:44.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:14:44.863+0000 7fd7cbdc3700 1 -- 172.21.15.161:0/1779266021 wait complete. 2023-07-14T11:14:45.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:44 smithi175 ceph-mon[129081]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:45.184 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.184 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.184 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:45.185 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:45.185 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.185 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:44 smithi161 ceph-mon[95526]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:45.213 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:45.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:45.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:45.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:44 smithi182 ceph-mon[129366]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:45.299 DEBUG:teuthology.orchestra.run.smithi161:osd.0> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.0.service 2023-07-14T11:14:45.303 INFO:tasks.cephadm:Deploying osd.1 on smithi175 with /dev/vg_nvme/lv_4... 2023-07-14T11:14:45.303 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T11:14:45.381 INFO:journalctl@ceph.osd.0.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:14:45.863 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:14:46.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:45 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:46.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:45 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:46.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:45 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:46.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:45 smithi175 ceph-mon[129081]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:14:46.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:45 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:46.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:45 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:46.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:45 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:46.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:45 smithi161 ceph-mon[95526]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:14:46.213 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:45 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:46.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:45 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:46.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:45 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:14:46.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:45 smithi182 ceph-mon[129366]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:14:46.957 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:46 smithi175 ceph-mon[129081]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T11:14:46.957 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:46 smithi175 ceph-mon[129081]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T11:14:46.957 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:46 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:46.958 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:46 smithi175 ceph-mon[129081]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:14:46.958 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:46 smithi175 ceph-mon[129081]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:47.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:46 smithi182 ceph-mon[129366]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T11:14:47.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:46 smithi182 ceph-mon[129366]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T11:14:47.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:46 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:47.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:46 smithi182 ceph-mon[129366]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:14:47.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:46 smithi182 ceph-mon[129366]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:47.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:46 smithi161 ceph-mon[95526]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T11:14:47.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:46 smithi161 ceph-mon[95526]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T11:14:47.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:46 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:47.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:46 smithi161 ceph-mon[95526]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:14:47.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:46 smithi161 ceph-mon[95526]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:47.247 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:14:46 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-0[108468]: 2023-07-14T11:14:46.846+0000 7f33face6700 -1 osd.0 0 waiting for initial osdmap 2023-07-14T11:14:47.247 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:14:46 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-0[108468]: 2023-07-14T11:14:46.852+0000 7f33f6adb700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:14:47.471 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T11:14:47.471 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T11:14:47.471 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2023-07-14T11:14:47.472 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2023-07-14T11:14:47.472 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111738 s, 938 MB/s 2023-07-14T11:14:47.472 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2023-07-14T11:14:47.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2023-07-14T11:14:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:47 smithi175 ceph-mon[129081]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:14:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:47 smithi175 ceph-mon[129081]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T11:14:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:47 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:47 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:48.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:47 smithi182 ceph-mon[129366]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:14:48.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:47 smithi182 ceph-mon[129366]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T11:14:48.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:47 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:48.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:47 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:48.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:47 smithi161 ceph-mon[95526]: from='osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:14:48.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:47 smithi161 ceph-mon[95526]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T11:14:48.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:47 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:48.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:47 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:48.366 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:14:49.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:14:49.124 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:14:49.124 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:49.124 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] boot 2023-07-14T11:14:49.124 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T11:14:49.124 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi161 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 17267M 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:49.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:49.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:14:49.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:14:49.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:49.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] boot 2023-07-14T11:14:49.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T11:14:49.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:49.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:49.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi161 2023-07-14T11:14:49.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:14:49.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 17267M 2023-07-14T11:14:49.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:49.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:49.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:14:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:14:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: osd.0 [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] boot 2023-07-14T11:14:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T11:14:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:14:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi161 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 17267M 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:49.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:14:49.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:14:49.564 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 -- 172.21.15.175:0/4192190284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c100e30 msgr2=0x7f6d7c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:49.565 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/4192190284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c100e30 0x7f6d7c101280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009990 tx=0x7f6d78009ca0 comp rx=0 tx=0).stop 2023-07-14T11:14:49.565 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 -- 172.21.15.175:0/4192190284 shutdown_connections 2023-07-14T11:14:49.565 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/4192190284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7c0fde50 0x7f6d7c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:49.565 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/4192190284 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d7c1017c0 0x7f6d7c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:49.565 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/4192190284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c100e30 0x7f6d7c101280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:49.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 -- 172.21.15.175:0/4192190284 >> 172.21.15.175:0/4192190284 conn(0x7f6d7c0f9860 msgr2=0x7f6d7c0fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:14:49.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 -- 172.21.15.175:0/4192190284 shutdown_connections 2023-07-14T11:14:49.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.561+0000 7f6d83ca0700 1 -- 172.21.15.175:0/4192190284 wait complete. 2023-07-14T11:14:49.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 Processor -- start 2023-07-14T11:14:49.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 -- start start 2023-07-14T11:14:49.567 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7c0fde50 0x7f6d7c100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:49.567 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d7c100e30 0x7f6d7c0fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:49.567 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c1017c0 0x7f6d7c0ff320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:49.567 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0ffaa0 con 0x7f6d7c0fde50 2023-07-14T11:14:49.567 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0ffc10 con 0x7f6d7c1017c0 2023-07-14T11:14:49.567 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d83ca0700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c19f860 con 0x7f6d7c100e30 2023-07-14T11:14:49.567 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c1017c0 0x7f6d7c0ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:49.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c1017c0 0x7f6d7c0ff320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58288/0 (socket says 172.21.15.175:58288) 2023-07-14T11:14:49.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 -- 172.21.15.175:0/2576691058 learned_addr learned my addr 172.21.15.175:0/2576691058 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-07-14T11:14:49.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8123b700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d7c100e30 0x7f6d7c0fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:49.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d81a3c700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7c0fde50 0x7f6d7c100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:49.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 -- 172.21.15.175:0/2576691058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d7c100e30 msgr2=0x7f6d7c0fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:49.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d7c100e30 0x7f6d7c0fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:49.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 -- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7c0fde50 msgr2=0x7f6d7c100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:14:49.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7c0fde50 0x7f6d7c100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:14:49.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 -- 172.21.15.175:0/2576691058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d780095f0 con 0x7f6d7c1017c0 2023-07-14T11:14:49.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d81a3c700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7c0fde50 0x7f6d7c100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:14:49.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.562+0000 7f6d8223d700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c1017c0 0x7f6d7c0ff320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ada0 tx=0x7f6d7400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:49.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.563+0000 7f6d72ffd700 1 -- 172.21.15.175:0/2576691058 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400f070 con 0x7f6d7c1017c0 2023-07-14T11:14:49.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.563+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c19fb40 con 0x7f6d7c1017c0 2023-07-14T11:14:49.570 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.563+0000 7f6d72ffd700 1 -- 172.21.15.175:0/2576691058 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d74007b30 con 0x7f6d7c1017c0 2023-07-14T11:14:49.570 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.563+0000 7f6d72ffd700 1 -- 172.21.15.175:0/2576691058 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d740053e0 con 0x7f6d7c1017c0 2023-07-14T11:14:49.570 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.563+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c1a0090 con 0x7f6d7c1017c0 2023-07-14T11:14:49.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.564+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c107180 con 0x7f6d7c1017c0 2023-07-14T11:14:49.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.565+0000 7f6d72ffd700 1 -- 172.21.15.175:0/2576691058 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f6d7400c980 con 0x7f6d7c1017c0 2023-07-14T11:14:49.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.565+0000 7f6d72ffd700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d680a10c0 0x7f6d680a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:14:49.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.565+0000 7f6d72ffd700 1 -- 172.21.15.175:0/2576691058 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f6d740ca230 con 0x7f6d7c1017c0 2023-07-14T11:14:49.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.566+0000 7f6d81a3c700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d680a10c0 0x7f6d680a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:14:49.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.566+0000 7f6d81a3c700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d680a10c0 0x7f6d680a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c067dd0 tx=0x7f6d6c00a680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:14:49.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.569+0000 7f6d72ffd700 1 -- 172.21.15.175:0/2576691058 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6d74095480 con 0x7f6d7c1017c0 2023-07-14T11:14:49.764 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:14:49.762+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d7c060950 con 0x7f6d680a10c0 2023-07-14T11:14:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:49 smithi175 ceph-mon[129081]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T11:14:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:49 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:14:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:49 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:14:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:49 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:50.213 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:49 smithi182 ceph-mon[129366]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T11:14:50.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:49 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:14:50.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:49 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:14:50.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:49 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:50.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:49 smithi161 ceph-mon[95526]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T11:14:50.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:49 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:14:50.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:49 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:14:50.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:49 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:51.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:50 smithi175 ceph-mon[129081]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:51.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:50 smithi175 ceph-mon[129081]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:51.213 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:50 smithi182 ceph-mon[129366]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:51.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:50 smithi182 ceph-mon[129366]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:51.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:50 smithi161 ceph-mon[95526]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:14:51.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:50 smithi161 ceph-mon[95526]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:53.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:53 smithi175 ceph-mon[129081]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:53.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:53 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/3869835041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]: dispatch 2023-07-14T11:14:53.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:53 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]: dispatch 2023-07-14T11:14:53.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:53 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]': finished 2023-07-14T11:14:53.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:53 smithi175 ceph-mon[129081]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T11:14:53.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:14:53.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:53 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/940868952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:14:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:53 smithi182 ceph-mon[129366]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:53 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/3869835041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]: dispatch 2023-07-14T11:14:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:53 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]: dispatch 2023-07-14T11:14:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:53 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]': finished 2023-07-14T11:14:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:53 smithi182 ceph-mon[129366]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T11:14:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:14:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:53 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/940868952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:14:53.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:53 smithi161 ceph-mon[95526]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:53.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:53 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/3869835041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]: dispatch 2023-07-14T11:14:53.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:53 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]: dispatch 2023-07-14T11:14:53.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:53 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660"}]': finished 2023-07-14T11:14:53.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:53 smithi161 ceph-mon[95526]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T11:14:53.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:14:53.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:53 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/940868952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:14:55.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:55 smithi175 ceph-mon[129081]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:55.463 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:55 smithi182 ceph-mon[129366]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:55.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:55 smithi161 ceph-mon[95526]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:57.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:57 smithi175 ceph-mon[129081]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:57.463 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:57 smithi182 ceph-mon[129366]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:57.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:57 smithi161 ceph-mon[95526]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:59.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:59 smithi175 ceph-mon[129081]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:59.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:59 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T11:14:59.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:59 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:59.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:14:59 smithi175 ceph-mon[129081]: Deploying daemon osd.1 on smithi175 2023-07-14T11:14:59.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:59 smithi182 ceph-mon[129366]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:59.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:59 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T11:14:59.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:59 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:59.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:14:59 smithi182 ceph-mon[129366]: Deploying daemon osd.1 on smithi175 2023-07-14T11:14:59.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:59 smithi161 ceph-mon[95526]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:14:59.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:59 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T11:14:59.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:59 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:14:59.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:14:59 smithi161 ceph-mon[95526]: Deploying daemon osd.1 on smithi175 2023-07-14T11:15:01.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:01 smithi182 ceph-mon[129366]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:01.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:01 smithi161 ceph-mon[95526]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:01.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:01 smithi175 ceph-mon[129081]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:03 smithi182 ceph-mon[129366]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:03 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:03 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:03 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:03.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:03 smithi161 ceph-mon[95526]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:03.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:03 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:03.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:03 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:03.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:03 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:03.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:03 smithi175 ceph-mon[129081]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:03.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:03.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:03.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:05.461 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:05 smithi175 ceph-mon[129081]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:05.461 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.461 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.462 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:05.462 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:05.462 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:05 smithi182 ceph-mon[129366]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:05 smithi161 ceph-mon[95526]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:05.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:05.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:05.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:05.878 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 1 on host 'smithi175' 2023-07-14T11:15:05.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.875+0000 7f6d72ffd700 1 -- 172.21.15.175:0/2576691058 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6d7c060950 con 0x7f6d680a10c0 2023-07-14T11:15:05.881 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d680a10c0 msgr2=0x7f6d680a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:05.881 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d680a10c0 0x7f6d680a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c067dd0 tx=0x7f6d6c00a680 comp rx=0 tx=0).stop 2023-07-14T11:15:05.881 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c1017c0 msgr2=0x7f6d7c0ff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:05.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c1017c0 0x7f6d7c0ff320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ada0 tx=0x7f6d7400c330 comp rx=0 tx=0).stop 2023-07-14T11:15:05.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 shutdown_connections 2023-07-14T11:15:05.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7c0fde50 0x7f6d7c100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:05.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d680a10c0 0x7f6d680a3570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:05.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d7c1017c0 0x7f6d7c0ff320 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:05.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 --2- 172.21.15.175:0/2576691058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d7c100e30 0x7f6d7c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:05.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.878+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 >> 172.21.15.175:0/2576691058 conn(0x7f6d7c0f9860 msgr2=0x7f6d7c1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:15:05.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.879+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 shutdown_connections 2023-07-14T11:15:05.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:05.879+0000 7f6d83ca0700 1 -- 172.21.15.175:0/2576691058 wait complete. 2023-07-14T11:15:06.362 DEBUG:teuthology.orchestra.run.smithi175:osd.1> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.1.service 2023-07-14T11:15:06.365 INFO:tasks.cephadm:Deploying osd.2 on smithi182 with /dev/vg_nvme/lv_4... 2023-07-14T11:15:06.365 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T11:15:06.402 INFO:journalctl@ceph.osd.1.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:15:06.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:06.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:06.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:06.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:06.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:06.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:06.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:06.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:06.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:06.929 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:15:07.418 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-1[134195]: 2023-07-14T11:15:07.116+0000 7f511401c640 -1 osd.1 0 log_to_monitors true 2023-07-14T11:15:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:07 smithi182 ceph-mon[129366]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:07 smithi182 ceph-mon[129366]: from='osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:15:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:07 smithi182 ceph-mon[129366]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:15:07.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:07 smithi161 ceph-mon[95526]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:07.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:07 smithi161 ceph-mon[95526]: from='osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:15:07.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:07 smithi161 ceph-mon[95526]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:15:07.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:07 smithi175 ceph-mon[129081]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:07.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:07 smithi175 ceph-mon[129081]: from='osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:15:07.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:07 smithi175 ceph-mon[129081]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:15:08.280 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T11:15:08.281 INFO:teuthology.orchestra.run.smithi182.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T11:15:08.281 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 10+0 records in 2023-07-14T11:15:08.281 INFO:teuthology.orchestra.run.smithi182.stderr:10+0 records out 2023-07-14T11:15:08.281 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120505 s, 870 MB/s 2023-07-14T11:15:08.281 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping successful for: 2023-07-14T11:15:08.681 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi182:vg_nvme/lv_4 2023-07-14T11:15:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T11:15:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T11:15:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:15:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:15:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi175 2023-07-14T11:15:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 17168M 2023-07-14T11:15:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:08.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.168 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-1[134195]: 2023-07-14T11:15:08.836+0000 7f5109694700 -1 osd.1 0 waiting for initial osdmap 2023-07-14T11:15:09.169 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-1[134195]: 2023-07-14T11:15:08.841+0000 7f5102c84700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:15:09.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T11:15:09.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T11:15:09.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:09.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi175 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 17168M 2023-07-14T11:15:09.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:09.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:09.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.230 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:15:09.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T11:15:09.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T11:15:09.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:09.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:15:09.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:15:09.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:09.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi175 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 17168M 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:09.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:10.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:09 smithi175 ceph-mon[129081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:15:10.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:09 smithi175 ceph-mon[129081]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T11:15:10.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:10.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:10.191 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:09 smithi182 ceph-mon[129366]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:15:10.191 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:09 smithi182 ceph-mon[129366]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T11:15:10.191 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:10.192 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:10.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:09 smithi161 ceph-mon[95526]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:15:10.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:09 smithi161 ceph-mon[95526]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T11:15:10.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:10.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:10.410 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2103804687 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 msgr2=0x7f2e740fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:10.410 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2103804687 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e740fffa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009990 tx=0x7f2e70009ca0 comp rx=0 tx=0).stop 2023-07-14T11:15:10.410 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2103804687 shutdown_connections 2023-07-14T11:15:10.410 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2103804687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e74101720 0x7f2e74107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:10.410 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2103804687 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e740fffa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:10.411 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2103804687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e74100d90 0x7f2e741011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:10.411 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2103804687 >> 172.21.15.182:0/2103804687 conn(0x7f2e740fb1a0 msgr2=0x7f2e740fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:15:10.411 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2103804687 shutdown_connections 2023-07-14T11:15:10.411 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2103804687 wait complete. 2023-07-14T11:15:10.413 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.407+0000 7f2e7bd3c700 1 Processor -- start 2023-07-14T11:15:10.413 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 -- start start 2023-07-14T11:15:10.413 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e74195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:10.413 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e74100d90 0x7f2e74195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:10.413 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e74101720 0x7f2e7419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:10.413 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7419d4d0 con 0x7f2e74101720 2023-07-14T11:15:10.413 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7419d610 con 0x7f2e74100d90 2023-07-14T11:15:10.414 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7419d7a0 con 0x7f2e740ffb90 2023-07-14T11:15:10.414 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e74195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:10.414 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e74195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.182:36936/0 (socket says 172.21.15.182:36936) 2023-07-14T11:15:10.414 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 -- 172.21.15.182:0/2242826527 learned_addr learned my addr 172.21.15.182:0/2242826527 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:15:10.414 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e792d7700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e74100d90 0x7f2e74195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:10.414 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7a2d9700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e74101720 0x7f2e7419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:10.414 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 -- 172.21.15.182:0/2242826527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e74100d90 msgr2=0x7f2e74195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e74100d90 0x7f2e74195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 -- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e74101720 msgr2=0x7f2e7419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e74101720 0x7f2e7419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 -- 172.21.15.182:0/2242826527 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e700095f0 con 0x7f2e740ffb90 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e79ad8700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e74195940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000b5c0 tx=0x7f2e7000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e6affd700 1 -- 172.21.15.182:0/2242826527 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7001d070 con 0x7f2e740ffb90 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7419da20 con 0x7f2e740ffb90 2023-07-14T11:15:10.415 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e6affd700 1 -- 172.21.15.182:0/2242826527 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e70022470 con 0x7f2e740ffb90 2023-07-14T11:15:10.416 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.408+0000 7f2e6affd700 1 -- 172.21.15.182:0/2242826527 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70017630 con 0x7f2e740ffb90 2023-07-14T11:15:10.416 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.409+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7419de30 con 0x7f2e740ffb90 2023-07-14T11:15:10.416 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.409+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e74107200 con 0x7f2e740ffb90 2023-07-14T11:15:10.417 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.410+0000 7f2e6affd700 1 -- 172.21.15.182:0/2242826527 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f2e700228e0 con 0x7f2e740ffb90 2023-07-14T11:15:10.417 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.411+0000 7f2e6affd700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2e600a10c0 0x7f2e600a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:10.417 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.411+0000 7f2e6affd700 1 -- 172.21.15.182:0/2242826527 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f2e700cc520 con 0x7f2e740ffb90 2023-07-14T11:15:10.417 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.411+0000 7f2e792d7700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2e600a10c0 0x7f2e600a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:10.417 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.412+0000 7f2e792d7700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2e600a10c0 0x7f2e600a3570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009a10 tx=0x7f2e64009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:15:10.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.415+0000 7f2e6affd700 1 -- 172.21.15.182:0/2242826527 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2e70097670 con 0x7f2e740ffb90 2023-07-14T11:15:10.611 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:10.609+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e74060950 con 0x7f2e600a10c0 2023-07-14T11:15:11.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:15:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:15:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] boot 2023-07-14T11:15:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T11:15:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:11.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:11.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:11.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:11.213 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:15:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:15:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] boot 2023-07-14T11:15:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T11:15:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:11.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:11.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:11.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:10 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:11.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:15:11.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:15:11.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: osd.1 [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] boot 2023-07-14T11:15:11.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T11:15:11.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:15:11.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:15:11.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:11.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:11.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:11.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:10 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:11.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:11 smithi182 ceph-mon[129366]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T11:15:12.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:11 smithi175 ceph-mon[129081]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T11:15:12.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:11 smithi161 ceph-mon[95526]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T11:15:13.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:13 smithi182 ceph-mon[129366]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:13.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:13 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/418006719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]: dispatch 2023-07-14T11:15:13.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:13 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]: dispatch 2023-07-14T11:15:13.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:13 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]': finished 2023-07-14T11:15:13.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:13 smithi182 ceph-mon[129366]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T11:15:13.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:13.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:13 smithi161 ceph-mon[95526]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:13.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:13 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/418006719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]: dispatch 2023-07-14T11:15:13.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:13 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]: dispatch 2023-07-14T11:15:13.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:13 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]': finished 2023-07-14T11:15:13.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:13 smithi161 ceph-mon[95526]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T11:15:13.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:13.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:13 smithi175 ceph-mon[129081]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:13 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/418006719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]: dispatch 2023-07-14T11:15:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:13 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]: dispatch 2023-07-14T11:15:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:13 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cfe220-ef88-4ab3-a40f-57786bafcf56"}]': finished 2023-07-14T11:15:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:13 smithi175 ceph-mon[129081]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T11:15:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:14.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:14 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/2137475089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:15:14.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:14 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/2137475089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:15:14.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:14 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/2137475089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:15:15.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:15 smithi182 ceph-mon[129366]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:15.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:15 smithi161 ceph-mon[95526]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:15.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:15 smithi175 ceph-mon[129081]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:17.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:17 smithi182 ceph-mon[129366]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:17.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:17 smithi161 ceph-mon[95526]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:17.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:17 smithi175 ceph-mon[129081]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:19 smithi182 ceph-mon[129366]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T11:15:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:19.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:19 smithi161 ceph-mon[95526]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T11:15:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:19.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:19 smithi175 ceph-mon[129081]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T11:15:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:20.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:20 smithi182 ceph-mon[129366]: Deploying daemon osd.2 on smithi182 2023-07-14T11:15:20.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:20 smithi161 ceph-mon[95526]: Deploying daemon osd.2 on smithi182 2023-07-14T11:15:20.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:20 smithi175 ceph-mon[129081]: Deploying daemon osd.2 on smithi182 2023-07-14T11:15:21.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:21 smithi182 ceph-mon[129366]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:21.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:21 smithi161 ceph-mon[95526]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:21.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:21 smithi175 ceph-mon[129081]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:23.588 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:23 smithi182 ceph-mon[129366]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:23.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:23 smithi161 ceph-mon[95526]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:23.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:23 smithi175 ceph-mon[129081]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:24.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:24.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:24.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:24.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:24.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:24.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:24.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:25 smithi175 ceph-mon[129081]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:25.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:25.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:25 smithi182 ceph-mon[129366]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:25.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:25 smithi161 ceph-mon[95526]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:25.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:25.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:25.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:25.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:26.427 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.424+0000 7f2e6affd700 1 -- 172.21.15.182:0/2242826527 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2e74060950 con 0x7f2e600a10c0 2023-07-14T11:15:26.427 INFO:teuthology.orchestra.run.smithi182.stdout:Created osd(s) 2 on host 'smithi182' 2023-07-14T11:15:26.429 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2e600a10c0 msgr2=0x7f2e600a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:26.429 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2e600a10c0 0x7f2e600a3570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009a10 tx=0x7f2e64009190 comp rx=0 tx=0).stop 2023-07-14T11:15:26.429 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 msgr2=0x7f2e74195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:26.429 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e74195940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000b5c0 tx=0x7f2e7000f690 comp rx=0 tx=0).stop 2023-07-14T11:15:26.429 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 shutdown_connections 2023-07-14T11:15:26.430 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e74101720 0x7f2e7419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:26.430 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2e600a10c0 0x7f2e600a3570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:26.430 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2e740ffb90 0x7f2e74195940 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:26.430 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 --2- 172.21.15.182:0/2242826527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e74100d90 0x7f2e74195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:26.430 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 >> 172.21.15.182:0/2242826527 conn(0x7f2e740fb1a0 msgr2=0x7f2e74104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:15:26.430 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 shutdown_connections 2023-07-14T11:15:26.430 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:15:26.426+0000 7f2e7bd3c700 1 -- 172.21.15.182:0/2242826527 wait complete. 2023-07-14T11:15:26.904 DEBUG:teuthology.orchestra.run.smithi182:osd.2> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.2.service 2023-07-14T11:15:26.907 INFO:tasks.cephadm:Deploying osd.3 on smithi161 with /dev/vg_nvme/lv_3... 2023-07-14T11:15:26.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T11:15:26.936 INFO:journalctl@ceph.osd.2.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:15:27.599 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:27 smithi182 ceph-mon[129366]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:27.599 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:27.599 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:27.600 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:27.600 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:27 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-2[135157]: 2023-07-14T11:15:27.556+0000 7ff08d701640 -1 osd.2 0 log_to_monitors true 2023-07-14T11:15:27.636 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:15:27.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:27 smithi175 ceph-mon[129081]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:27.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:27 smithi161 ceph-mon[95526]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:28.706 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:28 smithi182 ceph-mon[129366]: from='osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:15:28.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:28 smithi182 ceph-mon[129366]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:15:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:28 smithi161 ceph-mon[95526]: from='osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:15:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:28 smithi161 ceph-mon[95526]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:15:28.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:28 smithi175 ceph-mon[129081]: from='osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:15:28.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:28 smithi175 ceph-mon[129081]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T11:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T11:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T11:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T11:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0314691 s, 333 MB/s 2023-07-14T11:15:29.643 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T11:15:29.714 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-2[135157]: 2023-07-14T11:15:29.449+0000 7ff082d79700 -1 osd.2 0 waiting for initial osdmap 2023-07-14T11:15:29.714 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-2[135157]: 2023-07-14T11:15:29.453+0000 7ff07db6c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:15:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T11:15:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T11:15:29.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:29.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:15:29.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:15:29.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:29.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:29.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:29.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:29.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T11:15:29.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T11:15:29.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:29.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:15:29.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:15:29.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:29.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:29.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:29.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T11:15:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T11:15:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:29.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:15:29.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:15:29.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:29.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:29.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:29.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:29.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:30.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2023-07-14T11:15:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:30 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi182 2023-07-14T11:15:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:30 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 17168M 2023-07-14T11:15:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:30 smithi182 ceph-mon[129366]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:15:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:30 smithi182 ceph-mon[129366]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T11:15:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:30 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:30.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:30 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:30.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:30 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi182 2023-07-14T11:15:30.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:30 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 17168M 2023-07-14T11:15:30.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:30 smithi161 ceph-mon[95526]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:15:30.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:30 smithi161 ceph-mon[95526]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T11:15:30.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:30 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:30.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:30 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:30.828 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:15:30.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:30 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi182 2023-07-14T11:15:30.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:30 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 17168M 2023-07-14T11:15:30.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:30 smithi175 ceph-mon[129081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:15:30.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:30 smithi175 ceph-mon[129081]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T11:15:30.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:30 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:30.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:30 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:31 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:15:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:31 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:15:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:31 smithi182 ceph-mon[129366]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:31 smithi182 ceph-mon[129366]: osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] boot 2023-07-14T11:15:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:31 smithi182 ceph-mon[129366]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T11:15:31.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:31.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:31 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:15:31.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:31 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:15:31.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:31 smithi161 ceph-mon[95526]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:31.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:31.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:31 smithi161 ceph-mon[95526]: osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] boot 2023-07-14T11:15:31.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:31 smithi161 ceph-mon[95526]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T11:15:31.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:31.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:31 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:15:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:31 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:15:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:31 smithi175 ceph-mon[129081]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:15:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:31 smithi175 ceph-mon[129081]: osd.2 [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] boot 2023-07-14T11:15:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:31 smithi175 ceph-mon[129081]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T11:15:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:15:32.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 -- 172.21.15.161:0/3669001429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4101720 msgr2=0x7fa3e4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:32.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/3669001429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4101720 0x7fa3e4107fe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00b0a0 tx=0x7fa3dc00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:15:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 -- 172.21.15.161:0/3669001429 shutdown_connections 2023-07-14T11:15:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/3669001429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e40ffb90 0x7fa3e40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/3669001429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4101720 0x7fa3e4107fe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/3669001429 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e4100d90 0x7fa3e41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 -- 172.21.15.161:0/3669001429 >> 172.21.15.161:0/3669001429 conn(0x7fa3e40fb1a0 msgr2=0x7fa3e40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:15:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 -- 172.21.15.161:0/3669001429 shutdown_connections 2023-07-14T11:15:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.305+0000 7fa3e8b66700 1 -- 172.21.15.161:0/3669001429 wait complete. 2023-07-14T11:15:32.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 Processor -- start 2023-07-14T11:15:32.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 -- start start 2023-07-14T11:15:32.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e40ffb90 0x7fa3e4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:32.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4100d90 0x7fa3e4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:32.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e4101720 0x7fa3e419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:32.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e419f760 con 0x7fa3e4101720 2023-07-14T11:15:32.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e419f8a0 con 0x7fa3e4100d90 2023-07-14T11:15:32.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.306+0000 7fa3e8b66700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e419fa30 con 0x7fa3e40ffb90 2023-07-14T11:15:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e1d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4100d90 0x7fa3e4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e4101720 0x7fa3e419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e40ffb90 0x7fa3e4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e1d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4100d90 0x7fa3e4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33050/0 (socket says 172.21.15.161:33050) 2023-07-14T11:15:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e1d9b700 1 -- 172.21.15.161:0/1297821172 learned_addr learned my addr 172.21.15.161:0/1297821172 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:15:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e40ffb90 0x7fa3e4197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:51280/0 (socket says 172.21.15.161:51280) 2023-07-14T11:15:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e259c700 1 -- 172.21.15.161:0/1297821172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4100d90 msgr2=0x7fa3e4198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:32.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e259c700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4100d90 0x7fa3e4198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:32.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e259c700 1 -- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e4101720 msgr2=0x7fa3e419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:32.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e259c700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e4101720 0x7fa3e419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:32.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e259c700 1 -- 172.21.15.161:0/1297821172 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc009d00 con 0x7fa3e40ffb90 2023-07-14T11:15:32.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.307+0000 7fa3e1d9b700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4100d90 0x7fa3e4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:15:32.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.308+0000 7fa3e259c700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e40ffb90 0x7fa3e4197bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400d750 tx=0x7fa3d400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:15:32.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.308+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400f690 con 0x7fa3e40ffb90 2023-07-14T11:15:32.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.308+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e419fd10 con 0x7fa3e40ffb90 2023-07-14T11:15:32.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.308+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3d400fcb0 con 0x7fa3e40ffb90 2023-07-14T11:15:32.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.308+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4010df0 con 0x7fa3e40ffb90 2023-07-14T11:15:32.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.308+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e41a01b0 con 0x7fa3e40ffb90 2023-07-14T11:15:32.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.309+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e4192270 con 0x7fa3e40ffb90 2023-07-14T11:15:32.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.311+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7fa3d400f7f0 con 0x7fa3e40ffb90 2023-07-14T11:15:32.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.312+0000 7fa3d37fe700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3cc0a1140 0x7fa3cc0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.313+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fa3d40cb2f0 con 0x7fa3e40ffb90 2023-07-14T11:15:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.313+0000 7fa3e1d9b700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3cc0a1140 0x7fa3cc0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.313+0000 7fa3e1d9b700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3cc0a1140 0x7fa3cc0a35f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8011550 tx=0x7fa3d8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:15:32.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.316+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa3d4095fd0 con 0x7fa3e40ffb90 2023-07-14T11:15:32.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:32.529+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3e4060950 con 0x7fa3cc0a1140 2023-07-14T11:15:32.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:32 smithi175 ceph-mon[129081]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T11:15:32.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:32 smithi182 ceph-mon[129366]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T11:15:32.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:32 smithi161 ceph-mon[95526]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T11:15:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:15:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:33.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:33.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T11:15:33.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T11:15:33.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:15:33.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:15:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:33.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T11:15:33.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T11:15:33.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:15:33.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:33.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:15:33.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:33.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:33.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:33.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:33.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T11:15:33.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T11:15:33.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:15:34.651 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111641]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:15:34.651 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111641]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:34.651 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111641]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:34.652 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111641]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:34.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:34.654+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa3d40963a0 con 0x7fa3e40ffb90 2023-07-14T11:15:34.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:15:34.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:34 smithi175 ceph-mon[129081]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T11:15:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:15:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:34 smithi182 ceph-mon[129366]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T11:15:34.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:15:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:34 smithi161 ceph-mon[95526]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T11:15:34.996 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111689]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:15:34.996 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111689]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:34.996 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111689]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:34.996 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:15:34 smithi161 sudo[111689]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:35.354 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136525]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:15:35.354 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136525]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:35.354 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136525]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:35.355 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136525]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:35.650 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136528]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:15:35.650 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136528]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:35.650 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136528]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:35.650 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:15:35 smithi175 sudo[136528]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:35.878 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:35 smithi182 ceph-mon[129366]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:35.878 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:35 smithi182 ceph-mon[129366]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T11:15:35.878 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:35 smithi182 ceph-mon[129366]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T11:15:35.879 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:35 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:35.879 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:35 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:35.879 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137387]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:15:35.879 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137387]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:35.879 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137387]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:35.880 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137387]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:35.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:35 smithi175 ceph-mon[129081]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:35.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:35 smithi175 ceph-mon[129081]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T11:15:35.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:35 smithi175 ceph-mon[129081]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T11:15:35.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:35 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:35.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:35 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:35.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:35 smithi161 ceph-mon[95526]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:35 smithi161 ceph-mon[95526]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T11:15:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:35 smithi161 ceph-mon[95526]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T11:15:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:35 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:35 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:36.214 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137390]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:15:36.214 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:36.214 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137390]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:36.214 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:15:35 smithi182 sudo[137390]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:36.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111769]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:15:36.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:36.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111769]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:36.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111769]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:37.228 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:36 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1260845768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]: dispatch 2023-07-14T11:15:37.229 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:36 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]: dispatch 2023-07-14T11:15:37.229 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:36 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]': finished 2023-07-14T11:15:37.229 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:36 smithi182 ceph-mon[129366]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T11:15:37.229 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:37.229 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:37.229 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:36 smithi182 ceph-mon[129366]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:37.230 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137393]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:15:37.230 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:37.230 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137393]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:37.230 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137393]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:37.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111823]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:15:37.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111823]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 sudo[111823]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1260845768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]: dispatch 2023-07-14T11:15:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]: dispatch 2023-07-14T11:15:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]': finished 2023-07-14T11:15:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 ceph-mon[95526]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T11:15:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:36 smithi161 ceph-mon[95526]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:37.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:36 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1260845768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]: dispatch 2023-07-14T11:15:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:36 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]: dispatch 2023-07-14T11:15:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:36 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24ac073f-086c-4626-8080-a901f73a1397"}]': finished 2023-07-14T11:15:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:36 smithi175 ceph-mon[129081]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T11:15:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:36 smithi175 ceph-mon[129081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:37.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137396]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:15:37.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:37.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137396]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:37.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 sudo[137396]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:37.959 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136531]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:15:37.960 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136531]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:37.960 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136531]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:37.960 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136531]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:38.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/962396658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:15:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:37 smithi161 ceph-mon[95526]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136534]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:15:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:15:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136534]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:15:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 sudo[136534]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:15:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/962396658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:38.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:37 smithi175 ceph-mon[129081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/962396658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:15:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:15:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:15:38.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:15:38.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:37 smithi182 ceph-mon[129366]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:15:39.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:38 smithi161 ceph-mon[95526]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:39.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:38 smithi175 ceph-mon[129081]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:39.463 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:38 smithi182 ceph-mon[129366]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:41.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:41 smithi175 ceph-mon[129081]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:41.459 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:41 smithi161 ceph-mon[95526]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:41.463 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:41 smithi182 ceph-mon[129366]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:43 smithi175 ceph-mon[129081]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-14T11:15:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T11:15:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:43.956 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:43 smithi161 ceph-mon[95526]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-14T11:15:43.956 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T11:15:43.956 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:43 smithi182 ceph-mon[129366]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-14T11:15:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T11:15:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:44.876 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:44 smithi161 ceph-mon[95526]: Deploying daemon osd.3 on smithi161 2023-07-14T11:15:44.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:44 smithi175 ceph-mon[129081]: Deploying daemon osd.3 on smithi161 2023-07-14T11:15:44.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:44 smithi182 ceph-mon[129366]: Deploying daemon osd.3 on smithi161 2023-07-14T11:15:45.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:45 smithi175 ceph-mon[129081]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-07-14T11:15:45.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:45 smithi182 ceph-mon[129366]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-07-14T11:15:45.992 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:45 smithi161 ceph-mon[95526]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-07-14T11:15:47.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:47 smithi175 ceph-mon[129081]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-07-14T11:15:47.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:47 smithi161 ceph-mon[95526]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-07-14T11:15:47.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:47 smithi182 ceph-mon[129366]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-07-14T11:15:49.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:49 smithi175 ceph-mon[129081]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:49.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:49 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:49.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:49 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:49.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:49 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:49.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:49 smithi182 ceph-mon[129366]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:49.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:49 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:49.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:49 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:49.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:49 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:49.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:49 smithi161 ceph-mon[95526]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:49.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:49 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:49.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:49 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:49.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:49 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:51.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:51 smithi175 ceph-mon[129081]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:51.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:51 smithi182 ceph-mon[129366]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:51.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:51 smithi161 ceph-mon[95526]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.798+0000 7fa3d37fe700 1 -- 172.21.15.161:0/1297821172 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa3e4060950 con 0x7fa3cc0a1140 2023-07-14T11:15:52.801 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 3 on host 'smithi161' 2023-07-14T11:15:52.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3cc0a1140 msgr2=0x7fa3cc0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3cc0a1140 0x7fa3cc0a35f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8011550 tx=0x7fa3d8009260 comp rx=0 tx=0).stop 2023-07-14T11:15:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e40ffb90 msgr2=0x7fa3e4197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e40ffb90 0x7fa3e4197bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400d750 tx=0x7fa3d400da60 comp rx=0 tx=0).stop 2023-07-14T11:15:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 shutdown_connections 2023-07-14T11:15:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e4101720 0x7fa3e419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3cc0a1140 0x7fa3cc0a35f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:52.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3e40ffb90 0x7fa3e4197bd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:52.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 --2- 172.21.15.161:0/1297821172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3e4100d90 0x7fa3e4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:52.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 >> 172.21.15.161:0/1297821172 conn(0x7fa3e40fb1a0 msgr2=0x7fa3e4106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:15:52.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 shutdown_connections 2023-07-14T11:15:52.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:15:52.801+0000 7fa3e8b66700 1 -- 172.21.15.161:0/1297821172 wait complete. 2023-07-14T11:15:53.341 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:53.341 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.341 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.341 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:53.342 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:53.342 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.342 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:53.342 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.342 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.356 DEBUG:teuthology.orchestra.run.smithi161:osd.3> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.3.service 2023-07-14T11:15:53.360 INFO:tasks.cephadm:Deploying osd.4 on smithi175 with /dev/vg_nvme/lv_3... 2023-07-14T11:15:53.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T11:15:53.392 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:53.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:53.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:53.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:53.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.394 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.437 INFO:journalctl@ceph.osd.3.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:15:53.463 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-14T11:15:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:15:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:53.921 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:15:54.206 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:54 smithi161 ceph-mon[95526]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T11:15:54.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:54 smithi175 ceph-mon[129081]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T11:15:54.463 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:54 smithi182 ceph-mon[129366]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T11:15:55.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:55 smithi175 ceph-mon[129081]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:55.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:55 smithi175 ceph-mon[129081]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T11:15:55.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:55 smithi175 ceph-mon[129081]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T11:15:55.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:55 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:55.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:55 smithi175 ceph-mon[129081]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:15:55.431 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T11:15:55.431 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T11:15:55.431 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2023-07-14T11:15:55.431 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2023-07-14T11:15:55.431 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126808 s, 827 MB/s 2023-07-14T11:15:55.431 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2023-07-14T11:15:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:55 smithi182 ceph-mon[129366]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:55 smithi182 ceph-mon[129366]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T11:15:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:55 smithi182 ceph-mon[129366]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T11:15:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:55 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:55.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:55 smithi182 ceph-mon[129366]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:15:55.495 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:15:55 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:15:55.076+0000 7fda7f00e700 -1 osd.3 0 waiting for initial osdmap 2023-07-14T11:15:55.496 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:15:55 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:15:55.084+0000 7fda78dff700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:15:55.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:55 smithi161 ceph-mon[95526]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:55.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:55 smithi161 ceph-mon[95526]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T11:15:55.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:55 smithi161 ceph-mon[95526]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T11:15:55.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:55 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:55.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:55 smithi161 ceph-mon[95526]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:15:55.762 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2023-07-14T11:15:56.353 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:15:56.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:56 smithi175 ceph-mon[129081]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:15:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:56 smithi175 ceph-mon[129081]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T11:15:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:56 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:56 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:56.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:56 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:56 smithi182 ceph-mon[129366]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:15:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:56 smithi182 ceph-mon[129366]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T11:15:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:56 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:56 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:56.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:56 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:56.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:56 smithi161 ceph-mon[95526]: from='osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:15:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:56 smithi161 ceph-mon[95526]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T11:15:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:56 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:56 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:56 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:15:57.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:15:57.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:57.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi161 2023-07-14T11:15:57.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] boot 2023-07-14T11:15:57.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T11:15:57.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:57.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:57.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:57.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 8633M 2023-07-14T11:15:57.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:57.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:57.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:57 smithi175 ceph-mon[129081]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T11:15:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:15:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:15:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi161 2023-07-14T11:15:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] boot 2023-07-14T11:15:57.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T11:15:57.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:57.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:57.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:57.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 8633M 2023-07-14T11:15:57.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:57.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:57.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:57 smithi182 ceph-mon[129366]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T11:15:57.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:15:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:15:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:15:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi161 2023-07-14T11:15:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: osd.3 [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] boot 2023-07-14T11:15:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T11:15:57.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:15:57.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:57.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:15:57.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 8633M 2023-07-14T11:15:57.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:57.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:15:57.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:15:57.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:57 smithi161 ceph-mon[95526]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T11:15:57.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 -- 172.21.15.175:0/3566138355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 msgr2=0x7eff3c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:57.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 --2- 172.21.15.175:0/3566138355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c0fe260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7eff24009990 tx=0x7eff24009ca0 comp rx=0 tx=0).stop 2023-07-14T11:15:57.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 -- 172.21.15.175:0/3566138355 shutdown_connections 2023-07-14T11:15:57.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 --2- 172.21.15.175:0/3566138355 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7eff3c1017c0 0x7eff3c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:57.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 --2- 172.21.15.175:0/3566138355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c0fe260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:57.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 --2- 172.21.15.175:0/3566138355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff3c100e30 0x7eff3c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:57.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 -- 172.21.15.175:0/3566138355 >> 172.21.15.175:0/3566138355 conn(0x7eff3c0f9860 msgr2=0x7eff3c0fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:15:57.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 -- 172.21.15.175:0/3566138355 shutdown_connections 2023-07-14T11:15:57.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.512+0000 7eff4244e700 1 -- 172.21.15.175:0/3566138355 wait complete. 2023-07-14T11:15:57.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 Processor -- start 2023-07-14T11:15:57.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 -- start start 2023-07-14T11:15:57.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:57.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff3c100e30 0x7eff3c195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:57.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7eff3c1017c0 0x7eff3c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:57.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19d520 con 0x7eff3c100e30 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19d660 con 0x7eff3c0fde50 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff4244e700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19d7f0 con 0x7eff3c1017c0 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff3bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff3bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58360/0 (socket says 172.21.15.175:58360) 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.513+0000 7eff3bfff700 1 -- 172.21.15.175:0/2980193950 learned_addr learned my addr 172.21.15.175:0/2980193950 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff3bfff700 1 -- 172.21.15.175:0/2980193950 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7eff3c1017c0 msgr2=0x7eff3c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff3bfff700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7eff3c1017c0 0x7eff3c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff3bfff700 1 -- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff3c100e30 msgr2=0x7eff3c195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:15:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff3bfff700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff3c100e30 0x7eff3c195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:15:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff3bfff700 1 -- 172.21.15.175:0/2980193950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff240095f0 con 0x7eff3c0fde50 2023-07-14T11:15:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff3bfff700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c195990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff2400f730 tx=0x7eff2400f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:15:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff397fa700 1 -- 172.21.15.175:0/2980193950 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2401d070 con 0x7eff3c0fde50 2023-07-14T11:15:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff397fa700 1 -- 172.21.15.175:0/2980193950 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2400fd30 con 0x7eff3c0fde50 2023-07-14T11:15:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c19da70 con 0x7eff3c0fde50 2023-07-14T11:15:57.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff397fa700 1 -- 172.21.15.175:0/2980193950 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24005710 con 0x7eff3c0fde50 2023-07-14T11:15:57.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.514+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c19de80 con 0x7eff3c0fde50 2023-07-14T11:15:57.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.515+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c107250 con 0x7eff3c0fde50 2023-07-14T11:15:57.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.517+0000 7eff397fa700 1 -- 172.21.15.175:0/2980193950 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7eff240047e0 con 0x7eff3c0fde50 2023-07-14T11:15:57.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.517+0000 7eff397fa700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7eff280a1150 0x7eff280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:15:57.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.517+0000 7eff397fa700 1 -- 172.21.15.175:0/2980193950 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7eff240cce90 con 0x7eff3c0fde50 2023-07-14T11:15:57.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.518+0000 7eff3b7fe700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7eff280a1150 0x7eff280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:15:57.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.518+0000 7eff3b7fe700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7eff280a1150 0x7eff280a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0097b0 tx=0x7eff2c006d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:15:57.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.521+0000 7eff397fa700 1 -- 172.21.15.175:0/2980193950 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7eff24097870 con 0x7eff3c0fde50 2023-07-14T11:15:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:15:57.714+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7eff3c060950 con 0x7eff280a1150 2023-07-14T11:15:58.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:58 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:58.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:58 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:58.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:58 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:58.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:58 smithi175 ceph-mon[129081]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T11:15:58.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:58 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:58.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:58 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:58.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:58 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:58.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:58 smithi182 ceph-mon[129366]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T11:15:58.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:58 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:15:58.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:58 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:15:58.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:58 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:15:58.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:58 smithi161 ceph-mon[95526]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T11:15:59.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:59 smithi175 ceph-mon[129081]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:59.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:59 smithi175 ceph-mon[129081]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:15:59.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:15:59 smithi175 ceph-mon[129081]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T11:15:59.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:59 smithi182 ceph-mon[129366]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:59.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:59 smithi182 ceph-mon[129366]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:15:59.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:15:59 smithi182 ceph-mon[129366]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T11:15:59.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:59 smithi161 ceph-mon[95526]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:15:59.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:59 smithi161 ceph-mon[95526]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:15:59.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:15:59 smithi161 ceph-mon[95526]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T11:16:01.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T11:16:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/3839926380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]: dispatch 2023-07-14T11:16:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]: dispatch 2023-07-14T11:16:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]': finished 2023-07-14T11:16:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T11:16:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/2735814767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T11:16:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/3839926380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]: dispatch 2023-07-14T11:16:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]: dispatch 2023-07-14T11:16:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]': finished 2023-07-14T11:16:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T11:16:01.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:01.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/2735814767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:01.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T11:16:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/3839926380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]: dispatch 2023-07-14T11:16:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]: dispatch 2023-07-14T11:16:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4e7501f-63ff-4f01-ade2-136ab260607c"}]': finished 2023-07-14T11:16:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T11:16:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:01.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/2735814767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:03.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:03 smithi175 ceph-mon[129081]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:16:03.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:03 smithi182 ceph-mon[129366]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:16:03.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:03 smithi161 ceph-mon[95526]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:16:05.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:05 smithi175 ceph-mon[129081]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T11:16:05.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:05 smithi182 ceph-mon[129366]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T11:16:05.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:05 smithi161 ceph-mon[95526]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T11:16:06.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T11:16:06.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T11:16:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:06.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T11:16:06.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:07.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:07 smithi175 ceph-mon[129081]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-07-14T11:16:07.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:07 smithi175 ceph-mon[129081]: Deploying daemon osd.4 on smithi175 2023-07-14T11:16:07.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:07 smithi182 ceph-mon[129366]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-07-14T11:16:07.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:07 smithi182 ceph-mon[129366]: Deploying daemon osd.4 on smithi175 2023-07-14T11:16:07.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:07 smithi161 ceph-mon[95526]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-07-14T11:16:07.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:07 smithi161 ceph-mon[95526]: Deploying daemon osd.4 on smithi175 2023-07-14T11:16:09.893 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:09 smithi175 ceph-mon[129081]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:16:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:09 smithi182 ceph-mon[129366]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:16:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:09 smithi161 ceph-mon[95526]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:16:11.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:11 smithi175 ceph-mon[129081]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:16:11.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:11.752 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:11.752 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:11.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:11 smithi182 ceph-mon[129366]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:16:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:11.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:11 smithi161 ceph-mon[95526]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:16:11.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:13.329 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 4 on host 'smithi175' 2023-07-14T11:16:13.329 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.326+0000 7eff397fa700 1 -- 172.21.15.175:0/2980193950 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff3c060950 con 0x7eff280a1150 2023-07-14T11:16:13.331 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7eff280a1150 msgr2=0x7eff280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:13.331 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7eff280a1150 0x7eff280a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0097b0 tx=0x7eff2c006d20 comp rx=0 tx=0).stop 2023-07-14T11:16:13.331 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 msgr2=0x7eff3c195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:13.331 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c195990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff2400f730 tx=0x7eff2400f810 comp rx=0 tx=0).stop 2023-07-14T11:16:13.332 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 shutdown_connections 2023-07-14T11:16:13.332 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7eff280a1150 0x7eff280a3600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:13.332 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7eff3c1017c0 0x7eff3c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:13.332 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff3c0fde50 0x7eff3c195990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:13.332 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 --2- 172.21.15.175:0/2980193950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff3c100e30 0x7eff3c195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:13.332 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 >> 172.21.15.175:0/2980193950 conn(0x7eff3c0f9860 msgr2=0x7eff3c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:16:13.332 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 shutdown_connections 2023-07-14T11:16:13.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:16:13.328+0000 7eff4244e700 1 -- 172.21.15.175:0/2980193950 wait complete. 2023-07-14T11:16:13.638 DEBUG:teuthology.orchestra.run.smithi175:osd.4> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.4.service 2023-07-14T11:16:13.641 INFO:tasks.cephadm:Deploying osd.5 on smithi182 with /dev/vg_nvme/lv_3... 2023-07-14T11:16:13.641 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T11:16:13.670 INFO:journalctl@ceph.osd.4.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:16:13.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:16:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:16:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:13.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:13.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:16:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:13.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:14.206 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:16:14.655 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:16:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:16:14.590+0000 7fc8456ee640 -1 osd.4 0 log_to_monitors true 2023-07-14T11:16:14.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:14 smithi182 ceph-mon[129366]: from='osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:16:14.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:14 smithi182 ceph-mon[129366]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:16:14.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:14 smithi175 ceph-mon[129081]: from='osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:16:14.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:14 smithi175 ceph-mon[129081]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:16:14.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:14 smithi161 ceph-mon[95526]: from='osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:16:14.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:14 smithi161 ceph-mon[95526]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:16:15.671 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T11:16:15.671 INFO:teuthology.orchestra.run.smithi182.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T11:16:15.672 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 10+0 records in 2023-07-14T11:16:15.672 INFO:teuthology.orchestra.run.smithi182.stderr:10+0 records out 2023-07-14T11:16:15.672 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124277 s, 844 MB/s 2023-07-14T11:16:15.672 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping successful for: 2023-07-14T11:16:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:15 smithi182 ceph-mon[129366]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:15 smithi182 ceph-mon[129366]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T11:16:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:15 smithi182 ceph-mon[129366]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T11:16:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:15 smithi182 ceph-mon[129366]: from='osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:16:15.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:15 smithi182 ceph-mon[129366]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:16:15.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:15 smithi161 ceph-mon[95526]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:15 smithi161 ceph-mon[95526]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T11:16:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:15 smithi161 ceph-mon[95526]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T11:16:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:15 smithi161 ceph-mon[95526]: from='osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:16:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:15 smithi161 ceph-mon[95526]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:16:16.044 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi182:vg_nvme/lv_3 2023-07-14T11:16:16.168 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:16:15.679+0000 7fc83ad66700 -1 osd.4 0 waiting for initial osdmap 2023-07-14T11:16:16.169 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:16:15.685+0000 7fc835b59700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:16:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-mon[129081]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-mon[129081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T11:16:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-mon[129081]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T11:16:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-mon[129081]: from='osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:16:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:15 smithi175 ceph-mon[129081]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:16:16.596 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:16:16.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:16:16.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T11:16:16.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:16.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:16.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi175 2023-07-14T11:16:16.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:16.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:16.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 8584M 2023-07-14T11:16:16.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:16.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:16.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:16 smithi182 ceph-mon[129366]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:16.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:16:16.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T11:16:16.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:16.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:16.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi175 2023-07-14T11:16:16.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 8584M 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:16.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:16 smithi161 ceph-mon[95526]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:17.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:16:17.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T11:16:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi175 2023-07-14T11:16:17.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:17.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:17.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:17.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:17.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 8584M 2023-07-14T11:16:17.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:17.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:17.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:17.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:17.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:16 smithi175 ceph-mon[129081]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:16:17.810 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 -- 172.21.15.182:0/3128603217 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b34100db0 msgr2=0x7f0b34101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:17.810 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3128603217 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b34100db0 0x7f0b34101200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009990 tx=0x7f0b30009ca0 comp rx=0 tx=0).stop 2023-07-14T11:16:17.810 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 -- 172.21.15.182:0/3128603217 shutdown_connections 2023-07-14T11:16:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3128603217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b34101740 0x7f0b34108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3128603217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b340ffbb0 0x7f0b340fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3128603217 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b34100db0 0x7f0b34101200 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 -- 172.21.15.182:0/3128603217 >> 172.21.15.182:0/3128603217 conn(0x7f0b340fb1c0 msgr2=0x7f0b340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:16:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 -- 172.21.15.182:0/3128603217 shutdown_connections 2023-07-14T11:16:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.806+0000 7f0b3c754700 1 -- 172.21.15.182:0/3128603217 wait complete. 2023-07-14T11:16:17.811 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 Processor -- start 2023-07-14T11:16:17.812 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 -- start start 2023-07-14T11:16:17.812 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b340ffbb0 0x7f0b34195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:17.812 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b34100db0 0x7f0b34195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:17.812 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34101740 0x7f0b3419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:17.812 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419d3e0 con 0x7f0b34100db0 2023-07-14T11:16:17.812 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419d520 con 0x7f0b34101740 2023-07-14T11:16:17.812 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3c754700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419d6b0 con 0x7f0b340ffbb0 2023-07-14T11:16:17.813 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3a4f0700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b340ffbb0 0x7f0b34195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:17.813 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3a4f0700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b340ffbb0 0x7f0b34195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.182:37022/0 (socket says 172.21.15.182:37022) 2023-07-14T11:16:17.813 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.807+0000 7f0b3a4f0700 1 -- 172.21.15.182:0/3199831839 learned_addr learned my addr 172.21.15.182:0/3199831839 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:16:17.813 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b39cef700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b34100db0 0x7f0b34195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:17.813 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3acf1700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34101740 0x7f0b3419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:17.813 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3a4f0700 1 -- 172.21.15.182:0/3199831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34101740 msgr2=0x7f0b3419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:17.813 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3a4f0700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34101740 0x7f0b3419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3a4f0700 1 -- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b34100db0 msgr2=0x7f0b34195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3a4f0700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b34100db0 0x7f0b34195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3a4f0700 1 -- 172.21.15.182:0/3199831839 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300095f0 con 0x7f0b340ffbb0 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b39cef700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b34100db0 0x7f0b34195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3a4f0700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b340ffbb0 0x7f0b34195850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800d9b0 tx=0x7f0b2800dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b277fe700 1 -- 172.21.15.182:0/3199831839 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800bb60 con 0x7f0b340ffbb0 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b277fe700 1 -- 172.21.15.182:0/3199831839 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b28007620 con 0x7f0b340ffbb0 2023-07-14T11:16:17.814 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b277fe700 1 -- 172.21.15.182:0/3199831839 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800f620 con 0x7f0b340ffbb0 2023-07-14T11:16:17.815 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3419d990 con 0x7f0b340ffbb0 2023-07-14T11:16:17.815 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.808+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3419de40 con 0x7f0b340ffbb0 2023-07-14T11:16:17.816 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.809+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3410b370 con 0x7f0b340ffbb0 2023-07-14T11:16:17.816 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.810+0000 7f0b277fe700 1 -- 172.21.15.182:0/3199831839 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0b28010460 con 0x7f0b340ffbb0 2023-07-14T11:16:17.816 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.810+0000 7f0b277fe700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0b200a1150 0x7f0b200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:17.816 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.811+0000 7f0b277fe700 1 -- 172.21.15.182:0/3199831839 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f0b280cc010 con 0x7f0b340ffbb0 2023-07-14T11:16:17.816 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.811+0000 7f0b39cef700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0b200a1150 0x7f0b200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:17.816 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.812+0000 7f0b39cef700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0b200a1150 0x7f0b200a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000b5c0 tx=0x7f0b30005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:16:17.817 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:17.814+0000 7f0b277fe700 1 -- 172.21.15.182:0/3199831839 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0b280967d0 con 0x7f0b340ffbb0 2023-07-14T11:16:17.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:17 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:16:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:17 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:16:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:17 smithi182 ceph-mon[129366]: osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] boot 2023-07-14T11:16:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:17 smithi182 ceph-mon[129366]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T11:16:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:17 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:16:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:17 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:16:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:17 smithi161 ceph-mon[95526]: osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] boot 2023-07-14T11:16:17.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:17 smithi161 ceph-mon[95526]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T11:16:17.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:18.007 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:18.005+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b34060950 con 0x7f0b200a1150 2023-07-14T11:16:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:17 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:16:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:17 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:16:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:17 smithi175 ceph-mon[129081]: osd.4 [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] boot 2023-07-14T11:16:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:17 smithi175 ceph-mon[129081]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T11:16:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:16:19.073 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:18 smithi182 ceph-mon[129366]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T11:16:19.073 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:18 smithi182 ceph-mon[129366]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:16:19.074 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:18 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:16:19.074 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:18 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:16:19.074 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:18 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:19.074 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:18 smithi182 ceph-mon[129366]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:19.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:18 smithi175 ceph-mon[129081]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T11:16:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:18 smithi175 ceph-mon[129081]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:16:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:18 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:16:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:18 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:16:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:18 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:18 smithi175 ceph-mon[129081]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:19.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:18 smithi161 ceph-mon[95526]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T11:16:19.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:18 smithi161 ceph-mon[95526]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:16:19.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:18 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:16:19.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:18 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:16:19.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:18 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:19.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:18 smithi161 ceph-mon[95526]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:21 smithi175 ceph-mon[129081]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:21 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/1142447903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]: dispatch 2023-07-14T11:16:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:21 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]: dispatch 2023-07-14T11:16:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:21 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]': finished 2023-07-14T11:16:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:21 smithi175 ceph-mon[129081]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T11:16:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:21.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:21 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/4226378090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:21.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:21 smithi182 ceph-mon[129366]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:21.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:21 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/1142447903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]: dispatch 2023-07-14T11:16:21.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:21 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]: dispatch 2023-07-14T11:16:21.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:21 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]': finished 2023-07-14T11:16:21.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:21 smithi182 ceph-mon[129366]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T11:16:21.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:21.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:21 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/4226378090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:21.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:21 smithi161 ceph-mon[95526]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:21.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:21 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/1142447903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]: dispatch 2023-07-14T11:16:21.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:21 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]: dispatch 2023-07-14T11:16:21.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:21 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "228b77ca-2772-40c5-8556-030d1b432280"}]': finished 2023-07-14T11:16:21.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:21 smithi161 ceph-mon[95526]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T11:16:21.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:21.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:21 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/4226378090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:23.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:23 smithi175 ceph-mon[129081]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:23 smithi182 ceph-mon[129366]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:23.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:23 smithi161 ceph-mon[95526]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:25 smithi175 ceph-mon[129081]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:25.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:25 smithi182 ceph-mon[129366]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:25.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:25 smithi161 ceph-mon[95526]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:27.905 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:27 smithi182 ceph-mon[129366]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:27.906 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T11:16:27.906 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:27.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:27 smithi175 ceph-mon[129081]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:27.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T11:16:27.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:27.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:27 smithi161 ceph-mon[95526]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:27.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T11:16:27.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:28.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:28 smithi182 ceph-mon[129366]: Deploying daemon osd.5 on smithi182 2023-07-14T11:16:28.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:28 smithi175 ceph-mon[129081]: Deploying daemon osd.5 on smithi182 2023-07-14T11:16:28.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:28 smithi161 ceph-mon[95526]: Deploying daemon osd.5 on smithi182 2023-07-14T11:16:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:29 smithi182 ceph-mon[129366]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:29.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:29 smithi175 ceph-mon[129081]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:29.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:29 smithi161 ceph-mon[95526]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:31.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:31 smithi175 ceph-mon[129081]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:31.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:31 smithi182 ceph-mon[129366]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:31.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:31.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:31.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:31.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:31 smithi161 ceph-mon[95526]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:31.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:31.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:31.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:33 smithi175 ceph-mon[129081]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:33 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.940 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:33 smithi182 ceph-mon[129366]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:33.940 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.940 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.941 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:33.941 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:33.941 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:33 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:33 smithi161 ceph-mon[95526]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:33 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:34.024 INFO:teuthology.orchestra.run.smithi182.stdout:Created osd(s) 5 on host 'smithi182' 2023-07-14T11:16:34.024 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.021+0000 7f0b277fe700 1 -- 172.21.15.182:0/3199831839 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0b34060950 con 0x7f0b200a1150 2023-07-14T11:16:34.026 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.023+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0b200a1150 msgr2=0x7f0b200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:34.026 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.023+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0b200a1150 0x7f0b200a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000b5c0 tx=0x7f0b30005fb0 comp rx=0 tx=0).stop 2023-07-14T11:16:34.026 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.023+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b340ffbb0 msgr2=0x7f0b34195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:34.027 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.023+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b340ffbb0 0x7f0b34195850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800d9b0 tx=0x7f0b2800dd70 comp rx=0 tx=0).stop 2023-07-14T11:16:34.027 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 shutdown_connections 2023-07-14T11:16:34.027 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0b200a1150 0x7f0b200a3600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:34.027 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b34101740 0x7f0b3419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:34.027 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0b340ffbb0 0x7f0b34195850 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:34.027 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 --2- 172.21.15.182:0/3199831839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b34100db0 0x7f0b34195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:34.027 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 >> 172.21.15.182:0/3199831839 conn(0x7f0b340fb1c0 msgr2=0x7f0b340fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:16:34.028 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 shutdown_connections 2023-07-14T11:16:34.028 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:16:34.024+0000 7f0b3c754700 1 -- 172.21.15.182:0/3199831839 wait complete. 2023-07-14T11:16:34.490 DEBUG:teuthology.orchestra.run.smithi182:osd.5> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.5.service 2023-07-14T11:16:34.492 INFO:tasks.cephadm:Deploying osd.6 on smithi161 with /dev/vg_nvme/lv_2... 2023-07-14T11:16:34.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T11:16:34.533 INFO:journalctl@ceph.osd.5.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:16:35.121 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:35.121 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:35.122 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:35.122 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:34 smithi182 ceph-mon[129366]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:35.235 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:16:35.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:35.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:35.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:35.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:34 smithi161 ceph-mon[95526]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:35.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:35.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:35.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:35.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:34 smithi175 ceph-mon[129081]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:35.463 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:16:35 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:16:35.332+0000 7fa676c0b640 -1 osd.5 0 log_to_monitors true 2023-07-14T11:16:36.057 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:35 smithi161 ceph-mon[95526]: from='osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:16:36.057 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:35 smithi161 ceph-mon[95526]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:16:36.237 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:35 smithi182 ceph-mon[129366]: from='osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:16:36.237 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:35 smithi182 ceph-mon[129366]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:16:36.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:35 smithi175 ceph-mon[129081]: from='osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:16:36.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:35 smithi175 ceph-mon[129081]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:16:37.091 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T11:16:37.091 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T11:16:37.091 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T11:16:37.092 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T11:16:37.092 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0298694 s, 351 MB/s 2023-07-14T11:16:37.092 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T11:16:37.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T11:16:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T11:16:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:16:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:16:37.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi182 2023-07-14T11:16:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 8584M 2023-07-14T11:16:37.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:37.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:37.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T11:16:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T11:16:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:16:37.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:16:37.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:37.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi182 2023-07-14T11:16:37.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:37.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:37.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 8584M 2023-07-14T11:16:37.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:37.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:37.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.464 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:16:36.994+0000 7fa66c283700 -1 osd.5 0 waiting for initial osdmap 2023-07-14T11:16:37.464 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:16:37.000+0000 7fa667877700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:16:37.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T11:16:37.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T11:16:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:16:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:16:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:16:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi182 2023-07-14T11:16:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:37.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:16:37.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 8584M 2023-07-14T11:16:37.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:37.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:16:37.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:37.614 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2023-07-14T11:16:38.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:37 smithi161 ceph-mon[95526]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:16:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:37 smithi161 ceph-mon[95526]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T11:16:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:37 smithi161 ceph-mon[95526]: osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] boot 2023-07-14T11:16:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:37 smithi161 ceph-mon[95526]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T11:16:38.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:37 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.338 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:16:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:37 smithi175 ceph-mon[129081]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:16:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:37 smithi175 ceph-mon[129081]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T11:16:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:37 smithi175 ceph-mon[129081]: osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] boot 2023-07-14T11:16:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:37 smithi175 ceph-mon[129081]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T11:16:38.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:37 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-mon[129366]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:16:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-mon[129366]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T11:16:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-mon[129366]: osd.5 [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] boot 2023-07-14T11:16:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-mon[129366]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T11:16:38.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:37 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:16:39.086 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:38 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:16:39.087 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:38 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:16:39.087 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:38 smithi161 ceph-mon[95526]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:39.087 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:38 smithi161 ceph-mon[95526]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T11:16:39.418 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:38 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:16:39.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:38 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:16:39.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:38 smithi175 ceph-mon[129081]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:39.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:38 smithi175 ceph-mon[129081]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T11:16:39.463 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:38 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:16:39.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:38 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:16:39.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:38 smithi182 ceph-mon[129366]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:39.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:38 smithi182 ceph-mon[129366]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T11:16:39.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 -- 172.21.15.161:0/2380954640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 msgr2=0x7fbfe41011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:39.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/2380954640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe41011e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009990 tx=0x7fbfd8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:16:39.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 -- 172.21.15.161:0/2380954640 shutdown_connections 2023-07-14T11:16:39.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/2380954640 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfe4101720 0x7fbfe4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/2380954640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfe40ffb90 0x7fbfe40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/2380954640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe41011e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 -- 172.21.15.161:0/2380954640 >> 172.21.15.161:0/2380954640 conn(0x7fbfe40fb1a0 msgr2=0x7fbfe40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:16:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 -- 172.21.15.161:0/2380954640 shutdown_connections 2023-07-14T11:16:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.729+0000 7fbfe9b96700 1 -- 172.21.15.161:0/2380954640 wait complete. 2023-07-14T11:16:39.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.730+0000 7fbfe9b96700 1 Processor -- start 2023-07-14T11:16:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.730+0000 7fbfe9b96700 1 -- start start 2023-07-14T11:16:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe9b96700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfe40ffb90 0x7fbfe4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe9b96700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe9b96700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfe4101720 0x7fbfe419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:39.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe9b96700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe419f670 con 0x7fbfe4100d90 2023-07-14T11:16:39.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe9b96700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe419f7b0 con 0x7fbfe40ffb90 2023-07-14T11:16:39.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe9b96700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe419f940 con 0x7fbfe4101720 2023-07-14T11:16:39.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48780/0 (socket says 172.21.15.161:48780) 2023-07-14T11:16:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 -- 172.21.15.161:0/714910466 learned_addr learned my addr 172.21.15.161:0/714910466 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:16:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe37fe700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfe40ffb90 0x7fbfe4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe3fff700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfe4101720 0x7fbfe419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 -- 172.21.15.161:0/714910466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfe4101720 msgr2=0x7fbfe419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfe4101720 0x7fbfe419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 -- 172.21.15.161:0/714910466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfe40ffb90 msgr2=0x7fbfe4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:16:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfe40ffb90 0x7fbfe4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:16:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.731+0000 7fbfe2ffd700 1 -- 172.21.15.161:0/714910466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80095f0 con 0x7fbfe4100d90 2023-07-14T11:16:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.732+0000 7fbfe2ffd700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe4198020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8005910 tx=0x7fbfd80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:16:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.732+0000 7fbfe0ff9700 1 -- 172.21.15.161:0/714910466 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd801d070 con 0x7fbfe4100d90 2023-07-14T11:16:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.732+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe419fbc0 con 0x7fbfe4100d90 2023-07-14T11:16:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.732+0000 7fbfe0ff9700 1 -- 172.21.15.161:0/714910466 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd8022470 con 0x7fbfe4100d90 2023-07-14T11:16:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.732+0000 7fbfe0ff9700 1 -- 172.21.15.161:0/714910466 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8017630 con 0x7fbfe4100d90 2023-07-14T11:16:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.732+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe419ff70 con 0x7fbfe4100d90 2023-07-14T11:16:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.733+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe4192260 con 0x7fbfe4100d90 2023-07-14T11:16:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.735+0000 7fbfe0ff9700 1 -- 172.21.15.161:0/714910466 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbfd800fad0 con 0x7fbfe4100d90 2023-07-14T11:16:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.736+0000 7fbfe0ff9700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfcc0a1150 0x7fbfcc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:16:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.737+0000 7fbfe0ff9700 1 -- 172.21.15.161:0/714910466 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fbfd80cd4d0 con 0x7fbfe4100d90 2023-07-14T11:16:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.737+0000 7fbfe37fe700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfcc0a1150 0x7fbfcc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:16:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.737+0000 7fbfe37fe700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfcc0a1150 0x7fbfcc0a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4100bf0 tx=0x7fbfd4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:16:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.739+0000 7fbfe0ff9700 1 -- 172.21.15.161:0/714910466 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbfd8097ab0 con 0x7fbfe4100d90 2023-07-14T11:16:39.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:16:39.942+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfe4060950 con 0x7fbfcc0a1150 2023-07-14T11:16:40.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:40 smithi175 ceph-mon[129081]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T11:16:40.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:40 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:16:40.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:40 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:16:40.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:40 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:40.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:40 smithi182 ceph-mon[129366]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T11:16:40.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:40 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:16:40.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:40 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:16:40.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:40 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:40.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:40 smithi161 ceph-mon[95526]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T11:16:40.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:16:40.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:16:40.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:41.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:41 smithi175 ceph-mon[129081]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:16:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:41 smithi175 ceph-mon[129081]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:41 smithi175 ceph-mon[129081]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T11:16:41.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:41 smithi182 ceph-mon[129366]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:16:41.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:41 smithi182 ceph-mon[129366]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:41.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:41 smithi182 ceph-mon[129366]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T11:16:41.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:41 smithi161 ceph-mon[95526]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:16:41.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:41 smithi161 ceph-mon[95526]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:41.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:41 smithi161 ceph-mon[95526]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T11:16:42.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:42 smithi175 ceph-mon[129081]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T11:16:42.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:42 smithi182 ceph-mon[129366]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T11:16:42.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:42 smithi161 ceph-mon[95526]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T11:16:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:43 smithi175 ceph-mon[129081]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:16:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:43 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1165094708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcacbde1-0008-440a-bbe7-07275b13f16a"}]: dispatch 2023-07-14T11:16:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:43 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1165094708' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcacbde1-0008-440a-bbe7-07275b13f16a"}]': finished 2023-07-14T11:16:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:43 smithi175 ceph-mon[129081]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T11:16:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:16:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:43 smithi182 ceph-mon[129366]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:16:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:43 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1165094708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcacbde1-0008-440a-bbe7-07275b13f16a"}]: dispatch 2023-07-14T11:16:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:43 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1165094708' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcacbde1-0008-440a-bbe7-07275b13f16a"}]': finished 2023-07-14T11:16:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:43 smithi182 ceph-mon[129366]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T11:16:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:16:43.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:43 smithi161 ceph-mon[95526]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:16:43.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:43 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1165094708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcacbde1-0008-440a-bbe7-07275b13f16a"}]: dispatch 2023-07-14T11:16:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:43 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1165094708' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcacbde1-0008-440a-bbe7-07275b13f16a"}]': finished 2023-07-14T11:16:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:43 smithi161 ceph-mon[95526]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T11:16:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:16:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:44 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1858593806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:44.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:44 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1858593806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:44.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:44 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1858593806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:16:45.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:45 smithi175 ceph-mon[129081]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-14T11:16:45.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:45 smithi182 ceph-mon[129366]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-14T11:16:45.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:45 smithi161 ceph-mon[95526]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-14T11:16:47.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:47 smithi175 ceph-mon[129081]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:16:47.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:47 smithi182 ceph-mon[129366]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:16:47.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:47 smithi161 ceph-mon[95526]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:16:49.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:49 smithi175 ceph-mon[129081]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:16:49.956 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:49 smithi161 ceph-mon[95526]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:16:49.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:49 smithi182 ceph-mon[129366]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:16:50.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T11:16:50.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T11:16:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:50.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T11:16:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:16:51.786 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:51 smithi161 ceph-mon[95526]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:51.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:51 smithi161 ceph-mon[95526]: Deploying daemon osd.6 on smithi161 2023-07-14T11:16:51.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:51 smithi175 ceph-mon[129081]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:51.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:51 smithi175 ceph-mon[129081]: Deploying daemon osd.6 on smithi161 2023-07-14T11:16:51.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:51 smithi182 ceph-mon[129366]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:51.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:51 smithi182 ceph-mon[129366]: Deploying daemon osd.6 on smithi161 2023-07-14T11:16:53.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:53 smithi161 ceph-mon[95526]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:53.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:53 smithi175 ceph-mon[129081]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:53.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:53 smithi182 ceph-mon[129366]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:55.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:55 smithi175 ceph-mon[129081]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:55.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:55 smithi182 ceph-mon[129366]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:55.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:55 smithi161 ceph-mon[95526]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:57.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:57 smithi175 ceph-mon[129081]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:57.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:57.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:57.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:57 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:57.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:57 smithi182 ceph-mon[129366]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:57.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:57.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:57.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:57 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:57.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:57 smithi161 ceph-mon[95526]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:57.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:57.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:16:57.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:57 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:16:59.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:16:59 smithi175 ceph-mon[129081]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:59.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:16:59 smithi182 ceph-mon[129366]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:16:59.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:16:59 smithi161 ceph-mon[95526]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:00.505 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 6 on host 'smithi161' 2023-07-14T11:17:00.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.502+0000 7fbfe0ff9700 1 -- 172.21.15.161:0/714910466 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbfe4060950 con 0x7fbfcc0a1150 2023-07-14T11:17:00.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.505+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfcc0a1150 msgr2=0x7fbfcc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:00.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.505+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfcc0a1150 0x7fbfcc0a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4100bf0 tx=0x7fbfd4009260 comp rx=0 tx=0).stop 2023-07-14T11:17:00.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 msgr2=0x7fbfe4198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:00.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe4198020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8005910 tx=0x7fbfd80057b0 comp rx=0 tx=0).stop 2023-07-14T11:17:00.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 shutdown_connections 2023-07-14T11:17:00.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfcc0a1150 0x7fbfcc0a3600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:00.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfe4101720 0x7fbfe419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:00.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfe40ffb90 0x7fbfe4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:00.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 --2- 172.21.15.161:0/714910466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100d90 0x7fbfe4198020 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:00.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 >> 172.21.15.161:0/714910466 conn(0x7fbfe40fb1a0 msgr2=0x7fbfe41066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:17:00.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 shutdown_connections 2023-07-14T11:17:00.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:00.506+0000 7fbfe9b96700 1 -- 172.21.15.161:0/714910466 wait complete. 2023-07-14T11:17:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:00.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:00 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:00.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:00.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:00.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:00.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:00.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:00 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:01.065 DEBUG:teuthology.orchestra.run.smithi161:osd.6> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.6.service 2023-07-14T11:17:01.071 INFO:tasks.cephadm:Deploying osd.7 on smithi175 with /dev/vg_nvme/lv_2... 2023-07-14T11:17:01.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T11:17:01.115 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:01.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:01.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:01.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:01.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:01.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:01.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:01.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:00 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:01.146 INFO:journalctl@ceph.osd.6.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:17:01.642 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:17:01.938 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:01 smithi161 ceph-mon[95526]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:17:02.005 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:01 smithi175 ceph-mon[129081]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:17:02.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:01 smithi182 ceph-mon[129366]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:17:02.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:02 smithi161 ceph-mon[95526]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T11:17:02.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:02 smithi161 ceph-mon[95526]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T11:17:02.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:02 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:02.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:02 smithi161 ceph-mon[95526]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:17:02.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:02 smithi161 ceph-mon[95526]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:02.997 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:17:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:17:02.747+0000 7fa4db610700 -1 osd.6 0 waiting for initial osdmap 2023-07-14T11:17:02.997 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:17:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:17:02.761+0000 7fa4d33fd700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:17:03.019 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T11:17:03.020 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T11:17:03.020 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2023-07-14T11:17:03.020 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2023-07-14T11:17:03.020 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111483 s, 941 MB/s 2023-07-14T11:17:03.020 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2023-07-14T11:17:03.058 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:02 smithi175 ceph-mon[129081]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T11:17:03.058 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:02 smithi175 ceph-mon[129081]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T11:17:03.059 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:02 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:03.059 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:02 smithi175 ceph-mon[129081]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:17:03.059 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:02 smithi175 ceph-mon[129081]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:03.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:02 smithi182 ceph-mon[129366]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T11:17:03.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:02 smithi182 ceph-mon[129366]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T11:17:03.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:02 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:03.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:02 smithi182 ceph-mon[129366]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:17:03.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:02 smithi182 ceph-mon[129366]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:03.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2023-07-14T11:17:03.948 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:17:04.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:03 smithi175 ceph-mon[129081]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:17:04.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:03 smithi175 ceph-mon[129081]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T11:17:04.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:04.131 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:03 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:03 smithi182 ceph-mon[129366]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:17:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:03 smithi182 ceph-mon[129366]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T11:17:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:03 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:03 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:04.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:03 smithi161 ceph-mon[95526]: from='osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:17:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:03 smithi161 ceph-mon[95526]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T11:17:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:03 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:03 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:05.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.084+0000 7f056279a700 1 -- 172.21.15.175:0/2053604028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0feaa0 msgr2=0x7f055c0feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:05.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.084+0000 7f056279a700 1 --2- 172.21.15.175:0/2053604028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0feaa0 0x7f055c0feeb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f054c009990 tx=0x7f054c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:17:05.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 -- 172.21.15.175:0/2053604028 shutdown_connections 2023-07-14T11:17:05.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 --2- 172.21.15.175:0/2053604028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c1006f0 0x7f055c106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:05.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 --2- 172.21.15.175:0/2053604028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0feaa0 0x7f055c0feeb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:05.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 --2- 172.21.15.175:0/2053604028 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f055c0ffd40 0x7f055c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:05.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 -- 172.21.15.175:0/2053604028 >> 172.21.15.175:0/2053604028 conn(0x7f055c0fa110 msgr2=0x7f055c0fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:17:05.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 -- 172.21.15.175:0/2053604028 shutdown_connections 2023-07-14T11:17:05.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 -- 172.21.15.175:0/2053604028 wait complete. 2023-07-14T11:17:05.090 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 Processor -- start 2023-07-14T11:17:05.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.085+0000 7f056279a700 1 -- start start 2023-07-14T11:17:05.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f056279a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f055c0feaa0 0x7f055c195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:05.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f056279a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c0ffd40 0x7f055c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:05.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f056279a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c1006f0 0x7f055c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:05.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f056279a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c19d3e0 con 0x7f055c0ffd40 2023-07-14T11:17:05.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f056279a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c19d520 con 0x7f055c1006f0 2023-07-14T11:17:05.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f056279a700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c19d6b0 con 0x7f055c0feaa0 2023-07-14T11:17:05.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c1006f0 0x7f055c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:05.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c1006f0 0x7f055c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58474/0 (socket says 172.21.15.175:58474) 2023-07-14T11:17:05.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 -- 172.21.15.175:0/4232995373 learned_addr learned my addr 172.21.15.175:0/4232995373 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-07-14T11:17:05.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f055bfff700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f055c0feaa0 0x7f055c195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:05.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f055b7fe700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c0ffd40 0x7f055c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:05.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 -- 172.21.15.175:0/4232995373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f055c0feaa0 msgr2=0x7f055c195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:05.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f055c0feaa0 0x7f055c195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 -- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c0ffd40 msgr2=0x7f055c195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c0ffd40 0x7f055c195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 -- 172.21.15.175:0/4232995373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0095f0 con 0x7f055c1006f0 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f0560d37700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c1006f0 0x7f055c19ce10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0554007ee0 tx=0x7f055400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f05597fa700 1 -- 172.21.15.175:0/4232995373 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554017070 con 0x7f055c1006f0 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f05597fa700 1 -- 172.21.15.175:0/4232995373 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f055400f040 con 0x7f055c1006f0 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.086+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c19d990 con 0x7f055c1006f0 2023-07-14T11:17:05.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.087+0000 7f05597fa700 1 -- 172.21.15.175:0/4232995373 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554005610 con 0x7f055c1006f0 2023-07-14T11:17:05.094 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.087+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c19de40 con 0x7f055c1006f0 2023-07-14T11:17:05.094 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.087+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c10a270 con 0x7f055c1006f0 2023-07-14T11:17:05.094 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.089+0000 7f05597fa700 1 -- 172.21.15.175:0/4232995373 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0554004c40 con 0x7f055c1006f0 2023-07-14T11:17:05.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.089+0000 7f05597fa700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f05440a1150 0x7f05440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:05.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.089+0000 7f05597fa700 1 -- 172.21.15.175:0/4232995373 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f05540cb950 con 0x7f055c1006f0 2023-07-14T11:17:05.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.090+0000 7f055bfff700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f05440a1150 0x7f05440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:05.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.090+0000 7f055bfff700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f05440a1150 0x7f05440a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f054c00b2e0 tx=0x7f054c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:17:05.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.093+0000 7f05597fa700 1 -- 172.21.15.175:0/4232995373 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0554095db0 con 0x7f055c1006f0 2023-07-14T11:17:05.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:17:05.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:17:05.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:05.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] boot 2023-07-14T11:17:05.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T11:17:05.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi161 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 5755M 2023-07-14T11:17:05.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:05.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:05.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:04 smithi175 ceph-mon[129081]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:05.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:17:05.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:17:05.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:05.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] boot 2023-07-14T11:17:05.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T11:17:05.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:05.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi161 2023-07-14T11:17:05.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 5755M 2023-07-14T11:17:05.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:05.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:05.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:04 smithi182 ceph-mon[129366]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:05.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:17:05.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:17:05.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:05.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: osd.6 [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] boot 2023-07-14T11:17:05.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T11:17:05.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:17:05.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi161 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 5755M 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:05.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:05.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:05.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:04 smithi161 ceph-mon[95526]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:17:05.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:05.285+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f055c069060 con 0x7f05440a1150 2023-07-14T11:17:06.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:05 smithi175 ceph-mon[129081]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T11:17:06.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:05 smithi175 ceph-mon[129081]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:06.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:06.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:06.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:06.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:05 smithi182 ceph-mon[129366]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T11:17:06.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:05 smithi182 ceph-mon[129366]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:06.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:06.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:06.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:06.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:05 smithi161 ceph-mon[95526]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T11:17:06.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:05 smithi161 ceph-mon[95526]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:06.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:06.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:06.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:07.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:06 smithi175 ceph-mon[129081]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:07.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:06 smithi182 ceph-mon[129366]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:07.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:06 smithi161 ceph-mon[95526]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:08 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/2714787503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]: dispatch 2023-07-14T11:17:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:08 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]: dispatch 2023-07-14T11:17:08.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:08 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]': finished 2023-07-14T11:17:08.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:08 smithi175 ceph-mon[129081]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T11:17:08.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:08.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:08 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/146258480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:08 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/2714787503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]: dispatch 2023-07-14T11:17:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:08 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]: dispatch 2023-07-14T11:17:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:08 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]': finished 2023-07-14T11:17:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:08 smithi182 ceph-mon[129366]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T11:17:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:08 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/146258480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:08.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:08 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/2714787503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]: dispatch 2023-07-14T11:17:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:08 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]: dispatch 2023-07-14T11:17:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:08 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6669cd4-11cc-4457-8775-f11dbeb79ec4"}]': finished 2023-07-14T11:17:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:08 smithi161 ceph-mon[95526]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T11:17:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:08 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/146258480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:09.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:09 smithi175 ceph-mon[129081]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:09.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:09 smithi182 ceph-mon[129366]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:09 smithi161 ceph-mon[95526]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:11.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:11 smithi175 ceph-mon[129081]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:11 smithi182 ceph-mon[129366]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:11.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:11 smithi161 ceph-mon[95526]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:13 smithi175 ceph-mon[129081]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:13 smithi182 ceph-mon[129366]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:13.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:13 smithi161 ceph-mon[95526]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:14.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T11:17:14.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:14.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T11:17:14.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:14.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T11:17:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:15.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:15 smithi175 ceph-mon[129081]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:15.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:15 smithi175 ceph-mon[129081]: Deploying daemon osd.7 on smithi175 2023-07-14T11:17:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:15 smithi182 ceph-mon[129366]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:15 smithi182 ceph-mon[129366]: Deploying daemon osd.7 on smithi175 2023-07-14T11:17:15.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:15 smithi161 ceph-mon[95526]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:15 smithi161 ceph-mon[95526]: Deploying daemon osd.7 on smithi175 2023-07-14T11:17:17.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:17 smithi175 ceph-mon[129081]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:17 smithi182 ceph-mon[129366]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:17 smithi161 ceph-mon[95526]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:19.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:19 smithi175 ceph-mon[129081]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:19.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:19.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:19.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:19.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:19 smithi182 ceph-mon[129366]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:19.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:19.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:19.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:19.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:19 smithi161 ceph-mon[95526]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:19.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:20.953 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.951+0000 7f05597fa700 1 -- 172.21.15.175:0/4232995373 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f055c069060 con 0x7f05440a1150 2023-07-14T11:17:20.954 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 7 on host 'smithi175' 2023-07-14T11:17:20.955 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f05440a1150 msgr2=0x7f05440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:20.955 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f05440a1150 0x7f05440a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f054c00b2e0 tx=0x7f054c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:17:20.955 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c1006f0 msgr2=0x7f055c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:20.955 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c1006f0 0x7f055c19ce10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0554007ee0 tx=0x7f055400d3b0 comp rx=0 tx=0).stop 2023-07-14T11:17:20.955 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 shutdown_connections 2023-07-14T11:17:20.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f05440a1150 0x7f05440a3600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:20.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c1006f0 0x7f055c19ce10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:20.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f055c0feaa0 0x7f055c195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:20.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 --2- 172.21.15.175:0/4232995373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c0ffd40 0x7f055c195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:20.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 >> 172.21.15.175:0/4232995373 conn(0x7f055c0fa110 msgr2=0x7f055c068950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:17:20.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 shutdown_connections 2023-07-14T11:17:20.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:17:20.953+0000 7f056279a700 1 -- 172.21.15.175:0/4232995373 wait complete. 2023-07-14T11:17:21.431 DEBUG:teuthology.orchestra.run.smithi175:osd.7> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.7.service 2023-07-14T11:17:21.439 INFO:tasks.cephadm:Deploying osd.8 on smithi182 with /dev/vg_nvme/lv_2... 2023-07-14T11:17:21.439 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T11:17:21.477 INFO:journalctl@ceph.osd.7.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:17:21.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:21.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:21.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:21.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:21.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:21.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:21.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:22.004 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:17:22.452 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:17:22 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:17:22.181+0000 7fb4987ce640 -1 osd.7 0 log_to_monitors true 2023-07-14T11:17:22.453 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:22 smithi175 ceph-mon[129081]: from='osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:17:22.453 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:22 smithi175 ceph-mon[129081]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:17:22.555 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:22 smithi182 ceph-mon[129366]: from='osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:17:22.555 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:22 smithi182 ceph-mon[129366]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:17:22.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:22 smithi161 ceph-mon[95526]: from='osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:17:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:22 smithi161 ceph-mon[95526]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:17:23.480 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T11:17:23.480 INFO:teuthology.orchestra.run.smithi182.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T11:17:23.480 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 10+0 records in 2023-07-14T11:17:23.481 INFO:teuthology.orchestra.run.smithi182.stderr:10+0 records out 2023-07-14T11:17:23.481 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114535 s, 916 MB/s 2023-07-14T11:17:23.481 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping successful for: 2023-07-14T11:17:23.669 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:17:23.432+0000 7fb48de46700 -1 osd.7 0 waiting for initial osdmap 2023-07-14T11:17:23.669 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:17:23.444+0000 7fb487c37700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:17:23.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-mon[129081]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:23.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-mon[129081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T11:17:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-mon[129081]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T11:17:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-mon[129081]: from='osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:17:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:23 smithi175 ceph-mon[129081]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:17:23.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:23 smithi182 ceph-mon[129366]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:23.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:23 smithi182 ceph-mon[129366]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T11:17:23.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:23 smithi182 ceph-mon[129366]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T11:17:23.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:23 smithi182 ceph-mon[129366]: from='osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:17:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:23 smithi182 ceph-mon[129366]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:17:23.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:23 smithi161 ceph-mon[95526]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:23.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:23 smithi161 ceph-mon[95526]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T11:17:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:23 smithi161 ceph-mon[95526]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T11:17:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:23 smithi161 ceph-mon[95526]: from='osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:17:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:23 smithi161 ceph-mon[95526]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:17:23.844 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi182:vg_nvme/lv_2 2023-07-14T11:17:24.401 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:17:24.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:17:24.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T11:17:24.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:24.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:24.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:24.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:24.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:17:24.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T11:17:24.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:24.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:24.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:24.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:24.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:17:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T11:17:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:25.597 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 -- 172.21.15.182:0/1820735316 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780ffc80 msgr2=0x7f66781000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:25.597 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 --2- 172.21.15.182:0/1820735316 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780ffc80 0x7f66781000d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6668009990 tx=0x7f6668009ca0 comp rx=0 tx=0).stop 2023-07-14T11:17:25.597 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 -- 172.21.15.182:0/1820735316 shutdown_connections 2023-07-14T11:17:25.597 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 --2- 172.21.15.182:0/1820735316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66780fea80 0x7f66780fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:25.597 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 --2- 172.21.15.182:0/1820735316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6678100610 0x7f6678106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:25.597 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 --2- 172.21.15.182:0/1820735316 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780ffc80 0x7f66781000d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:25.598 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 -- 172.21.15.182:0/1820735316 >> 172.21.15.182:0/1820735316 conn(0x7f66780fa0b0 msgr2=0x7f66780fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:17:25.598 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 -- 172.21.15.182:0/1820735316 shutdown_connections 2023-07-14T11:17:25.598 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.592+0000 7f66806a3700 1 -- 172.21.15.182:0/1820735316 wait complete. 2023-07-14T11:17:25.598 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 Processor -- start 2023-07-14T11:17:25.598 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 -- start start 2023-07-14T11:17:25.598 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780fea80 0x7f6678069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:25.598 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66780ffc80 0x7f667819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6678100610 0x7f667819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667819d480 con 0x7f66780ffc80 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667819d5f0 con 0x7f6678100610 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f66806a3700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667819d7b0 con 0x7f66780fea80 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780fea80 0x7f6678069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780fea80 0x7f6678069420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.182:37126/0 (socket says 172.21.15.182:37126) 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 -- 172.21.15.182:0/1217290360 learned_addr learned my addr 172.21.15.182:0/1217290360 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:17:25.599 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667ec40700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6678100610 0x7f667819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:25.600 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 -- 172.21.15.182:0/1217290360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6678100610 msgr2=0x7f667819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:25.600 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6678100610 0x7f667819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:25.600 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 -- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66780ffc80 msgr2=0x7f667819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:25.600 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66780ffc80 0x7f667819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:25.600 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.593+0000 7f667e43f700 1 -- 172.21.15.182:0/1217290360 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680095f0 con 0x7f66780fea80 2023-07-14T11:17:25.600 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.594+0000 7f667e43f700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780fea80 0x7f6678069420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f667400b500 tx=0x7f667400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:17:25.600 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.594+0000 7f666f7fe700 1 -- 172.21.15.182:0/1217290360 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674011830 con 0x7f66780fea80 2023-07-14T11:17:25.601 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.594+0000 7f666f7fe700 1 -- 172.21.15.182:0/1217290360 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6674011e50 con 0x7f66780fea80 2023-07-14T11:17:25.601 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.594+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667819da90 con 0x7f66780fea80 2023-07-14T11:17:25.601 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.594+0000 7f666f7fe700 1 -- 172.21.15.182:0/1217290360 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667400f2e0 con 0x7f66780fea80 2023-07-14T11:17:25.601 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.594+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667819df90 con 0x7f66780fea80 2023-07-14T11:17:25.602 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.595+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678106090 con 0x7f66780fea80 2023-07-14T11:17:25.602 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.596+0000 7f666f7fe700 1 -- 172.21.15.182:0/1217290360 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6674010580 con 0x7f66780fea80 2023-07-14T11:17:25.602 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.596+0000 7f666f7fe700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f66640a1150 0x7f66640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:25.602 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.596+0000 7f666f7fe700 1 -- 172.21.15.182:0/1217290360 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f66740cbda0 con 0x7f66780fea80 2023-07-14T11:17:25.602 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.597+0000 7f667dc3e700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f66640a1150 0x7f66640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:25.602 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.597+0000 7f667dc3e700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f66640a1150 0x7f66640a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66680098e0 tx=0x7f6668005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:17:25.603 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.600+0000 7f666f7fe700 1 -- 172.21.15.182:0/1217290360 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6674096030 con 0x7f66780fea80 2023-07-14T11:17:25.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:17:25.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:17:25.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi175 2023-07-14T11:17:25.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 5722M 2023-07-14T11:17:25.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:25.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] boot 2023-07-14T11:17:25.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T11:17:25.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:25.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:17:25.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:17:25.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi175 2023-07-14T11:17:25.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 5722M 2023-07-14T11:17:25.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:25.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] boot 2023-07-14T11:17:25.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T11:17:25.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:25.797 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:25.794+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6678060950 con 0x7f66640a1150 2023-07-14T11:17:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:17:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:17:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi175 2023-07-14T11:17:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 5722M 2023-07-14T11:17:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:17:25.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: osd.7 [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] boot 2023-07-14T11:17:25.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T11:17:25.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:17:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:26 smithi182 ceph-mon[129366]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T11:17:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:26 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:26 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:26.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:26 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:26.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:26 smithi161 ceph-mon[95526]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T11:17:26.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:26.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:26.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:26.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:26 smithi175 ceph-mon[129081]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T11:17:26.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:26 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:26.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:26 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:26.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:26 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:27.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:27 smithi182 ceph-mon[129366]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:27.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:27 smithi182 ceph-mon[129366]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:27.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:27 smithi182 ceph-mon[129366]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T11:17:27.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:27 smithi161 ceph-mon[95526]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:27 smithi161 ceph-mon[95526]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:27 smithi161 ceph-mon[95526]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T11:17:27.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:27 smithi175 ceph-mon[129081]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:27.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:27 smithi175 ceph-mon[129081]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:27.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:27 smithi175 ceph-mon[129081]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T11:17:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:28 smithi182 ceph-mon[129366]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T11:17:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:28 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/2752129892' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]: dispatch 2023-07-14T11:17:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:28 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]: dispatch 2023-07-14T11:17:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:28 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]': finished 2023-07-14T11:17:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:28 smithi182 ceph-mon[129366]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T11:17:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:28 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:28.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:28 smithi161 ceph-mon[95526]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T11:17:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:28 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/2752129892' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]: dispatch 2023-07-14T11:17:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:28 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]: dispatch 2023-07-14T11:17:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:28 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]': finished 2023-07-14T11:17:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:28 smithi161 ceph-mon[95526]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T11:17:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:28 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:28.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:28 smithi175 ceph-mon[129081]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T11:17:28.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:28 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/2752129892' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]: dispatch 2023-07-14T11:17:28.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:28 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]: dispatch 2023-07-14T11:17:28.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:28 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56d1548f-b2d9-44bb-914a-9dbfdcd39b6d"}]': finished 2023-07-14T11:17:28.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:28 smithi175 ceph-mon[129081]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T11:17:28.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:28 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:29 smithi175 ceph-mon[129081]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:29 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/2069454510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:29.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:29 smithi182 ceph-mon[129366]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:29.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:29 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/2069454510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:29.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:29 smithi161 ceph-mon[95526]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:29 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/2069454510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:31.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:31 smithi175 ceph-mon[129081]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:31.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:31 smithi182 ceph-mon[129366]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:31.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:31 smithi161 ceph-mon[95526]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:33 smithi175 ceph-mon[129081]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:33 smithi182 ceph-mon[129366]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:33 smithi161 ceph-mon[95526]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:34.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T11:17:34.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:34.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T11:17:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:34.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T11:17:34.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:35.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:35 smithi175 ceph-mon[129081]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:35.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:35 smithi175 ceph-mon[129081]: Deploying daemon osd.8 on smithi182 2023-07-14T11:17:35.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:35 smithi182 ceph-mon[129366]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:35.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:35 smithi182 ceph-mon[129366]: Deploying daemon osd.8 on smithi182 2023-07-14T11:17:35.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:35 smithi161 ceph-mon[95526]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:35 smithi161 ceph-mon[95526]: Deploying daemon osd.8 on smithi182 2023-07-14T11:17:37.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:37 smithi182 ceph-mon[129366]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:37.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:37 smithi175 ceph-mon[129081]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:37.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:37 smithi161 ceph-mon[95526]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:39.902 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:39 smithi182 ceph-mon[129366]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:39.902 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:39 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:39.902 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:39 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:39.903 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:39 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:39.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:39 smithi175 ceph-mon[129081]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:39.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:39 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:39.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:39 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:39.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:39 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:39.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:39 smithi161 ceph-mon[95526]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:39.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:39 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:39.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:39 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:39.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:39 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:41.416 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.413+0000 7f666f7fe700 1 -- 172.21.15.182:0/1217290360 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6678060950 con 0x7f66640a1150 2023-07-14T11:17:41.416 INFO:teuthology.orchestra.run.smithi182.stdout:Created osd(s) 8 on host 'smithi182' 2023-07-14T11:17:41.417 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f66640a1150 msgr2=0x7f66640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f66640a1150 0x7f66640a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66680098e0 tx=0x7f6668005fb0 comp rx=0 tx=0).stop 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780fea80 msgr2=0x7f6678069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780fea80 0x7f6678069420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f667400b500 tx=0x7f667400b810 comp rx=0 tx=0).stop 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 shutdown_connections 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f66640a1150 0x7f66640a3600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6678100610 0x7f667819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66780fea80 0x7f6678069420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:41.418 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 --2- 172.21.15.182:0/1217290360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66780ffc80 0x7f667819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:41.419 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.415+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 >> 172.21.15.182:0/1217290360 conn(0x7f66780fa0b0 msgr2=0x7f66781031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:17:41.419 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.416+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 shutdown_connections 2023-07-14T11:17:41.419 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:17:41.416+0000 7f66806a3700 1 -- 172.21.15.182:0/1217290360 wait complete. 2023-07-14T11:17:41.879 DEBUG:teuthology.orchestra.run.smithi182:osd.8> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.8.service 2023-07-14T11:17:41.882 INFO:tasks.cephadm:Deploying osd.9 on smithi161 with /dev/vg_nvme/lv_1... 2023-07-14T11:17:41.883 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T11:17:41.909 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:41.909 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.909 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.909 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:41.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:41.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:41.915 INFO:journalctl@ceph.osd.8.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:17:41.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:41.953 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:41.953 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.953 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.953 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:41.953 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:41.954 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.954 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.954 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:41.954 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:17:42.615 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:17:42.986 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:17:42 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:17:42.728+0000 7fd4f028d640 -1 osd.8 0 log_to_monitors true 2023-07-14T11:17:43.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:43 smithi161 ceph-mon[95526]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:43.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:43 smithi161 ceph-mon[95526]: from='osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:17:43.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:43 smithi161 ceph-mon[95526]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:17:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:43 smithi175 ceph-mon[129081]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:43 smithi175 ceph-mon[129081]: from='osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:17:43.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:43 smithi175 ceph-mon[129081]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:17:43.959 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:43 smithi182 ceph-mon[129366]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:43.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:43 smithi182 ceph-mon[129366]: from='osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:17:43.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:43 smithi182 ceph-mon[129366]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:17:44.631 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T11:17:44.631 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T11:17:44.631 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T11:17:44.631 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T11:17:44.631 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0302058 s, 347 MB/s 2023-07-14T11:17:44.632 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T11:17:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T11:17:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T11:17:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:44.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:44.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.964 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:17:44.670+0000 7fd4e5905700 -1 osd.8 0 waiting for initial osdmap 2023-07-14T11:17:44.964 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:17:44.679+0000 7fd4df6f6700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:17:44.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T11:17:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T11:17:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:17:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:17:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:44.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:44.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T11:17:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T11:17:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:17:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:17:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:17:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:17:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:17:45.195 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2023-07-14T11:17:45.923 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:17:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi182 2023-07-14T11:17:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 5722M 2023-07-14T11:17:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:17:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T11:17:45.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:45.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:45.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] boot 2023-07-14T11:17:45.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T11:17:45.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:45 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:45.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:45.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi182 2023-07-14T11:17:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 5722M 2023-07-14T11:17:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:17:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T11:17:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] boot 2023-07-14T11:17:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T11:17:45.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:45 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:17:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi182 2023-07-14T11:17:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 5722M 2023-07-14T11:17:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:17:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T11:17:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:46.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:46.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: osd.8 [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] boot 2023-07-14T11:17:46.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T11:17:46.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:45 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:17:46.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:46 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:17:46.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:46 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:17:46.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:46 smithi161 ceph-mon[95526]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T11:17:46.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:46 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:17:46.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:46 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:17:46.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:46 smithi182 ceph-mon[129366]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T11:17:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:46 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:17:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:46 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:17:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:46 smithi175 ceph-mon[129081]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T11:17:47.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 -- 172.21.15.161:0/2675011282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea4101720 msgr2=0x7f6ea4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:47.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/2675011282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea4101720 0x7f6ea4107fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00b0a0 tx=0x7f6e9c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:17:47.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 -- 172.21.15.161:0/2675011282 shutdown_connections 2023-07-14T11:17:47.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/2675011282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea4101720 0x7f6ea4107fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:47.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/2675011282 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ea40ffb90 0x7f6ea40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:47.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/2675011282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4100d90 0x7f6ea41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:47.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 -- 172.21.15.161:0/2675011282 >> 172.21.15.161:0/2675011282 conn(0x7f6ea40fb1a0 msgr2=0x7f6ea40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:17:47.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 -- 172.21.15.161:0/2675011282 shutdown_connections 2023-07-14T11:17:47.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.458+0000 7f6ea8adf700 1 -- 172.21.15.161:0/2675011282 wait complete. 2023-07-14T11:17:47.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.459+0000 7f6ea8adf700 1 Processor -- start 2023-07-14T11:17:47.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.459+0000 7f6ea8adf700 1 -- start start 2023-07-14T11:17:47.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea8adf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ffb90 0x7f6ea4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:47.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea8adf700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ea4100d90 0x7f6ea41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:47.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea8adf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4101720 0x7f6ea419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:47.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea8adf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea419f6f0 con 0x7f6ea40ffb90 2023-07-14T11:17:47.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea8adf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea419f830 con 0x7f6ea4101720 2023-07-14T11:17:47.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea8adf700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea419f9c0 con 0x7f6ea4100d90 2023-07-14T11:17:47.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ffb90 0x7f6ea4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:47.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea1d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ea4100d90 0x7f6ea41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:47.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ffb90 0x7f6ea4197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37110/0 (socket says 172.21.15.161:37110) 2023-07-14T11:17:47.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 -- 172.21.15.161:0/4073960180 learned_addr learned my addr 172.21.15.161:0/4073960180 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:17:47.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea2d9d700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4101720 0x7f6ea419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:47.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 -- 172.21.15.161:0/4073960180 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ea4100d90 msgr2=0x7f6ea41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:47.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ea4100d90 0x7f6ea41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:47.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 -- 172.21.15.161:0/4073960180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4101720 msgr2=0x7f6ea419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:17:47.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4101720 0x7f6ea419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:17:47.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea259c700 1 -- 172.21.15.161:0/4073960180 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c009d00 con 0x7f6ea40ffb90 2023-07-14T11:17:47.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.460+0000 7f6ea2d9d700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4101720 0x7f6ea419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:17:47.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.461+0000 7f6ea259c700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ffb90 0x7f6ea4197b60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400e9a0 tx=0x7f6e9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:17:47.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.461+0000 7f6e937fe700 1 -- 172.21.15.161:0/4073960180 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94009790 con 0x7f6ea40ffb90 2023-07-14T11:17:47.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.461+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea419fca0 con 0x7f6ea40ffb90 2023-07-14T11:17:47.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.461+0000 7f6e937fe700 1 -- 172.21.15.161:0/4073960180 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e940044e0 con 0x7f6ea40ffb90 2023-07-14T11:17:47.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.461+0000 7f6e937fe700 1 -- 172.21.15.161:0/4073960180 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94003d20 con 0x7f6ea40ffb90 2023-07-14T11:17:47.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.462+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea41a01a0 con 0x7f6ea40ffb90 2023-07-14T11:17:47.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.462+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea4192220 con 0x7f6ea40ffb90 2023-07-14T11:17:47.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.464+0000 7f6e937fe700 1 -- 172.21.15.161:0/4073960180 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6e94003e80 con 0x7f6ea40ffb90 2023-07-14T11:17:47.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.466+0000 7f6e937fe700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6e8c0a1150 0x7f6e8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:17:47.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.466+0000 7f6e937fe700 1 -- 172.21.15.161:0/4073960180 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f6e94014070 con 0x7f6ea40ffb90 2023-07-14T11:17:47.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.466+0000 7f6ea1d9b700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6e8c0a1150 0x7f6e8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:17:47.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.466+0000 7f6ea1d9b700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6e8c0a1150 0x7f6e8c0a3600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009c60 tx=0x7f6e98009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:17:47.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.470+0000 7f6e937fe700 1 -- 172.21.15.161:0/4073960180 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6e940962e0 con 0x7f6ea40ffb90 2023-07-14T11:17:47.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:17:47.677+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ea4060950 con 0x7f6e8c0a1150 2023-07-14T11:17:47.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:47 smithi182 ceph-mon[129366]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:47.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:47 smithi182 ceph-mon[129366]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T11:17:47.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:47 smithi161 ceph-mon[95526]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:47.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:47 smithi161 ceph-mon[95526]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T11:17:48.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:47 smithi175 ceph-mon[129081]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:47 smithi175 ceph-mon[129081]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T11:17:48.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:48 smithi182 ceph-mon[129366]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:48.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:48.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:48.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:48.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:48 smithi161 ceph-mon[95526]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:48.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:49.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:48 smithi175 ceph-mon[129081]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:17:49.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:17:49.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:17:49.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:49.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:49 smithi182 ceph-mon[129366]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:49.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:49 smithi182 ceph-mon[129366]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T11:17:49.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:49 smithi161 ceph-mon[95526]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:49.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:49 smithi161 ceph-mon[95526]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T11:17:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:49 smithi175 ceph-mon[129081]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:49 smithi175 ceph-mon[129081]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T11:17:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:50 smithi182 ceph-mon[129366]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T11:17:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:50 smithi182 ceph-mon[129366]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:17:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:50 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3872862822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c1241-9eaf-48e6-b54a-4cfc973aa73f"}]: dispatch 2023-07-14T11:17:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:50 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3872862822' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "016c1241-9eaf-48e6-b54a-4cfc973aa73f"}]': finished 2023-07-14T11:17:50.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:50 smithi182 ceph-mon[129366]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T11:17:50.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:17:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:50 smithi161 ceph-mon[95526]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T11:17:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:50 smithi161 ceph-mon[95526]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:17:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:50 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3872862822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c1241-9eaf-48e6-b54a-4cfc973aa73f"}]: dispatch 2023-07-14T11:17:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:50 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3872862822' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "016c1241-9eaf-48e6-b54a-4cfc973aa73f"}]': finished 2023-07-14T11:17:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:50 smithi161 ceph-mon[95526]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T11:17:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:17:51.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:50 smithi175 ceph-mon[129081]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T11:17:51.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:50 smithi175 ceph-mon[129081]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:17:51.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:50 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3872862822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c1241-9eaf-48e6-b54a-4cfc973aa73f"}]: dispatch 2023-07-14T11:17:51.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:50 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3872862822' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "016c1241-9eaf-48e6-b54a-4cfc973aa73f"}]': finished 2023-07-14T11:17:51.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:50 smithi175 ceph-mon[129081]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T11:17:51.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:17:51.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:51 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2242447733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:51.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:51 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2242447733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:52.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:51 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2242447733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:17:53.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:53 smithi175 ceph-mon[129081]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-07-14T11:17:53.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:53 smithi182 ceph-mon[129366]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-07-14T11:17:53.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:53 smithi161 ceph-mon[95526]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-07-14T11:17:55.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:55 smithi175 ceph-mon[129081]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:17:55.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:55 smithi182 ceph-mon[129366]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:17:55.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:55 smithi161 ceph-mon[95526]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:17:57.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:57 smithi175 ceph-mon[129081]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:17:57.963 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:57 smithi161 ceph-mon[95526]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:17:57.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:57 smithi182 ceph-mon[129366]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:17:58.900 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:58 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T11:17:58.900 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:58 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:58.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:58 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T11:17:58.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:58 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:58.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:58 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T11:17:58.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:58 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:17:59.896 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:59 smithi161 ceph-mon[95526]: Deploying daemon osd.9 on smithi161 2023-07-14T11:17:59.896 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:17:59 smithi161 ceph-mon[95526]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:59.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:59 smithi175 ceph-mon[129081]: Deploying daemon osd.9 on smithi161 2023-07-14T11:17:59.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:17:59 smithi175 ceph-mon[129081]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:17:59.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:59 smithi182 ceph-mon[129366]: Deploying daemon osd.9 on smithi161 2023-07-14T11:17:59.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:17:59 smithi182 ceph-mon[129366]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:01.701 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:01 smithi161 ceph-mon[95526]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:01.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:01 smithi175 ceph-mon[129081]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:01.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:01 smithi182 ceph-mon[129366]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:03.903 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:03 smithi161 ceph-mon[95526]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:03.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:03 smithi175 ceph-mon[129081]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:03.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:03 smithi182 ceph-mon[129366]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:05.514 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:05 smithi161 ceph-mon[95526]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:05.515 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:05.515 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:05.515 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:05.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:05 smithi175 ceph-mon[129081]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:05.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:05.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:05.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:05 smithi182 ceph-mon[129366]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:05.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:07.527 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:07 smithi161 ceph-mon[95526]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:07.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:07 smithi175 ceph-mon[129081]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:07 smithi182 ceph-mon[129366]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:07.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.943+0000 7f6e937fe700 1 -- 172.21.15.161:0/4073960180 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6ea4060950 con 0x7f6e8c0a1150 2023-07-14T11:18:07.947 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 9 on host 'smithi161' 2023-07-14T11:18:07.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.946+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6e8c0a1150 msgr2=0x7f6e8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:07.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.946+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6e8c0a1150 0x7f6e8c0a3600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009c60 tx=0x7f6e98009260 comp rx=0 tx=0).stop 2023-07-14T11:18:07.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.946+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ffb90 msgr2=0x7f6ea4197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:07.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.946+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ffb90 0x7f6ea4197b60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400e9a0 tx=0x7f6e9400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:18:07.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 shutdown_connections 2023-07-14T11:18:07.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ffb90 0x7f6ea4197b60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:07.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6e8c0a1150 0x7f6e8c0a3600 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:07.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4101720 0x7f6ea419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:07.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 --2- 172.21.15.161:0/4073960180 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ea4100d90 0x7f6ea41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:07.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 >> 172.21.15.161:0/4073960180 conn(0x7f6ea40fb1a0 msgr2=0x7f6ea4106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:07.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 shutdown_connections 2023-07-14T11:18:07.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:07.947+0000 7f6ea8adf700 1 -- 172.21.15.161:0/4073960180 wait complete. 2023-07-14T11:18:08.445 DEBUG:teuthology.orchestra.run.smithi161:osd.9> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.9.service 2023-07-14T11:18:08.449 INFO:tasks.cephadm:Deploying osd.10 on smithi175 with /dev/vg_nvme/lv_1... 2023-07-14T11:18:08.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T11:18:08.523 INFO:journalctl@ceph.osd.9.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:18:08.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:08.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:08 smithi161 ceph-mon[95526]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T11:18:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:08.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:08 smithi175 ceph-mon[129081]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T11:18:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:08.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:08 smithi182 ceph-mon[129366]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T11:18:09.014 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:18:09.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:09 smithi175 ceph-mon[129081]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:09.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:09 smithi175 ceph-mon[129081]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T11:18:09.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:09 smithi175 ceph-mon[129081]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T11:18:09.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:09.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:09 smithi175 ceph-mon[129081]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:18:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:09 smithi161 ceph-mon[95526]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:09 smithi161 ceph-mon[95526]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T11:18:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:09 smithi161 ceph-mon[95526]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T11:18:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:09 smithi161 ceph-mon[95526]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:18:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:09 smithi182 ceph-mon[129366]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:09 smithi182 ceph-mon[129366]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T11:18:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:09 smithi182 ceph-mon[129366]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T11:18:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:09 smithi182 ceph-mon[129366]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T11:18:10.246 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:18:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:18:09.930+0000 7f206f7c5700 -1 osd.9 0 waiting for initial osdmap 2023-07-14T11:18:10.246 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:18:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:18:09.945+0000 7f2067db3700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:18:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T11:18:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T11:18:10.461 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2023-07-14T11:18:10.461 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2023-07-14T11:18:10.461 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128249 s, 818 MB/s 2023-07-14T11:18:10.461 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2023-07-14T11:18:10.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2023-07-14T11:18:10.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:10 smithi161 ceph-mon[95526]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:18:10.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:10 smithi161 ceph-mon[95526]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T11:18:10.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:10 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:10.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:10 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:10.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:10 smithi161 ceph-mon[95526]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:10 smithi175 ceph-mon[129081]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:18:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:10 smithi175 ceph-mon[129081]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T11:18:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:10 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:11.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:10 smithi175 ceph-mon[129081]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:10 smithi182 ceph-mon[129366]: from='osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T11:18:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:10 smithi182 ceph-mon[129366]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T11:18:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:10 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:10 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:10 smithi182 ceph-mon[129366]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:18:11.414 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:18:12.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:18:12.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:18:12.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] boot 2023-07-14T11:18:12.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T11:18:12.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:12.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi161 2023-07-14T11:18:12.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 4316M 2023-07-14T11:18:12.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:12.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:12.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:18:12.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:18:12.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] boot 2023-07-14T11:18:12.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T11:18:12.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:12.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi161 2023-07-14T11:18:12.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 4316M 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:12.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:12.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.377 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:18:12.377 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:18:12.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: osd.9 [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] boot 2023-07-14T11:18:12.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T11:18:12.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:18:12.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi161 2023-07-14T11:18:12.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 4316M 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:12.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:12.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/4267590430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740fea80 msgr2=0x7f9a740fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:12.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/4267590430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740fea80 0x7f9a740fee90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009990 tx=0x7f9a68009ca0 comp rx=0 tx=0).stop 2023-07-14T11:18:12.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/4267590430 shutdown_connections 2023-07-14T11:18:12.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/4267590430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a74100610 0x7f9a74106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:12.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/4267590430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740fea80 0x7f9a740fee90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:12.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/4267590430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9a740ffc80 0x7f9a741000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:12.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/4267590430 >> 172.21.15.175:0/4267590430 conn(0x7f9a740fa0b0 msgr2=0x7f9a740fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:12.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/4267590430 shutdown_connections 2023-07-14T11:18:12.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/4267590430 wait complete. 2023-07-14T11:18:12.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 Processor -- start 2023-07-14T11:18:12.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.546+0000 7f9a7a2c6700 1 -- start start 2023-07-14T11:18:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a740fea80 0x7f9a74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740ffc80 0x7f9a7419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9a74100610 0x7f9a7419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419d400 con 0x7f9a740fea80 2023-07-14T11:18:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419d570 con 0x7f9a740ffc80 2023-07-14T11:18:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419d730 con 0x7f9a74100610 2023-07-14T11:18:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740ffc80 0x7f9a7419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740ffc80 0x7f9a7419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58614/0 (socket says 172.21.15.175:58614) 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 -- 172.21.15.175:0/3414014596 learned_addr learned my addr 172.21.15.175:0/3414014596 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a78863700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9a74100610 0x7f9a7419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 -- 172.21.15.175:0/3414014596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9a74100610 msgr2=0x7f9a7419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9a74100610 0x7f9a7419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 -- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a740fea80 msgr2=0x7f9a74068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a740fea80 0x7f9a74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:12.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 -- 172.21.15.175:0/3414014596 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680095f0 con 0x7f9a740ffc80 2023-07-14T11:18:12.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a737fe700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740ffc80 0x7f9a7419a610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b530 tx=0x7f9a6400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:12.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a717fa700 1 -- 172.21.15.175:0/3414014596 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64011830 con 0x7f9a740ffc80 2023-07-14T11:18:12.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7419da10 con 0x7f9a740ffc80 2023-07-14T11:18:12.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a717fa700 1 -- 172.21.15.175:0/3414014596 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a64011e50 con 0x7f9a740ffc80 2023-07-14T11:18:12.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7419df60 con 0x7f9a740ffc80 2023-07-14T11:18:12.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.547+0000 7f9a717fa700 1 -- 172.21.15.175:0/3414014596 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400f2e0 con 0x7f9a740ffc80 2023-07-14T11:18:12.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.548+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a60005320 con 0x7f9a740ffc80 2023-07-14T11:18:12.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.549+0000 7f9a717fa700 1 -- 172.21.15.175:0/3414014596 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9a64011990 con 0x7f9a740ffc80 2023-07-14T11:18:12.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.550+0000 7f9a717fa700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9a5c0a11d0 0x7f9a5c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:12.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.550+0000 7f9a717fa700 1 -- 172.21.15.175:0/3414014596 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9a640cbb70 con 0x7f9a740ffc80 2023-07-14T11:18:12.556 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.551+0000 7f9a73fff700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9a5c0a11d0 0x7f9a5c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:12.556 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.551+0000 7f9a73fff700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9a5c0a11d0 0x7f9a5c0a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a680095c0 tx=0x7f9a68009530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:12.557 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.554+0000 7f9a717fa700 1 -- 172.21.15.175:0/3414014596 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9a64095ac0 con 0x7f9a740ffc80 2023-07-14T11:18:12.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:12.754+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a60000bf0 con 0x7f9a5c0a11d0 2023-07-14T11:18:13.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:13 smithi161 ceph-mon[95526]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:13 smithi161 ceph-mon[95526]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T11:18:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:18:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:18:13.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:13.566 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:13 smithi175 ceph-mon[129081]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:13.566 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:13 smithi175 ceph-mon[129081]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T11:18:13.567 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:18:13.567 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:18:13.567 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:13 smithi182 ceph-mon[129366]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:13 smithi182 ceph-mon[129366]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T11:18:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:18:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:18:13.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:14.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:14 smithi175 ceph-mon[129081]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:18:14.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:14 smithi161 ceph-mon[95526]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:18:14.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:14 smithi182 ceph-mon[129366]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:18:15.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:15 smithi161 ceph-mon[95526]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:15.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:15 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/3769196590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]: dispatch 2023-07-14T11:18:15.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:15 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]: dispatch 2023-07-14T11:18:15.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:15 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]': finished 2023-07-14T11:18:15.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:15 smithi161 ceph-mon[95526]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T11:18:15.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:15.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:15 smithi175 ceph-mon[129081]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:15.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:15 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/3769196590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]: dispatch 2023-07-14T11:18:15.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:15 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]: dispatch 2023-07-14T11:18:15.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:15 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]': finished 2023-07-14T11:18:15.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:15 smithi175 ceph-mon[129081]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T11:18:15.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:15.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:15 smithi182 ceph-mon[129366]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:15.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:15 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/3769196590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]: dispatch 2023-07-14T11:18:15.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:15 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]: dispatch 2023-07-14T11:18:15.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:15 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2755e32a-49c5-4aaa-ad39-861f12b47889"}]': finished 2023-07-14T11:18:15.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:15 smithi182 ceph-mon[129366]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T11:18:15.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:16 smithi161 ceph-mon[95526]: from='client.? 172.21.15.175:0/1052064512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:18:16.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:16 smithi175 ceph-mon[129081]: from='client.? 172.21.15.175:0/1052064512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:18:16.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:16 smithi182 ceph-mon[129366]: from='client.? 172.21.15.175:0/1052064512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:18:17.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:17 smithi161 ceph-mon[95526]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:17.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:17 smithi175 ceph-mon[129081]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:17.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:17 smithi182 ceph-mon[129366]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:19 smithi175 ceph-mon[129081]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:19.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:19 smithi182 ceph-mon[129366]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:19 smithi161 ceph-mon[95526]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:21.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:21 smithi175 ceph-mon[129081]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:21.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:21 smithi182 ceph-mon[129366]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:21.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:21 smithi161 ceph-mon[95526]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:22.796 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:22 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T11:18:22.796 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:22 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:22.796 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:22 smithi175 ceph-mon[129081]: Deploying daemon osd.10 on smithi175 2023-07-14T11:18:22.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:22 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T11:18:22.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:22 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:22.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:22 smithi182 ceph-mon[129366]: Deploying daemon osd.10 on smithi175 2023-07-14T11:18:22.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:22 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T11:18:22.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:22 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:22.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:22 smithi161 ceph-mon[95526]: Deploying daemon osd.10 on smithi175 2023-07-14T11:18:23.835 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:23 smithi175 ceph-mon[129081]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:23 smithi182 ceph-mon[129366]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:23 smithi161 ceph-mon[95526]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:25.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:25 smithi175 ceph-mon[129081]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:25.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:25 smithi182 ceph-mon[129366]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:25.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:25 smithi161 ceph-mon[95526]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:27 smithi175 ceph-mon[129081]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:27.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:27 smithi182 ceph-mon[129366]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:27.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:27.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:27.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:27.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:27 smithi161 ceph-mon[95526]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:28.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.967+0000 7f9a717fa700 1 -- 172.21.15.175:0/3414014596 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9a60000bf0 con 0x7f9a5c0a11d0 2023-07-14T11:18:28.969 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 10 on host 'smithi175' 2023-07-14T11:18:28.971 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.969+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9a5c0a11d0 msgr2=0x7f9a5c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:28.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.969+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9a5c0a11d0 0x7f9a5c0a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a680095c0 tx=0x7f9a68009530 comp rx=0 tx=0).stop 2023-07-14T11:18:28.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.969+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740ffc80 msgr2=0x7f9a7419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:28.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.969+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740ffc80 0x7f9a7419a610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b530 tx=0x7f9a6400b840 comp rx=0 tx=0).stop 2023-07-14T11:18:28.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 shutdown_connections 2023-07-14T11:18:28.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a740fea80 0x7f9a74068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:28.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9a5c0a11d0 0x7f9a5c0a3680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:28.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9a74100610 0x7f9a7419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:28.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 --2- 172.21.15.175:0/3414014596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740ffc80 0x7f9a7419a610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:28.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 >> 172.21.15.175:0/3414014596 conn(0x7f9a740fa0b0 msgr2=0x7f9a741031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:28.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 shutdown_connections 2023-07-14T11:18:28.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-07-14T11:18:28.970+0000 7f9a7a2c6700 1 -- 172.21.15.175:0/3414014596 wait complete. 2023-07-14T11:18:29.403 DEBUG:teuthology.orchestra.run.smithi175:osd.10> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.10.service 2023-07-14T11:18:29.405 INFO:tasks.cephadm:Deploying osd.11 on smithi182 with /dev/vg_nvme/lv_1... 2023-07-14T11:18:29.405 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T11:18:29.433 INFO:journalctl@ceph.osd.10.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:18:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:29 smithi182 ceph-mon[129366]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:29 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:29 smithi175 ceph-mon[129081]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:29 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.966 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:18:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:29 smithi161 ceph-mon[95526]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:29 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:30.299 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:18:29 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:18:29.927+0000 7f300a5ff640 -1 osd.10 0 log_to_monitors true 2023-07-14T11:18:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:30 smithi182 ceph-mon[129366]: from='osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:18:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:30 smithi182 ceph-mon[129366]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:18:30.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:30 smithi175 ceph-mon[129081]: from='osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:18:30.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:30 smithi175 ceph-mon[129081]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:18:30.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:30 smithi161 ceph-mon[95526]: from='osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:18:30.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:30 smithi161 ceph-mon[95526]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:18:31.497 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T11:18:31.497 INFO:teuthology.orchestra.run.smithi182.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T11:18:31.497 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 10+0 records in 2023-07-14T11:18:31.497 INFO:teuthology.orchestra.run.smithi182.stderr:10+0 records out 2023-07-14T11:18:31.497 INFO:teuthology.orchestra.run.smithi182.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01448 s, 724 MB/s 2023-07-14T11:18:31.498 INFO:teuthology.orchestra.run.smithi182.stderr: stderr: 2023-07-14T11:18:31.498 INFO:teuthology.orchestra.run.smithi182.stderr:--> Zapping successful for: 2023-07-14T11:18:31.918 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:31 smithi182 ceph-mon[129366]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:31.918 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:31 smithi182 ceph-mon[129366]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T11:18:31.918 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:31 smithi182 ceph-mon[129366]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T11:18:31.919 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:31.919 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:31 smithi182 ceph-mon[129366]: from='osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:18:31.919 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:31 smithi182 ceph-mon[129366]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:18:31.919 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:18:31.665+0000 7f2fffc77700 -1 osd.10 0 waiting for initial osdmap 2023-07-14T11:18:31.920 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:18:31.675+0000 7f2ffb26b700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:18:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-mon[129081]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-mon[129081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T11:18:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-mon[129081]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T11:18:31.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:31.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-mon[129081]: from='osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:18:31.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:31 smithi175 ceph-mon[129081]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:18:31.921 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi182:vg_nvme/lv_1 2023-07-14T11:18:31.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:31 smithi161 ceph-mon[95526]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:31.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:31 smithi161 ceph-mon[95526]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T11:18:31.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:31 smithi161 ceph-mon[95526]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T11:18:31.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:31.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:31 smithi161 ceph-mon[95526]: from='osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:18:31.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:31 smithi161 ceph-mon[95526]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2023-07-14T11:18:32.476 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:18:32.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:18:32.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T11:18:32.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:32.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:32.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:32.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:32.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:32.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] boot 2023-07-14T11:18:32.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T11:18:32.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:32 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:18:32.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T11:18:32.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:32.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:32.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] boot 2023-07-14T11:18:32.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T11:18:32.967 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:32 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2023-07-14T11:18:32.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T11:18:32.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:32.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:32.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:32.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:32.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:32.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:32.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:32.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: osd.10 [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] boot 2023-07-14T11:18:32.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T11:18:32.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:32 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:18:33.665 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.663+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/3738744051 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b980ff6e0 msgr2=0x7f2b98106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:33.666 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.663+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/3738744051 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b980ff6e0 0x7f2b98106f90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b0a0 tx=0x7f2b9000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:18:33.666 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.663+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/3738744051 shutdown_connections 2023-07-14T11:18:33.666 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/3738744051 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b980ff6e0 0x7f2b98106f90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:33.666 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/3738744051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b98100160 0x7f2b98100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:33.667 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/3738744051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b980fed30 0x7f2b980ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:33.667 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/3738744051 >> 172.21.15.182:0/3738744051 conn(0x7f2b980f9890 msgr2=0x7f2b980fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:33.667 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/3738744051 shutdown_connections 2023-07-14T11:18:33.667 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/3738744051 wait complete. 2023-07-14T11:18:33.668 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 Processor -- start 2023-07-14T11:18:33.668 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.664+0000 7f2b9e0bf700 1 -- start start 2023-07-14T11:18:33.668 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b9e0bf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b980fed30 0x7f2b980fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:33.669 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b9e0bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b980ff6e0 0x7f2b9819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:33.669 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b9e0bf700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b98100160 0x7f2b9819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:33.669 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b9e0bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9819d4c0 con 0x7f2b980ff6e0 2023-07-14T11:18:33.669 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b9e0bf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9819d630 con 0x7f2b980fed30 2023-07-14T11:18:33.669 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b9e0bf700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9819d7f0 con 0x7f2b98100160 2023-07-14T11:18:33.669 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b977fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b980fed30 0x7f2b980fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:33.669 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b97fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b98100160 0x7f2b9819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b977fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b980fed30 0x7f2b980fe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.182:33144/0 (socket says 172.21.15.182:33144) 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b977fe700 1 -- 172.21.15.182:0/2534705900 learned_addr learned my addr 172.21.15.182:0/2534705900 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b96ffd700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b980ff6e0 0x7f2b9819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b97fff700 1 -- 172.21.15.182:0/2534705900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b980fed30 msgr2=0x7f2b980fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b97fff700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b980fed30 0x7f2b980fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b97fff700 1 -- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b980ff6e0 msgr2=0x7f2b9819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b97fff700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b980ff6e0 0x7f2b9819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:33.670 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b97fff700 1 -- 172.21.15.182:0/2534705900 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90009d00 con 0x7f2b98100160 2023-07-14T11:18:33.671 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b977fe700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b980fed30 0x7f2b980fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:18:33.671 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.665+0000 7f2b97fff700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b98100160 0x7f2b9819cd20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b070 tx=0x7f2b90004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:33.672 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.667+0000 7f2b94ff9700 1 -- 172.21.15.182:0/2534705900 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000e040 con 0x7f2b98100160 2023-07-14T11:18:33.672 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.667+0000 7f2b94ff9700 1 -- 172.21.15.182:0/2534705900 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b9000cec0 con 0x7f2b98100160 2023-07-14T11:18:33.672 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.667+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b981945a0 con 0x7f2b98100160 2023-07-14T11:18:33.672 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.667+0000 7f2b94ff9700 1 -- 172.21.15.182:0/2534705900 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9001bac0 con 0x7f2b98100160 2023-07-14T11:18:33.672 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.667+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b98194aa0 con 0x7f2b98100160 2023-07-14T11:18:33.672 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.668+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b98106040 con 0x7f2b98100160 2023-07-14T11:18:33.672 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.669+0000 7f2b94ff9700 1 -- 172.21.15.182:0/2534705900 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2b9001bc20 con 0x7f2b98100160 2023-07-14T11:18:33.673 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.670+0000 7f2b94ff9700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2b800a11d0 0x7f2b800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:33.673 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.670+0000 7f2b94ff9700 1 -- 172.21.15.182:0/2534705900 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f2b900ce170 con 0x7f2b98100160 2023-07-14T11:18:33.673 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.670+0000 7f2b977fe700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2b800a11d0 0x7f2b800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:33.673 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.671+0000 7f2b977fe700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2b800a11d0 0x7f2b800a3680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005950 tx=0x7f2b8800a400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:33.676 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.674+0000 7f2b94ff9700 1 -- 172.21.15.182:0/2534705900 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2b90097f80 con 0x7f2b98100160 2023-07-14T11:18:33.871 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:33.869+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b98060950 con 0x7f2b800a11d0 2023-07-14T11:18:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:33 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:18:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:33 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:18:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:33 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi175 2023-07-14T11:18:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:33 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 4292M 2023-07-14T11:18:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:33 smithi175 ceph-mon[129081]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:33.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:33 smithi175 ceph-mon[129081]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T11:18:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:33 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:18:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:33 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:18:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:33 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi175 2023-07-14T11:18:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:33 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 4292M 2023-07-14T11:18:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:33 smithi182 ceph-mon[129366]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:33.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:33 smithi182 ceph-mon[129366]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T11:18:33.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:33 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:18:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:33 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:18:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:33 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi175 2023-07-14T11:18:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:33 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 4292M 2023-07-14T11:18:33.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:33 smithi161 ceph-mon[95526]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:18:33.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:33 smithi161 ceph-mon[95526]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T11:18:34.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:18:34.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:18:34.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:34.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:34 smithi182 ceph-mon[129366]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T11:18:34.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:18:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:18:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:34 smithi161 ceph-mon[95526]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T11:18:35.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:18:35.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:18:35.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:35.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:34 smithi175 ceph-mon[129081]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T11:18:35.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:35 smithi182 ceph-mon[129366]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:18:35.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:35 smithi182 ceph-mon[129366]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:35.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:35 smithi182 ceph-mon[129366]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T11:18:35.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:35 smithi161 ceph-mon[95526]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:18:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:35 smithi161 ceph-mon[95526]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:35 smithi161 ceph-mon[95526]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T11:18:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:35 smithi175 ceph-mon[129081]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi182:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:18:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:35 smithi175 ceph-mon[129081]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:35 smithi175 ceph-mon[129081]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T11:18:36.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:36 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/4199608018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]: dispatch 2023-07-14T11:18:36.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:36 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]: dispatch 2023-07-14T11:18:36.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:36 smithi182 ceph-mon[129366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]': finished 2023-07-14T11:18:36.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:36 smithi182 ceph-mon[129366]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T11:18:36.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:36.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:36 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/4199608018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]: dispatch 2023-07-14T11:18:36.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:36 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]: dispatch 2023-07-14T11:18:36.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:36 smithi161 ceph-mon[95526]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]': finished 2023-07-14T11:18:36.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:36 smithi161 ceph-mon[95526]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T11:18:36.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:36 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/4199608018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]: dispatch 2023-07-14T11:18:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:36 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]: dispatch 2023-07-14T11:18:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:36 smithi175 ceph-mon[129081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d1d4d3c-b2d7-41e9-ae67-771e27332ebb"}]': finished 2023-07-14T11:18:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:36 smithi175 ceph-mon[129081]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T11:18:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:37.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:37 smithi182 ceph-mon[129366]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:37.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:37 smithi182 ceph-mon[129366]: from='client.? 172.21.15.182:0/3399497525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:18:37.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:37 smithi161 ceph-mon[95526]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:37.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.182:0/3399497525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:18:38.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:37 smithi175 ceph-mon[129081]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:38.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:37 smithi175 ceph-mon[129081]: from='client.? 172.21.15.182:0/3399497525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:18:39.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:39 smithi182 ceph-mon[129366]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:39.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:39 smithi161 ceph-mon[95526]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:40.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:39 smithi175 ceph-mon[129081]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:41.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:41 smithi182 ceph-mon[129366]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:18:41.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:41 smithi161 ceph-mon[95526]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:18:42.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:41 smithi175 ceph-mon[129081]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:18:43.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:43 smithi182 ceph-mon[129366]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:18:43.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T11:18:43.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:43.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:43 smithi161 ceph-mon[95526]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:18:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T11:18:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:43 smithi175 ceph-mon[129081]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-14T11:18:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T11:18:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:44.941 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:44 smithi182 ceph-mon[129366]: Deploying daemon osd.11 on smithi182 2023-07-14T11:18:44.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:44 smithi161 ceph-mon[95526]: Deploying daemon osd.11 on smithi182 2023-07-14T11:18:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:44 smithi175 ceph-mon[129081]: Deploying daemon osd.11 on smithi182 2023-07-14T11:18:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:45 smithi182 ceph-mon[129366]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T11:18:45.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:45 smithi161 ceph-mon[95526]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T11:18:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:45 smithi175 ceph-mon[129081]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T11:18:46.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:46 smithi182 ceph-mon[129366]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:18:46.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:46 smithi161 ceph-mon[95526]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:18:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:46 smithi175 ceph-mon[129081]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:18:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:48 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:48.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:48 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:48.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:48.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:48.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:48 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:49.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:49 smithi175 ceph-mon[129081]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-14T11:18:49.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:49 smithi182 ceph-mon[129366]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-14T11:18:49.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:49 smithi161 ceph-mon[95526]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-14T11:18:50.190 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.187+0000 7f2b94ff9700 1 -- 172.21.15.182:0/2534705900 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f2b98060950 con 0x7f2b800a11d0 2023-07-14T11:18:50.190 INFO:teuthology.orchestra.run.smithi182.stdout:Created osd(s) 11 on host 'smithi182' 2023-07-14T11:18:50.192 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2b800a11d0 msgr2=0x7f2b800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:50.192 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2b800a11d0 0x7f2b800a3680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005950 tx=0x7f2b8800a400 comp rx=0 tx=0).stop 2023-07-14T11:18:50.192 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b98100160 msgr2=0x7f2b9819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:50.192 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b98100160 0x7f2b9819cd20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b070 tx=0x7f2b90004c80 comp rx=0 tx=0).stop 2023-07-14T11:18:50.192 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 shutdown_connections 2023-07-14T11:18:50.193 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2b800a11d0 0x7f2b800a3680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:50.193 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b98100160 0x7f2b9819cd20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:50.193 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b980fed30 0x7f2b980fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:50.193 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 --2- 172.21.15.182:0/2534705900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b980ff6e0 0x7f2b9819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:50.193 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 >> 172.21.15.182:0/2534705900 conn(0x7f2b980f9890 msgr2=0x7f2b98103180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:50.193 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 shutdown_connections 2023-07-14T11:18:50.194 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:18:50.189+0000 7f2b9e0bf700 1 -- 172.21.15.182:0/2534705900 wait complete. 2023-07-14T11:18:50.655 DEBUG:teuthology.orchestra.run.smithi182:osd.11> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.11.service 2023-07-14T11:18:50.658 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-14T11:18:50.658 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd stat -f json 2023-07-14T11:18:50.688 INFO:journalctl@ceph.osd.11.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:18:50.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:50.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:50.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:50.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:50 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:50.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:50.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:50 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:50.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:50.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:18:50.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:50 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:51.395 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:18:51.414 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:18:51 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:18:51.155+0000 7fe7c6785640 -1 osd.11 0 log_to_monitors true 2023-07-14T11:18:51.940 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:51 smithi182 ceph-mon[129366]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:18:51.940 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:51 smithi182 ceph-mon[129366]: from='osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:18:51.940 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:51 smithi182 ceph-mon[129366]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:18:51.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:51 smithi161 ceph-mon[95526]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:18:51.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:51 smithi161 ceph-mon[95526]: from='osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:18:51.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:51 smithi161 ceph-mon[95526]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:18:52.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:51 smithi175 ceph-mon[129081]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:18:52.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:51 smithi175 ceph-mon[129081]: from='osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:18:52.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:51 smithi175 ceph-mon[129081]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:18:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.917+0000 7f2bd5270700 1 -- 172.21.15.161:0/2740726152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0074540 msgr2=0x7f2bd00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.917+0000 7f2bd5270700 1 --2- 172.21.15.161:0/2740726152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0074540 0x7f2bd00729a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009990 tx=0x7f2bc0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:18:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.918+0000 7f2bd5270700 1 -- 172.21.15.161:0/2740726152 shutdown_connections 2023-07-14T11:18:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.918+0000 7f2bd5270700 1 --2- 172.21.15.161:0/2740726152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0074540 0x7f2bd00729a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.918+0000 7f2bd5270700 1 --2- 172.21.15.161:0/2740726152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2bd0107a60 0x7f2bd0109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.918+0000 7f2bd5270700 1 --2- 172.21.15.161:0/2740726152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0072f70 0x7f2bd00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.918+0000 7f2bd5270700 1 -- 172.21.15.161:0/2740726152 >> 172.21.15.161:0/2740726152 conn(0x7f2bd00fb1a0 msgr2=0x7f2bd00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.918+0000 7f2bd5270700 1 -- 172.21.15.161:0/2740726152 shutdown_connections 2023-07-14T11:18:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.918+0000 7f2bd5270700 1 -- 172.21.15.161:0/2740726152 wait complete. 2023-07-14T11:18:52.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 Processor -- start 2023-07-14T11:18:52.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 -- start start 2023-07-14T11:18:52.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0072f70 0x7f2bd0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:52.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0074540 0x7f2bd0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:52.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2bd0107a60 0x7f2bd019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:52.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019f650 con 0x7f2bd0072f70 2023-07-14T11:18:52.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019f790 con 0x7f2bd0074540 2023-07-14T11:18:52.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.919+0000 7f2bd5270700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019f920 con 0x7f2bd0107a60 2023-07-14T11:18:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bce7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0074540 0x7f2bd0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0072f70 0x7f2bd0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2bd0107a60 0x7f2bd019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bce7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0074540 0x7f2bd0198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48700/0 (socket says 172.21.15.161:48700) 2023-07-14T11:18:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bce7fc700 1 -- 172.21.15.161:0/3857382670 learned_addr learned my addr 172.21.15.161:0/3857382670 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:18:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bcf7fe700 1 -- 172.21.15.161:0/3857382670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0074540 msgr2=0x7f2bd0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bcf7fe700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0074540 0x7f2bd0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bcf7fe700 1 -- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0072f70 msgr2=0x7f2bd0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bcf7fe700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0072f70 0x7f2bd0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bcf7fe700 1 -- 172.21.15.161:0/3857382670 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00095f0 con 0x7f2bd0107a60 2023-07-14T11:18:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bceffd700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0072f70 0x7f2bd0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:18:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.920+0000 7f2bcf7fe700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2bd0107a60 0x7f2bd019f080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009f60 tx=0x7f2bc8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.921+0000 7f2bb7fff700 1 -- 172.21.15.161:0/3857382670 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8018070 con 0x7f2bd0107a60 2023-07-14T11:18:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.921+0000 7f2bb7fff700 1 -- 172.21.15.161:0/3857382670 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bc800f040 con 0x7f2bd0107a60 2023-07-14T11:18:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.921+0000 7f2bb7fff700 1 -- 172.21.15.161:0/3857382670 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ccc0 con 0x7f2bd0107a60 2023-07-14T11:18:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.921+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd019fba0 con 0x7f2bd0107a60 2023-07-14T11:18:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.921+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd01a0050 con 0x7f2bd0107a60 2023-07-14T11:18:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.922+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0106110 con 0x7f2bd0107a60 2023-07-14T11:18:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.924+0000 7f2bb7fff700 1 -- 172.21.15.161:0/3857382670 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2bc8003a00 con 0x7f2bd0107a60 2023-07-14T11:18:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.925+0000 7f2bb7fff700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bb80a11d0 0x7f2bb80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.925+0000 7f2bb7fff700 1 -- 172.21.15.161:0/3857382670 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f2bc80cbe10 con 0x7f2bd0107a60 2023-07-14T11:18:52.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.925+0000 7f2bceffd700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bb80a11d0 0x7f2bb80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:52.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.926+0000 7f2bceffd700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bb80a11d0 0x7f2bb80a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b2e0 tx=0x7f2bc0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:52.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:52.929+0000 7f2bb7fff700 1 -- 172.21.15.161:0/3857382670 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2bc80cf030 con 0x7f2bd0107a60 2023-07-14T11:18:52.964 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:18:52.688+0000 7fe7bbdfd700 -1 osd.11 0 waiting for initial osdmap 2023-07-14T11:18:52.964 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:18:52.702+0000 7fe7b53ed700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:18:52.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T11:18:52.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T11:18:52.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: from='osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:18:52.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:52.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:18:52.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:18:52.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T11:18:52.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:52 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:52.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T11:18:52.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T11:18:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: from='osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:18:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:18:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:18:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T11:18:52.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:52 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:53.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.129+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2bd002c850 con 0x7f2bd0107a60 2023-07-14T11:18:53.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.130+0000 7f2bb7fff700 1 -- 172.21.15.161:0/3857382670 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f2bc8095b10 con 0x7f2bd0107a60 2023-07-14T11:18:53.133 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:18:53.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bb80a11d0 msgr2=0x7f2bb80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:53.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bb80a11d0 0x7f2bb80a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b2e0 tx=0x7f2bc0005fb0 comp rx=0 tx=0).stop 2023-07-14T11:18:53.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2bd0107a60 msgr2=0x7f2bd019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:53.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2bd0107a60 0x7f2bd019f080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009f60 tx=0x7f2bc8009f90 comp rx=0 tx=0).stop 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 shutdown_connections 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd0072f70 0x7f2bd0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bb80a11d0 0x7f2bb80a3680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2bd0107a60 0x7f2bd019f080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 --2- 172.21.15.161:0/3857382670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0074540 0x7f2bd0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.132+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 >> 172.21.15.161:0/3857382670 conn(0x7f2bd00fb1a0 msgr2=0x7f2bd00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.133+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 shutdown_connections 2023-07-14T11:18:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:53.133+0000 7f2bd5270700 1 -- 172.21.15.161:0/3857382670 wait complete. 2023-07-14T11:18:53.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T11:18:53.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T11:18:53.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: from='osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:18:53.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:53.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]: dispatch 2023-07-14T11:18:53.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi182", "root=default"]}]': finished 2023-07-14T11:18:53.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T11:18:53.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:52 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:53.673 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1689333512,"num_in_osds":12,"osd_in_since":1689333516,"num_remapped_pgs":0} 2023-07-14T11:18:53.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:53.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:53.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:53.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:53.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:53 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3857382670' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:18:53.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:53.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:53.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:53.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:53.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:53 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3857382670' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:18:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:18:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:18:54.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:53 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3857382670' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:18:54.674 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd stat -f json 2023-07-14T11:18:54.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: purged_snaps scrub starts 2023-07-14T11:18:54.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: purged_snaps scrub ok 2023-07-14T11:18:54.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: Detected new or changed devices on smithi182 2023-07-14T11:18:54.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 4292M 2023-07-14T11:18:54.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:54.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] boot 2023-07-14T11:18:54.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T11:18:54.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:54 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:54.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: purged_snaps scrub starts 2023-07-14T11:18:54.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: purged_snaps scrub ok 2023-07-14T11:18:54.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: Detected new or changed devices on smithi182 2023-07-14T11:18:54.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 4292M 2023-07-14T11:18:54.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:54.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] boot 2023-07-14T11:18:54.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T11:18:54.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:54 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:55.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: purged_snaps scrub starts 2023-07-14T11:18:55.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: purged_snaps scrub ok 2023-07-14T11:18:55.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: Detected new or changed devices on smithi182 2023-07-14T11:18:55.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 4292M 2023-07-14T11:18:55.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:55.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: osd.11 [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] boot 2023-07-14T11:18:55.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T11:18:55.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:54 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:18:55.409 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:18:55.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:55 smithi182 ceph-mon[129366]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:55.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:55 smithi182 ceph-mon[129366]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T11:18:55.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:55 smithi161 ceph-mon[95526]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:55.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:55 smithi161 ceph-mon[95526]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T11:18:56.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:55 smithi175 ceph-mon[129081]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:18:56.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:55 smithi175 ceph-mon[129081]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T11:18:56.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 -- 172.21.15.161:0/1541217333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c480ffb90 msgr2=0x7f5c480fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:56.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/1541217333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c480ffb90 0x7f5c480fffa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009990 tx=0x7f5c38009ca0 comp rx=0 tx=0).stop 2023-07-14T11:18:56.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 -- 172.21.15.161:0/1541217333 shutdown_connections 2023-07-14T11:18:56.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/1541217333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c480ffb90 0x7f5c480fffa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:56.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/1541217333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c48101720 0x7f5c48107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:56.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/1541217333 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c48100d90 0x7f5c481011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:56.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 -- 172.21.15.161:0/1541217333 >> 172.21.15.161:0/1541217333 conn(0x7f5c480fb1a0 msgr2=0x7f5c480fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:56.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 -- 172.21.15.161:0/1541217333 shutdown_connections 2023-07-14T11:18:56.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.926+0000 7f5c4dd12700 1 -- 172.21.15.161:0/1541217333 wait complete. 2023-07-14T11:18:56.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.927+0000 7f5c4dd12700 1 Processor -- start 2023-07-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.927+0000 7f5c4dd12700 1 -- start start 2023-07-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c4dd12700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c480ffb90 0x7f5c480742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c4dd12700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c48100d90 0x7f5c48072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c4dd12700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c48101720 0x7f5c48072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c4dd12700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c480735c0 con 0x7f5c48101720 2023-07-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c4dd12700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48073730 con 0x7f5c480ffb90 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c4dd12700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4819f950 con 0x7f5c48100d90 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c48101720 0x7f5c48072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c477fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c480ffb90 0x7f5c480742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c48101720 0x7f5c48072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53108/0 (socket says 172.21.15.161:53108) 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 -- 172.21.15.161:0/2448034695 learned_addr learned my addr 172.21.15.161:0/2448034695 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c46ffd700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c48100d90 0x7f5c48072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 -- 172.21.15.161:0/2448034695 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c48100d90 msgr2=0x7f5c48072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c48100d90 0x7f5c48072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 -- 172.21.15.161:0/2448034695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c480ffb90 msgr2=0x7f5c480742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c480ffb90 0x7f5c480742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.928+0000 7f5c47fff700 1 -- 172.21.15.161:0/2448034695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380095f0 con 0x7f5c48101720 2023-07-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.929+0000 7f5c47fff700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c48101720 0x7f5c48072e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000ef30 tx=0x7f5c4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.929+0000 7f5c44ff9700 1 -- 172.21.15.161:0/2448034695 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40018070 con 0x7f5c48101720 2023-07-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.929+0000 7f5c44ff9700 1 -- 172.21.15.161:0/2448034695 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c4000f040 con 0x7f5c48101720 2023-07-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.929+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4819fbd0 con 0x7f5c48101720 2023-07-14T11:18:56.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.929+0000 7f5c44ff9700 1 -- 172.21.15.161:0/2448034695 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4000cd00 con 0x7f5c48101720 2023-07-14T11:18:56.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.929+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c481a00d0 con 0x7f5c48101720 2023-07-14T11:18:56.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.930+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c481071e0 con 0x7f5c48101720 2023-07-14T11:18:56.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.932+0000 7f5c44ff9700 1 -- 172.21.15.161:0/2448034695 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5c40003a00 con 0x7f5c48101720 2023-07-14T11:18:56.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.933+0000 7f5c44ff9700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c300a1150 0x7f5c300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:56.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.934+0000 7f5c44ff9700 1 -- 172.21.15.161:0/2448034695 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c400cbff0 con 0x7f5c48101720 2023-07-14T11:18:56.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.934+0000 7f5c477fe700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c300a1150 0x7f5c300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:56.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.934+0000 7f5c477fe700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c300a1150 0x7f5c300a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800b2e0 tx=0x7f5c38005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:56.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:56.937+0000 7f5c44ff9700 1 -- 172.21.15.161:0/2448034695 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c40095bd0 con 0x7f5c48101720 2023-07-14T11:18:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.134+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5c4802c850 con 0x7f5c48101720 2023-07-14T11:18:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.135+0000 7f5c44ff9700 1 -- 172.21.15.161:0/2448034695 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f5c400991f0 con 0x7f5c48101720 2023-07-14T11:18:57.137 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:18:57.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c300a1150 msgr2=0x7f5c300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:57.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c300a1150 0x7f5c300a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800b2e0 tx=0x7f5c38005fb0 comp rx=0 tx=0).stop 2023-07-14T11:18:57.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c48101720 msgr2=0x7f5c48072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:57.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c48101720 0x7f5c48072e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000ef30 tx=0x7f5c4000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:18:57.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 shutdown_connections 2023-07-14T11:18:57.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c48101720 0x7f5c48072e40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:57.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c300a1150 0x7f5c300a3600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:57.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c480ffb90 0x7f5c480742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:57.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 --2- 172.21.15.161:0/2448034695 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c48100d90 0x7f5c48072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:57.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 >> 172.21.15.161:0/2448034695 conn(0x7f5c480fb1a0 msgr2=0x7f5c48104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:57.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 shutdown_connections 2023-07-14T11:18:57.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:57.137+0000 7f5c4dd12700 1 -- 172.21.15.161:0/2448034695 wait complete. 2023-07-14T11:18:57.583 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1689333533,"num_in_osds":12,"osd_in_since":1689333516,"num_remapped_pgs":0} 2023-07-14T11:18:57.584 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd dump --format=json 2023-07-14T11:18:57.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:57 smithi182 ceph-mon[129366]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:18:57.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:57 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2448034695' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:18:57.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:57 smithi161 ceph-mon[95526]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:18:57.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:57 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2448034695' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:18:58.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:57 smithi175 ceph-mon[129081]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:18:58.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:57 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2448034695' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:18:58.311 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:18:58.944 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:18:58 smithi161 ceph-mon[95526]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:18:59.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:18:58 smithi175 ceph-mon[129081]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:18:59.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:18:58 smithi182 ceph-mon[129366]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:18:59.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 -- 172.21.15.161:0/3639370986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0480ffe00 msgr2=0x7fb0481001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:59.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/3639370986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0480ffe00 0x7fb0481001d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0380099c0 tx=0x7fb038009cd0 comp rx=0 tx=0).stop 2023-07-14T11:18:59.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 -- 172.21.15.161:0/3639370986 shutdown_connections 2023-07-14T11:18:59.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/3639370986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0480ffe00 0x7fb0481001d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:59.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/3639370986 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb048104ea0 0x7fb04810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:59.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/3639370986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0481007a0 0x7fb048104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:59.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 -- 172.21.15.161:0/3639370986 >> 172.21.15.161:0/3639370986 conn(0x7fb0480fb660 msgr2=0x7fb0480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:18:59.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 -- 172.21.15.161:0/3639370986 shutdown_connections 2023-07-14T11:18:59.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.918+0000 7fb04ea3f700 1 -- 172.21.15.161:0/3639370986 wait complete. 2023-07-14T11:18:59.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.919+0000 7fb04ea3f700 1 Processor -- start 2023-07-14T11:18:59.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.919+0000 7fb04ea3f700 1 -- start start 2023-07-14T11:18:59.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.919+0000 7fb04ea3f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0480ffe00 0x7fb048197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:59.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.919+0000 7fb04ea3f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0481007a0 0x7fb048198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:59.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.919+0000 7fb04ea3f700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb048104ea0 0x7fb04819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:59.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb04ea3f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04819ce90 con 0x7fb0481007a0 2023-07-14T11:18:59.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb04ea3f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04819d000 con 0x7fb0480ffe00 2023-07-14T11:18:59.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb04ea3f700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04819d1c0 con 0x7fb048104ea0 2023-07-14T11:18:59.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0480ffe00 0x7fb048197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:59.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb0477fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0481007a0 0x7fb048198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:59.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0480ffe00 0x7fb048197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52812/0 (socket says 172.21.15.161:52812) 2023-07-14T11:18:59.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 -- 172.21.15.161:0/2999007938 learned_addr learned my addr 172.21.15.161:0/2999007938 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:18:59.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb04cfdc700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb048104ea0 0x7fb04819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:59.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 -- 172.21.15.161:0/2999007938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb048104ea0 msgr2=0x7fb04819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb048104ea0 0x7fb04819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 -- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0481007a0 msgr2=0x7fb048198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0481007a0 0x7fb048198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 -- 172.21.15.161:0/2999007938 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380095f0 con 0x7fb0480ffe00 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.920+0000 7fb047fff700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0480ffe00 0x7fb048197ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0380098e0 tx=0x7fb03800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.921+0000 7fb0457fa700 1 -- 172.21.15.161:0/2999007938 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03801d070 con 0x7fb0480ffe00 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.921+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04819d440 con 0x7fb0480ffe00 2023-07-14T11:18:59.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.921+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0481a4d00 con 0x7fb0480ffe00 2023-07-14T11:18:59.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.921+0000 7fb0457fa700 1 -- 172.21.15.161:0/2999007938 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb03800fb80 con 0x7fb0480ffe00 2023-07-14T11:18:59.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.921+0000 7fb0457fa700 1 -- 172.21.15.161:0/2999007938 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0380173f0 con 0x7fb0480ffe00 2023-07-14T11:18:59.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.922+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb034005320 con 0x7fb0480ffe00 2023-07-14T11:18:59.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.924+0000 7fb0457fa700 1 -- 172.21.15.161:0/2999007938 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb03800fcf0 con 0x7fb0480ffe00 2023-07-14T11:18:59.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.925+0000 7fb0457fa700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb0300a1150 0x7fb0300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:18:59.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.926+0000 7fb0457fa700 1 -- 172.21.15.161:0/2999007938 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb0380cd4f0 con 0x7fb0480ffe00 2023-07-14T11:18:59.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.926+0000 7fb0477fe700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb0300a1150 0x7fb0300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:18:59.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.926+0000 7fb0477fe700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb0300a1150 0x7fb0300a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0481994f0 tx=0x7fb03c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:18:59.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:18:59.927+0000 7fb0457fa700 1 -- 172.21.15.161:0/2999007938 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb0380970d0 con 0x7fb0480ffe00 2023-07-14T11:19:00.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.125+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb034005190 con 0x7fb0480ffe00 2023-07-14T11:19:00.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.126+0000 7fb0457fa700 1 -- 172.21.15.161:0/2999007938 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18455 (secure 0 0 0) 0x7fb038022470 con 0x7fb0480ffe00 2023-07-14T11:19:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:00.129 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":80,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:18:54.699706+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:00.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb0300a1150 msgr2=0x7fb0300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:00.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb0300a1150 0x7fb0300a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0481994f0 tx=0x7fb03c009260 comp rx=0 tx=0).stop 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0480ffe00 msgr2=0x7fb048197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0480ffe00 0x7fb048197ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0380098e0 tx=0x7fb03800f690 comp rx=0 tx=0).stop 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 shutdown_connections 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb0300a1150 0x7fb0300a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb048104ea0 0x7fb04819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0480ffe00 0x7fb048197ed0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 --2- 172.21.15.161:0/2999007938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0481007a0 0x7fb048198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:00.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 >> 172.21.15.161:0/2999007938 conn(0x7fb0480fb660 msgr2=0x7fb0480fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:00.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 shutdown_connections 2023-07-14T11:19:00.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:00.130+0000 7fb04ea3f700 1 -- 172.21.15.161:0/2999007938 wait complete. 2023-07-14T11:19:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:00 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2999007938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:00.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:00 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2999007938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:00.584 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:19:00.584 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-14T11:19:00.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:00 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2999007938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:19:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:01 smithi182 ceph-mon[129366]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:01.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:01 smithi161 ceph-mon[95526]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:01.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:01 smithi175 ceph-mon[129081]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 -- 172.21.15.161:0/1510211336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94981007a0 msgr2=0x7f9498104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 --2- 172.21.15.161:0/1510211336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94981007a0 0x7f9498104770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f94880099c0 tx=0x7f9488009cd0 comp rx=0 tx=0).stop 2023-07-14T11:19:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 -- 172.21.15.161:0/1510211336 shutdown_connections 2023-07-14T11:19:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 --2- 172.21.15.161:0/1510211336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9498104ea0 0x7f949810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 --2- 172.21.15.161:0/1510211336 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94980ffe00 0x7f94981001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 --2- 172.21.15.161:0/1510211336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94981007a0 0x7f9498104770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 -- 172.21.15.161:0/1510211336 >> 172.21.15.161:0/1510211336 conn(0x7f94980fb660 msgr2=0x7f94980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:02.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 -- 172.21.15.161:0/1510211336 shutdown_connections 2023-07-14T11:19:02.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.813+0000 7f94a0431700 1 -- 172.21.15.161:0/1510211336 wait complete. 2023-07-14T11:19:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.814+0000 7f94a0431700 1 Processor -- start 2023-07-14T11:19:02.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.814+0000 7f94a0431700 1 -- start start 2023-07-14T11:19:02.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f94a0431700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94980ffe00 0x7f9498197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:02.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f94a0431700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94981007a0 0x7f9498198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:02.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f94a0431700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9498104ea0 0x7f949819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:02.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f94a0431700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949819ce90 con 0x7f94980ffe00 2023-07-14T11:19:02.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f94a0431700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949819d000 con 0x7f94981007a0 2023-07-14T11:19:02.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f94a0431700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949819d1c0 con 0x7f9498104ea0 2023-07-14T11:19:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94980ffe00 0x7f9498197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94980ffe00 0x7f9498197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53528/0 (socket says 172.21.15.161:53528) 2023-07-14T11:19:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 -- 172.21.15.161:0/4145760011 learned_addr learned my addr 172.21.15.161:0/4145760011 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949d9cc700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94981007a0 0x7f9498198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e9ce700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9498104ea0 0x7f949819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 -- 172.21.15.161:0/4145760011 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9498104ea0 msgr2=0x7f949819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9498104ea0 0x7f949819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 -- 172.21.15.161:0/4145760011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94981007a0 msgr2=0x7f9498198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94981007a0 0x7f9498198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.815+0000 7f949e1cd700 1 -- 172.21.15.161:0/4145760011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94880095f0 con 0x7f94980ffe00 2023-07-14T11:19:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.816+0000 7f949e1cd700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94980ffe00 0x7f9498197ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9a0 tx=0x7f949400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.816+0000 7f948f7fe700 1 -- 172.21.15.161:0/4145760011 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494009790 con 0x7f94980ffe00 2023-07-14T11:19:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.816+0000 7f948f7fe700 1 -- 172.21.15.161:0/4145760011 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9494010430 con 0x7f94980ffe00 2023-07-14T11:19:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.816+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949819d4a0 con 0x7f94980ffe00 2023-07-14T11:19:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.816+0000 7f948f7fe700 1 -- 172.21.15.161:0/4145760011 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494018680 con 0x7f94980ffe00 2023-07-14T11:19:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.816+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94981a4d00 con 0x7f94980ffe00 2023-07-14T11:19:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.817+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949804e210 con 0x7f94980ffe00 2023-07-14T11:19:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.819+0000 7f948f7fe700 1 -- 172.21.15.161:0/4145760011 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f94940105a0 con 0x7f94980ffe00 2023-07-14T11:19:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.820+0000 7f948f7fe700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94840a11d0 0x7f94840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.821+0000 7f948f7fe700 1 -- 172.21.15.161:0/4145760011 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9494014070 con 0x7f94980ffe00 2023-07-14T11:19:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.821+0000 7f949d9cc700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94840a11d0 0x7f94840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:02.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.821+0000 7f949d9cc700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94840a11d0 0x7f94840a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9488009fd0 tx=0x7f9488005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:02.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:02.825+0000 7f948f7fe700 1 -- 172.21.15.161:0/4145760011 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9494051790 con 0x7f94980ffe00 2023-07-14T11:19:03.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.025+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9498199230 con 0x7f94980ffe00 2023-07-14T11:19:03.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.026+0000 7f948f7fe700 1 -- 172.21.15.161:0/4145760011 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94940961d0 con 0x7f94980ffe00 2023-07-14T11:19:03.029 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:19:03.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94840a11d0 msgr2=0x7f94840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:03.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94840a11d0 0x7f94840a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9488009fd0 tx=0x7f9488005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:03.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94980ffe00 msgr2=0x7f9498197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:03.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94980ffe00 0x7f9498197ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9a0 tx=0x7f949400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:19:03.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 shutdown_connections 2023-07-14T11:19:03.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94980ffe00 0x7f9498197ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:03.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94840a11d0 0x7f94840a3680 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:03.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9498104ea0 0x7f949819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:03.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 --2- 172.21.15.161:0/4145760011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94981007a0 0x7f9498198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:03.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 >> 172.21.15.161:0/4145760011 conn(0x7f94980fb660 msgr2=0x7f94980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:03.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 shutdown_connections 2023-07-14T11:19:03.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:03.028+0000 7f94a0431700 1 -- 172.21.15.161:0/4145760011 wait complete. 2023-07-14T11:19:03.490 INFO:tasks.cephadm:Adding mds.a on smithi161 2023-07-14T11:19:03.491 INFO:tasks.cephadm:Adding mds.d on smithi161 2023-07-14T11:19:03.491 INFO:tasks.cephadm:Adding mds.g on smithi161 2023-07-14T11:19:03.491 INFO:tasks.cephadm:Adding mds.j on smithi161 2023-07-14T11:19:03.491 INFO:tasks.cephadm:Adding mds.b on smithi175 2023-07-14T11:19:03.492 INFO:tasks.cephadm:Adding mds.e on smithi175 2023-07-14T11:19:03.492 INFO:tasks.cephadm:Adding mds.h on smithi175 2023-07-14T11:19:03.492 INFO:tasks.cephadm:Adding mds.k on smithi175 2023-07-14T11:19:03.492 INFO:tasks.cephadm:Adding mds.c on smithi182 2023-07-14T11:19:03.492 INFO:tasks.cephadm:Adding mds.f on smithi182 2023-07-14T11:19:03.492 INFO:tasks.cephadm:Adding mds.i on smithi182 2023-07-14T11:19:03.493 INFO:tasks.cephadm:Adding mds.l on smithi182 2023-07-14T11:19:03.493 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph orch apply mds all '12;smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l' 2023-07-14T11:19:03.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:03 smithi161 ceph-mon[95526]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:03.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:03 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4145760011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:03.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:03 smithi175 ceph-mon[129081]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:03.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:03 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4145760011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:03.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:03 smithi182 ceph-mon[129366]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:03.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:03 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4145760011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:04.062 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:19:05.350 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 -- 172.21.15.182:0/113019552 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80ff410 msgr2=0x7f34d80ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:05.351 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 --2- 172.21.15.182:0/113019552 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80ff410 0x7f34d80ff860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009990 tx=0x7f34c4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.351 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 -- 172.21.15.182:0/113019552 shutdown_connections 2023-07-14T11:19:05.351 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 --2- 172.21.15.182:0/113019552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d80fe210 0x7f34d80fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.351 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 --2- 172.21.15.182:0/113019552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d80ffda0 0x7f34d8106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.351 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 --2- 172.21.15.182:0/113019552 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80ff410 0x7f34d80ff860 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.352 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 -- 172.21.15.182:0/113019552 >> 172.21.15.182:0/113019552 conn(0x7f34d80f9860 msgr2=0x7f34d80fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:05.352 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 -- 172.21.15.182:0/113019552 shutdown_connections 2023-07-14T11:19:05.352 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.348+0000 7f34dea64700 1 -- 172.21.15.182:0/113019552 wait complete. 2023-07-14T11:19:05.352 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 Processor -- start 2023-07-14T11:19:05.352 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 -- start start 2023-07-14T11:19:05.354 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80fe210 0x7f34d80fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:05.354 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d80ff410 0x7f34d819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:05.354 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d80ffda0 0x7f34d819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:05.354 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d819d330 con 0x7f34d80ffda0 2023-07-14T11:19:05.354 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d819d4a0 con 0x7f34d80ff410 2023-07-14T11:19:05.354 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dea64700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d819d660 con 0x7f34d80fe210 2023-07-14T11:19:05.354 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80fe210 0x7f34d80fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:05.355 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80fe210 0x7f34d80fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.182:37402/0 (socket says 172.21.15.182:37402) 2023-07-14T11:19:05.355 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 -- 172.21.15.182:0/2679400306 learned_addr learned my addr 172.21.15.182:0/2679400306 (peer_addr_for_me v2:172.21.15.182:0/0) 2023-07-14T11:19:05.355 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34d7fff700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d80ff410 0x7f34d819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:05.355 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 -- 172.21.15.182:0/2679400306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d80ff410 msgr2=0x7f34d819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:05.355 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d80ff410 0x7f34d819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.355 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 -- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d80ffda0 msgr2=0x7f34d819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:05.355 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d80ffda0 0x7f34d819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 -- 172.21.15.182:0/2679400306 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c40095f0 con 0x7f34d80fe210 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34dc800700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80fe210 0x7f34d80fbbd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00c990 tx=0x7f34cc00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.349+0000 7f34d5ffb700 1 -- 172.21.15.182:0/2679400306 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc007b00 con 0x7f34d80fe210 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.350+0000 7f34d5ffb700 1 -- 172.21.15.182:0/2679400306 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34cc007c60 con 0x7f34d80fe210 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.350+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d819d940 con 0x7f34d80fe210 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.350+0000 7f34d5ffb700 1 -- 172.21.15.182:0/2679400306 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc018660 con 0x7f34d80fe210 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.350+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d819de40 con 0x7f34d80fe210 2023-07-14T11:19:05.356 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.351+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d8105d90 con 0x7f34d80fe210 2023-07-14T11:19:05.357 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.352+0000 7f34d5ffb700 1 -- 172.21.15.182:0/2679400306 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f34cc00f450 con 0x7f34d80fe210 2023-07-14T11:19:05.357 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.352+0000 7f34d5ffb700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f34c80a1150 0x7f34c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:05.358 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.352+0000 7f34d5ffb700 1 -- 172.21.15.182:0/2679400306 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34cc0cc4c0 con 0x7f34d80fe210 2023-07-14T11:19:05.358 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.353+0000 7f34d7fff700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f34c80a1150 0x7f34c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:05.358 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.353+0000 7f34d7fff700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f34c80a1150 0x7f34c80a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006010 tx=0x7f34c400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:05.358 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.356+0000 7f34d5ffb700 1 -- 172.21.15.182:0/2679400306 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f34cc0960b0 con 0x7f34d80fe210 2023-07-14T11:19:05.551 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.549+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f34d8060c10 con 0x7f34c80a1150 2023-07-14T11:19:05.560 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.558+0000 7f34d5ffb700 1 -- 172.21.15.182:0/2679400306 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f34d8060c10 con 0x7f34c80a1150 2023-07-14T11:19:05.561 INFO:teuthology.orchestra.run.smithi182.stdout:Scheduled mds.all update... 2023-07-14T11:19:05.562 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f34c80a1150 msgr2=0x7f34c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:05.563 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f34c80a1150 0x7f34c80a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006010 tx=0x7f34c400b540 comp rx=0 tx=0).stop 2023-07-14T11:19:05.563 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80fe210 msgr2=0x7f34d80fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:05.563 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80fe210 0x7f34d80fbbd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00c990 tx=0x7f34cc00cd50 comp rx=0 tx=0).stop 2023-07-14T11:19:05.563 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 shutdown_connections 2023-07-14T11:19:05.563 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d80ffda0 0x7f34d819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.563 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f34c80a1150 0x7f34c80a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.563 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f34d80fe210 0x7f34d80fbbd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.564 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 --2- 172.21.15.182:0/2679400306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d80ff410 0x7f34d819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:05.564 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 >> 172.21.15.182:0/2679400306 conn(0x7f34d80f9860 msgr2=0x7f34d8102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:05.564 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 shutdown_connections 2023-07-14T11:19:05.564 INFO:teuthology.orchestra.run.smithi182.stderr:2023-07-14T11:19:05.560+0000 7f34dea64700 1 -- 172.21.15.182:0/2679400306 wait complete. 2023-07-14T11:19:05.915 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:05 smithi182 ceph-mon[129366]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:05.915 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:05.916 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:05.917 DEBUG:teuthology.orchestra.run.smithi161:mds.a> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.a.service 2023-07-14T11:19:05.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:05 smithi175 ceph-mon[129081]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:05.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:05.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:05.922 DEBUG:teuthology.orchestra.run.smithi161:mds.d> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.d.service 2023-07-14T11:19:05.967 DEBUG:teuthology.orchestra.run.smithi161:mds.g> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.g.service 2023-07-14T11:19:05.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:05 smithi161 ceph-mon[95526]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:05.988 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:05.988 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:05.991 DEBUG:teuthology.orchestra.run.smithi161:mds.j> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.j.service 2023-07-14T11:19:06.003 INFO:journalctl@ceph.mds.a.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:19:06.007 DEBUG:teuthology.orchestra.run.smithi175:mds.b> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.b.service 2023-07-14T11:19:06.009 DEBUG:teuthology.orchestra.run.smithi175:mds.e> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.e.service 2023-07-14T11:19:06.040 INFO:journalctl@ceph.mds.b.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:19:06.041 DEBUG:teuthology.orchestra.run.smithi175:mds.h> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.h.service 2023-07-14T11:19:06.042 INFO:journalctl@ceph.mds.d.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:19:06.064 INFO:journalctl@ceph.mds.g.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:19:06.069 INFO:journalctl@ceph.mds.e.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:19:06.069 DEBUG:teuthology.orchestra.run.smithi175:mds.k> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.k.service 2023-07-14T11:19:06.079 INFO:journalctl@ceph.mds.j.smithi161.stdout:-- Logs begin at Fri 2023-07-14 10:49:08 UTC. -- 2023-07-14T11:19:06.095 INFO:journalctl@ceph.mds.h.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:19:06.096 DEBUG:teuthology.orchestra.run.smithi182:mds.c> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.c.service 2023-07-14T11:19:06.098 DEBUG:teuthology.orchestra.run.smithi182:mds.f> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.f.service 2023-07-14T11:19:06.120 INFO:journalctl@ceph.mds.k.smithi175.stdout:-- Logs begin at Fri 2023-07-14 10:32:00 UTC. -- 2023-07-14T11:19:06.127 INFO:journalctl@ceph.mds.c.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:19:06.128 DEBUG:teuthology.orchestra.run.smithi182:mds.i> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.i.service 2023-07-14T11:19:06.153 INFO:journalctl@ceph.mds.f.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:19:06.154 DEBUG:teuthology.orchestra.run.smithi182:mds.l> sudo journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.l.service 2023-07-14T11:19:06.179 INFO:journalctl@ceph.mds.i.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:19:06.180 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-14T11:19:06.181 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:19:06.206 INFO:journalctl@ceph.mds.l.smithi182.stdout:-- Logs begin at Fri 2023-07-14 10:38:47 UTC. -- 2023-07-14T11:19:06.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.399+0000 7f374c3ff700 1 -- 172.21.15.161:0/3041744553 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744101680 msgr2=0x7f3744107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.399+0000 7f374c3ff700 1 --2- 172.21.15.161:0/3041744553 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744101680 0x7f3744107f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f373c009f90 tx=0x7f373c00b330 comp rx=0 tx=0).stop 2023-07-14T11:19:06.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.399+0000 7f374c3ff700 1 -- 172.21.15.161:0/3041744553 shutdown_connections 2023-07-14T11:19:06.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.399+0000 7f374c3ff700 1 --2- 172.21.15.161:0/3041744553 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744101680 0x7f3744107f40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.399+0000 7f374c3ff700 1 --2- 172.21.15.161:0/3041744553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37440ffb00 0x7f37440fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.399+0000 7f374c3ff700 1 --2- 172.21.15.161:0/3041744553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744100cf0 0x7f3744101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.399+0000 7f374c3ff700 1 -- 172.21.15.161:0/3041744553 >> 172.21.15.161:0/3041744553 conn(0x7f37440fb0e0 msgr2=0x7f37440fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:06.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.400+0000 7f374c3ff700 1 -- 172.21.15.161:0/3041744553 shutdown_connections 2023-07-14T11:19:06.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.400+0000 7f374c3ff700 1 -- 172.21.15.161:0/3041744553 wait complete. 2023-07-14T11:19:06.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.400+0000 7f374c3ff700 1 Processor -- start 2023-07-14T11:19:06.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374c3ff700 1 -- start start 2023-07-14T11:19:06.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374c3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37440ffb00 0x7f37440742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374c3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744100cf0 0x7f3744072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374c3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744101680 0x7f3744072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374c3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440735e0 con 0x7f3744101680 2023-07-14T11:19:06.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374c3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744073750 con 0x7f37440ffb00 2023-07-14T11:19:06.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374c3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374419f7b0 con 0x7f3744100cf0 2023-07-14T11:19:06.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374a99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744101680 0x7f3744072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374999a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744100cf0 0x7f3744072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374a99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744101680 0x7f3744072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53556/0 (socket says 172.21.15.161:53556) 2023-07-14T11:19:06.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374a99c700 1 -- 172.21.15.161:0/963144493 learned_addr learned my addr 172.21.15.161:0/963144493 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:06.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.401+0000 7f374a19b700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37440ffb00 0x7f37440742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374999a700 1 -- 172.21.15.161:0/963144493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37440ffb00 msgr2=0x7f37440742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374999a700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37440ffb00 0x7f37440742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374999a700 1 -- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744101680 msgr2=0x7f3744072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374999a700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744101680 0x7f3744072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374999a700 1 -- 172.21.15.161:0/963144493 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340095f0 con 0x7f3744100cf0 2023-07-14T11:19:06.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374a99c700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744101680 0x7f3744072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:06.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374999a700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744100cf0 0x7f3744072900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f373400eb90 tx=0x7f373400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:06.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f373b7fe700 1 -- 172.21.15.161:0/963144493 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734009970 con 0x7f3744100cf0 2023-07-14T11:19:06.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f373b7fe700 1 -- 172.21.15.161:0/963144493 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37340044e0 con 0x7f3744100cf0 2023-07-14T11:19:06.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f373b7fe700 1 -- 172.21.15.161:0/963144493 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734003e40 con 0x7f3744100cf0 2023-07-14T11:19:06.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c009c40 con 0x7f3744100cf0 2023-07-14T11:19:06.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.402+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374419fdc0 con 0x7f3744100cf0 2023-07-14T11:19:06.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.403+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3744107000 con 0x7f3744100cf0 2023-07-14T11:19:06.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.405+0000 7f373b7fe700 1 -- 172.21.15.161:0/963144493 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3734004000 con 0x7f3744100cf0 2023-07-14T11:19:06.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.406+0000 7f373b7fe700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f37300a1220 0x7f37300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.407+0000 7f373b7fe700 1 -- 172.21.15.161:0/963144493 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3734014070 con 0x7f3744100cf0 2023-07-14T11:19:06.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.407+0000 7f374a19b700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f37300a1220 0x7f37300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.407+0000 7f374a19b700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f37300a1220 0x7f37300a36d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3744100b50 tx=0x7f3740009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:06.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.408+0000 7f373b7fe700 1 -- 172.21.15.161:0/963144493 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3734096a40 con 0x7f3744100cf0 2023-07-14T11:19:06.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.611+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f374402c850 con 0x7f3744100cf0 2023-07-14T11:19:06.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.612+0000 7f373b7fe700 1 -- 172.21.15.161:0/963144493 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18455 (secure 0 0 0) 0x7f373409a060 con 0x7f3744100cf0 2023-07-14T11:19:06.615 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:06.615 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":80,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:18:54.699706+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:06.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f37300a1220 msgr2=0x7f37300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f37300a1220 0x7f37300a36d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3744100b50 tx=0x7f3740009260 comp rx=0 tx=0).stop 2023-07-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744100cf0 msgr2=0x7f3744072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744100cf0 0x7f3744072900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f373400eb90 tx=0x7f373400eea0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 shutdown_connections 2023-07-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3744101680 0x7f3744072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f37300a1220 0x7f37300a36d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37440ffb00 0x7f37440742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 --2- 172.21.15.161:0/963144493 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3744100cf0 0x7f3744072900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 >> 172.21.15.161:0/963144493 conn(0x7f37440fb0e0 msgr2=0x7f3744104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:06.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 shutdown_connections 2023-07-14T11:19:06.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.614+0000 7f374c3ff700 1 -- 172.21.15.161:0/963144493 wait complete. 2023-07-14T11:19:06.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:19:06.637 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:19:06.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 -- 172.21.15.161:0/404647614 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924101670 msgr2=0x7f4924107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 --2- 172.21.15.161:0/404647614 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924101670 0x7f4924107f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f491c00b0a0 tx=0x7f491c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 -- 172.21.15.161:0/404647614 shutdown_connections 2023-07-14T11:19:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 --2- 172.21.15.161:0/404647614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49240ffaf0 0x7f49240fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 --2- 172.21.15.161:0/404647614 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924101670 0x7f4924107f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 --2- 172.21.15.161:0/404647614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4924100ce0 0x7f4924101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 -- 172.21.15.161:0/404647614 >> 172.21.15.161:0/404647614 conn(0x7f49240fb130 msgr2=0x7f49240fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:06.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 -- 172.21.15.161:0/404647614 shutdown_connections 2023-07-14T11:19:06.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.836+0000 7f4928dff700 1 -- 172.21.15.161:0/404647614 wait complete. 2023-07-14T11:19:06.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.837+0000 7f4928dff700 1 Processor -- start 2023-07-14T11:19:06.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.837+0000 7f4928dff700 1 -- start start 2023-07-14T11:19:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.837+0000 7f4928dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49240ffaf0 0x7f49240742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.837+0000 7f4928dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924100ce0 0x7f4924072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.837+0000 7f4928dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4924101670 0x7f4924072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.837+0000 7f4928dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49240735e0 con 0x7f4924101670 2023-07-14T11:19:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f4928dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924073750 con 0x7f49240ffaf0 2023-07-14T11:19:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f4928dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492419f7a0 con 0x7f4924100ce0 2023-07-14T11:19:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f491bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924100ce0 0x7f4924072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f491bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924100ce0 0x7f4924072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47118/0 (socket says 172.21.15.161:47118) 2023-07-14T11:19:06.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f491bfff700 1 -- 172.21.15.161:0/1041756072 learned_addr learned my addr 172.21.15.161:0/1041756072 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:06.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f492259c700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49240ffaf0 0x7f49240742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f4922d9d700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4924101670 0x7f4924072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f492259c700 1 -- 172.21.15.161:0/1041756072 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924100ce0 msgr2=0x7f4924072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f492259c700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924100ce0 0x7f4924072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f492259c700 1 -- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4924101670 msgr2=0x7f4924072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:06.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f492259c700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4924101670 0x7f4924072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:06.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f492259c700 1 -- 172.21.15.161:0/1041756072 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c009d00 con 0x7f49240ffaf0 2023-07-14T11:19:06.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f491bfff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924100ce0 0x7f4924072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:06.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f492259c700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49240ffaf0 0x7f49240742c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9c0 tx=0x7f491400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:06.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.838+0000 7f491b7fe700 1 -- 172.21.15.161:0/1041756072 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914004220 con 0x7f49240ffaf0 2023-07-14T11:19:06.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.839+0000 7f491b7fe700 1 -- 172.21.15.161:0/1041756072 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4914004380 con 0x7f49240ffaf0 2023-07-14T11:19:06.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.839+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492419fa80 con 0x7f49240ffaf0 2023-07-14T11:19:06.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.839+0000 7f491b7fe700 1 -- 172.21.15.161:0/1041756072 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914003be0 con 0x7f49240ffaf0 2023-07-14T11:19:06.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.839+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492419ff60 con 0x7f49240ffaf0 2023-07-14T11:19:06.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.840+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492410b260 con 0x7f49240ffaf0 2023-07-14T11:19:06.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.841+0000 7f491b7fe700 1 -- 172.21.15.161:0/1041756072 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4914003d40 con 0x7f49240ffaf0 2023-07-14T11:19:06.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.843+0000 7f491b7fe700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49100a1150 0x7f49100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:06.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.843+0000 7f491b7fe700 1 -- 172.21.15.161:0/1041756072 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4914014070 con 0x7f49240ffaf0 2023-07-14T11:19:06.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.843+0000 7f491bfff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49100a1150 0x7f49100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:06.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.844+0000 7f491bfff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49100a1150 0x7f49100a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f490c011550 tx=0x7f490c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:06.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:06.845+0000 7f491b7fe700 1 -- 172.21.15.161:0/1041756072 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f49140968d0 con 0x7f49240ffaf0 2023-07-14T11:19:06.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:19:06.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: Saving service mds.all spec with placement smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l;count:12 2023-07-14T11:19:06.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:06.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:06.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:06.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:06.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:06.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:06.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:06 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/963144493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:19:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: Saving service mds.all spec with placement smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l;count:12 2023-07-14T11:19:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:06.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:06.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:06.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:06.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:06 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/963144493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:06.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:19:06.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: Saving service mds.all spec with placement smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l;count:12 2023-07-14T11:19:06.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:06.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:06.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:06.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:06.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:06.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:06.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:06 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/963144493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:07.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.044+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4924106950 con 0x7f49240ffaf0 2023-07-14T11:19:07.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.044+0000 7f491b7fe700 1 -- 172.21.15.161:0/1041756072 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f491401f340 con 0x7f49240ffaf0 2023-07-14T11:19:07.047 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:19:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49100a1150 msgr2=0x7f49100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49100a1150 0x7f49100a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f490c011550 tx=0x7f490c009260 comp rx=0 tx=0).stop 2023-07-14T11:19:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49240ffaf0 msgr2=0x7f49240742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49240ffaf0 0x7f49240742c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9c0 tx=0x7f491400ecd0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 shutdown_connections 2023-07-14T11:19:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4924101670 0x7f4924072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49100a1150 0x7f49100a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49240ffaf0 0x7f49240742c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 --2- 172.21.15.161:0/1041756072 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4924100ce0 0x7f4924072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 >> 172.21.15.161:0/1041756072 conn(0x7f49240fb130 msgr2=0x7f49240fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 shutdown_connections 2023-07-14T11:19:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.047+0000 7f4928dff700 1 -- 172.21.15.161:0/1041756072 wait complete. 2023-07-14T11:19:07.068 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 -- 172.21.15.161:0/352759372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9640ffb00 msgr2=0x7fd9640fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 --2- 172.21.15.161:0/352759372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9640ffb00 0x7fd9640fff10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd958009990 tx=0x7fd958009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 -- 172.21.15.161:0/352759372 shutdown_connections 2023-07-14T11:19:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 --2- 172.21.15.161:0/352759372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9640ffb00 0x7fd9640fff10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 --2- 172.21.15.161:0/352759372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd964101680 0x7fd964107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 --2- 172.21.15.161:0/352759372 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd964100cf0 0x7fd964101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 -- 172.21.15.161:0/352759372 >> 172.21.15.161:0/352759372 conn(0x7fd9640fb0e0 msgr2=0x7fd9640fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 -- 172.21.15.161:0/352759372 shutdown_connections 2023-07-14T11:19:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.263+0000 7fd96adff700 1 -- 172.21.15.161:0/352759372 wait complete. 2023-07-14T11:19:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 Processor -- start 2023-07-14T11:19:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 -- start start 2023-07-14T11:19:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9640ffb00 0x7fd9640742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd964100cf0 0x7fd964072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd964101680 0x7fd964072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640735e0 con 0x7fd964100cf0 2023-07-14T11:19:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd964073750 con 0x7fd964101680 2023-07-14T11:19:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.264+0000 7fd96adff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96419f870 con 0x7fd9640ffb00 2023-07-14T11:19:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9640ffb00 0x7fd9640742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9640ffb00 0x7fd9640742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47136/0 (socket says 172.21.15.161:47136) 2023-07-14T11:19:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 -- 172.21.15.161:0/3014520838 learned_addr learned my addr 172.21.15.161:0/3014520838 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd963fff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd964100cf0 0x7fd964072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd96939c700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd964101680 0x7fd964072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 -- 172.21.15.161:0/3014520838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd964101680 msgr2=0x7fd964072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd964101680 0x7fd964072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 -- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd964100cf0 msgr2=0x7fd964072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd964100cf0 0x7fd964072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 -- 172.21.15.161:0/3014520838 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9580095f0 con 0x7fd9640ffb00 2023-07-14T11:19:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd968b9b700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9640ffb00 0x7fd9640742c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd958005740 tx=0x7fd958005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd961ffb700 1 -- 172.21.15.161:0/3014520838 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95801d070 con 0x7fd9640ffb00 2023-07-14T11:19:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd961ffb700 1 -- 172.21.15.161:0/3014520838 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd958021410 con 0x7fd9640ffb00 2023-07-14T11:19:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96419faf0 con 0x7fd9640ffb00 2023-07-14T11:19:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.265+0000 7fd961ffb700 1 -- 172.21.15.161:0/3014520838 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd958017650 con 0x7fd9640ffb00 2023-07-14T11:19:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.266+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96419ff10 con 0x7fd9640ffb00 2023-07-14T11:19:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.266+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964107050 con 0x7fd9640ffb00 2023-07-14T11:19:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.269+0000 7fd961ffb700 1 -- 172.21.15.161:0/3014520838 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd958021580 con 0x7fd9640ffb00 2023-07-14T11:19:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.270+0000 7fd961ffb700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9540a11a0 0x7fd9540a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.270+0000 7fd961ffb700 1 -- 172.21.15.161:0/3014520838 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9580cd670 con 0x7fd9640ffb00 2023-07-14T11:19:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.270+0000 7fd963fff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9540a11a0 0x7fd9540a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.271+0000 7fd963fff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9540a11a0 0x7fd9540a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd950009c60 tx=0x7fd950009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.272+0000 7fd961ffb700 1 -- 172.21.15.161:0/3014520838 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd958097250 con 0x7fd9640ffb00 2023-07-14T11:19:07.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.513+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd96402c850 con 0x7fd9640ffb00 2023-07-14T11:19:07.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.513+0000 7fd961ffb700 1 -- 172.21.15.161:0/3014520838 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fd95809a870 con 0x7fd9640ffb00 2023-07-14T11:19:07.516 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:07.516 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-14T11:19:07.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9540a11a0 msgr2=0x7fd9540a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:07.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9540a11a0 0x7fd9540a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd950009c60 tx=0x7fd950009260 comp rx=0 tx=0).stop 2023-07-14T11:19:07.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9640ffb00 msgr2=0x7fd9640742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:07.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9640ffb00 0x7fd9640742c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd958005740 tx=0x7fd958005770 comp rx=0 tx=0).stop 2023-07-14T11:19:07.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 shutdown_connections 2023-07-14T11:19:07.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9540a11a0 0x7fd9540a3650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd964101680 0x7fd964072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9640ffb00 0x7fd9640742c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 --2- 172.21.15.161:0/3014520838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd964100cf0 0x7fd964072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:07.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 >> 172.21.15.161:0/3014520838 conn(0x7fd9640fb0e0 msgr2=0x7fd964104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:07.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 shutdown_connections 2023-07-14T11:19:07.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:07.516+0000 7fd96adff700 1 -- 172.21.15.161:0/3014520838 wait complete. 2023-07-14T11:19:07.520 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 1 2023-07-14T11:19:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:07 smithi182 ceph-mon[129366]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:07 smithi182 ceph-mon[129366]: Deploying daemon mds.l on smithi182 2023-07-14T11:19:07.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:07 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1041756072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:07.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:07 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3014520838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:07.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:07 smithi175 ceph-mon[129081]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:07.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:07 smithi175 ceph-mon[129081]: Deploying daemon mds.l on smithi182 2023-07-14T11:19:07.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:07 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1041756072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:07.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:07 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3014520838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:07.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:07 smithi161 ceph-mon[95526]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:07.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:07 smithi161 ceph-mon[95526]: Deploying daemon mds.l on smithi182 2023-07-14T11:19:07.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:07 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1041756072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:07.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:07 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3014520838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:08.010 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:19:07 smithi182 systemd[1]: Starting Ceph mds.l for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:08.265 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:19:08 smithi182 podman[154792]: 2023-07-14T11:19:08.714 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:19:08 smithi182 ceph-mds[154832]: -- 172.21.15.182:0/1199306800 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b1e3ed91e0 con 0x55b1e3eaf000 2023-07-14T11:19:08.714 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:19:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-l[154808]: starting mds.l at 2023-07-14T11:19:08.714 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:19:08 smithi182 bash[154792]: 037286ff5c428c06a3843ac1ddd23baef3c0f1506bacc98cea55154b3a627e9d 2023-07-14T11:19:08.714 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:19:08 smithi182 systemd[1]: Started Ceph mds.l for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:09.334 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:09.334 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6832/2580899731,v1:172.21.15.182:6833/2580899731] up:boot 2023-07-14T11:19:09.334 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: fsmap 1 up:standby 2023-07-14T11:19:09.334 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:19:09.335 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.335 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.335 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.335 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:09.335 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:09.335 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:09.335 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:09 smithi175 ceph-mon[129081]: Deploying daemon mds.k on smithi175 2023-07-14T11:19:09.537 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:09.604 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:09.604 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6832/2580899731,v1:172.21.15.182:6833/2580899731] up:boot 2023-07-14T11:19:09.605 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: fsmap 1 up:standby 2023-07-14T11:19:09.605 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:19:09.605 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.605 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.605 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.605 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:09.605 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:09.606 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:09.606 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:09 smithi161 ceph-mon[95526]: Deploying daemon mds.k on smithi175 2023-07-14T11:19:09.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:09.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6832/2580899731,v1:172.21.15.182:6833/2580899731] up:boot 2023-07-14T11:19:09.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: fsmap 1 up:standby 2023-07-14T11:19:09.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:19:09.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:09.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:09.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:09.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:09.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:09 smithi182 ceph-mon[129366]: Deploying daemon mds.k on smithi175 2023-07-14T11:19:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 -- 172.21.15.161:0/779219556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62100ffb00 msgr2=0x7f62100fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 --2- 172.21.15.161:0/779219556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62100ffb00 0x7f62100fff10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f620c009990 tx=0x7f620c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 -- 172.21.15.161:0/779219556 shutdown_connections 2023-07-14T11:19:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 --2- 172.21.15.161:0/779219556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6210101680 0x7f6210107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 --2- 172.21.15.161:0/779219556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62100ffb00 0x7f62100fff10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:09.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 --2- 172.21.15.161:0/779219556 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6210100cf0 0x7f6210101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:09.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 -- 172.21.15.161:0/779219556 >> 172.21.15.161:0/779219556 conn(0x7f62100fb0e0 msgr2=0x7f62100fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:09.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 -- 172.21.15.161:0/779219556 shutdown_connections 2023-07-14T11:19:09.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.743+0000 7f62185ff700 1 -- 172.21.15.161:0/779219556 wait complete. 2023-07-14T11:19:09.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.744+0000 7f62185ff700 1 Processor -- start 2023-07-14T11:19:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f62185ff700 1 -- start start 2023-07-14T11:19:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f62185ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f62100ffb00 0x7f62100742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f62185ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210100cf0 0x7f6210072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f62185ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6210101680 0x7f6210072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f62185ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100735e0 con 0x7f6210101680 2023-07-14T11:19:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f62185ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210073750 con 0x7f6210100cf0 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f62185ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621019f890 con 0x7f62100ffb00 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210100cf0 0x7f6210072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210100cf0 0x7f6210072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:58726/0 (socket says 172.21.15.161:58726) 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 -- 172.21.15.161:0/1919902197 learned_addr learned my addr 172.21.15.161:0/1919902197 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f621639b700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f62100ffb00 0x7f62100742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6216b9c700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6210101680 0x7f6210072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 -- 172.21.15.161:0/1919902197 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f62100ffb00 msgr2=0x7f62100742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f62100ffb00 0x7f62100742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 -- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6210101680 msgr2=0x7f6210072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6210101680 0x7f6210072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.745+0000 7f6215b9a700 1 -- 172.21.15.161:0/1919902197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0095f0 con 0x7f6210100cf0 2023-07-14T11:19:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.746+0000 7f6215b9a700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210100cf0 0x7f6210072900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f620000e9a0 tx=0x7f620000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.746+0000 7f62077fe700 1 -- 172.21.15.161:0/1919902197 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200009790 con 0x7f6210100cf0 2023-07-14T11:19:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.746+0000 7f62077fe700 1 -- 172.21.15.161:0/1919902197 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6200010430 con 0x7f6210100cf0 2023-07-14T11:19:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.746+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621019fb70 con 0x7f6210100cf0 2023-07-14T11:19:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.746+0000 7f62077fe700 1 -- 172.21.15.161:0/1919902197 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200018680 con 0x7f6210100cf0 2023-07-14T11:19:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.746+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62101a0080 con 0x7f6210100cf0 2023-07-14T11:19:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.747+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6210107070 con 0x7f6210100cf0 2023-07-14T11:19:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.749+0000 7f62077fe700 1 -- 172.21.15.161:0/1919902197 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f62000105a0 con 0x7f6210100cf0 2023-07-14T11:19:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.750+0000 7f62077fe700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f61fc0a11a0 0x7f61fc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.750+0000 7f62077fe700 1 -- 172.21.15.161:0/1919902197 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6200014070 con 0x7f6210100cf0 2023-07-14T11:19:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.751+0000 7f621639b700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f61fc0a11a0 0x7f61fc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.751+0000 7f621639b700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f61fc0a11a0 0x7f61fc0a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f620c00b2e0 tx=0x7f620c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.752+0000 7f62077fe700 1 -- 172.21.15.161:0/1919902197 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6200096400 con 0x7f6210100cf0 2023-07-14T11:19:10.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.999+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f621002c850 con 0x7f6210100cf0 2023-07-14T11:19:10.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:09.999+0000 7f62077fe700 1 -- 172.21.15.161:0/1919902197 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f6200099a20 con 0x7f6210100cf0 2023-07-14T11:19:10.002 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:10.002 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T11:19:10.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f61fc0a11a0 msgr2=0x7f61fc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:10.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f61fc0a11a0 0x7f61fc0a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f620c00b2e0 tx=0x7f620c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:10.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210100cf0 msgr2=0x7f6210072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:10.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210100cf0 0x7f6210072900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f620000e9a0 tx=0x7f620000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:19:10.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 shutdown_connections 2023-07-14T11:19:10.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6210101680 0x7f6210072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:10.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f61fc0a11a0 0x7f61fc0a3650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:10.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f62100ffb00 0x7f62100742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:10.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 --2- 172.21.15.161:0/1919902197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210100cf0 0x7f6210072900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:10.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 >> 172.21.15.161:0/1919902197 conn(0x7f62100fb0e0 msgr2=0x7f62101041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:10.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 shutdown_connections 2023-07-14T11:19:10.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:10.002+0000 7f62185ff700 1 -- 172.21.15.161:0/1919902197 wait complete. 2023-07-14T11:19:10.006 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 2 2023-07-14T11:19:10.169 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:19:09 smithi175 systemd[1]: Starting Ceph mds.k for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:10.450 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:10 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1919902197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:10.450 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:19:10 smithi175 podman[153534]: 2023-07-14T11:19:10.451 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:19:10 smithi175 ceph-mds[153574]: -- 172.21.15.175:0/249766931 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f2b21f51e0 con 0x55f2b21cac00 2023-07-14T11:19:10.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:10 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1919902197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:10.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:10 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1919902197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:10.919 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:19:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-k[153550]: starting mds.k at 2023-07-14T11:19:10.919 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:19:10 smithi175 bash[153534]: 27b662b4f2eeff83b241d678143cb429ded94c7e900c50654451066168444922 2023-07-14T11:19:10.919 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:19:10 smithi175 systemd[1]: Started Ceph mds.k for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:11.817 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:11.817 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.817 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.817 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.818 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:11.818 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:11.818 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:11.818 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: Deploying daemon mds.j on smithi161 2023-07-14T11:19:11.818 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6832/3899543425,v1:172.21.15.175:6833/3899543425] up:boot 2023-07-14T11:19:11.818 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: fsmap 2 up:standby 2023-07-14T11:19:11.818 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:11 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:19:11.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:11.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:11.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:11.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:11.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: Deploying daemon mds.j on smithi161 2023-07-14T11:19:11.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6832/3899543425,v1:172.21.15.175:6833/3899543425] up:boot 2023-07-14T11:19:11.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: fsmap 2 up:standby 2023-07-14T11:19:11.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:11 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:19:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: Deploying daemon mds.j on smithi161 2023-07-14T11:19:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6832/3899543425,v1:172.21.15.175:6833/3899543425] up:boot 2023-07-14T11:19:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: fsmap 2 up:standby 2023-07-14T11:19:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:11 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:19:12.023 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:12.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 -- 172.21.15.161:0/2512509350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 msgr2=0x7fdc4c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:12.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 --2- 172.21.15.161:0/2512509350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 0x7fdc4c101150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009990 tx=0x7fdc40009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 -- 172.21.15.161:0/2512509350 shutdown_connections 2023-07-14T11:19:12.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 --2- 172.21.15.161:0/2512509350 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c101690 0x7fdc4c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 --2- 172.21.15.161:0/2512509350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c0ffb10 0x7fdc4c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 --2- 172.21.15.161:0/2512509350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 0x7fdc4c101150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 -- 172.21.15.161:0/2512509350 >> 172.21.15.161:0/2512509350 conn(0x7fdc4c0fb110 msgr2=0x7fdc4c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:12.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.249+0000 7fdc513ff700 1 -- 172.21.15.161:0/2512509350 shutdown_connections 2023-07-14T11:19:12.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.250+0000 7fdc513ff700 1 -- 172.21.15.161:0/2512509350 wait complete. 2023-07-14T11:19:12.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 Processor -- start 2023-07-14T11:19:12.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 -- start start 2023-07-14T11:19:12.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c0ffb10 0x7fdc4c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:12.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 0x7fdc4c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:12.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c101690 0x7fdc4c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:12.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19f5b0 con 0x7fdc4c100d00 2023-07-14T11:19:12.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19f6f0 con 0x7fdc4c101690 2023-07-14T11:19:12.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.251+0000 7fdc513ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19f880 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c0ffb10 0x7fdc4c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:12.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c0ffb10 0x7fdc4c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:54026/0 (socket says 172.21.15.161:54026) 2023-07-14T11:19:12.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 0x7fdc4c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:12.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 -- 172.21.15.161:0/3464159489 learned_addr learned my addr 172.21.15.161:0/3464159489 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:12.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4b7fe700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c101690 0x7fdc4c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:12.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 -- 172.21.15.161:0/3464159489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c101690 msgr2=0x7fdc4c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:12.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c101690 0x7fdc4c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 -- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 msgr2=0x7fdc4c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:12.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 0x7fdc4c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 -- 172.21.15.161:0/3464159489 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009520 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.252+0000 7fdc4affd700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c0ffb10 0x7fdc4c197a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ea50 tx=0x7fdc3c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:12.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.253+0000 7fdc33fff700 1 -- 172.21.15.161:0/3464159489 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00eee0 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.253+0000 7fdc33fff700 1 -- 172.21.15.161:0/3464159489 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3c00cbd0 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.253+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc44009c40 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.253+0000 7fdc33fff700 1 -- 172.21.15.161:0/3464159489 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c018680 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.253+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400095f0 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.254+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c1921f0 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.256+0000 7fdc33fff700 1 -- 172.21.15.161:0/3464159489 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdc3c010430 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.257+0000 7fdc33fff700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc340a11a0 0x7fdc340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:12.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.257+0000 7fdc33fff700 1 -- 172.21.15.161:0/3464159489 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdc3c014070 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.258+0000 7fdc4a7fc700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc340a11a0 0x7fdc340a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:12.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.258+0000 7fdc4a7fc700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc340a11a0 0x7fdc340a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000b2e0 tx=0x7fdc40005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:12.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.259+0000 7fdc33fff700 1 -- 172.21.15.161:0/3464159489 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdc3c096590 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.528+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc4c02c850 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.529+0000 7fdc33fff700 1 -- 172.21.15.161:0/3464159489 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7fdc3c094e60 con 0x7fdc4c0ffb10 2023-07-14T11:19:12.531 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:12.532 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-14T11:19:12.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.532+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc340a11a0 msgr2=0x7fdc340a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:12.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.532+0000 7fdc513ff700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc340a11a0 0x7fdc340a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000b2e0 tx=0x7fdc40005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.532+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c0ffb10 msgr2=0x7fdc4c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:12.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.532+0000 7fdc513ff700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c0ffb10 0x7fdc4c197a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ea50 tx=0x7fdc3c00ed60 comp rx=0 tx=0).stop 2023-07-14T11:19:12.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 shutdown_connections 2023-07-14T11:19:12.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc340a11a0 0x7fdc340a3650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc4c101690 0x7fdc4c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc4c0ffb10 0x7fdc4c197a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 --2- 172.21.15.161:0/3464159489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc4c100d00 0x7fdc4c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:12.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 >> 172.21.15.161:0/3464159489 conn(0x7fdc4c0fb110 msgr2=0x7fdc4c1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:12.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 shutdown_connections 2023-07-14T11:19:12.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:12.533+0000 7fdc513ff700 1 -- 172.21.15.161:0/3464159489 wait complete. 2023-07-14T11:19:12.537 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 3 2023-07-14T11:19:12.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:12 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3464159489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:12.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:12 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3464159489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:12.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:12 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3464159489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:12.996 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:19:12 smithi161 systemd[1]: Starting Ceph mds.j for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:13.473 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:19:13 smithi161 podman[129031]: 2023-07-14T11:19:13.745 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:19:13 smithi161 bash[129031]: d210d876b8a2e471eaf5bffca91260b4650a4d02a6de78636f8362db75873cde 2023-07-14T11:19:13.746 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mds[129071]: -- 172.21.15.161:0/835415390 <== mon.0 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573b56a91e0 con 0x5573b567ec00 2023-07-14T11:19:13.746 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-j[129047]: starting mds.j at 2023-07-14T11:19:13.746 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:19:13 smithi161 systemd[1]: Started Ceph mds.j for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:13.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mon[95526]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:13.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:13.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:13.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:13 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:13 smithi175 ceph-mon[129081]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:13.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:13 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:13 smithi182 ceph-mon[129366]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:13.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:13.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:13 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:14.568 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:14.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 -- 172.21.15.161:0/2001860748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b40ffb00 msgr2=0x7f78b40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:14.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/2001860748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b40ffb00 0x7f78b40fff10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009990 tx=0x7f78a4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:14.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 -- 172.21.15.161:0/2001860748 shutdown_connections 2023-07-14T11:19:14.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/2001860748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b40ffb00 0x7f78b40fff10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:14.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/2001860748 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b4101680 0x7f78b4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:14.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/2001860748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b4100cf0 0x7f78b4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:14.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 -- 172.21.15.161:0/2001860748 >> 172.21.15.161:0/2001860748 conn(0x7f78b40fb0e0 msgr2=0x7f78b40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:14.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 -- 172.21.15.161:0/2001860748 shutdown_connections 2023-07-14T11:19:14.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.781+0000 7f78ba7ff700 1 -- 172.21.15.161:0/2001860748 wait complete. 2023-07-14T11:19:14.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.782+0000 7f78ba7ff700 1 Processor -- start 2023-07-14T11:19:14.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.782+0000 7f78ba7ff700 1 -- start start 2023-07-14T11:19:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.782+0000 7f78ba7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b40ffb00 0x7f78b4195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78ba7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b4100cf0 0x7f78b4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78ba7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b4101680 0x7f78b419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78ba7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b419d390 con 0x7f78b4101680 2023-07-14T11:19:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78ba7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b419d4d0 con 0x7f78b4100cf0 2023-07-14T11:19:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78ba7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b419d660 con 0x7f78b40ffb00 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b8d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b4101680 0x7f78b419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b40ffb00 0x7f78b4195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b40ffb00 0x7f78b4195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:54056/0 (socket says 172.21.15.161:54056) 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b4100cf0 0x7f78b4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 -- 172.21.15.161:0/4088069517 learned_addr learned my addr 172.21.15.161:0/4088069517 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 -- 172.21.15.161:0/4088069517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b4100cf0 msgr2=0x7f78b4195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b4100cf0 0x7f78b4195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 -- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b4101680 msgr2=0x7f78b419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b4101680 0x7f78b419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b3fff700 1 -- 172.21.15.161:0/4088069517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40095f0 con 0x7f78b40ffb00 2023-07-14T11:19:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.783+0000 7f78b8d9c700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b4101680 0x7f78b419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.784+0000 7f78b3fff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b40ffb00 0x7f78b4195810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78a4004730 tx=0x7f78a4004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.784+0000 7f78b17fa700 1 -- 172.21.15.161:0/4088069517 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a401d070 con 0x7f78b40ffb00 2023-07-14T11:19:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.784+0000 7f78b17fa700 1 -- 172.21.15.161:0/4088069517 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a4021410 con 0x7f78b40ffb00 2023-07-14T11:19:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.784+0000 7f78b17fa700 1 -- 172.21.15.161:0/4088069517 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4017650 con 0x7f78b40ffb00 2023-07-14T11:19:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.784+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b419d8e0 con 0x7f78b40ffb00 2023-07-14T11:19:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.784+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b419dcb0 con 0x7f78b40ffb00 2023-07-14T11:19:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.785+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b4107050 con 0x7f78b40ffb00 2023-07-14T11:19:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.787+0000 7f78b17fa700 1 -- 172.21.15.161:0/4088069517 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f78a4021580 con 0x7f78b40ffb00 2023-07-14T11:19:14.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.789+0000 7f78b17fa700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f789c0a1220 0x7f789c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.789+0000 7f78b17fa700 1 -- 172.21.15.161:0/4088069517 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f78a40cd7f0 con 0x7f78b40ffb00 2023-07-14T11:19:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.790+0000 7f78b37fe700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f789c0a1220 0x7f789c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.790+0000 7f78b37fe700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f789c0a1220 0x7f789c0a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009c60 tx=0x7f78a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:14.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:14.793+0000 7f78b17fa700 1 -- 172.21.15.161:0/4088069517 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78a4097450 con 0x7f78b40ffb00 2023-07-14T11:19:14.842 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:14 smithi161 ceph-mon[95526]: Deploying daemon mds.d on smithi161 2023-07-14T11:19:14.842 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:14 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.161:6834/2631325720,v1:172.21.15.161:6835/2631325720] up:boot 2023-07-14T11:19:14.842 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:14 smithi161 ceph-mon[95526]: fsmap 3 up:standby 2023-07-14T11:19:14.842 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:19:14.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:14 smithi182 ceph-mon[129366]: Deploying daemon mds.d on smithi161 2023-07-14T11:19:14.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:14 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.161:6834/2631325720,v1:172.21.15.161:6835/2631325720] up:boot 2023-07-14T11:19:14.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:14 smithi182 ceph-mon[129366]: fsmap 3 up:standby 2023-07-14T11:19:14.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:19:15.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.045+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78b402c850 con 0x7f78b40ffb00 2023-07-14T11:19:15.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.045+0000 7f78b17fa700 1 -- 172.21.15.161:0/4088069517 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2780 (secure 0 0 0) 0x7f78a409aa70 con 0x7f78b40ffb00 2023-07-14T11:19:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:15.048 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-14T11:19:15.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f789c0a1220 msgr2=0x7f789c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:15.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f789c0a1220 0x7f789c0a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009c60 tx=0x7f78a8009260 comp rx=0 tx=0).stop 2023-07-14T11:19:15.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b40ffb00 msgr2=0x7f78b4195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:15.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b40ffb00 0x7f78b4195810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78a4004730 tx=0x7f78a4004760 comp rx=0 tx=0).stop 2023-07-14T11:19:15.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 shutdown_connections 2023-07-14T11:19:15.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78b4101680 0x7f78b419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:15.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f789c0a1220 0x7f789c0a36d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:15.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78b40ffb00 0x7f78b4195810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:15.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 --2- 172.21.15.161:0/4088069517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b4100cf0 0x7f78b4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:15.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 >> 172.21.15.161:0/4088069517 conn(0x7f78b40fb0e0 msgr2=0x7f78b4104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:15.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 shutdown_connections 2023-07-14T11:19:15.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:15.048+0000 7f78ba7ff700 1 -- 172.21.15.161:0/4088069517 wait complete. 2023-07-14T11:19:15.052 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 4 2023-07-14T11:19:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:14 smithi175 ceph-mon[129081]: Deploying daemon mds.d on smithi161 2023-07-14T11:19:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:14 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.161:6834/2631325720,v1:172.21.15.161:6835/2631325720] up:boot 2023-07-14T11:19:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:14 smithi175 ceph-mon[129081]: fsmap 3 up:standby 2023-07-14T11:19:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:19:15.777 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:15 smithi161 ceph-mon[95526]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:15.777 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:15 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4088069517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:15.777 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:19:15 smithi161 systemd[1]: Starting Ceph mds.d for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:15 smithi182 ceph-mon[129366]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:15.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:15 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4088069517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:15 smithi175 ceph-mon[129081]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:15 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4088069517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:16.489 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:19:16 smithi161 podman[129414]: 2023-07-14T11:19:16.746 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:19:16 smithi161 bash[129414]: 6f02dba4a649cbf04181af33ea16530c0b25defeb6e06e947e9579640ded8ed1 2023-07-14T11:19:16.746 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:19:16 smithi161 ceph-mds[129454]: -- 172.21.15.161:0/3919381479 <== mon.0 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e46642b1e0 con 0x55e466400c00 2023-07-14T11:19:16.746 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:19:16 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-d[129430]: starting mds.d at 2023-07-14T11:19:16.746 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:19:16 smithi161 systemd[1]: Started Ceph mds.d for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:17.071 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:17.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3114085317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 msgr2=0x7fb5ec101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:17.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3114085317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec101020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009990 tx=0x7fb5d4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3114085317 shutdown_connections 2023-07-14T11:19:17.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3114085317 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5ec101560 0x7fb5ec107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3114085317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ec0ff9e0 0x7fb5ec0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3114085317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec101020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3114085317 >> 172.21.15.161:0/3114085317 conn(0x7fb5ec0fafe0 msgr2=0x7fb5ec0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:17.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3114085317 shutdown_connections 2023-07-14T11:19:17.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.274+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3114085317 wait complete. 2023-07-14T11:19:17.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.275+0000 7fb5ebfff700 1 Processor -- start 2023-07-14T11:19:17.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.275+0000 7fb5ebfff700 1 -- start start 2023-07-14T11:19:17.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5ec0ff9e0 0x7fb5ec0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:17.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:17.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ec101560 0x7fb5ec072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:17.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5ebfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0735e0 con 0x7fb5ec100bd0 2023-07-14T11:19:17.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5ebfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec073750 con 0x7fb5ec101560 2023-07-14T11:19:17.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5ebfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec19f5b0 con 0x7fb5ec0ff9e0 2023-07-14T11:19:17.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:17.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54994/0 (socket says 172.21.15.161:54994) 2023-07-14T11:19:17.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 -- 172.21.15.161:0/3685366865 learned_addr learned my addr 172.21.15.161:0/3685366865 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:17.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5eaffd700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5ec0ff9e0 0x7fb5ec0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:17.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5eb7fe700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ec101560 0x7fb5ec072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:17.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 -- 172.21.15.161:0/3685366865 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5ec0ff9e0 msgr2=0x7fb5ec0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:17.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5ec0ff9e0 0x7fb5ec0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 -- 172.21.15.161:0/3685366865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ec101560 msgr2=0x7fb5ec072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:17.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ec101560 0x7fb5ec072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5e3fff700 1 -- 172.21.15.161:0/3685366865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40095f0 con 0x7fb5ec100bd0 2023-07-14T11:19:17.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.276+0000 7fb5eb7fe700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ec101560 0x7fb5ec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:17.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.277+0000 7fb5e3fff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec072900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4005910 tx=0x7fb5d4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:17.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.277+0000 7fb5e8ff9700 1 -- 172.21.15.161:0/3685366865 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d401d070 con 0x7fb5ec100bd0 2023-07-14T11:19:17.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.277+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec19f830 con 0x7fb5ec100bd0 2023-07-14T11:19:17.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.277+0000 7fb5e8ff9700 1 -- 172.21.15.161:0/3685366865 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5d4003f60 con 0x7fb5ec100bd0 2023-07-14T11:19:17.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.277+0000 7fb5e8ff9700 1 -- 172.21.15.161:0/3685366865 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4022ca0 con 0x7fb5ec100bd0 2023-07-14T11:19:17.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.277+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec19fc50 con 0x7fb5ec100bd0 2023-07-14T11:19:17.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.278+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec106e60 con 0x7fb5ec100bd0 2023-07-14T11:19:17.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.280+0000 7fb5e8ff9700 1 -- 172.21.15.161:0/3685366865 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb5d40040d0 con 0x7fb5ec100bd0 2023-07-14T11:19:17.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.281+0000 7fb5e8ff9700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb5d80a1240 0x7fb5d80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:17.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.281+0000 7fb5e8ff9700 1 -- 172.21.15.161:0/3685366865 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb5d40cdab0 con 0x7fb5ec100bd0 2023-07-14T11:19:17.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.281+0000 7fb5eaffd700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb5d80a1240 0x7fb5d80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:17.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.282+0000 7fb5eaffd700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb5d80a1240 0x7fb5d80a36f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec100a30 tx=0x7fb5dc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:17.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.285+0000 7fb5e8ff9700 1 -- 172.21.15.161:0/3685366865 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb5d4097690 con 0x7fb5ec100bd0 2023-07-14T11:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.529+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5ec02c850 con 0x7fb5ec100bd0 2023-07-14T11:19:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.529+0000 7fb5e8ff9700 1 -- 172.21.15.161:0/3685366865 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3540 (secure 0 0 0) 0x7fb5d4027df0 con 0x7fb5ec100bd0 2023-07-14T11:19:17.532 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:17.532 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2023-07-14T11:19:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.531+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb5d80a1240 msgr2=0x7fb5d80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.531+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb5d80a1240 0x7fb5d80a36f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec100a30 tx=0x7fb5dc00b410 comp rx=0 tx=0).stop 2023-07-14T11:19:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.531+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 msgr2=0x7fb5ec072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.531+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec072900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4005910 tx=0x7fb5d4005740 comp rx=0 tx=0).stop 2023-07-14T11:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.531+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 shutdown_connections 2023-07-14T11:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.531+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb5d80a1240 0x7fb5d80a36f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.532+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ec101560 0x7fb5ec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.532+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5ec0ff9e0 0x7fb5ec0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.532+0000 7fb5ebfff700 1 --2- 172.21.15.161:0/3685366865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5ec100bd0 0x7fb5ec072900 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.532+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 >> 172.21.15.161:0/3685366865 conn(0x7fb5ec0fafe0 msgr2=0x7fb5ec103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.532+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 shutdown_connections 2023-07-14T11:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:17.532+0000 7fb5ebfff700 1 -- 172.21.15.161:0/3685366865 wait complete. 2023-07-14T11:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 5 2023-07-14T11:19:17.859 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:17.860 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.860 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.860 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.860 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:17.860 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:17.860 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:17.861 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: Deploying daemon mds.e on smithi175 2023-07-14T11:19:17.861 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.161:6836/4200056265,v1:172.21.15.161:6837/4200056265] up:boot 2023-07-14T11:19:17.861 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: fsmap 4 up:standby 2023-07-14T11:19:17.861 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:19:17.861 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:17 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3685366865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: Deploying daemon mds.e on smithi175 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.161:6836/4200056265,v1:172.21.15.161:6837/4200056265] up:boot 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: fsmap 4 up:standby 2023-07-14T11:19:17.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:19:17.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:17 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3685366865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:17.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:17.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:17.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: Deploying daemon mds.e on smithi175 2023-07-14T11:19:17.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.161:6836/4200056265,v1:172.21.15.161:6837/4200056265] up:boot 2023-07-14T11:19:17.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: fsmap 4 up:standby 2023-07-14T11:19:17.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:19:17.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:17 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3685366865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:18.419 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:19:18 smithi175 systemd[1]: Starting Ceph mds.e for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:18.744 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:19:18 smithi175 podman[153874]: 2023-07-14T11:19:19.169 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:19:18 smithi175 ceph-mds[153915]: -- 172.21.15.175:0/1777032412 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5636201071e0 con 0x5636200dd000 2023-07-14T11:19:19.169 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:19:18 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-e[153890]: starting mds.e at 2023-07-14T11:19:19.169 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:19:18 smithi175 bash[153874]: 4f90790fb86932bd27db95723fd40ba60bad2b23ed1f184b5391694014652572 2023-07-14T11:19:19.169 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:19:18 smithi175 systemd[1]: Started Ceph mds.e for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:19.553 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:19.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 -- 172.21.15.161:0/1058463889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb4074550 msgr2=0x7fdcb40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:19.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/1058463889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb4074550 0x7fdcb40729a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009990 tx=0x7fdcb0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:19.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 -- 172.21.15.161:0/1058463889 shutdown_connections 2023-07-14T11:19:19.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/1058463889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb4074550 0x7fdcb40729a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:19.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/1058463889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb41079c0 0x7fdcb4109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:19.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/1058463889 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcb4072f70 0x7fdcb40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:19.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 -- 172.21.15.161:0/1058463889 >> 172.21.15.161:0/1058463889 conn(0x7fdcb40fb0e0 msgr2=0x7fdcb40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:19.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 -- 172.21.15.161:0/1058463889 shutdown_connections 2023-07-14T11:19:19.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.764+0000 7fdcbbdff700 1 -- 172.21.15.161:0/1058463889 wait complete. 2023-07-14T11:19:19.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 Processor -- start 2023-07-14T11:19:19.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 -- start start 2023-07-14T11:19:19.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb4072f70 0x7fdcb4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:19.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcb4074550 0x7fdcb4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:19.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41079c0 0x7fdcb419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:19.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb419f580 con 0x7fdcb41079c0 2023-07-14T11:19:19.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb419f6c0 con 0x7fdcb4072f70 2023-07-14T11:19:19.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.765+0000 7fdcbbdff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb419f850 con 0x7fdcb4074550 2023-07-14T11:19:19.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcba39c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41079c0 0x7fdcb419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:19.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcb9b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb4072f70 0x7fdcb4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:19.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcb9b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb4072f70 0x7fdcb4197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:42112/0 (socket says 172.21.15.161:42112) 2023-07-14T11:19:19.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcb9b9b700 1 -- 172.21.15.161:0/2413873776 learned_addr learned my addr 172.21.15.161:0/2413873776 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:19.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcb939a700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcb4074550 0x7fdcb4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:19.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcba39c700 1 -- 172.21.15.161:0/2413873776 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcb4074550 msgr2=0x7fdcb4197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:19.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcba39c700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcb4074550 0x7fdcb4197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:19.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcba39c700 1 -- 172.21.15.161:0/2413873776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb4072f70 msgr2=0x7fdcb4197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:19.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcba39c700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb4072f70 0x7fdcb4197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:19.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcba39c700 1 -- 172.21.15.161:0/2413873776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb00095f0 con 0x7fdcb41079c0 2023-07-14T11:19:19.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcb9b9b700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb4072f70 0x7fdcb4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:19.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcba39c700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41079c0 0x7fdcb419efb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00efc0 tx=0x7fdcac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:19.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.766+0000 7fdcaaffd700 1 -- 172.21.15.161:0/2413873776 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac018070 con 0x7fdcb41079c0 2023-07-14T11:19:19.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.767+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb419fad0 con 0x7fdcb41079c0 2023-07-14T11:19:19.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.767+0000 7fdcaaffd700 1 -- 172.21.15.161:0/2413873776 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcac00f040 con 0x7fdcb41079c0 2023-07-14T11:19:19.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.767+0000 7fdcaaffd700 1 -- 172.21.15.161:0/2413873776 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00cd40 con 0x7fdcb41079c0 2023-07-14T11:19:19.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.767+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb41a0030 con 0x7fdcb41079c0 2023-07-14T11:19:19.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.768+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb4106070 con 0x7fdcb41079c0 2023-07-14T11:19:19.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.770+0000 7fdcaaffd700 1 -- 172.21.15.161:0/2413873776 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdcac003a00 con 0x7fdcb41079c0 2023-07-14T11:19:19.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.771+0000 7fdcaaffd700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdca00a1220 0x7fdca00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:19.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.771+0000 7fdcaaffd700 1 -- 172.21.15.161:0/2413873776 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdcac0cc9e0 con 0x7fdcb41079c0 2023-07-14T11:19:19.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.771+0000 7fdcb9b9b700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdca00a1220 0x7fdca00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:19.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.772+0000 7fdcb9b9b700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdca00a1220 0x7fdca00a36d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b680 tx=0x7fdcb0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:19.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:19.773+0000 7fdcaaffd700 1 -- 172.21.15.161:0/2413873776 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdcac0965c0 con 0x7fdcb41079c0 2023-07-14T11:19:19.843 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:19 smithi182 ceph-mon[129366]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:19.844 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.844 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.844 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.844 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:19.844 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:19.844 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:19 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:19 smithi175 ceph-mon[129081]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:19.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:19.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:19 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:19.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:19 smithi161 ceph-mon[95526]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:19 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:20.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.017+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcb402c850 con 0x7fdcb41079c0 2023-07-14T11:19:20.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.017+0000 7fdcaaffd700 1 -- 172.21.15.161:0/2413873776 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4300 (secure 0 0 0) 0x7fdcac099be0 con 0x7fdcb41079c0 2023-07-14T11:19:20.020 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:20.020 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-14T11:19:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdca00a1220 msgr2=0x7fdca00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdca00a1220 0x7fdca00a36d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b680 tx=0x7fdcb0005fd0 comp rx=0 tx=0).stop 2023-07-14T11:19:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41079c0 msgr2=0x7fdcb419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41079c0 0x7fdcb419efb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00efc0 tx=0x7fdcac00c3a0 comp rx=0 tx=0).stop 2023-07-14T11:19:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 shutdown_connections 2023-07-14T11:19:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41079c0 0x7fdcb419efb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdca00a1220 0x7fdca00a36d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb4072f70 0x7fdcb4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 --2- 172.21.15.161:0/2413873776 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcb4074550 0x7fdcb4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 >> 172.21.15.161:0/2413873776 conn(0x7fdcb40fb0e0 msgr2=0x7fdcb40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 shutdown_connections 2023-07-14T11:19:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:20.020+0000 7fdcbbdff700 1 -- 172.21.15.161:0/2413873776 wait complete. 2023-07-14T11:19:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 6 2023-07-14T11:19:20.464 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:19:20 smithi182 systemd[1]: Starting Ceph mds.f for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:20.908 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:19:20 smithi182 podman[155131]: 2023-07-14T11:19:20.909 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:19:20 smithi182 ceph-mds[155171]: -- 172.21.15.182:0/1165833193 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a13cc0b1e0 con 0x55a13cbe1000 2023-07-14T11:19:20.909 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:19:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-f[155147]: starting mds.f at 2023-07-14T11:19:20.909 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:20 smithi182 ceph-mon[129366]: Deploying daemon mds.f on smithi182 2023-07-14T11:19:20.909 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:20 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6834/4098726991,v1:172.21.15.175:6835/4098726991] up:boot 2023-07-14T11:19:20.909 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:20 smithi182 ceph-mon[129366]: fsmap 5 up:standby 2023-07-14T11:19:20.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:20 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:19:20.910 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:20 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2413873776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:20.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:20 smithi175 ceph-mon[129081]: Deploying daemon mds.f on smithi182 2023-07-14T11:19:20.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:20 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6834/4098726991,v1:172.21.15.175:6835/4098726991] up:boot 2023-07-14T11:19:20.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:20 smithi175 ceph-mon[129081]: fsmap 5 up:standby 2023-07-14T11:19:20.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:20 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:19:20.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:20 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2413873776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:20.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:20 smithi161 ceph-mon[95526]: Deploying daemon mds.f on smithi182 2023-07-14T11:19:20.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:20 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6834/4098726991,v1:172.21.15.175:6835/4098726991] up:boot 2023-07-14T11:19:20.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:20 smithi161 ceph-mon[95526]: fsmap 5 up:standby 2023-07-14T11:19:20.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:20 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:19:20.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:20 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2413873776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:21.214 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:19:20 smithi182 bash[155131]: 8f57f0ad8be042f0b5b7856ed6a101da179655baa2667d470bfa5b7ed1f3fe81 2023-07-14T11:19:21.214 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:19:20 smithi182 systemd[1]: Started Ceph mds.f for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:21.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:21 smithi182 ceph-mon[129366]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:21.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:21.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:21.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:21.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:21.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:21.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:21 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:21.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:21 smithi161 ceph-mon[95526]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:21.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:21 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:22.042 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:21 smithi175 ceph-mon[129081]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:21 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/1659181206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f04100cf0 msgr2=0x7f4f04101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/1659181206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f04100cf0 0x7f4f04101140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009990 tx=0x7f4eec009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/1659181206 shutdown_connections 2023-07-14T11:19:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/1659181206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f04101680 0x7f4f04107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/1659181206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f040ffb00 0x7f4f040fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/1659181206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f04100cf0 0x7f4f04101140 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/1659181206 >> 172.21.15.161:0/1659181206 conn(0x7f4f040fb0e0 msgr2=0x7f4f040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/1659181206 shutdown_connections 2023-07-14T11:19:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.248+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/1659181206 wait complete. 2023-07-14T11:19:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 Processor -- start 2023-07-14T11:19:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 -- start start 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f040ffb00 0x7f4f041957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f04100cf0 0x7f4f04195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f04101680 0x7f4f0419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0419d320 con 0x7f4f04101680 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0419d460 con 0x7f4f04100cf0 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f0a1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0419d5f0 con 0x7f4f040ffb00 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.249+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f040ffb00 0x7f4f041957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f040ffb00 0x7f4f041957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:48138/0 (socket says 172.21.15.161:48138) 2023-07-14T11:19:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4efbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f04100cf0 0x7f4f04195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 -- 172.21.15.161:0/2493441745 learned_addr learned my addr 172.21.15.161:0/2493441745 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f03fff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f04101680 0x7f4f0419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 -- 172.21.15.161:0/2493441745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f04100cf0 msgr2=0x7f4f04195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f04100cf0 0x7f4f04195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 -- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f04101680 msgr2=0x7f4f0419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f04101680 0x7f4f0419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 -- 172.21.15.161:0/2493441745 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec0095f0 con 0x7f4f040ffb00 2023-07-14T11:19:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4efbfff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f04100cf0 0x7f4f04195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f037fe700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f040ffb00 0x7f4f041957a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400e970 tx=0x7f4ef400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f017fa700 1 -- 172.21.15.161:0/2493441745 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4009790 con 0x7f4f040ffb00 2023-07-14T11:19:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.250+0000 7f4f017fa700 1 -- 172.21.15.161:0/2493441745 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ef400c7e0 con 0x7f4f040ffb00 2023-07-14T11:19:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.251+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0419d8d0 con 0x7f4f040ffb00 2023-07-14T11:19:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.251+0000 7f4f017fa700 1 -- 172.21.15.161:0/2493441745 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4003c70 con 0x7f4f040ffb00 2023-07-14T11:19:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.251+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0419de30 con 0x7f4f040ffb00 2023-07-14T11:19:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.251+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0410b2c0 con 0x7f4f040ffb00 2023-07-14T11:19:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.254+0000 7f4f017fa700 1 -- 172.21.15.161:0/2493441745 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4ef400cc50 con 0x7f4f040ffb00 2023-07-14T11:19:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.255+0000 7f4f017fa700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ef00a1220 0x7f4ef00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.255+0000 7f4f017fa700 1 -- 172.21.15.161:0/2493441745 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4ef4014070 con 0x7f4f040ffb00 2023-07-14T11:19:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.255+0000 7f4efbfff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ef00a1220 0x7f4ef00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.256+0000 7f4efbfff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ef00a1220 0x7f4ef00a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00b2e0 tx=0x7f4eec005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.257+0000 7f4f017fa700 1 -- 172.21.15.161:0/2493441745 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4ef4096780 con 0x7f4f040ffb00 2023-07-14T11:19:22.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.497+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f04106960 con 0x7f4f040ffb00 2023-07-14T11:19:22.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.498+0000 7f4f017fa700 1 -- 172.21.15.161:0/2493441745 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5060 (secure 0 0 0) 0x7f4ef4099da0 con 0x7f4f040ffb00 2023-07-14T11:19:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:22.501 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-14T11:19:22.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ef00a1220 msgr2=0x7f4ef00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:22.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ef00a1220 0x7f4ef00a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00b2e0 tx=0x7f4eec005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f040ffb00 msgr2=0x7f4f041957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:22.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f040ffb00 0x7f4f041957a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400e970 tx=0x7f4ef400ec80 comp rx=0 tx=0).stop 2023-07-14T11:19:22.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 shutdown_connections 2023-07-14T11:19:22.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f04101680 0x7f4f0419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ef00a1220 0x7f4ef00a36d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4f040ffb00 0x7f4f041957a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 --2- 172.21.15.161:0/2493441745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f04100cf0 0x7f4f04195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 >> 172.21.15.161:0/2493441745 conn(0x7f4f040fb0e0 msgr2=0x7f4f040fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 shutdown_connections 2023-07-14T11:19:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:22.500+0000 7f4f0a1ff700 1 -- 172.21.15.161:0/2493441745 wait complete. 2023-07-14T11:19:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 7 2023-07-14T11:19:22.680 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:19:22 smithi182 systemd[1]: Starting Ceph mds.c for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:22.943 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:19:22 smithi182 podman[155469]: 2023-07-14T11:19:22.943 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:22 smithi182 ceph-mon[129366]: Deploying daemon mds.c on smithi182 2023-07-14T11:19:22.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:22 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6834/1383844371,v1:172.21.15.182:6835/1383844371] up:boot 2023-07-14T11:19:22.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:22 smithi182 ceph-mon[129366]: fsmap 6 up:standby 2023-07-14T11:19:22.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:22 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:19:22.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:22 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2493441745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:22.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:22 smithi161 ceph-mon[95526]: Deploying daemon mds.c on smithi182 2023-07-14T11:19:22.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:22 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6834/1383844371,v1:172.21.15.182:6835/1383844371] up:boot 2023-07-14T11:19:22.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:22 smithi161 ceph-mon[95526]: fsmap 6 up:standby 2023-07-14T11:19:22.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:22 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:19:22.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:22 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2493441745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:23.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:22 smithi175 ceph-mon[129081]: Deploying daemon mds.c on smithi182 2023-07-14T11:19:23.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:22 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6834/1383844371,v1:172.21.15.182:6835/1383844371] up:boot 2023-07-14T11:19:23.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:22 smithi175 ceph-mon[129081]: fsmap 6 up:standby 2023-07-14T11:19:23.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:22 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:19:23.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:22 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2493441745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:23.464 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mds[155509]: -- 172.21.15.182:0/1044050947 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5628bad851e0 con 0x5628bad5b000 2023-07-14T11:19:23.464 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-c[155485]: starting mds.c at 2023-07-14T11:19:23.465 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:19:23 smithi182 bash[155469]: c3bd4257c7ad76687d5e138ffdc97af2c53d49ee98b500b8fa29e053ad605d99 2023-07-14T11:19:23.465 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:19:23 smithi182 systemd[1]: Started Ceph mds.c for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:23.936 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:23 smithi175 ceph-mon[129081]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:23.936 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:23.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:23.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:23 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mon[129366]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:23.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:23.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:23 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:23.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:23 smithi161 ceph-mon[95526]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:23 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:24.522 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:24.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 -- 172.21.15.161:0/295453131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 msgr2=0x7f3ce8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:24.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/295453131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8101140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009990 tx=0x7f3cd8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 -- 172.21.15.161:0/295453131 shutdown_connections 2023-07-14T11:19:24.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/295453131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8101680 0x7f3ce8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/295453131 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ce80ffb00 0x7f3ce80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/295453131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8101140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 -- 172.21.15.161:0/295453131 >> 172.21.15.161:0/295453131 conn(0x7f3ce80fb0e0 msgr2=0x7f3ce80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:24.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 -- 172.21.15.161:0/295453131 shutdown_connections 2023-07-14T11:19:24.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.729+0000 7f3cef9ff700 1 -- 172.21.15.161:0/295453131 wait complete. 2023-07-14T11:19:24.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.730+0000 7f3cef9ff700 1 Processor -- start 2023-07-14T11:19:24.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cef9ff700 1 -- start start 2023-07-14T11:19:24.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cef9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ce80ffb00 0x7f3ce8195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:24.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cef9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:24.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cef9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8101680 0x7f3ce819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:24.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cef9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce819d390 con 0x7f3ce8100cf0 2023-07-14T11:19:24.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cef9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce819d4d0 con 0x7f3ce8101680 2023-07-14T11:19:24.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cef9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce819d660 con 0x7f3ce80ffb00 2023-07-14T11:19:24.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cecf9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:24.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.731+0000 7f3cecf9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48678/0 (socket says 172.21.15.161:48678) 2023-07-14T11:19:24.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cecf9a700 1 -- 172.21.15.161:0/3145547621 learned_addr learned my addr 172.21.15.161:0/3145547621 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:24.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cedf9c700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8101680 0x7f3ce819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:24.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3ced79b700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ce80ffb00 0x7f3ce8195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:24.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cecf9a700 1 -- 172.21.15.161:0/3145547621 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ce80ffb00 msgr2=0x7f3ce8195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:24.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cecf9a700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ce80ffb00 0x7f3ce8195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cecf9a700 1 -- 172.21.15.161:0/3145547621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8101680 msgr2=0x7f3ce819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:24.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cecf9a700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8101680 0x7f3ce819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cecf9a700 1 -- 172.21.15.161:0/3145547621 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80095f0 con 0x7f3ce8100cf0 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cedf9c700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8101680 0x7f3ce819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cecf9a700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8195d50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b620 tx=0x7f3cd8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cde7fc700 1 -- 172.21.15.161:0/3145547621 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd801d070 con 0x7f3ce8100cf0 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cde7fc700 1 -- 172.21.15.161:0/3145547621 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd8022470 con 0x7f3ce8100cf0 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce8072740 con 0x7f3ce8100cf0 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.732+0000 7f3cde7fc700 1 -- 172.21.15.161:0/3145547621 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8017650 con 0x7f3ce8100cf0 2023-07-14T11:19:24.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.733+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce8072bf0 con 0x7f3ce8100cf0 2023-07-14T11:19:24.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.733+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce8107050 con 0x7f3ce8100cf0 2023-07-14T11:19:24.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.735+0000 7f3cde7fc700 1 -- 172.21.15.161:0/3145547621 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3cd80225e0 con 0x7f3ce8100cf0 2023-07-14T11:19:24.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.737+0000 7f3cde7fc700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3cd40a11a0 0x7f3cd40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:24.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.737+0000 7f3cde7fc700 1 -- 172.21.15.161:0/3145547621 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3cd80ce080 con 0x7f3ce8100cf0 2023-07-14T11:19:24.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.737+0000 7f3ced79b700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3cd40a11a0 0x7f3cd40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:24.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.738+0000 7f3ced79b700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3cd40a11a0 0x7f3cd40a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8100b50 tx=0x7f3ce4012040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:24.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.739+0000 7f3cde7fc700 1 -- 172.21.15.161:0/3145547621 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3cd8097d80 con 0x7f3ce8100cf0 2023-07-14T11:19:24.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:24 smithi175 ceph-mon[129081]: Deploying daemon mds.h on smithi175 2023-07-14T11:19:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:24 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6836/1326049792,v1:172.21.15.182:6837/1326049792] up:boot 2023-07-14T11:19:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:24 smithi175 ceph-mon[129081]: fsmap 7 up:standby 2023-07-14T11:19:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:24 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:19:24.920 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:19:24 smithi175 systemd[1]: Starting Ceph mds.h for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:24.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:24 smithi182 ceph-mon[129366]: Deploying daemon mds.h on smithi175 2023-07-14T11:19:24.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:24 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6836/1326049792,v1:172.21.15.182:6837/1326049792] up:boot 2023-07-14T11:19:24.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:24 smithi182 ceph-mon[129366]: fsmap 7 up:standby 2023-07-14T11:19:24.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:24 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:19:24.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.982+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ce802c850 con 0x7f3ce8100cf0 2023-07-14T11:19:24.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.982+0000 7f3cde7fc700 1 -- 172.21.15.161:0/3145547621 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5820 (secure 0 0 0) 0x7f3cd809b3a0 con 0x7f3ce8100cf0 2023-07-14T11:19:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:24.986 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-14T11:19:24.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3cd40a11a0 msgr2=0x7f3cd40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3cd40a11a0 0x7f3cd40a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8100b50 tx=0x7f3ce4012040 comp rx=0 tx=0).stop 2023-07-14T11:19:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 msgr2=0x7f3ce8195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8195d50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b620 tx=0x7f3cd8005740 comp rx=0 tx=0).stop 2023-07-14T11:19:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 shutdown_connections 2023-07-14T11:19:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3cd40a11a0 0x7f3cd40a3650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8101680 0x7f3ce819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ce80ffb00 0x7f3ce8195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 --2- 172.21.15.161:0/3145547621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce8100cf0 0x7f3ce8195d50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 >> 172.21.15.161:0/3145547621 conn(0x7f3ce80fb0e0 msgr2=0x7f3ce8104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 shutdown_connections 2023-07-14T11:19:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:24.985+0000 7f3cef9ff700 1 -- 172.21.15.161:0/3145547621 wait complete. 2023-07-14T11:19:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2023-07-14T11:19:24.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:24 smithi161 ceph-mon[95526]: Deploying daemon mds.h on smithi175 2023-07-14T11:19:24.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:24 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6836/1326049792,v1:172.21.15.182:6837/1326049792] up:boot 2023-07-14T11:19:24.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:24 smithi161 ceph-mon[95526]: fsmap 7 up:standby 2023-07-14T11:19:24.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:24 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:19:25.311 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:19:24 smithi175 podman[154218]: 2023-07-14T11:19:25.311 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mds[154258]: -- 172.21.15.175:0/679996233 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559515831e0 con 0x555951558800 2023-07-14T11:19:25.311 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-h[154234]: starting mds.h at 2023-07-14T11:19:25.312 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:19:25 smithi175 bash[154218]: cdafe630856836f50493319b3bab908a5ef146286c911d674525e7365ec68fd4 2023-07-14T11:19:25.669 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:19:25 smithi175 systemd[1]: Started Ceph mds.h for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:25.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3145547621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:25.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:25 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:25.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:25.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3145547621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:25.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:25.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:25.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:25.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:25.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:25.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:25 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:26.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:26.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3145547621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:26.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:26.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:26.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:26.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:26.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:26.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:25 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:26.964 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:19:26 smithi182 systemd[1]: Starting Ceph mds.i for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:26.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:26 smithi182 ceph-mon[129366]: Deploying daemon mds.i on smithi182 2023-07-14T11:19:26.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:26 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] up:boot 2023-07-14T11:19:26.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:26 smithi182 ceph-mon[129366]: fsmap 8 up:standby 2023-07-14T11:19:26.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:26 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:19:26.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:26 smithi161 ceph-mon[95526]: Deploying daemon mds.i on smithi182 2023-07-14T11:19:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:26 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] up:boot 2023-07-14T11:19:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:26 smithi161 ceph-mon[95526]: fsmap 8 up:standby 2023-07-14T11:19:26.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:26 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:19:27.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:26 smithi175 ceph-mon[129081]: Deploying daemon mds.i on smithi182 2023-07-14T11:19:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:26 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] up:boot 2023-07-14T11:19:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:26 smithi175 ceph-mon[129081]: fsmap 8 up:standby 2023-07-14T11:19:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:26 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:19:27.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 -- 172.21.15.161:0/421398782 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 msgr2=0x7fdbbc073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:27.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/421398782 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc073360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdba40098f0 tx=0x7fdba4009c00 comp rx=0 tx=0).stop 2023-07-14T11:19:27.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 -- 172.21.15.161:0/421398782 shutdown_connections 2023-07-14T11:19:27.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/421398782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0744d0 0x7fdbbc072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/421398782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc1079b0 0x7fdbbc109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/421398782 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc073360 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 -- 172.21.15.161:0/421398782 >> 172.21.15.161:0/421398782 conn(0x7fdbbc0fb0e0 msgr2=0x7fdbbc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:27.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 -- 172.21.15.161:0/421398782 shutdown_connections 2023-07-14T11:19:27.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.213+0000 7fdbc11ff700 1 -- 172.21.15.161:0/421398782 wait complete. 2023-07-14T11:19:27.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 Processor -- start 2023-07-14T11:19:27.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 -- start start 2023-07-14T11:19:27.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:27.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0744d0 0x7fdbbc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:27.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc1079b0 0x7fdbbc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:27.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc19f5e0 con 0x7fdbbc0744d0 2023-07-14T11:19:27.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc19f720 con 0x7fdbbc1079b0 2023-07-14T11:19:27.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbc11ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc19f8b0 con 0x7fdbbc072ef0 2023-07-14T11:19:27.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.214+0000 7fdbbad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:27.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:48176/0 (socket says 172.21.15.161:48176) 2023-07-14T11:19:27.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 -- 172.21.15.161:0/1481976857 learned_addr learned my addr 172.21.15.161:0/1481976857 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:27.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbb3fff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0744d0 0x7fdbbc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 -- 172.21.15.161:0/1481976857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc1079b0 msgr2=0x7fdbbc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbb59e700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc1079b0 0x7fdbbc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc1079b0 0x7fdbbc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 -- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0744d0 msgr2=0x7fdbbc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0744d0 0x7fdbbc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 -- 172.21.15.161:0/1481976857 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac0095f0 con 0x7fdbbc072ef0 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbb59e700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc1079b0 0x7fdbbc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:19:27.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbb3fff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0744d0 0x7fdbbc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:27.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbbad9d700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc197a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00ec10 tx=0x7fdbac00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:27.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbb8d99700 1 -- 172.21.15.161:0/1481976857 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00cd40 con 0x7fdbbc072ef0 2023-07-14T11:19:27.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbb8d99700 1 -- 172.21.15.161:0/1481976857 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbac0044e0 con 0x7fdbbc072ef0 2023-07-14T11:19:27.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb4009c40 con 0x7fdbbc072ef0 2023-07-14T11:19:27.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbb8d99700 1 -- 172.21.15.161:0/1481976857 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac010690 con 0x7fdbbc072ef0 2023-07-14T11:19:27.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.215+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba4009520 con 0x7fdbbc072ef0 2023-07-14T11:19:27.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.217+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc1060f0 con 0x7fdbbc072ef0 2023-07-14T11:19:27.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.218+0000 7fdbb8d99700 1 -- 172.21.15.161:0/1481976857 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdbac003710 con 0x7fdbbc072ef0 2023-07-14T11:19:27.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.220+0000 7fdbb8d99700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdba80a1220 0x7fdba80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:27.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.220+0000 7fdbb8d99700 1 -- 172.21.15.161:0/1481976857 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbac014070 con 0x7fdbbc072ef0 2023-07-14T11:19:27.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.220+0000 7fdbb3fff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdba80a1220 0x7fdba80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:27.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.221+0000 7fdbb3fff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdba80a1220 0x7fdba80a36d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdba4005e90 tx=0x7fdba4019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:27.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.223+0000 7fdbb8d99700 1 -- 172.21.15.161:0/1481976857 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdbac096560 con 0x7fdbbc072ef0 2023-07-14T11:19:27.423 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:19:27 smithi182 podman[155811]: 2023-07-14T11:19:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.463+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbbc02c850 con 0x7fdbbc072ef0 2023-07-14T11:19:27.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.464+0000 7fdbb8d99700 1 -- 172.21.15.161:0/1481976857 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6580 (secure 0 0 0) 0x7fdbac099b80 con 0x7fdbbc072ef0 2023-07-14T11:19:27.467 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:27.468 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-14T11:19:27.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdba80a1220 msgr2=0x7fdba80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:27.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdba80a1220 0x7fdba80a36d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdba4005e90 tx=0x7fdba4019040 comp rx=0 tx=0).stop 2023-07-14T11:19:27.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 msgr2=0x7fdbbc197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:27.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc197a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00ec10 tx=0x7fdbac00ef20 comp rx=0 tx=0).stop 2023-07-14T11:19:27.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 shutdown_connections 2023-07-14T11:19:27.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdba80a1220 0x7fdba80a36d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbbc1079b0 0x7fdbbc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdbbc072ef0 0x7fdbbc197a60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 --2- 172.21.15.161:0/1481976857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0744d0 0x7fdbbc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:27.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 >> 172.21.15.161:0/1481976857 conn(0x7fdbbc0fb0e0 msgr2=0x7fdbbc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:27.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 shutdown_connections 2023-07-14T11:19:27.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:27.466+0000 7fdbc11ff700 1 -- 172.21.15.161:0/1481976857 wait complete. 2023-07-14T11:19:27.472 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 9 2023-07-14T11:19:27.706 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mds[155851]: -- 172.21.15.182:0/3790088583 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b8d98231e0 con 0x55b8d97f8c00 2023-07-14T11:19:27.707 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-i[155827]: starting mds.i at 2023-07-14T11:19:27.707 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:19:27 smithi182 bash[155811]: f74abdfc7b1ea68238c32c1beed2b31122ae41e84bc764d070413806defa2cd9 2023-07-14T11:19:27.707 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:19:27 smithi182 systemd[1]: Started Ceph mds.i for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:27.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1481976857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:27.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:27.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:27 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:27.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:27.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1481976857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:27.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:27.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:27.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:27.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:27.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:27.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:27 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:28.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:28.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1481976857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:28.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:27 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:28 smithi161 ceph-mon[95526]: Deploying daemon mds.g on smithi161 2023-07-14T11:19:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:28 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] up:boot 2023-07-14T11:19:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:28 smithi161 ceph-mon[95526]: fsmap 9 up:standby 2023-07-14T11:19:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:28 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:19:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:28 smithi161 ceph-mon[95526]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:28.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:28 smithi182 ceph-mon[129366]: Deploying daemon mds.g on smithi161 2023-07-14T11:19:28.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:28 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] up:boot 2023-07-14T11:19:28.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:28 smithi182 ceph-mon[129366]: fsmap 9 up:standby 2023-07-14T11:19:28.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:28 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:19:28.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:28 smithi182 ceph-mon[129366]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:29.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:28 smithi175 ceph-mon[129081]: Deploying daemon mds.g on smithi161 2023-07-14T11:19:29.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:28 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] up:boot 2023-07-14T11:19:29.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:28 smithi175 ceph-mon[129081]: fsmap 9 up:standby 2023-07-14T11:19:29.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:28 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:19:29.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:28 smithi175 ceph-mon[129081]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:29.489 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:29.684 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:19:29 smithi161 systemd[1]: Starting Ceph mds.g for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:29.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 -- 172.21.15.161:0/1444169269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 msgr2=0x7fb478101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:29.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/1444169269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 0x7fb478101140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb468009990 tx=0x7fb468009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:29.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 -- 172.21.15.161:0/1444169269 shutdown_connections 2023-07-14T11:19:29.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/1444169269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101680 0x7fb478107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:29.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/1444169269 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb4780ffb00 0x7fb4780fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:29.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/1444169269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 0x7fb478101140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:29.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 -- 172.21.15.161:0/1444169269 >> 172.21.15.161:0/1444169269 conn(0x7fb4780fb0e0 msgr2=0x7fb4780fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:29.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 -- 172.21.15.161:0/1444169269 shutdown_connections 2023-07-14T11:19:29.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.824+0000 7fb47f7ff700 1 -- 172.21.15.161:0/1444169269 wait complete. 2023-07-14T11:19:29.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 Processor -- start 2023-07-14T11:19:29.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 -- start start 2023-07-14T11:19:29.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffb00 0x7fb478105880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:29.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 0x7fb478105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:29.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101680 0x7fb478102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:29.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478102bb0 con 0x7fb478100cf0 2023-07-14T11:19:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478102d20 con 0x7fb4780ffb00 2023-07-14T11:19:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.826+0000 7fb47f7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478102ee0 con 0x7fb478101680 2023-07-14T11:19:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47d59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffb00 0x7fb478105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47cd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 0x7fb478105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47d59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffb00 0x7fb478105880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57206/0 (socket says 172.21.15.161:57206) 2023-07-14T11:19:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47d59b700 1 -- 172.21.15.161:0/2161362216 learned_addr learned my addr 172.21.15.161:0/2161362216 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47dd9c700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101680 0x7fb478102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:29.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47cd9a700 1 -- 172.21.15.161:0/2161362216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101680 msgr2=0x7fb478102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:29.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47cd9a700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101680 0x7fb478102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:29.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47cd9a700 1 -- 172.21.15.161:0/2161362216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffb00 msgr2=0x7fb478105880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:29.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47cd9a700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffb00 0x7fb478105880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:29.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47cd9a700 1 -- 172.21.15.161:0/2161362216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680095f0 con 0x7fb478100cf0 2023-07-14T11:19:29.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47d59b700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffb00 0x7fb478105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47dd9c700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101680 0x7fb478102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.827+0000 7fb47cd9a700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 0x7fb478105dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb46800b620 tx=0x7fb468004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.828+0000 7fb46e7fc700 1 -- 172.21.15.161:0/2161362216 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46801d070 con 0x7fb478100cf0 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.828+0000 7fb46e7fc700 1 -- 172.21.15.161:0/2161362216 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb468021410 con 0x7fb478100cf0 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.828+0000 7fb46e7fc700 1 -- 172.21.15.161:0/2161362216 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468017650 con 0x7fb478100cf0 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.828+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4781a3f00 con 0x7fb478100cf0 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.828+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4781a4270 con 0x7fb478100cf0 2023-07-14T11:19:29.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.829+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47810b320 con 0x7fb478100cf0 2023-07-14T11:19:29.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.833+0000 7fb46e7fc700 1 -- 172.21.15.161:0/2161362216 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb4680217f0 con 0x7fb478100cf0 2023-07-14T11:19:29.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.835+0000 7fb46e7fc700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a12a0 0x7fb4640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:29.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.835+0000 7fb46e7fc700 1 -- 172.21.15.161:0/2161362216 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb4680cdf00 con 0x7fb478100cf0 2023-07-14T11:19:29.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.835+0000 7fb47d59b700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a12a0 0x7fb4640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:29.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.836+0000 7fb47d59b700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a12a0 0x7fb4640a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb474003ef0 tx=0x7fb47400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:29.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:29.838+0000 7fb46e7fc700 1 -- 172.21.15.161:0/2161362216 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb468097ae0 con 0x7fb478100cf0 2023-07-14T11:19:30.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.109+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb47810b600 con 0x7fb478100cf0 2023-07-14T11:19:30.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.109+0000 7fb46e7fc700 1 -- 172.21.15.161:0/2161362216 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7339 (secure 0 0 0) 0x7fb47810b600 con 0x7fb478100cf0 2023-07-14T11:19:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:30.113 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T11:19:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a12a0 msgr2=0x7fb4640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a12a0 0x7fb4640a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb474003ef0 tx=0x7fb47400b040 comp rx=0 tx=0).stop 2023-07-14T11:19:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 msgr2=0x7fb478105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 0x7fb478105dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb46800b620 tx=0x7fb468004730 comp rx=0 tx=0).stop 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 shutdown_connections 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a12a0 0x7fb4640a3750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101680 0x7fb478102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffb00 0x7fb478105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2161362216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cf0 0x7fb478105dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 >> 172.21.15.161:0/2161362216 conn(0x7fb4780fb0e0 msgr2=0x7fb4780fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 shutdown_connections 2023-07-14T11:19:30.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:30.112+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2161362216 wait complete. 2023-07-14T11:19:30.117 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 10 2023-07-14T11:19:30.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:30 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2161362216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:30.420 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:30 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2161362216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:30.421 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:19:30 smithi161 podman[130031]: 2023-07-14T11:19:30.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:30 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2161362216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:30.746 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:19:30 smithi161 bash[130031]: 34a8f12f858bec0715d2607f93a6ca83bae79f79fe9fc63b91fa6e0049d2452b 2023-07-14T11:19:30.746 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:19:30 smithi161 ceph-mds[130073]: -- 172.21.15.161:0/159957945 <== mon.0 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558f81d6d1e0 con 0x558f81d43000 2023-07-14T11:19:30.746 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:19:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-g[130049]: starting mds.g at 2023-07-14T11:19:30.747 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:19:30 smithi161 systemd[1]: Started Ceph mds.g for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:31.840 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:31.840 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.840 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.840 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.841 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:31.841 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:31.841 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:31.841 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: Deploying daemon mds.a on smithi161 2023-07-14T11:19:31.841 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.161:6838/1651269993,v1:172.21.15.161:6839/1651269993] up:boot 2023-07-14T11:19:31.841 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: fsmap 10 up:standby 2023-07-14T11:19:31.841 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:31 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:19:31.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:31.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: Deploying daemon mds.a on smithi161 2023-07-14T11:19:31.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.161:6838/1651269993,v1:172.21.15.161:6839/1651269993] up:boot 2023-07-14T11:19:31.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: fsmap 10 up:standby 2023-07-14T11:19:31.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:31 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:19:31.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: Deploying daemon mds.a on smithi161 2023-07-14T11:19:31.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.161:6838/1651269993,v1:172.21.15.161:6839/1651269993] up:boot 2023-07-14T11:19:31.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: fsmap 10 up:standby 2023-07-14T11:19:31.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:31 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:19:32.135 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 -- 172.21.15.161:0/3851493083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c102ef0 msgr2=0x7fd58c103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 --2- 172.21.15.161:0/3851493083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c102ef0 0x7fd58c103340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009990 tx=0x7fd57c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 -- 172.21.15.161:0/3851493083 shutdown_connections 2023-07-14T11:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 --2- 172.21.15.161:0/3851493083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c103880 0x7fd58c10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 --2- 172.21.15.161:0/3851493083 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd58c101d00 0x7fd58c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 --2- 172.21.15.161:0/3851493083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c102ef0 0x7fd58c103340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 -- 172.21.15.161:0/3851493083 >> 172.21.15.161:0/3851493083 conn(0x7fd58c0fd320 msgr2=0x7fd58c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 -- 172.21.15.161:0/3851493083 shutdown_connections 2023-07-14T11:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.351+0000 7fd5943ff700 1 -- 172.21.15.161:0/3851493083 wait complete. 2023-07-14T11:19:32.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 Processor -- start 2023-07-14T11:19:32.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 -- start start 2023-07-14T11:19:32.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c101d00 0x7fd58c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c102ef0 0x7fd58c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd58c103880 0x7fd58c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c19f5b0 con 0x7fd58c101d00 2023-07-14T11:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c19f6f0 con 0x7fd58c102ef0 2023-07-14T11:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.353+0000 7fd5943ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c19f880 con 0x7fd58c103880 2023-07-14T11:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c102ef0 0x7fd58c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59219b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c101d00 0x7fd58c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c102ef0 0x7fd58c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57234/0 (socket says 172.21.15.161:57234) 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 -- 172.21.15.161:0/4010026883 learned_addr learned my addr 172.21.15.161:0/4010026883 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59299c700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd58c103880 0x7fd58c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 -- 172.21.15.161:0/4010026883 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd58c103880 msgr2=0x7fd58c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd58c103880 0x7fd58c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 -- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c101d00 msgr2=0x7fd58c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c101d00 0x7fd58c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 -- 172.21.15.161:0/4010026883 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c0095f0 con 0x7fd58c102ef0 2023-07-14T11:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59219b700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c101d00 0x7fd58c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.354+0000 7fd59199a700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c102ef0 0x7fd58c197f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00b620 tx=0x7fd57c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.355+0000 7fd5837fe700 1 -- 172.21.15.161:0/4010026883 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01d070 con 0x7fd58c102ef0 2023-07-14T11:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.355+0000 7fd5837fe700 1 -- 172.21.15.161:0/4010026883 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57c022470 con 0x7fd58c102ef0 2023-07-14T11:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.355+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c19fb00 con 0x7fd58c102ef0 2023-07-14T11:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.355+0000 7fd5837fe700 1 -- 172.21.15.161:0/4010026883 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c017650 con 0x7fd58c102ef0 2023-07-14T11:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.355+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c19ff20 con 0x7fd58c102ef0 2023-07-14T11:19:32.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.356+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c10d590 con 0x7fd58c102ef0 2023-07-14T11:19:32.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.358+0000 7fd5837fe700 1 -- 172.21.15.161:0/4010026883 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd57c0225e0 con 0x7fd58c102ef0 2023-07-14T11:19:32.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.359+0000 7fd5837fe700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd5780a11a0 0x7fd5780a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:32.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.359+0000 7fd5837fe700 1 -- 172.21.15.161:0/4010026883 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd57c0ce140 con 0x7fd58c102ef0 2023-07-14T11:19:32.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.360+0000 7fd59219b700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd5780a11a0 0x7fd5780a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:32.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.360+0000 7fd59219b700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd5780a11a0 0x7fd5780a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd58c102d50 tx=0x7fd588009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:32.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.361+0000 7fd5837fe700 1 -- 172.21.15.161:0/4010026883 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd57c097d20 con 0x7fd58c102ef0 2023-07-14T11:19:32.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.651+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd58c108b90 con 0x7fd58c102ef0 2023-07-14T11:19:32.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.652+0000 7fd5837fe700 1 -- 172.21.15.161:0/4010026883 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8100 (secure 0 0 0) 0x7fd57c09b340 con 0x7fd58c102ef0 2023-07-14T11:19:32.656 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:32.656 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T11:19:32.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd5780a11a0 msgr2=0x7fd5780a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:32.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd5780a11a0 0x7fd5780a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd58c102d50 tx=0x7fd588009260 comp rx=0 tx=0).stop 2023-07-14T11:19:32.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c102ef0 msgr2=0x7fd58c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:32.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c102ef0 0x7fd58c197f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00b620 tx=0x7fd57c005740 comp rx=0 tx=0).stop 2023-07-14T11:19:32.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 shutdown_connections 2023-07-14T11:19:32.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd58c101d00 0x7fd58c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd5780a11a0 0x7fd5780a3650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd58c103880 0x7fd58c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 --2- 172.21.15.161:0/4010026883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c102ef0 0x7fd58c197f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:32.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 >> 172.21.15.161:0/4010026883 conn(0x7fd58c0fd320 msgr2=0x7fd58c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 shutdown_connections 2023-07-14T11:19:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:32.655+0000 7fd5943ff700 1 -- 172.21.15.161:0/4010026883 wait complete. 2023-07-14T11:19:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 11 2023-07-14T11:19:33.246 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:19:32 smithi161 systemd[1]: Starting Ceph mds.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:33.555 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:19:33 smithi161 podman[130422]: 2023-07-14T11:19:33.556 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:33 smithi161 ceph-mon[95526]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:33.556 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:33 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4010026883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:33 smithi175 ceph-mon[129081]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:33.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:33 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4010026883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:33 smithi182 ceph-mon[129366]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:33 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4010026883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:33.995 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:19:33 smithi161 ceph-mds[130462]: -- 172.21.15.161:0/3711054079 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559caa76d1e0 con 0x559caa742800 2023-07-14T11:19:33.996 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:19:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-a[130438]: starting mds.a at 2023-07-14T11:19:33.996 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:19:33 smithi161 bash[130422]: a5edc65f8c88981cd80474273363ec60b11a446be149078599ca0d4584fb493d 2023-07-14T11:19:33.996 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:19:33 smithi161 systemd[1]: Started Ceph mds.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:34.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:34.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:34.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:34.874 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:34.874 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: Deploying daemon mds.b on smithi175 2023-07-14T11:19:34.874 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.161:6840/208844218,v1:172.21.15.161:6841/208844218] up:boot 2023-07-14T11:19:34.874 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: fsmap 11 up:standby 2023-07-14T11:19:34.874 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:34 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:19:34.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.888+0000 7f9bb77ff700 1 -- 172.21.15.161:0/2693545105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb01079c0 msgr2=0x7f9bb0109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:34.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.888+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/2693545105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb01079c0 0x7f9bb0109e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800b0a0 tx=0x7f9ba800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:19:34.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.888+0000 7f9bb77ff700 1 -- 172.21.15.161:0/2693545105 shutdown_connections 2023-07-14T11:19:34.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.888+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/2693545105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb01079c0 0x7f9bb0109e40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:34.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.888+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/2693545105 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9bb0074550 0x7f9bb00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:34.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.888+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/2693545105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb0072f70 0x7f9bb00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:34.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.888+0000 7f9bb77ff700 1 -- 172.21.15.161:0/2693545105 >> 172.21.15.161:0/2693545105 conn(0x7f9bb00fb0e0 msgr2=0x7f9bb00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:34.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.889+0000 7f9bb77ff700 1 -- 172.21.15.161:0/2693545105 shutdown_connections 2023-07-14T11:19:34.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.889+0000 7f9bb77ff700 1 -- 172.21.15.161:0/2693545105 wait complete. 2023-07-14T11:19:34.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 Processor -- start 2023-07-14T11:19:34.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 -- start start 2023-07-14T11:19:34.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0072f70 0x7f9bb0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:34.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb0074550 0x7f9bb0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:34.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9bb01079c0 0x7f9bb019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:34.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb019f680 con 0x7f9bb0074550 2023-07-14T11:19:34.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb019f7c0 con 0x7f9bb0072f70 2023-07-14T11:19:34.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb77ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb019f950 con 0x7f9bb01079c0 2023-07-14T11:19:34.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb559b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0072f70 0x7f9bb0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:34.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb559b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0072f70 0x7f9bb0197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57278/0 (socket says 172.21.15.161:57278) 2023-07-14T11:19:34.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb559b700 1 -- 172.21.15.161:0/1382042402 learned_addr learned my addr 172.21.15.161:0/1382042402 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:34.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb5d9c700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9bb01079c0 0x7f9bb019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:34.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.890+0000 7f9bb4d9a700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb0074550 0x7f9bb0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb559b700 1 -- 172.21.15.161:0/1382042402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9bb01079c0 msgr2=0x7f9bb019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb559b700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9bb01079c0 0x7f9bb019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb559b700 1 -- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb0074550 msgr2=0x7f9bb0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb559b700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb0074550 0x7f9bb0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb559b700 1 -- 172.21.15.161:0/1382042402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8009d00 con 0x7f9bb0072f70 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb559b700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0072f70 0x7f9bb0197b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d980 tx=0x7f9bac00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9ba67fc700 1 -- 172.21.15.161:0/1382042402 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00f890 con 0x7f9bb0072f70 2023-07-14T11:19:34.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb019fc30 con 0x7f9bb0072f70 2023-07-14T11:19:34.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9ba67fc700 1 -- 172.21.15.161:0/1382042402 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bac010460 con 0x7f9bb0072f70 2023-07-14T11:19:34.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9ba67fc700 1 -- 172.21.15.161:0/1382042402 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00e610 con 0x7f9bb0072f70 2023-07-14T11:19:34.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.891+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb01a0140 con 0x7f9bb0072f70 2023-07-14T11:19:34.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.892+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb0106780 con 0x7f9bb0072f70 2023-07-14T11:19:34.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.895+0000 7f9ba67fc700 1 -- 172.21.15.161:0/1382042402 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9bac0105d0 con 0x7f9bb0072f70 2023-07-14T11:19:34.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.896+0000 7f9ba67fc700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b9c0a11a0 0x7f9b9c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:34.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.896+0000 7f9ba67fc700 1 -- 172.21.15.161:0/1382042402 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9bac0cbe60 con 0x7f9bb0072f70 2023-07-14T11:19:34.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.896+0000 7f9bb4d9a700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b9c0a11a0 0x7f9b9c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:34.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.897+0000 7f9bb4d9a700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b9c0a11a0 0x7f9b9c0a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0074270 tx=0x7f9ba0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:34.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:34.898+0000 7f9ba67fc700 1 -- 172.21.15.161:0/1382042402 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9bac095a30 con 0x7f9bb0072f70 2023-07-14T11:19:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:34.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:34.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: Deploying daemon mds.b on smithi175 2023-07-14T11:19:34.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.161:6840/208844218,v1:172.21.15.161:6841/208844218] up:boot 2023-07-14T11:19:34.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: fsmap 11 up:standby 2023-07-14T11:19:34.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:34 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:19:34.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:19:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:19:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:34.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: Deploying daemon mds.b on smithi175 2023-07-14T11:19:34.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.161:6840/208844218,v1:172.21.15.161:6841/208844218] up:boot 2023-07-14T11:19:34.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: fsmap 11 up:standby 2023-07-14T11:19:34.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:34 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:19:35.128 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:19:35 smithi175 systemd[1]: Starting Ceph mds.b for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:19:35.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.138+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bb002c850 con 0x7f9bb0072f70 2023-07-14T11:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.139+0000 7f9ba67fc700 1 -- 172.21.15.161:0/1382042402 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8858 (secure 0 0 0) 0x7f9bac099050 con 0x7f9bb0072f70 2023-07-14T11:19:35.142 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:35.142 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T11:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b9c0a11a0 msgr2=0x7f9b9c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b9c0a11a0 0x7f9b9c0a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0074270 tx=0x7f9ba0009260 comp rx=0 tx=0).stop 2023-07-14T11:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0072f70 msgr2=0x7f9bb0197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0072f70 0x7f9bb0197b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d980 tx=0x7f9bac00dc90 comp rx=0 tx=0).stop 2023-07-14T11:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 shutdown_connections 2023-07-14T11:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b9c0a11a0 0x7f9b9c0a3650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:35.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9bb01079c0 0x7f9bb019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:35.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bb0072f70 0x7f9bb0197b00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:35.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 --2- 172.21.15.161:0/1382042402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb0074550 0x7f9bb0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:35.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 >> 172.21.15.161:0/1382042402 conn(0x7f9bb00fb0e0 msgr2=0x7f9bb01038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:35.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 shutdown_connections 2023-07-14T11:19:35.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:35.141+0000 7f9bb77ff700 1 -- 172.21.15.161:0/1382042402 wait complete. 2023-07-14T11:19:35.147 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 12 2023-07-14T11:19:35.669 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:19:35 smithi175 podman[154559]: 2023-07-14T11:19:35.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:35 smithi182 ceph-mon[129366]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:35.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:35 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1382042402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:35.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:35 smithi161 ceph-mon[95526]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:35 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1382042402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:35 smithi175 ceph-mon[129081]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:35 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1382042402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:36.169 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:19:35 smithi175 ceph-mds[154599]: -- 172.21.15.175:0/1166821339 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e7a5471e0 con 0x558e7a51d000 2023-07-14T11:19:36.169 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:19:35 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: starting mds.b at 2023-07-14T11:19:36.170 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:19:35 smithi175 bash[154559]: aeddc20983dd45c34d9182599e8098c080b0bb535b1fd9d82c269223b8724336 2023-07-14T11:19:36.170 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:19:35 smithi175 systemd[1]: Started Ceph mds.b for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:19:37.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:37.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:37.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] up:boot 2023-07-14T11:19:37.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: fsmap 12 up:standby 2023-07-14T11:19:37.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:36 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:19:37.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:37.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:37.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] up:boot 2023-07-14T11:19:37.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: fsmap 12 up:standby 2023-07-14T11:19:37.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:36 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:19:37.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:37.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:37.230 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:37.230 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] up:boot 2023-07-14T11:19:37.230 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: fsmap 12 up:standby 2023-07-14T11:19:37.230 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:36 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:19:37.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.373+0000 7f19f41ff700 1 -- 172.21.15.161:0/1538460243 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec0ffb00 msgr2=0x7f19ec0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.373+0000 7f19f41ff700 1 --2- 172.21.15.161:0/1538460243 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec0ffb00 0x7f19ec0fff10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009990 tx=0x7f19e8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.373+0000 7f19f41ff700 1 -- 172.21.15.161:0/1538460243 shutdown_connections 2023-07-14T11:19:37.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.373+0000 7f19f41ff700 1 --2- 172.21.15.161:0/1538460243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec101680 0x7f19ec107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.373+0000 7f19f41ff700 1 --2- 172.21.15.161:0/1538460243 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec0ffb00 0x7f19ec0fff10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.373+0000 7f19f41ff700 1 --2- 172.21.15.161:0/1538460243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec100cf0 0x7f19ec101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.373+0000 7f19f41ff700 1 -- 172.21.15.161:0/1538460243 >> 172.21.15.161:0/1538460243 conn(0x7f19ec0fb0e0 msgr2=0x7f19ec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:37.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.374+0000 7f19f41ff700 1 -- 172.21.15.161:0/1538460243 shutdown_connections 2023-07-14T11:19:37.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.374+0000 7f19f41ff700 1 -- 172.21.15.161:0/1538460243 wait complete. 2023-07-14T11:19:37.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.375+0000 7f19f41ff700 1 Processor -- start 2023-07-14T11:19:37.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.375+0000 7f19f41ff700 1 -- start start 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f41ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec0ffb00 0x7f19ec1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f41ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec100cf0 0x7f19ec197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f41ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101680 0x7f19ec19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f41ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19f560 con 0x7f19ec101680 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f41ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19f6a0 con 0x7f19ec0ffb00 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f41ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19f830 con 0x7f19ec100cf0 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec100cf0 0x7f19ec197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec0ffb00 0x7f19ec1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec100cf0 0x7f19ec197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:42164/0 (socket says 172.21.15.161:42164) 2023-07-14T11:19:37.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec0ffb00 0x7f19ec1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57302/0 (socket says 172.21.15.161:57302) 2023-07-14T11:19:37.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 -- 172.21.15.161:0/2758669452 learned_addr learned my addr 172.21.15.161:0/2758669452 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f279c700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101680 0x7f19ec19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 -- 172.21.15.161:0/2758669452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec0ffb00 msgr2=0x7f19ec1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec0ffb00 0x7f19ec1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 -- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101680 msgr2=0x7f19ec19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101680 0x7f19ec19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f179a700 1 -- 172.21.15.161:0/2758669452 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80095f0 con 0x7f19ec100cf0 2023-07-14T11:19:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.376+0000 7f19f279c700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101680 0x7f19ec19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.377+0000 7f19f179a700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec100cf0 0x7f19ec197f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e9a0 tx=0x7f19dc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.377+0000 7f19e2ffd700 1 -- 172.21.15.161:0/2758669452 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc009790 con 0x7f19ec100cf0 2023-07-14T11:19:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.377+0000 7f19e2ffd700 1 -- 172.21.15.161:0/2758669452 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19dc00cbd0 con 0x7f19ec100cf0 2023-07-14T11:19:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.377+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec19fb10 con 0x7f19ec100cf0 2023-07-14T11:19:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.377+0000 7f19e2ffd700 1 -- 172.21.15.161:0/2758669452 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc018680 con 0x7f19ec100cf0 2023-07-14T11:19:37.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.377+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1a0020 con 0x7f19ec100cf0 2023-07-14T11:19:37.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.378+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec10b2c0 con 0x7f19ec100cf0 2023-07-14T11:19:37.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.380+0000 7f19e2ffd700 1 -- 172.21.15.161:0/2758669452 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f19dc00cd40 con 0x7f19ec100cf0 2023-07-14T11:19:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.382+0000 7f19e2ffd700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f19d80a11a0 0x7f19d80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.382+0000 7f19e2ffd700 1 -- 172.21.15.161:0/2758669452 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f19dc014070 con 0x7f19ec100cf0 2023-07-14T11:19:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.382+0000 7f19f1f9b700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f19d80a11a0 0x7f19d80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.382+0000 7f19f1f9b700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f19d80a11a0 0x7f19d80a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19e800b2e0 tx=0x7f19e8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:37.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.384+0000 7f19e2ffd700 1 -- 172.21.15.161:0/2758669452 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f19dc096370 con 0x7f19ec100cf0 2023-07-14T11:19:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.653+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19ec1a05d0 con 0x7f19ec100cf0 2023-07-14T11:19:37.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.654+0000 7f19e2ffd700 1 -- 172.21.15.161:0/2758669452 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9619 (secure 0 0 0) 0x7f19dc099990 con 0x7f19ec100cf0 2023-07-14T11:19:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:37.657 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-14T11:19:37.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f19d80a11a0 msgr2=0x7f19d80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f19d80a11a0 0x7f19d80a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19e800b2e0 tx=0x7f19e8005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec100cf0 msgr2=0x7f19ec197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec100cf0 0x7f19ec197f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e9a0 tx=0x7f19dc00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 shutdown_connections 2023-07-14T11:19:37.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101680 0x7f19ec19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f19d80a11a0 0x7f19d80a3650 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec0ffb00 0x7f19ec1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 --2- 172.21.15.161:0/2758669452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f19ec100cf0 0x7f19ec197f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 >> 172.21.15.161:0/2758669452 conn(0x7f19ec0fb0e0 msgr2=0x7f19ec0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:37.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 shutdown_connections 2023-07-14T11:19:37.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.657+0000 7f19f41ff700 1 -- 172.21.15.161:0/2758669452 wait complete. 2023-07-14T11:19:37.661 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 13 2023-07-14T11:19:37.679 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:19:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 -- 172.21.15.161:0/703978691 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5238074550 msgr2=0x7f52380729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 --2- 172.21.15.161:0/703978691 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5238074550 0x7f52380729a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52340098f0 tx=0x7f5234009c00 comp rx=0 tx=0).stop 2023-07-14T11:19:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 -- 172.21.15.161:0/703978691 shutdown_connections 2023-07-14T11:19:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 --2- 172.21.15.161:0/703978691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f523810c0d0 0x7f523810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 --2- 172.21.15.161:0/703978691 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5238074550 0x7f52380729a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 --2- 172.21.15.161:0/703978691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238072f70 0x7f52380733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 -- 172.21.15.161:0/703978691 >> 172.21.15.161:0/703978691 conn(0x7f52380fb0e0 msgr2=0x7f52380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 -- 172.21.15.161:0/703978691 shutdown_connections 2023-07-14T11:19:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.891+0000 7f523fdff700 1 -- 172.21.15.161:0/703978691 wait complete. 2023-07-14T11:19:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.893+0000 7f523fdff700 1 Processor -- start 2023-07-14T11:19:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.893+0000 7f523fdff700 1 -- start start 2023-07-14T11:19:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.893+0000 7f523fdff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5238072f70 0x7f523819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.893+0000 7f523fdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238074550 0x7f523819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523fdff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f523810c0d0 0x7f52381a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523fdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a3a30 con 0x7f5238074550 2023-07-14T11:19:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523fdff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a3b70 con 0x7f5238072f70 2023-07-14T11:19:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523fdff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a3d00 con 0x7f523810c0d0 2023-07-14T11:19:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523d39a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238074550 0x7f523819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523d39a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238074550 0x7f523819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48628/0 (socket says 172.21.15.161:48628) 2023-07-14T11:19:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523d39a700 1 -- 172.21.15.161:0/1724007195 learned_addr learned my addr 172.21.15.161:0/1724007195 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523e39c700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f523810c0d0 0x7f52381a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523db9b700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5238072f70 0x7f523819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523e39c700 1 -- 172.21.15.161:0/1724007195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5238072f70 msgr2=0x7f523819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523e39c700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5238072f70 0x7f523819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523e39c700 1 -- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238074550 msgr2=0x7f523819c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523e39c700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238074550 0x7f523819c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523e39c700 1 -- 172.21.15.161:0/1724007195 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280095f0 con 0x7f523810c0d0 2023-07-14T11:19:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523d39a700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238074550 0x7f523819c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523db9b700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5238072f70 0x7f523819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.894+0000 7f523e39c700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f523810c0d0 0x7f52381a3460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f523000efc0 tx=0x7f523000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.895+0000 7f522effd700 1 -- 172.21.15.161:0/1724007195 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230018070 con 0x7f523810c0d0 2023-07-14T11:19:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.895+0000 7f522effd700 1 -- 172.21.15.161:0/1724007195 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523000f040 con 0x7f523810c0d0 2023-07-14T11:19:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.895+0000 7f522effd700 1 -- 172.21.15.161:0/1724007195 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523000cd40 con 0x7f523810c0d0 2023-07-14T11:19:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.895+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5234009520 con 0x7f523810c0d0 2023-07-14T11:19:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.895+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381a4220 con 0x7f523810c0d0 2023-07-14T11:19:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.896+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5238101c20 con 0x7f523810c0d0 2023-07-14T11:19:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.899+0000 7f522effd700 1 -- 172.21.15.161:0/1724007195 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5230003a00 con 0x7f523810c0d0 2023-07-14T11:19:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.900+0000 7f522effd700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f52240a1220 0x7f52240a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.901+0000 7f522effd700 1 -- 172.21.15.161:0/1724007195 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f52300cbf40 con 0x7f523810c0d0 2023-07-14T11:19:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.901+0000 7f523db9b700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f52240a1220 0x7f52240a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.901+0000 7f523db9b700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f52240a1220 0x7f52240a36d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5234009fd0 tx=0x7f5234019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:37.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:37.904+0000 7f522effd700 1 -- 172.21.15.161:0/1724007195 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5230095bc0 con 0x7f523810c0d0 2023-07-14T11:19:37.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2758669452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:38.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.113+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52381119f0 con 0x7f523810c0d0 2023-07-14T11:19:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.114+0000 7f522effd700 1 -- 172.21.15.161:0/1724007195 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18455 (secure 0 0 0) 0x7f52300991e0 con 0x7f523810c0d0 2023-07-14T11:19:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:38.117 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":80,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:18:54.699706+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:38.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f52240a1220 msgr2=0x7f52240a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f52240a1220 0x7f52240a36d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5234009fd0 tx=0x7f5234019040 comp rx=0 tx=0).stop 2023-07-14T11:19:38.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f523810c0d0 msgr2=0x7f52381a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f523810c0d0 0x7f52381a3460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f523000efc0 tx=0x7f523000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 shutdown_connections 2023-07-14T11:19:38.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f52240a1220 0x7f52240a36d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f523810c0d0 0x7f52381a3460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5238072f70 0x7f523819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 --2- 172.21.15.161:0/1724007195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5238074550 0x7f523819c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 >> 172.21.15.161:0/1724007195 conn(0x7f52380fb0e0 msgr2=0x7f52380fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:38.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 shutdown_connections 2023-07-14T11:19:38.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.117+0000 7f523fdff700 1 -- 172.21.15.161:0/1724007195 wait complete. 2023-07-14T11:19:38.140 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:19:38.140 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:19:38.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:37 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2758669452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:38.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:37 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2758669452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:38.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.353+0000 7f55d75ff700 1 -- 172.21.15.161:0/1704878473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d0100560 msgr2=0x7f55d0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.353+0000 7f55d75ff700 1 --2- 172.21.15.161:0/1704878473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d0100560 0x7f55d0102940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009990 tx=0x7f55cc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.353+0000 7f55d75ff700 1 -- 172.21.15.161:0/1704878473 shutdown_connections 2023-07-14T11:19:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.354+0000 7f55d75ff700 1 --2- 172.21.15.161:0/1704878473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d0100560 0x7f55d0102940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.354+0000 7f55d75ff700 1 --2- 172.21.15.161:0/1704878473 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55d01057a0 0x7f55d0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.354+0000 7f55d75ff700 1 --2- 172.21.15.161:0/1704878473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0102e80 0x7f55d0105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.354+0000 7f55d75ff700 1 -- 172.21.15.161:0/1704878473 >> 172.21.15.161:0/1704878473 conn(0x7f55d00fa000 msgr2=0x7f55d00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.354+0000 7f55d75ff700 1 -- 172.21.15.161:0/1704878473 shutdown_connections 2023-07-14T11:19:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.354+0000 7f55d75ff700 1 -- 172.21.15.161:0/1704878473 wait complete. 2023-07-14T11:19:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.355+0000 7f55d75ff700 1 Processor -- start 2023-07-14T11:19:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.355+0000 7f55d75ff700 1 -- start start 2023-07-14T11:19:38.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d75ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0100560 0x7f55d0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d75ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55d0102e80 0x7f55d0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d75ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d01057a0 0x7f55d019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d75ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019f5f0 con 0x7f55d01057a0 2023-07-14T11:19:38.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d75ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019f730 con 0x7f55d0100560 2023-07-14T11:19:38.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d75ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019f8c0 con 0x7f55d0102e80 2023-07-14T11:19:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0100560 0x7f55d0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0100560 0x7f55d0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49818/0 (socket says 172.21.15.161:49818) 2023-07-14T11:19:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 -- 172.21.15.161:0/2401215412 learned_addr learned my addr 172.21.15.161:0/2401215412 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d5b9c700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d01057a0 0x7f55d019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d4b9a700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55d0102e80 0x7f55d0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 -- 172.21.15.161:0/2401215412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55d0102e80 msgr2=0x7f55d0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55d0102e80 0x7f55d0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 -- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d01057a0 msgr2=0x7f55d019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d01057a0 0x7f55d019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d539b700 1 -- 172.21.15.161:0/2401215412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0095f0 con 0x7f55d0100560 2023-07-14T11:19:38.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d4b9a700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55d0102e80 0x7f55d0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:38.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.356+0000 7f55d5b9c700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d01057a0 0x7f55d019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:38.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.357+0000 7f55d539b700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0100560 0x7f55d0197a70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55cc004740 tx=0x7f55cc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:38.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.357+0000 7f55c67fc700 1 -- 172.21.15.161:0/2401215412 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc01d070 con 0x7f55d0100560 2023-07-14T11:19:38.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.357+0000 7f55c67fc700 1 -- 172.21.15.161:0/2401215412 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55cc004830 con 0x7f55d0100560 2023-07-14T11:19:38.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.357+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d019fb40 con 0x7f55d0100560 2023-07-14T11:19:38.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.357+0000 7f55c67fc700 1 -- 172.21.15.161:0/2401215412 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0053c0 con 0x7f55d0100560 2023-07-14T11:19:38.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.357+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d019ff00 con 0x7f55d0100560 2023-07-14T11:19:38.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.358+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d0107360 con 0x7f55d0100560 2023-07-14T11:19:38.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.362+0000 7f55c67fc700 1 -- 172.21.15.161:0/2401215412 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f55cc005560 con 0x7f55d0100560 2023-07-14T11:19:38.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.363+0000 7f55c67fc700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f55bc0a1150 0x7f55bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.363+0000 7f55c67fc700 1 -- 172.21.15.161:0/2401215412 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f55cc0cd5a0 con 0x7f55d0100560 2023-07-14T11:19:38.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.364+0000 7f55d4b9a700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f55bc0a1150 0x7f55bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.364+0000 7f55c67fc700 1 -- 172.21.15.161:0/2401215412 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55cc097220 con 0x7f55d0100560 2023-07-14T11:19:38.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.364+0000 7f55d4b9a700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f55bc0a1150 0x7f55bc0a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f55c0011490 tx=0x7f55c0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:38.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.582+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55d002c9e0 con 0x7f55d0100560 2023-07-14T11:19:38.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.582+0000 7f55c67fc700 1 -- 172.21.15.161:0/2401215412 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55cc09a840 con 0x7f55d0100560 2023-07-14T11:19:38.585 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:19:38.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f55bc0a1150 msgr2=0x7f55bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f55bc0a1150 0x7f55bc0a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f55c0011490 tx=0x7f55c0009260 comp rx=0 tx=0).stop 2023-07-14T11:19:38.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0100560 msgr2=0x7f55d0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0100560 0x7f55d0197a70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55cc004740 tx=0x7f55cc009580 comp rx=0 tx=0).stop 2023-07-14T11:19:38.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 shutdown_connections 2023-07-14T11:19:38.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55d01057a0 0x7f55d019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f55bc0a1150 0x7f55bc0a3600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d0100560 0x7f55d0197a70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 --2- 172.21.15.161:0/2401215412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55d0102e80 0x7f55d0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 >> 172.21.15.161:0/2401215412 conn(0x7f55d00fa000 msgr2=0x7f55d00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:38.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 shutdown_connections 2023-07-14T11:19:38.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.585+0000 7f55d75ff700 1 -- 172.21.15.161:0/2401215412 wait complete. 2023-07-14T11:19:38.608 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T11:19:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 -- 172.21.15.161:0/2722951043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 msgr2=0x7f9520072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 --2- 172.21.15.161:0/2722951043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 0x7f9520072940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f951c009990 tx=0x7f951c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 -- 172.21.15.161:0/2722951043 shutdown_connections 2023-07-14T11:19:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 --2- 172.21.15.161:0/2722951043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 0x7f9520072940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 --2- 172.21.15.161:0/2722951043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9520107990 0x7f9520109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 --2- 172.21.15.161:0/2722951043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 0x7f9520073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 -- 172.21.15.161:0/2722951043 >> 172.21.15.161:0/2722951043 conn(0x7f95200fb0e0 msgr2=0x7f95200fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 -- 172.21.15.161:0/2722951043 shutdown_connections 2023-07-14T11:19:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.835+0000 7f95275ff700 1 -- 172.21.15.161:0/2722951043 wait complete. 2023-07-14T11:19:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 Processor -- start 2023-07-14T11:19:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 -- start start 2023-07-14T11:19:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 0x7f9520197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 0x7f9520197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9520107990 0x7f952019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952019f5a0 con 0x7f95200744f0 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952019f6e0 con 0x7f9520072f10 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.836+0000 7f95275ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952019f870 con 0x7f9520107990 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 0x7f9520197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 0x7f9520197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49832/0 (socket says 172.21.15.161:49832) 2023-07-14T11:19:38.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 -- 172.21.15.161:0/214861988 learned_addr learned my addr 172.21.15.161:0/214861988 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:38.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f9524b9a700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 0x7f9520197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f9525b9c700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9520107990 0x7f952019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 -- 172.21.15.161:0/214861988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9520107990 msgr2=0x7f952019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9520107990 0x7f952019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 -- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 msgr2=0x7f9520197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 0x7f9520197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 -- 172.21.15.161:0/214861988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0095f0 con 0x7f9520072f10 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f952539b700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 0x7f9520197a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f951c004730 tx=0x7f951c004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f95167fc700 1 -- 172.21.15.161:0/214861988 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c01d070 con 0x7f9520072f10 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f95167fc700 1 -- 172.21.15.161:0/214861988 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951c022470 con 0x7f9520072f10 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f95167fc700 1 -- 172.21.15.161:0/214861988 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c017650 con 0x7f9520072f10 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.837+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952019faf0 con 0x7f9520072f10 2023-07-14T11:19:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.838+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952019fec0 con 0x7f9520072f10 2023-07-14T11:19:38.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.839+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95201060d0 con 0x7f9520072f10 2023-07-14T11:19:38.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.840+0000 7f95167fc700 1 -- 172.21.15.161:0/214861988 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f951c0225e0 con 0x7f9520072f10 2023-07-14T11:19:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.841+0000 7f95167fc700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f950c0a11a0 0x7f950c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.842+0000 7f95167fc700 1 -- 172.21.15.161:0/214861988 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f951c0cd8d0 con 0x7f9520072f10 2023-07-14T11:19:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.842+0000 7f9524b9a700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f950c0a11a0 0x7f950c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.842+0000 7f9524b9a700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f950c0a11a0 0x7f950c0a3650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9510009c60 tx=0x7f9510009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:38.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:38.845+0000 7f95167fc700 1 -- 172.21.15.161:0/214861988 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f951c0974b0 con 0x7f9520072f10 2023-07-14T11:19:38.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:38 smithi161 ceph-mon[95526]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:38.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:38 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1724007195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:38.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:38 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2401215412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:38.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:38 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:38.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:38 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:38.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:38 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:38.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:38 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.067+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f952002cab0 con 0x7f9520072f10 2023-07-14T11:19:39.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.068+0000 7f95167fc700 1 -- 172.21.15.161:0/214861988 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f951c09aad0 con 0x7f9520072f10 2023-07-14T11:19:39.070 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:39.070 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2023-07-14T11:19:39.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:19:39.071 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2023-07-14T11:19:39.071 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2023-07-14T11:19:39.071 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:19:39.071 INFO:teuthology.orchestra.run.smithi161.stdout:] 2023-07-14T11:19:39.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f950c0a11a0 msgr2=0x7f950c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:39.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f950c0a11a0 0x7f950c0a3650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9510009c60 tx=0x7f9510009260 comp rx=0 tx=0).stop 2023-07-14T11:19:39.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 msgr2=0x7f9520197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:39.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 0x7f9520197a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f951c004730 tx=0x7f951c004760 comp rx=0 tx=0).stop 2023-07-14T11:19:39.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 shutdown_connections 2023-07-14T11:19:39.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f950c0a11a0 0x7f950c0a3650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9520107990 0x7f952019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9520072f10 0x7f9520197a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 --2- 172.21.15.161:0/214861988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95200744f0 0x7f9520197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 >> 172.21.15.161:0/214861988 conn(0x7f95200fb0e0 msgr2=0x7f95200fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.070+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 shutdown_connections 2023-07-14T11:19:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.071+0000 7f95275ff700 1 -- 172.21.15.161:0/214861988 wait complete. 2023-07-14T11:19:39.092 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T11:19:39.092 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T11:19:39.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:38 smithi175 ceph-mon[129081]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:39.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:38 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1724007195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:39.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:38 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2401215412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:39.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:38 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:38 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:38 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:38 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:38 smithi182 ceph-mon[129366]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:39.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:38 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1724007195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:39.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:38 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2401215412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:39.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:38 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:38 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:38 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:38 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:39.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3028298154 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 msgr2=0x7ff2d810e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:39.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3028298154 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 0x7ff2d810e4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009f90 tx=0x7ff2d000b330 comp rx=0 tx=0).stop 2023-07-14T11:19:39.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3028298154 shutdown_connections 2023-07-14T11:19:39.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3028298154 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 0x7ff2d810e4a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3028298154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2d8074550 0x7ff2d80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3028298154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 0x7ff2d80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3028298154 >> 172.21.15.161:0/3028298154 conn(0x7ff2d80fb130 msgr2=0x7ff2d80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:39.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3028298154 shutdown_connections 2023-07-14T11:19:39.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.311+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3028298154 wait complete. 2023-07-14T11:19:39.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.312+0000 7ff2dd9ff700 1 Processor -- start 2023-07-14T11:19:39.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.312+0000 7ff2dd9ff700 1 -- start start 2023-07-14T11:19:39.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2dd9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 0x7ff2d819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:39.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2dd9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2d8074550 0x7ff2d819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:39.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2dd9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 0x7ff2d81a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:39.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2dd9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d81a3ac0 con 0x7ff2d8072f70 2023-07-14T11:19:39.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2dd9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d81a3c00 con 0x7ff2d8074550 2023-07-14T11:19:39.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2dd9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d81a3d90 con 0x7ff2d810c0c0 2023-07-14T11:19:39.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 0x7ff2d819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:39.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2cffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2d8074550 0x7ff2d819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:39.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 0x7ff2d819bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48686/0 (socket says 172.21.15.161:48686) 2023-07-14T11:19:39.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 -- 172.21.15.161:0/3720080689 learned_addr learned my addr 172.21.15.161:0/3720080689 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:39.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d77fe700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 0x7ff2d81a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:39.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 -- 172.21.15.161:0/3720080689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 msgr2=0x7ff2d81a34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:39.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 0x7ff2d81a34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 -- 172.21.15.161:0/3720080689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2d8074550 msgr2=0x7ff2d819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2d8074550 0x7ff2d819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2d6ffd700 1 -- 172.21.15.161:0/3720080689 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d0009c40 con 0x7ff2d8072f70 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.313+0000 7ff2cffff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2d8074550 0x7ff2d819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.314+0000 7ff2d6ffd700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 0x7ff2d819bf40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800e970 tx=0x7ff2c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.314+0000 7ff2dc9fd700 1 -- 172.21.15.161:0/3720080689 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8009790 con 0x7ff2d8072f70 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.314+0000 7ff2dc9fd700 1 -- 172.21.15.161:0/3720080689 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c800c7c0 con 0x7ff2d8072f70 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.314+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d81a4070 con 0x7ff2d8072f70 2023-07-14T11:19:39.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.314+0000 7ff2dc9fd700 1 -- 172.21.15.161:0/3720080689 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80054b0 con 0x7ff2d8072f70 2023-07-14T11:19:39.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.314+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d81a45d0 con 0x7ff2d8072f70 2023-07-14T11:19:39.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.315+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d8101c10 con 0x7ff2d8072f70 2023-07-14T11:19:39.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.317+0000 7ff2dc9fd700 1 -- 172.21.15.161:0/3720080689 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff2c800c930 con 0x7ff2d8072f70 2023-07-14T11:19:39.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.319+0000 7ff2dc9fd700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2c40a11d0 0x7ff2c40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:39.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.319+0000 7ff2dc9fd700 1 -- 172.21.15.161:0/3720080689 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff2c8014070 con 0x7ff2d8072f70 2023-07-14T11:19:39.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.319+0000 7ff2cffff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2c40a11d0 0x7ff2c40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:39.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.319+0000 7ff2cffff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2c40a11d0 0x7ff2c40a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8074270 tx=0x7ff2c0009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.322+0000 7ff2dc9fd700 1 -- 172.21.15.161:0/3720080689 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff2c8096160 con 0x7ff2d8072f70 2023-07-14T11:19:39.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:39.561+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff2d802c850 con 0x7ff2d8072f70 2023-07-14T11:19:40.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.089+0000 7ff2dc9fd700 1 -- 172.21.15.161:0/3720080689 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff2c8099780 con 0x7ff2d8072f70 2023-07-14T11:19:40.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.094+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2c40a11d0 msgr2=0x7ff2c40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:40.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.094+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2c40a11d0 0x7ff2c40a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8074270 tx=0x7ff2c0009310 comp rx=0 tx=0).stop 2023-07-14T11:19:40.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.094+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 msgr2=0x7ff2d819bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:40.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.094+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 0x7ff2d819bf40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800e970 tx=0x7ff2c800ec80 comp rx=0 tx=0).stop 2023-07-14T11:19:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.095+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 shutdown_connections 2023-07-14T11:19:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.095+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2d8072f70 0x7ff2d819bf40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.095+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2c40a11d0 0x7ff2c40a3680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.095+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2d810c0c0 0x7ff2d81a34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.095+0000 7ff2dd9ff700 1 --2- 172.21.15.161:0/3720080689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2d8074550 0x7ff2d819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.095+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 >> 172.21.15.161:0/3720080689 conn(0x7ff2d80fb130 msgr2=0x7ff2d810e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.096+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 shutdown_connections 2023-07-14T11:19:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.096+0000 7ff2dd9ff700 1 -- 172.21.15.161:0/3720080689 wait complete. 2023-07-14T11:19:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2023-07-14T11:19:40.118 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-14T11:19:40.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 -- 172.21.15.161:0/605751045 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75300ffaf0 msgr2=0x7f75300fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:40.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 --2- 172.21.15.161:0/605751045 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75300ffaf0 0x7f75300fff00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f752c009990 tx=0x7f752c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 -- 172.21.15.161:0/605751045 shutdown_connections 2023-07-14T11:19:40.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 --2- 172.21.15.161:0/605751045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7530101670 0x7f7530107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 --2- 172.21.15.161:0/605751045 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75300ffaf0 0x7f75300fff00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 --2- 172.21.15.161:0/605751045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 0x7f7530101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 -- 172.21.15.161:0/605751045 >> 172.21.15.161:0/605751045 conn(0x7f75300fb130 msgr2=0x7f75300fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:40.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.327+0000 7f75381ff700 1 -- 172.21.15.161:0/605751045 shutdown_connections 2023-07-14T11:19:40.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.328+0000 7f75381ff700 1 -- 172.21.15.161:0/605751045 wait complete. 2023-07-14T11:19:40.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.329+0000 7f75381ff700 1 Processor -- start 2023-07-14T11:19:40.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.329+0000 7f75381ff700 1 -- start start 2023-07-14T11:19:40.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f75381ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75300ffaf0 0x7f75300742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f75381ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 0x7f7530072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f75381ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7530101670 0x7f7530072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f75381ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75300735e0 con 0x7f75300ffaf0 2023-07-14T11:19:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f75381ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530073750 con 0x7f7530100ce0 2023-07-14T11:19:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f75381ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753019f880 con 0x7f7530101670 2023-07-14T11:19:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 0x7f7530072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f7535f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75300ffaf0 0x7f75300742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 0x7f7530072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49880/0 (socket says 172.21.15.161:49880) 2023-07-14T11:19:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 -- 172.21.15.161:0/741833795 learned_addr learned my addr 172.21.15.161:0/741833795 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753679c700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7530101670 0x7f7530072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:40.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 -- 172.21.15.161:0/741833795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7530101670 msgr2=0x7f7530072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:40.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7530101670 0x7f7530072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 -- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75300ffaf0 msgr2=0x7f75300742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:40.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75300ffaf0 0x7f75300742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:40.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f753579a700 1 -- 172.21.15.161:0/741833795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7520009520 con 0x7f7530100ce0 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.330+0000 7f7535f9b700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75300ffaf0 0x7f75300742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.331+0000 7f753579a700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 0x7f7530072900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f752000eb30 tx=0x7f752000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.331+0000 7f7526ffd700 1 -- 172.21.15.161:0/741833795 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520009880 con 0x7f7530100ce0 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.331+0000 7f7526ffd700 1 -- 172.21.15.161:0/741833795 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75200044e0 con 0x7f7530100ce0 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.331+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7528009c40 con 0x7f7530100ce0 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.331+0000 7f7526ffd700 1 -- 172.21.15.161:0/741833795 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520003e90 con 0x7f7530100ce0 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.331+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c0095f0 con 0x7f7530100ce0 2023-07-14T11:19:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.332+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75301070b0 con 0x7f7530100ce0 2023-07-14T11:19:40.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.335+0000 7f7526ffd700 1 -- 172.21.15.161:0/741833795 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7520004000 con 0x7f7530100ce0 2023-07-14T11:19:40.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.337+0000 7f7526ffd700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f751c0a1150 0x7f751c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:40.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.337+0000 7f7526ffd700 1 -- 172.21.15.161:0/741833795 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f7520014070 con 0x7f7530100ce0 2023-07-14T11:19:40.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.337+0000 7f7535f9b700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f751c0a1150 0x7f751c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:40.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.338+0000 7f7535f9b700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f751c0a1150 0x7f751c0a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f752c00b2e0 tx=0x7f752c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:40.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.340+0000 7f7526ffd700 1 -- 172.21.15.161:0/741833795 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f75200964d0 con 0x7f7530100ce0 2023-07-14T11:19:40.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:40 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/214861988' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T11:19:40.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:40 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:40 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/214861988' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T11:19:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:40 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:40.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:40 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/214861988' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T11:19:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:40 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:40.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:40.551+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f753002c850 con 0x7f7530100ce0 2023-07-14T11:19:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.099+0000 7f7526ffd700 1 -- 172.21.15.161:0/741833795 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f7520099af0 con 0x7f7530100ce0 2023-07-14T11:19:41.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f751c0a1150 msgr2=0x7f751c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f751c0a1150 0x7f751c0a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f752c00b2e0 tx=0x7f752c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 msgr2=0x7f7530072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 0x7f7530072900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f752000eb30 tx=0x7f752000ee40 comp rx=0 tx=0).stop 2023-07-14T11:19:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 shutdown_connections 2023-07-14T11:19:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75300ffaf0 0x7f75300742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f751c0a1150 0x7f751c0a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7530101670 0x7f7530072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 --2- 172.21.15.161:0/741833795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7530100ce0 0x7f7530072900 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 >> 172.21.15.161:0/741833795 conn(0x7f75300fb130 msgr2=0x7f75301041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:41.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 shutdown_connections 2023-07-14T11:19:41.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.114+0000 7f75381ff700 1 -- 172.21.15.161:0/741833795 wait complete. 2023-07-14T11:19:41.119 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2023-07-14T11:19:41.136 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T11:19:41.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/785331659 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48100cd0 msgr2=0x7fdc48101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/785331659 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48100cd0 0x7fdc48101120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009990 tx=0x7fdc3c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/785331659 shutdown_connections 2023-07-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/785331659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc48101660 0x7fdc48107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/785331659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc480ffae0 0x7fdc480ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/785331659 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48100cd0 0x7fdc48101120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/785331659 >> 172.21.15.161:0/785331659 conn(0x7fdc480fb0e0 msgr2=0x7fdc480fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/785331659 shutdown_connections 2023-07-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.346+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/785331659 wait complete. 2023-07-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.347+0000 7fdc4e9ff700 1 Processor -- start 2023-07-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4e9ff700 1 -- start start 2023-07-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4e9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc480ffae0 0x7fdc48195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4e9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc48100cd0 0x7fdc48195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4e9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48101660 0x7fdc4819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4e9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4819d380 con 0x7fdc48100cd0 2023-07-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4e9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4819d4c0 con 0x7fdc480ffae0 2023-07-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4e9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4819d650 con 0x7fdc48101660 2023-07-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc477fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc48100cd0 0x7fdc48195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc4cf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48101660 0x7fdc4819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc47fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc480ffae0 0x7fdc48195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc477fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc48100cd0 0x7fdc48195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48720/0 (socket says 172.21.15.161:48720) 2023-07-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.348+0000 7fdc477fe700 1 -- 172.21.15.161:0/3163350024 learned_addr learned my addr 172.21.15.161:0/3163350024 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4cf9c700 1 -- 172.21.15.161:0/3163350024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc480ffae0 msgr2=0x7fdc48195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4cf9c700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc480ffae0 0x7fdc48195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4cf9c700 1 -- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc48100cd0 msgr2=0x7fdc48195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4cf9c700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc48100cd0 0x7fdc48195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4cf9c700 1 -- 172.21.15.161:0/3163350024 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0095f0 con 0x7fdc48101660 2023-07-14T11:19:41.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4cf9c700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48101660 0x7fdc4819cdb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000efc0 tx=0x7fdc4000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:41.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc457fa700 1 -- 172.21.15.161:0/3163350024 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40018070 con 0x7fdc48101660 2023-07-14T11:19:41.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc457fa700 1 -- 172.21.15.161:0/3163350024 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4000f040 con 0x7fdc48101660 2023-07-14T11:19:41.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc48072740 con 0x7fdc48101660 2023-07-14T11:19:41.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc457fa700 1 -- 172.21.15.161:0/3163350024 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000cd40 con 0x7fdc48101660 2023-07-14T11:19:41.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.349+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc48072c50 con 0x7fdc48101660 2023-07-14T11:19:41.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.351+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4810b2a0 con 0x7fdc48101660 2023-07-14T11:19:41.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.352+0000 7fdc457fa700 1 -- 172.21.15.161:0/3163350024 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdc40003a00 con 0x7fdc48101660 2023-07-14T11:19:41.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.355+0000 7fdc457fa700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc300a1150 0x7fdc300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:41.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.355+0000 7fdc457fa700 1 -- 172.21.15.161:0/3163350024 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fdc400cc3c0 con 0x7fdc48101660 2023-07-14T11:19:41.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.355+0000 7fdc47fff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc300a1150 0x7fdc300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:41.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.355+0000 7fdc47fff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc300a1150 0x7fdc300a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc48100b30 tx=0x7fdc38009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:41.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.357+0000 7fdc457fa700 1 -- 172.21.15.161:0/3163350024 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdc40095cb0 con 0x7fdc48101660 2023-07-14T11:19:41.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3720080689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:19:41.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:41.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:41.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3720080689' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T11:19:41.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T11:19:41.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:41 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:41.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3720080689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:19:41.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:41.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:41.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3720080689' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T11:19:41.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T11:19:41.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:41 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:41.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3720080689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:19:41.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:41.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:41.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:41.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3720080689' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T11:19:41.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T11:19:41.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:41 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:19:41.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:41.648+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdc480733b0 con 0x7fdc48101660 2023-07-14T11:19:42.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.097+0000 7fdc457fa700 1 -- 172.21.15.161:0/3163350024 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdc400992d0 con 0x7fdc48101660 2023-07-14T11:19:42.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.100+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc300a1150 msgr2=0x7fdc300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.100+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc300a1150 0x7fdc300a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc48100b30 tx=0x7fdc38009260 comp rx=0 tx=0).stop 2023-07-14T11:19:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.100+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48101660 msgr2=0x7fdc4819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.100+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48101660 0x7fdc4819cdb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000efc0 tx=0x7fdc4000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.101+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 shutdown_connections 2023-07-14T11:19:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.101+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fdc300a1150 0x7fdc300a3600 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc48100b30 tx=0x7fdc38009260 comp rx=0 tx=0).stop 2023-07-14T11:19:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.101+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdc48101660 0x7fdc4819cdb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.101+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc480ffae0 0x7fdc48195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.101+0000 7fdc4e9ff700 1 --2- 172.21.15.161:0/3163350024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc48100cd0 0x7fdc48195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.101+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 >> 172.21.15.161:0/3163350024 conn(0x7fdc480fb0e0 msgr2=0x7fdc480fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.102+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 shutdown_connections 2023-07-14T11:19:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.102+0000 7fdc4e9ff700 1 -- 172.21.15.161:0/3163350024 wait complete. 2023-07-14T11:19:42.107 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 2 and data pool 3 2023-07-14T11:19:42.126 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-14T11:19:42.127 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-14T11:19:42.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 -- 172.21.15.161:0/3800559144 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4074550 msgr2=0x7f97b40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:42.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/3800559144 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4074550 0x7f97b40729a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009990 tx=0x7f97b0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 -- 172.21.15.161:0/3800559144 shutdown_connections 2023-07-14T11:19:42.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/3800559144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b41079b0 0x7f97b4109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/3800559144 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4074550 0x7f97b40729a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/3800559144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b4072f70 0x7f97b40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 -- 172.21.15.161:0/3800559144 >> 172.21.15.161:0/3800559144 conn(0x7f97b40fb130 msgr2=0x7f97b40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:42.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 -- 172.21.15.161:0/3800559144 shutdown_connections 2023-07-14T11:19:42.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.339+0000 7f97bc3ff700 1 -- 172.21.15.161:0/3800559144 wait complete. 2023-07-14T11:19:42.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.340+0000 7f97bc3ff700 1 Processor -- start 2023-07-14T11:19:42.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.340+0000 7f97bc3ff700 1 -- start start 2023-07-14T11:19:42.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97bc3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4072f70 0x7f97b4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:42.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97bc3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b4074550 0x7f97b4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:42.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97bc3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b41079b0 0x7f97b419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:42.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97bc3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b419f620 con 0x7f97b4074550 2023-07-14T11:19:42.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97bc3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b419f760 con 0x7f97b41079b0 2023-07-14T11:19:42.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97bc3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b419f8f0 con 0x7f97b4072f70 2023-07-14T11:19:42.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97b999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b4074550 0x7f97b4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:42.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97b999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b4074550 0x7f97b4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48742/0 (socket says 172.21.15.161:48742) 2023-07-14T11:19:42.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97b999a700 1 -- 172.21.15.161:0/1027693573 learned_addr learned my addr 172.21.15.161:0/1027693573 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:42.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba19b700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4072f70 0x7f97b4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba99c700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b41079b0 0x7f97b419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba19b700 1 -- 172.21.15.161:0/1027693573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b41079b0 msgr2=0x7f97b419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba19b700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b41079b0 0x7f97b419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba19b700 1 -- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b4074550 msgr2=0x7f97b4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba19b700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b4074550 0x7f97b4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba19b700 1 -- 172.21.15.161:0/1027693573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00095f0 con 0x7f97b4072f70 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97b999a700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b4074550 0x7f97b4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:42.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.341+0000 7f97ba99c700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b41079b0 0x7f97b419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:42.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.342+0000 7f97ba19b700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4072f70 0x7f97b4197aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f97b00098e0 tx=0x7f97b0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:42.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.342+0000 7f97ab7fe700 1 -- 172.21.15.161:0/1027693573 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b001d070 con 0x7f97b4072f70 2023-07-14T11:19:42.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.342+0000 7f97ab7fe700 1 -- 172.21.15.161:0/1027693573 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b0021410 con 0x7f97b4072f70 2023-07-14T11:19:42.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.342+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b419fb70 con 0x7f97b4072f70 2023-07-14T11:19:42.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.342+0000 7f97ab7fe700 1 -- 172.21.15.161:0/1027693573 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0005720 con 0x7f97b4072f70 2023-07-14T11:19:42.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.342+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b419fff0 con 0x7f97b4072f70 2023-07-14T11:19:42.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.343+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4106060 con 0x7f97b4072f70 2023-07-14T11:19:42.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.347+0000 7f97ab7fe700 1 -- 172.21.15.161:0/1027693573 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f97b0021580 con 0x7f97b4072f70 2023-07-14T11:19:42.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.348+0000 7f97ab7fe700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97a00a11d0 0x7f97a00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:42.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.348+0000 7f97ab7fe700 1 -- 172.21.15.161:0/1027693573 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97b00cd950 con 0x7f97b4072f70 2023-07-14T11:19:42.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.348+0000 7f97b999a700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97a00a11d0 0x7f97a00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:42.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.349+0000 7f97b999a700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97a00a11d0 0x7f97a00a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005950 tx=0x7f97a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:42.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.352+0000 7f97ab7fe700 1 -- 172.21.15.161:0/1027693573 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f97b0097280 con 0x7f97b4072f70 2023-07-14T11:19:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/741833795' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:19:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:19:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T11:19:42.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T11:19:42.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3163350024' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:19:42.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:19:42.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:42 smithi175 ceph-mon[129081]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T11:19:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/741833795' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:19:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:19:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T11:19:42.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T11:19:42.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3163350024' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:19:42.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:19:42.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:42 smithi182 ceph-mon[129366]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T11:19:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/741833795' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:19:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:19:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T11:19:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T11:19:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3163350024' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:19:42.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:19:42.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:42 smithi161 ceph-mon[95526]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T11:19:42.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:42.572+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f97b402cab0 con 0x7f97b4072f70 2023-07-14T11:19:43.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.072+0000 7f97ab7fe700 1 -- 172.21.15.161:0/1027693573 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f97b009a8a0 con 0x7f97b4072f70 2023-07-14T11:19:43.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97a00a11d0 msgr2=0x7f97a00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:43.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97a00a11d0 0x7f97a00a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005950 tx=0x7f97a4009260 comp rx=0 tx=0).stop 2023-07-14T11:19:43.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4072f70 msgr2=0x7f97b4197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:43.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4072f70 0x7f97b4197aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f97b00098e0 tx=0x7f97b0009580 comp rx=0 tx=0).stop 2023-07-14T11:19:43.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 shutdown_connections 2023-07-14T11:19:43.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97a00a11d0 0x7f97a00a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b41079b0 0x7f97b419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97b4072f70 0x7f97b4197aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 --2- 172.21.15.161:0/1027693573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b4074550 0x7f97b4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 >> 172.21.15.161:0/1027693573 conn(0x7f97b40fb130 msgr2=0x7f97b40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:43.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 shutdown_connections 2023-07-14T11:19:43.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.074+0000 7f97bc3ff700 1 -- 172.21.15.161:0/1027693573 wait complete. 2023-07-14T11:19:43.096 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T11:19:43.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.308+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/696624041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 msgr2=0x7f4b18101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:43.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.308+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/696624041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 0x7f4b18101130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009990 tx=0x7f4b0c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.309+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/696624041 shutdown_connections 2023-07-14T11:19:43.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.309+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/696624041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b18101670 0x7f4b18107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.309+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/696624041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 0x7f4b180fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.309+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/696624041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 0x7f4b18101130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.309+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/696624041 >> 172.21.15.161:0/696624041 conn(0x7f4b180fb0d0 msgr2=0x7f4b180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:43.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.309+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/696624041 shutdown_connections 2023-07-14T11:19:43.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.309+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/696624041 wait complete. 2023-07-14T11:19:43.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 Processor -- start 2023-07-14T11:19:43.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 -- start start 2023-07-14T11:19:43.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 0x7f4b181979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:43.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 0x7f4b18197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b18101670 0x7f4b1819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1819f560 con 0x7f4b18101670 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1819f6a0 con 0x7f4b18100ce0 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b1d1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1819f830 con 0x7f4b180ffaf0 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 0x7f4b181979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 0x7f4b181979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45934/0 (socket says 172.21.15.161:45934) 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.310+0000 7f4b16d9d700 1 -- 172.21.15.161:0/891160842 learned_addr learned my addr 172.21.15.161:0/891160842 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:43.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b1659c700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 0x7f4b18197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:43.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b1759e700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b18101670 0x7f4b1819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:43.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b16d9d700 1 -- 172.21.15.161:0/891160842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 msgr2=0x7f4b18197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:43.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b16d9d700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 0x7f4b18197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b16d9d700 1 -- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b18101670 msgr2=0x7f4b1819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:43.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b16d9d700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b18101670 0x7f4b1819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:43.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b16d9d700 1 -- 172.21.15.161:0/891160842 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0095f0 con 0x7f4b180ffaf0 2023-07-14T11:19:43.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b1759e700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b18101670 0x7f4b1819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:43.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b16d9d700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 0x7f4b181979e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800d980 tx=0x7f4b0800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:43.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4afffff700 1 -- 172.21.15.161:0/891160842 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800b950 con 0x7f4b180ffaf0 2023-07-14T11:19:43.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1819fb10 con 0x7f4b180ffaf0 2023-07-14T11:19:43.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4afffff700 1 -- 172.21.15.161:0/891160842 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b08007620 con 0x7f4b180ffaf0 2023-07-14T11:19:43.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4afffff700 1 -- 172.21.15.161:0/891160842 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800f3c0 con 0x7f4b180ffaf0 2023-07-14T11:19:43.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.311+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b181a0070 con 0x7f4b180ffaf0 2023-07-14T11:19:43.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.312+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1810b310 con 0x7f4b180ffaf0 2023-07-14T11:19:43.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.314+0000 7f4afffff700 1 -- 172.21.15.161:0/891160842 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4b08010640 con 0x7f4b180ffaf0 2023-07-14T11:19:43.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.316+0000 7f4afffff700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4b000a11f0 0x7f4b000a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:43.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.316+0000 7f4afffff700 1 -- 172.21.15.161:0/891160842 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f4b080cc830 con 0x7f4b180ffaf0 2023-07-14T11:19:43.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.316+0000 7f4b1659c700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4b000a11f0 0x7f4b000a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:43.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.317+0000 7f4b1659c700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4b000a11f0 0x7f4b000a36a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b5c0 tx=0x7f4b0c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:43.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.319+0000 7f4afffff700 1 -- 172.21.15.161:0/891160842 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4b08095f40 con 0x7f4b180ffaf0 2023-07-14T11:19:43.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T11:19:43.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: pgmap v267: 97 pgs: 2 creating+activating, 21 creating+peering, 7 active+clean, 67 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:43.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1027693573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] up:active 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:43 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T11:19:43.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: pgmap v267: 97 pgs: 2 creating+activating, 21 creating+peering, 7 active+clean, 67 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:43.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T11:19:43.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T11:19:43.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T11:19:43.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1027693573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:19:43.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:19:43.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] up:active 2023-07-14T11:19:43.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:19:43.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:43 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T11:19:43.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: pgmap v267: 97 pgs: 2 creating+activating, 21 creating+peering, 7 active+clean, 67 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:19:43.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T11:19:43.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T11:19:43.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T11:19:43.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1027693573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:19:43.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:19:43.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] up:active 2023-07-14T11:19:43.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:19:43.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:43 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:43.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:43.533+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4b181a06d0 con 0x7f4b180ffaf0 2023-07-14T11:19:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T11:19:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T11:19:44.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/891160842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:19:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:19:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:44.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T11:19:44.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T11:19:44.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/891160842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:19:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:19:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:44.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T11:19:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T11:19:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/891160842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:19:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:19:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:44.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:19:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:19:44.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:19:45.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.076+0000 7f4afffff700 1 -- 172.21.15.161:0/891160842 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f4b08099560 con 0x7f4b180ffaf0 2023-07-14T11:19:45.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.082+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4b000a11f0 msgr2=0x7f4b000a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.082+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4b000a11f0 0x7f4b000a36a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b5c0 tx=0x7f4b0c005fd0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.082+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 msgr2=0x7f4b181979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.082+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 0x7f4b181979e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800d980 tx=0x7f4b0800dc90 comp rx=0 tx=0).stop 2023-07-14T11:19:45.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.084+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 shutdown_connections 2023-07-14T11:19:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.084+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b18101670 0x7f4b1819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.084+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4b000a11f0 0x7f4b000a36a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.084+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b180ffaf0 0x7f4b181979e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.084+0000 7f4b1d1ff700 1 --2- 172.21.15.161:0/891160842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18100ce0 0x7f4b18197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.084+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 >> 172.21.15.161:0/891160842 conn(0x7f4b180fb0d0 msgr2=0x7f4b18106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:45.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.085+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 shutdown_connections 2023-07-14T11:19:45.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.085+0000 7f4b1d1ff700 1 -- 172.21.15.161:0/891160842 wait complete. 2023-07-14T11:19:45.089 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data_ec' created 2023-07-14T11:19:45.107 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-14T11:19:45.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 -- 172.21.15.161:0/2710500530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 msgr2=0x7fd924107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:45.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 --2- 172.21.15.161:0/2710500530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 0x7fd924107f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b0a0 tx=0x7fd91c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 -- 172.21.15.161:0/2710500530 shutdown_connections 2023-07-14T11:19:45.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 --2- 172.21.15.161:0/2710500530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 0x7fd924107f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 --2- 172.21.15.161:0/2710500530 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd9240ffaf0 0x7fd9240fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 --2- 172.21.15.161:0/2710500530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd924100ce0 0x7fd924101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 -- 172.21.15.161:0/2710500530 >> 172.21.15.161:0/2710500530 conn(0x7fd9240fb130 msgr2=0x7fd9240fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:45.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 -- 172.21.15.161:0/2710500530 shutdown_connections 2023-07-14T11:19:45.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.306+0000 7fd92bfff700 1 -- 172.21.15.161:0/2710500530 wait complete. 2023-07-14T11:19:45.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.307+0000 7fd92bfff700 1 Processor -- start 2023-07-14T11:19:45.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.307+0000 7fd92bfff700 1 -- start start 2023-07-14T11:19:45.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9240ffaf0 0x7fd924197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:45.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd924100ce0 0x7fd9241980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:45.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 0x7fd92419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:45.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92bfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419f700 con 0x7fd924101670 2023-07-14T11:19:45.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419f840 con 0x7fd9240ffaf0 2023-07-14T11:19:45.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92bfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419f9d0 con 0x7fd924100ce0 2023-07-14T11:19:45.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 0x7fd92419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:45.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd929d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9240ffaf0 0x7fd924197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:45.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd929d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9240ffaf0 0x7fd924197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49936/0 (socket says 172.21.15.161:49936) 2023-07-14T11:19:45.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd929d9b700 1 -- 172.21.15.161:0/4179815996 learned_addr learned my addr 172.21.15.161:0/4179815996 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:45.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92959a700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd924100ce0 0x7fd9241980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:45.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92a59c700 1 -- 172.21.15.161:0/4179815996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd924100ce0 msgr2=0x7fd9241980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:45.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92a59c700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd924100ce0 0x7fd9241980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92a59c700 1 -- 172.21.15.161:0/4179815996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9240ffaf0 msgr2=0x7fd924197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:45.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92a59c700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9240ffaf0 0x7fd924197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:45.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.308+0000 7fd92a59c700 1 -- 172.21.15.161:0/4179815996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c009d00 con 0x7fd924101670 2023-07-14T11:19:45.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.309+0000 7fd92a59c700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 0x7fd92419f130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b070 tx=0x7fd91c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:45.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.309+0000 7fd91affd700 1 -- 172.21.15.161:0/4179815996 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00e040 con 0x7fd924101670 2023-07-14T11:19:45.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.309+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92419fc50 con 0x7fd924101670 2023-07-14T11:19:45.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.309+0000 7fd91affd700 1 -- 172.21.15.161:0/4179815996 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91c003e10 con 0x7fd924101670 2023-07-14T11:19:45.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.309+0000 7fd91affd700 1 -- 172.21.15.161:0/4179815996 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0052a0 con 0x7fd924101670 2023-07-14T11:19:45.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.309+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9241a0020 con 0x7fd924101670 2023-07-14T11:19:45.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.310+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92410b320 con 0x7fd924101670 2023-07-14T11:19:45.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.312+0000 7fd91affd700 1 -- 172.21.15.161:0/4179815996 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd91c01d480 con 0x7fd924101670 2023-07-14T11:19:45.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.313+0000 7fd91affd700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9100a1150 0x7fd9100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:45.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.313+0000 7fd91affd700 1 -- 172.21.15.161:0/4179815996 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fd91c0cded0 con 0x7fd924101670 2023-07-14T11:19:45.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.314+0000 7fd929d9b700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9100a1150 0x7fd9100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:45.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.314+0000 7fd929d9b700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9100a1150 0x7fd9100a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd920005950 tx=0x7fd920009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:45.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.316+0000 7fd91affd700 1 -- 172.21.15.161:0/4179815996 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd91c097510 con 0x7fd924101670 2023-07-14T11:19:45.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:45.517+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd92410b510 con 0x7fd924101670 2023-07-14T11:19:45.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:45 smithi175 ceph-mon[129081]: pgmap v270: 97 pgs: 2 creating+activating, 21 creating+peering, 37 active+clean, 37 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-07-14T11:19:45.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:45 smithi175 ceph-mon[129081]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T11:19:45.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:45 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T11:19:45.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:45 smithi175 ceph-mon[129081]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T11:19:45.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:45 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4179815996' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:19:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:45 smithi182 ceph-mon[129366]: pgmap v270: 97 pgs: 2 creating+activating, 21 creating+peering, 37 active+clean, 37 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-07-14T11:19:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:45 smithi182 ceph-mon[129366]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T11:19:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:45 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T11:19:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:45 smithi182 ceph-mon[129366]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T11:19:45.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:45 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4179815996' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:19:45.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:45 smithi161 ceph-mon[95526]: pgmap v270: 97 pgs: 2 creating+activating, 21 creating+peering, 37 active+clean, 37 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-07-14T11:19:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:45 smithi161 ceph-mon[95526]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T11:19:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:45 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T11:19:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:45 smithi161 ceph-mon[95526]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T11:19:45.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:45 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4179815996' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:19:46.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.079+0000 7fd91affd700 1 -- 172.21.15.161:0/4179815996 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd91c09ab30 con 0x7fd924101670 2023-07-14T11:19:46.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.084+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9100a1150 msgr2=0x7fd9100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:46.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.084+0000 7fd92bfff700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9100a1150 0x7fd9100a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd920005950 tx=0x7fd920009260 comp rx=0 tx=0).stop 2023-07-14T11:19:46.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.084+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 msgr2=0x7fd92419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:46.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.084+0000 7fd92bfff700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 0x7fd92419f130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b070 tx=0x7fd91c00bc80 comp rx=0 tx=0).stop 2023-07-14T11:19:46.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.087+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 shutdown_connections 2023-07-14T11:19:46.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.087+0000 7fd92bfff700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd924101670 0x7fd92419f130 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.087+0000 7fd92bfff700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd9100a1150 0x7fd9100a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.087+0000 7fd92bfff700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9240ffaf0 0x7fd924197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.087+0000 7fd92bfff700 1 --2- 172.21.15.161:0/4179815996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd924100ce0 0x7fd9241980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.087+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 >> 172.21.15.161:0/4179815996 conn(0x7fd9240fb130 msgr2=0x7fd924106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:46.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.095+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 shutdown_connections 2023-07-14T11:19:46.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.099+0000 7fd92bfff700 1 -- 172.21.15.161:0/4179815996 wait complete. 2023-07-14T11:19:46.105 INFO:teuthology.orchestra.run.smithi161.stderr:set pool 4 allow_ec_overwrites to true 2023-07-14T11:19:46.123 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-14T11:19:46.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 -- 172.21.15.161:0/3070057585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4072f70 msgr2=0x7f2fa40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/3070057585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4072f70 0x7f2fa40733e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009990 tx=0x7f2f94009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 -- 172.21.15.161:0/3070057585 shutdown_connections 2023-07-14T11:19:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/3070057585 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2fa41079b0 0x7f2fa4109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/3070057585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fa4074550 0x7f2fa40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/3070057585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4072f70 0x7f2fa40733e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 -- 172.21.15.161:0/3070057585 >> 172.21.15.161:0/3070057585 conn(0x7f2fa40fb130 msgr2=0x7f2fa40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:46.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 -- 172.21.15.161:0/3070057585 shutdown_connections 2023-07-14T11:19:46.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.324+0000 7f2fab9ff700 1 -- 172.21.15.161:0/3070057585 wait complete. 2023-07-14T11:19:46.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.325+0000 7f2fab9ff700 1 Processor -- start 2023-07-14T11:19:46.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.325+0000 7f2fab9ff700 1 -- start start 2023-07-14T11:19:46.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fab9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2fa4072f70 0x7f2fa4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:46.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fab9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4074550 0x7f2fa4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:46.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fab9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fa41079b0 0x7f2fa419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:46.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fab9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa419f660 con 0x7f2fa4074550 2023-07-14T11:19:46.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fab9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa419f7a0 con 0x7f2fa41079b0 2023-07-14T11:19:46.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fab9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa419f930 con 0x7f2fa4072f70 2023-07-14T11:19:46.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4074550 0x7f2fa4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa979b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2fa4072f70 0x7f2fa4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4074550 0x7f2fa4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48776/0 (socket says 172.21.15.161:48776) 2023-07-14T11:19:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 -- 172.21.15.161:0/1345158055 learned_addr learned my addr 172.21.15.161:0/1345158055 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa9f9c700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fa41079b0 0x7f2fa419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 -- 172.21.15.161:0/1345158055 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2fa4072f70 msgr2=0x7f2fa4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2fa4072f70 0x7f2fa4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 -- 172.21.15.161:0/1345158055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fa41079b0 msgr2=0x7f2fa419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fa41079b0 0x7f2fa419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa8f9a700 1 -- 172.21.15.161:0/1345158055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940095f0 con 0x7f2fa4074550 2023-07-14T11:19:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.326+0000 7f2fa9f9c700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fa41079b0 0x7f2fa419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.327+0000 7f2fa8f9a700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4074550 0x7f2fa4198020 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f94004710 tx=0x7f2f94009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.327+0000 7f2f9a7fc700 1 -- 172.21.15.161:0/1345158055 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9401d070 con 0x7f2fa4074550 2023-07-14T11:19:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.327+0000 7f2f9a7fc700 1 -- 172.21.15.161:0/1345158055 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f94021410 con 0x7f2fa4074550 2023-07-14T11:19:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.327+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa419fbb0 con 0x7f2fa4074550 2023-07-14T11:19:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.327+0000 7f2f9a7fc700 1 -- 172.21.15.161:0/1345158055 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94017650 con 0x7f2fa4074550 2023-07-14T11:19:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.327+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa41a0030 con 0x7f2fa4074550 2023-07-14T11:19:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.328+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa4106060 con 0x7f2fa4074550 2023-07-14T11:19:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.330+0000 7f2f9a7fc700 1 -- 172.21.15.161:0/1345158055 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2f94021580 con 0x7f2fa4074550 2023-07-14T11:19:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.331+0000 7f2f9a7fc700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f900a11d0 0x7f2f900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.331+0000 7f2f9a7fc700 1 -- 172.21.15.161:0/1345158055 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f2f940cdf10 con 0x7f2fa4074550 2023-07-14T11:19:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.331+0000 7f2fa979b700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f900a11d0 0x7f2f900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.332+0000 7f2fa979b700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f900a11d0 0x7f2f900a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4074920 tx=0x7f2fa0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:46.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.335+0000 7f2f9a7fc700 1 -- 172.21.15.161:0/1345158055 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2f940974c0 con 0x7f2fa4074550 2023-07-14T11:19:46.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:46.581+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f2fa402c850 con 0x7f2fa4074550 2023-07-14T11:19:47.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.082+0000 7f2f9a7fc700 1 -- 172.21.15.161:0/1345158055 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7f2f9409aae0 con 0x7f2fa4074550 2023-07-14T11:19:47.086 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:19:46 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: 2023-07-14T11:19:46.679+0000 7f252077c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:19:47.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f900a11d0 msgr2=0x7f2f900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f900a11d0 0x7f2f900a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4074920 tx=0x7f2fa0009260 comp rx=0 tx=0).stop 2023-07-14T11:19:47.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4074550 msgr2=0x7f2fa4198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4074550 0x7f2fa4198020 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f94004710 tx=0x7f2f94009580 comp rx=0 tx=0).stop 2023-07-14T11:19:47.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 shutdown_connections 2023-07-14T11:19:47.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f900a11d0 0x7f2f900a3680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fa41079b0 0x7f2fa419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2fa4072f70 0x7f2fa4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 --2- 172.21.15.161:0/1345158055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fa4074550 0x7f2fa4198020 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 >> 172.21.15.161:0/1345158055 conn(0x7f2fa40fb130 msgr2=0x7f2fa40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:47.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 shutdown_connections 2023-07-14T11:19:47.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.085+0000 7f2fab9ff700 1 -- 172.21.15.161:0/1345158055 wait complete. 2023-07-14T11:19:47.089 INFO:teuthology.orchestra.run.smithi161.stderr:added data pool 4 to fsmap 2023-07-14T11:19:47.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:47.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 -- 172.21.15.161:0/3884037086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97780ff9f0 msgr2=0x7f97780ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 --2- 172.21.15.161:0/3884037086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97780ff9f0 0x7f97780ffe00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9774009990 tx=0x7f9774009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 -- 172.21.15.161:0/3884037086 shutdown_connections 2023-07-14T11:19:47.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 --2- 172.21.15.161:0/3884037086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97780ff9f0 0x7f97780ffe00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 --2- 172.21.15.161:0/3884037086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9778101570 0x7f9778107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 --2- 172.21.15.161:0/3884037086 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9778100be0 0x7f9778101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 -- 172.21.15.161:0/3884037086 >> 172.21.15.161:0/3884037086 conn(0x7f97780fb010 msgr2=0x7f97780fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:47.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 -- 172.21.15.161:0/3884037086 shutdown_connections 2023-07-14T11:19:47.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.304+0000 7f977e79d700 1 -- 172.21.15.161:0/3884037086 wait complete. 2023-07-14T11:19:47.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.305+0000 7f977e79d700 1 Processor -- start 2023-07-14T11:19:47.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.305+0000 7f977e79d700 1 -- start start 2023-07-14T11:19:47.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.305+0000 7f977e79d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97780ff9f0 0x7f9778197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977e79d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778100be0 0x7f9778197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977e79d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9778101570 0x7f977819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977e79d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977819f2e0 con 0x7f9778100be0 2023-07-14T11:19:47.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977e79d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977819f420 con 0x7f97780ff9f0 2023-07-14T11:19:47.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977e79d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977819f5b0 con 0x7f9778101570 2023-07-14T11:19:47.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778100be0 0x7f9778197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778100be0 0x7f9778197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48786/0 (socket says 172.21.15.161:48786) 2023-07-14T11:19:47.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 -- 172.21.15.161:0/1588376420 learned_addr learned my addr 172.21.15.161:0/1588376420 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:47.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977d79b700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97780ff9f0 0x7f9778197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977df9c700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9778101570 0x7f977819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 -- 172.21.15.161:0/1588376420 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9778101570 msgr2=0x7f977819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9778101570 0x7f977819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 -- 172.21.15.161:0/1588376420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97780ff9f0 msgr2=0x7f9778197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97780ff9f0 0x7f9778197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 -- 172.21.15.161:0/1588376420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740095f0 con 0x7f9778100be0 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977df9c700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9778101570 0x7f977819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.306+0000 7f977cf9a700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778100be0 0x7f9778197ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f976800d950 tx=0x7f976800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.307+0000 7f976e7fc700 1 -- 172.21.15.161:0/1588376420 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768009940 con 0x7f9778100be0 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.307+0000 7f976e7fc700 1 -- 172.21.15.161:0/1588376420 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9768010460 con 0x7f9778100be0 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.307+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977819f890 con 0x7f9778100be0 2023-07-14T11:19:47.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.307+0000 7f976e7fc700 1 -- 172.21.15.161:0/1588376420 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800f620 con 0x7f9778100be0 2023-07-14T11:19:47.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.307+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977819fdf0 con 0x7f9778100be0 2023-07-14T11:19:47.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.308+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977810b210 con 0x7f9778100be0 2023-07-14T11:19:47.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.310+0000 7f976e7fc700 1 -- 172.21.15.161:0/1588376420 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f97680108d0 con 0x7f9778100be0 2023-07-14T11:19:47.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.312+0000 7f976e7fc700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97640a12c0 0x7f97640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.312+0000 7f976e7fc700 1 -- 172.21.15.161:0/1588376420 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f97680cc960 con 0x7f9778100be0 2023-07-14T11:19:47.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.312+0000 7f977d79b700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97640a12c0 0x7f97640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.313+0000 7f977d79b700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97640a12c0 0x7f97640a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f977400b2e0 tx=0x7f9774005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:47.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.316+0000 7f976e7fc700 1 -- 172.21.15.161:0/1588376420 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9768095e60 con 0x7f9778100be0 2023-07-14T11:19:47.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:47 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4179815996' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T11:19:47.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:47 smithi175 ceph-mon[129081]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T11:19:47.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:47 smithi175 ceph-mon[129081]: pgmap v273: 129 pgs: 2 creating+activating, 16 creating+peering, 81 active+clean, 30 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-07-14T11:19:47.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:47 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1345158055' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:19:47.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:47 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4179815996' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T11:19:47.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:47 smithi182 ceph-mon[129366]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T11:19:47.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:47 smithi182 ceph-mon[129366]: pgmap v273: 129 pgs: 2 creating+activating, 16 creating+peering, 81 active+clean, 30 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-07-14T11:19:47.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:47 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1345158055' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:19:47.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:47 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4179815996' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T11:19:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:47 smithi161 ceph-mon[95526]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T11:19:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:47 smithi161 ceph-mon[95526]: pgmap v273: 129 pgs: 2 creating+activating, 16 creating+peering, 81 active+clean, 30 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-07-14T11:19:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:47 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1345158055' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:19:47.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.556+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f977810b3d0 con 0x7f9778100be0 2023-07-14T11:19:47.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.557+0000 7f976e7fc700 1 -- 172.21.15.161:0/1588376420 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+10803 (secure 0 0 0) 0x7f9768016090 con 0x7f9778100be0 2023-07-14T11:19:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:47.560 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:19:47.075743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T11:19:47.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97640a12c0 msgr2=0x7f97640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97640a12c0 0x7f97640a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f977400b2e0 tx=0x7f9774005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778100be0 msgr2=0x7f9778197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778100be0 0x7f9778197ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f976800d950 tx=0x7f976800dc60 comp rx=0 tx=0).stop 2023-07-14T11:19:47.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 shutdown_connections 2023-07-14T11:19:47.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97640a12c0 0x7f97640a3770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9778101570 0x7f977819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97780ff9f0 0x7f9778197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 --2- 172.21.15.161:0/1588376420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778100be0 0x7f9778197ca0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 >> 172.21.15.161:0/1588376420 conn(0x7f97780fb010 msgr2=0x7f97781063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:47.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 shutdown_connections 2023-07-14T11:19:47.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.559+0000 7f977e79d700 1 -- 172.21.15.161:0/1588376420 wait complete. 2023-07-14T11:19:47.564 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2023-07-14T11:19:47.580 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:19:47.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 -- 172.21.15.161:0/2382424592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c101d20 msgr2=0x7f761c102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 --2- 172.21.15.161:0/2382424592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c101d20 0x7f761c102130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f760c009990 tx=0x7f760c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 -- 172.21.15.161:0/2382424592 shutdown_connections 2023-07-14T11:19:47.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 --2- 172.21.15.161:0/2382424592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c101d20 0x7f761c102130 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 --2- 172.21.15.161:0/2382424592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f761c1038a0 0x7f761c10a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 --2- 172.21.15.161:0/2382424592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761c102f10 0x7f761c103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 -- 172.21.15.161:0/2382424592 >> 172.21.15.161:0/2382424592 conn(0x7f761c0fd360 msgr2=0x7f761c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:47.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 -- 172.21.15.161:0/2382424592 shutdown_connections 2023-07-14T11:19:47.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.779+0000 7f7620bff700 1 -- 172.21.15.161:0/2382424592 wait complete. 2023-07-14T11:19:47.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.780+0000 7f7620bff700 1 Processor -- start 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.780+0000 7f7620bff700 1 -- start start 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.780+0000 7f7620bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761c101d20 0x7f761c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.780+0000 7f7620bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f761c102f10 0x7f761c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f7620bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1038a0 0x7f761c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f7620bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c19f620 con 0x7f761c1038a0 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f7620bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c19f760 con 0x7f761c101d20 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f7620bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c19f8f0 con 0x7f761c102f10 2023-07-14T11:19:47.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1038a0 0x7f761c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1038a0 0x7f761c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48806/0 (socket says 172.21.15.161:48806) 2023-07-14T11:19:47.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 -- 172.21.15.161:0/235043697 learned_addr learned my addr 172.21.15.161:0/235043697 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:47.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761a59c700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761c101d20 0x7f761c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f7619d9b700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f761c102f10 0x7f761c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 -- 172.21.15.161:0/235043697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f761c102f10 msgr2=0x7f761c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f761c102f10 0x7f761c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 -- 172.21.15.161:0/235043697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761c101d20 msgr2=0x7f761c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761c101d20 0x7f761c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 -- 172.21.15.161:0/235043697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c0095f0 con 0x7f761c1038a0 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f7619d9b700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f761c102f10 0x7f761c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.781+0000 7f761ad9d700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1038a0 0x7f761c19f050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f761400ef30 tx=0x7f761400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.782+0000 7f760b7fe700 1 -- 172.21.15.161:0/235043697 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614018070 con 0x7f761c1038a0 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.782+0000 7f760b7fe700 1 -- 172.21.15.161:0/235043697 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761400f040 con 0x7f761c1038a0 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.782+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c19fb70 con 0x7f761c1038a0 2023-07-14T11:19:47.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.782+0000 7f760b7fe700 1 -- 172.21.15.161:0/235043697 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400cdb0 con 0x7f761c1038a0 2023-07-14T11:19:47.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.782+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c1a00d0 con 0x7f761c1038a0 2023-07-14T11:19:47.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.783+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c109290 con 0x7f761c1038a0 2023-07-14T11:19:47.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.785+0000 7f760b7fe700 1 -- 172.21.15.161:0/235043697 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7614003a00 con 0x7f761c1038a0 2023-07-14T11:19:47.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.786+0000 7f760b7fe700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76040a1220 0x7f76040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:47.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.786+0000 7f760b7fe700 1 -- 172.21.15.161:0/235043697 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f76140cc630 con 0x7f761c1038a0 2023-07-14T11:19:47.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.786+0000 7f761a59c700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76040a1220 0x7f76040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:47.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.787+0000 7f761a59c700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76040a1220 0x7f76040a36d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b2e0 tx=0x7f760c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:47.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.790+0000 7f760b7fe700 1 -- 172.21.15.161:0/235043697 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7614095bc0 con 0x7f761c1038a0 2023-07-14T11:19:47.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.996+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f761c02c850 con 0x7f761c1038a0 2023-07-14T11:19:48.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:47.997+0000 7f760b7fe700 1 -- 172.21.15.161:0/235043697 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f76140991e0 con 0x7f761c1038a0 2023-07-14T11:19:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:48.001 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:19:46.582122+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:48.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76040a1220 msgr2=0x7f76040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76040a1220 0x7f76040a36d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b2e0 tx=0x7f760c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1038a0 msgr2=0x7f761c19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1038a0 0x7f761c19f050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f761400ef30 tx=0x7f761400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 shutdown_connections 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1038a0 0x7f761c19f050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76040a1220 0x7f76040a36d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f761c101d20 0x7f761c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 --2- 172.21.15.161:0/235043697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f761c102f10 0x7f761c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 >> 172.21.15.161:0/235043697 conn(0x7f761c0fd360 msgr2=0x7f761c1063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:48.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 shutdown_connections 2023-07-14T11:19:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.000+0000 7f7620bff700 1 -- 172.21.15.161:0/235043697 wait complete. 2023-07-14T11:19:48.022 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:19:48.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 -- 172.21.15.161:0/891449596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6004100cf0 msgr2=0x7f6004101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 --2- 172.21.15.161:0/891449596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6004100cf0 0x7f6004101140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009990 tx=0x7f5ff8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 -- 172.21.15.161:0/891449596 shutdown_connections 2023-07-14T11:19:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 --2- 172.21.15.161:0/891449596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6004101680 0x7f6004107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 --2- 172.21.15.161:0/891449596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60040ffb00 0x7f60040fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 --2- 172.21.15.161:0/891449596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6004100cf0 0x7f6004101140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 -- 172.21.15.161:0/891449596 >> 172.21.15.161:0/891449596 conn(0x7f60040fb0e0 msgr2=0x7f60040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 -- 172.21.15.161:0/891449596 shutdown_connections 2023-07-14T11:19:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.219+0000 7f6009dff700 1 -- 172.21.15.161:0/891449596 wait complete. 2023-07-14T11:19:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.220+0000 7f6009dff700 1 Processor -- start 2023-07-14T11:19:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6009dff700 1 -- start start 2023-07-14T11:19:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6009dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60040ffb00 0x7f60040742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6009dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6004100cf0 0x7f6004072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6009dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6004101680 0x7f6004072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6009dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040735e0 con 0x7f60040ffb00 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6009dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6004073750 con 0x7f6004100cf0 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6009dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600419f860 con 0x7f6004101680 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f60037fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60040ffb00 0x7f60040742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f60037fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60040ffb00 0x7f60040742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58532/0 (socket says 172.21.15.161:58532) 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f60037fe700 1 -- 172.21.15.161:0/1628241873 learned_addr learned my addr 172.21.15.161:0/1628241873 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6003fff700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6004101680 0x7f6004072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f6002ffd700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6004100cf0 0x7f6004072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f60037fe700 1 -- 172.21.15.161:0/1628241873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6004101680 msgr2=0x7f6004072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f60037fe700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6004101680 0x7f6004072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f60037fe700 1 -- 172.21.15.161:0/1628241873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6004100cf0 msgr2=0x7f6004072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.221+0000 7f60037fe700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6004100cf0 0x7f6004072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.222+0000 7f60037fe700 1 -- 172.21.15.161:0/1628241873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80095f0 con 0x7f60040ffb00 2023-07-14T11:19:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.222+0000 7f60037fe700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60040ffb00 0x7f60040742c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e970 tx=0x7f5ff400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.222+0000 7f6000ff9700 1 -- 172.21.15.161:0/1628241873 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4009790 con 0x7f60040ffb00 2023-07-14T11:19:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.222+0000 7f6000ff9700 1 -- 172.21.15.161:0/1628241873 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ff400eec0 con 0x7f60040ffb00 2023-07-14T11:19:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.222+0000 7f6000ff9700 1 -- 172.21.15.161:0/1628241873 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4003cf0 con 0x7f60040ffb00 2023-07-14T11:19:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.222+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600419fb40 con 0x7f60040ffb00 2023-07-14T11:19:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.222+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60041a0000 con 0x7f60040ffb00 2023-07-14T11:19:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.223+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004106ff0 con 0x7f60040ffb00 2023-07-14T11:19:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.225+0000 7f6000ff9700 1 -- 172.21.15.161:0/1628241873 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5ff400ccd0 con 0x7f60040ffb00 2023-07-14T11:19:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.226+0000 7f6000ff9700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5fec0a1220 0x7f5fec0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.226+0000 7f6000ff9700 1 -- 172.21.15.161:0/1628241873 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5ff4014070 con 0x7f60040ffb00 2023-07-14T11:19:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.226+0000 7f6002ffd700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5fec0a1220 0x7f5fec0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.227+0000 7f6002ffd700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5fec0a1220 0x7f5fec0a36d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800b2e0 tx=0x7f5ff8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:48.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.230+0000 7f6000ff9700 1 -- 172.21.15.161:0/1628241873 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5ff4096210 con 0x7f60040ffb00 2023-07-14T11:19:48.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:48 smithi175 ceph-mon[129081]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T11:19:48.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:48 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1345158055' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T11:19:48.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:48 smithi175 ceph-mon[129081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:19:48.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:48 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1588376420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:48.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:48 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/235043697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:48.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.430+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f600402c850 con 0x7f60040ffb00 2023-07-14T11:19:48.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.431+0000 7f6000ff9700 1 -- 172.21.15.161:0/1628241873 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f5ff4099830 con 0x7f60040ffb00 2023-07-14T11:19:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:48.434 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:19:46.582122+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:48.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5fec0a1220 msgr2=0x7f5fec0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5fec0a1220 0x7f5fec0a36d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800b2e0 tx=0x7f5ff8005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60040ffb00 msgr2=0x7f60040742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60040ffb00 0x7f60040742c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e970 tx=0x7f5ff400ec80 comp rx=0 tx=0).stop 2023-07-14T11:19:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 shutdown_connections 2023-07-14T11:19:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60040ffb00 0x7f60040742c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5fec0a1220 0x7f5fec0a36d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6004101680 0x7f6004072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 --2- 172.21.15.161:0/1628241873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6004100cf0 0x7f6004072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 >> 172.21.15.161:0/1628241873 conn(0x7f60040fb0e0 msgr2=0x7f6004104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 shutdown_connections 2023-07-14T11:19:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.434+0000 7f6009dff700 1 -- 172.21.15.161:0/1628241873 wait complete. 2023-07-14T11:19:48.452 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:48 smithi161 ceph-mon[95526]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T11:19:48.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:48 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1345158055' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T11:19:48.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:48 smithi161 ceph-mon[95526]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:19:48.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:48 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1588376420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:48.453 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:48 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/235043697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:48.472 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:19:48.473 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:48 smithi182 ceph-mon[129366]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T11:19:48.474 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:48 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1345158055' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T11:19:48.474 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:48 smithi182 ceph-mon[129366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:19:48.474 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:48 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1588376420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:48.474 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:48 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/235043697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:48.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 -- 172.21.15.161:0/472338583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 msgr2=0x7f14680fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 --2- 172.21.15.161:0/472338583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f14680fff10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1464009990 tx=0x7f1464009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 -- 172.21.15.161:0/472338583 shutdown_connections 2023-07-14T11:19:48.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 --2- 172.21.15.161:0/472338583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f14680fff10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 --2- 172.21.15.161:0/472338583 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1468101680 0x7f1468107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 --2- 172.21.15.161:0/472338583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1468100cf0 0x7f1468101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 -- 172.21.15.161:0/472338583 >> 172.21.15.161:0/472338583 conn(0x7f14680fb0e0 msgr2=0x7f14680fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 -- 172.21.15.161:0/472338583 shutdown_connections 2023-07-14T11:19:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.670+0000 7f14701ff700 1 -- 172.21.15.161:0/472338583 wait complete. 2023-07-14T11:19:48.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.671+0000 7f14701ff700 1 Processor -- start 2023-07-14T11:19:48.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.671+0000 7f14701ff700 1 -- start start 2023-07-14T11:19:48.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f14701ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f1468197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f14701ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1468100cf0 0x7f1468197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f14701ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1468101680 0x7f146819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f14701ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146819f5b0 con 0x7f14680ffb00 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f14701ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146819f6f0 con 0x7f1468100cf0 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f14701ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146819f880 con 0x7f1468101680 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f1468197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f1468197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58546/0 (socket says 172.21.15.161:58546) 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 -- 172.21.15.161:0/1844217334 learned_addr learned my addr 172.21.15.161:0/1844217334 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146d79a700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1468100cf0 0x7f1468197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146e79c700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1468101680 0x7f146819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 -- 172.21.15.161:0/1844217334 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1468101680 msgr2=0x7f146819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1468101680 0x7f146819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 -- 172.21.15.161:0/1844217334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1468100cf0 msgr2=0x7f1468197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1468100cf0 0x7f1468197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.672+0000 7f146df9b700 1 -- 172.21.15.161:0/1844217334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640095f0 con 0x7f14680ffb00 2023-07-14T11:19:48.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.673+0000 7f146df9b700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f1468197a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1464000c00 tx=0x7f1464005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:48.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.673+0000 7f145effd700 1 -- 172.21.15.161:0/1844217334 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146401d070 con 0x7f14680ffb00 2023-07-14T11:19:48.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.673+0000 7f145effd700 1 -- 172.21.15.161:0/1844217334 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1464003f60 con 0x7f14680ffb00 2023-07-14T11:19:48.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.673+0000 7f145effd700 1 -- 172.21.15.161:0/1844217334 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464022c70 con 0x7f14680ffb00 2023-07-14T11:19:48.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.673+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146819fb00 con 0x7f14680ffb00 2023-07-14T11:19:48.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.673+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146819ff20 con 0x7f14680ffb00 2023-07-14T11:19:48.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.674+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14681921e0 con 0x7f14680ffb00 2023-07-14T11:19:48.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.676+0000 7f145effd700 1 -- 172.21.15.161:0/1844217334 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f14640040d0 con 0x7f14680ffb00 2023-07-14T11:19:48.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.677+0000 7f145effd700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f14540a1220 0x7f14540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:48.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.677+0000 7f145effd700 1 -- 172.21.15.161:0/1844217334 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f14640ce0b0 con 0x7f14680ffb00 2023-07-14T11:19:48.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.678+0000 7f146d79a700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f14540a1220 0x7f14540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:48.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.678+0000 7f146d79a700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f14540a1220 0x7f14540a36d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1458007c60 tx=0x7f14580073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:48.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.681+0000 7f145effd700 1 -- 172.21.15.161:0/1844217334 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1464097640 con 0x7f14680ffb00 2023-07-14T11:19:48.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.923+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f146802c850 con 0x7f14680ffb00 2023-07-14T11:19:48.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.924+0000 7f145effd700 1 -- 172.21.15.161:0/1844217334 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+10803 (secure 0 0 0) 0x7f146409ac60 con 0x7f14680ffb00 2023-07-14T11:19:48.927 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:48.927 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:19:47.075743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T11:19:48.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.926+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f14540a1220 msgr2=0x7f14540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.926+0000 7f14701ff700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f14540a1220 0x7f14540a36d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1458007c60 tx=0x7f14580073d0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.926+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 msgr2=0x7f1468197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:48.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.926+0000 7f14701ff700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f1468197a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1464000c00 tx=0x7f1464005740 comp rx=0 tx=0).stop 2023-07-14T11:19:48.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.926+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 shutdown_connections 2023-07-14T11:19:48.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.926+0000 7f14701ff700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14680ffb00 0x7f1468197a30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.927+0000 7f14701ff700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f14540a1220 0x7f14540a36d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.927+0000 7f14701ff700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1468101680 0x7f146819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.927+0000 7f14701ff700 1 --2- 172.21.15.161:0/1844217334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1468100cf0 0x7f1468197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:48.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.927+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 >> 172.21.15.161:0/1844217334 conn(0x7f14680fb0e0 msgr2=0x7f14681065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:48.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.927+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 shutdown_connections 2023-07-14T11:19:48.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:48.927+0000 7f14701ff700 1 -- 172.21.15.161:0/1844217334 wait complete. 2023-07-14T11:19:48.931 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2023-07-14T11:19:48.947 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:19:49.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.146+0000 7f27ce7ff700 1 -- 172.21.15.161:0/4155884507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 msgr2=0x7f27c8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.146+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/4155884507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8101140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009990 tx=0x7f27bc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.147+0000 7f27ce7ff700 1 -- 172.21.15.161:0/4155884507 shutdown_connections 2023-07-14T11:19:49.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.147+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/4155884507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27c8101680 0x7f27c8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.147+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/4155884507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c80ffb00 0x7f27c80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.147+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/4155884507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8101140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.147+0000 7f27ce7ff700 1 -- 172.21.15.161:0/4155884507 >> 172.21.15.161:0/4155884507 conn(0x7f27c80fb0e0 msgr2=0x7f27c80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.147+0000 7f27ce7ff700 1 -- 172.21.15.161:0/4155884507 shutdown_connections 2023-07-14T11:19:49.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.147+0000 7f27ce7ff700 1 -- 172.21.15.161:0/4155884507 wait complete. 2023-07-14T11:19:49.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 Processor -- start 2023-07-14T11:19:49.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 -- start start 2023-07-14T11:19:49.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c80ffb00 0x7f27c8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27c8101680 0x7f27c819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819f5a0 con 0x7f27c8100cf0 2023-07-14T11:19:49.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819f6e0 con 0x7f27c80ffb00 2023-07-14T11:19:49.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27ce7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819f870 con 0x7f27c8101680 2023-07-14T11:19:49.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27c77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27c77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58560/0 (socket says 172.21.15.161:58560) 2023-07-14T11:19:49.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27c77fe700 1 -- 172.21.15.161:0/870199009 learned_addr learned my addr 172.21.15.161:0/870199009 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:49.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.148+0000 7f27c7fff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c80ffb00 0x7f27c8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27ccd9c700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27c8101680 0x7f27c819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c77fe700 1 -- 172.21.15.161:0/870199009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27c8101680 msgr2=0x7f27c819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c77fe700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27c8101680 0x7f27c819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c77fe700 1 -- 172.21.15.161:0/870199009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c80ffb00 msgr2=0x7f27c8197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c77fe700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c80ffb00 0x7f27c8197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c77fe700 1 -- 172.21.15.161:0/870199009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc0095f0 con 0x7f27c8100cf0 2023-07-14T11:19:49.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c77fe700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8197f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f27bc004760 tx=0x7f27bc004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c57fa700 1 -- 172.21.15.161:0/870199009 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc01d070 con 0x7f27c8100cf0 2023-07-14T11:19:49.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c57fa700 1 -- 172.21.15.161:0/870199009 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27bc004830 con 0x7f27c8100cf0 2023-07-14T11:19:49.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c819faf0 con 0x7f27c8100cf0 2023-07-14T11:19:49.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27c57fa700 1 -- 172.21.15.161:0/870199009 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc005480 con 0x7f27c8100cf0 2023-07-14T11:19:49.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.149+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c819ff10 con 0x7f27c8100cf0 2023-07-14T11:19:49.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.150+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c810b320 con 0x7f27c8100cf0 2023-07-14T11:19:49.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.152+0000 7f27c57fa700 1 -- 172.21.15.161:0/870199009 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f27bc00fad0 con 0x7f27c8100cf0 2023-07-14T11:19:49.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.153+0000 7f27c57fa700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27b00a1220 0x7f27b00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.154+0000 7f27c57fa700 1 -- 172.21.15.161:0/870199009 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f27bc0cdc90 con 0x7f27c8100cf0 2023-07-14T11:19:49.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.154+0000 7f27c7fff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27b00a1220 0x7f27b00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.154+0000 7f27c7fff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27b00a1220 0x7f27b00a36d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f27c8100b50 tx=0x7f27b8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.158+0000 7f27c57fa700 1 -- 172.21.15.161:0/870199009 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f27bc097220 con 0x7f27c8100cf0 2023-07-14T11:19:49.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:49 smithi161 ceph-mon[95526]: pgmap v275: 129 pgs: 1 creating+activating, 2 creating+peering, 117 active+clean, 9 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2023-07-14T11:19:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1628241873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1844217334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:49.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.359+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27c81a0580 con 0x7f27c8100cf0 2023-07-14T11:19:49.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.360+0000 7f27c57fa700 1 -- 172.21.15.161:0/870199009 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f27bc0a0090 con 0x7f27c8100cf0 2023-07-14T11:19:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:49.363 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:19:46.582122+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:49.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27b00a1220 msgr2=0x7f27b00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27b00a1220 0x7f27b00a36d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f27c8100b50 tx=0x7f27b8009260 comp rx=0 tx=0).stop 2023-07-14T11:19:49.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 msgr2=0x7f27c8197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8197f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f27bc004760 tx=0x7f27bc004790 comp rx=0 tx=0).stop 2023-07-14T11:19:49.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 shutdown_connections 2023-07-14T11:19:49.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27b00a1220 0x7f27b00a36d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27c8101680 0x7f27c819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c80ffb00 0x7f27c8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 --2- 172.21.15.161:0/870199009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27c8100cf0 0x7f27c8197f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 >> 172.21.15.161:0/870199009 conn(0x7f27c80fb0e0 msgr2=0x7f27c8106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 shutdown_connections 2023-07-14T11:19:49.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.363+0000 7f27ce7ff700 1 -- 172.21.15.161:0/870199009 wait complete. 2023-07-14T11:19:49.385 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-14T11:19:41.650966+0000', 'modified': '2023-07-14T11:19:47.075743+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6839/2144664422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 2144664422}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 2144664422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-14T11:19:49.385 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-14T11:19:49.385 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T11:19:49.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:49 smithi175 ceph-mon[129081]: pgmap v275: 129 pgs: 1 creating+activating, 2 creating+peering, 117 active+clean, 9 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2023-07-14T11:19:49.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:49 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1628241873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:49.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:49 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1844217334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:49.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:49 smithi182 ceph-mon[129366]: pgmap v275: 129 pgs: 1 creating+activating, 2 creating+peering, 117 active+clean, 9 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2023-07-14T11:19:49.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:49 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1628241873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:49.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:49 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1844217334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:19:49.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 -- 172.21.15.161:0/1802685859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c600ffaf0 msgr2=0x7f5c600fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 --2- 172.21.15.161:0/1802685859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c600ffaf0 0x7f5c600fff00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009990 tx=0x7f5c5c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 -- 172.21.15.161:0/1802685859 shutdown_connections 2023-07-14T11:19:49.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 --2- 172.21.15.161:0/1802685859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c60101670 0x7f5c60107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 --2- 172.21.15.161:0/1802685859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c600ffaf0 0x7f5c600fff00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 --2- 172.21.15.161:0/1802685859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60100ce0 0x7f5c60101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 -- 172.21.15.161:0/1802685859 >> 172.21.15.161:0/1802685859 conn(0x7f5c600fb130 msgr2=0x7f5c600fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 -- 172.21.15.161:0/1802685859 shutdown_connections 2023-07-14T11:19:49.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.582+0000 7f5c677ff700 1 -- 172.21.15.161:0/1802685859 wait complete. 2023-07-14T11:19:49.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 Processor -- start 2023-07-14T11:19:49.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 -- start start 2023-07-14T11:19:49.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c600ffaf0 0x7f5c601979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c60100ce0 0x7f5c60197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60101670 0x7f5c6019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6019f570 con 0x7f5c600ffaf0 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6019f6b0 con 0x7f5c60101670 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c677ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6019f840 con 0x7f5c60100ce0 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c6559b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c600ffaf0 0x7f5c601979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c6559b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c600ffaf0 0x7f5c601979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58588/0 (socket says 172.21.15.161:58588) 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c6559b700 1 -- 172.21.15.161:0/2087451627 learned_addr learned my addr 172.21.15.161:0/2087451627 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.583+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c60100ce0 0x7f5c60197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c65d9c700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60101670 0x7f5c6019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c6559b700 1 -- 172.21.15.161:0/2087451627 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c60100ce0 msgr2=0x7f5c60197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c6559b700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c60100ce0 0x7f5c60197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c6559b700 1 -- 172.21.15.161:0/2087451627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60101670 msgr2=0x7f5c6019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c6559b700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60101670 0x7f5c6019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c6559b700 1 -- 172.21.15.161:0/2087451627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0095f0 con 0x7f5c600ffaf0 2023-07-14T11:19:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c6559b700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c600ffaf0 0x7f5c601979f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00b680 tx=0x7f5c5c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c567fc700 1 -- 172.21.15.161:0/2087451627 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c01d070 con 0x7f5c600ffaf0 2023-07-14T11:19:49.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6019fac0 con 0x7f5c600ffaf0 2023-07-14T11:19:49.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c567fc700 1 -- 172.21.15.161:0/2087451627 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c5c021410 con 0x7f5c600ffaf0 2023-07-14T11:19:49.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c567fc700 1 -- 172.21.15.161:0/2087451627 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c017650 con 0x7f5c600ffaf0 2023-07-14T11:19:49.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.584+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6019ff40 con 0x7f5c600ffaf0 2023-07-14T11:19:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.589+0000 7f5c567fc700 1 -- 172.21.15.161:0/2087451627 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5c5c021580 con 0x7f5c600ffaf0 2023-07-14T11:19:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 Processor -- start 2023-07-14T11:19:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 -- start start 2023-07-14T11:19:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c567fc700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c4c0a1150 0x7f5c4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c567fc700 1 -- 172.21.15.161:0/2087451627 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c5c0cdfb0 con 0x7f5c600ffaf0 2023-07-14T11:19:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e3e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39d43c60 con 0x560a39e3edd0 2023-07-14T11:19:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39d969f0 con 0x560a39e3fc20 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.590+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39d8fe80 con 0x560a39e3e480 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c65d9c700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c4c0a1150 0x7f5c4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e3f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e3f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58594/0 (socket says 172.21.15.161:58594) 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3090019740 learned_addr learned my addr 172.21.15.161:0/3090019740 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e3e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3090019740 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 msgr2=0x560a39e3e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e3e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3090019740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 msgr2=0x560a39e40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3090019740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a39bbdd90 con 0x560a39e3edd0 2023-07-14T11:19:49.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e3e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:49.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c65d9c700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c4c0a1150 0x7f5c4c0a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c58003f20 tx=0x7f5c58009ba0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.591+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e3f690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009d80 tx=0x7f5c50011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63c2f5d79a551a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c4b7fe700 1 -- 172.21.15.161:0/3090019740 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50004230 con 0x560a39e3edd0 2023-07-14T11:19:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c4b7fe700 1 -- 172.21.15.161:0/3090019740 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c50004390 con 0x560a39e3edd0 2023-07-14T11:19:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c4b7fe700 1 -- 172.21.15.161:0/3090019740 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500197c0 con 0x560a39e3edd0 2023-07-14T11:19:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3090019740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 msgr2=0x560a39e3f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e3f690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009d80 tx=0x7f5c50011720 comp rx=0 tx=0).stop 2023-07-14T11:19:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3090019740 shutdown_connections 2023-07-14T11:19:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e3e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3090019740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e3f690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.592+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3090019740 >> 172.21.15.161:0/3090019740 conn(0x560a39e33d50 msgr2=0x560a39e39180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.593+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3090019740 shutdown_connections 2023-07-14T11:19:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.593+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3090019740 wait complete. 2023-07-14T11:19:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.593+0000 7f5c7b1cab80 1 -- start start 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.593+0000 7f5c7b1cab80 1 Processor -- start 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e48660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e4f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39d2fa70 con 0x560a39e3edd0 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39d2f770 con 0x560a39e3fc20 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39c138d0 con 0x560a39e3e480 2023-07-14T11:19:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c7b1cab80 10 client.0 _ll_register_callbacks cb 0x560a39d2d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:19:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e48660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e48660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58604/0 (socket says 172.21.15.161:58604) 2023-07-14T11:19:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3086708921 learned_addr learned my addr 172.21.15.161:0/3086708921 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c65d9c700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e4f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3086708921 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 msgr2=0x560a39e48120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e48120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 msgr2=0x560a39e4f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e4f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c64d9a700 1 -- 172.21.15.161:0/3086708921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6019ff40 con 0x560a39e3edd0 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.594+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e48120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e48660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c50005c50 tx=0x7f5c50003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4f75f5b85714ee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c48ff9700 1 -- 172.21.15.161:0/3086708921 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50005240 con 0x560a39e3edd0 2023-07-14T11:19:49.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a39ce94a0 con 0x560a39e3edd0 2023-07-14T11:19:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c48ff9700 1 -- 172.21.15.161:0/3086708921 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c50005ce0 con 0x560a39e3edd0 2023-07-14T11:19:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c48ff9700 1 -- 172.21.15.161:0/3086708921 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c500053a0 con 0x560a39e3edd0 2023-07-14T11:19:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c7b1cab80 10 client.14643 fetch_fsmap learned FSMap version 16 2023-07-14T11:19:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x7f5c500099e0 con 0x560a39e3edd0 2023-07-14T11:19:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c48ff9700 1 -- 172.21.15.161:0/3086708921 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a39ce94a0 con 0x560a39e3edd0 2023-07-14T11:19:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.595+0000 7f5c48ff9700 1 -- 172.21.15.161:0/3086708921 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 16) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5c50021040 con 0x560a39e3edd0 2023-07-14T11:19:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c7b1cab80 10 client.14643 fetch_fsmap finished waiting for FSMap version 16 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c7b1cab80 10 client.14643 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c7b1cab80 20 client.14643 populate_metadata read hostname 'smithi161' 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39e532f0 0x560a39e52750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c7b1cab80 4 client.14643 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a39d88e60 con 0x560a39e532f0 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39e532f0 0x560a39e52750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c48ff9700 10 client.14643 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:19:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.596+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39e532f0 0x560a39e52750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b94e31eb9428bbd7 server_cookie=12ba93a4e37c718f in_seq=0 out_seq=0 2023-07-14T11:19:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.597+0000 7f5c48ff9700 1 -- 172.21.15.161:0/3086708921 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a39d88e60 con 0x560a39e532f0 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.597+0000 7f5c48ff9700 10 client.14643 handle_command_reply: tid=0 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 client.14643 shutdown 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 msgr2=0x560a39e48660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e48660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c50005c50 tx=0x7f5c50003960 comp rx=0 tx=0).stop 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 shutdown_connections 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39e532f0 0x560a39e52750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e4f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3e480 0x560a39e48120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c48ff9700 0 client.14643 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:19:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3086708921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e48660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 >> 172.21.15.161:0/3086708921 conn(0x560a39e33d50 msgr2=0x560a39e38860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 shutdown_connections 2023-07-14T11:19:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3086708921 wait complete. 2023-07-14T11:19:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.598+0000 7f5c7b1cab80 20 client.14643 trim_cache size 0 max 16384 2023-07-14T11:19:49.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 Processor -- start 2023-07-14T11:19:49.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 -- start start 2023-07-14T11:19:49.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3e480 0x560a39e46e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e46010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39d5e790 con 0x560a39e3e480 2023-07-14T11:19:49.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39c1fde0 con 0x560a39e3fc20 2023-07-14T11:19:49.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39d4e480 con 0x560a39e3edd0 2023-07-14T11:19:49.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c6559b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3e480 0x560a39e46e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c6559b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3e480 0x560a39e46e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58616/0 (socket says 172.21.15.161:58616) 2023-07-14T11:19:49.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c6559b700 1 -- 172.21.15.161:0/3714386053 learned_addr learned my addr 172.21.15.161:0/3714386053 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:49.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c65d9c700 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e46010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.616+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c6559b700 1 -- 172.21.15.161:0/3714386053 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3edd0 msgr2=0x560a39e4f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c6559b700 1 -- 172.21.15.161:0/3714386053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 msgr2=0x560a39e46010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e46010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c6559b700 1 -- 172.21.15.161:0/3714386053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a39ba6c10 con 0x560a39e3e480 2023-07-14T11:19:49.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c6559b700 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3e480 0x560a39e46e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c013ff0 tx=0x7f5c5c0cb730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ece358d6d2bc92 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c4bfff700 1 -- 172.21.15.161:0/3714386053 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0a35c0 con 0x560a39e3e480 2023-07-14T11:19:49.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c4bfff700 1 -- 172.21.15.161:0/3714386053 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c5c0995a0 con 0x560a39e3e480 2023-07-14T11:19:49.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.617+0000 7f5c4bfff700 1 -- 172.21.15.161:0/3714386053 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c09a950 con 0x560a39e3e480 2023-07-14T11:19:49.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3714386053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3e480 msgr2=0x560a39e46e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3e480 0x560a39e46e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c013ff0 tx=0x7f5c5c0cb730 comp rx=0 tx=0).stop 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3714386053 shutdown_connections 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3e480 0x560a39e46e00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3fc20 0x560a39e46010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/3714386053 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3714386053 >> 172.21.15.161:0/3714386053 conn(0x560a39e33d50 msgr2=0x560a39e38860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3714386053 shutdown_connections 2023-07-14T11:19:49.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/3714386053 wait complete. 2023-07-14T11:19:49.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 -- start start 2023-07-14T11:19:49.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.618+0000 7f5c7b1cab80 1 Processor -- start 2023-07-14T11:19:49.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3e480 0x560a39e45540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c7b1cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3fc20 0x560a39ea7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39ea7950 con 0x560a39e3edd0 2023-07-14T11:19:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39ea7ac0 con 0x560a39e3e480 2023-07-14T11:19:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c7b1cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a39ea7c80 con 0x560a39e3fc20 2023-07-14T11:19:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c7b1cab80 10 client.0 _ll_register_callbacks cb 0x7f5c5c0057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:19:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58630/0 (socket says 172.21.15.161:58630) 2023-07-14T11:19:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 -- 172.21.15.161:0/47554361 learned_addr learned my addr 172.21.15.161:0/47554361 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c65d9c700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3fc20 0x560a39ea7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c6559b700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3e480 0x560a39e45540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 -- 172.21.15.161:0/47554361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3fc20 msgr2=0x560a39ea7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3fc20 0x560a39ea7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 -- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3e480 msgr2=0x560a39e45540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:19:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3e480 0x560a39e45540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c64d9a700 1 -- 172.21.15.161:0/47554361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6019fac0 con 0x560a39e3edd0 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.619+0000 7f5c65d9c700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3fc20 0x560a39ea7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c64d9a700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009cd0 tx=0x7f5c50019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671919dfa5808b1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c48ff9700 1 -- 172.21.15.161:0/47554361 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50005450 con 0x560a39e3edd0 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c50004390 con 0x560a39e3edd0 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c48ff9700 1 -- 172.21.15.161:0/47554361 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c50000e40 con 0x560a39e3edd0 2023-07-14T11:19:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c48ff9700 1 -- 172.21.15.161:0/47554361 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c50019420 con 0x560a39e3edd0 2023-07-14T11:19:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c7b1cab80 10 client.14649 fetch_fsmap learned FSMap version 16 2023-07-14T11:19:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x560a39be4200 con 0x560a39e3edd0 2023-07-14T11:19:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.620+0000 7f5c48ff9700 1 -- 172.21.15.161:0/47554361 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c50004390 con 0x560a39e3edd0 2023-07-14T11:19:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c48ff9700 1 -- 172.21.15.161:0/47554361 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 16) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5c50011790 con 0x560a39e3edd0 2023-07-14T11:19:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c7b1cab80 10 client.14649 fetch_fsmap finished waiting for FSMap version 16 2023-07-14T11:19:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c7b1cab80 10 client.14649 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T11:19:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c7b1cab80 20 client.14649 populate_metadata read hostname 'smithi161' 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39eac570 0x560a39ea8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c7b1cab80 4 client.14649 mds_command: new command op to 24575 tid=0[{"prefix": "status"}] 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a39d4b000 con 0x560a39eac570 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c6559b700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39eac570 0x560a39ea8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c48ff9700 10 client.14649 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.621+0000 7f5c6559b700 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39eac570 0x560a39ea8f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a13e00f55682663 server_cookie=e4cfa80867066726 in_seq=0 out_seq=0 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c48ff9700 1 -- 172.21.15.161:0/47554361 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x560a39d4b000 con 0x560a39eac570 2023-07-14T11:19:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c48ff9700 10 client.14649 handle_command_reply: tid=0 2023-07-14T11:19:49.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 client.14649 shutdown 2023-07-14T11:19:49.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 msgr2=0x560a39e4f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009cd0 tx=0x7f5c50019760 comp rx=0 tx=0).stop 2023-07-14T11:19:49.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 shutdown_connections 2023-07-14T11:19:49.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560a39eac570 0x560a39ea8f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560a39e3fc20 0x560a39ea7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a39e3e480 0x560a39e45540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c48ff9700 0 client.14649 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:19:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 --2- 172.21.15.161:0/47554361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a39e3edd0 0x560a39e4f660 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 >> 172.21.15.161:0/47554361 conn(0x560a39e33d50 msgr2=0x560a39e52750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 shutdown_connections 2023-07-14T11:19:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 1 -- 172.21.15.161:0/47554361 wait complete. 2023-07-14T11:19:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.622+0000 7f5c7b1cab80 20 client.14649 trim_cache size 0 max 16384 2023-07-14T11:19:49.633 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:19:49.634 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:19:49.634 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2023-07-14T11:19:49.634 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24575, 2023-07-14T11:19:49.634 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2023-07-14T11:19:49.634 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2023-07-14T11:19:49.634 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2023-07-14T11:19:49.635 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 7.5265134590000002, 2023-07-14T11:19:49.635 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 16, 2023-07-14T11:19:49.635 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 88, 2023-07-14T11:19:49.635 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 87, 2023-07-14T11:19:49.635 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 13.943565075 2023-07-14T11:19:49.635 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:19:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c4c0a1150 msgr2=0x7f5c4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c4c0a1150 0x7f5c4c0a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c58003f20 tx=0x7f5c58009ba0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c600ffaf0 msgr2=0x7f5c601979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c600ffaf0 0x7f5c601979f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00b680 tx=0x7f5c5c005740 comp rx=0 tx=0).stop 2023-07-14T11:19:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 shutdown_connections 2023-07-14T11:19:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c600ffaf0 0x7f5c601979f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c4c0a1150 0x7f5c4c0a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60101670 0x7f5c6019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 --2- 172.21.15.161:0/2087451627 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c60100ce0 0x7f5c60197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 >> 172.21.15.161:0/2087451627 conn(0x7f5c600fb130 msgr2=0x7f5c601065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 shutdown_connections 2023-07-14T11:19:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:49.623+0000 7f5c677ff700 1 -- 172.21.15.161:0/2087451627 wait complete. 2023-07-14T11:19:49.643 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:19:49.644 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:19:49.644 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph fs ls 2023-07-14T11:19:50.363 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:19:50.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:50 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/870199009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:50 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/870199009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:50 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/870199009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:51.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:51 smithi161 ceph-mon[95526]: pgmap v276: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2023-07-14T11:19:51.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:51 smithi175 ceph-mon[129081]: pgmap v276: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2023-07-14T11:19:51.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:51 smithi182 ceph-mon[129366]: pgmap v276: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2023-07-14T11:19:51.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.878+0000 7ff2a57cb700 1 -- 172.21.15.161:0/1099376179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a01007a0 msgr2=0x7ff2a0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.878+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/1099376179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a01007a0 0x7ff2a0104770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff2940099c0 tx=0x7ff294009cd0 comp rx=0 tx=0).stop 2023-07-14T11:19:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.879+0000 7ff2a57cb700 1 -- 172.21.15.161:0/1099376179 shutdown_connections 2023-07-14T11:19:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.879+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/1099376179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a0104ea0 0x7ff2a010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.879+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/1099376179 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2a00ffe00 0x7ff2a01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.879+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/1099376179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a01007a0 0x7ff2a0104770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.879+0000 7ff2a57cb700 1 -- 172.21.15.161:0/1099376179 >> 172.21.15.161:0/1099376179 conn(0x7ff2a00fb660 msgr2=0x7ff2a00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.879+0000 7ff2a57cb700 1 -- 172.21.15.161:0/1099376179 shutdown_connections 2023-07-14T11:19:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.879+0000 7ff2a57cb700 1 -- 172.21.15.161:0/1099376179 wait complete. 2023-07-14T11:19:51.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 Processor -- start 2023-07-14T11:19:51.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 -- start start 2023-07-14T11:19:51.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a00ffe00 0x7ff2a0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2a01007a0 0x7ff2a0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a0104ea0 0x7ff2a019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019cea0 con 0x7ff2a0104ea0 2023-07-14T11:19:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019d010 con 0x7ff2a00ffe00 2023-07-14T11:19:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff2a57cb700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019d1d0 con 0x7ff2a01007a0 2023-07-14T11:19:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff29f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a0104ea0 0x7ff2a019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.880+0000 7ff29e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2a01007a0 0x7ff2a0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a0104ea0 0x7ff2a019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58646/0 (socket says 172.21.15.161:58646) 2023-07-14T11:19:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a00ffe00 0x7ff2a0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 -- 172.21.15.161:0/410672318 learned_addr learned my addr 172.21.15.161:0/410672318 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 -- 172.21.15.161:0/410672318 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2a01007a0 msgr2=0x7ff2a0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2a01007a0 0x7ff2a0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 -- 172.21.15.161:0/410672318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a00ffe00 msgr2=0x7ff2a0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a00ffe00 0x7ff2a0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 -- 172.21.15.161:0/410672318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940095f0 con 0x7ff2a0104ea0 2023-07-14T11:19:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff29f7fe700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a0104ea0 0x7ff2a019c890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff298007ee0 tx=0x7ff29800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff287fff700 1 -- 172.21.15.161:0/410672318 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298017070 con 0x7ff2a0104ea0 2023-07-14T11:19:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff287fff700 1 -- 172.21.15.161:0/410672318 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff29800f040 con 0x7ff2a0104ea0 2023-07-14T11:19:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a019d4b0 con 0x7ff2a0104ea0 2023-07-14T11:19:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.881+0000 7ff287fff700 1 -- 172.21.15.161:0/410672318 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298005580 con 0x7ff2a0104ea0 2023-07-14T11:19:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.882+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a01a4d10 con 0x7ff2a0104ea0 2023-07-14T11:19:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.883+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a004e210 con 0x7ff2a0104ea0 2023-07-14T11:19:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.884+0000 7ff287fff700 1 -- 172.21.15.161:0/410672318 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff29800db20 con 0x7ff2a0104ea0 2023-07-14T11:19:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.887+0000 7ff287fff700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2880a11d0 0x7ff2880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.887+0000 7ff287fff700 1 -- 172.21.15.161:0/410672318 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff2980cc1d0 con 0x7ff2a0104ea0 2023-07-14T11:19:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.887+0000 7ff29effd700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2880a11d0 0x7ff2880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:51.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.887+0000 7ff29effd700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2880a11d0 0x7ff2880a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff290009c40 tx=0x7ff290009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:51.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:51.891+0000 7ff287fff700 1 -- 172.21.15.161:0/410672318 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff2980956d0 con 0x7ff2a0104ea0 2023-07-14T11:19:52.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.138+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2a01a4ea0 con 0x7ff2a0104ea0 2023-07-14T11:19:52.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.138+0000 7ff287fff700 1 -- 172.21.15.161:0/410672318 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff298098cf0 con 0x7ff2a0104ea0 2023-07-14T11:19:52.141 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:19:52.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2880a11d0 msgr2=0x7ff2880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:52.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2880a11d0 0x7ff2880a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff290009c40 tx=0x7ff290009260 comp rx=0 tx=0).stop 2023-07-14T11:19:52.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a0104ea0 msgr2=0x7ff2a019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:52.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a0104ea0 0x7ff2a019c890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff298007ee0 tx=0x7ff29800d3b0 comp rx=0 tx=0).stop 2023-07-14T11:19:52.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 shutdown_connections 2023-07-14T11:19:52.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2a0104ea0 0x7ff2a019c890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:52.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff2880a11d0 0x7ff2880a3680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:52.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a00ffe00 0x7ff2a0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:52.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 --2- 172.21.15.161:0/410672318 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff2a01007a0 0x7ff2a0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:52.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 >> 172.21.15.161:0/410672318 conn(0x7ff2a00fb660 msgr2=0x7ff2a00fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:52.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 shutdown_connections 2023-07-14T11:19:52.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:52.141+0000 7ff2a57cb700 1 -- 172.21.15.161:0/410672318 wait complete. 2023-07-14T11:19:52.537 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-14T11:19:52.538 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:19:52.538 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-07-14T11:19:52.538 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-14T11:19:52.538 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:19:52.539 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-14T11:19:52.539 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:19:52.539 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:19:52.539 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T11:19:52.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:52.540 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-07-14T11:19:52.587 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:19:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:19:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:19:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:19:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-07-14T11:19:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:19:52.589 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:19:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:19:52.590 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-07-14T11:19:52.590 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:19:52.590 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:19:52.590 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-07-14T11:19:52.591 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:19:52.591 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:19:52.591 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:19:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:19:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-14T11:19:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 41373sec preferred_lft 41373sec 2023-07-14T11:19:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::7941:741e:f751:64dd/64 scope link noprefixroute 2023-07-14T11:19:52.593 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:19:52.594 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T11:19:52.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:52.596 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:19:52.596 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:19:52.597 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add name ceph-brx type bridge 2023-07-14T11:19:52.597 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr flush dev ceph-brx 2023-07-14T11:19:52.598 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set ceph-brx up 2023-07-14T11:19:52.598 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T11:19:52.599 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:19:53.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:53.028 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:19:53.099 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-07-14T11:19:53.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:53.103 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-07-14T11:19:53.158 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-07-14T11:19:53.159 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:19:53.159 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:19:53.159 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-14T11:19:53.159 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:19:53.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:53.161 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:19:53.161 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:19:53.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:19:53.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:19:53.162 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:19:53.162 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:19:53.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:53 smithi161 ceph-mon[95526]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2023-07-14T11:19:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:53 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/410672318' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:19:53.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:53 smithi175 ceph-mon[129081]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2023-07-14T11:19:53.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:53 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/410672318' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:19:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:53 smithi182 ceph-mon[129366]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2023-07-14T11:19:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:53 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/410672318' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:19:53.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:53.871 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-07-14T11:19:53.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:53.917 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2023-07-14T11:19:54.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:54.000 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:19:54.001 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:19:54.001 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-14T11:19:54.001 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-14T11:19:54.001 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:19:54.187 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-14T11:19:54.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:54.188 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:19:54.188 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:19:54.189 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-14T11:19:54.189 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T11:19:54.189 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-14T11:19:54.189 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-14T11:19:54.190 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-14T11:19:54.190 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:19:54.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:54.463 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:19:54.463 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:19:54.464 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-07-14T11:19:54.464 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T11:19:54.464 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:19:54.639 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-14T11:19:54.639 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:19:54.640 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:19:54.683 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T11:19:54.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:19:54.684 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:19:54.766 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2023-07-14T11:19:54.876 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:19:54.876 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:19:54.944 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:19:54.946 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:19:54.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:54.947 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2023-07-14T11:19:54.991 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:19:54.991 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-14T11:19:55.035 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2023-07-14T11:19:55.106 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:19:55.107 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:19:55.142 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:55 smithi161 ceph-mon[95526]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 B/s wr, 0 op/s 2023-07-14T11:19:55.178 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:19:55.181 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:19:55.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:55.182 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2023-07-14T11:19:55.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi161.stderr:2023-07-14T11:19:55.240+0000 7f6ed1c415c0 -1 init, newargv = 0x5560df401d80 newargc=15 2023-07-14T11:19:55.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi161.stderr:ceph-fuse[132903]: starting ceph client 2023-07-14T11:19:55.252 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi161.stderr:ceph-fuse[132903]: starting fuse 2023-07-14T11:19:55.271 INFO:teuthology.orchestra.run.smithi161.stdout:121 2023-07-14T11:19:55.271 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-07-14T11:19:55.272 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:19:55.272 DEBUG:teuthology.orchestra.run.smithi161:> import glob 2023-07-14T11:19:55.272 DEBUG:teuthology.orchestra.run.smithi161:> import re 2023-07-14T11:19:55.273 DEBUG:teuthology.orchestra.run.smithi161:> import os 2023-07-14T11:19:55.273 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2023-07-14T11:19:55.273 DEBUG:teuthology.orchestra.run.smithi161:> 2023-07-14T11:19:55.274 DEBUG:teuthology.orchestra.run.smithi161:> def _find_admin_socket(client_name): 2023-07-14T11:19:55.274 DEBUG:teuthology.orchestra.run.smithi161:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-14T11:19:55.274 DEBUG:teuthology.orchestra.run.smithi161:> files = glob.glob(asok_path) 2023-07-14T11:19:55.274 DEBUG:teuthology.orchestra.run.smithi161:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-14T11:19:55.275 DEBUG:teuthology.orchestra.run.smithi161:> 2023-07-14T11:19:55.275 DEBUG:teuthology.orchestra.run.smithi161:> # Given a non-glob path, it better be there 2023-07-14T11:19:55.275 DEBUG:teuthology.orchestra.run.smithi161:> if "*" not in asok_path: 2023-07-14T11:19:55.275 DEBUG:teuthology.orchestra.run.smithi161:> assert(len(files) == 1) 2023-07-14T11:19:55.276 DEBUG:teuthology.orchestra.run.smithi161:> return files[0] 2023-07-14T11:19:55.276 DEBUG:teuthology.orchestra.run.smithi161:> 2023-07-14T11:19:55.276 DEBUG:teuthology.orchestra.run.smithi161:> for f in files: 2023-07-14T11:19:55.276 DEBUG:teuthology.orchestra.run.smithi161:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:19:55.277 DEBUG:teuthology.orchestra.run.smithi161:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:19:55.277 DEBUG:teuthology.orchestra.run.smithi161:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:19:55.277 DEBUG:teuthology.orchestra.run.smithi161:> contents = proc_f.read() 2023-07-14T11:19:55.278 DEBUG:teuthology.orchestra.run.smithi161:> if mountpoint in contents: 2023-07-14T11:19:55.278 DEBUG:teuthology.orchestra.run.smithi161:> return f 2023-07-14T11:19:55.278 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:19:55.279 DEBUG:teuthology.orchestra.run.smithi161:> 2023-07-14T11:19:55.279 DEBUG:teuthology.orchestra.run.smithi161:> print(_find_admin_socket("client.admin")) 2023-07-14T11:19:55.279 DEBUG:teuthology.orchestra.run.smithi161:> ' 2023-07-14T11:19:55.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:55 smithi175 ceph-mon[129081]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 B/s wr, 0 op/s 2023-07-14T11:19:55.455 INFO:teuthology.orchestra.run.smithi161.stdout:/var/run/ceph/ceph-client.admin.132903.asok 2023-07-14T11:19:55.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:55 smithi182 ceph-mon[129366]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 B/s wr, 0 op/s 2023-07-14T11:19:55.633 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.132903.asok 2023-07-14T11:19:55.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:55.634 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.132903.asok status 2023-07-14T11:19:55.858 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:19:55.859 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": { 2023-07-14T11:19:55.859 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:19:55.859 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:19:55.859 INFO:teuthology.orchestra.run.smithi161.stdout: "entity_id": "admin", 2023-07-14T11:19:55.859 INFO:teuthology.orchestra.run.smithi161.stdout: "hostname": "smithi161", 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: "pid": "132903", 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: "root": "/" 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_count": 0, 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_pinned_count": 0, 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24703, 2023-07-14T11:19:55.860 INFO:teuthology.orchestra.run.smithi161.stdout: "inst": { 2023-07-14T11:19:55.861 INFO:teuthology.orchestra.run.smithi161.stdout: "name": { 2023-07-14T11:19:55.861 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "client", 2023-07-14T11:19:55.861 INFO:teuthology.orchestra.run.smithi161.stdout: "num": 24703 2023-07-14T11:19:55.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:19:55.861 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2023-07-14T11:19:55.861 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2023-07-14T11:19:55.861 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 2213890863 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 2213890863 2023-07-14T11:19:55.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "inst_str": "client.24703 172.21.15.161:0/2213890863", 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "addr_str": "172.21.15.161:0/2213890863", 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "inode_count": 1, 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_epoch": 16, 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch": 88, 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklisted": false, 2023-07-14T11:19:55.863 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs" 2023-07-14T11:19:55.864 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:19:55.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:55.873 DEBUG:teuthology.orchestra.run.smithi161:> stat --file-system '--printf=%T 2023-07-14T11:19:55.874 DEBUG:teuthology.orchestra.run.smithi161:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:19:55.919 INFO:teuthology.orchestra.run.smithi161.stdout:fuseblk 2023-07-14T11:19:55.919 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:19:55.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:55.921 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:19:56.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:19:56.031 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-14T11:19:56.156 INFO:teuthology.orchestra.run.smithi161.stdout:# file: . 2023-07-14T11:19:56.156 INFO:teuthology.orchestra.run.smithi161.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-14T11:19:56.156 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:56.157 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:19:56.157 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:19:56.202 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:19:56.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 -- 172.21.15.161:0/992789304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0200ffb00 msgr2=0x7ff0200fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 --2- 172.21.15.161:0/992789304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0200ffb00 0x7ff0200fff10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff014009990 tx=0x7ff014009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 -- 172.21.15.161:0/992789304 shutdown_connections 2023-07-14T11:19:56.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 --2- 172.21.15.161:0/992789304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0200ffb00 0x7ff0200fff10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 --2- 172.21.15.161:0/992789304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff020101680 0x7ff020107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 --2- 172.21.15.161:0/992789304 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff020100cf0 0x7ff020101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 -- 172.21.15.161:0/992789304 >> 172.21.15.161:0/992789304 conn(0x7ff0200fb0e0 msgr2=0x7ff0200fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:56.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 -- 172.21.15.161:0/992789304 shutdown_connections 2023-07-14T11:19:56.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.437+0000 7ff026bff700 1 -- 172.21.15.161:0/992789304 wait complete. 2023-07-14T11:19:56.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.438+0000 7ff026bff700 1 Processor -- start 2023-07-14T11:19:56.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.438+0000 7ff026bff700 1 -- start start 2023-07-14T11:19:56.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff026bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0200ffb00 0x7ff0200742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff026bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020100cf0 0x7ff020072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff026bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff020101680 0x7ff020072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff026bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200735e0 con 0x7ff020100cf0 2023-07-14T11:19:56.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff026bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff020073750 con 0x7ff0200ffb00 2023-07-14T11:19:56.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff026bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02019f8e0 con 0x7ff020101680 2023-07-14T11:19:56.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020100cf0 0x7ff020072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020100cf0 0x7ff020072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58666/0 (socket says 172.21.15.161:58666) 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 -- 172.21.15.161:0/2930562637 learned_addr learned my addr 172.21.15.161:0/2930562637 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff02499b700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0200ffb00 0x7ff0200742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff02519c700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff020101680 0x7ff020072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 -- 172.21.15.161:0/2930562637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff020101680 msgr2=0x7ff020072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff020101680 0x7ff020072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 -- 172.21.15.161:0/2930562637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0200ffb00 msgr2=0x7ff0200742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0200ffb00 0x7ff0200742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff01ffff700 1 -- 172.21.15.161:0/2930562637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140095f0 con 0x7ff020100cf0 2023-07-14T11:19:56.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.439+0000 7ff02519c700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff020101680 0x7ff020072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:56.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.440+0000 7ff01ffff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020100cf0 0x7ff020072900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00e9a0 tx=0x7ff00c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:56.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.440+0000 7ff01dffb700 1 -- 172.21.15.161:0/2930562637 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c009790 con 0x7ff020100cf0 2023-07-14T11:19:56.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.440+0000 7ff01dffb700 1 -- 172.21.15.161:0/2930562637 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00c00cbd0 con 0x7ff020100cf0 2023-07-14T11:19:56.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.440+0000 7ff01dffb700 1 -- 172.21.15.161:0/2930562637 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c018680 con 0x7ff020100cf0 2023-07-14T11:19:56.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.440+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02019fbc0 con 0x7ff020100cf0 2023-07-14T11:19:56.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.440+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0201a0080 con 0x7ff020100cf0 2023-07-14T11:19:56.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.441+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0201070a0 con 0x7ff020100cf0 2023-07-14T11:19:56.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.443+0000 7ff01dffb700 1 -- 172.21.15.161:0/2930562637 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff00c00cd40 con 0x7ff020100cf0 2023-07-14T11:19:56.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.444+0000 7ff01dffb700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff0100a1220 0x7ff0100a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.445+0000 7ff01dffb700 1 -- 172.21.15.161:0/2930562637 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff00c014070 con 0x7ff020100cf0 2023-07-14T11:19:56.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.445+0000 7ff02499b700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff0100a1220 0x7ff0100a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.445+0000 7ff02499b700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff0100a1220 0x7ff0100a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff01400b2e0 tx=0x7ff014005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:56.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.448+0000 7ff01dffb700 1 -- 172.21.15.161:0/2930562637 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff00c051790 con 0x7ff020100cf0 2023-07-14T11:19:56.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.651+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff02002c8a0 con 0x7ff020100cf0 2023-07-14T11:19:56.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.653+0000 7ff01dffb700 1 -- 172.21.15.161:0/2930562637 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23993 (secure 0 0 0) 0x7ff00c078e60 con 0x7ff020100cf0 2023-07-14T11:19:56.655 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:56.655 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:19:46.582122+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff0100a1220 msgr2=0x7ff0100a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff0100a1220 0x7ff0100a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff01400b2e0 tx=0x7ff014005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020100cf0 msgr2=0x7ff020072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020100cf0 0x7ff020072900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00e9a0 tx=0x7ff00c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 shutdown_connections 2023-07-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff0100a1220 0x7ff0100a36d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff020101680 0x7ff020072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0200ffb00 0x7ff0200742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 --2- 172.21.15.161:0/2930562637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020100cf0 0x7ff020072900 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.655+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 >> 172.21.15.161:0/2930562637 conn(0x7ff0200fb0e0 msgr2=0x7ff0201041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:56.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.656+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 shutdown_connections 2023-07-14T11:19:56.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.656+0000 7ff026bff700 1 -- 172.21.15.161:0/2930562637 wait complete. 2023-07-14T11:19:56.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:19:56.677 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:19:56.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 -- 172.21.15.161:0/379632836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598100cd0 msgr2=0x7f9598101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 --2- 172.21.15.161:0/379632836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598100cd0 0x7f9598101120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9588009990 tx=0x7f9588009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 -- 172.21.15.161:0/379632836 shutdown_connections 2023-07-14T11:19:56.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 --2- 172.21.15.161:0/379632836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9598101660 0x7f9598107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 --2- 172.21.15.161:0/379632836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95980ffae0 0x7f95980ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 --2- 172.21.15.161:0/379632836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598100cd0 0x7f9598101120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 -- 172.21.15.161:0/379632836 >> 172.21.15.161:0/379632836 conn(0x7f95980fb0e0 msgr2=0x7f95980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:56.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 -- 172.21.15.161:0/379632836 shutdown_connections 2023-07-14T11:19:56.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.877+0000 7f95a01ff700 1 -- 172.21.15.161:0/379632836 wait complete. 2023-07-14T11:19:56.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.878+0000 7f95a01ff700 1 Processor -- start 2023-07-14T11:19:56.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.878+0000 7f95a01ff700 1 -- start start 2023-07-14T11:19:56.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f95a01ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95980ffae0 0x7f9598195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f95a01ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9598100cd0 0x7f9598195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f95a01ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598101660 0x7f959819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f95a01ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959819d400 con 0x7f9598101660 2023-07-14T11:19:56.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f95a01ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959819d540 con 0x7f95980ffae0 2023-07-14T11:19:56.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f95a01ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959819d6d0 con 0x7f9598100cd0 2023-07-14T11:19:56.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598101660 0x7f959819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598101660 0x7f959819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58682/0 (socket says 172.21.15.161:58682) 2023-07-14T11:19:56.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 -- 172.21.15.161:0/1301351230 learned_addr learned my addr 172.21.15.161:0/1301351230 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:56.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959df9b700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95980ffae0 0x7f9598195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959d79a700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9598100cd0 0x7f9598195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 -- 172.21.15.161:0/1301351230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9598100cd0 msgr2=0x7f9598195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9598100cd0 0x7f9598195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 -- 172.21.15.161:0/1301351230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95980ffae0 msgr2=0x7f9598195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95980ffae0 0x7f9598195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.879+0000 7f959e79c700 1 -- 172.21.15.161:0/1301351230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9594009520 con 0x7f9598101660 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.880+0000 7f959e79c700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598101660 0x7f959819ce30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f959000ef30 tx=0x7f959000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.880+0000 7f958effd700 1 -- 172.21.15.161:0/1301351230 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590018070 con 0x7f9598101660 2023-07-14T11:19:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.880+0000 7f958effd700 1 -- 172.21.15.161:0/1301351230 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f959000f040 con 0x7f9598101660 2023-07-14T11:19:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.880+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95880095f0 con 0x7f9598101660 2023-07-14T11:19:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.880+0000 7f958effd700 1 -- 172.21.15.161:0/1301351230 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959000cd00 con 0x7f9598101660 2023-07-14T11:19:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.880+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959819dc80 con 0x7f9598101660 2023-07-14T11:19:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.881+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959810b300 con 0x7f9598101660 2023-07-14T11:19:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.883+0000 7f958effd700 1 -- 172.21.15.161:0/1301351230 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9590003a00 con 0x7f9598101660 2023-07-14T11:19:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.885+0000 7f958effd700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f95840a1150 0x7f95840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:56.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.886+0000 7f958effd700 1 -- 172.21.15.161:0/1301351230 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f95900cc6b0 con 0x7f9598101660 2023-07-14T11:19:56.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.886+0000 7f959df9b700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f95840a1150 0x7f95840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:56.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.886+0000 7f959df9b700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f95840a1150 0x7f95840a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9598100b30 tx=0x7f9594009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:56.889+0000 7f958effd700 1 -- 172.21.15.161:0/1301351230 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9590095c40 con 0x7f9598101660 2023-07-14T11:19:57.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.087+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95981069e0 con 0x7f9598101660 2023-07-14T11:19:57.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.087+0000 7f958effd700 1 -- 172.21.15.161:0/1301351230 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9590099260 con 0x7f9598101660 2023-07-14T11:19:57.090 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:19:57.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f95840a1150 msgr2=0x7f95840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f95840a1150 0x7f95840a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9598100b30 tx=0x7f9594009260 comp rx=0 tx=0).stop 2023-07-14T11:19:57.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598101660 msgr2=0x7f959819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598101660 0x7f959819ce30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f959000ef30 tx=0x7f959000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 shutdown_connections 2023-07-14T11:19:57.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9598101660 0x7f959819ce30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f95840a1150 0x7f95840a3600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95980ffae0 0x7f9598195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 --2- 172.21.15.161:0/1301351230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9598100cd0 0x7f9598195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 >> 172.21.15.161:0/1301351230 conn(0x7f95980fb0e0 msgr2=0x7f95980fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:57.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 shutdown_connections 2023-07-14T11:19:57.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.090+0000 7f95a01ff700 1 -- 172.21.15.161:0/1301351230 wait complete. 2023-07-14T11:19:57.111 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:19:57.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 -- 172.21.15.161:0/1625301599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c10c0b0 msgr2=0x7f6c1c10e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 --2- 172.21.15.161:0/1625301599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c10c0b0 0x7f6c1c10e490 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b0a0 tx=0x7f6c1400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 -- 172.21.15.161:0/1625301599 shutdown_connections 2023-07-14T11:19:57.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 --2- 172.21.15.161:0/1625301599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c10c0b0 0x7f6c1c10e490 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 --2- 172.21.15.161:0/1625301599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c074550 0x7f6c1c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 --2- 172.21.15.161:0/1625301599 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c1c072f70 0x7f6c1c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 -- 172.21.15.161:0/1625301599 >> 172.21.15.161:0/1625301599 conn(0x7f6c1c0fb0e0 msgr2=0x7f6c1c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:57.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 -- 172.21.15.161:0/1625301599 shutdown_connections 2023-07-14T11:19:57.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.310+0000 7f6c21bff700 1 -- 172.21.15.161:0/1625301599 wait complete. 2023-07-14T11:19:57.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.311+0000 7f6c21bff700 1 Processor -- start 2023-07-14T11:19:57.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.311+0000 7f6c21bff700 1 -- start start 2023-07-14T11:19:57.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.311+0000 7f6c21bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c072f70 0x7f6c1c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.311+0000 7f6c21bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c1c074550 0x7f6c1c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.311+0000 7f6c21bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c10c0b0 0x7f6c1c1a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c21bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1a3b00 con 0x7f6c1c072f70 2023-07-14T11:19:57.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c21bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1a3c40 con 0x7f6c1c10c0b0 2023-07-14T11:19:57.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c21bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1a3dd0 con 0x7f6c1c074550 2023-07-14T11:19:57.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c072f70 0x7f6c1c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c072f70 0x7f6c1c19bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58698/0 (socket says 172.21.15.161:58698) 2023-07-14T11:19:57.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 -- 172.21.15.161:0/3861225390 learned_addr learned my addr 172.21.15.161:0/3861225390 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:57.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c13fff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c1c074550 0x7f6c1c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1bfff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c10c0b0 0x7f6c1c1a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 -- 172.21.15.161:0/3861225390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c1c074550 msgr2=0x7f6c1c19c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c1c074550 0x7f6c1c19c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 -- 172.21.15.161:0/3861225390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c10c0b0 msgr2=0x7f6c1c1a3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c10c0b0 0x7f6c1c1a3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 -- 172.21.15.161:0/3861225390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c009520 con 0x7f6c1c072f70 2023-07-14T11:19:57.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.312+0000 7f6c1b7fe700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c072f70 0x7f6c1c19bf80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00ea30 tx=0x7f6c0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:57.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.313+0000 7f6c197fa700 1 -- 172.21.15.161:0/3861225390 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c009820 con 0x7f6c1c072f70 2023-07-14T11:19:57.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.313+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c14009d00 con 0x7f6c1c072f70 2023-07-14T11:19:57.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.313+0000 7f6c197fa700 1 -- 172.21.15.161:0/3861225390 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c0c0044e0 con 0x7f6c1c072f70 2023-07-14T11:19:57.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.313+0000 7f6c197fa700 1 -- 172.21.15.161:0/3861225390 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c003e80 con 0x7f6c1c072f70 2023-07-14T11:19:57.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.313+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c1a4430 con 0x7f6c1c072f70 2023-07-14T11:19:57.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.314+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c101c00 con 0x7f6c1c072f70 2023-07-14T11:19:57.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.316+0000 7f6c197fa700 1 -- 172.21.15.161:0/3861225390 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6c0c004000 con 0x7f6c1c072f70 2023-07-14T11:19:57.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.317+0000 7f6c197fa700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6c080a1150 0x7f6c080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.317+0000 7f6c197fa700 1 -- 172.21.15.161:0/3861225390 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6c0c014070 con 0x7f6c1c072f70 2023-07-14T11:19:57.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.317+0000 7f6c13fff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6c080a1150 0x7f6c080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.318+0000 7f6c13fff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6c080a1150 0x7f6c080a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c074270 tx=0x7f6c04009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:57.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.321+0000 7f6c197fa700 1 -- 172.21.15.161:0/3861225390 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c0c0965c0 con 0x7f6c1c072f70 2023-07-14T11:19:57.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:57 smithi175 ceph-mon[129081]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 0 op/s 2023-07-14T11:19:57.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:57 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2930562637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:57.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:57 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1301351230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:57 smithi182 ceph-mon[129366]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 0 op/s 2023-07-14T11:19:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:57 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2930562637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:57 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1301351230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:57.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:57 smithi161 ceph-mon[95526]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 0 op/s 2023-07-14T11:19:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:57 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2930562637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:19:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:57 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1301351230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:19:57.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.518+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c1c061150 con 0x7f6c1c072f70 2023-07-14T11:19:57.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.519+0000 7f6c197fa700 1 -- 172.21.15.161:0/3861225390 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c0c099be0 con 0x7f6c1c072f70 2023-07-14T11:19:57.521 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:19:57.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.521+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6c080a1150 msgr2=0x7f6c080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.521+0000 7f6c21bff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6c080a1150 0x7f6c080a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c074270 tx=0x7f6c04009260 comp rx=0 tx=0).stop 2023-07-14T11:19:57.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.521+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c072f70 msgr2=0x7f6c1c19bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.521+0000 7f6c21bff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c072f70 0x7f6c1c19bf80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00ea30 tx=0x7f6c0c00edf0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 shutdown_connections 2023-07-14T11:19:57.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c072f70 0x7f6c1c19bf80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6c080a1150 0x7f6c080a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c1c10c0b0 0x7f6c1c1a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 --2- 172.21.15.161:0/3861225390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c1c074550 0x7f6c1c19c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 >> 172.21.15.161:0/3861225390 conn(0x7f6c1c0fb0e0 msgr2=0x7f6c1c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:57.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 shutdown_connections 2023-07-14T11:19:57.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.522+0000 7f6c21bff700 1 -- 172.21.15.161:0/3861225390 wait complete. 2023-07-14T11:19:57.542 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:19:57.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 -- 172.21.15.161:0/1073679807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc100ce0 msgr2=0x7f28bc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 --2- 172.21.15.161:0/1073679807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc100ce0 0x7f28bc101130 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009990 tx=0x7f28ac009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 -- 172.21.15.161:0/1073679807 shutdown_connections 2023-07-14T11:19:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 --2- 172.21.15.161:0/1073679807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc101670 0x7f28bc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 --2- 172.21.15.161:0/1073679807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28bc0ffaf0 0x7f28bc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 --2- 172.21.15.161:0/1073679807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc100ce0 0x7f28bc101130 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 -- 172.21.15.161:0/1073679807 >> 172.21.15.161:0/1073679807 conn(0x7f28bc0fb130 msgr2=0x7f28bc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 -- 172.21.15.161:0/1073679807 shutdown_connections 2023-07-14T11:19:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.741+0000 7f28c43ff700 1 -- 172.21.15.161:0/1073679807 wait complete. 2023-07-14T11:19:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.742+0000 7f28c43ff700 1 Processor -- start 2023-07-14T11:19:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.742+0000 7f28c43ff700 1 -- start start 2023-07-14T11:19:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c43ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc0ffaf0 0x7f28bc195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c43ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28bc100ce0 0x7f28bc195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c43ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc101670 0x7f28bc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c43ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc19d3c0 con 0x7f28bc101670 2023-07-14T11:19:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c43ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc19d500 con 0x7f28bc100ce0 2023-07-14T11:19:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c43ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc19d690 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c299c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc101670 0x7f28bc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c219b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc0ffaf0 0x7f28bc195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c199a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28bc100ce0 0x7f28bc195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c219b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc0ffaf0 0x7f28bc195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47014/0 (socket says 172.21.15.161:47014) 2023-07-14T11:19:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.743+0000 7f28c219b700 1 -- 172.21.15.161:0/2557390065 learned_addr learned my addr 172.21.15.161:0/2557390065 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:57.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c219b700 1 -- 172.21.15.161:0/2557390065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28bc100ce0 msgr2=0x7f28bc195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c219b700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28bc100ce0 0x7f28bc195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c219b700 1 -- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc101670 msgr2=0x7f28bc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c219b700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc101670 0x7f28bc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c219b700 1 -- 172.21.15.161:0/2557390065 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8009520 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c199a700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28bc100ce0 0x7f28bc195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c299c700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc101670 0x7f28bc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:19:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c219b700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc0ffaf0 0x7f28bc195840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28b800ea50 tx=0x7f28b800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28b37fe700 1 -- 172.21.15.161:0/2557390065 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800cbd0 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac0095f0 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28b37fe700 1 -- 172.21.15.161:0/2557390065 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b800cd30 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28b37fe700 1 -- 172.21.15.161:0/2557390065 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8018680 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.744+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc19dcf0 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.745+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc10b310 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.747+0000 7f28b37fe700 1 -- 172.21.15.161:0/2557390065 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f28b8010430 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.749+0000 7f28b37fe700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f28a80a1150 0x7f28a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:57.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.749+0000 7f28b37fe700 1 -- 172.21.15.161:0/2557390065 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f28b8014070 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.749+0000 7f28c199a700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f28a80a1150 0x7f28a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:57.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.750+0000 7f28c199a700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f28a80a1150 0x7f28a80a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b2e0 tx=0x7f28ac005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:57.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.751+0000 7f28b37fe700 1 -- 172.21.15.161:0/2557390065 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f28b80968b0 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.948+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28bc1069f0 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.949+0000 7f28b37fe700 1 -- 172.21.15.161:0/2557390065 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28b8099ed0 con 0x7f28bc0ffaf0 2023-07-14T11:19:57.951 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:19:57.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f28a80a1150 msgr2=0x7f28a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f28a80a1150 0x7f28a80a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b2e0 tx=0x7f28ac005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc0ffaf0 msgr2=0x7f28bc195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc0ffaf0 0x7f28bc195840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28b800ea50 tx=0x7f28b800ed60 comp rx=0 tx=0).stop 2023-07-14T11:19:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 shutdown_connections 2023-07-14T11:19:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28bc101670 0x7f28bc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f28a80a1150 0x7f28a80a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f28bc0ffaf0 0x7f28bc195840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 --2- 172.21.15.161:0/2557390065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28bc100ce0 0x7f28bc195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:57.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 >> 172.21.15.161:0/2557390065 conn(0x7f28bc0fb130 msgr2=0x7f28bc0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:57.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 shutdown_connections 2023-07-14T11:19:57.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:57.951+0000 7f28c43ff700 1 -- 172.21.15.161:0/2557390065 wait complete. 2023-07-14T11:19:57.972 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:19:58.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/1097409323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 msgr2=0x7fd1e4101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/1097409323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 0x7fd1e4101130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009990 tx=0x7fd1d0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/1097409323 shutdown_connections 2023-07-14T11:19:58.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/1097409323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e4101670 0x7fd1e4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/1097409323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e40ffaf0 0x7fd1e40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/1097409323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 0x7fd1e4101130 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/1097409323 >> 172.21.15.161:0/1097409323 conn(0x7fd1e40fb130 msgr2=0x7fd1e40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:58.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/1097409323 shutdown_connections 2023-07-14T11:19:58.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.177+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/1097409323 wait complete. 2023-07-14T11:19:58.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.178+0000 7fd1eb1ff700 1 Processor -- start 2023-07-14T11:19:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.178+0000 7fd1eb1ff700 1 -- start start 2023-07-14T11:19:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1eb1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e40ffaf0 0x7fd1e41957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1eb1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 0x7fd1e4195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1eb1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4101670 0x7fd1e419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1eb1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419d360 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1eb1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419d4a0 con 0x7fd1e4101670 2023-07-14T11:19:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1eb1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419d630 con 0x7fd1e4100ce0 2023-07-14T11:19:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e40ffaf0 0x7fd1e41957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 0x7fd1e4195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e40ffaf0 0x7fd1e41957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51668/0 (socket says 172.21.15.161:51668) 2023-07-14T11:19:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 -- 172.21.15.161:0/2759087829 learned_addr learned my addr 172.21.15.161:0/2759087829 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e3fff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 0x7fd1e4195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33188/0 (socket says 172.21.15.161:33188) 2023-07-14T11:19:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e979c700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4101670 0x7fd1e419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 -- 172.21.15.161:0/2759087829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 msgr2=0x7fd1e4195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 0x7fd1e4195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 -- 172.21.15.161:0/2759087829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4101670 msgr2=0x7fd1e419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4101670 0x7fd1e419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.179+0000 7fd1e8f9b700 1 -- 172.21.15.161:0/2759087829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00095f0 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.180+0000 7fd1e8f9b700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e40ffaf0 0x7fd1e41957e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800e970 tx=0x7fd1d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.180+0000 7fd1e1ffb700 1 -- 172.21.15.161:0/2759087829 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8009790 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.180+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e419d910 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.180+0000 7fd1e1ffb700 1 -- 172.21.15.161:0/2759087829 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d800c7e0 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.180+0000 7fd1e1ffb700 1 -- 172.21.15.161:0/2759087829 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003c70 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.180+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e419de20 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.181+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e410b2b0 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.183+0000 7fd1e1ffb700 1 -- 172.21.15.161:0/2759087829 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd1d800cc50 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.185+0000 7fd1e1ffb700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1d40a1150 0x7fd1d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.185+0000 7fd1e1ffb700 1 -- 172.21.15.161:0/2759087829 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd1d8014070 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.185+0000 7fd1e3fff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1d40a1150 0x7fd1d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.186+0000 7fd1e3fff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1d40a1150 0x7fd1d40a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b2e0 tx=0x7fd1d0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:58.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.189+0000 7fd1e1ffb700 1 -- 172.21.15.161:0/2759087829 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd1d80961f0 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:58 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3861225390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:19:58.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:58 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2557390065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:19:58.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.388+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd1e41069a0 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.388+0000 7fd1e1ffb700 1 -- 172.21.15.161:0/2759087829 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd1d8003dd0 con 0x7fd1e40ffaf0 2023-07-14T11:19:58.391 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:19:58.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1d40a1150 msgr2=0x7fd1d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1d40a1150 0x7fd1d40a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b2e0 tx=0x7fd1d0005fb0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e40ffaf0 msgr2=0x7fd1e41957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e40ffaf0 0x7fd1e41957e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800e970 tx=0x7fd1d800ec80 comp rx=0 tx=0).stop 2023-07-14T11:19:58.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 shutdown_connections 2023-07-14T11:19:58.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e40ffaf0 0x7fd1e41957e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1d40a1150 0x7fd1d40a3600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4101670 0x7fd1e419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 --2- 172.21.15.161:0/2759087829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd1e4100ce0 0x7fd1e4195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 >> 172.21.15.161:0/2759087829 conn(0x7fd1e40fb130 msgr2=0x7fd1e40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:58.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 shutdown_connections 2023-07-14T11:19:58.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.391+0000 7fd1eb1ff700 1 -- 172.21.15.161:0/2759087829 wait complete. 2023-07-14T11:19:58.412 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:19:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:58 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3861225390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:19:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:58 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2557390065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:19:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:58 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3861225390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:19:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:58 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2557390065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:19:58.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 -- 172.21.15.161:0/3229478361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc101670 msgr2=0x7fcbdc107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/3229478361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc101670 0x7fcbdc107f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400b0a0 tx=0x7fcbd400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 -- 172.21.15.161:0/3229478361 shutdown_connections 2023-07-14T11:19:58.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/3229478361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc101670 0x7fcbdc107f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/3229478361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbdc0ffaf0 0x7fcbdc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/3229478361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbdc100ce0 0x7fcbdc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 -- 172.21.15.161:0/3229478361 >> 172.21.15.161:0/3229478361 conn(0x7fcbdc0fb130 msgr2=0x7fcbdc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:58.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 -- 172.21.15.161:0/3229478361 shutdown_connections 2023-07-14T11:19:58.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.608+0000 7fcbe0fff700 1 -- 172.21.15.161:0/3229478361 wait complete. 2023-07-14T11:19:58.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.609+0000 7fcbe0fff700 1 Processor -- start 2023-07-14T11:19:58.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.609+0000 7fcbe0fff700 1 -- start start 2023-07-14T11:19:58.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.609+0000 7fcbe0fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbdc0ffaf0 0x7fcbdc195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.609+0000 7fcbe0fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc100ce0 0x7fcbdc195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.609+0000 7fcbe0fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbdc101670 0x7fcbdc19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbe0fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc19d4b0 con 0x7fcbdc100ce0 2023-07-14T11:19:58.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbe0fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc19d5f0 con 0x7fcbdc101670 2023-07-14T11:19:58.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbe0fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc19d780 con 0x7fcbdc0ffaf0 2023-07-14T11:19:58.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc100ce0 0x7fcbdc195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc100ce0 0x7fcbdc195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51686/0 (socket says 172.21.15.161:51686) 2023-07-14T11:19:58.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbda59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbdc0ffaf0 0x7fcbdc195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 -- 172.21.15.161:0/421026145 learned_addr learned my addr 172.21.15.161:0/421026145 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:58.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbdad9d700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbdc101670 0x7fcbdc19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 -- 172.21.15.161:0/421026145 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbdc0ffaf0 msgr2=0x7fcbdc195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbdc0ffaf0 0x7fcbdc195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 -- 172.21.15.161:0/421026145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbdc101670 msgr2=0x7fcbdc19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbdc101670 0x7fcbdc19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 -- 172.21.15.161:0/421026145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4009d00 con 0x7fcbdc100ce0 2023-07-14T11:19:58.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbdad9d700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbdc101670 0x7fcbdc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:19:58.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.610+0000 7fcbd9d9b700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc100ce0 0x7fcbdc195e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000e9a0 tx=0x7fcbd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:58.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.611+0000 7fcbcb7fe700 1 -- 172.21.15.161:0/421026145 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0009790 con 0x7fcbdc100ce0 2023-07-14T11:19:58.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.611+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc19da60 con 0x7fcbdc100ce0 2023-07-14T11:19:58.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.611+0000 7fcbcb7fe700 1 -- 172.21.15.161:0/421026145 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbd000cbd0 con 0x7fcbdc100ce0 2023-07-14T11:19:58.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.611+0000 7fcbcb7fe700 1 -- 172.21.15.161:0/421026145 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0018680 con 0x7fcbdc100ce0 2023-07-14T11:19:58.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.611+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc19dfc0 con 0x7fcbdc100ce0 2023-07-14T11:19:58.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.612+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc10b320 con 0x7fcbdc100ce0 2023-07-14T11:19:58.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.614+0000 7fcbcb7fe700 1 -- 172.21.15.161:0/421026145 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcbd000cd40 con 0x7fcbdc100ce0 2023-07-14T11:19:58.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.615+0000 7fcbcb7fe700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcbc40a1150 0x7fcbc40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:58.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.615+0000 7fcbcb7fe700 1 -- 172.21.15.161:0/421026145 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcbd0014070 con 0x7fcbdc100ce0 2023-07-14T11:19:58.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.615+0000 7fcbda59c700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcbc40a1150 0x7fcbc40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:58.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.616+0000 7fcbda59c700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcbc40a1150 0x7fcbc40a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc100b40 tx=0x7fcbcc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:58.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.617+0000 7fcbcb7fe700 1 -- 172.21.15.161:0/421026145 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcbd0096590 con 0x7fcbdc100ce0 2023-07-14T11:19:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.846+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcbdc10b510 con 0x7fcbdc100ce0 2023-07-14T11:19:58.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.847+0000 7fcbcb7fe700 1 -- 172.21.15.161:0/421026145 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+599 (secure 0 0 0) 0x7fcbd0026270 con 0x7fcbdc100ce0 2023-07-14T11:19:58.850 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/871206111 2023-07-15T11:13:14.005108+0000 2023-07-14T11:19:58.850 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6801/3292405426 2023-07-15T11:13:14.005108+0000 2023-07-14T11:19:58.850 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6800/3292405426 2023-07-15T11:13:14.005108+0000 2023-07-14T11:19:58.850 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/629268496 2023-07-15T11:12:47.212143+0000 2023-07-14T11:19:58.851 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/832098793 2023-07-15T11:13:14.005108+0000 2023-07-14T11:19:58.851 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6800/4131434778 2023-07-15T11:12:47.212143+0000 2023-07-14T11:19:58.851 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6801/4131434778 2023-07-15T11:12:47.212143+0000 2023-07-14T11:19:58.851 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/3333551885 2023-07-15T11:12:47.212143+0000 2023-07-14T11:19:58.851 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/1691634661 2023-07-15T11:12:47.212143+0000 2023-07-14T11:19:58.852 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/3118869366 2023-07-15T11:13:14.005108+0000 2023-07-14T11:19:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcbc40a1150 msgr2=0x7fcbc40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcbc40a1150 0x7fcbc40a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc100b40 tx=0x7fcbcc009260 comp rx=0 tx=0).stop 2023-07-14T11:19:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc100ce0 msgr2=0x7fcbdc195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc100ce0 0x7fcbdc195e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000e9a0 tx=0x7fcbd000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 shutdown_connections 2023-07-14T11:19:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcbc40a1150 0x7fcbc40a3600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbdc101670 0x7fcbdc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbdc0ffaf0 0x7fcbdc195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 --2- 172.21.15.161:0/421026145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbdc100ce0 0x7fcbdc195e70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:58.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 >> 172.21.15.161:0/421026145 conn(0x7fcbdc0fb130 msgr2=0x7fcbdc0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:58.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 shutdown_connections 2023-07-14T11:19:58.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:58.849+0000 7fcbe0fff700 1 -- 172.21.15.161:0/421026145 wait complete. 2023-07-14T11:19:58.856 INFO:teuthology.orchestra.run.smithi161.stderr:listed 10 entries 2023-07-14T11:19:58.870 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:19:58.870 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:19:58.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:19:59.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.150+0000 7efca49ff700 1 -- 172.21.15.161:0/2134854326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 msgr2=0x7efca00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.150+0000 7efca49ff700 1 --2- 172.21.15.161:0/2134854326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca00fff10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc90009990 tx=0x7efc90009ca0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.151+0000 7efca49ff700 1 -- 172.21.15.161:0/2134854326 shutdown_connections 2023-07-14T11:19:59.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.151+0000 7efca49ff700 1 --2- 172.21.15.161:0/2134854326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca00fff10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.151+0000 7efca49ff700 1 --2- 172.21.15.161:0/2134854326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efca0101680 0x7efca0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.151+0000 7efca49ff700 1 --2- 172.21.15.161:0/2134854326 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efca0100cf0 0x7efca0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.151+0000 7efca49ff700 1 -- 172.21.15.161:0/2134854326 >> 172.21.15.161:0/2134854326 conn(0x7efca00fb0e0 msgr2=0x7efca00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:59.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.151+0000 7efca49ff700 1 -- 172.21.15.161:0/2134854326 shutdown_connections 2023-07-14T11:19:59.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.151+0000 7efca49ff700 1 -- 172.21.15.161:0/2134854326 wait complete. 2023-07-14T11:19:59.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 Processor -- start 2023-07-14T11:19:59.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 -- start start 2023-07-14T11:19:59.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efca0100cf0 0x7efca0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efca0101680 0x7efca019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca019f630 con 0x7efca00ffb00 2023-07-14T11:19:59.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca019f770 con 0x7efca0101680 2023-07-14T11:19:59.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efca49ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca019f900 con 0x7efca0100cf0 2023-07-14T11:19:59.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.152+0000 7efc9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca0197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51706/0 (socket says 172.21.15.161:51706) 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 -- 172.21.15.161:0/2846446319 learned_addr learned my addr 172.21.15.161:0/2846446319 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9dd9b700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efca0100cf0 0x7efca0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9ed9d700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efca0101680 0x7efca019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 -- 172.21.15.161:0/2846446319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efca0100cf0 msgr2=0x7efca0197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efca0100cf0 0x7efca0197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 -- 172.21.15.161:0/2846446319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efca0101680 msgr2=0x7efca019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efca0101680 0x7efca019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 -- 172.21.15.161:0/2846446319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900095f0 con 0x7efca00ffb00 2023-07-14T11:19:59.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc9e59c700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca0197ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc90005740 tx=0x7efc90005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:59.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc8f7fe700 1 -- 172.21.15.161:0/2846446319 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9001d070 con 0x7efca00ffb00 2023-07-14T11:19:59.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca019fb80 con 0x7efca00ffb00 2023-07-14T11:19:59.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc8f7fe700 1 -- 172.21.15.161:0/2846446319 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc90021410 con 0x7efca00ffb00 2023-07-14T11:19:59.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.153+0000 7efc8f7fe700 1 -- 172.21.15.161:0/2846446319 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90017650 con 0x7efca00ffb00 2023-07-14T11:19:59.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.154+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca019ffa0 con 0x7efca00ffb00 2023-07-14T11:19:59.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.155+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca010b320 con 0x7efca00ffb00 2023-07-14T11:19:59.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.156+0000 7efc8f7fe700 1 -- 172.21.15.161:0/2846446319 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efc90021580 con 0x7efca00ffb00 2023-07-14T11:19:59.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.158+0000 7efc8f7fe700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7efc880a11a0 0x7efc880a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.158+0000 7efc8f7fe700 1 -- 172.21.15.161:0/2846446319 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc900cdc80 con 0x7efca00ffb00 2023-07-14T11:19:59.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.158+0000 7efc9dd9b700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7efc880a11a0 0x7efc880a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.158+0000 7efc9dd9b700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7efc880a11a0 0x7efc880a3650 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc94009c60 tx=0x7efc94009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:59.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.161+0000 7efc8f7fe700 1 -- 172.21.15.161:0/2846446319 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efc90097330 con 0x7efca00ffb00 2023-07-14T11:19:59.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.358+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efca01a0550 con 0x7efca00ffb00 2023-07-14T11:19:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.360+0000 7efc8f7fe700 1 -- 172.21.15.161:0/2846446319 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23993 (secure 0 0 0) 0x7efc90026090 con 0x7efca00ffb00 2023-07-14T11:19:59.363 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:19:59.363 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:19:46.582122+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:19:59.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.362+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7efc880a11a0 msgr2=0x7efc880a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.362+0000 7efca49ff700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7efc880a11a0 0x7efc880a3650 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc94009c60 tx=0x7efc94009260 comp rx=0 tx=0).stop 2023-07-14T11:19:59.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 msgr2=0x7efca0197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca0197ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc90005740 tx=0x7efc90005770 comp rx=0 tx=0).stop 2023-07-14T11:19:59.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 shutdown_connections 2023-07-14T11:19:59.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efca00ffb00 0x7efca0197ab0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7efc880a11a0 0x7efc880a3650 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efca0101680 0x7efca019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 --2- 172.21.15.161:0/2846446319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efca0100cf0 0x7efca0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 >> 172.21.15.161:0/2846446319 conn(0x7efca00fb0e0 msgr2=0x7efca0106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:59.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 shutdown_connections 2023-07-14T11:19:59.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.363+0000 7efca49ff700 1 -- 172.21.15.161:0/2846446319 wait complete. 2023-07-14T11:19:59.381 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:59 smithi161 ceph-mon[95526]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 278 B/s wr, 0 op/s 2023-07-14T11:19:59.381 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:59 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2759087829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:19:59.381 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:19:59 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/421026145' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:19:59.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:19:59.384 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:19:59.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:59 smithi175 ceph-mon[129081]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 278 B/s wr, 0 op/s 2023-07-14T11:19:59.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:59 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2759087829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:19:59.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:19:59 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/421026145' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:19:59.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:59 smithi182 ceph-mon[129366]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 278 B/s wr, 0 op/s 2023-07-14T11:19:59.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:59 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2759087829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:19:59.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:19:59 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/421026145' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:19:59.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2718811166 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 msgr2=0x7fb478107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2718811166 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 0x7fb478107f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb47000b0a0 tx=0x7fb47000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2718811166 shutdown_connections 2023-07-14T11:19:59.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2718811166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4780ffae0 0x7fb4780ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2718811166 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 0x7fb478107f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/2718811166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478100cd0 0x7fb478101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2718811166 >> 172.21.15.161:0/2718811166 conn(0x7fb4780fb0e0 msgr2=0x7fb4780fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:59.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2718811166 shutdown_connections 2023-07-14T11:19:59.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.582+0000 7fb47f7ff700 1 -- 172.21.15.161:0/2718811166 wait complete. 2023-07-14T11:19:59.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.583+0000 7fb47f7ff700 1 Processor -- start 2023-07-14T11:19:59.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.583+0000 7fb47f7ff700 1 -- start start 2023-07-14T11:19:59.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.583+0000 7fb47f7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffae0 0x7fb478197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.583+0000 7fb47f7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cd0 0x7fb478197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47f7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 0x7fb47819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47f7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47819f4d0 con 0x7fb478100cd0 2023-07-14T11:19:59.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47f7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47819f610 con 0x7fb4780ffae0 2023-07-14T11:19:59.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47f7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47819f7a0 con 0x7fb478101660 2023-07-14T11:19:59.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cd0 0x7fb478197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cd0 0x7fb478197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51720/0 (socket says 172.21.15.161:51720) 2023-07-14T11:19:59.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 -- 172.21.15.161:0/3305518581 learned_addr learned my addr 172.21.15.161:0/3305518581 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:19:59.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47d59b700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffae0 0x7fb478197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47dd9c700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 0x7fb47819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 -- 172.21.15.161:0/3305518581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 msgr2=0x7fb47819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 0x7fb47819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 -- 172.21.15.161:0/3305518581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffae0 msgr2=0x7fb478197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffae0 0x7fb478197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 -- 172.21.15.161:0/3305518581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb470009d00 con 0x7fb478100cd0 2023-07-14T11:19:59.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.584+0000 7fb47cd9a700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cd0 0x7fb478197e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9a0 tx=0x7fb46800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:59.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.585+0000 7fb46e7fc700 1 -- 172.21.15.161:0/3305518581 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468009790 con 0x7fb478100cd0 2023-07-14T11:19:59.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.585+0000 7fb46e7fc700 1 -- 172.21.15.161:0/3305518581 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb46800cbd0 con 0x7fb478100cd0 2023-07-14T11:19:59.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.585+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47819fa80 con 0x7fb478100cd0 2023-07-14T11:19:59.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.585+0000 7fb46e7fc700 1 -- 172.21.15.161:0/3305518581 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468018680 con 0x7fb478100cd0 2023-07-14T11:19:59.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.585+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47819ff90 con 0x7fb478100cd0 2023-07-14T11:19:59.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.586+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47810b250 con 0x7fb478100cd0 2023-07-14T11:19:59.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.588+0000 7fb46e7fc700 1 -- 172.21.15.161:0/3305518581 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb46800cd40 con 0x7fb478100cd0 2023-07-14T11:19:59.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.589+0000 7fb46e7fc700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a1150 0x7fb4640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:19:59.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.589+0000 7fb46e7fc700 1 -- 172.21.15.161:0/3305518581 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb468014070 con 0x7fb478100cd0 2023-07-14T11:19:59.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.589+0000 7fb47d59b700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a1150 0x7fb4640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:19:59.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.590+0000 7fb47d59b700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a1150 0x7fb4640a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb478100b30 tx=0x7fb474009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:19:59.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.592+0000 7fb46e7fc700 1 -- 172.21.15.161:0/3305518581 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb468051790 con 0x7fb478100cd0 2023-07-14T11:19:59.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.794+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4781a06d0 con 0x7fb478100cd0 2023-07-14T11:19:59.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.794+0000 7fb46e7fc700 1 -- 172.21.15.161:0/3305518581 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4680962b0 con 0x7fb478100cd0 2023-07-14T11:19:59.797 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:19:59.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.796+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a1150 msgr2=0x7fb4640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.796+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a1150 0x7fb4640a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb478100b30 tx=0x7fb474009260 comp rx=0 tx=0).stop 2023-07-14T11:19:59.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.796+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cd0 msgr2=0x7fb478197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:19:59.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.796+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cd0 0x7fb478197e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9a0 tx=0x7fb46800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 shutdown_connections 2023-07-14T11:19:59.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4640a1150 0x7fb4640a3600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb478101660 0x7fb47819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4780ffae0 0x7fb478197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 --2- 172.21.15.161:0/3305518581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb478100cd0 0x7fb478197e90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:19:59.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 >> 172.21.15.161:0/3305518581 conn(0x7fb4780fb0e0 msgr2=0x7fb478106560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:19:59.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 shutdown_connections 2023-07-14T11:19:59.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:19:59.797+0000 7fb47f7ff700 1 -- 172.21.15.161:0/3305518581 wait complete. 2023-07-14T11:19:59.817 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:20:00.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 -- 172.21.15.161:0/1381591320 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e00ffaf0 msgr2=0x7f22e00fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1381591320 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e00ffaf0 0x7f22e00fff00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009990 tx=0x7f22d0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 -- 172.21.15.161:0/1381591320 shutdown_connections 2023-07-14T11:20:00.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1381591320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e0101670 0x7f22e0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1381591320 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e00ffaf0 0x7f22e00fff00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1381591320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22e0100ce0 0x7f22e0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 -- 172.21.15.161:0/1381591320 >> 172.21.15.161:0/1381591320 conn(0x7f22e00fb130 msgr2=0x7f22e00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:00.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 -- 172.21.15.161:0/1381591320 shutdown_connections 2023-07-14T11:20:00.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.015+0000 7f22e65ff700 1 -- 172.21.15.161:0/1381591320 wait complete. 2023-07-14T11:20:00.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.016+0000 7f22e65ff700 1 Processor -- start 2023-07-14T11:20:00.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.016+0000 7f22e65ff700 1 -- start start 2023-07-14T11:20:00.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22e65ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e00ffaf0 0x7f22e01957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22e65ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22e0100ce0 0x7f22e0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22e65ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e0101670 0x7f22e019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22e65ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019d370 con 0x7f22e0100ce0 2023-07-14T11:20:00.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22e65ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019d4b0 con 0x7f22e00ffaf0 2023-07-14T11:20:00.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22e65ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019d640 con 0x7f22e0101670 2023-07-14T11:20:00.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22e0100ce0 0x7f22e0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e00ffaf0 0x7f22e01957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e00ffaf0 0x7f22e01957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:50972/0 (socket says 172.21.15.161:50972) 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 -- 172.21.15.161:0/1553699659 learned_addr learned my addr 172.21.15.161:0/1553699659 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22e4b9c700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e0101670 0x7f22e019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 -- 172.21.15.161:0/1553699659 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e0101670 msgr2=0x7f22e019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e0101670 0x7f22e019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 -- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22e0100ce0 msgr2=0x7f22e0195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22e0100ce0 0x7f22e0195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22dffff700 1 -- 172.21.15.161:0/1553699659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00095f0 con 0x7f22e00ffaf0 2023-07-14T11:20:00.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.017+0000 7f22df7fe700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22e0100ce0 0x7f22e0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:00.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.018+0000 7f22dffff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e00ffaf0 0x7f22e01957f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22d0004710 tx=0x7f22d0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:00.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.018+0000 7f22dd7fa700 1 -- 172.21.15.161:0/1553699659 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d001d070 con 0x7f22e00ffaf0 2023-07-14T11:20:00.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.018+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e019d8c0 con 0x7f22e00ffaf0 2023-07-14T11:20:00.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.018+0000 7f22dd7fa700 1 -- 172.21.15.161:0/1553699659 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d0021410 con 0x7f22e00ffaf0 2023-07-14T11:20:00.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.018+0000 7f22dd7fa700 1 -- 172.21.15.161:0/1553699659 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0017650 con 0x7f22e00ffaf0 2023-07-14T11:20:00.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.018+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e019dd40 con 0x7f22e00ffaf0 2023-07-14T11:20:00.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.019+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e0106a20 con 0x7f22e00ffaf0 2023-07-14T11:20:00.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.021+0000 7f22dd7fa700 1 -- 172.21.15.161:0/1553699659 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f22d0021580 con 0x7f22e00ffaf0 2023-07-14T11:20:00.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.022+0000 7f22dd7fa700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f22c80a1150 0x7f22c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.022+0000 7f22dd7fa700 1 -- 172.21.15.161:0/1553699659 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f22d00ce1d0 con 0x7f22e00ffaf0 2023-07-14T11:20:00.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.022+0000 7f22df7fe700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f22c80a1150 0x7f22c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.023+0000 7f22df7fe700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f22c80a1150 0x7f22c80a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22d4011490 tx=0x7f22d4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:00.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.024+0000 7f22dd7fa700 1 -- 172.21.15.161:0/1553699659 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f22d0097860 con 0x7f22e00ffaf0 2023-07-14T11:20:00.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.231+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22e002c9e0 con 0x7f22e00ffaf0 2023-07-14T11:20:00.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.232+0000 7f22dd7fa700 1 -- 172.21.15.161:0/1553699659 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22d009ae80 con 0x7f22e00ffaf0 2023-07-14T11:20:00.234 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:20:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.234+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f22c80a1150 msgr2=0x7f22c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.234+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f22c80a1150 0x7f22c80a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22d4011490 tx=0x7f22d4009260 comp rx=0 tx=0).stop 2023-07-14T11:20:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.234+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e00ffaf0 msgr2=0x7f22e01957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.234+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e00ffaf0 0x7f22e01957f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22d0004710 tx=0x7f22d0009580 comp rx=0 tx=0).stop 2023-07-14T11:20:00.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 shutdown_connections 2023-07-14T11:20:00.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f22c80a1150 0x7f22c80a3600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f22e0101670 0x7f22e019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e00ffaf0 0x7f22e01957f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 --2- 172.21.15.161:0/1553699659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22e0100ce0 0x7f22e0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 >> 172.21.15.161:0/1553699659 conn(0x7f22e00fb130 msgr2=0x7f22e00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:00.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 shutdown_connections 2023-07-14T11:20:00.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.235+0000 7f22e65ff700 1 -- 172.21.15.161:0/1553699659 wait complete. 2023-07-14T11:20:00.255 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:20:00.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:00 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2846446319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:00.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:00 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3305518581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:20:00.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:00 smithi175 ceph-mon[129081]: overall HEALTH_OK 2023-07-14T11:20:00.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.461+0000 7f2045dff700 1 -- 172.21.15.161:0/707794773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040100cd0 msgr2=0x7f2040101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.461+0000 7f2045dff700 1 --2- 172.21.15.161:0/707794773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040100cd0 0x7f2040101120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2034009990 tx=0x7f2034009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.462+0000 7f2045dff700 1 -- 172.21.15.161:0/707794773 shutdown_connections 2023-07-14T11:20:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.462+0000 7f2045dff700 1 --2- 172.21.15.161:0/707794773 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2040101660 0x7f2040107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.462+0000 7f2045dff700 1 --2- 172.21.15.161:0/707794773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20400ffae0 0x7f20400ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.462+0000 7f2045dff700 1 --2- 172.21.15.161:0/707794773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040100cd0 0x7f2040101120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.462+0000 7f2045dff700 1 -- 172.21.15.161:0/707794773 >> 172.21.15.161:0/707794773 conn(0x7f20400fb0e0 msgr2=0x7f20400fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.462+0000 7f2045dff700 1 -- 172.21.15.161:0/707794773 shutdown_connections 2023-07-14T11:20:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.462+0000 7f2045dff700 1 -- 172.21.15.161:0/707794773 wait complete. 2023-07-14T11:20:00.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:00 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2846446319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:00.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:00 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3305518581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:20:00.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:00 smithi182 ceph-mon[129366]: overall HEALTH_OK 2023-07-14T11:20:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 Processor -- start 2023-07-14T11:20:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 -- start start 2023-07-14T11:20:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f20400ffae0 0x7f20400742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040100cd0 0x7f2040072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040101660 0x7f2040072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400735e0 con 0x7f2040101660 2023-07-14T11:20:00.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2040073750 con 0x7f2040100cd0 2023-07-14T11:20:00.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f2045dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204019f8e0 con 0x7f20400ffae0 2023-07-14T11:20:00.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f203ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040101660 0x7f2040072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f203ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040101660 0x7f2040072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51764/0 (socket says 172.21.15.161:51764) 2023-07-14T11:20:00.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f203ffff700 1 -- 172.21.15.161:0/2028453327 learned_addr learned my addr 172.21.15.161:0/2028453327 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:00.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f203f7fe700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f20400ffae0 0x7f20400742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.463+0000 7f203effd700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040100cd0 0x7f2040072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203ffff700 1 -- 172.21.15.161:0/2028453327 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f20400ffae0 msgr2=0x7f20400742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203ffff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f20400ffae0 0x7f20400742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203ffff700 1 -- 172.21.15.161:0/2028453327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040100cd0 msgr2=0x7f2040072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203ffff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040100cd0 0x7f2040072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203ffff700 1 -- 172.21.15.161:0/2028453327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340095f0 con 0x7f2040101660 2023-07-14T11:20:00.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203ffff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040101660 0x7f2040072e40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f203800ef30 tx=0x7f203800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:00.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203cff9700 1 -- 172.21.15.161:0/2028453327 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038018070 con 0x7f2040101660 2023-07-14T11:20:00.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203cff9700 1 -- 172.21.15.161:0/2028453327 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f203800f040 con 0x7f2040101660 2023-07-14T11:20:00.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204019fb60 con 0x7f2040101660 2023-07-14T11:20:00.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f203cff9700 1 -- 172.21.15.161:0/2028453327 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203800cd00 con 0x7f2040101660 2023-07-14T11:20:00.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.464+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401a0070 con 0x7f2040101660 2023-07-14T11:20:00.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.465+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204010b300 con 0x7f2040101660 2023-07-14T11:20:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.467+0000 7f203cff9700 1 -- 172.21.15.161:0/2028453327 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2038003a00 con 0x7f2040101660 2023-07-14T11:20:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.469+0000 7f203cff9700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f20280a1150 0x7f20280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.469+0000 7f203cff9700 1 -- 172.21.15.161:0/2028453327 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f20380cc420 con 0x7f2040101660 2023-07-14T11:20:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.469+0000 7f203f7fe700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f20280a1150 0x7f20280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.469+0000 7f203f7fe700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f20280a1150 0x7f20280a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2040100b30 tx=0x7f2030009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:00.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.472+0000 7f203cff9700 1 -- 172.21.15.161:0/2028453327 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2038095a10 con 0x7f2040101660 2023-07-14T11:20:00.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:00 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2846446319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:00 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3305518581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:20:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:00 smithi161 ceph-mon[95526]: overall HEALTH_OK 2023-07-14T11:20:00.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.671+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20401069e0 con 0x7f2040101660 2023-07-14T11:20:00.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.672+0000 7f203cff9700 1 -- 172.21.15.161:0/2028453327 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2038099030 con 0x7f2040101660 2023-07-14T11:20:00.674 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:20:00.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f20280a1150 msgr2=0x7f20280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f20280a1150 0x7f20280a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2040100b30 tx=0x7f2030009260 comp rx=0 tx=0).stop 2023-07-14T11:20:00.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040101660 msgr2=0x7f2040072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040101660 0x7f2040072e40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f203800ef30 tx=0x7f203800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 shutdown_connections 2023-07-14T11:20:00.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2040101660 0x7f2040072e40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f20280a1150 0x7f20280a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f20400ffae0 0x7f20400742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 --2- 172.21.15.161:0/2028453327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040100cd0 0x7f2040072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 >> 172.21.15.161:0/2028453327 conn(0x7f20400fb0e0 msgr2=0x7f20400fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:00.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 shutdown_connections 2023-07-14T11:20:00.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.674+0000 7f2045dff700 1 -- 172.21.15.161:0/2028453327 wait complete. 2023-07-14T11:20:00.695 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:20:00.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 -- 172.21.15.161:0/1527840025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c101670 msgr2=0x7f1a1c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1527840025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c101670 0x7f1a1c107f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b0a0 tx=0x7f1a1400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 -- 172.21.15.161:0/1527840025 shutdown_connections 2023-07-14T11:20:00.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1527840025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c101670 0x7f1a1c107f30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1527840025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a1c0ffaf0 0x7f1a1c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1527840025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1a1c100ce0 0x7f1a1c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 -- 172.21.15.161:0/1527840025 >> 172.21.15.161:0/1527840025 conn(0x7f1a1c0fb130 msgr2=0x7f1a1c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:00.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 -- 172.21.15.161:0/1527840025 shutdown_connections 2023-07-14T11:20:00.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.894+0000 7f1a237ff700 1 -- 172.21.15.161:0/1527840025 wait complete. 2023-07-14T11:20:00.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 Processor -- start 2023-07-14T11:20:00.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 -- start start 2023-07-14T11:20:00.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c0ffaf0 0x7f1a1c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a1c100ce0 0x7f1a1c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1a1c101670 0x7f1a1c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0735e0 con 0x7f1a1c0ffaf0 2023-07-14T11:20:00.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c073750 con 0x7f1a1c100ce0 2023-07-14T11:20:00.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.895+0000 7f1a237ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c19f910 con 0x7f1a1c101670 2023-07-14T11:20:00.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a2159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c0ffaf0 0x7f1a1c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a20d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a1c100ce0 0x7f1a1c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a2159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c0ffaf0 0x7f1a1c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51782/0 (socket says 172.21.15.161:51782) 2023-07-14T11:20:00.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a2159b700 1 -- 172.21.15.161:0/1758704690 learned_addr learned my addr 172.21.15.161:0/1758704690 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:00.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a21d9c700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1a1c101670 0x7f1a1c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a20d9a700 1 -- 172.21.15.161:0/1758704690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1a1c101670 msgr2=0x7f1a1c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a20d9a700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1a1c101670 0x7f1a1c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a20d9a700 1 -- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c0ffaf0 msgr2=0x7f1a1c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:00.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a20d9a700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c0ffaf0 0x7f1a1c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:00.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a20d9a700 1 -- 172.21.15.161:0/1758704690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18009520 con 0x7f1a1c100ce0 2023-07-14T11:20:00.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a2159b700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c0ffaf0 0x7f1a1c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:00.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.896+0000 7f1a20d9a700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a1c100ce0 0x7f1a1c072900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00e9f0 tx=0x7f1a0c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:00.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.897+0000 7f1a127fc700 1 -- 172.21.15.161:0/1758704690 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00c9c0 con 0x7f1a1c100ce0 2023-07-14T11:20:00.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.897+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a14009d00 con 0x7f1a1c100ce0 2023-07-14T11:20:00.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.897+0000 7f1a127fc700 1 -- 172.21.15.161:0/1758704690 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a0c010430 con 0x7f1a1c100ce0 2023-07-14T11:20:00.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.897+0000 7f1a127fc700 1 -- 172.21.15.161:0/1758704690 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c018680 con 0x7f1a1c100ce0 2023-07-14T11:20:00.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.897+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c19ffc0 con 0x7f1a1c100ce0 2023-07-14T11:20:00.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.898+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c106fc0 con 0x7f1a1c100ce0 2023-07-14T11:20:00.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.900+0000 7f1a127fc700 1 -- 172.21.15.161:0/1758704690 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1a0c010860 con 0x7f1a1c100ce0 2023-07-14T11:20:00.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.901+0000 7f1a127fc700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1a080a11d0 0x7f1a080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:00.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.901+0000 7f1a127fc700 1 -- 172.21.15.161:0/1758704690 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1a0c014070 con 0x7f1a1c100ce0 2023-07-14T11:20:00.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.901+0000 7f1a2159b700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1a080a11d0 0x7f1a080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:00.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.902+0000 7f1a2159b700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1a080a11d0 0x7f1a080a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009fd0 tx=0x7f1a18009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:00.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:00.903+0000 7f1a127fc700 1 -- 172.21.15.161:0/1758704690 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1a0c0960b0 con 0x7f1a1c100ce0 2023-07-14T11:20:01.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.104+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1a1c02c850 con 0x7f1a1c100ce0 2023-07-14T11:20:01.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.105+0000 7f1a127fc700 1 -- 172.21.15.161:0/1758704690 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1a0c0996d0 con 0x7f1a1c100ce0 2023-07-14T11:20:01.108 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:20:01.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1a080a11d0 msgr2=0x7f1a080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:01.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1a080a11d0 0x7f1a080a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009fd0 tx=0x7f1a18009260 comp rx=0 tx=0).stop 2023-07-14T11:20:01.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a1c100ce0 msgr2=0x7f1a1c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:01.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a1c100ce0 0x7f1a1c072900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00e9f0 tx=0x7f1a0c00ed00 comp rx=0 tx=0).stop 2023-07-14T11:20:01.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 shutdown_connections 2023-07-14T11:20:01.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a1c0ffaf0 0x7f1a1c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1a080a11d0 0x7f1a080a3680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1a1c101670 0x7f1a1c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 --2- 172.21.15.161:0/1758704690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a1c100ce0 0x7f1a1c072900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 >> 172.21.15.161:0/1758704690 conn(0x7f1a1c0fb130 msgr2=0x7f1a1c104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:01.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 shutdown_connections 2023-07-14T11:20:01.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.108+0000 7f1a237ff700 1 -- 172.21.15.161:0/1758704690 wait complete. 2023-07-14T11:20:01.129 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:20:01.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 -- 172.21.15.161:0/4104158385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 msgr2=0x7fb2a40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:01.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/4104158385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a40ffef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb294009990 tx=0x7fb294009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 -- 172.21.15.161:0/4104158385 shutdown_connections 2023-07-14T11:20:01.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/4104158385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a40ffef0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/4104158385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a4101660 0x7fb2a4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/4104158385 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb2a4100cd0 0x7fb2a4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 -- 172.21.15.161:0/4104158385 >> 172.21.15.161:0/4104158385 conn(0x7fb2a40fb0e0 msgr2=0x7fb2a40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:01.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 -- 172.21.15.161:0/4104158385 shutdown_connections 2023-07-14T11:20:01.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.325+0000 7fb2a99ff700 1 -- 172.21.15.161:0/4104158385 wait complete. 2023-07-14T11:20:01.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.326+0000 7fb2a99ff700 1 Processor -- start 2023-07-14T11:20:01.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.326+0000 7fb2a99ff700 1 -- start start 2023-07-14T11:20:01.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.326+0000 7fb2a99ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a4195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:01.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.326+0000 7fb2a99ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a4100cd0 0x7fb2a4195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:01.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a99ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb2a4101660 0x7fb2a419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:01.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a99ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419d3e0 con 0x7fb2a40ffae0 2023-07-14T11:20:01.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a99ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419d520 con 0x7fb2a4100cd0 2023-07-14T11:20:01.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a99ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419d6b0 con 0x7fb2a4101660 2023-07-14T11:20:01.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a4195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:01.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a4195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51806/0 (socket says 172.21.15.161:51806) 2023-07-14T11:20:01.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 -- 172.21.15.161:0/1653795109 learned_addr learned my addr 172.21.15.161:0/1653795109 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:01.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a27fc700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a4100cd0 0x7fb2a4195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:01.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a37fe700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb2a4101660 0x7fb2a419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:01.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 -- 172.21.15.161:0/1653795109 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb2a4101660 msgr2=0x7fb2a419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:01.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb2a4101660 0x7fb2a419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 -- 172.21.15.161:0/1653795109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a4100cd0 msgr2=0x7fb2a4195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:01.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a4100cd0 0x7fb2a4195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 -- 172.21.15.161:0/1653795109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940095f0 con 0x7fb2a40ffae0 2023-07-14T11:20:01.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.327+0000 7fb2a2ffd700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a4195860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2940098e0 tx=0x7fb294004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:01.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.328+0000 7fb2a89fd700 1 -- 172.21.15.161:0/1653795109 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29401d070 con 0x7fb2a40ffae0 2023-07-14T11:20:01.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.328+0000 7fb2a89fd700 1 -- 172.21.15.161:0/1653795109 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb294005030 con 0x7fb2a40ffae0 2023-07-14T11:20:01.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.328+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a419d930 con 0x7fb2a40ffae0 2023-07-14T11:20:01.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.328+0000 7fb2a89fd700 1 -- 172.21.15.161:0/1653795109 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294004b50 con 0x7fb2a40ffae0 2023-07-14T11:20:01.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.328+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a419dcf0 con 0x7fb2a40ffae0 2023-07-14T11:20:01.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.329+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a410b300 con 0x7fb2a40ffae0 2023-07-14T11:20:01.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.331+0000 7fb2a89fd700 1 -- 172.21.15.161:0/1653795109 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb294004d30 con 0x7fb2a40ffae0 2023-07-14T11:20:01.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.332+0000 7fb2a89fd700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb28c0a1150 0x7fb28c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:01.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.332+0000 7fb2a89fd700 1 -- 172.21.15.161:0/1653795109 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb2940cddc0 con 0x7fb2a40ffae0 2023-07-14T11:20:01.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.332+0000 7fb2a27fc700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb28c0a1150 0x7fb28c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:01.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.333+0000 7fb2a27fc700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb28c0a1150 0x7fb28c0a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb298009c60 tx=0x7fb298009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:01.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.336+0000 7fb2a89fd700 1 -- 172.21.15.161:0/1653795109 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb29405a790 con 0x7fb2a40ffae0 2023-07-14T11:20:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:01 smithi175 ceph-mon[129081]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T11:20:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1553699659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2028453327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1758704690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:01 smithi182 ceph-mon[129366]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T11:20:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1553699659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2028453327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1758704690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:01 smithi161 ceph-mon[95526]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T11:20:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1553699659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2028453327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1758704690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:20:01.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.554+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb2a419e390 con 0x7fb2a40ffae0 2023-07-14T11:20:01.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.554+0000 7fb2a89fd700 1 -- 172.21.15.161:0/1653795109 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+599 (secure 0 0 0) 0x7fb29409ade0 con 0x7fb2a40ffae0 2023-07-14T11:20:01.557 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/871206111 2023-07-15T11:13:14.005108+0000 2023-07-14T11:20:01.557 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6801/3292405426 2023-07-15T11:13:14.005108+0000 2023-07-14T11:20:01.557 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6800/3292405426 2023-07-15T11:13:14.005108+0000 2023-07-14T11:20:01.557 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/629268496 2023-07-15T11:12:47.212143+0000 2023-07-14T11:20:01.557 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/832098793 2023-07-15T11:13:14.005108+0000 2023-07-14T11:20:01.557 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6800/4131434778 2023-07-15T11:12:47.212143+0000 2023-07-14T11:20:01.558 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6801/4131434778 2023-07-15T11:12:47.212143+0000 2023-07-14T11:20:01.558 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/3333551885 2023-07-15T11:12:47.212143+0000 2023-07-14T11:20:01.558 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/1691634661 2023-07-15T11:12:47.212143+0000 2023-07-14T11:20:01.558 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/3118869366 2023-07-15T11:13:14.005108+0000 2023-07-14T11:20:01.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb28c0a1150 msgr2=0x7fb28c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:01.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb28c0a1150 0x7fb28c0a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb298009c60 tx=0x7fb298009260 comp rx=0 tx=0).stop 2023-07-14T11:20:01.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 msgr2=0x7fb2a4195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:01.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a4195860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2940098e0 tx=0x7fb294004730 comp rx=0 tx=0).stop 2023-07-14T11:20:01.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 shutdown_connections 2023-07-14T11:20:01.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a40ffae0 0x7fb2a4195860 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb28c0a1150 0x7fb28c0a3600 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb2a4101660 0x7fb2a419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 --2- 172.21.15.161:0/1653795109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a4100cd0 0x7fb2a4195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:01.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 >> 172.21.15.161:0/1653795109 conn(0x7fb2a40fb0e0 msgr2=0x7fb2a40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:01.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.556+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 shutdown_connections 2023-07-14T11:20:01.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:01.557+0000 7fb2a99ff700 1 -- 172.21.15.161:0/1653795109 wait complete. 2023-07-14T11:20:01.561 INFO:teuthology.orchestra.run.smithi161.stderr:listed 10 entries 2023-07-14T11:20:01.577 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi161.front.sepia.ceph.com... 2023-07-14T11:20:01.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:20:01.578 DEBUG:teuthology.orchestra.run.smithi161:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:20:01.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi161.stderr:ceph-fuse[132903]: fuse finished with error 0 and tester_r 0 2023-07-14T11:20:01.649 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:20:02.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:02 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1653795109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:20:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:02 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1653795109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:20:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:02 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1653795109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:20:03.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:03 smithi182 ceph-mon[129366]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T11:20:03.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:03 smithi161 ceph-mon[95526]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T11:20:04.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:03 smithi175 ceph-mon[129081]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T11:20:05.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:05 smithi182 ceph-mon[129366]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:05.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:05 smithi161 ceph-mon[95526]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:06.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:05 smithi175 ceph-mon[129081]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:07.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T11:20:07.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:20:07.751 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-14T11:20:07.812 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:20:07.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:07 smithi182 ceph-mon[129366]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:07.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:07 smithi161 ceph-mon[95526]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:08.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3208564599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f8100cf0 msgr2=0x7f90f8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3208564599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f8100cf0 0x7f90f8101140 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009990 tx=0x7f90ec009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3208564599 shutdown_connections 2023-07-14T11:20:08.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3208564599 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90f8101680 0x7f90f8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3208564599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90f80ffb00 0x7f90f80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3208564599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f8100cf0 0x7f90f8101140 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3208564599 >> 172.21.15.161:0/3208564599 conn(0x7f90f80fb0e0 msgr2=0x7f90f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:08.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3208564599 shutdown_connections 2023-07-14T11:20:08.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.011+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3208564599 wait complete. 2023-07-14T11:20:08.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.012+0000 7f90fe3ff700 1 Processor -- start 2023-07-14T11:20:08.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fe3ff700 1 -- start start 2023-07-14T11:20:08.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fe3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f80ffb00 0x7f90f8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fe3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90f8100cf0 0x7f90f8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fe3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90f8101680 0x7f90f819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fe3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f819f5c0 con 0x7f90f80ffb00 2023-07-14T11:20:08.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fe3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f819f700 con 0x7f90f8100cf0 2023-07-14T11:20:08.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fe3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f819f890 con 0x7f90f8101680 2023-07-14T11:20:08.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f80ffb00 0x7f90f8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f80ffb00 0x7f90f8197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44934/0 (socket says 172.21.15.161:44934) 2023-07-14T11:20:08.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90f7fff700 1 -- 172.21.15.161:0/3266939884 learned_addr learned my addr 172.21.15.161:0/3266939884 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:08.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90fc99c700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90f8101680 0x7f90f819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.013+0000 7f90f77fe700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90f8100cf0 0x7f90f8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f7fff700 1 -- 172.21.15.161:0/3266939884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90f8101680 msgr2=0x7f90f819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f7fff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90f8101680 0x7f90f819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f7fff700 1 -- 172.21.15.161:0/3266939884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90f8100cf0 msgr2=0x7f90f8197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f7fff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90f8100cf0 0x7f90f8197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f7fff700 1 -- 172.21.15.161:0/3266939884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec0095f0 con 0x7f90f80ffb00 2023-07-14T11:20:08.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90fc99c700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90f8101680 0x7f90f819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:08.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f7fff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f80ffb00 0x7f90f8197a40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e970 tx=0x7f90e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:08.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f57fa700 1 -- 172.21.15.161:0/3266939884 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8009790 con 0x7f90f80ffb00 2023-07-14T11:20:08.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f57fa700 1 -- 172.21.15.161:0/3266939884 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90e800c7e0 con 0x7f90f80ffb00 2023-07-14T11:20:08.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f819fb70 con 0x7f90f80ffb00 2023-07-14T11:20:08.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90f57fa700 1 -- 172.21.15.161:0/3266939884 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8003c70 con 0x7f90f80ffb00 2023-07-14T11:20:08.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.014+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f81a0080 con 0x7f90f80ffb00 2023-07-14T11:20:08.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.015+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f810b320 con 0x7f90f80ffb00 2023-07-14T11:20:08.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.017+0000 7f90f57fa700 1 -- 172.21.15.161:0/3266939884 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f90e800cc50 con 0x7f90f80ffb00 2023-07-14T11:20:08.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.019+0000 7f90f57fa700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90e00a11a0 0x7f90e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.019+0000 7f90f57fa700 1 -- 172.21.15.161:0/3266939884 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f90e8014070 con 0x7f90f80ffb00 2023-07-14T11:20:08.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.019+0000 7f90f77fe700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90e00a11a0 0x7f90e00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.019+0000 7f90f77fe700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90e00a11a0 0x7f90e00a3650 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00b2e0 tx=0x7f90ec005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:08.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.022+0000 7f90f57fa700 1 -- 172.21.15.161:0/3266939884 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f90e8096210 con 0x7f90f80ffb00 2023-07-14T11:20:08.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:07 smithi175 ceph-mon[129081]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:08.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.223+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90f810b4e0 con 0x7f90f80ffb00 2023-07-14T11:20:08.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.225+0000 7f90f57fa700 1 -- 172.21.15.161:0/3266939884 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f90e8003dd0 con 0x7f90f80ffb00 2023-07-14T11:20:08.229 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:08.229 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:19:46.582122+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:20:08.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.227+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90e00a11a0 msgr2=0x7f90e00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.227+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90e00a11a0 0x7f90e00a3650 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00b2e0 tx=0x7f90ec005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.227+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f80ffb00 msgr2=0x7f90f8197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.227+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f80ffb00 0x7f90f8197a40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e970 tx=0x7f90e800ec80 comp rx=0 tx=0).stop 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 shutdown_connections 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f80ffb00 0x7f90f8197a40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90e00a11a0 0x7f90e00a3650 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90f8101680 0x7f90f819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 --2- 172.21.15.161:0/3266939884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90f8100cf0 0x7f90f8197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 >> 172.21.15.161:0/3266939884 conn(0x7f90f80fb0e0 msgr2=0x7f90f80fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 shutdown_connections 2023-07-14T11:20:08.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.228+0000 7f90fe3ff700 1 -- 172.21.15.161:0/3266939884 wait complete. 2023-07-14T11:20:08.250 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:20:08.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 -- 172.21.15.161:0/4063413442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c00ffb00 msgr2=0x7f49c00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 --2- 172.21.15.161:0/4063413442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c00ffb00 0x7f49c00fff10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009990 tx=0x7f49b0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 -- 172.21.15.161:0/4063413442 shutdown_connections 2023-07-14T11:20:08.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 --2- 172.21.15.161:0/4063413442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c00ffb00 0x7f49c00fff10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 --2- 172.21.15.161:0/4063413442 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49c0101680 0x7f49c0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 --2- 172.21.15.161:0/4063413442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c0100cf0 0x7f49c0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 -- 172.21.15.161:0/4063413442 >> 172.21.15.161:0/4063413442 conn(0x7f49c00fb0e0 msgr2=0x7f49c00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:08.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 -- 172.21.15.161:0/4063413442 shutdown_connections 2023-07-14T11:20:08.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.446+0000 7f49c5bff700 1 -- 172.21.15.161:0/4063413442 wait complete. 2023-07-14T11:20:08.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 Processor -- start 2023-07-14T11:20:08.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 -- start start 2023-07-14T11:20:08.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c00ffb00 0x7f49c0195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c0100cf0 0x7f49c0195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49c0101680 0x7f49c019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c019d3c0 con 0x7f49c0100cf0 2023-07-14T11:20:08.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c019d500 con 0x7f49c00ffb00 2023-07-14T11:20:08.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.447+0000 7f49c5bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c019d690 con 0x7f49c0101680 2023-07-14T11:20:08.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c0100cf0 0x7f49c0195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c0100cf0 0x7f49c0195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44952/0 (socket says 172.21.15.161:44952) 2023-07-14T11:20:08.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bf7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c00ffb00 0x7f49c0195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49b7fff700 1 -- 172.21.15.161:0/19749093 learned_addr learned my addr 172.21.15.161:0/19749093 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:08.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bffff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49c0101680 0x7f49c019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bf7fe700 1 -- 172.21.15.161:0/19749093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49c0101680 msgr2=0x7f49c019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bf7fe700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49c0101680 0x7f49c019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bf7fe700 1 -- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c0100cf0 msgr2=0x7f49c0195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bf7fe700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c0100cf0 0x7f49c0195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bf7fe700 1 -- 172.21.15.161:0/19749093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00095f0 con 0x7f49c00ffb00 2023-07-14T11:20:08.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49b7fff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c0100cf0 0x7f49c0195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:08.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.448+0000 7f49bf7fe700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c00ffb00 0x7f49c0195840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49b00098e0 tx=0x7f49b0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:08.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.449+0000 7f49bd7fa700 1 -- 172.21.15.161:0/19749093 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b001d070 con 0x7f49c00ffb00 2023-07-14T11:20:08.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.449+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c019d910 con 0x7f49c00ffb00 2023-07-14T11:20:08.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.449+0000 7f49bd7fa700 1 -- 172.21.15.161:0/19749093 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b0021410 con 0x7f49c00ffb00 2023-07-14T11:20:08.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.449+0000 7f49bd7fa700 1 -- 172.21.15.161:0/19749093 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0017650 con 0x7f49c00ffb00 2023-07-14T11:20:08.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.449+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c019dd30 con 0x7f49c00ffb00 2023-07-14T11:20:08.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.450+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c010b320 con 0x7f49c00ffb00 2023-07-14T11:20:08.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.452+0000 7f49bd7fa700 1 -- 172.21.15.161:0/19749093 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f49b0021580 con 0x7f49c00ffb00 2023-07-14T11:20:08.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.453+0000 7f49bd7fa700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49ac0a11a0 0x7f49ac0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.454+0000 7f49bd7fa700 1 -- 172.21.15.161:0/19749093 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f49b00ce4f0 con 0x7f49c00ffb00 2023-07-14T11:20:08.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.454+0000 7f49b7fff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49ac0a11a0 0x7f49ac0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.454+0000 7f49b7fff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49ac0a11a0 0x7f49ac0a3650 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f49a8011550 tx=0x7f49a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:08.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.455+0000 7f49bd7fa700 1 -- 172.21.15.161:0/19749093 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f49b0097a80 con 0x7f49c00ffb00 2023-07-14T11:20:08.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.655+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49c019e370 con 0x7f49c00ffb00 2023-07-14T11:20:08.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.657+0000 7f49bd7fa700 1 -- 172.21.15.161:0/19749093 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f49b009b0a0 con 0x7f49c00ffb00 2023-07-14T11:20:08.661 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:08.661 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:19:46.582122+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:20:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49ac0a11a0 msgr2=0x7f49ac0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49ac0a11a0 0x7f49ac0a3650 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f49a8011550 tx=0x7f49a8009260 comp rx=0 tx=0).stop 2023-07-14T11:20:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c00ffb00 msgr2=0x7f49c0195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c00ffb00 0x7f49c0195840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49b00098e0 tx=0x7f49b0005740 comp rx=0 tx=0).stop 2023-07-14T11:20:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 shutdown_connections 2023-07-14T11:20:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f49ac0a11a0 0x7f49ac0a3650 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49c0101680 0x7f49c019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c00ffb00 0x7f49c0195840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 --2- 172.21.15.161:0/19749093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c0100cf0 0x7f49c0195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 >> 172.21.15.161:0/19749093 conn(0x7f49c00fb0e0 msgr2=0x7f49c00fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.659+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 shutdown_connections 2023-07-14T11:20:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.660+0000 7f49c5bff700 1 -- 172.21.15.161:0/19749093 wait complete. 2023-07-14T11:20:08.682 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T11:20:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 -- 172.21.15.161:0/1608516326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98440ffaa0 msgr2=0x7f98440ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 --2- 172.21.15.161:0/1608516326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98440ffaa0 0x7f98440ffeb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f984000b0a0 tx=0x7f984000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 -- 172.21.15.161:0/1608516326 shutdown_connections 2023-07-14T11:20:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 --2- 172.21.15.161:0/1608516326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98440ffaa0 0x7f98440ffeb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 --2- 172.21.15.161:0/1608516326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98441016e0 0x7f9844107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 --2- 172.21.15.161:0/1608516326 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9844100d30 0x7f98441011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 -- 172.21.15.161:0/1608516326 >> 172.21.15.161:0/1608516326 conn(0x7f98440fb0c0 msgr2=0x7f98440fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 -- 172.21.15.161:0/1608516326 shutdown_connections 2023-07-14T11:20:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.881+0000 7f984c5ff700 1 -- 172.21.15.161:0/1608516326 wait complete. 2023-07-14T11:20:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 Processor -- start 2023-07-14T11:20:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 -- start start 2023-07-14T11:20:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440ffaa0 0x7f9844197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9844100d30 0x7f9844198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441016e0 0x7f984419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984419f670 con 0x7f98441016e0 2023-07-14T11:20:08.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984419f7b0 con 0x7f98440ffaa0 2023-07-14T11:20:08.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.883+0000 7f984c5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984419f940 con 0x7f9844100d30 2023-07-14T11:20:08.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984ab9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441016e0 0x7f984419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440ffaa0 0x7f9844197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440ffaa0 0x7f9844197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34860/0 (socket says 172.21.15.161:34860) 2023-07-14T11:20:08.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 -- 172.21.15.161:0/921530913 learned_addr learned my addr 172.21.15.161:0/921530913 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:08.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f9849b9a700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9844100d30 0x7f9844198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 -- 172.21.15.161:0/921530913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9844100d30 msgr2=0x7f9844198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9844100d30 0x7f9844198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 -- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441016e0 msgr2=0x7f984419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441016e0 0x7f984419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 -- 172.21.15.161:0/921530913 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840009d00 con 0x7f98440ffaa0 2023-07-14T11:20:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984ab9c700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441016e0 0x7f984419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.884+0000 7f984a39b700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440ffaa0 0x7f9844197af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f984000b070 tx=0x7f984000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.885+0000 7f983b7fe700 1 -- 172.21.15.161:0/921530913 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984000e040 con 0x7f98440ffaa0 2023-07-14T11:20:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.885+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984419fbc0 con 0x7f98440ffaa0 2023-07-14T11:20:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.885+0000 7f983b7fe700 1 -- 172.21.15.161:0/921530913 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9840003c00 con 0x7f98440ffaa0 2023-07-14T11:20:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.885+0000 7f983b7fe700 1 -- 172.21.15.161:0/921530913 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984001b6c0 con 0x7f98440ffaa0 2023-07-14T11:20:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.885+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98441a0040 con 0x7f98440ffaa0 2023-07-14T11:20:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.886+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984410b270 con 0x7f98440ffaa0 2023-07-14T11:20:08.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.888+0000 7f983b7fe700 1 -- 172.21.15.161:0/921530913 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f984001d470 con 0x7f98440ffaa0 2023-07-14T11:20:08.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.889+0000 7f983b7fe700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f98300a1150 0x7f98300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:08.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.889+0000 7f983b7fe700 1 -- 172.21.15.161:0/921530913 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f98400cdc30 con 0x7f98440ffaa0 2023-07-14T11:20:08.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.889+0000 7f9849b9a700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f98300a1150 0x7f98300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:08.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.890+0000 7f9849b9a700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f98300a1150 0x7f98300a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9834009bf0 tx=0x7f9834009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:08.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:08.893+0000 7f983b7fe700 1 -- 172.21.15.161:0/921530913 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f98400971c0 con 0x7f98440ffaa0 2023-07-14T11:20:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:08 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3266939884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:08.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:08 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/19749093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:08.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:08 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3266939884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:08.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:08 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/19749093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:09.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.141+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f984410b400 con 0x7f98440ffaa0 2023-07-14T11:20:09.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:08 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3266939884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:09.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:08 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/19749093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.685+0000 7f983b7fe700 1 -- 172.21.15.161:0/921530913 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f984009a7e0 con 0x7f98440ffaa0 2023-07-14T11:20:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f98300a1150 msgr2=0x7f98300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f98300a1150 0x7f98300a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9834009bf0 tx=0x7f9834009260 comp rx=0 tx=0).stop 2023-07-14T11:20:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440ffaa0 msgr2=0x7f9844197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440ffaa0 0x7f9844197af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f984000b070 tx=0x7f984000bab0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 shutdown_connections 2023-07-14T11:20:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441016e0 0x7f984419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f98300a1150 0x7f98300a3600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440ffaa0 0x7f9844197af0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 --2- 172.21.15.161:0/921530913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9844100d30 0x7f9844198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 >> 172.21.15.161:0/921530913 conn(0x7f98440fb0c0 msgr2=0x7f98440fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 shutdown_connections 2023-07-14T11:20:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.688+0000 7f984c5ff700 1 -- 172.21.15.161:0/921530913 wait complete. 2023-07-14T11:20:09.711 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-07-14T11:20:09.712 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-14T11:20:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 -- 172.21.15.161:0/1249478711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4101670 msgr2=0x7f92c4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/1249478711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4101670 0x7f92c4107f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00b0a0 tx=0x7f92bc00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 -- 172.21.15.161:0/1249478711 shutdown_connections 2023-07-14T11:20:09.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/1249478711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4101670 0x7f92c4107f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/1249478711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40ffaf0 0x7f92c40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/1249478711 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f92c4100ce0 0x7f92c4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 -- 172.21.15.161:0/1249478711 >> 172.21.15.161:0/1249478711 conn(0x7f92c40fb130 msgr2=0x7f92c40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:09.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 -- 172.21.15.161:0/1249478711 shutdown_connections 2023-07-14T11:20:09.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.911+0000 7f92ca1ff700 1 -- 172.21.15.161:0/1249478711 wait complete. 2023-07-14T11:20:09.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.913+0000 7f92ca1ff700 1 Processor -- start 2023-07-14T11:20:09.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.913+0000 7f92ca1ff700 1 -- start start 2023-07-14T11:20:09.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92ca1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c40ffaf0 0x7f92c4195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:09.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92ca1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c4100ce0 0x7f92c4195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92ca1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f92c4101670 0x7f92c419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92ca1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419d480 con 0x7f92c40ffaf0 2023-07-14T11:20:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92ca1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419d5c0 con 0x7f92c4100ce0 2023-07-14T11:20:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92ca1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c419d750 con 0x7f92c4101670 2023-07-14T11:20:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c40ffaf0 0x7f92c4195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c40ffaf0 0x7f92c4195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44990/0 (socket says 172.21.15.161:44990) 2023-07-14T11:20:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92bbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c4100ce0 0x7f92c4195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 -- 172.21.15.161:0/3102977694 learned_addr learned my addr 172.21.15.161:0/3102977694 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c3fff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f92c4101670 0x7f92c419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 -- 172.21.15.161:0/3102977694 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f92c4101670 msgr2=0x7f92c419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f92c4101670 0x7f92c419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 -- 172.21.15.161:0/3102977694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c4100ce0 msgr2=0x7f92c4195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c4100ce0 0x7f92c4195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c37fe700 1 -- 172.21.15.161:0/3102977694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc009d00 con 0x7f92c40ffaf0 2023-07-14T11:20:09.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.914+0000 7f92c3fff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f92c4101670 0x7f92c419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:09.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.915+0000 7f92c37fe700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c40ffaf0 0x7f92c4195900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f92b400e970 tx=0x7f92b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:09.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.915+0000 7f92c17fa700 1 -- 172.21.15.161:0/3102977694 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4009790 con 0x7f92c40ffaf0 2023-07-14T11:20:09.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.915+0000 7f92c17fa700 1 -- 172.21.15.161:0/3102977694 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92b400eec0 con 0x7f92c40ffaf0 2023-07-14T11:20:09.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.915+0000 7f92c17fa700 1 -- 172.21.15.161:0/3102977694 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4003cf0 con 0x7f92c40ffaf0 2023-07-14T11:20:09.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.915+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c419da30 con 0x7f92c40ffaf0 2023-07-14T11:20:09.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.915+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c419def0 con 0x7f92c40ffaf0 2023-07-14T11:20:09.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.916+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c4106fc0 con 0x7f92c40ffaf0 2023-07-14T11:20:09.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.918+0000 7f92c17fa700 1 -- 172.21.15.161:0/3102977694 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f92b400ccd0 con 0x7f92c40ffaf0 2023-07-14T11:20:09.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.919+0000 7f92c17fa700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f92b00a11d0 0x7f92b00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:09.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.919+0000 7f92c17fa700 1 -- 172.21.15.161:0/3102977694 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f92b4014070 con 0x7f92c40ffaf0 2023-07-14T11:20:09.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.919+0000 7f92bbfff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f92b00a11d0 0x7f92b00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:09.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.920+0000 7f92bbfff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f92b00a11d0 0x7f92b00a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009c60 tx=0x7f92ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:09.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:09.923+0000 7f92c17fa700 1 -- 172.21.15.161:0/3102977694 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f92b40962b0 con 0x7f92c40ffaf0 2023-07-14T11:20:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:09 smithi182 ceph-mon[129366]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:09 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/921530913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T11:20:09.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:09 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T11:20:09.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:09 smithi161 ceph-mon[95526]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:09.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:09 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/921530913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T11:20:09.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:09 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T11:20:10.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:09 smithi175 ceph-mon[129081]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T11:20:10.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:09 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/921530913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T11:20:10.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:09 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T11:20:10.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.171+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f92c40611c0 con 0x7f92c40ffaf0 2023-07-14T11:20:10.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.697+0000 7f92c17fa700 1 -- 172.21.15.161:0/3102977694 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f92b40998d0 con 0x7f92c40ffaf0 2023-07-14T11:20:10.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f92b00a11d0 msgr2=0x7f92b00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:10.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f92b00a11d0 0x7f92b00a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009c60 tx=0x7f92ac009260 comp rx=0 tx=0).stop 2023-07-14T11:20:10.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c40ffaf0 msgr2=0x7f92c4195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:10.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c40ffaf0 0x7f92c4195900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f92b400e970 tx=0x7f92b400ec80 comp rx=0 tx=0).stop 2023-07-14T11:20:10.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 shutdown_connections 2023-07-14T11:20:10.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c40ffaf0 0x7f92c4195900 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f92b00a11d0 0x7f92b00a3680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f92c4101670 0x7f92c419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 --2- 172.21.15.161:0/3102977694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c4100ce0 0x7f92c4195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.700+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 >> 172.21.15.161:0/3102977694 conn(0x7f92c40fb130 msgr2=0x7f92c4104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:10.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.701+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 shutdown_connections 2023-07-14T11:20:10.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.701+0000 7f92ca1ff700 1 -- 172.21.15.161:0/3102977694 wait complete. 2023-07-14T11:20:10.726 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-14T11:20:10.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 -- 172.21.15.161:0/2082674372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 msgr2=0x7f4644072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 --2- 172.21.15.161:0/2082674372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 0x7f4644072940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4634009990 tx=0x7f4634009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 -- 172.21.15.161:0/2082674372 shutdown_connections 2023-07-14T11:20:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 --2- 172.21.15.161:0/2082674372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 0x7f4644072940 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 --2- 172.21.15.161:0/2082674372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644107980 0x7f4644109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 --2- 172.21.15.161:0/2082674372 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4644072f10 0x7f4644073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 -- 172.21.15.161:0/2082674372 >> 172.21.15.161:0/2082674372 conn(0x7f46440fb130 msgr2=0x7f46440fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:10.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 -- 172.21.15.161:0/2082674372 shutdown_connections 2023-07-14T11:20:10.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.927+0000 7f4649dff700 1 -- 172.21.15.161:0/2082674372 wait complete. 2023-07-14T11:20:10.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.928+0000 7f4649dff700 1 Processor -- start 2023-07-14T11:20:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.928+0000 7f4649dff700 1 -- start start 2023-07-14T11:20:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.928+0000 7f4649dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644072f10 0x7f4644197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.928+0000 7f4649dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 0x7f4644197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4649dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4644107980 0x7f464419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4649dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464419f5e0 con 0x7f46440744f0 2023-07-14T11:20:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4649dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464419f720 con 0x7f4644072f10 2023-07-14T11:20:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4649dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464419f8b0 con 0x7f4644107980 2023-07-14T11:20:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4642ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 0x7f4644197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f46437fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644072f10 0x7f4644197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f46437fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644072f10 0x7f4644197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34898/0 (socket says 172.21.15.161:34898) 2023-07-14T11:20:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f46437fe700 1 -- 172.21.15.161:0/3373138358 learned_addr learned my addr 172.21.15.161:0/3373138358 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4643fff700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4644107980 0x7f464419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4642ffd700 1 -- 172.21.15.161:0/3373138358 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4644107980 msgr2=0x7f464419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4642ffd700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4644107980 0x7f464419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4642ffd700 1 -- 172.21.15.161:0/3373138358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644072f10 msgr2=0x7f4644197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4642ffd700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644072f10 0x7f4644197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4642ffd700 1 -- 172.21.15.161:0/3373138358 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340095f0 con 0x7f46440744f0 2023-07-14T11:20:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f46437fe700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644072f10 0x7f4644197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.929+0000 7f4642ffd700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 0x7f4644197fa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f463800e9f0 tx=0x7f463800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.930+0000 7f4640ff9700 1 -- 172.21.15.161:0/3373138358 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800c9c0 con 0x7f46440744f0 2023-07-14T11:20:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.930+0000 7f4640ff9700 1 -- 172.21.15.161:0/3373138358 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4638010430 con 0x7f46440744f0 2023-07-14T11:20:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.930+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464419fb90 con 0x7f46440744f0 2023-07-14T11:20:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.930+0000 7f4640ff9700 1 -- 172.21.15.161:0/3373138358 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638018680 con 0x7f46440744f0 2023-07-14T11:20:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.930+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46441a00a0 con 0x7f46440744f0 2023-07-14T11:20:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.931+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46441060c0 con 0x7f46440744f0 2023-07-14T11:20:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.933+0000 7f4640ff9700 1 -- 172.21.15.161:0/3373138358 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4638010860 con 0x7f46440744f0 2023-07-14T11:20:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.934+0000 7f4640ff9700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f462c0a1150 0x7f462c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.934+0000 7f4640ff9700 1 -- 172.21.15.161:0/3373138358 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4638014070 con 0x7f46440744f0 2023-07-14T11:20:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.935+0000 7f46437fe700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f462c0a1150 0x7f462c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.935+0000 7f46437fe700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f462c0a1150 0x7f462c0a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f463400b2e0 tx=0x7f4634005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:10.938+0000 7f4640ff9700 1 -- 172.21.15.161:0/3373138358 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4638051790 con 0x7f46440744f0 2023-07-14T11:20:10.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:10 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T11:20:10.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:10 smithi182 ceph-mon[129366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:20:10.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:10 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3102977694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T11:20:10.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:10 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T11:20:10.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:10 smithi161 ceph-mon[95526]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:20:10.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:10 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3102977694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T11:20:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:10 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T11:20:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:10 smithi175 ceph-mon[129081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:20:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:10 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3102977694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T11:20:11.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.182+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f464402c850 con 0x7f46440744f0 2023-07-14T11:20:11.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.703+0000 7f4640ff9700 1 -- 172.21.15.161:0/3373138358 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4638096160 con 0x7f46440744f0 2023-07-14T11:20:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f462c0a1150 msgr2=0x7f462c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f462c0a1150 0x7f462c0a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f463400b2e0 tx=0x7f4634005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 msgr2=0x7f4644197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 0x7f4644197fa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f463800e9f0 tx=0x7f463800ed00 comp rx=0 tx=0).stop 2023-07-14T11:20:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 shutdown_connections 2023-07-14T11:20:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f462c0a1150 0x7f462c0a3600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4644107980 0x7f464419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4644072f10 0x7f4644197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 --2- 172.21.15.161:0/3373138358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46440744f0 0x7f4644197fa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 >> 172.21.15.161:0/3373138358 conn(0x7f46440fb130 msgr2=0x7f46440fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.705+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 shutdown_connections 2023-07-14T11:20:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.706+0000 7f4649dff700 1 -- 172.21.15.161:0/3373138358 wait complete. 2023-07-14T11:20:11.732 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-14T11:20:11.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.930+0000 7f911d7ff700 1 -- 172.21.15.161:0/2241791459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 msgr2=0x7f9118107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:11.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.930+0000 7f911d7ff700 1 --2- 172.21.15.161:0/2241791459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 0x7f9118107f20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f911000b0a0 tx=0x7f911000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.931+0000 7f911d7ff700 1 -- 172.21.15.161:0/2241791459 shutdown_connections 2023-07-14T11:20:11.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.931+0000 7f911d7ff700 1 --2- 172.21.15.161:0/2241791459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 0x7f9118107f20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.931+0000 7f911d7ff700 1 --2- 172.21.15.161:0/2241791459 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 0x7f91180ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.931+0000 7f911d7ff700 1 --2- 172.21.15.161:0/2241791459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9118100cd0 0x7f9118101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.931+0000 7f911d7ff700 1 -- 172.21.15.161:0/2241791459 >> 172.21.15.161:0/2241791459 conn(0x7f91180fb0e0 msgr2=0x7f91180fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:11.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.931+0000 7f911d7ff700 1 -- 172.21.15.161:0/2241791459 shutdown_connections 2023-07-14T11:20:11.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.931+0000 7f911d7ff700 1 -- 172.21.15.161:0/2241791459 wait complete. 2023-07-14T11:20:11.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 Processor -- start 2023-07-14T11:20:11.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 -- start start 2023-07-14T11:20:11.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 0x7f9118195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:11.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9118100cd0 0x7f9118195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:11.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 0x7f911819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:11.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911819d4d0 con 0x7f9118101660 2023-07-14T11:20:11.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911819d610 con 0x7f9118100cd0 2023-07-14T11:20:11.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.932+0000 7f911d7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911819d7a0 con 0x7f91180ffae0 2023-07-14T11:20:11.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f9116ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 0x7f9118195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:11.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91177fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 0x7f911819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:11.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f9116ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 0x7f9118195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:54820/0 (socket says 172.21.15.161:54820) 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f9116ffd700 1 -- 172.21.15.161:0/869629216 learned_addr learned my addr 172.21.15.161:0/869629216 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91167fc700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9118100cd0 0x7f9118195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91177fe700 1 -- 172.21.15.161:0/869629216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 msgr2=0x7f9118195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91177fe700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 0x7f9118195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91177fe700 1 -- 172.21.15.161:0/869629216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9118100cd0 msgr2=0x7f9118195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91177fe700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9118100cd0 0x7f9118195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91177fe700 1 -- 172.21.15.161:0/869629216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110009d00 con 0x7f9118101660 2023-07-14T11:20:11.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f9116ffd700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 0x7f9118195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:11.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f91177fe700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 0x7f911819cf00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f911000b070 tx=0x7f911000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:11.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f90fffff700 1 -- 172.21.15.161:0/869629216 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000e040 con 0x7f9118101660 2023-07-14T11:20:11.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f90fffff700 1 -- 172.21.15.161:0/869629216 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9110009480 con 0x7f9118101660 2023-07-14T11:20:11.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911819da20 con 0x7f9118101660 2023-07-14T11:20:11.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.933+0000 7f90fffff700 1 -- 172.21.15.161:0/869629216 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91100052a0 con 0x7f9118101660 2023-07-14T11:20:11.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.934+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911819ddf0 con 0x7f9118101660 2023-07-14T11:20:11.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.935+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9118107000 con 0x7f9118101660 2023-07-14T11:20:11.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.937+0000 7f90fffff700 1 -- 172.21.15.161:0/869629216 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f911001d480 con 0x7f9118101660 2023-07-14T11:20:11.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.938+0000 7f90fffff700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91000a1150 0x7f91000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:11.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.938+0000 7f90fffff700 1 -- 172.21.15.161:0/869629216 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f91100cdc80 con 0x7f9118101660 2023-07-14T11:20:11.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.939+0000 7f9116ffd700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91000a1150 0x7f91000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:11.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.940+0000 7f9116ffd700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91000a1150 0x7f91000a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9108005950 tx=0x7f9108009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:11.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:11.942+0000 7f90fffff700 1 -- 172.21.15.161:0/869629216 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9110097210 con 0x7f9118101660 2023-07-14T11:20:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-07-14T11:20:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T11:20:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3102977694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T11:20:11.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T11:20:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:20:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T11:20:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T11:20:11.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:11 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3373138358' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T11:20:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-07-14T11:20:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T11:20:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3102977694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T11:20:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T11:20:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:20:11.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T11:20:11.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T11:20:11.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:11 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3373138358' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T11:20:12.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-07-14T11:20:12.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T11:20:12.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3102977694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T11:20:12.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T11:20:12.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T11:20:12.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T11:20:12.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T11:20:12.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:11 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3373138358' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T11:20:12.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.194+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f911802c850 con 0x7f9118101660 2023-07-14T11:20:12.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.671+0000 7f90fffff700 1 -- 172.21.15.161:0/869629216 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f911009a830 con 0x7f9118101660 2023-07-14T11:20:12.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.673+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91000a1150 msgr2=0x7f91000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:12.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.673+0000 7f911d7ff700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91000a1150 0x7f91000a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9108005950 tx=0x7f9108009260 comp rx=0 tx=0).stop 2023-07-14T11:20:12.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.673+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 msgr2=0x7f911819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:12.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.673+0000 7f911d7ff700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 0x7f911819cf00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f911000b070 tx=0x7f911000bc80 comp rx=0 tx=0).stop 2023-07-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 shutdown_connections 2023-07-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118101660 0x7f911819cf00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91000a1150 0x7f91000a3600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f91180ffae0 0x7f9118195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 --2- 172.21.15.161:0/869629216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9118100cd0 0x7f9118195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 >> 172.21.15.161:0/869629216 conn(0x7f91180fb0e0 msgr2=0x7f9118104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 shutdown_connections 2023-07-14T11:20:12.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.674+0000 7f911d7ff700 1 -- 172.21.15.161:0/869629216 wait complete. 2023-07-14T11:20:12.696 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-07-14T11:20:12.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-14T11:20:12.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 -- 172.21.15.161:0/228995676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 msgr2=0x7f3f50101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:12.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 --2- 172.21.15.161:0/228995676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50101130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009990 tx=0x7f3f44009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 -- 172.21.15.161:0/228995676 shutdown_connections 2023-07-14T11:20:12.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 --2- 172.21.15.161:0/228995676 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3f50101670 0x7f3f50107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 --2- 172.21.15.161:0/228995676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f500ffaf0 0x7f3f500fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 --2- 172.21.15.161:0/228995676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50101130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 -- 172.21.15.161:0/228995676 >> 172.21.15.161:0/228995676 conn(0x7f3f500fb130 msgr2=0x7f3f500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:12.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 -- 172.21.15.161:0/228995676 shutdown_connections 2023-07-14T11:20:12.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.898+0000 7f3f557ff700 1 -- 172.21.15.161:0/228995676 wait complete. 2023-07-14T11:20:12.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.899+0000 7f3f557ff700 1 Processor -- start 2023-07-14T11:20:12.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.899+0000 7f3f557ff700 1 -- start start 2023-07-14T11:20:12.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f557ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f500ffaf0 0x7f3f50197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:12.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f557ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:12.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f557ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3f50101670 0x7f3f5019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:12.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f557ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5019f5e0 con 0x7f3f50100ce0 2023-07-14T11:20:12.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f557ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5019f720 con 0x7f3f500ffaf0 2023-07-14T11:20:12.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f557ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5019f8b0 con 0x7f3f50101670 2023-07-14T11:20:12.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:12.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45044/0 (socket says 172.21.15.161:45044) 2023-07-14T11:20:12.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 -- 172.21.15.161:0/501632630 learned_addr learned my addr 172.21.15.161:0/501632630 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:12.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4f7fe700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3f50101670 0x7f3f5019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:12.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4effd700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f500ffaf0 0x7f3f50197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 -- 172.21.15.161:0/501632630 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3f50101670 msgr2=0x7f3f5019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3f50101670 0x7f3f5019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 -- 172.21.15.161:0/501632630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f500ffaf0 msgr2=0x7f3f50197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f500ffaf0 0x7f3f50197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.900+0000 7f3f4e7fc700 1 -- 172.21.15.161:0/501632630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440095f0 con 0x7f3f50100ce0 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.901+0000 7f3f4e7fc700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50197fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400b620 tx=0x7f3f44005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.901+0000 7f3f37fff700 1 -- 172.21.15.161:0/501632630 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4401d070 con 0x7f3f50100ce0 2023-07-14T11:20:12.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.901+0000 7f3f37fff700 1 -- 172.21.15.161:0/501632630 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f44022470 con 0x7f3f50100ce0 2023-07-14T11:20:12.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.901+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5019fb30 con 0x7f3f50100ce0 2023-07-14T11:20:12.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.901+0000 7f3f37fff700 1 -- 172.21.15.161:0/501632630 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44017650 con 0x7f3f50100ce0 2023-07-14T11:20:12.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.901+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5019ff50 con 0x7f3f50100ce0 2023-07-14T11:20:12.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.902+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f501921d0 con 0x7f3f50100ce0 2023-07-14T11:20:12.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.904+0000 7f3f37fff700 1 -- 172.21.15.161:0/501632630 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3f440225e0 con 0x7f3f50100ce0 2023-07-14T11:20:12.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.905+0000 7f3f37fff700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3f380a1150 0x7f3f380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:12.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.906+0000 7f3f37fff700 1 -- 172.21.15.161:0/501632630 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3f440ce0f0 con 0x7f3f50100ce0 2023-07-14T11:20:12.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.906+0000 7f3f4effd700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3f380a1150 0x7f3f380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:12.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.906+0000 7f3f4effd700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3f380a1150 0x7f3f380a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f50100b40 tx=0x7f3f40007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:12.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:12.909+0000 7f3f37fff700 1 -- 172.21.15.161:0/501632630 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3f44097680 con 0x7f3f50100ce0 2023-07-14T11:20:12.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] up:active 2023-07-14T11:20:12.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3373138358' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T11:20:12.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T11:20:12.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T11:20:12.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: Cluster is now healthy 2023-07-14T11:20:12.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-07-14T11:20:12.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-07-14T11:20:12.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-14T11:20:12.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/869629216' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T11:20:12.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] up:active 2023-07-14T11:20:12.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/869629216' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T11:20:12.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:12 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:20:12.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] up:active 2023-07-14T11:20:12.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3373138358' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T11:20:12.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T11:20:12.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T11:20:12.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: Cluster is now healthy 2023-07-14T11:20:12.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-07-14T11:20:12.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-07-14T11:20:12.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-14T11:20:12.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/869629216' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T11:20:12.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] up:active 2023-07-14T11:20:12.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/869629216' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T11:20:12.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:12 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:20:13.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.144+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f50060950 con 0x7f3f380a1150 2023-07-14T11:20:13.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] up:active 2023-07-14T11:20:13.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3373138358' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T11:20:13.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T11:20:13.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T11:20:13.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: Cluster is now healthy 2023-07-14T11:20:13.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-07-14T11:20:13.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-07-14T11:20:13.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-14T11:20:13.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/869629216' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T11:20:13.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] up:active 2023-07-14T11:20:13.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/869629216' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T11:20:13.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:12 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:20:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.184+0000 7f3f37fff700 1 -- 172.21.15.161:0/501632630 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3f50060950 con 0x7f3f380a1150 2023-07-14T11:20:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3f380a1150 msgr2=0x7f3f380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3f380a1150 0x7f3f380a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f50100b40 tx=0x7f3f40007400 comp rx=0 tx=0).stop 2023-07-14T11:20:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 msgr2=0x7f3f50197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50197fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400b620 tx=0x7f3f44005740 comp rx=0 tx=0).stop 2023-07-14T11:20:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 shutdown_connections 2023-07-14T11:20:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3f380a1150 0x7f3f380a3600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3f50101670 0x7f3f5019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f500ffaf0 0x7f3f50197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 --2- 172.21.15.161:0/501632630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f50100ce0 0x7f3f50197fa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 >> 172.21.15.161:0/501632630 conn(0x7f3f500fb130 msgr2=0x7f3f501065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 shutdown_connections 2023-07-14T11:20:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.187+0000 7f3f557ff700 1 -- 172.21.15.161:0/501632630 wait complete. 2023-07-14T11:20:13.208 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-14T11:20:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 -- 172.21.15.161:0/3164981010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 msgr2=0x7fb4fc109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 --2- 172.21.15.161:0/3164981010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 0x7fb4fc109e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400b0a0 tx=0x7fb4f400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 -- 172.21.15.161:0/3164981010 shutdown_connections 2023-07-14T11:20:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 --2- 172.21.15.161:0/3164981010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 0x7fb4fc109e00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 --2- 172.21.15.161:0/3164981010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc0744f0 0x7fb4fc072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 --2- 172.21.15.161:0/3164981010 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb4fc072f10 0x7fb4fc073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 -- 172.21.15.161:0/3164981010 >> 172.21.15.161:0/3164981010 conn(0x7fb4fc0fb130 msgr2=0x7fb4fc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:13.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 -- 172.21.15.161:0/3164981010 shutdown_connections 2023-07-14T11:20:13.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.407+0000 7fb5021ff700 1 -- 172.21.15.161:0/3164981010 wait complete. 2023-07-14T11:20:13.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 Processor -- start 2023-07-14T11:20:13.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 -- start start 2023-07-14T11:20:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc072f10 0x7fb4fc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb4fc0744f0 0x7fb4fc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 0x7fb4fc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc19f6a0 con 0x7fb4fc107980 2023-07-14T11:20:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc19f7e0 con 0x7fb4fc072f10 2023-07-14T11:20:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.409+0000 7fb5021ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc19f970 con 0x7fb4fc0744f0 2023-07-14T11:20:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 0x7fb4fc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc072f10 0x7fb4fc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4faffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb4fc0744f0 0x7fb4fc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc072f10 0x7fb4fc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34962/0 (socket says 172.21.15.161:34962) 2023-07-14T11:20:13.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fb7fe700 1 -- 172.21.15.161:0/1053454852 learned_addr learned my addr 172.21.15.161:0/1053454852 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:13.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fbfff700 1 -- 172.21.15.161:0/1053454852 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb4fc0744f0 msgr2=0x7fb4fc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fbfff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb4fc0744f0 0x7fb4fc198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fbfff700 1 -- 172.21.15.161:0/1053454852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc072f10 msgr2=0x7fb4fc197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fbfff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc072f10 0x7fb4fc197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fbfff700 1 -- 172.21.15.161:0/1053454852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009520 con 0x7fb4fc107980 2023-07-14T11:20:13.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fb7fe700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc072f10 0x7fb4fc197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:13.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.410+0000 7fb4fbfff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 0x7fb4fc19f0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009fd0 tx=0x7fb4f4007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:13.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.411+0000 7fb4f8ff9700 1 -- 172.21.15.161:0/1053454852 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400e040 con 0x7fb4fc107980 2023-07-14T11:20:13.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.411+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f4009d00 con 0x7fb4fc107980 2023-07-14T11:20:13.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.411+0000 7fb4f8ff9700 1 -- 172.21.15.161:0/1053454852 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f4004c50 con 0x7fb4fc107980 2023-07-14T11:20:13.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.411+0000 7fb4f8ff9700 1 -- 172.21.15.161:0/1053454852 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f401b6c0 con 0x7fb4fc107980 2023-07-14T11:20:13.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.411+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc19fe80 con 0x7fb4fc107980 2023-07-14T11:20:13.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.412+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc1060c0 con 0x7fb4fc107980 2023-07-14T11:20:13.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.414+0000 7fb4f8ff9700 1 -- 172.21.15.161:0/1053454852 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb4f401d470 con 0x7fb4fc107980 2023-07-14T11:20:13.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.415+0000 7fb4f8ff9700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4e40a1150 0x7fb4e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.415+0000 7fb4f8ff9700 1 -- 172.21.15.161:0/1053454852 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb4f40cdd60 con 0x7fb4fc107980 2023-07-14T11:20:13.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.415+0000 7fb4fb7fe700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4e40a1150 0x7fb4e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.416+0000 7fb4fb7fe700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4e40a1150 0x7fb4e40a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec011550 tx=0x7fb4ec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:13.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.418+0000 7fb4f8ff9700 1 -- 172.21.15.161:0/1053454852 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb4f4097350 con 0x7fb4fc107980 2023-07-14T11:20:13.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:20:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:20:13.151+0000 7f6c525ca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:20:13.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:20:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:20:13.151+0000 7f6c525ca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:20:13.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:20:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:20:13.151+0000 7f6c525ca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:20:13.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:20:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:20:13.151+0000 7f6c525ca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:20:13.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:20:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:20:13.151+0000 7f6c525ca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:20:13.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.648+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4fc060950 con 0x7fb4e40a1150 2023-07-14T11:20:13.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.652+0000 7fb4f8ff9700 1 -- 172.21.15.161:0/1053454852 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fb4fc060950 con 0x7fb4e40a1150 2023-07-14T11:20:13.656 INFO:teuthology.orchestra.run.smithi161.stdout:/volumes/_nogroup/sv_0/82797a72-16c1-4f9b-82c0-afd79a56d986 2023-07-14T11:20:13.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4e40a1150 msgr2=0x7fb4e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4e40a1150 0x7fb4e40a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec011550 tx=0x7fb4ec009260 comp rx=0 tx=0).stop 2023-07-14T11:20:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 msgr2=0x7fb4fc19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 0x7fb4fc19f0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009fd0 tx=0x7fb4f4007930 comp rx=0 tx=0).stop 2023-07-14T11:20:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 shutdown_connections 2023-07-14T11:20:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4fc107980 0x7fb4fc19f0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb4e40a1150 0x7fb4e40a3600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4fc072f10 0x7fb4fc197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 --2- 172.21.15.161:0/1053454852 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb4fc0744f0 0x7fb4fc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 >> 172.21.15.161:0/1053454852 conn(0x7fb4fc0fb130 msgr2=0x7fb4fc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 shutdown_connections 2023-07-14T11:20:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.655+0000 7fb5021ff700 1 -- 172.21.15.161:0/1053454852 wait complete. 2023-07-14T11:20:13.677 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-14T11:20:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1963943958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa0101670 msgr2=0x7f9fa0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1963943958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa0101670 0x7f9fa0107f30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800b0a0 tx=0x7f9f9800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1963943958 shutdown_connections 2023-07-14T11:20:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1963943958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa0101670 0x7f9fa0107f30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1963943958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa00ffaf0 0x7f9fa00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1963943958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9fa0100ce0 0x7f9fa0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1963943958 >> 172.21.15.161:0/1963943958 conn(0x7f9fa00fb130 msgr2=0x7f9fa00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1963943958 shutdown_connections 2023-07-14T11:20:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.877+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1963943958 wait complete. 2023-07-14T11:20:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.878+0000 7f9fa5bff700 1 Processor -- start 2023-07-14T11:20:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.878+0000 7f9fa5bff700 1 -- start start 2023-07-14T11:20:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9fa5bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa00ffaf0 0x7f9fa0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9fa5bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0100ce0 0x7f9fa0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9fa5bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9fa0101670 0x7f9fa019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9fa5bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa019f6a0 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9fa5bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa019f7e0 con 0x7f9fa0100ce0 2023-07-14T11:20:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9fa5bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa019f970 con 0x7f9fa0101670 2023-07-14T11:20:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa00ffaf0 0x7f9fa0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa00ffaf0 0x7f9fa0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45084/0 (socket says 172.21.15.161:45084) 2023-07-14T11:20:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 -- 172.21.15.161:0/1768450577 learned_addr learned my addr 172.21.15.161:0/1768450577 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9effd700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0100ce0 0x7f9fa0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9ffff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9fa0101670 0x7f9fa019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 -- 172.21.15.161:0/1768450577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9fa0101670 msgr2=0x7f9fa019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9fa0101670 0x7f9fa019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 -- 172.21.15.161:0/1768450577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0100ce0 msgr2=0x7f9fa0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0100ce0 0x7f9fa0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9f7fe700 1 -- 172.21.15.161:0/1768450577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f90009520 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.879+0000 7f9f9ffff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9fa0101670 0x7f9fa019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.880+0000 7f9f9f7fe700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa00ffaf0 0x7f9fa0197b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000ea50 tx=0x7f9f9000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.880+0000 7f9f9cff9700 1 -- 172.21.15.161:0/1768450577 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000cbd0 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.880+0000 7f9f9cff9700 1 -- 172.21.15.161:0/1768450577 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f9000cd30 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.880+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f98009d00 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.880+0000 7f9f9cff9700 1 -- 172.21.15.161:0/1768450577 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90018680 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.880+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa019ffd0 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.881+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa010b320 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.883+0000 7f9f9cff9700 1 -- 172.21.15.161:0/1768450577 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9f900188a0 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.884+0000 7f9f9cff9700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9f880a1150 0x7f9f880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.884+0000 7f9f9cff9700 1 -- 172.21.15.161:0/1768450577 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9f90014070 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.885+0000 7f9f9effd700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9f880a1150 0x7f9f880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.885+0000 7f9f9effd700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9f880a1150 0x7f9f880a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009c60 tx=0x7f9f94009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:13.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:13.888+0000 7f9f9cff9700 1 -- 172.21.15.161:0/1768450577 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9f90096580 con 0x7f9fa00ffaf0 2023-07-14T11:20:13.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:13 smithi182 ceph-mon[129366]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-07-14T11:20:13.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:13 smithi161 ceph-mon[95526]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-07-14T11:20:14.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.129+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fa0107540 con 0x7f9f880a1150 2023-07-14T11:20:14.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.153+0000 7f9f9cff9700 1 -- 172.21.15.161:0/1768450577 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9fa0107540 con 0x7f9f880a1150 2023-07-14T11:20:14.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.155+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9f880a1150 msgr2=0x7f9f880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.155+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9f880a1150 0x7f9f880a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009c60 tx=0x7f9f94009260 comp rx=0 tx=0).stop 2023-07-14T11:20:14.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.155+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa00ffaf0 msgr2=0x7f9fa0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.155+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa00ffaf0 0x7f9fa0197b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000ea50 tx=0x7f9f9000ed60 comp rx=0 tx=0).stop 2023-07-14T11:20:14.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 shutdown_connections 2023-07-14T11:20:14.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fa00ffaf0 0x7f9fa0197b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9f880a1150 0x7f9f880a3600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9fa0101670 0x7f9fa019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 --2- 172.21.15.161:0/1768450577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0100ce0 0x7f9fa0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 >> 172.21.15.161:0/1768450577 conn(0x7f9fa00fb130 msgr2=0x7f9fa00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:14.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 shutdown_connections 2023-07-14T11:20:14.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.156+0000 7f9fa5bff700 1 -- 172.21.15.161:0/1768450577 wait complete. 2023-07-14T11:20:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:13 smithi175 ceph-mon[129081]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-07-14T11:20:14.178 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-14T11:20:14.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 -- 172.21.15.161:0/2509073349 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45001005c0 msgr2=0x7f45001029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 --2- 172.21.15.161:0/2509073349 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45001005c0 0x7f45001029a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f44f40098f0 tx=0x7f44f4009c00 comp rx=0 tx=0).stop 2023-07-14T11:20:14.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 -- 172.21.15.161:0/2509073349 shutdown_connections 2023-07-14T11:20:14.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 --2- 172.21.15.161:0/2509073349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4500105800 0x7f4500107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 --2- 172.21.15.161:0/2509073349 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45001005c0 0x7f45001029a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 --2- 172.21.15.161:0/2509073349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4500102ee0 0x7f45001052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 -- 172.21.15.161:0/2509073349 >> 172.21.15.161:0/2509073349 conn(0x7f45000fa000 msgr2=0x7f45000fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:14.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 -- 172.21.15.161:0/2509073349 shutdown_connections 2023-07-14T11:20:14.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.376+0000 7f4506bff700 1 -- 172.21.15.161:0/2509073349 wait complete. 2023-07-14T11:20:14.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.377+0000 7f4506bff700 1 Processor -- start 2023-07-14T11:20:14.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.378+0000 7f4506bff700 1 -- start start 2023-07-14T11:20:14.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.378+0000 7f4506bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45001005c0 0x7f45000fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.378+0000 7f4506bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4500102ee0 0x7f45000fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.378+0000 7f4506bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4500105800 0x7f45000feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.378+0000 7f4506bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000ff250 con 0x7f45001005c0 2023-07-14T11:20:14.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.378+0000 7f4506bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000ff3c0 con 0x7f4500102ee0 2023-07-14T11:20:14.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.378+0000 7f4506bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450019f8c0 con 0x7f4500105800 2023-07-14T11:20:14.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45001005c0 0x7f45000fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45001005c0 0x7f45000fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45106/0 (socket says 172.21.15.161:45106) 2023-07-14T11:20:14.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450519c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4500105800 0x7f45000feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 -- 172.21.15.161:0/1655506763 learned_addr learned my addr 172.21.15.161:0/1655506763 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:14.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f44fffff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4500102ee0 0x7f45000fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 -- 172.21.15.161:0/1655506763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4500105800 msgr2=0x7f45000feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4500105800 0x7f45000feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 -- 172.21.15.161:0/1655506763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4500102ee0 msgr2=0x7f45000fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4500102ee0 0x7f45000fe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 -- 172.21.15.161:0/1655506763 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec0095f0 con 0x7f45001005c0 2023-07-14T11:20:14.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450519c700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4500105800 0x7f45000feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:14.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f450499b700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45001005c0 0x7f45000fff30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009230 tx=0x7f44f400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:14.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.379+0000 7f44fdffb700 1 -- 172.21.15.161:0/1655506763 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f401c070 con 0x7f45001005c0 2023-07-14T11:20:14.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.380+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f4009520 con 0x7f45001005c0 2023-07-14T11:20:14.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.380+0000 7f44fdffb700 1 -- 172.21.15.161:0/1655506763 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f400f860 con 0x7f45001005c0 2023-07-14T11:20:14.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.380+0000 7f44fdffb700 1 -- 172.21.15.161:0/1655506763 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4004b30 con 0x7f45001005c0 2023-07-14T11:20:14.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.380+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450019fde0 con 0x7f45001005c0 2023-07-14T11:20:14.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.381+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45000fbb70 con 0x7f45001005c0 2023-07-14T11:20:14.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.383+0000 7f44fdffb700 1 -- 172.21.15.161:0/1655506763 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f44f4020070 con 0x7f45001005c0 2023-07-14T11:20:14.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.384+0000 7f44fdffb700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44f00a1150 0x7f44f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.384+0000 7f44fdffb700 1 -- 172.21.15.161:0/1655506763 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f44f40cd9d0 con 0x7f45001005c0 2023-07-14T11:20:14.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.384+0000 7f44fffff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44f00a1150 0x7f44f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.385+0000 7f44fffff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44f00a1150 0x7f44f00a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f44ec005fd0 tx=0x7f44ec009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.388+0000 7f44fdffb700 1 -- 172.21.15.161:0/1655506763 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f44f4096f60 con 0x7f45001005c0 2023-07-14T11:20:14.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.622+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4500060950 con 0x7f44f00a1150 2023-07-14T11:20:14.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.626+0000 7f44fdffb700 1 -- 172.21.15.161:0/1655506763 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f4500060950 con 0x7f44f00a1150 2023-07-14T11:20:14.629 INFO:teuthology.orchestra.run.smithi161.stdout:/volumes/_nogroup/sv_1/fbc93191-b860-463a-be29-85439d88d9de 2023-07-14T11:20:14.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.628+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44f00a1150 msgr2=0x7f44f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.628+0000 7f4506bff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44f00a1150 0x7f44f00a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f44ec005fd0 tx=0x7f44ec009310 comp rx=0 tx=0).stop 2023-07-14T11:20:14.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.628+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45001005c0 msgr2=0x7f45000fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.628+0000 7f4506bff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45001005c0 0x7f45000fff30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009230 tx=0x7f44f400b790 comp rx=0 tx=0).stop 2023-07-14T11:20:14.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 shutdown_connections 2023-07-14T11:20:14.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45001005c0 0x7f45000fff30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44f00a1150 0x7f44f00a3600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4500105800 0x7f45000feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 --2- 172.21.15.161:0/1655506763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4500102ee0 0x7f45000fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 >> 172.21.15.161:0/1655506763 conn(0x7f45000fa000 msgr2=0x7f4500101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:14.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 shutdown_connections 2023-07-14T11:20:14.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.629+0000 7f4506bff700 1 -- 172.21.15.161:0/1655506763 wait complete. 2023-07-14T11:20:14.651 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:20:14.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 -- 172.21.15.161:0/4036104653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 msgr2=0x7f41cc072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 --2- 172.21.15.161:0/4036104653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 0x7f41cc072940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009990 tx=0x7f41c8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 -- 172.21.15.161:0/4036104653 shutdown_connections 2023-07-14T11:20:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 --2- 172.21.15.161:0/4036104653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc1079a0 0x7f41cc109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 --2- 172.21.15.161:0/4036104653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 0x7f41cc072940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 --2- 172.21.15.161:0/4036104653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41cc072f10 0x7f41cc073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 -- 172.21.15.161:0/4036104653 >> 172.21.15.161:0/4036104653 conn(0x7f41cc0fb110 msgr2=0x7f41cc0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 -- 172.21.15.161:0/4036104653 shutdown_connections 2023-07-14T11:20:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.859+0000 7f41d41ff700 1 -- 172.21.15.161:0/4036104653 wait complete. 2023-07-14T11:20:14.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 Processor -- start 2023-07-14T11:20:14.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 -- start start 2023-07-14T11:20:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc072f10 0x7f41cc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 0x7f41cc197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41cc1079a0 0x7f41cc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19f550 con 0x7f41cc1079a0 2023-07-14T11:20:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19f690 con 0x7f41cc072f10 2023-07-14T11:20:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.860+0000 7f41d41ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19f820 con 0x7f41cc0744f0 2023-07-14T11:20:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc072f10 0x7f41cc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc072f10 0x7f41cc1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35020/0 (socket says 172.21.15.161:35020) 2023-07-14T11:20:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 -- 172.21.15.161:0/2922587973 learned_addr learned my addr 172.21.15.161:0/2922587973 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d279c700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41cc1079a0 0x7f41cc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d179a700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 0x7f41cc197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 -- 172.21.15.161:0/2922587973 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 msgr2=0x7f41cc197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 0x7f41cc197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 -- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41cc1079a0 msgr2=0x7f41cc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41cc1079a0 0x7f41cc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 -- 172.21.15.161:0/2922587973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc009520 con 0x7f41cc072f10 2023-07-14T11:20:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.861+0000 7f41d1f9b700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc072f10 0x7f41cc1979d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f41c8004740 tx=0x7f41c8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.862+0000 7f41c2ffd700 1 -- 172.21.15.161:0/2922587973 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c801d070 con 0x7f41cc072f10 2023-07-14T11:20:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.862+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c80095f0 con 0x7f41cc072f10 2023-07-14T11:20:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.862+0000 7f41c2ffd700 1 -- 172.21.15.161:0/2922587973 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c8004830 con 0x7f41cc072f10 2023-07-14T11:20:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.862+0000 7f41c2ffd700 1 -- 172.21.15.161:0/2922587973 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80053c0 con 0x7f41cc072f10 2023-07-14T11:20:14.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.862+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc19fce0 con 0x7f41cc072f10 2023-07-14T11:20:14.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.863+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc1060e0 con 0x7f41cc072f10 2023-07-14T11:20:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.865+0000 7f41c2ffd700 1 -- 172.21.15.161:0/2922587973 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f41c8005580 con 0x7f41cc072f10 2023-07-14T11:20:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.866+0000 7f41c2ffd700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f41b80a1220 0x7f41b80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.866+0000 7f41c2ffd700 1 -- 172.21.15.161:0/2922587973 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f41c80cdf10 con 0x7f41cc072f10 2023-07-14T11:20:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.866+0000 7f41d179a700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f41b80a1220 0x7f41b80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.867+0000 7f41d179a700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f41b80a1220 0x7f41b80a36d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009df0 tx=0x7f41bc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:14.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:14.868+0000 7f41c2ffd700 1 -- 172.21.15.161:0/2922587973 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f41c8097220 con 0x7f41cc072f10 2023-07-14T11:20:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-14T11:20:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2023-07-14T11:20:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-07-14T11:20:14.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-07-14T11:20:14.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-07-14T11:20:14.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-07-14T11:20:14.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-07-14T11:20:14.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-07-14T11:20:14.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-07-14T11:20:14.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:14 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]: dispatch 2023-07-14T11:20:15.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.113+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41cc02c850 con 0x7f41cc072f10 2023-07-14T11:20:15.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.114+0000 7f41c2ffd700 1 -- 172.21.15.161:0/2922587973 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f41c80a0020 con 0x7f41cc072f10 2023-07-14T11:20:15.118 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:15.119 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:20:15.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f41b80a1220 msgr2=0x7f41b80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f41b80a1220 0x7f41b80a36d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009df0 tx=0x7f41bc009260 comp rx=0 tx=0).stop 2023-07-14T11:20:15.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc072f10 msgr2=0x7f41cc1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc072f10 0x7f41cc1979d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f41c8004740 tx=0x7f41c8009580 comp rx=0 tx=0).stop 2023-07-14T11:20:15.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 shutdown_connections 2023-07-14T11:20:15.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41cc1079a0 0x7f41cc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f41b80a1220 0x7f41b80a36d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41cc072f10 0x7f41cc1979d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 --2- 172.21.15.161:0/2922587973 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41cc0744f0 0x7f41cc197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 >> 172.21.15.161:0/2922587973 conn(0x7f41cc0fb110 msgr2=0x7f41cc0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:15.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 shutdown_connections 2023-07-14T11:20:15.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.117+0000 7f41d41ff700 1 -- 172.21.15.161:0/2922587973 wait complete. 2023-07-14T11:20:15.124 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:20:15.140 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:20:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-14T11:20:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2023-07-14T11:20:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-07-14T11:20:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-07-14T11:20:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-07-14T11:20:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-07-14T11:20:15.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-07-14T11:20:15.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-07-14T11:20:15.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-07-14T11:20:15.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:14 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]: dispatch 2023-07-14T11:20:15.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:15.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:15.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-14T11:20:15.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2023-07-14T11:20:15.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-07-14T11:20:15.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-07-14T11:20:15.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-07-14T11:20:15.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-07-14T11:20:15.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-07-14T11:20:15.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-07-14T11:20:15.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-07-14T11:20:15.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:14 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]: dispatch 2023-07-14T11:20:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.334+0000 7f36dd9ff700 1 -- 172.21.15.161:0/195696134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d80ffb00 msgr2=0x7f36d80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.334+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/195696134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d80ffb00 0x7f36d80fff10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009990 tx=0x7f36c8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.335+0000 7f36dd9ff700 1 -- 172.21.15.161:0/195696134 shutdown_connections 2023-07-14T11:20:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.335+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/195696134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8101680 0x7f36d8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.335+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/195696134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d80ffb00 0x7f36d80fff10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.335+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/195696134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d8100cf0 0x7f36d8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.335+0000 7f36dd9ff700 1 -- 172.21.15.161:0/195696134 >> 172.21.15.161:0/195696134 conn(0x7f36d80fb0e0 msgr2=0x7f36d80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.335+0000 7f36dd9ff700 1 -- 172.21.15.161:0/195696134 shutdown_connections 2023-07-14T11:20:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.335+0000 7f36dd9ff700 1 -- 172.21.15.161:0/195696134 wait complete. 2023-07-14T11:20:15.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 Processor -- start 2023-07-14T11:20:15.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 -- start start 2023-07-14T11:20:15.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d80ffb00 0x7f36d8195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d8100cf0 0x7f36d8195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8101680 0x7f36d819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d819d3c0 con 0x7f36d8101680 2023-07-14T11:20:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d819d500 con 0x7f36d8100cf0 2023-07-14T11:20:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36dd9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d819d690 con 0x7f36d80ffb00 2023-07-14T11:20:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36d6ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d80ffb00 0x7f36d8195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36d77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8101680 0x7f36d819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36d6ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d80ffb00 0x7f36d8195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:54980/0 (socket says 172.21.15.161:54980) 2023-07-14T11:20:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.336+0000 7f36d6ffd700 1 -- 172.21.15.161:0/1300013126 learned_addr learned my addr 172.21.15.161:0/1300013126 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d67fc700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d8100cf0 0x7f36d8195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d6ffd700 1 -- 172.21.15.161:0/1300013126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d8100cf0 msgr2=0x7f36d8195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d6ffd700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d8100cf0 0x7f36d8195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d6ffd700 1 -- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8101680 msgr2=0x7f36d819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d6ffd700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8101680 0x7f36d819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d6ffd700 1 -- 172.21.15.161:0/1300013126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc009520 con 0x7f36d80ffb00 2023-07-14T11:20:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d77fe700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8101680 0x7f36d819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36d6ffd700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d80ffb00 0x7f36d8195840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36c8004730 tx=0x7f36c8004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36dc9fd700 1 -- 172.21.15.161:0/1300013126 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c801d070 con 0x7f36d80ffb00 2023-07-14T11:20:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36dc9fd700 1 -- 172.21.15.161:0/1300013126 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36c8021410 con 0x7f36d80ffb00 2023-07-14T11:20:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c80095f0 con 0x7f36d80ffb00 2023-07-14T11:20:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.337+0000 7f36dc9fd700 1 -- 172.21.15.161:0/1300013126 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8017650 con 0x7f36d80ffb00 2023-07-14T11:20:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.338+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d819dba0 con 0x7f36d80ffb00 2023-07-14T11:20:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.338+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d810b320 con 0x7f36d80ffb00 2023-07-14T11:20:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.341+0000 7f36dc9fd700 1 -- 172.21.15.161:0/1300013126 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f36c8021580 con 0x7f36d80ffb00 2023-07-14T11:20:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.342+0000 7f36dc9fd700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f36c00a1220 0x7f36c00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.342+0000 7f36dc9fd700 1 -- 172.21.15.161:0/1300013126 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f36c80ce340 con 0x7f36d80ffb00 2023-07-14T11:20:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.342+0000 7f36d67fc700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f36c00a1220 0x7f36c00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.343+0000 7f36d67fc700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f36c00a1220 0x7f36c00a36d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009df0 tx=0x7f36cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.344+0000 7f36dc9fd700 1 -- 172.21.15.161:0/1300013126 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f36c80976b0 con 0x7f36d80ffb00 2023-07-14T11:20:15.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.544+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36d810b4e0 con 0x7f36d80ffb00 2023-07-14T11:20:15.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.546+0000 7f36dc9fd700 1 -- 172.21.15.161:0/1300013126 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24461 (secure 0 0 0) 0x7f36c809acd0 con 0x7f36d80ffb00 2023-07-14T11:20:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:15.550 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":89,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:20:14.705816+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:20:15.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f36c00a1220 msgr2=0x7f36c00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f36c00a1220 0x7f36c00a36d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009df0 tx=0x7f36cc009260 comp rx=0 tx=0).stop 2023-07-14T11:20:15.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d80ffb00 msgr2=0x7f36d8195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d80ffb00 0x7f36d8195840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36c8004730 tx=0x7f36c8004760 comp rx=0 tx=0).stop 2023-07-14T11:20:15.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 shutdown_connections 2023-07-14T11:20:15.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8101680 0x7f36d819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f36c00a1220 0x7f36c00a36d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f36d80ffb00 0x7f36d8195840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 --2- 172.21.15.161:0/1300013126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d8100cf0 0x7f36d8195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 >> 172.21.15.161:0/1300013126 conn(0x7f36d80fb0e0 msgr2=0x7f36d80fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:15.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 shutdown_connections 2023-07-14T11:20:15.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.549+0000 7f36dd9ff700 1 -- 172.21.15.161:0/1300013126 wait complete. 2023-07-14T11:20:15.571 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T11:20:15.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:15.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 -- 172.21.15.161:0/1698942153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8101680 msgr2=0x7f94c8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 --2- 172.21.15.161:0/1698942153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8101680 0x7f94c8107f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94c000b0a0 tx=0x7f94c000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 -- 172.21.15.161:0/1698942153 shutdown_connections 2023-07-14T11:20:15.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 --2- 172.21.15.161:0/1698942153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8101680 0x7f94c8107f40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 --2- 172.21.15.161:0/1698942153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94c80ffb00 0x7f94c80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 --2- 172.21.15.161:0/1698942153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c8100cf0 0x7f94c8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 -- 172.21.15.161:0/1698942153 >> 172.21.15.161:0/1698942153 conn(0x7f94c80fb0e0 msgr2=0x7f94c80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:15.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 -- 172.21.15.161:0/1698942153 shutdown_connections 2023-07-14T11:20:15.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.774+0000 7f94d03ff700 1 -- 172.21.15.161:0/1698942153 wait complete. 2023-07-14T11:20:15.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.775+0000 7f94d03ff700 1 Processor -- start 2023-07-14T11:20:15.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.775+0000 7f94d03ff700 1 -- start start 2023-07-14T11:20:15.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94d03ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c80ffb00 0x7f94c8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94d03ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c8100cf0 0x7f94c8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94d03ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94c8101680 0x7f94c819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94d03ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c819f660 con 0x7f94c80ffb00 2023-07-14T11:20:15.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94d03ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c819f7a0 con 0x7f94c8100cf0 2023-07-14T11:20:15.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94d03ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c819f930 con 0x7f94c8101680 2023-07-14T11:20:15.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c80ffb00 0x7f94c8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c80ffb00 0x7f94c8197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45146/0 (socket says 172.21.15.161:45146) 2023-07-14T11:20:15.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 -- 172.21.15.161:0/2952052369 learned_addr learned my addr 172.21.15.161:0/2952052369 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:15.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce99c700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94c8101680 0x7f94c819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94cd99a700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c8100cf0 0x7f94c8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 -- 172.21.15.161:0/2952052369 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94c8101680 msgr2=0x7f94c819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94c8101680 0x7f94c819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 -- 172.21.15.161:0/2952052369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c8100cf0 msgr2=0x7f94c8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c8100cf0 0x7f94c8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce19b700 1 -- 172.21.15.161:0/2952052369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0009d00 con 0x7f94c80ffb00 2023-07-14T11:20:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.776+0000 7f94ce99c700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94c8101680 0x7f94c819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.777+0000 7f94ce19b700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c80ffb00 0x7f94c8197ae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e970 tx=0x7f94c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.777+0000 7f94bf7fe700 1 -- 172.21.15.161:0/2952052369 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4009790 con 0x7f94c80ffb00 2023-07-14T11:20:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.777+0000 7f94bf7fe700 1 -- 172.21.15.161:0/2952052369 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c400c7e0 con 0x7f94c80ffb00 2023-07-14T11:20:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.777+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c819fc10 con 0x7f94c80ffb00 2023-07-14T11:20:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.777+0000 7f94bf7fe700 1 -- 172.21.15.161:0/2952052369 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4003c70 con 0x7f94c80ffb00 2023-07-14T11:20:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.777+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c81a0120 con 0x7f94c80ffb00 2023-07-14T11:20:15.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.778+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c810b330 con 0x7f94c80ffb00 2023-07-14T11:20:15.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.780+0000 7f94bf7fe700 1 -- 172.21.15.161:0/2952052369 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f94c400cc50 con 0x7f94c80ffb00 2023-07-14T11:20:15.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.781+0000 7f94bf7fe700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94b40a1220 0x7f94b40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.781+0000 7f94bf7fe700 1 -- 172.21.15.161:0/2952052369 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f94c4014070 con 0x7f94c80ffb00 2023-07-14T11:20:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.781+0000 7f94cd99a700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94b40a1220 0x7f94b40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.782+0000 7f94cd99a700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94b40a1220 0x7f94b40a36d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009c60 tx=0x7f94b8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:15.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.785+0000 7f94bf7fe700 1 -- 172.21.15.161:0/2952052369 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f94c4096220 con 0x7f94c80ffb00 2023-07-14T11:20:15.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.978+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94c8107c10 con 0x7f94b40a1220 2023-07-14T11:20:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: pgmap v288: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.2 KiB/s wr, 4 op/s 2023-07-14T11:20:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-14T11:20:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2023-07-14T11:20:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-07-14T11:20:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-07-14T11:20:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-07-14T11:20:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-07-14T11:20:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-07-14T11:20:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-07-14T11:20:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-07-14T11:20:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]': finished 2023-07-14T11:20:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T11:20:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T11:20:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2922587973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:20:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:15 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1300013126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:15.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.993+0000 7f94bf7fe700 1 -- 172.21.15.161:0/2952052369 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422824 (secure 0 0 0) 0x7f94c8107c10 con 0x7f94b40a1220 2023-07-14T11:20:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:16.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94b40a1220 msgr2=0x7f94b40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94b40a1220 0x7f94b40a36d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009c60 tx=0x7f94b8009260 comp rx=0 tx=0).stop 2023-07-14T11:20:16.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c80ffb00 msgr2=0x7f94c8197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c80ffb00 0x7f94c8197ae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e970 tx=0x7f94c400ec80 comp rx=0 tx=0).stop 2023-07-14T11:20:16.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 shutdown_connections 2023-07-14T11:20:16.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c80ffb00 0x7f94c8197ae0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f94b40a1220 0x7f94b40a36d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f94c8101680 0x7f94c819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 --2- 172.21.15.161:0/2952052369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c8100cf0 0x7f94c8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 >> 172.21.15.161:0/2952052369 conn(0x7f94c80fb0e0 msgr2=0x7f94c81064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:16.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 shutdown_connections 2023-07-14T11:20:16.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:15.999+0000 7f94d03ff700 1 -- 172.21.15.161:0/2952052369 wait complete. 2023-07-14T11:20:16.008 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:16.022 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2023-07-14T11:20:15.721622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597217,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":156,"num_read_kb":137,"num_write":303,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334880,"kb_used_data":11456,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606944,"statfs":{"total":1151940427776,"available":1151597510656,"internally_reserved":0,"allocated":11730944,"data_stored":7418003,"data_compressed":250499,"data_compressed_allocated":335872,"data_compressed_original":2170880,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2257,"num_objects":35,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.620460"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080953+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:19:47.080953+0000","last_peered":"2023-07-14T11:19:47.080953+0000","last_clean":"2023-07-14T11:19:47.080953+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:19:47.080953+0000","last_undegraded":"2023-07-14T11:19:47.080953+0000","last_fullsized":"2023-07-14T11:19:47.080953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:57.564463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737248+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:19:47.737248+0000","last_peered":"2023-07-14T11:19:47.737248+0000","last_clean":"2023-07-14T11:19:47.737248+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:19:47.737248+0000","last_undegraded":"2023-07-14T11:19:47.737248+0000","last_fullsized":"2023-07-14T11:19:47.737248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:19.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737244+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:19:47.737244+0000","last_peered":"2023-07-14T11:19:47.737244+0000","last_clean":"2023-07-14T11:19:47.737244+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:19:47.737244+0000","last_undegraded":"2023-07-14T11:19:47.737244+0000","last_fullsized":"2023-07-14T11:19:47.737244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:47.796849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968258+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:19:47.968258+0000","last_peered":"2023-07-14T11:19:47.968258+0000","last_clean":"2023-07-14T11:19:47.968258+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:19:47.968258+0000","last_undegraded":"2023-07-14T11:19:47.968258+0000","last_fullsized":"2023-07-14T11:19:47.968258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:18:31.399322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717211+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:14.717211+0000","last_peered":"2023-07-14T11:20:14.717211+0000","last_clean":"2023-07-14T11:20:14.717211+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:14.717211+0000","last_undegraded":"2023-07-14T11:20:14.717211+0000","last_fullsized":"2023-07-14T11:20:14.717211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:45:02.418574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081056+0000","last_change":"2023-07-14T11:19:42.092136+0000","last_active":"2023-07-14T11:19:47.081056+0000","last_peered":"2023-07-14T11:19:47.081056+0000","last_clean":"2023-07-14T11:19:47.081056+0000","last_became_active":"2023-07-14T11:19:42.091820+0000","last_became_peered":"2023-07-14T11:19:42.091820+0000","last_unstale":"2023-07-14T11:19:47.081056+0000","last_undegraded":"2023-07-14T11:19:47.081056+0000","last_fullsized":"2023-07-14T11:19:47.081056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:01.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968087+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:19:47.968087+0000","last_peered":"2023-07-14T11:19:47.968087+0000","last_clean":"2023-07-14T11:19:47.968087+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:19:47.968087+0000","last_undegraded":"2023-07-14T11:19:47.968087+0000","last_fullsized":"2023-07-14T11:19:47.968087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:21.428012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967192+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:19:47.967192+0000","last_peered":"2023-07-14T11:19:47.967192+0000","last_clean":"2023-07-14T11:19:47.967192+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:19:47.967192+0000","last_undegraded":"2023-07-14T11:19:47.967192+0000","last_fullsized":"2023-07-14T11:19:47.967192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:38.965332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969763+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:19:47.969763+0000","last_peered":"2023-07-14T11:19:47.969763+0000","last_clean":"2023-07-14T11:19:47.969763+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:19:47.969763+0000","last_undegraded":"2023-07-14T11:19:47.969763+0000","last_fullsized":"2023-07-14T11:19:47.969763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:16.828892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081420+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:19:47.081420+0000","last_peered":"2023-07-14T11:19:47.081420+0000","last_clean":"2023-07-14T11:19:47.081420+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:19:47.081420+0000","last_undegraded":"2023-07-14T11:19:47.081420+0000","last_fullsized":"2023-07-14T11:19:47.081420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:51.712868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969101+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:19:47.969101+0000","last_peered":"2023-07-14T11:19:47.969101+0000","last_clean":"2023-07-14T11:19:47.969101+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:19:47.969101+0000","last_undegraded":"2023-07-14T11:19:47.969101+0000","last_fullsized":"2023-07-14T11:19:47.969101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:48:50.962800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968204+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:19:47.968204+0000","last_peered":"2023-07-14T11:19:47.968204+0000","last_clean":"2023-07-14T11:19:47.968204+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:19:47.968204+0000","last_undegraded":"2023-07-14T11:19:47.968204+0000","last_fullsized":"2023-07-14T11:19:47.968204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:47.264465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727198+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:14.727198+0000","last_peered":"2023-07-14T11:20:14.727198+0000","last_clean":"2023-07-14T11:20:14.727198+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:14.727198+0000","last_undegraded":"2023-07-14T11:20:14.727198+0000","last_fullsized":"2023-07-14T11:20:14.727198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:42.155482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736832+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:19:47.736832+0000","last_peered":"2023-07-14T11:19:47.736832+0000","last_clean":"2023-07-14T11:19:47.736832+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:19:47.736832+0000","last_undegraded":"2023-07-14T11:19:47.736832+0000","last_fullsized":"2023-07-14T11:19:47.736832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:26.314511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736956+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:19:47.736956+0000","last_peered":"2023-07-14T11:19:47.736956+0000","last_clean":"2023-07-14T11:19:47.736956+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:19:47.736956+0000","last_undegraded":"2023-07-14T11:19:47.736956+0000","last_fullsized":"2023-07-14T11:19:47.736956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:10:23.738358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969113+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:19:47.969113+0000","last_peered":"2023-07-14T11:19:47.969113+0000","last_clean":"2023-07-14T11:19:47.969113+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:19:47.969113+0000","last_undegraded":"2023-07-14T11:19:47.969113+0000","last_fullsized":"2023-07-14T11:19:47.969113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:23:46.442897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081137+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:19:47.081137+0000","last_peered":"2023-07-14T11:19:47.081137+0000","last_clean":"2023-07-14T11:19:47.081137+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:19:47.081137+0000","last_undegraded":"2023-07-14T11:19:47.081137+0000","last_fullsized":"2023-07-14T11:19:47.081137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:51.121664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082305+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:19:47.082305+0000","last_peered":"2023-07-14T11:19:47.082305+0000","last_clean":"2023-07-14T11:19:47.082305+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:19:47.082305+0000","last_undegraded":"2023-07-14T11:19:47.082305+0000","last_fullsized":"2023-07-14T11:19:47.082305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:07:45.552513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197608+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:19:47.197608+0000","last_peered":"2023-07-14T11:19:47.197608+0000","last_clean":"2023-07-14T11:19:47.197608+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:19:47.197608+0000","last_undegraded":"2023-07-14T11:19:47.197608+0000","last_fullsized":"2023-07-14T11:19:47.197608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:44.466407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197036+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:19:47.197036+0000","last_peered":"2023-07-14T11:19:47.197036+0000","last_clean":"2023-07-14T11:19:47.197036+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:19:47.197036+0000","last_undegraded":"2023-07-14T11:19:47.197036+0000","last_fullsized":"2023-07-14T11:19:47.197036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:54.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.719668+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:11.719668+0000","last_peered":"2023-07-14T11:20:11.719668+0000","last_clean":"2023-07-14T11:20:11.719668+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:11.719668+0000","last_undegraded":"2023-07-14T11:20:11.719668+0000","last_fullsized":"2023-07-14T11:20:11.719668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:13:47.991877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967522+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:19:47.967522+0000","last_peered":"2023-07-14T11:19:47.967522+0000","last_clean":"2023-07-14T11:19:47.967522+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:19:47.967522+0000","last_undegraded":"2023-07-14T11:19:47.967522+0000","last_fullsized":"2023-07-14T11:19:47.967522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:21.029051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967474+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:19:47.967474+0000","last_peered":"2023-07-14T11:19:47.967474+0000","last_clean":"2023-07-14T11:19:47.967474+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:19:47.967474+0000","last_undegraded":"2023-07-14T11:19:47.967474+0000","last_fullsized":"2023-07-14T11:19:47.967474+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:10.344354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967640+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:19:47.967640+0000","last_peered":"2023-07-14T11:19:47.967640+0000","last_clean":"2023-07-14T11:19:47.967640+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:19:47.967640+0000","last_undegraded":"2023-07-14T11:19:47.967640+0000","last_fullsized":"2023-07-14T11:19:47.967640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:14.404398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737187+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:19:47.737187+0000","last_peered":"2023-07-14T11:19:47.737187+0000","last_clean":"2023-07-14T11:19:47.737187+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:19:47.737187+0000","last_undegraded":"2023-07-14T11:19:47.737187+0000","last_fullsized":"2023-07-14T11:19:47.737187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:32:25.980256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967578+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:19:47.967578+0000","last_peered":"2023-07-14T11:19:47.967578+0000","last_clean":"2023-07-14T11:19:47.967578+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:19:47.967578+0000","last_undegraded":"2023-07-14T11:19:47.967578+0000","last_fullsized":"2023-07-14T11:19:47.967578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:02:46.721446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718300+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:14.718300+0000","last_peered":"2023-07-14T11:20:14.718300+0000","last_clean":"2023-07-14T11:20:14.718300+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:14.718300+0000","last_undegraded":"2023-07-14T11:20:14.718300+0000","last_fullsized":"2023-07-14T11:20:14.718300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:53:43.048114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969462+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:19:47.969462+0000","last_peered":"2023-07-14T11:19:47.969462+0000","last_clean":"2023-07-14T11:19:47.969462+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:19:47.969462+0000","last_undegraded":"2023-07-14T11:19:47.969462+0000","last_fullsized":"2023-07-14T11:19:47.969462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:45.135535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967447+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:19:47.967447+0000","last_peered":"2023-07-14T11:19:47.967447+0000","last_clean":"2023-07-14T11:19:47.967447+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:19:47.967447+0000","last_undegraded":"2023-07-14T11:19:47.967447+0000","last_fullsized":"2023-07-14T11:19:47.967447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:38.925420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717638+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:14.717638+0000","last_peered":"2023-07-14T11:20:14.717638+0000","last_clean":"2023-07-14T11:20:14.717638+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:14.717638+0000","last_undegraded":"2023-07-14T11:20:14.717638+0000","last_fullsized":"2023-07-14T11:20:14.717638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:20.321448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968913+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:19:47.968913+0000","last_peered":"2023-07-14T11:19:47.968913+0000","last_clean":"2023-07-14T11:19:47.968913+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:19:47.968913+0000","last_undegraded":"2023-07-14T11:19:47.968913+0000","last_fullsized":"2023-07-14T11:19:47.968913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:18.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197409+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:19:47.197409+0000","last_peered":"2023-07-14T11:19:47.197409+0000","last_clean":"2023-07-14T11:19:47.197409+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:19:47.197409+0000","last_undegraded":"2023-07-14T11:19:47.197409+0000","last_fullsized":"2023-07-14T11:19:47.197409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:17.534514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.678617+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:12.678617+0000","last_peered":"2023-07-14T11:20:12.678617+0000","last_clean":"2023-07-14T11:20:12.678617+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:12.678617+0000","last_undegraded":"2023-07-14T11:20:12.678617+0000","last_fullsized":"2023-07-14T11:20:12.678617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.545637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718422+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:14.718422+0000","last_peered":"2023-07-14T11:20:14.718422+0000","last_clean":"2023-07-14T11:20:14.718422+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:14.718422+0000","last_undegraded":"2023-07-14T11:20:14.718422+0000","last_fullsized":"2023-07-14T11:20:14.718422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:25.009048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197411+0000","last_change":"2023-07-14T11:19:46.096809+0000","last_active":"2023-07-14T11:19:47.197411+0000","last_peered":"2023-07-14T11:19:47.197411+0000","last_clean":"2023-07-14T11:19:47.197411+0000","last_became_active":"2023-07-14T11:19:46.096322+0000","last_became_peered":"2023-07-14T11:19:46.096322+0000","last_unstale":"2023-07-14T11:19:47.197411+0000","last_undegraded":"2023-07-14T11:19:47.197411+0000","last_fullsized":"2023-07-14T11:19:47.197411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:48:14.145380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718924+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:11.718924+0000","last_peered":"2023-07-14T11:20:11.718924+0000","last_clean":"2023-07-14T11:20:11.718924+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:11.718924+0000","last_undegraded":"2023-07-14T11:20:11.718924+0000","last_fullsized":"2023-07-14T11:20:11.718924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:52.081384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968898+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:19:47.968898+0000","last_peered":"2023-07-14T11:19:47.968898+0000","last_clean":"2023-07-14T11:19:47.968898+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:19:47.968898+0000","last_undegraded":"2023-07-14T11:19:47.968898+0000","last_fullsized":"2023-07-14T11:19:47.968898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:04.665047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081037+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:19:47.081037+0000","last_peered":"2023-07-14T11:19:47.081037+0000","last_clean":"2023-07-14T11:19:47.081037+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:19:47.081037+0000","last_undegraded":"2023-07-14T11:19:47.081037+0000","last_fullsized":"2023-07-14T11:19:47.081037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:25.917595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731318+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:14.731318+0000","last_peered":"2023-07-14T11:20:14.731318+0000","last_clean":"2023-07-14T11:20:14.731318+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:14.731318+0000","last_undegraded":"2023-07-14T11:20:14.731318+0000","last_fullsized":"2023-07-14T11:20:14.731318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:39.839896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967040+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:19:47.967040+0000","last_peered":"2023-07-14T11:19:47.967040+0000","last_clean":"2023-07-14T11:19:47.967040+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:19:47.967040+0000","last_undegraded":"2023-07-14T11:19:47.967040+0000","last_fullsized":"2023-07-14T11:19:47.967040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:43.960608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737292+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:19:47.737292+0000","last_peered":"2023-07-14T11:19:47.737292+0000","last_clean":"2023-07-14T11:19:47.737292+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:19:47.737292+0000","last_undegraded":"2023-07-14T11:19:47.737292+0000","last_fullsized":"2023-07-14T11:19:47.737292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:38.334013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967956+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:19:47.967956+0000","last_peered":"2023-07-14T11:19:47.967956+0000","last_clean":"2023-07-14T11:19:47.967956+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:19:47.967956+0000","last_undegraded":"2023-07-14T11:19:47.967956+0000","last_fullsized":"2023-07-14T11:19:47.967956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:16.510067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081653+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:19:47.081653+0000","last_peered":"2023-07-14T11:19:47.081653+0000","last_clean":"2023-07-14T11:19:47.081653+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:19:47.081653+0000","last_undegraded":"2023-07-14T11:19:47.081653+0000","last_fullsized":"2023-07-14T11:19:47.081653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:35.054594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967147+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:19:47.967147+0000","last_peered":"2023-07-14T11:19:47.967147+0000","last_clean":"2023-07-14T11:19:47.967147+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:19:47.967147+0000","last_undegraded":"2023-07-14T11:19:47.967147+0000","last_fullsized":"2023-07-14T11:19:47.967147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:21:05.922286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711077+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:10.711077+0000","last_peered":"2023-07-14T11:20:10.711077+0000","last_clean":"2023-07-14T11:20:10.711077+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:10.711077+0000","last_undegraded":"2023-07-14T11:20:10.711077+0000","last_fullsized":"2023-07-14T11:20:10.711077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:02:01.788231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969766+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:19:47.969766+0000","last_peered":"2023-07-14T11:19:47.969766+0000","last_clean":"2023-07-14T11:19:47.969766+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:19:47.969766+0000","last_undegraded":"2023-07-14T11:19:47.969766+0000","last_fullsized":"2023-07-14T11:19:47.969766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:43:31.070780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081629+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:19:47.081629+0000","last_peered":"2023-07-14T11:19:47.081629+0000","last_clean":"2023-07-14T11:19:47.081629+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:19:47.081629+0000","last_undegraded":"2023-07-14T11:19:47.081629+0000","last_fullsized":"2023-07-14T11:19:47.081629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:08.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711857+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:10.711857+0000","last_peered":"2023-07-14T11:20:10.711857+0000","last_clean":"2023-07-14T11:20:10.711857+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:10.711857+0000","last_undegraded":"2023-07-14T11:20:10.711857+0000","last_fullsized":"2023-07-14T11:20:10.711857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:05.009517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737241+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:19:47.737241+0000","last_peered":"2023-07-14T11:19:47.737241+0000","last_clean":"2023-07-14T11:19:47.737241+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:19:47.737241+0000","last_undegraded":"2023-07-14T11:19:47.737241+0000","last_fullsized":"2023-07-14T11:19:47.737241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:30:37.665531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967912+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:19:47.967912+0000","last_peered":"2023-07-14T11:19:47.967912+0000","last_clean":"2023-07-14T11:19:47.967912+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:19:47.967912+0000","last_undegraded":"2023-07-14T11:19:47.967912+0000","last_fullsized":"2023-07-14T11:19:47.967912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:39:26.015436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.710731+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:10.710731+0000","last_peered":"2023-07-14T11:20:10.710731+0000","last_clean":"2023-07-14T11:20:10.710731+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:10.710731+0000","last_undegraded":"2023-07-14T11:20:10.710731+0000","last_fullsized":"2023-07-14T11:20:10.710731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:44.419475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969789+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:19:47.969789+0000","last_peered":"2023-07-14T11:19:47.969789+0000","last_clean":"2023-07-14T11:19:47.969789+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:19:47.969789+0000","last_undegraded":"2023-07-14T11:19:47.969789+0000","last_fullsized":"2023-07-14T11:19:47.969789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:12:08.517212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081641+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:19:47.081641+0000","last_peered":"2023-07-14T11:19:47.081641+0000","last_clean":"2023-07-14T11:19:47.081641+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:19:47.081641+0000","last_undegraded":"2023-07-14T11:19:47.081641+0000","last_fullsized":"2023-07-14T11:19:47.081641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:47.740065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711843+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:10.711843+0000","last_peered":"2023-07-14T11:20:10.711843+0000","last_clean":"2023-07-14T11:20:10.711843+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:10.711843+0000","last_undegraded":"2023-07-14T11:20:10.711843+0000","last_fullsized":"2023-07-14T11:20:10.711843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:13:57.943632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717973+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:14.717973+0000","last_peered":"2023-07-14T11:20:14.717973+0000","last_clean":"2023-07-14T11:20:14.717973+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:14.717973+0000","last_undegraded":"2023-07-14T11:20:14.717973+0000","last_fullsized":"2023-07-14T11:20:14.717973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:59.311439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969168+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:19:47.969168+0000","last_peered":"2023-07-14T11:19:47.969168+0000","last_clean":"2023-07-14T11:19:47.969168+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:19:47.969168+0000","last_undegraded":"2023-07-14T11:19:47.969168+0000","last_fullsized":"2023-07-14T11:19:47.969168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:56:28.718616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711234+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:10.711234+0000","last_peered":"2023-07-14T11:20:10.711234+0000","last_clean":"2023-07-14T11:20:10.711234+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:10.711234+0000","last_undegraded":"2023-07-14T11:20:10.711234+0000","last_fullsized":"2023-07-14T11:20:10.711234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:29:12.273324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728223+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:14.728223+0000","last_peered":"2023-07-14T11:20:14.728223+0000","last_clean":"2023-07-14T11:20:14.728223+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:14.728223+0000","last_undegraded":"2023-07-14T11:20:14.728223+0000","last_fullsized":"2023-07-14T11:20:14.728223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:45.374719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080735+0000","last_change":"2023-07-14T11:19:46.097705+0000","last_active":"2023-07-14T11:19:47.080735+0000","last_peered":"2023-07-14T11:19:47.080735+0000","last_clean":"2023-07-14T11:19:47.080735+0000","last_became_active":"2023-07-14T11:19:46.097093+0000","last_became_peered":"2023-07-14T11:19:46.097093+0000","last_unstale":"2023-07-14T11:19:47.080735+0000","last_undegraded":"2023-07-14T11:19:47.080735+0000","last_fullsized":"2023-07-14T11:19:47.080735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:16:13.697886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713460+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:10.713460+0000","last_peered":"2023-07-14T11:20:10.713460+0000","last_clean":"2023-07-14T11:20:10.713460+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:10.713460+0000","last_undegraded":"2023-07-14T11:20:10.713460+0000","last_fullsized":"2023-07-14T11:20:10.713460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:44.692262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968550+0000","last_change":"2023-07-14T11:19:42.077628+0000","last_active":"2023-07-14T11:19:47.968550+0000","last_peered":"2023-07-14T11:19:47.968550+0000","last_clean":"2023-07-14T11:19:47.968550+0000","last_became_active":"2023-07-14T11:19:42.077341+0000","last_became_peered":"2023-07-14T11:19:42.077341+0000","last_unstale":"2023-07-14T11:19:47.968550+0000","last_undegraded":"2023-07-14T11:19:47.968550+0000","last_fullsized":"2023-07-14T11:19:47.968550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:03.185626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082158+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:19:47.082158+0000","last_peered":"2023-07-14T11:19:47.082158+0000","last_clean":"2023-07-14T11:19:47.082158+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:19:47.082158+0000","last_undegraded":"2023-07-14T11:19:47.082158+0000","last_fullsized":"2023-07-14T11:19:47.082158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:07.141091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.680816+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:12.680816+0000","last_peered":"2023-07-14T11:20:12.680816+0000","last_clean":"2023-07-14T11:20:12.680816+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:12.680816+0000","last_undegraded":"2023-07-14T11:20:12.680816+0000","last_fullsized":"2023-07-14T11:20:12.680816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:29.624865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968266+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:19:47.968266+0000","last_peered":"2023-07-14T11:19:47.968266+0000","last_clean":"2023-07-14T11:19:47.968266+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:19:47.968266+0000","last_undegraded":"2023-07-14T11:19:47.968266+0000","last_fullsized":"2023-07-14T11:19:47.968266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:59.325039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728868+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:14.728868+0000","last_peered":"2023-07-14T11:20:14.728868+0000","last_clean":"2023-07-14T11:20:14.728868+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:14.728868+0000","last_undegraded":"2023-07-14T11:20:14.728868+0000","last_fullsized":"2023-07-14T11:20:14.728868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:31.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718779+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:14.718779+0000","last_peered":"2023-07-14T11:20:14.718779+0000","last_clean":"2023-07-14T11:20:14.718779+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:14.718779+0000","last_undegraded":"2023-07-14T11:20:14.718779+0000","last_fullsized":"2023-07-14T11:20:14.718779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:25:31.744731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967979+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:19:47.967979+0000","last_peered":"2023-07-14T11:19:47.967979+0000","last_clean":"2023-07-14T11:19:47.967979+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:19:47.967979+0000","last_undegraded":"2023-07-14T11:19:47.967979+0000","last_fullsized":"2023-07-14T11:19:47.967979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:58.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967884+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:19:47.967884+0000","last_peered":"2023-07-14T11:19:47.967884+0000","last_clean":"2023-07-14T11:19:47.967884+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:19:47.967884+0000","last_undegraded":"2023-07-14T11:19:47.967884+0000","last_fullsized":"2023-07-14T11:19:47.967884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:07.208804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717860+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:14.717860+0000","last_peered":"2023-07-14T11:20:14.717860+0000","last_clean":"2023-07-14T11:20:14.717860+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:14.717860+0000","last_undegraded":"2023-07-14T11:20:14.717860+0000","last_fullsized":"2023-07-14T11:20:14.717860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:15:42.131659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080903+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:19:47.080903+0000","last_peered":"2023-07-14T11:19:47.080903+0000","last_clean":"2023-07-14T11:19:47.080903+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:19:47.080903+0000","last_undegraded":"2023-07-14T11:19:47.080903+0000","last_fullsized":"2023-07-14T11:19:47.080903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:29.469059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T11:19:47.967804+0000","last_change":"2023-07-14T11:20:14.717622+0000","last_active":"2023-07-14T11:20:14.717622+0000","last_peered":"2023-07-14T11:19:47.967804+0000","last_clean":"2023-07-14T11:19:47.967804+0000","last_became_active":"2023-07-14T11:19:46.273092+0000","last_became_peered":"2023-07-14T11:19:46.273092+0000","last_unstale":"2023-07-14T11:19:47.967804+0000","last_undegraded":"2023-07-14T11:19:47.967804+0000","last_fullsized":"2023-07-14T11:19:47.967804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081920+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:19:47.081920+0000","last_peered":"2023-07-14T11:19:47.081920+0000","last_clean":"2023-07-14T11:19:47.081920+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:19:47.081920+0000","last_undegraded":"2023-07-14T11:19:47.081920+0000","last_fullsized":"2023-07-14T11:19:47.081920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:21:18.903408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081118+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:19:47.081118+0000","last_peered":"2023-07-14T11:19:47.081118+0000","last_clean":"2023-07-14T11:19:47.081118+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:19:47.081118+0000","last_undegraded":"2023-07-14T11:19:47.081118+0000","last_fullsized":"2023-07-14T11:19:47.081118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:37.312351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730373+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:14.730373+0000","last_peered":"2023-07-14T11:20:14.730373+0000","last_clean":"2023-07-14T11:20:14.730373+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:14.730373+0000","last_undegraded":"2023-07-14T11:20:14.730373+0000","last_fullsized":"2023-07-14T11:20:14.730373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:10.476499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968436+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:19:47.968436+0000","last_peered":"2023-07-14T11:19:47.968436+0000","last_clean":"2023-07-14T11:19:47.968436+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:19:47.968436+0000","last_undegraded":"2023-07-14T11:19:47.968436+0000","last_fullsized":"2023-07-14T11:19:47.968436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081008+0000","last_change":"2023-07-14T11:19:42.093066+0000","last_active":"2023-07-14T11:19:47.081008+0000","last_peered":"2023-07-14T11:19:47.081008+0000","last_clean":"2023-07-14T11:19:47.081008+0000","last_became_active":"2023-07-14T11:19:42.092719+0000","last_became_peered":"2023-07-14T11:19:42.092719+0000","last_unstale":"2023-07-14T11:19:47.081008+0000","last_undegraded":"2023-07-14T11:19:47.081008+0000","last_fullsized":"2023-07-14T11:19:47.081008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:35:05.018230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.729003+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:14.729003+0000","last_peered":"2023-07-14T11:20:14.729003+0000","last_clean":"2023-07-14T11:20:14.729003+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:14.729003+0000","last_undegraded":"2023-07-14T11:20:14.729003+0000","last_fullsized":"2023-07-14T11:20:14.729003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:29.125385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968186+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:19:47.968186+0000","last_peered":"2023-07-14T11:19:47.968186+0000","last_clean":"2023-07-14T11:19:47.968186+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:19:47.968186+0000","last_undegraded":"2023-07-14T11:19:47.968186+0000","last_fullsized":"2023-07-14T11:19:47.968186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:06.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731196+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:14.731196+0000","last_peered":"2023-07-14T11:20:14.731196+0000","last_clean":"2023-07-14T11:20:14.731196+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:14.731196+0000","last_undegraded":"2023-07-14T11:20:14.731196+0000","last_fullsized":"2023-07-14T11:20:14.731196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:16:34.542159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:49.452867+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:19:49.452867+0000","last_peered":"2023-07-14T11:19:49.452867+0000","last_clean":"2023-07-14T11:19:49.452867+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:19:49.452867+0000","last_undegraded":"2023-07-14T11:19:49.452867+0000","last_fullsized":"2023-07-14T11:19:49.452867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:22.680690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967124+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:19:47.967124+0000","last_peered":"2023-07-14T11:19:47.967124+0000","last_clean":"2023-07-14T11:19:47.967124+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:19:47.967124+0000","last_undegraded":"2023-07-14T11:19:47.967124+0000","last_fullsized":"2023-07-14T11:19:47.967124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:14:39.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.676206+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:12.676206+0000","last_peered":"2023-07-14T11:20:12.676206+0000","last_clean":"2023-07-14T11:20:12.676206+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:12.676206+0000","last_undegraded":"2023-07-14T11:20:12.676206+0000","last_fullsized":"2023-07-14T11:20:12.676206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:22:45.207699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968042+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:19:47.968042+0000","last_peered":"2023-07-14T11:19:47.968042+0000","last_clean":"2023-07-14T11:19:47.968042+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:19:47.968042+0000","last_undegraded":"2023-07-14T11:19:47.968042+0000","last_fullsized":"2023-07-14T11:19:47.968042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:23.767120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968619+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:19:47.968619+0000","last_peered":"2023-07-14T11:19:47.968619+0000","last_clean":"2023-07-14T11:19:47.968619+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:19:47.968619+0000","last_undegraded":"2023-07-14T11:19:47.968619+0000","last_fullsized":"2023-07-14T11:19:47.968619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:13.375758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731070+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:14.731070+0000","last_peered":"2023-07-14T11:20:14.731070+0000","last_clean":"2023-07-14T11:20:14.731070+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:14.731070+0000","last_undegraded":"2023-07-14T11:20:14.731070+0000","last_fullsized":"2023-07-14T11:20:14.731070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:59:07.405076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.717379+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:11.717379+0000","last_peered":"2023-07-14T11:20:11.717379+0000","last_clean":"2023-07-14T11:20:11.717379+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:11.717379+0000","last_undegraded":"2023-07-14T11:20:11.717379+0000","last_fullsized":"2023-07-14T11:20:11.717379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:24.177685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080724+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:19:47.080724+0000","last_peered":"2023-07-14T11:19:47.080724+0000","last_clean":"2023-07-14T11:19:47.080724+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:19:47.080724+0000","last_undegraded":"2023-07-14T11:19:47.080724+0000","last_fullsized":"2023-07-14T11:19:47.080724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:52:43.413334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718394+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:14.718394+0000","last_peered":"2023-07-14T11:20:14.718394+0000","last_clean":"2023-07-14T11:20:14.718394+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:14.718394+0000","last_undegraded":"2023-07-14T11:20:14.718394+0000","last_fullsized":"2023-07-14T11:20:14.718394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:22.172922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730551+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:14.730551+0000","last_peered":"2023-07-14T11:20:14.730551+0000","last_clean":"2023-07-14T11:20:14.730551+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:14.730551+0000","last_undegraded":"2023-07-14T11:20:14.730551+0000","last_fullsized":"2023-07-14T11:20:14.730551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:37.007482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967049+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:19:47.967049+0000","last_peered":"2023-07-14T11:19:47.967049+0000","last_clean":"2023-07-14T11:19:47.967049+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:19:47.967049+0000","last_undegraded":"2023-07-14T11:19:47.967049+0000","last_fullsized":"2023-07-14T11:19:47.967049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:39:36.064722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718705+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:11.718705+0000","last_peered":"2023-07-14T11:20:11.718705+0000","last_clean":"2023-07-14T11:20:11.718705+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:11.718705+0000","last_undegraded":"2023-07-14T11:20:11.718705+0000","last_fullsized":"2023-07-14T11:20:11.718705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:26:49.607683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967586+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:19:47.967586+0000","last_peered":"2023-07-14T11:19:47.967586+0000","last_clean":"2023-07-14T11:19:47.967586+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:19:47.967586+0000","last_undegraded":"2023-07-14T11:19:47.967586+0000","last_fullsized":"2023-07-14T11:19:47.967586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:35:51.028974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.196923+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:19:47.196923+0000","last_peered":"2023-07-14T11:19:47.196923+0000","last_clean":"2023-07-14T11:19:47.196923+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:19:47.196923+0000","last_undegraded":"2023-07-14T11:19:47.196923+0000","last_fullsized":"2023-07-14T11:19:47.196923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:20.885447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711775+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:10.711775+0000","last_peered":"2023-07-14T11:20:10.711775+0000","last_clean":"2023-07-14T11:20:10.711775+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:10.711775+0000","last_undegraded":"2023-07-14T11:20:10.711775+0000","last_fullsized":"2023-07-14T11:20:10.711775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:36.953428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737388+0000","last_change":"2023-07-14T11:19:42.085058+0000","last_active":"2023-07-14T11:19:47.737388+0000","last_peered":"2023-07-14T11:19:47.737388+0000","last_clean":"2023-07-14T11:19:47.737388+0000","last_became_active":"2023-07-14T11:19:42.083380+0000","last_became_peered":"2023-07-14T11:19:42.083380+0000","last_unstale":"2023-07-14T11:19:47.737388+0000","last_undegraded":"2023-07-14T11:19:47.737388+0000","last_fullsized":"2023-07-14T11:19:47.737388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:52.430115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967817+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:19:47.967817+0000","last_peered":"2023-07-14T11:19:47.967817+0000","last_clean":"2023-07-14T11:19:47.967817+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:19:47.967817+0000","last_undegraded":"2023-07-14T11:19:47.967817+0000","last_fullsized":"2023-07-14T11:19:47.967817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:25:48.734999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.712915+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:10.712915+0000","last_peered":"2023-07-14T11:20:10.712915+0000","last_clean":"2023-07-14T11:20:10.712915+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:10.712915+0000","last_undegraded":"2023-07-14T11:20:10.712915+0000","last_fullsized":"2023-07-14T11:20:10.712915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:31:26.836255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717656+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:14.717656+0000","last_peered":"2023-07-14T11:20:14.717656+0000","last_clean":"2023-07-14T11:20:14.717656+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:14.717656+0000","last_undegraded":"2023-07-14T11:20:14.717656+0000","last_fullsized":"2023-07-14T11:20:14.717656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:28.963610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082443+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:19:47.082443+0000","last_peered":"2023-07-14T11:19:47.082443+0000","last_clean":"2023-07-14T11:19:47.082443+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:19:47.082443+0000","last_undegraded":"2023-07-14T11:19:47.082443+0000","last_fullsized":"2023-07-14T11:19:47.082443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:43.469946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713281+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:10.713281+0000","last_peered":"2023-07-14T11:20:10.713281+0000","last_clean":"2023-07-14T11:20:10.713281+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:10.713281+0000","last_undegraded":"2023-07-14T11:20:10.713281+0000","last_fullsized":"2023-07-14T11:20:10.713281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:45:36.543244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080522+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:19:47.080522+0000","last_peered":"2023-07-14T11:19:47.080522+0000","last_clean":"2023-07-14T11:19:47.080522+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:19:47.080522+0000","last_undegraded":"2023-07-14T11:19:47.080522+0000","last_fullsized":"2023-07-14T11:19:47.080522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:44.548171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967752+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:19:47.967752+0000","last_peered":"2023-07-14T11:19:47.967752+0000","last_clean":"2023-07-14T11:19:47.967752+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:19:47.967752+0000","last_undegraded":"2023-07-14T11:19:47.967752+0000","last_fullsized":"2023-07-14T11:19:47.967752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:50:53.130285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197038+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:19:47.197038+0000","last_peered":"2023-07-14T11:19:47.197038+0000","last_clean":"2023-07-14T11:19:47.197038+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:19:47.197038+0000","last_undegraded":"2023-07-14T11:19:47.197038+0000","last_fullsized":"2023-07-14T11:19:47.197038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:15.617687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967360+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:19:47.967360+0000","last_peered":"2023-07-14T11:19:47.967360+0000","last_clean":"2023-07-14T11:19:47.967360+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:19:47.967360+0000","last_undegraded":"2023-07-14T11:19:47.967360+0000","last_fullsized":"2023-07-14T11:19:47.967360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:21.159717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967674+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:19:47.967674+0000","last_peered":"2023-07-14T11:19:47.967674+0000","last_clean":"2023-07-14T11:19:47.967674+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:19:47.967674+0000","last_undegraded":"2023-07-14T11:19:47.967674+0000","last_fullsized":"2023-07-14T11:19:47.967674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:57:50.109303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717280+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:14.717280+0000","last_peered":"2023-07-14T11:20:14.717280+0000","last_clean":"2023-07-14T11:20:14.717280+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:14.717280+0000","last_undegraded":"2023-07-14T11:20:14.717280+0000","last_fullsized":"2023-07-14T11:20:14.717280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:38:55.878246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968641+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:19:47.968641+0000","last_peered":"2023-07-14T11:19:47.968641+0000","last_clean":"2023-07-14T11:19:47.968641+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:19:47.968641+0000","last_undegraded":"2023-07-14T11:19:47.968641+0000","last_fullsized":"2023-07-14T11:19:47.968641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:57.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969226+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:19:47.969226+0000","last_peered":"2023-07-14T11:19:47.969226+0000","last_clean":"2023-07-14T11:19:47.969226+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:19:47.969226+0000","last_undegraded":"2023-07-14T11:19:47.969226+0000","last_fullsized":"2023-07-14T11:19:47.969226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:17:50.327405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197577+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:19:47.197577+0000","last_peered":"2023-07-14T11:19:47.197577+0000","last_clean":"2023-07-14T11:19:47.197577+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:19:47.197577+0000","last_undegraded":"2023-07-14T11:19:47.197577+0000","last_fullsized":"2023-07-14T11:19:47.197577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:48.609439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969031+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:19:47.969031+0000","last_peered":"2023-07-14T11:19:47.969031+0000","last_clean":"2023-07-14T11:19:47.969031+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:19:47.969031+0000","last_undegraded":"2023-07-14T11:19:47.969031+0000","last_fullsized":"2023-07-14T11:19:47.969031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:55.843732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736878+0000","last_change":"2023-07-14T11:19:46.088344+0000","last_active":"2023-07-14T11:19:47.736878+0000","last_peered":"2023-07-14T11:19:47.736878+0000","last_clean":"2023-07-14T11:19:47.736878+0000","last_became_active":"2023-07-14T11:19:46.087922+0000","last_became_peered":"2023-07-14T11:19:46.087922+0000","last_unstale":"2023-07-14T11:19:47.736878+0000","last_undegraded":"2023-07-14T11:19:47.736878+0000","last_fullsized":"2023-07-14T11:19:47.736878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:02:04.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968974+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:19:47.968974+0000","last_peered":"2023-07-14T11:19:47.968974+0000","last_clean":"2023-07-14T11:19:47.968974+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:19:47.968974+0000","last_undegraded":"2023-07-14T11:19:47.968974+0000","last_fullsized":"2023-07-14T11:19:47.968974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:56:35.324459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968655+0000","last_change":"2023-07-14T11:19:42.073286+0000","last_active":"2023-07-14T11:19:47.968655+0000","last_peered":"2023-07-14T11:19:47.968655+0000","last_clean":"2023-07-14T11:19:47.968655+0000","last_became_active":"2023-07-14T11:19:42.072976+0000","last_became_peered":"2023-07-14T11:19:42.072976+0000","last_unstale":"2023-07-14T11:19:47.968655+0000","last_undegraded":"2023-07-14T11:19:47.968655+0000","last_fullsized":"2023-07-14T11:19:47.968655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:37:29.066629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967707+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:19:47.967707+0000","last_peered":"2023-07-14T11:19:47.967707+0000","last_clean":"2023-07-14T11:19:47.967707+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:19:47.967707+0000","last_undegraded":"2023-07-14T11:19:47.967707+0000","last_fullsized":"2023-07-14T11:19:47.967707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:15.123989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728744+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:14.728744+0000","last_peered":"2023-07-14T11:20:14.728744+0000","last_clean":"2023-07-14T11:20:14.728744+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:14.728744+0000","last_undegraded":"2023-07-14T11:20:14.728744+0000","last_fullsized":"2023-07-14T11:20:14.728744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:03:42.954256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968714+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:19:47.968714+0000","last_peered":"2023-07-14T11:19:47.968714+0000","last_clean":"2023-07-14T11:19:47.968714+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:19:47.968714+0000","last_undegraded":"2023-07-14T11:19:47.968714+0000","last_fullsized":"2023-07-14T11:19:47.968714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:03.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080574+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:19:47.080574+0000","last_peered":"2023-07-14T11:19:47.080574+0000","last_clean":"2023-07-14T11:19:47.080574+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:19:47.080574+0000","last_undegraded":"2023-07-14T11:19:47.080574+0000","last_fullsized":"2023-07-14T11:19:47.080574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:06.602465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197215+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:19:47.197215+0000","last_peered":"2023-07-14T11:19:47.197215+0000","last_clean":"2023-07-14T11:19:47.197215+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:19:47.197215+0000","last_undegraded":"2023-07-14T11:19:47.197215+0000","last_fullsized":"2023-07-14T11:19:47.197215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:52:37.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197079+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:19:47.197079+0000","last_peered":"2023-07-14T11:19:47.197079+0000","last_clean":"2023-07-14T11:19:47.197079+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:19:47.197079+0000","last_undegraded":"2023-07-14T11:19:47.197079+0000","last_fullsized":"2023-07-14T11:19:47.197079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:26:40.329142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736992+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:19:47.736992+0000","last_peered":"2023-07-14T11:19:47.736992+0000","last_clean":"2023-07-14T11:19:47.736992+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:19:47.736992+0000","last_undegraded":"2023-07-14T11:19:47.736992+0000","last_fullsized":"2023-07-14T11:19:47.736992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:09:09.081413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730659+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:14.730659+0000","last_peered":"2023-07-14T11:20:14.730659+0000","last_clean":"2023-07-14T11:20:14.730659+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:14.730659+0000","last_undegraded":"2023-07-14T11:20:14.730659+0000","last_fullsized":"2023-07-14T11:20:14.730659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:18.996495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081049+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:19:47.081049+0000","last_peered":"2023-07-14T11:19:47.081049+0000","last_clean":"2023-07-14T11:19:47.081049+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:19:47.081049+0000","last_undegraded":"2023-07-14T11:19:47.081049+0000","last_fullsized":"2023-07-14T11:19:47.081049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:56.251387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197612+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:19:47.197612+0000","last_peered":"2023-07-14T11:19:47.197612+0000","last_clean":"2023-07-14T11:19:47.197612+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:19:47.197612+0000","last_undegraded":"2023-07-14T11:19:47.197612+0000","last_fullsized":"2023-07-14T11:19:47.197612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:32:51.167057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967746+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:19:47.967746+0000","last_peered":"2023-07-14T11:19:47.967746+0000","last_clean":"2023-07-14T11:19:47.967746+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:19:47.967746+0000","last_undegraded":"2023-07-14T11:19:47.967746+0000","last_fullsized":"2023-07-14T11:19:47.967746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:37.267773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-14T11:19:42.068794+0000","last_change":"2023-07-14T11:20:15.683458+0000","last_active":"2023-07-14T11:19:41.086153+0000","last_peered":"2023-07-14T11:19:41.086153+0000","last_clean":"2023-07-14T11:19:41.086153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:19:42.068794+0000","last_undegraded":"2023-07-14T11:19:42.068794+0000","last_fullsized":"2023-07-14T11:19:42.068794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967293+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:19:47.967293+0000","last_peered":"2023-07-14T11:19:47.967293+0000","last_clean":"2023-07-14T11:19:47.967293+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:19:47.967293+0000","last_undegraded":"2023-07-14T11:19:47.967293+0000","last_fullsized":"2023-07-14T11:19:47.967293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:58:16.557411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969388+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:19:47.969388+0000","last_peered":"2023-07-14T11:19:47.969388+0000","last_clean":"2023-07-14T11:19:47.969388+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:19:47.969388+0000","last_undegraded":"2023-07-14T11:19:47.969388+0000","last_fullsized":"2023-07-14T11:19:47.969388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:21:35.735038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727007+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:14.727007+0000","last_peered":"2023-07-14T11:20:14.727007+0000","last_clean":"2023-07-14T11:20:14.727007+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:14.727007+0000","last_undegraded":"2023-07-14T11:20:14.727007+0000","last_fullsized":"2023-07-14T11:20:14.727007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:33.588223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080793+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:19:47.080793+0000","last_peered":"2023-07-14T11:19:47.080793+0000","last_clean":"2023-07-14T11:19:47.080793+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:19:47.080793+0000","last_undegraded":"2023-07-14T11:19:47.080793+0000","last_fullsized":"2023-07-14T11:19:47.080793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:57.953734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":121,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":21140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":471801,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.37},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.755},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.78000000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1062815,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.876},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82999999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":467907,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.628},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.1639999999999997,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70299999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.59799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":481467,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93000000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0129999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1062952,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":466940,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.439},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.85099999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.876},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":470917,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80900000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.377},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.433}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58999999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri Jul 14 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.82399999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.746}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":468328,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":1.042},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.55800000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.621},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.64800000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":1.147}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.99099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":471252,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56299999999999994}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467317,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.376}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.5},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.622}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73899999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467719,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54000000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1058588,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.624}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50800000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:16.035 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: pgmap v288: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.2 KiB/s wr, 4 op/s 2023-07-14T11:20:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-07-14T11:20:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-07-14T11:20:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-07-14T11:20:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-07-14T11:20:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]': finished 2023-07-14T11:20:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T11:20:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T11:20:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2922587973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:20:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:15 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1300013126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:16.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: pgmap v288: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.2 KiB/s wr, 4 op/s 2023-07-14T11:20:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:20:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-14T11:20:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2023-07-14T11:20:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-07-14T11:20:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-07-14T11:20:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-07-14T11:20:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-07-14T11:20:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-07-14T11:20:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-07-14T11:20:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-07-14T11:20:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]': finished 2023-07-14T11:20:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T11:20:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T11:20:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2922587973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:20:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:15 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1300013126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:16.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 -- 172.21.15.161:0/4150217940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 msgr2=0x7f82100fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 --2- 172.21.15.161:0/4150217940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100fff10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f820c009990 tx=0x7f820c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 -- 172.21.15.161:0/4150217940 shutdown_connections 2023-07-14T11:20:16.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 --2- 172.21.15.161:0/4150217940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100fff10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 --2- 172.21.15.161:0/4150217940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8210101680 0x7f8210107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 --2- 172.21.15.161:0/4150217940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8210100cf0 0x7f8210101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 -- 172.21.15.161:0/4150217940 >> 172.21.15.161:0/4150217940 conn(0x7f82100fb0e0 msgr2=0x7f82100fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:16.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 -- 172.21.15.161:0/4150217940 shutdown_connections 2023-07-14T11:20:16.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.258+0000 7f82177ff700 1 -- 172.21.15.161:0/4150217940 wait complete. 2023-07-14T11:20:16.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.259+0000 7f82177ff700 1 Processor -- start 2023-07-14T11:20:16.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.259+0000 7f82177ff700 1 -- start start 2023-07-14T11:20:16.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f82177ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f82177ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8210100cf0 0x7f8210072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f82177ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8210101680 0x7f8210072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f82177ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82100735e0 con 0x7f82100ffb00 2023-07-14T11:20:16.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f82177ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8210073750 con 0x7f8210101680 2023-07-14T11:20:16.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f82177ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821019f860 con 0x7f8210100cf0 2023-07-14T11:20:16.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45174/0 (socket says 172.21.15.161:45174) 2023-07-14T11:20:16.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 -- 172.21.15.161:0/3004817638 learned_addr learned my addr 172.21.15.161:0/3004817638 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:16.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f8214d9a700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8210100cf0 0x7f8210072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f8215d9c700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8210101680 0x7f8210072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 -- 172.21.15.161:0/3004817638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8210100cf0 msgr2=0x7f8210072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8210100cf0 0x7f8210072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 -- 172.21.15.161:0/3004817638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8210101680 msgr2=0x7f8210072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8210101680 0x7f8210072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f821559b700 1 -- 172.21.15.161:0/3004817638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200009520 con 0x7f82100ffb00 2023-07-14T11:20:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f8214d9a700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8210100cf0 0x7f8210072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.260+0000 7f8215d9c700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8210101680 0x7f8210072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:20:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.261+0000 7f821559b700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100742c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f820c005740 tx=0x7f820c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.261+0000 7f82067fc700 1 -- 172.21.15.161:0/3004817638 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c01d070 con 0x7f82100ffb00 2023-07-14T11:20:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.261+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c0095f0 con 0x7f82100ffb00 2023-07-14T11:20:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.261+0000 7f82067fc700 1 -- 172.21.15.161:0/3004817638 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820c021410 con 0x7f82100ffb00 2023-07-14T11:20:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.261+0000 7f82067fc700 1 -- 172.21.15.161:0/3004817638 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c017650 con 0x7f82100ffb00 2023-07-14T11:20:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.261+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821019fd80 con 0x7f82100ffb00 2023-07-14T11:20:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.262+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821010b320 con 0x7f82100ffb00 2023-07-14T11:20:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.264+0000 7f82067fc700 1 -- 172.21.15.161:0/3004817638 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f820c021810 con 0x7f82100ffb00 2023-07-14T11:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.266+0000 7f82067fc700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81fc0a1220 0x7f81fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.267+0000 7f82067fc700 1 -- 172.21.15.161:0/3004817638 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f820c0ce300 con 0x7f82100ffb00 2023-07-14T11:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.267+0000 7f8214d9a700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81fc0a1220 0x7f81fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.267+0000 7f8214d9a700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81fc0a1220 0x7f81fc0a36d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8200009e40 tx=0x7f8200009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:16.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.270+0000 7f82067fc700 1 -- 172.21.15.161:0/3004817638 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f820c097610 con 0x7f82100ffb00 2023-07-14T11:20:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.463+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82101a0420 con 0x7f81fc0a1220 2023-07-14T11:20:16.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.476+0000 7f82067fc700 1 -- 172.21.15.161:0/3004817638 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422840 (secure 0 0 0) 0x7f82101a0420 con 0x7f81fc0a1220 2023-07-14T11:20:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81fc0a1220 msgr2=0x7f81fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81fc0a1220 0x7f81fc0a36d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8200009e40 tx=0x7f8200009260 comp rx=0 tx=0).stop 2023-07-14T11:20:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 msgr2=0x7f82100742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100742c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f820c005740 tx=0x7f820c009510 comp rx=0 tx=0).stop 2023-07-14T11:20:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 shutdown_connections 2023-07-14T11:20:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82100ffb00 0x7f82100742c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f81fc0a1220 0x7f81fc0a36d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8210101680 0x7f8210072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 --2- 172.21.15.161:0/3004817638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8210100cf0 0x7f8210072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 >> 172.21.15.161:0/3004817638 conn(0x7f82100fb0e0 msgr2=0x7f82100fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 shutdown_connections 2023-07-14T11:20:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.481+0000 7f82177ff700 1 -- 172.21.15.161:0/3004817638 wait complete. 2023-07-14T11:20:16.488 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:16.503 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-07-14T11:20:16.103870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597217,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":156,"num_read_kb":137,"num_write":303,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334928,"kb_used_data":11504,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606896,"statfs":{"total":1151940427776,"available":1151597461504,"internally_reserved":0,"allocated":11780096,"data_stored":7462234,"data_compressed":253459,"data_compressed_allocated":339968,"data_compressed_original":2179072,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2257,"num_objects":35,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002134"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080953+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:19:47.080953+0000","last_peered":"2023-07-14T11:19:47.080953+0000","last_clean":"2023-07-14T11:19:47.080953+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:19:47.080953+0000","last_undegraded":"2023-07-14T11:19:47.080953+0000","last_fullsized":"2023-07-14T11:19:47.080953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:57.564463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737248+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:19:47.737248+0000","last_peered":"2023-07-14T11:19:47.737248+0000","last_clean":"2023-07-14T11:19:47.737248+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:19:47.737248+0000","last_undegraded":"2023-07-14T11:19:47.737248+0000","last_fullsized":"2023-07-14T11:19:47.737248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:19.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737244+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:19:47.737244+0000","last_peered":"2023-07-14T11:19:47.737244+0000","last_clean":"2023-07-14T11:19:47.737244+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:19:47.737244+0000","last_undegraded":"2023-07-14T11:19:47.737244+0000","last_fullsized":"2023-07-14T11:19:47.737244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:47.796849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968258+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:19:47.968258+0000","last_peered":"2023-07-14T11:19:47.968258+0000","last_clean":"2023-07-14T11:19:47.968258+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:19:47.968258+0000","last_undegraded":"2023-07-14T11:19:47.968258+0000","last_fullsized":"2023-07-14T11:19:47.968258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:18:31.399322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717211+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:14.717211+0000","last_peered":"2023-07-14T11:20:14.717211+0000","last_clean":"2023-07-14T11:20:14.717211+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:14.717211+0000","last_undegraded":"2023-07-14T11:20:14.717211+0000","last_fullsized":"2023-07-14T11:20:14.717211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:45:02.418574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081056+0000","last_change":"2023-07-14T11:19:42.092136+0000","last_active":"2023-07-14T11:19:47.081056+0000","last_peered":"2023-07-14T11:19:47.081056+0000","last_clean":"2023-07-14T11:19:47.081056+0000","last_became_active":"2023-07-14T11:19:42.091820+0000","last_became_peered":"2023-07-14T11:19:42.091820+0000","last_unstale":"2023-07-14T11:19:47.081056+0000","last_undegraded":"2023-07-14T11:19:47.081056+0000","last_fullsized":"2023-07-14T11:19:47.081056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:01.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968087+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:19:47.968087+0000","last_peered":"2023-07-14T11:19:47.968087+0000","last_clean":"2023-07-14T11:19:47.968087+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:19:47.968087+0000","last_undegraded":"2023-07-14T11:19:47.968087+0000","last_fullsized":"2023-07-14T11:19:47.968087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:21.428012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967192+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:19:47.967192+0000","last_peered":"2023-07-14T11:19:47.967192+0000","last_clean":"2023-07-14T11:19:47.967192+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:19:47.967192+0000","last_undegraded":"2023-07-14T11:19:47.967192+0000","last_fullsized":"2023-07-14T11:19:47.967192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:38.965332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081420+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:19:47.081420+0000","last_peered":"2023-07-14T11:19:47.081420+0000","last_clean":"2023-07-14T11:19:47.081420+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:19:47.081420+0000","last_undegraded":"2023-07-14T11:19:47.081420+0000","last_fullsized":"2023-07-14T11:19:47.081420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:51.712868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969101+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:19:47.969101+0000","last_peered":"2023-07-14T11:19:47.969101+0000","last_clean":"2023-07-14T11:19:47.969101+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:19:47.969101+0000","last_undegraded":"2023-07-14T11:19:47.969101+0000","last_fullsized":"2023-07-14T11:19:47.969101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:48:50.962800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968204+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:19:47.968204+0000","last_peered":"2023-07-14T11:19:47.968204+0000","last_clean":"2023-07-14T11:19:47.968204+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:19:47.968204+0000","last_undegraded":"2023-07-14T11:19:47.968204+0000","last_fullsized":"2023-07-14T11:19:47.968204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:47.264465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727198+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:14.727198+0000","last_peered":"2023-07-14T11:20:14.727198+0000","last_clean":"2023-07-14T11:20:14.727198+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:14.727198+0000","last_undegraded":"2023-07-14T11:20:14.727198+0000","last_fullsized":"2023-07-14T11:20:14.727198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:42.155482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736832+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:19:47.736832+0000","last_peered":"2023-07-14T11:19:47.736832+0000","last_clean":"2023-07-14T11:19:47.736832+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:19:47.736832+0000","last_undegraded":"2023-07-14T11:19:47.736832+0000","last_fullsized":"2023-07-14T11:19:47.736832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:26.314511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736956+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:19:47.736956+0000","last_peered":"2023-07-14T11:19:47.736956+0000","last_clean":"2023-07-14T11:19:47.736956+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:19:47.736956+0000","last_undegraded":"2023-07-14T11:19:47.736956+0000","last_fullsized":"2023-07-14T11:19:47.736956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:10:23.738358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081137+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:19:47.081137+0000","last_peered":"2023-07-14T11:19:47.081137+0000","last_clean":"2023-07-14T11:19:47.081137+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:19:47.081137+0000","last_undegraded":"2023-07-14T11:19:47.081137+0000","last_fullsized":"2023-07-14T11:19:47.081137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:51.121664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082305+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:19:47.082305+0000","last_peered":"2023-07-14T11:19:47.082305+0000","last_clean":"2023-07-14T11:19:47.082305+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:19:47.082305+0000","last_undegraded":"2023-07-14T11:19:47.082305+0000","last_fullsized":"2023-07-14T11:19:47.082305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:07:45.552513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197608+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:19:47.197608+0000","last_peered":"2023-07-14T11:19:47.197608+0000","last_clean":"2023-07-14T11:19:47.197608+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:19:47.197608+0000","last_undegraded":"2023-07-14T11:19:47.197608+0000","last_fullsized":"2023-07-14T11:19:47.197608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:44.466407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197036+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:19:47.197036+0000","last_peered":"2023-07-14T11:19:47.197036+0000","last_clean":"2023-07-14T11:19:47.197036+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:19:47.197036+0000","last_undegraded":"2023-07-14T11:19:47.197036+0000","last_fullsized":"2023-07-14T11:19:47.197036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:54.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.719668+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:11.719668+0000","last_peered":"2023-07-14T11:20:11.719668+0000","last_clean":"2023-07-14T11:20:11.719668+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:11.719668+0000","last_undegraded":"2023-07-14T11:20:11.719668+0000","last_fullsized":"2023-07-14T11:20:11.719668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:13:47.991877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967522+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:19:47.967522+0000","last_peered":"2023-07-14T11:19:47.967522+0000","last_clean":"2023-07-14T11:19:47.967522+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:19:47.967522+0000","last_undegraded":"2023-07-14T11:19:47.967522+0000","last_fullsized":"2023-07-14T11:19:47.967522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:21.029051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967474+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:19:47.967474+0000","last_peered":"2023-07-14T11:19:47.967474+0000","last_clean":"2023-07-14T11:19:47.967474+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:19:47.967474+0000","last_undegraded":"2023-07-14T11:19:47.967474+0000","last_fullsized":"2023-07-14T11:19:47.967474+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:10.344354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967640+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:19:47.967640+0000","last_peered":"2023-07-14T11:19:47.967640+0000","last_clean":"2023-07-14T11:19:47.967640+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:19:47.967640+0000","last_undegraded":"2023-07-14T11:19:47.967640+0000","last_fullsized":"2023-07-14T11:19:47.967640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:14.404398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737187+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:19:47.737187+0000","last_peered":"2023-07-14T11:19:47.737187+0000","last_clean":"2023-07-14T11:19:47.737187+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:19:47.737187+0000","last_undegraded":"2023-07-14T11:19:47.737187+0000","last_fullsized":"2023-07-14T11:19:47.737187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:32:25.980256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967578+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:19:47.967578+0000","last_peered":"2023-07-14T11:19:47.967578+0000","last_clean":"2023-07-14T11:19:47.967578+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:19:47.967578+0000","last_undegraded":"2023-07-14T11:19:47.967578+0000","last_fullsized":"2023-07-14T11:19:47.967578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:02:46.721446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718300+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:14.718300+0000","last_peered":"2023-07-14T11:20:14.718300+0000","last_clean":"2023-07-14T11:20:14.718300+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:14.718300+0000","last_undegraded":"2023-07-14T11:20:14.718300+0000","last_fullsized":"2023-07-14T11:20:14.718300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:53:43.048114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967447+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:19:47.967447+0000","last_peered":"2023-07-14T11:19:47.967447+0000","last_clean":"2023-07-14T11:19:47.967447+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:19:47.967447+0000","last_undegraded":"2023-07-14T11:19:47.967447+0000","last_fullsized":"2023-07-14T11:19:47.967447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:38.925420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717638+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:14.717638+0000","last_peered":"2023-07-14T11:20:14.717638+0000","last_clean":"2023-07-14T11:20:14.717638+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:14.717638+0000","last_undegraded":"2023-07-14T11:20:14.717638+0000","last_fullsized":"2023-07-14T11:20:14.717638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:20.321448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968913+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:19:47.968913+0000","last_peered":"2023-07-14T11:19:47.968913+0000","last_clean":"2023-07-14T11:19:47.968913+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:19:47.968913+0000","last_undegraded":"2023-07-14T11:19:47.968913+0000","last_fullsized":"2023-07-14T11:19:47.968913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:18.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197409+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:19:47.197409+0000","last_peered":"2023-07-14T11:19:47.197409+0000","last_clean":"2023-07-14T11:19:47.197409+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:19:47.197409+0000","last_undegraded":"2023-07-14T11:19:47.197409+0000","last_fullsized":"2023-07-14T11:19:47.197409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:17.534514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.678617+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:12.678617+0000","last_peered":"2023-07-14T11:20:12.678617+0000","last_clean":"2023-07-14T11:20:12.678617+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:12.678617+0000","last_undegraded":"2023-07-14T11:20:12.678617+0000","last_fullsized":"2023-07-14T11:20:12.678617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.545637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718422+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:14.718422+0000","last_peered":"2023-07-14T11:20:14.718422+0000","last_clean":"2023-07-14T11:20:14.718422+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:14.718422+0000","last_undegraded":"2023-07-14T11:20:14.718422+0000","last_fullsized":"2023-07-14T11:20:14.718422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:25.009048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197411+0000","last_change":"2023-07-14T11:19:46.096809+0000","last_active":"2023-07-14T11:19:47.197411+0000","last_peered":"2023-07-14T11:19:47.197411+0000","last_clean":"2023-07-14T11:19:47.197411+0000","last_became_active":"2023-07-14T11:19:46.096322+0000","last_became_peered":"2023-07-14T11:19:46.096322+0000","last_unstale":"2023-07-14T11:19:47.197411+0000","last_undegraded":"2023-07-14T11:19:47.197411+0000","last_fullsized":"2023-07-14T11:19:47.197411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:48:14.145380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718924+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:11.718924+0000","last_peered":"2023-07-14T11:20:11.718924+0000","last_clean":"2023-07-14T11:20:11.718924+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:11.718924+0000","last_undegraded":"2023-07-14T11:20:11.718924+0000","last_fullsized":"2023-07-14T11:20:11.718924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:52.081384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968898+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:19:47.968898+0000","last_peered":"2023-07-14T11:19:47.968898+0000","last_clean":"2023-07-14T11:19:47.968898+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:19:47.968898+0000","last_undegraded":"2023-07-14T11:19:47.968898+0000","last_fullsized":"2023-07-14T11:19:47.968898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:04.665047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081037+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:19:47.081037+0000","last_peered":"2023-07-14T11:19:47.081037+0000","last_clean":"2023-07-14T11:19:47.081037+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:19:47.081037+0000","last_undegraded":"2023-07-14T11:19:47.081037+0000","last_fullsized":"2023-07-14T11:19:47.081037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:25.917595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731318+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:14.731318+0000","last_peered":"2023-07-14T11:20:14.731318+0000","last_clean":"2023-07-14T11:20:14.731318+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:14.731318+0000","last_undegraded":"2023-07-14T11:20:14.731318+0000","last_fullsized":"2023-07-14T11:20:14.731318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:39.839896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967040+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:19:47.967040+0000","last_peered":"2023-07-14T11:19:47.967040+0000","last_clean":"2023-07-14T11:19:47.967040+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:19:47.967040+0000","last_undegraded":"2023-07-14T11:19:47.967040+0000","last_fullsized":"2023-07-14T11:19:47.967040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:43.960608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737292+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:19:47.737292+0000","last_peered":"2023-07-14T11:19:47.737292+0000","last_clean":"2023-07-14T11:19:47.737292+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:19:47.737292+0000","last_undegraded":"2023-07-14T11:19:47.737292+0000","last_fullsized":"2023-07-14T11:19:47.737292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:38.334013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967956+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:19:47.967956+0000","last_peered":"2023-07-14T11:19:47.967956+0000","last_clean":"2023-07-14T11:19:47.967956+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:19:47.967956+0000","last_undegraded":"2023-07-14T11:19:47.967956+0000","last_fullsized":"2023-07-14T11:19:47.967956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:16.510067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081653+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:19:47.081653+0000","last_peered":"2023-07-14T11:19:47.081653+0000","last_clean":"2023-07-14T11:19:47.081653+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:19:47.081653+0000","last_undegraded":"2023-07-14T11:19:47.081653+0000","last_fullsized":"2023-07-14T11:19:47.081653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:35.054594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967147+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:19:47.967147+0000","last_peered":"2023-07-14T11:19:47.967147+0000","last_clean":"2023-07-14T11:19:47.967147+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:19:47.967147+0000","last_undegraded":"2023-07-14T11:19:47.967147+0000","last_fullsized":"2023-07-14T11:19:47.967147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:21:05.922286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711077+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:10.711077+0000","last_peered":"2023-07-14T11:20:10.711077+0000","last_clean":"2023-07-14T11:20:10.711077+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:10.711077+0000","last_undegraded":"2023-07-14T11:20:10.711077+0000","last_fullsized":"2023-07-14T11:20:10.711077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:02:01.788231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081629+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:19:47.081629+0000","last_peered":"2023-07-14T11:19:47.081629+0000","last_clean":"2023-07-14T11:19:47.081629+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:19:47.081629+0000","last_undegraded":"2023-07-14T11:19:47.081629+0000","last_fullsized":"2023-07-14T11:19:47.081629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:08.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711857+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:10.711857+0000","last_peered":"2023-07-14T11:20:10.711857+0000","last_clean":"2023-07-14T11:20:10.711857+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:10.711857+0000","last_undegraded":"2023-07-14T11:20:10.711857+0000","last_fullsized":"2023-07-14T11:20:10.711857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:05.009517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737241+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:19:47.737241+0000","last_peered":"2023-07-14T11:19:47.737241+0000","last_clean":"2023-07-14T11:19:47.737241+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:19:47.737241+0000","last_undegraded":"2023-07-14T11:19:47.737241+0000","last_fullsized":"2023-07-14T11:19:47.737241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:30:37.665531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967912+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:19:47.967912+0000","last_peered":"2023-07-14T11:19:47.967912+0000","last_clean":"2023-07-14T11:19:47.967912+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:19:47.967912+0000","last_undegraded":"2023-07-14T11:19:47.967912+0000","last_fullsized":"2023-07-14T11:19:47.967912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:39:26.015436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.710731+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:10.710731+0000","last_peered":"2023-07-14T11:20:10.710731+0000","last_clean":"2023-07-14T11:20:10.710731+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:10.710731+0000","last_undegraded":"2023-07-14T11:20:10.710731+0000","last_fullsized":"2023-07-14T11:20:10.710731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:44.419475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081641+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:19:47.081641+0000","last_peered":"2023-07-14T11:19:47.081641+0000","last_clean":"2023-07-14T11:19:47.081641+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:19:47.081641+0000","last_undegraded":"2023-07-14T11:19:47.081641+0000","last_fullsized":"2023-07-14T11:19:47.081641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:47.740065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711843+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:10.711843+0000","last_peered":"2023-07-14T11:20:10.711843+0000","last_clean":"2023-07-14T11:20:10.711843+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:10.711843+0000","last_undegraded":"2023-07-14T11:20:10.711843+0000","last_fullsized":"2023-07-14T11:20:10.711843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:13:57.943632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717973+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:14.717973+0000","last_peered":"2023-07-14T11:20:14.717973+0000","last_clean":"2023-07-14T11:20:14.717973+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:14.717973+0000","last_undegraded":"2023-07-14T11:20:14.717973+0000","last_fullsized":"2023-07-14T11:20:14.717973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:59.311439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969168+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:19:47.969168+0000","last_peered":"2023-07-14T11:19:47.969168+0000","last_clean":"2023-07-14T11:19:47.969168+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:19:47.969168+0000","last_undegraded":"2023-07-14T11:19:47.969168+0000","last_fullsized":"2023-07-14T11:19:47.969168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:56:28.718616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711234+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:10.711234+0000","last_peered":"2023-07-14T11:20:10.711234+0000","last_clean":"2023-07-14T11:20:10.711234+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:10.711234+0000","last_undegraded":"2023-07-14T11:20:10.711234+0000","last_fullsized":"2023-07-14T11:20:10.711234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:29:12.273324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728223+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:14.728223+0000","last_peered":"2023-07-14T11:20:14.728223+0000","last_clean":"2023-07-14T11:20:14.728223+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:14.728223+0000","last_undegraded":"2023-07-14T11:20:14.728223+0000","last_fullsized":"2023-07-14T11:20:14.728223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:45.374719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080735+0000","last_change":"2023-07-14T11:19:46.097705+0000","last_active":"2023-07-14T11:19:47.080735+0000","last_peered":"2023-07-14T11:19:47.080735+0000","last_clean":"2023-07-14T11:19:47.080735+0000","last_became_active":"2023-07-14T11:19:46.097093+0000","last_became_peered":"2023-07-14T11:19:46.097093+0000","last_unstale":"2023-07-14T11:19:47.080735+0000","last_undegraded":"2023-07-14T11:19:47.080735+0000","last_fullsized":"2023-07-14T11:19:47.080735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:16:13.697886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713460+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:10.713460+0000","last_peered":"2023-07-14T11:20:10.713460+0000","last_clean":"2023-07-14T11:20:10.713460+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:10.713460+0000","last_undegraded":"2023-07-14T11:20:10.713460+0000","last_fullsized":"2023-07-14T11:20:10.713460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:44.692262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968550+0000","last_change":"2023-07-14T11:19:42.077628+0000","last_active":"2023-07-14T11:19:47.968550+0000","last_peered":"2023-07-14T11:19:47.968550+0000","last_clean":"2023-07-14T11:19:47.968550+0000","last_became_active":"2023-07-14T11:19:42.077341+0000","last_became_peered":"2023-07-14T11:19:42.077341+0000","last_unstale":"2023-07-14T11:19:47.968550+0000","last_undegraded":"2023-07-14T11:19:47.968550+0000","last_fullsized":"2023-07-14T11:19:47.968550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:03.185626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082158+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:19:47.082158+0000","last_peered":"2023-07-14T11:19:47.082158+0000","last_clean":"2023-07-14T11:19:47.082158+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:19:47.082158+0000","last_undegraded":"2023-07-14T11:19:47.082158+0000","last_fullsized":"2023-07-14T11:19:47.082158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:07.141091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.680816+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:12.680816+0000","last_peered":"2023-07-14T11:20:12.680816+0000","last_clean":"2023-07-14T11:20:12.680816+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:12.680816+0000","last_undegraded":"2023-07-14T11:20:12.680816+0000","last_fullsized":"2023-07-14T11:20:12.680816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:29.624865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968266+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:19:47.968266+0000","last_peered":"2023-07-14T11:19:47.968266+0000","last_clean":"2023-07-14T11:19:47.968266+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:19:47.968266+0000","last_undegraded":"2023-07-14T11:19:47.968266+0000","last_fullsized":"2023-07-14T11:19:47.968266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:59.325039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728868+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:14.728868+0000","last_peered":"2023-07-14T11:20:14.728868+0000","last_clean":"2023-07-14T11:20:14.728868+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:14.728868+0000","last_undegraded":"2023-07-14T11:20:14.728868+0000","last_fullsized":"2023-07-14T11:20:14.728868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:31.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718779+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:14.718779+0000","last_peered":"2023-07-14T11:20:14.718779+0000","last_clean":"2023-07-14T11:20:14.718779+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:14.718779+0000","last_undegraded":"2023-07-14T11:20:14.718779+0000","last_fullsized":"2023-07-14T11:20:14.718779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:25:31.744731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967979+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:19:47.967979+0000","last_peered":"2023-07-14T11:19:47.967979+0000","last_clean":"2023-07-14T11:19:47.967979+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:19:47.967979+0000","last_undegraded":"2023-07-14T11:19:47.967979+0000","last_fullsized":"2023-07-14T11:19:47.967979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:58.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967884+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:19:47.967884+0000","last_peered":"2023-07-14T11:19:47.967884+0000","last_clean":"2023-07-14T11:19:47.967884+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:19:47.967884+0000","last_undegraded":"2023-07-14T11:19:47.967884+0000","last_fullsized":"2023-07-14T11:19:47.967884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:07.208804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717860+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:14.717860+0000","last_peered":"2023-07-14T11:20:14.717860+0000","last_clean":"2023-07-14T11:20:14.717860+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:14.717860+0000","last_undegraded":"2023-07-14T11:20:14.717860+0000","last_fullsized":"2023-07-14T11:20:14.717860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:15:42.131659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080903+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:19:47.080903+0000","last_peered":"2023-07-14T11:19:47.080903+0000","last_clean":"2023-07-14T11:19:47.080903+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:19:47.080903+0000","last_undegraded":"2023-07-14T11:19:47.080903+0000","last_fullsized":"2023-07-14T11:19:47.080903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:29.469059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T11:19:47.967804+0000","last_change":"2023-07-14T11:20:14.717622+0000","last_active":"2023-07-14T11:20:14.717622+0000","last_peered":"2023-07-14T11:19:47.967804+0000","last_clean":"2023-07-14T11:19:47.967804+0000","last_became_active":"2023-07-14T11:19:46.273092+0000","last_became_peered":"2023-07-14T11:19:46.273092+0000","last_unstale":"2023-07-14T11:19:47.967804+0000","last_undegraded":"2023-07-14T11:19:47.967804+0000","last_fullsized":"2023-07-14T11:19:47.967804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081920+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:19:47.081920+0000","last_peered":"2023-07-14T11:19:47.081920+0000","last_clean":"2023-07-14T11:19:47.081920+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:19:47.081920+0000","last_undegraded":"2023-07-14T11:19:47.081920+0000","last_fullsized":"2023-07-14T11:19:47.081920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:21:18.903408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081118+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:19:47.081118+0000","last_peered":"2023-07-14T11:19:47.081118+0000","last_clean":"2023-07-14T11:19:47.081118+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:19:47.081118+0000","last_undegraded":"2023-07-14T11:19:47.081118+0000","last_fullsized":"2023-07-14T11:19:47.081118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:37.312351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730373+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:14.730373+0000","last_peered":"2023-07-14T11:20:14.730373+0000","last_clean":"2023-07-14T11:20:14.730373+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:14.730373+0000","last_undegraded":"2023-07-14T11:20:14.730373+0000","last_fullsized":"2023-07-14T11:20:14.730373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:10.476499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968436+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:19:47.968436+0000","last_peered":"2023-07-14T11:19:47.968436+0000","last_clean":"2023-07-14T11:19:47.968436+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:19:47.968436+0000","last_undegraded":"2023-07-14T11:19:47.968436+0000","last_fullsized":"2023-07-14T11:19:47.968436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081008+0000","last_change":"2023-07-14T11:19:42.093066+0000","last_active":"2023-07-14T11:19:47.081008+0000","last_peered":"2023-07-14T11:19:47.081008+0000","last_clean":"2023-07-14T11:19:47.081008+0000","last_became_active":"2023-07-14T11:19:42.092719+0000","last_became_peered":"2023-07-14T11:19:42.092719+0000","last_unstale":"2023-07-14T11:19:47.081008+0000","last_undegraded":"2023-07-14T11:19:47.081008+0000","last_fullsized":"2023-07-14T11:19:47.081008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:35:05.018230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.729003+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:14.729003+0000","last_peered":"2023-07-14T11:20:14.729003+0000","last_clean":"2023-07-14T11:20:14.729003+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:14.729003+0000","last_undegraded":"2023-07-14T11:20:14.729003+0000","last_fullsized":"2023-07-14T11:20:14.729003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:29.125385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968186+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:19:47.968186+0000","last_peered":"2023-07-14T11:19:47.968186+0000","last_clean":"2023-07-14T11:19:47.968186+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:19:47.968186+0000","last_undegraded":"2023-07-14T11:19:47.968186+0000","last_fullsized":"2023-07-14T11:19:47.968186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:06.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731196+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:14.731196+0000","last_peered":"2023-07-14T11:20:14.731196+0000","last_clean":"2023-07-14T11:20:14.731196+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:14.731196+0000","last_undegraded":"2023-07-14T11:20:14.731196+0000","last_fullsized":"2023-07-14T11:20:14.731196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:16:34.542159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:49.452867+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:19:49.452867+0000","last_peered":"2023-07-14T11:19:49.452867+0000","last_clean":"2023-07-14T11:19:49.452867+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:19:49.452867+0000","last_undegraded":"2023-07-14T11:19:49.452867+0000","last_fullsized":"2023-07-14T11:19:49.452867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:22.680690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967124+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:19:47.967124+0000","last_peered":"2023-07-14T11:19:47.967124+0000","last_clean":"2023-07-14T11:19:47.967124+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:19:47.967124+0000","last_undegraded":"2023-07-14T11:19:47.967124+0000","last_fullsized":"2023-07-14T11:19:47.967124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:14:39.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.676206+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:12.676206+0000","last_peered":"2023-07-14T11:20:12.676206+0000","last_clean":"2023-07-14T11:20:12.676206+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:12.676206+0000","last_undegraded":"2023-07-14T11:20:12.676206+0000","last_fullsized":"2023-07-14T11:20:12.676206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:22:45.207699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968042+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:19:47.968042+0000","last_peered":"2023-07-14T11:19:47.968042+0000","last_clean":"2023-07-14T11:19:47.968042+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:19:47.968042+0000","last_undegraded":"2023-07-14T11:19:47.968042+0000","last_fullsized":"2023-07-14T11:19:47.968042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:23.767120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968619+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:19:47.968619+0000","last_peered":"2023-07-14T11:19:47.968619+0000","last_clean":"2023-07-14T11:19:47.968619+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:19:47.968619+0000","last_undegraded":"2023-07-14T11:19:47.968619+0000","last_fullsized":"2023-07-14T11:19:47.968619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:13.375758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731070+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:14.731070+0000","last_peered":"2023-07-14T11:20:14.731070+0000","last_clean":"2023-07-14T11:20:14.731070+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:14.731070+0000","last_undegraded":"2023-07-14T11:20:14.731070+0000","last_fullsized":"2023-07-14T11:20:14.731070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:59:07.405076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.717379+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:11.717379+0000","last_peered":"2023-07-14T11:20:11.717379+0000","last_clean":"2023-07-14T11:20:11.717379+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:11.717379+0000","last_undegraded":"2023-07-14T11:20:11.717379+0000","last_fullsized":"2023-07-14T11:20:11.717379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:24.177685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080724+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:19:47.080724+0000","last_peered":"2023-07-14T11:19:47.080724+0000","last_clean":"2023-07-14T11:19:47.080724+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:19:47.080724+0000","last_undegraded":"2023-07-14T11:19:47.080724+0000","last_fullsized":"2023-07-14T11:19:47.080724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:52:43.413334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718394+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:14.718394+0000","last_peered":"2023-07-14T11:20:14.718394+0000","last_clean":"2023-07-14T11:20:14.718394+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:14.718394+0000","last_undegraded":"2023-07-14T11:20:14.718394+0000","last_fullsized":"2023-07-14T11:20:14.718394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:22.172922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730551+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:14.730551+0000","last_peered":"2023-07-14T11:20:14.730551+0000","last_clean":"2023-07-14T11:20:14.730551+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:14.730551+0000","last_undegraded":"2023-07-14T11:20:14.730551+0000","last_fullsized":"2023-07-14T11:20:14.730551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:37.007482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967049+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:19:47.967049+0000","last_peered":"2023-07-14T11:19:47.967049+0000","last_clean":"2023-07-14T11:19:47.967049+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:19:47.967049+0000","last_undegraded":"2023-07-14T11:19:47.967049+0000","last_fullsized":"2023-07-14T11:19:47.967049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:39:36.064722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718705+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:11.718705+0000","last_peered":"2023-07-14T11:20:11.718705+0000","last_clean":"2023-07-14T11:20:11.718705+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:11.718705+0000","last_undegraded":"2023-07-14T11:20:11.718705+0000","last_fullsized":"2023-07-14T11:20:11.718705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:26:49.607683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967586+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:19:47.967586+0000","last_peered":"2023-07-14T11:19:47.967586+0000","last_clean":"2023-07-14T11:19:47.967586+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:19:47.967586+0000","last_undegraded":"2023-07-14T11:19:47.967586+0000","last_fullsized":"2023-07-14T11:19:47.967586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:35:51.028974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.196923+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:19:47.196923+0000","last_peered":"2023-07-14T11:19:47.196923+0000","last_clean":"2023-07-14T11:19:47.196923+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:19:47.196923+0000","last_undegraded":"2023-07-14T11:19:47.196923+0000","last_fullsized":"2023-07-14T11:19:47.196923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:20.885447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711775+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:10.711775+0000","last_peered":"2023-07-14T11:20:10.711775+0000","last_clean":"2023-07-14T11:20:10.711775+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:10.711775+0000","last_undegraded":"2023-07-14T11:20:10.711775+0000","last_fullsized":"2023-07-14T11:20:10.711775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:36.953428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737388+0000","last_change":"2023-07-14T11:19:42.085058+0000","last_active":"2023-07-14T11:19:47.737388+0000","last_peered":"2023-07-14T11:19:47.737388+0000","last_clean":"2023-07-14T11:19:47.737388+0000","last_became_active":"2023-07-14T11:19:42.083380+0000","last_became_peered":"2023-07-14T11:19:42.083380+0000","last_unstale":"2023-07-14T11:19:47.737388+0000","last_undegraded":"2023-07-14T11:19:47.737388+0000","last_fullsized":"2023-07-14T11:19:47.737388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:52.430115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967817+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:19:47.967817+0000","last_peered":"2023-07-14T11:19:47.967817+0000","last_clean":"2023-07-14T11:19:47.967817+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:19:47.967817+0000","last_undegraded":"2023-07-14T11:19:47.967817+0000","last_fullsized":"2023-07-14T11:19:47.967817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:25:48.734999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717656+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:14.717656+0000","last_peered":"2023-07-14T11:20:14.717656+0000","last_clean":"2023-07-14T11:20:14.717656+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:14.717656+0000","last_undegraded":"2023-07-14T11:20:14.717656+0000","last_fullsized":"2023-07-14T11:20:14.717656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:28.963610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082443+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:19:47.082443+0000","last_peered":"2023-07-14T11:19:47.082443+0000","last_clean":"2023-07-14T11:19:47.082443+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:19:47.082443+0000","last_undegraded":"2023-07-14T11:19:47.082443+0000","last_fullsized":"2023-07-14T11:19:47.082443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:43.469946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713281+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:10.713281+0000","last_peered":"2023-07-14T11:20:10.713281+0000","last_clean":"2023-07-14T11:20:10.713281+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:10.713281+0000","last_undegraded":"2023-07-14T11:20:10.713281+0000","last_fullsized":"2023-07-14T11:20:10.713281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:45:36.543244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080522+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:19:47.080522+0000","last_peered":"2023-07-14T11:19:47.080522+0000","last_clean":"2023-07-14T11:19:47.080522+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:19:47.080522+0000","last_undegraded":"2023-07-14T11:19:47.080522+0000","last_fullsized":"2023-07-14T11:19:47.080522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:44.548171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967752+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:19:47.967752+0000","last_peered":"2023-07-14T11:19:47.967752+0000","last_clean":"2023-07-14T11:19:47.967752+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:19:47.967752+0000","last_undegraded":"2023-07-14T11:19:47.967752+0000","last_fullsized":"2023-07-14T11:19:47.967752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:50:53.130285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197038+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:19:47.197038+0000","last_peered":"2023-07-14T11:19:47.197038+0000","last_clean":"2023-07-14T11:19:47.197038+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:19:47.197038+0000","last_undegraded":"2023-07-14T11:19:47.197038+0000","last_fullsized":"2023-07-14T11:19:47.197038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:15.617687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967360+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:19:47.967360+0000","last_peered":"2023-07-14T11:19:47.967360+0000","last_clean":"2023-07-14T11:19:47.967360+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:19:47.967360+0000","last_undegraded":"2023-07-14T11:19:47.967360+0000","last_fullsized":"2023-07-14T11:19:47.967360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:21.159717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967674+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:19:47.967674+0000","last_peered":"2023-07-14T11:19:47.967674+0000","last_clean":"2023-07-14T11:19:47.967674+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:19:47.967674+0000","last_undegraded":"2023-07-14T11:19:47.967674+0000","last_fullsized":"2023-07-14T11:19:47.967674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:57:50.109303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717280+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:14.717280+0000","last_peered":"2023-07-14T11:20:14.717280+0000","last_clean":"2023-07-14T11:20:14.717280+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:14.717280+0000","last_undegraded":"2023-07-14T11:20:14.717280+0000","last_fullsized":"2023-07-14T11:20:14.717280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:38:55.878246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968641+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:19:47.968641+0000","last_peered":"2023-07-14T11:19:47.968641+0000","last_clean":"2023-07-14T11:19:47.968641+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:19:47.968641+0000","last_undegraded":"2023-07-14T11:19:47.968641+0000","last_fullsized":"2023-07-14T11:19:47.968641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:57.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197577+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:19:47.197577+0000","last_peered":"2023-07-14T11:19:47.197577+0000","last_clean":"2023-07-14T11:19:47.197577+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:19:47.197577+0000","last_undegraded":"2023-07-14T11:19:47.197577+0000","last_fullsized":"2023-07-14T11:19:47.197577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:48.609439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969031+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:19:47.969031+0000","last_peered":"2023-07-14T11:19:47.969031+0000","last_clean":"2023-07-14T11:19:47.969031+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:19:47.969031+0000","last_undegraded":"2023-07-14T11:19:47.969031+0000","last_fullsized":"2023-07-14T11:19:47.969031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:55.843732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736878+0000","last_change":"2023-07-14T11:19:46.088344+0000","last_active":"2023-07-14T11:19:47.736878+0000","last_peered":"2023-07-14T11:19:47.736878+0000","last_clean":"2023-07-14T11:19:47.736878+0000","last_became_active":"2023-07-14T11:19:46.087922+0000","last_became_peered":"2023-07-14T11:19:46.087922+0000","last_unstale":"2023-07-14T11:19:47.736878+0000","last_undegraded":"2023-07-14T11:19:47.736878+0000","last_fullsized":"2023-07-14T11:19:47.736878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:02:04.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968974+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:19:47.968974+0000","last_peered":"2023-07-14T11:19:47.968974+0000","last_clean":"2023-07-14T11:19:47.968974+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:19:47.968974+0000","last_undegraded":"2023-07-14T11:19:47.968974+0000","last_fullsized":"2023-07-14T11:19:47.968974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:56:35.324459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968655+0000","last_change":"2023-07-14T11:19:42.073286+0000","last_active":"2023-07-14T11:19:47.968655+0000","last_peered":"2023-07-14T11:19:47.968655+0000","last_clean":"2023-07-14T11:19:47.968655+0000","last_became_active":"2023-07-14T11:19:42.072976+0000","last_became_peered":"2023-07-14T11:19:42.072976+0000","last_unstale":"2023-07-14T11:19:47.968655+0000","last_undegraded":"2023-07-14T11:19:47.968655+0000","last_fullsized":"2023-07-14T11:19:47.968655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:37:29.066629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967707+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:19:47.967707+0000","last_peered":"2023-07-14T11:19:47.967707+0000","last_clean":"2023-07-14T11:19:47.967707+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:19:47.967707+0000","last_undegraded":"2023-07-14T11:19:47.967707+0000","last_fullsized":"2023-07-14T11:19:47.967707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:15.123989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728744+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:14.728744+0000","last_peered":"2023-07-14T11:20:14.728744+0000","last_clean":"2023-07-14T11:20:14.728744+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:14.728744+0000","last_undegraded":"2023-07-14T11:20:14.728744+0000","last_fullsized":"2023-07-14T11:20:14.728744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:03:42.954256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968714+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:19:47.968714+0000","last_peered":"2023-07-14T11:19:47.968714+0000","last_clean":"2023-07-14T11:19:47.968714+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:19:47.968714+0000","last_undegraded":"2023-07-14T11:19:47.968714+0000","last_fullsized":"2023-07-14T11:19:47.968714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:03.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080574+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:19:47.080574+0000","last_peered":"2023-07-14T11:19:47.080574+0000","last_clean":"2023-07-14T11:19:47.080574+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:19:47.080574+0000","last_undegraded":"2023-07-14T11:19:47.080574+0000","last_fullsized":"2023-07-14T11:19:47.080574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:06.602465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197215+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:19:47.197215+0000","last_peered":"2023-07-14T11:19:47.197215+0000","last_clean":"2023-07-14T11:19:47.197215+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:19:47.197215+0000","last_undegraded":"2023-07-14T11:19:47.197215+0000","last_fullsized":"2023-07-14T11:19:47.197215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:52:37.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197079+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:19:47.197079+0000","last_peered":"2023-07-14T11:19:47.197079+0000","last_clean":"2023-07-14T11:19:47.197079+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:19:47.197079+0000","last_undegraded":"2023-07-14T11:19:47.197079+0000","last_fullsized":"2023-07-14T11:19:47.197079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:26:40.329142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736992+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:19:47.736992+0000","last_peered":"2023-07-14T11:19:47.736992+0000","last_clean":"2023-07-14T11:19:47.736992+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:19:47.736992+0000","last_undegraded":"2023-07-14T11:19:47.736992+0000","last_fullsized":"2023-07-14T11:19:47.736992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:09:09.081413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730659+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:14.730659+0000","last_peered":"2023-07-14T11:20:14.730659+0000","last_clean":"2023-07-14T11:20:14.730659+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:14.730659+0000","last_undegraded":"2023-07-14T11:20:14.730659+0000","last_fullsized":"2023-07-14T11:20:14.730659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:18.996495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081049+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:19:47.081049+0000","last_peered":"2023-07-14T11:19:47.081049+0000","last_clean":"2023-07-14T11:19:47.081049+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:19:47.081049+0000","last_undegraded":"2023-07-14T11:19:47.081049+0000","last_fullsized":"2023-07-14T11:19:47.081049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:56.251387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197612+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:19:47.197612+0000","last_peered":"2023-07-14T11:19:47.197612+0000","last_clean":"2023-07-14T11:19:47.197612+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:19:47.197612+0000","last_undegraded":"2023-07-14T11:19:47.197612+0000","last_fullsized":"2023-07-14T11:19:47.197612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:32:51.167057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967746+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:19:47.967746+0000","last_peered":"2023-07-14T11:19:47.967746+0000","last_clean":"2023-07-14T11:19:47.967746+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:19:47.967746+0000","last_undegraded":"2023-07-14T11:19:47.967746+0000","last_fullsized":"2023-07-14T11:19:47.967746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:37.267773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-14T11:19:42.068794+0000","last_change":"2023-07-14T11:20:15.683458+0000","last_active":"2023-07-14T11:19:41.086153+0000","last_peered":"2023-07-14T11:19:41.086153+0000","last_clean":"2023-07-14T11:19:41.086153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:19:42.068794+0000","last_undegraded":"2023-07-14T11:19:42.068794+0000","last_fullsized":"2023-07-14T11:19:42.068794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967293+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:19:47.967293+0000","last_peered":"2023-07-14T11:19:47.967293+0000","last_clean":"2023-07-14T11:19:47.967293+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:19:47.967293+0000","last_undegraded":"2023-07-14T11:19:47.967293+0000","last_fullsized":"2023-07-14T11:19:47.967293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:58:16.557411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727007+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:14.727007+0000","last_peered":"2023-07-14T11:20:14.727007+0000","last_clean":"2023-07-14T11:20:14.727007+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:14.727007+0000","last_undegraded":"2023-07-14T11:20:14.727007+0000","last_fullsized":"2023-07-14T11:20:14.727007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:33.588223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080793+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:19:47.080793+0000","last_peered":"2023-07-14T11:19:47.080793+0000","last_clean":"2023-07-14T11:19:47.080793+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:19:47.080793+0000","last_undegraded":"2023-07-14T11:19:47.080793+0000","last_fullsized":"2023-07-14T11:19:47.080793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:57.953734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":121,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":60474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":471801,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.37},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.755},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.78000000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1062815,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.876},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82999999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":467907,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.628},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.1639999999999997,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70299999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.59799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":481467,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93000000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0129999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1062952,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":466940,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.439},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.85099999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.876},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":515148,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.84799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.623},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.752}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":468328,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":1.042},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.55800000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.621},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.64800000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":1.147}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.99099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":471252,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56299999999999994}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467317,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.376}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.5},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.622}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73899999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467719,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54000000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1058588,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.624}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50800000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:16.514 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:16.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 -- 172.21.15.161:0/1518487913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c0100d00 msgr2=0x7f73c0101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 --2- 172.21.15.161:0/1518487913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c0100d00 0x7f73c0101150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009990 tx=0x7f73b0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 -- 172.21.15.161:0/1518487913 shutdown_connections 2023-07-14T11:20:16.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 --2- 172.21.15.161:0/1518487913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c0101690 0x7f73c0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 --2- 172.21.15.161:0/1518487913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f73c00ffb10 0x7f73c00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 --2- 172.21.15.161:0/1518487913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c0100d00 0x7f73c0101150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 -- 172.21.15.161:0/1518487913 >> 172.21.15.161:0/1518487913 conn(0x7f73c00fb110 msgr2=0x7f73c00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:16.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 -- 172.21.15.161:0/1518487913 shutdown_connections 2023-07-14T11:20:16.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.743+0000 7f73c85ff700 1 -- 172.21.15.161:0/1518487913 wait complete. 2023-07-14T11:20:16.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 Processor -- start 2023-07-14T11:20:16.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 -- start start 2023-07-14T11:20:16.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ffb10 0x7f73c01957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f73c0100d00 0x7f73c0195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c0101690 0x7f73c019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019d330 con 0x7f73c00ffb10 2023-07-14T11:20:16.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019d470 con 0x7f73c0101690 2023-07-14T11:20:16.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.744+0000 7f73c85ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019d600 con 0x7f73c0100d00 2023-07-14T11:20:16.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ffb10 0x7f73c01957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ffb10 0x7f73c01957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45188/0 (socket says 172.21.15.161:45188) 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 -- 172.21.15.161:0/468144253 learned_addr learned my addr 172.21.15.161:0/468144253 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c5b9a700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f73c0100d00 0x7f73c0195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c6b9c700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c0101690 0x7f73c019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 -- 172.21.15.161:0/468144253 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f73c0100d00 msgr2=0x7f73c0195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f73c0100d00 0x7f73c0195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 -- 172.21.15.161:0/468144253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c0101690 msgr2=0x7f73c019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c0101690 0x7f73c019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 -- 172.21.15.161:0/468144253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b00095f0 con 0x7f73c00ffb10 2023-07-14T11:20:16.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c639b700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ffb10 0x7f73c01957b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e970 tx=0x7f73bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:16.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73b77fe700 1 -- 172.21.15.161:0/468144253 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc009790 con 0x7f73c00ffb10 2023-07-14T11:20:16.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73b77fe700 1 -- 172.21.15.161:0/468144253 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73bc00c7e0 con 0x7f73c00ffb10 2023-07-14T11:20:16.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c019d8e0 con 0x7f73c00ffb10 2023-07-14T11:20:16.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.745+0000 7f73b77fe700 1 -- 172.21.15.161:0/468144253 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc003c70 con 0x7f73c00ffb10 2023-07-14T11:20:16.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.746+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c019ddf0 con 0x7f73c00ffb10 2023-07-14T11:20:16.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.747+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c0107000 con 0x7f73c00ffb10 2023-07-14T11:20:16.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.748+0000 7f73b77fe700 1 -- 172.21.15.161:0/468144253 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f73bc00cc50 con 0x7f73c00ffb10 2023-07-14T11:20:16.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.751+0000 7f73b77fe700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f73ac0a1220 0x7f73ac0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:16.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.751+0000 7f73b77fe700 1 -- 172.21.15.161:0/468144253 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f73bc014070 con 0x7f73c00ffb10 2023-07-14T11:20:16.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.751+0000 7f73c5b9a700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f73ac0a1220 0x7f73ac0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:16.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.751+0000 7f73c5b9a700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f73ac0a1220 0x7f73ac0a36d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f73b000b620 tx=0x7f73b0005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:16.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.754+0000 7f73b77fe700 1 -- 172.21.15.161:0/468144253 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f73bc096220 con 0x7f73c00ffb10 2023-07-14T11:20:16.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.949+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73c0060950 con 0x7f73ac0a1220 2023-07-14T11:20:16.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.962+0000 7f73b77fe700 1 -- 172.21.15.161:0/468144253 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422840 (secure 0 0 0) 0x7f73c0060950 con 0x7f73ac0a1220 2023-07-14T11:20:16.972 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:16.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f73ac0a1220 msgr2=0x7f73ac0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f73ac0a1220 0x7f73ac0a36d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f73b000b620 tx=0x7f73b0005960 comp rx=0 tx=0).stop 2023-07-14T11:20:16.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ffb10 msgr2=0x7f73c01957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:16.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ffb10 0x7f73c01957b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e970 tx=0x7f73bc00ec80 comp rx=0 tx=0).stop 2023-07-14T11:20:16.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 shutdown_connections 2023-07-14T11:20:16.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ffb10 0x7f73c01957b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f73ac0a1220 0x7f73ac0a36d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c0101690 0x7f73c019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 --2- 172.21.15.161:0/468144253 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f73c0100d00 0x7f73c0195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:16.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 >> 172.21.15.161:0/468144253 conn(0x7f73c00fb110 msgr2=0x7f73c0104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:16.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 shutdown_connections 2023-07-14T11:20:16.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:16.967+0000 7f73c85ff700 1 -- 172.21.15.161:0/468144253 wait complete. 2023-07-14T11:20:16.979 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:16.986 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:16 smithi161 ceph-mon[95526]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T11:20:16.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:16 smithi161 ceph-mon[95526]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:16.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:16 smithi161 ceph-mon[95526]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2023-07-14T11:20:16.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:16 smithi161 ceph-mon[95526]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:16.989 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-07-14T11:20:16.103870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597217,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":156,"num_read_kb":137,"num_write":303,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334928,"kb_used_data":11504,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606896,"statfs":{"total":1151940427776,"available":1151597461504,"internally_reserved":0,"allocated":11780096,"data_stored":7462234,"data_compressed":253459,"data_compressed_allocated":339968,"data_compressed_original":2179072,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2257,"num_objects":35,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002134"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080953+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:19:47.080953+0000","last_peered":"2023-07-14T11:19:47.080953+0000","last_clean":"2023-07-14T11:19:47.080953+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:19:47.080953+0000","last_undegraded":"2023-07-14T11:19:47.080953+0000","last_fullsized":"2023-07-14T11:19:47.080953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:57.564463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737248+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:19:47.737248+0000","last_peered":"2023-07-14T11:19:47.737248+0000","last_clean":"2023-07-14T11:19:47.737248+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:19:47.737248+0000","last_undegraded":"2023-07-14T11:19:47.737248+0000","last_fullsized":"2023-07-14T11:19:47.737248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:19.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737244+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:19:47.737244+0000","last_peered":"2023-07-14T11:19:47.737244+0000","last_clean":"2023-07-14T11:19:47.737244+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:19:47.737244+0000","last_undegraded":"2023-07-14T11:19:47.737244+0000","last_fullsized":"2023-07-14T11:19:47.737244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:47.796849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968258+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:19:47.968258+0000","last_peered":"2023-07-14T11:19:47.968258+0000","last_clean":"2023-07-14T11:19:47.968258+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:19:47.968258+0000","last_undegraded":"2023-07-14T11:19:47.968258+0000","last_fullsized":"2023-07-14T11:19:47.968258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:18:31.399322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717211+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:14.717211+0000","last_peered":"2023-07-14T11:20:14.717211+0000","last_clean":"2023-07-14T11:20:14.717211+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:14.717211+0000","last_undegraded":"2023-07-14T11:20:14.717211+0000","last_fullsized":"2023-07-14T11:20:14.717211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:45:02.418574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081056+0000","last_change":"2023-07-14T11:19:42.092136+0000","last_active":"2023-07-14T11:19:47.081056+0000","last_peered":"2023-07-14T11:19:47.081056+0000","last_clean":"2023-07-14T11:19:47.081056+0000","last_became_active":"2023-07-14T11:19:42.091820+0000","last_became_peered":"2023-07-14T11:19:42.091820+0000","last_unstale":"2023-07-14T11:19:47.081056+0000","last_undegraded":"2023-07-14T11:19:47.081056+0000","last_fullsized":"2023-07-14T11:19:47.081056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:01.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968087+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:19:47.968087+0000","last_peered":"2023-07-14T11:19:47.968087+0000","last_clean":"2023-07-14T11:19:47.968087+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:19:47.968087+0000","last_undegraded":"2023-07-14T11:19:47.968087+0000","last_fullsized":"2023-07-14T11:19:47.968087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:21.428012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967192+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:19:47.967192+0000","last_peered":"2023-07-14T11:19:47.967192+0000","last_clean":"2023-07-14T11:19:47.967192+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:19:47.967192+0000","last_undegraded":"2023-07-14T11:19:47.967192+0000","last_fullsized":"2023-07-14T11:19:47.967192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:38.965332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081420+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:19:47.081420+0000","last_peered":"2023-07-14T11:19:47.081420+0000","last_clean":"2023-07-14T11:19:47.081420+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:19:47.081420+0000","last_undegraded":"2023-07-14T11:19:47.081420+0000","last_fullsized":"2023-07-14T11:19:47.081420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:51.712868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969101+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:19:47.969101+0000","last_peered":"2023-07-14T11:19:47.969101+0000","last_clean":"2023-07-14T11:19:47.969101+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:19:47.969101+0000","last_undegraded":"2023-07-14T11:19:47.969101+0000","last_fullsized":"2023-07-14T11:19:47.969101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:48:50.962800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968204+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:19:47.968204+0000","last_peered":"2023-07-14T11:19:47.968204+0000","last_clean":"2023-07-14T11:19:47.968204+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:19:47.968204+0000","last_undegraded":"2023-07-14T11:19:47.968204+0000","last_fullsized":"2023-07-14T11:19:47.968204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:47.264465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727198+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:14.727198+0000","last_peered":"2023-07-14T11:20:14.727198+0000","last_clean":"2023-07-14T11:20:14.727198+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:14.727198+0000","last_undegraded":"2023-07-14T11:20:14.727198+0000","last_fullsized":"2023-07-14T11:20:14.727198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:42.155482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736832+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:19:47.736832+0000","last_peered":"2023-07-14T11:19:47.736832+0000","last_clean":"2023-07-14T11:19:47.736832+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:19:47.736832+0000","last_undegraded":"2023-07-14T11:19:47.736832+0000","last_fullsized":"2023-07-14T11:19:47.736832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:26.314511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736956+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:19:47.736956+0000","last_peered":"2023-07-14T11:19:47.736956+0000","last_clean":"2023-07-14T11:19:47.736956+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:19:47.736956+0000","last_undegraded":"2023-07-14T11:19:47.736956+0000","last_fullsized":"2023-07-14T11:19:47.736956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:10:23.738358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081137+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:19:47.081137+0000","last_peered":"2023-07-14T11:19:47.081137+0000","last_clean":"2023-07-14T11:19:47.081137+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:19:47.081137+0000","last_undegraded":"2023-07-14T11:19:47.081137+0000","last_fullsized":"2023-07-14T11:19:47.081137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:51.121664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082305+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:19:47.082305+0000","last_peered":"2023-07-14T11:19:47.082305+0000","last_clean":"2023-07-14T11:19:47.082305+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:19:47.082305+0000","last_undegraded":"2023-07-14T11:19:47.082305+0000","last_fullsized":"2023-07-14T11:19:47.082305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:07:45.552513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197608+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:19:47.197608+0000","last_peered":"2023-07-14T11:19:47.197608+0000","last_clean":"2023-07-14T11:19:47.197608+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:19:47.197608+0000","last_undegraded":"2023-07-14T11:19:47.197608+0000","last_fullsized":"2023-07-14T11:19:47.197608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:44.466407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197036+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:19:47.197036+0000","last_peered":"2023-07-14T11:19:47.197036+0000","last_clean":"2023-07-14T11:19:47.197036+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:19:47.197036+0000","last_undegraded":"2023-07-14T11:19:47.197036+0000","last_fullsized":"2023-07-14T11:19:47.197036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:54.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.719668+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:11.719668+0000","last_peered":"2023-07-14T11:20:11.719668+0000","last_clean":"2023-07-14T11:20:11.719668+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:11.719668+0000","last_undegraded":"2023-07-14T11:20:11.719668+0000","last_fullsized":"2023-07-14T11:20:11.719668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:13:47.991877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967522+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:19:47.967522+0000","last_peered":"2023-07-14T11:19:47.967522+0000","last_clean":"2023-07-14T11:19:47.967522+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:19:47.967522+0000","last_undegraded":"2023-07-14T11:19:47.967522+0000","last_fullsized":"2023-07-14T11:19:47.967522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:21.029051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967474+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:19:47.967474+0000","last_peered":"2023-07-14T11:19:47.967474+0000","last_clean":"2023-07-14T11:19:47.967474+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:19:47.967474+0000","last_undegraded":"2023-07-14T11:19:47.967474+0000","last_fullsized":"2023-07-14T11:19:47.967474+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:10.344354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967640+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:19:47.967640+0000","last_peered":"2023-07-14T11:19:47.967640+0000","last_clean":"2023-07-14T11:19:47.967640+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:19:47.967640+0000","last_undegraded":"2023-07-14T11:19:47.967640+0000","last_fullsized":"2023-07-14T11:19:47.967640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:14.404398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737187+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:19:47.737187+0000","last_peered":"2023-07-14T11:19:47.737187+0000","last_clean":"2023-07-14T11:19:47.737187+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:19:47.737187+0000","last_undegraded":"2023-07-14T11:19:47.737187+0000","last_fullsized":"2023-07-14T11:19:47.737187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:32:25.980256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967578+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:19:47.967578+0000","last_peered":"2023-07-14T11:19:47.967578+0000","last_clean":"2023-07-14T11:19:47.967578+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:19:47.967578+0000","last_undegraded":"2023-07-14T11:19:47.967578+0000","last_fullsized":"2023-07-14T11:19:47.967578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:02:46.721446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718300+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:14.718300+0000","last_peered":"2023-07-14T11:20:14.718300+0000","last_clean":"2023-07-14T11:20:14.718300+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:14.718300+0000","last_undegraded":"2023-07-14T11:20:14.718300+0000","last_fullsized":"2023-07-14T11:20:14.718300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:53:43.048114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967447+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:19:47.967447+0000","last_peered":"2023-07-14T11:19:47.967447+0000","last_clean":"2023-07-14T11:19:47.967447+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:19:47.967447+0000","last_undegraded":"2023-07-14T11:19:47.967447+0000","last_fullsized":"2023-07-14T11:19:47.967447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:38.925420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717638+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:14.717638+0000","last_peered":"2023-07-14T11:20:14.717638+0000","last_clean":"2023-07-14T11:20:14.717638+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:14.717638+0000","last_undegraded":"2023-07-14T11:20:14.717638+0000","last_fullsized":"2023-07-14T11:20:14.717638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:20.321448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968913+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:19:47.968913+0000","last_peered":"2023-07-14T11:19:47.968913+0000","last_clean":"2023-07-14T11:19:47.968913+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:19:47.968913+0000","last_undegraded":"2023-07-14T11:19:47.968913+0000","last_fullsized":"2023-07-14T11:19:47.968913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:18.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197409+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:19:47.197409+0000","last_peered":"2023-07-14T11:19:47.197409+0000","last_clean":"2023-07-14T11:19:47.197409+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:19:47.197409+0000","last_undegraded":"2023-07-14T11:19:47.197409+0000","last_fullsized":"2023-07-14T11:19:47.197409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:17.534514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.678617+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:12.678617+0000","last_peered":"2023-07-14T11:20:12.678617+0000","last_clean":"2023-07-14T11:20:12.678617+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:12.678617+0000","last_undegraded":"2023-07-14T11:20:12.678617+0000","last_fullsized":"2023-07-14T11:20:12.678617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.545637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718422+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:14.718422+0000","last_peered":"2023-07-14T11:20:14.718422+0000","last_clean":"2023-07-14T11:20:14.718422+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:14.718422+0000","last_undegraded":"2023-07-14T11:20:14.718422+0000","last_fullsized":"2023-07-14T11:20:14.718422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:25.009048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197411+0000","last_change":"2023-07-14T11:19:46.096809+0000","last_active":"2023-07-14T11:19:47.197411+0000","last_peered":"2023-07-14T11:19:47.197411+0000","last_clean":"2023-07-14T11:19:47.197411+0000","last_became_active":"2023-07-14T11:19:46.096322+0000","last_became_peered":"2023-07-14T11:19:46.096322+0000","last_unstale":"2023-07-14T11:19:47.197411+0000","last_undegraded":"2023-07-14T11:19:47.197411+0000","last_fullsized":"2023-07-14T11:19:47.197411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:48:14.145380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718924+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:11.718924+0000","last_peered":"2023-07-14T11:20:11.718924+0000","last_clean":"2023-07-14T11:20:11.718924+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:11.718924+0000","last_undegraded":"2023-07-14T11:20:11.718924+0000","last_fullsized":"2023-07-14T11:20:11.718924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:52.081384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968898+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:19:47.968898+0000","last_peered":"2023-07-14T11:19:47.968898+0000","last_clean":"2023-07-14T11:19:47.968898+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:19:47.968898+0000","last_undegraded":"2023-07-14T11:19:47.968898+0000","last_fullsized":"2023-07-14T11:19:47.968898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:04.665047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081037+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:19:47.081037+0000","last_peered":"2023-07-14T11:19:47.081037+0000","last_clean":"2023-07-14T11:19:47.081037+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:19:47.081037+0000","last_undegraded":"2023-07-14T11:19:47.081037+0000","last_fullsized":"2023-07-14T11:19:47.081037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:25.917595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731318+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:14.731318+0000","last_peered":"2023-07-14T11:20:14.731318+0000","last_clean":"2023-07-14T11:20:14.731318+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:14.731318+0000","last_undegraded":"2023-07-14T11:20:14.731318+0000","last_fullsized":"2023-07-14T11:20:14.731318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:39.839896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967040+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:19:47.967040+0000","last_peered":"2023-07-14T11:19:47.967040+0000","last_clean":"2023-07-14T11:19:47.967040+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:19:47.967040+0000","last_undegraded":"2023-07-14T11:19:47.967040+0000","last_fullsized":"2023-07-14T11:19:47.967040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:43.960608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737292+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:19:47.737292+0000","last_peered":"2023-07-14T11:19:47.737292+0000","last_clean":"2023-07-14T11:19:47.737292+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:19:47.737292+0000","last_undegraded":"2023-07-14T11:19:47.737292+0000","last_fullsized":"2023-07-14T11:19:47.737292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:38.334013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967956+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:19:47.967956+0000","last_peered":"2023-07-14T11:19:47.967956+0000","last_clean":"2023-07-14T11:19:47.967956+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:19:47.967956+0000","last_undegraded":"2023-07-14T11:19:47.967956+0000","last_fullsized":"2023-07-14T11:19:47.967956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:16.510067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081653+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:19:47.081653+0000","last_peered":"2023-07-14T11:19:47.081653+0000","last_clean":"2023-07-14T11:19:47.081653+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:19:47.081653+0000","last_undegraded":"2023-07-14T11:19:47.081653+0000","last_fullsized":"2023-07-14T11:19:47.081653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:35.054594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967147+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:19:47.967147+0000","last_peered":"2023-07-14T11:19:47.967147+0000","last_clean":"2023-07-14T11:19:47.967147+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:19:47.967147+0000","last_undegraded":"2023-07-14T11:19:47.967147+0000","last_fullsized":"2023-07-14T11:19:47.967147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:21:05.922286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711077+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:10.711077+0000","last_peered":"2023-07-14T11:20:10.711077+0000","last_clean":"2023-07-14T11:20:10.711077+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:10.711077+0000","last_undegraded":"2023-07-14T11:20:10.711077+0000","last_fullsized":"2023-07-14T11:20:10.711077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:02:01.788231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081629+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:19:47.081629+0000","last_peered":"2023-07-14T11:19:47.081629+0000","last_clean":"2023-07-14T11:19:47.081629+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:19:47.081629+0000","last_undegraded":"2023-07-14T11:19:47.081629+0000","last_fullsized":"2023-07-14T11:19:47.081629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:08.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711857+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:10.711857+0000","last_peered":"2023-07-14T11:20:10.711857+0000","last_clean":"2023-07-14T11:20:10.711857+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:10.711857+0000","last_undegraded":"2023-07-14T11:20:10.711857+0000","last_fullsized":"2023-07-14T11:20:10.711857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:05.009517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737241+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:19:47.737241+0000","last_peered":"2023-07-14T11:19:47.737241+0000","last_clean":"2023-07-14T11:19:47.737241+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:19:47.737241+0000","last_undegraded":"2023-07-14T11:19:47.737241+0000","last_fullsized":"2023-07-14T11:19:47.737241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:30:37.665531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967912+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:19:47.967912+0000","last_peered":"2023-07-14T11:19:47.967912+0000","last_clean":"2023-07-14T11:19:47.967912+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:19:47.967912+0000","last_undegraded":"2023-07-14T11:19:47.967912+0000","last_fullsized":"2023-07-14T11:19:47.967912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:39:26.015436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.710731+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:10.710731+0000","last_peered":"2023-07-14T11:20:10.710731+0000","last_clean":"2023-07-14T11:20:10.710731+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:10.710731+0000","last_undegraded":"2023-07-14T11:20:10.710731+0000","last_fullsized":"2023-07-14T11:20:10.710731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:44.419475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081641+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:19:47.081641+0000","last_peered":"2023-07-14T11:19:47.081641+0000","last_clean":"2023-07-14T11:19:47.081641+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:19:47.081641+0000","last_undegraded":"2023-07-14T11:19:47.081641+0000","last_fullsized":"2023-07-14T11:19:47.081641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:47.740065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711843+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:10.711843+0000","last_peered":"2023-07-14T11:20:10.711843+0000","last_clean":"2023-07-14T11:20:10.711843+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:10.711843+0000","last_undegraded":"2023-07-14T11:20:10.711843+0000","last_fullsized":"2023-07-14T11:20:10.711843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:13:57.943632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717973+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:14.717973+0000","last_peered":"2023-07-14T11:20:14.717973+0000","last_clean":"2023-07-14T11:20:14.717973+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:14.717973+0000","last_undegraded":"2023-07-14T11:20:14.717973+0000","last_fullsized":"2023-07-14T11:20:14.717973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:59.311439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969168+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:19:47.969168+0000","last_peered":"2023-07-14T11:19:47.969168+0000","last_clean":"2023-07-14T11:19:47.969168+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:19:47.969168+0000","last_undegraded":"2023-07-14T11:19:47.969168+0000","last_fullsized":"2023-07-14T11:19:47.969168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:56:28.718616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711234+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:10.711234+0000","last_peered":"2023-07-14T11:20:10.711234+0000","last_clean":"2023-07-14T11:20:10.711234+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:10.711234+0000","last_undegraded":"2023-07-14T11:20:10.711234+0000","last_fullsized":"2023-07-14T11:20:10.711234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:29:12.273324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728223+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:14.728223+0000","last_peered":"2023-07-14T11:20:14.728223+0000","last_clean":"2023-07-14T11:20:14.728223+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:14.728223+0000","last_undegraded":"2023-07-14T11:20:14.728223+0000","last_fullsized":"2023-07-14T11:20:14.728223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:45.374719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080735+0000","last_change":"2023-07-14T11:19:46.097705+0000","last_active":"2023-07-14T11:19:47.080735+0000","last_peered":"2023-07-14T11:19:47.080735+0000","last_clean":"2023-07-14T11:19:47.080735+0000","last_became_active":"2023-07-14T11:19:46.097093+0000","last_became_peered":"2023-07-14T11:19:46.097093+0000","last_unstale":"2023-07-14T11:19:47.080735+0000","last_undegraded":"2023-07-14T11:19:47.080735+0000","last_fullsized":"2023-07-14T11:19:47.080735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:16:13.697886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713460+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:10.713460+0000","last_peered":"2023-07-14T11:20:10.713460+0000","last_clean":"2023-07-14T11:20:10.713460+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:10.713460+0000","last_undegraded":"2023-07-14T11:20:10.713460+0000","last_fullsized":"2023-07-14T11:20:10.713460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:44.692262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968550+0000","last_change":"2023-07-14T11:19:42.077628+0000","last_active":"2023-07-14T11:19:47.968550+0000","last_peered":"2023-07-14T11:19:47.968550+0000","last_clean":"2023-07-14T11:19:47.968550+0000","last_became_active":"2023-07-14T11:19:42.077341+0000","last_became_peered":"2023-07-14T11:19:42.077341+0000","last_unstale":"2023-07-14T11:19:47.968550+0000","last_undegraded":"2023-07-14T11:19:47.968550+0000","last_fullsized":"2023-07-14T11:19:47.968550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:03.185626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082158+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:19:47.082158+0000","last_peered":"2023-07-14T11:19:47.082158+0000","last_clean":"2023-07-14T11:19:47.082158+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:19:47.082158+0000","last_undegraded":"2023-07-14T11:19:47.082158+0000","last_fullsized":"2023-07-14T11:19:47.082158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:07.141091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.680816+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:12.680816+0000","last_peered":"2023-07-14T11:20:12.680816+0000","last_clean":"2023-07-14T11:20:12.680816+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:12.680816+0000","last_undegraded":"2023-07-14T11:20:12.680816+0000","last_fullsized":"2023-07-14T11:20:12.680816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:29.624865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968266+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:19:47.968266+0000","last_peered":"2023-07-14T11:19:47.968266+0000","last_clean":"2023-07-14T11:19:47.968266+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:19:47.968266+0000","last_undegraded":"2023-07-14T11:19:47.968266+0000","last_fullsized":"2023-07-14T11:19:47.968266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:59.325039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728868+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:14.728868+0000","last_peered":"2023-07-14T11:20:14.728868+0000","last_clean":"2023-07-14T11:20:14.728868+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:14.728868+0000","last_undegraded":"2023-07-14T11:20:14.728868+0000","last_fullsized":"2023-07-14T11:20:14.728868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:31.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718779+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:14.718779+0000","last_peered":"2023-07-14T11:20:14.718779+0000","last_clean":"2023-07-14T11:20:14.718779+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:14.718779+0000","last_undegraded":"2023-07-14T11:20:14.718779+0000","last_fullsized":"2023-07-14T11:20:14.718779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:25:31.744731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967979+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:19:47.967979+0000","last_peered":"2023-07-14T11:19:47.967979+0000","last_clean":"2023-07-14T11:19:47.967979+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:19:47.967979+0000","last_undegraded":"2023-07-14T11:19:47.967979+0000","last_fullsized":"2023-07-14T11:19:47.967979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:58.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967884+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:19:47.967884+0000","last_peered":"2023-07-14T11:19:47.967884+0000","last_clean":"2023-07-14T11:19:47.967884+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:19:47.967884+0000","last_undegraded":"2023-07-14T11:19:47.967884+0000","last_fullsized":"2023-07-14T11:19:47.967884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:07.208804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717860+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:14.717860+0000","last_peered":"2023-07-14T11:20:14.717860+0000","last_clean":"2023-07-14T11:20:14.717860+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:14.717860+0000","last_undegraded":"2023-07-14T11:20:14.717860+0000","last_fullsized":"2023-07-14T11:20:14.717860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:15:42.131659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080903+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:19:47.080903+0000","last_peered":"2023-07-14T11:19:47.080903+0000","last_clean":"2023-07-14T11:19:47.080903+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:19:47.080903+0000","last_undegraded":"2023-07-14T11:19:47.080903+0000","last_fullsized":"2023-07-14T11:19:47.080903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:29.469059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T11:19:47.967804+0000","last_change":"2023-07-14T11:20:14.717622+0000","last_active":"2023-07-14T11:20:14.717622+0000","last_peered":"2023-07-14T11:19:47.967804+0000","last_clean":"2023-07-14T11:19:47.967804+0000","last_became_active":"2023-07-14T11:19:46.273092+0000","last_became_peered":"2023-07-14T11:19:46.273092+0000","last_unstale":"2023-07-14T11:19:47.967804+0000","last_undegraded":"2023-07-14T11:19:47.967804+0000","last_fullsized":"2023-07-14T11:19:47.967804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081920+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:19:47.081920+0000","last_peered":"2023-07-14T11:19:47.081920+0000","last_clean":"2023-07-14T11:19:47.081920+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:19:47.081920+0000","last_undegraded":"2023-07-14T11:19:47.081920+0000","last_fullsized":"2023-07-14T11:19:47.081920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:21:18.903408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081118+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:19:47.081118+0000","last_peered":"2023-07-14T11:19:47.081118+0000","last_clean":"2023-07-14T11:19:47.081118+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:19:47.081118+0000","last_undegraded":"2023-07-14T11:19:47.081118+0000","last_fullsized":"2023-07-14T11:19:47.081118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:37.312351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730373+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:14.730373+0000","last_peered":"2023-07-14T11:20:14.730373+0000","last_clean":"2023-07-14T11:20:14.730373+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:14.730373+0000","last_undegraded":"2023-07-14T11:20:14.730373+0000","last_fullsized":"2023-07-14T11:20:14.730373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:10.476499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968436+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:19:47.968436+0000","last_peered":"2023-07-14T11:19:47.968436+0000","last_clean":"2023-07-14T11:19:47.968436+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:19:47.968436+0000","last_undegraded":"2023-07-14T11:19:47.968436+0000","last_fullsized":"2023-07-14T11:19:47.968436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081008+0000","last_change":"2023-07-14T11:19:42.093066+0000","last_active":"2023-07-14T11:19:47.081008+0000","last_peered":"2023-07-14T11:19:47.081008+0000","last_clean":"2023-07-14T11:19:47.081008+0000","last_became_active":"2023-07-14T11:19:42.092719+0000","last_became_peered":"2023-07-14T11:19:42.092719+0000","last_unstale":"2023-07-14T11:19:47.081008+0000","last_undegraded":"2023-07-14T11:19:47.081008+0000","last_fullsized":"2023-07-14T11:19:47.081008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:35:05.018230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.729003+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:14.729003+0000","last_peered":"2023-07-14T11:20:14.729003+0000","last_clean":"2023-07-14T11:20:14.729003+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:14.729003+0000","last_undegraded":"2023-07-14T11:20:14.729003+0000","last_fullsized":"2023-07-14T11:20:14.729003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:29.125385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968186+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:19:47.968186+0000","last_peered":"2023-07-14T11:19:47.968186+0000","last_clean":"2023-07-14T11:19:47.968186+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:19:47.968186+0000","last_undegraded":"2023-07-14T11:19:47.968186+0000","last_fullsized":"2023-07-14T11:19:47.968186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:06.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731196+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:14.731196+0000","last_peered":"2023-07-14T11:20:14.731196+0000","last_clean":"2023-07-14T11:20:14.731196+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:14.731196+0000","last_undegraded":"2023-07-14T11:20:14.731196+0000","last_fullsized":"2023-07-14T11:20:14.731196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:16:34.542159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:49.452867+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:19:49.452867+0000","last_peered":"2023-07-14T11:19:49.452867+0000","last_clean":"2023-07-14T11:19:49.452867+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:19:49.452867+0000","last_undegraded":"2023-07-14T11:19:49.452867+0000","last_fullsized":"2023-07-14T11:19:49.452867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:22.680690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967124+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:19:47.967124+0000","last_peered":"2023-07-14T11:19:47.967124+0000","last_clean":"2023-07-14T11:19:47.967124+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:19:47.967124+0000","last_undegraded":"2023-07-14T11:19:47.967124+0000","last_fullsized":"2023-07-14T11:19:47.967124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:14:39.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.676206+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:12.676206+0000","last_peered":"2023-07-14T11:20:12.676206+0000","last_clean":"2023-07-14T11:20:12.676206+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:12.676206+0000","last_undegraded":"2023-07-14T11:20:12.676206+0000","last_fullsized":"2023-07-14T11:20:12.676206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:22:45.207699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968042+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:19:47.968042+0000","last_peered":"2023-07-14T11:19:47.968042+0000","last_clean":"2023-07-14T11:19:47.968042+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:19:47.968042+0000","last_undegraded":"2023-07-14T11:19:47.968042+0000","last_fullsized":"2023-07-14T11:19:47.968042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:23.767120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968619+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:19:47.968619+0000","last_peered":"2023-07-14T11:19:47.968619+0000","last_clean":"2023-07-14T11:19:47.968619+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:19:47.968619+0000","last_undegraded":"2023-07-14T11:19:47.968619+0000","last_fullsized":"2023-07-14T11:19:47.968619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:13.375758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731070+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:14.731070+0000","last_peered":"2023-07-14T11:20:14.731070+0000","last_clean":"2023-07-14T11:20:14.731070+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:14.731070+0000","last_undegraded":"2023-07-14T11:20:14.731070+0000","last_fullsized":"2023-07-14T11:20:14.731070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:59:07.405076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.717379+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:11.717379+0000","last_peered":"2023-07-14T11:20:11.717379+0000","last_clean":"2023-07-14T11:20:11.717379+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:11.717379+0000","last_undegraded":"2023-07-14T11:20:11.717379+0000","last_fullsized":"2023-07-14T11:20:11.717379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:24.177685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080724+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:19:47.080724+0000","last_peered":"2023-07-14T11:19:47.080724+0000","last_clean":"2023-07-14T11:19:47.080724+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:19:47.080724+0000","last_undegraded":"2023-07-14T11:19:47.080724+0000","last_fullsized":"2023-07-14T11:19:47.080724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:52:43.413334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718394+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:14.718394+0000","last_peered":"2023-07-14T11:20:14.718394+0000","last_clean":"2023-07-14T11:20:14.718394+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:14.718394+0000","last_undegraded":"2023-07-14T11:20:14.718394+0000","last_fullsized":"2023-07-14T11:20:14.718394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:22.172922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730551+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:14.730551+0000","last_peered":"2023-07-14T11:20:14.730551+0000","last_clean":"2023-07-14T11:20:14.730551+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:14.730551+0000","last_undegraded":"2023-07-14T11:20:14.730551+0000","last_fullsized":"2023-07-14T11:20:14.730551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:37.007482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967049+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:19:47.967049+0000","last_peered":"2023-07-14T11:19:47.967049+0000","last_clean":"2023-07-14T11:19:47.967049+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:19:47.967049+0000","last_undegraded":"2023-07-14T11:19:47.967049+0000","last_fullsized":"2023-07-14T11:19:47.967049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:39:36.064722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718705+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:11.718705+0000","last_peered":"2023-07-14T11:20:11.718705+0000","last_clean":"2023-07-14T11:20:11.718705+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:11.718705+0000","last_undegraded":"2023-07-14T11:20:11.718705+0000","last_fullsized":"2023-07-14T11:20:11.718705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:26:49.607683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967586+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:19:47.967586+0000","last_peered":"2023-07-14T11:19:47.967586+0000","last_clean":"2023-07-14T11:19:47.967586+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:19:47.967586+0000","last_undegraded":"2023-07-14T11:19:47.967586+0000","last_fullsized":"2023-07-14T11:19:47.967586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:35:51.028974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.196923+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:19:47.196923+0000","last_peered":"2023-07-14T11:19:47.196923+0000","last_clean":"2023-07-14T11:19:47.196923+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:19:47.196923+0000","last_undegraded":"2023-07-14T11:19:47.196923+0000","last_fullsized":"2023-07-14T11:19:47.196923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:20.885447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711775+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:10.711775+0000","last_peered":"2023-07-14T11:20:10.711775+0000","last_clean":"2023-07-14T11:20:10.711775+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:10.711775+0000","last_undegraded":"2023-07-14T11:20:10.711775+0000","last_fullsized":"2023-07-14T11:20:10.711775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:36.953428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737388+0000","last_change":"2023-07-14T11:19:42.085058+0000","last_active":"2023-07-14T11:19:47.737388+0000","last_peered":"2023-07-14T11:19:47.737388+0000","last_clean":"2023-07-14T11:19:47.737388+0000","last_became_active":"2023-07-14T11:19:42.083380+0000","last_became_peered":"2023-07-14T11:19:42.083380+0000","last_unstale":"2023-07-14T11:19:47.737388+0000","last_undegraded":"2023-07-14T11:19:47.737388+0000","last_fullsized":"2023-07-14T11:19:47.737388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:52.430115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967817+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:19:47.967817+0000","last_peered":"2023-07-14T11:19:47.967817+0000","last_clean":"2023-07-14T11:19:47.967817+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:19:47.967817+0000","last_undegraded":"2023-07-14T11:19:47.967817+0000","last_fullsized":"2023-07-14T11:19:47.967817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:25:48.734999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717656+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:14.717656+0000","last_peered":"2023-07-14T11:20:14.717656+0000","last_clean":"2023-07-14T11:20:14.717656+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:14.717656+0000","last_undegraded":"2023-07-14T11:20:14.717656+0000","last_fullsized":"2023-07-14T11:20:14.717656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:28.963610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082443+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:19:47.082443+0000","last_peered":"2023-07-14T11:19:47.082443+0000","last_clean":"2023-07-14T11:19:47.082443+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:19:47.082443+0000","last_undegraded":"2023-07-14T11:19:47.082443+0000","last_fullsized":"2023-07-14T11:19:47.082443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:43.469946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713281+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:10.713281+0000","last_peered":"2023-07-14T11:20:10.713281+0000","last_clean":"2023-07-14T11:20:10.713281+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:10.713281+0000","last_undegraded":"2023-07-14T11:20:10.713281+0000","last_fullsized":"2023-07-14T11:20:10.713281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:45:36.543244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080522+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:19:47.080522+0000","last_peered":"2023-07-14T11:19:47.080522+0000","last_clean":"2023-07-14T11:19:47.080522+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:19:47.080522+0000","last_undegraded":"2023-07-14T11:19:47.080522+0000","last_fullsized":"2023-07-14T11:19:47.080522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:44.548171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967752+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:19:47.967752+0000","last_peered":"2023-07-14T11:19:47.967752+0000","last_clean":"2023-07-14T11:19:47.967752+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:19:47.967752+0000","last_undegraded":"2023-07-14T11:19:47.967752+0000","last_fullsized":"2023-07-14T11:19:47.967752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:50:53.130285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197038+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:19:47.197038+0000","last_peered":"2023-07-14T11:19:47.197038+0000","last_clean":"2023-07-14T11:19:47.197038+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:19:47.197038+0000","last_undegraded":"2023-07-14T11:19:47.197038+0000","last_fullsized":"2023-07-14T11:19:47.197038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:15.617687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967360+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:19:47.967360+0000","last_peered":"2023-07-14T11:19:47.967360+0000","last_clean":"2023-07-14T11:19:47.967360+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:19:47.967360+0000","last_undegraded":"2023-07-14T11:19:47.967360+0000","last_fullsized":"2023-07-14T11:19:47.967360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:21.159717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967674+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:19:47.967674+0000","last_peered":"2023-07-14T11:19:47.967674+0000","last_clean":"2023-07-14T11:19:47.967674+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:19:47.967674+0000","last_undegraded":"2023-07-14T11:19:47.967674+0000","last_fullsized":"2023-07-14T11:19:47.967674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:57:50.109303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717280+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:14.717280+0000","last_peered":"2023-07-14T11:20:14.717280+0000","last_clean":"2023-07-14T11:20:14.717280+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:14.717280+0000","last_undegraded":"2023-07-14T11:20:14.717280+0000","last_fullsized":"2023-07-14T11:20:14.717280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:38:55.878246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968641+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:19:47.968641+0000","last_peered":"2023-07-14T11:19:47.968641+0000","last_clean":"2023-07-14T11:19:47.968641+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:19:47.968641+0000","last_undegraded":"2023-07-14T11:19:47.968641+0000","last_fullsized":"2023-07-14T11:19:47.968641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:57.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197577+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:19:47.197577+0000","last_peered":"2023-07-14T11:19:47.197577+0000","last_clean":"2023-07-14T11:19:47.197577+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:19:47.197577+0000","last_undegraded":"2023-07-14T11:19:47.197577+0000","last_fullsized":"2023-07-14T11:19:47.197577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:48.609439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969031+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:19:47.969031+0000","last_peered":"2023-07-14T11:19:47.969031+0000","last_clean":"2023-07-14T11:19:47.969031+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:19:47.969031+0000","last_undegraded":"2023-07-14T11:19:47.969031+0000","last_fullsized":"2023-07-14T11:19:47.969031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:55.843732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736878+0000","last_change":"2023-07-14T11:19:46.088344+0000","last_active":"2023-07-14T11:19:47.736878+0000","last_peered":"2023-07-14T11:19:47.736878+0000","last_clean":"2023-07-14T11:19:47.736878+0000","last_became_active":"2023-07-14T11:19:46.087922+0000","last_became_peered":"2023-07-14T11:19:46.087922+0000","last_unstale":"2023-07-14T11:19:47.736878+0000","last_undegraded":"2023-07-14T11:19:47.736878+0000","last_fullsized":"2023-07-14T11:19:47.736878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:02:04.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968974+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:19:47.968974+0000","last_peered":"2023-07-14T11:19:47.968974+0000","last_clean":"2023-07-14T11:19:47.968974+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:19:47.968974+0000","last_undegraded":"2023-07-14T11:19:47.968974+0000","last_fullsized":"2023-07-14T11:19:47.968974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:56:35.324459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968655+0000","last_change":"2023-07-14T11:19:42.073286+0000","last_active":"2023-07-14T11:19:47.968655+0000","last_peered":"2023-07-14T11:19:47.968655+0000","last_clean":"2023-07-14T11:19:47.968655+0000","last_became_active":"2023-07-14T11:19:42.072976+0000","last_became_peered":"2023-07-14T11:19:42.072976+0000","last_unstale":"2023-07-14T11:19:47.968655+0000","last_undegraded":"2023-07-14T11:19:47.968655+0000","last_fullsized":"2023-07-14T11:19:47.968655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:37:29.066629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967707+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:19:47.967707+0000","last_peered":"2023-07-14T11:19:47.967707+0000","last_clean":"2023-07-14T11:19:47.967707+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:19:47.967707+0000","last_undegraded":"2023-07-14T11:19:47.967707+0000","last_fullsized":"2023-07-14T11:19:47.967707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:15.123989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728744+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:14.728744+0000","last_peered":"2023-07-14T11:20:14.728744+0000","last_clean":"2023-07-14T11:20:14.728744+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:14.728744+0000","last_undegraded":"2023-07-14T11:20:14.728744+0000","last_fullsized":"2023-07-14T11:20:14.728744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:03:42.954256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968714+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:19:47.968714+0000","last_peered":"2023-07-14T11:19:47.968714+0000","last_clean":"2023-07-14T11:19:47.968714+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:19:47.968714+0000","last_undegraded":"2023-07-14T11:19:47.968714+0000","last_fullsized":"2023-07-14T11:19:47.968714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:03.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080574+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:19:47.080574+0000","last_peered":"2023-07-14T11:19:47.080574+0000","last_clean":"2023-07-14T11:19:47.080574+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:19:47.080574+0000","last_undegraded":"2023-07-14T11:19:47.080574+0000","last_fullsized":"2023-07-14T11:19:47.080574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:06.602465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197215+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:19:47.197215+0000","last_peered":"2023-07-14T11:19:47.197215+0000","last_clean":"2023-07-14T11:19:47.197215+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:19:47.197215+0000","last_undegraded":"2023-07-14T11:19:47.197215+0000","last_fullsized":"2023-07-14T11:19:47.197215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:52:37.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197079+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:19:47.197079+0000","last_peered":"2023-07-14T11:19:47.197079+0000","last_clean":"2023-07-14T11:19:47.197079+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:19:47.197079+0000","last_undegraded":"2023-07-14T11:19:47.197079+0000","last_fullsized":"2023-07-14T11:19:47.197079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:26:40.329142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736992+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:19:47.736992+0000","last_peered":"2023-07-14T11:19:47.736992+0000","last_clean":"2023-07-14T11:19:47.736992+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:19:47.736992+0000","last_undegraded":"2023-07-14T11:19:47.736992+0000","last_fullsized":"2023-07-14T11:19:47.736992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:09:09.081413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730659+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:14.730659+0000","last_peered":"2023-07-14T11:20:14.730659+0000","last_clean":"2023-07-14T11:20:14.730659+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:14.730659+0000","last_undegraded":"2023-07-14T11:20:14.730659+0000","last_fullsized":"2023-07-14T11:20:14.730659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:18.996495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081049+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:19:47.081049+0000","last_peered":"2023-07-14T11:19:47.081049+0000","last_clean":"2023-07-14T11:19:47.081049+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:19:47.081049+0000","last_undegraded":"2023-07-14T11:19:47.081049+0000","last_fullsized":"2023-07-14T11:19:47.081049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:56.251387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197612+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:19:47.197612+0000","last_peered":"2023-07-14T11:19:47.197612+0000","last_clean":"2023-07-14T11:19:47.197612+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:19:47.197612+0000","last_undegraded":"2023-07-14T11:19:47.197612+0000","last_fullsized":"2023-07-14T11:19:47.197612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:32:51.167057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967746+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:19:47.967746+0000","last_peered":"2023-07-14T11:19:47.967746+0000","last_clean":"2023-07-14T11:19:47.967746+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:19:47.967746+0000","last_undegraded":"2023-07-14T11:19:47.967746+0000","last_fullsized":"2023-07-14T11:19:47.967746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:37.267773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-14T11:19:42.068794+0000","last_change":"2023-07-14T11:20:15.683458+0000","last_active":"2023-07-14T11:19:41.086153+0000","last_peered":"2023-07-14T11:19:41.086153+0000","last_clean":"2023-07-14T11:19:41.086153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:19:42.068794+0000","last_undegraded":"2023-07-14T11:19:42.068794+0000","last_fullsized":"2023-07-14T11:19:42.068794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967293+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:19:47.967293+0000","last_peered":"2023-07-14T11:19:47.967293+0000","last_clean":"2023-07-14T11:19:47.967293+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:19:47.967293+0000","last_undegraded":"2023-07-14T11:19:47.967293+0000","last_fullsized":"2023-07-14T11:19:47.967293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:58:16.557411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727007+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:14.727007+0000","last_peered":"2023-07-14T11:20:14.727007+0000","last_clean":"2023-07-14T11:20:14.727007+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:14.727007+0000","last_undegraded":"2023-07-14T11:20:14.727007+0000","last_fullsized":"2023-07-14T11:20:14.727007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:33.588223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080793+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:19:47.080793+0000","last_peered":"2023-07-14T11:19:47.080793+0000","last_clean":"2023-07-14T11:19:47.080793+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:19:47.080793+0000","last_undegraded":"2023-07-14T11:19:47.080793+0000","last_fullsized":"2023-07-14T11:19:47.080793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:57.953734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":121,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":60474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":471801,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.37},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.755},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.78000000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1062815,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.876},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82999999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":467907,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.628},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.1639999999999997,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70299999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.59799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":481467,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93000000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0129999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1062952,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":466940,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.439},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.85099999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.876},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":515148,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.84799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.623},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.752}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":468328,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":1.042},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.55800000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.621},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.64800000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":1.147}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.99099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":471252,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56299999999999994}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467317,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.376}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.5},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.622}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73899999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467719,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54000000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1058588,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.624}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50800000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:16.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-07-14T11:20:16.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-07-14T11:19:47.967804+0000', 'last_change': '2023-07-14T11:20:14.717622+0000', 'last_active': '2023-07-14T11:20:14.717622+0000', 'last_peered': '2023-07-14T11:19:47.967804+0000', 'last_clean': '2023-07-14T11:19:47.967804+0000', 'last_became_active': '2023-07-14T11:19:46.273092+0000', 'last_became_peered': '2023-07-14T11:19:46.273092+0000', 'last_unstale': '2023-07-14T11:19:47.967804+0000', 'last_undegraded': '2023-07-14T11:19:47.967804+0000', 'last_fullsized': '2023-07-14T11:19:47.967804+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 86, 'last_epoch_clean': 87, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:19:45.066838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:19:45.066838+0000', 'last_clean_scrub_stamp': '2023-07-14T11:19:45.066838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 10, 11, 1], 'acting': [8, 10, 11, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 9, 10, 11], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-07-14T11:20:16.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-07-14T11:20:17.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 89, 'state': 'peering', 'last_fresh': '2023-07-14T11:19:42.068794+0000', 'last_change': '2023-07-14T11:20:15.683458+0000', 'last_active': '2023-07-14T11:19:41.086153+0000', 'last_peered': '2023-07-14T11:19:41.086153+0000', 'last_clean': '2023-07-14T11:19:41.086153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:19:42.068794+0000', 'last_undegraded': '2023-07-14T11:19:42.068794+0000', 'last_fullsized': '2023-07-14T11:19:42.068794+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 83, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:19:41.086153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:19:41.086153+0000', 'last_clean_scrub_stamp': '2023-07-14T11:19:41.086153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 9, 3], 'acting': [8, 9, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-07-14T11:20:17.000 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-14T11:20:17.169 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:20:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: 2023-07-14T11:20:16.680+0000 7f252077c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-07-14T11:20:17.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:16 smithi175 ceph-mon[129081]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T11:20:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:16 smithi175 ceph-mon[129081]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:16 smithi175 ceph-mon[129081]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2023-07-14T11:20:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:16 smithi175 ceph-mon[129081]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:16 smithi182 ceph-mon[129366]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T11:20:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:16 smithi182 ceph-mon[129366]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:16 smithi182 ceph-mon[129366]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2023-07-14T11:20:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:16 smithi182 ceph-mon[129366]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:17.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2113618889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b94100cf0 msgr2=0x7f8b94101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2113618889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b94100cf0 0x7f8b94101140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009990 tx=0x7f8b7c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2113618889 shutdown_connections 2023-07-14T11:20:17.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2113618889 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b94101680 0x7f8b94107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2113618889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b940ffb00 0x7f8b940fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2113618889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b94100cf0 0x7f8b94101140 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2113618889 >> 172.21.15.161:0/2113618889 conn(0x7f8b940fb0e0 msgr2=0x7f8b940fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:17.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2113618889 shutdown_connections 2023-07-14T11:20:17.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.222+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2113618889 wait complete. 2023-07-14T11:20:17.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.223+0000 7f8b9a9ff700 1 Processor -- start 2023-07-14T11:20:17.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.223+0000 7f8b9a9ff700 1 -- start start 2023-07-14T11:20:17.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.223+0000 7f8b9a9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b940ffb00 0x7f8b94197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.223+0000 7f8b9a9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b94100cf0 0x7f8b94197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.223+0000 7f8b9a9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b94101680 0x7f8b9419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b9a9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419f590 con 0x7f8b940ffb00 2023-07-14T11:20:17.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b9a9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419f6d0 con 0x7f8b94100cf0 2023-07-14T11:20:17.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b9a9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419f860 con 0x7f8b94101680 2023-07-14T11:20:17.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b940ffb00 0x7f8b94197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b940ffb00 0x7f8b94197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45212/0 (socket says 172.21.15.161:45212) 2023-07-14T11:20:17.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 -- 172.21.15.161:0/2285566743 learned_addr learned my addr 172.21.15.161:0/2285566743 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:17.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b8bfff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b94100cf0 0x7f8b94197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b98f9c700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b94101680 0x7f8b9419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 -- 172.21.15.161:0/2285566743 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b94101680 msgr2=0x7f8b9419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b94101680 0x7f8b9419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 -- 172.21.15.161:0/2285566743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b94100cf0 msgr2=0x7f8b94197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b94100cf0 0x7f8b94197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 -- 172.21.15.161:0/2285566743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b84009520 con 0x7f8b940ffb00 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.224+0000 7f8b93fff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b940ffb00 0x7f8b94197a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400ea50 tx=0x7f8b8400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.225+0000 7f8b91ffb700 1 -- 172.21.15.161:0/2285566743 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400cbd0 con 0x7f8b940ffb00 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.225+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c0095f0 con 0x7f8b940ffb00 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.225+0000 7f8b91ffb700 1 -- 172.21.15.161:0/2285566743 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8400cd30 con 0x7f8b940ffb00 2023-07-14T11:20:17.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.225+0000 7f8b91ffb700 1 -- 172.21.15.161:0/2285566743 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84018680 con 0x7f8b940ffb00 2023-07-14T11:20:17.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.225+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9419ff10 con 0x7f8b940ffb00 2023-07-14T11:20:17.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.226+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9410b320 con 0x7f8b940ffb00 2023-07-14T11:20:17.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.228+0000 7f8b91ffb700 1 -- 172.21.15.161:0/2285566743 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8b84010430 con 0x7f8b940ffb00 2023-07-14T11:20:17.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.229+0000 7f8b91ffb700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b800a1220 0x7f8b800a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.229+0000 7f8b91ffb700 1 -- 172.21.15.161:0/2285566743 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8b84014070 con 0x7f8b940ffb00 2023-07-14T11:20:17.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.230+0000 7f8b8bfff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b800a1220 0x7f8b800a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.230+0000 7f8b8bfff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b800a1220 0x7f8b800a36d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00b2e0 tx=0x7f8b7c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:17.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.233+0000 7f8b91ffb700 1 -- 172.21.15.161:0/2285566743 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8b84096550 con 0x7f8b940ffb00 2023-07-14T11:20:17.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.512+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8b9410b4e0 con 0x7f8b940ffb00 2023-07-14T11:20:17.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.513+0000 7f8b91ffb700 1 -- 172.21.15.161:0/2285566743 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2766 (secure 0 0 0) 0x7f8b84099b70 con 0x7f8b940ffb00 2023-07-14T11:20:17.516 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:17.517 INFO:teuthology.orchestra.run.smithi161.stdout:{"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":366,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":90,"num_osds":12,"num_up_osds":12,"osd_up_since":1689333533,"num_in_osds":12,"osd_in_since":1689333516,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":2}],"num_pgs":129,"num_pools":4,"num_objects":59,"data_bytes":597217,"bytes_used":342966272,"bytes_avail":1151597461504,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"read_bytes_sec":2303,"write_bytes_sec":3071,"read_op_per_sec":2,"write_op_per_sec":9},"fsmap":{"epoch":22,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24503},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24514},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24526},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24533},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24544},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24575}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.161:8443/"}},"servicemap":{"epoch":21,"modified":"2023-07-14T11:20:14.103464+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-07-14T11:20:17.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b800a1220 msgr2=0x7f8b800a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b800a1220 0x7f8b800a36d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00b2e0 tx=0x7f8b7c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b940ffb00 msgr2=0x7f8b94197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b940ffb00 0x7f8b94197a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400ea50 tx=0x7f8b8400ed60 comp rx=0 tx=0).stop 2023-07-14T11:20:17.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 shutdown_connections 2023-07-14T11:20:17.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b940ffb00 0x7f8b94197a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b800a1220 0x7f8b800a36d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b94101680 0x7f8b9419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 --2- 172.21.15.161:0/2285566743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b94100cf0 0x7f8b94197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 >> 172.21.15.161:0/2285566743 conn(0x7f8b940fb0e0 msgr2=0x7f8b940fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:17.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 shutdown_connections 2023-07-14T11:20:17.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.515+0000 7f8b9a9ff700 1 -- 172.21.15.161:0/2285566743 wait complete. 2023-07-14T11:20:17.537 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-14T11:20:17.538 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:17.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 -- 172.21.15.161:0/710591402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc100cf0 msgr2=0x7fa9dc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/710591402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc100cf0 0x7fa9dc101140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009990 tx=0x7fa9c4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 -- 172.21.15.161:0/710591402 shutdown_connections 2023-07-14T11:20:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/710591402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa9dc101680 0x7fa9dc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/710591402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 0x7fa9dc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/710591402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc100cf0 0x7fa9dc101140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 -- 172.21.15.161:0/710591402 >> 172.21.15.161:0/710591402 conn(0x7fa9dc0fb0e0 msgr2=0x7fa9dc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 -- 172.21.15.161:0/710591402 shutdown_connections 2023-07-14T11:20:17.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.743+0000 7fa9e09ff700 1 -- 172.21.15.161:0/710591402 wait complete. 2023-07-14T11:20:17.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.744+0000 7fa9e09ff700 1 Processor -- start 2023-07-14T11:20:17.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.744+0000 7fa9e09ff700 1 -- start start 2023-07-14T11:20:17.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.744+0000 7fa9e09ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 0x7fa9dc197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.744+0000 7fa9e09ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa9dc100cf0 0x7fa9dc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9e09ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc101680 0x7fa9dc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9e09ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19f590 con 0x7fa9dc101680 2023-07-14T11:20:17.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9e09ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19f6d0 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9e09ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19f860 con 0x7fa9dc100cf0 2023-07-14T11:20:17.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 0x7fa9dc197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 0x7fa9dc197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35116/0 (socket says 172.21.15.161:35116) 2023-07-14T11:20:17.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 -- 172.21.15.161:0/3080267206 learned_addr learned my addr 172.21.15.161:0/3080267206 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:17.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9dad9d700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc101680 0x7fa9dc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9d3fff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa9dc100cf0 0x7fa9dc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 -- 172.21.15.161:0/3080267206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa9dc100cf0 msgr2=0x7fa9dc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa9dc100cf0 0x7fa9dc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 -- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc101680 msgr2=0x7fa9dc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc101680 0x7fa9dc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 -- 172.21.15.161:0/3080267206 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c40095f0 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9dad9d700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc101680 0x7fa9dc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.745+0000 7fa9da59c700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 0x7fa9dc197a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e970 tx=0x7fa9cc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.746+0000 7fa9d37fe700 1 -- 172.21.15.161:0/3080267206 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc009790 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.746+0000 7fa9d37fe700 1 -- 172.21.15.161:0/3080267206 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9cc00c7e0 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.746+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc19fb40 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.746+0000 7fa9d37fe700 1 -- 172.21.15.161:0/3080267206 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc003c70 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.746+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc1a00a0 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.747+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc10b320 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.749+0000 7fa9d37fe700 1 -- 172.21.15.161:0/3080267206 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa9cc00cc50 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.750+0000 7fa9d37fe700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa9c80a12a0 0x7fa9c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:17.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.750+0000 7fa9d37fe700 1 -- 172.21.15.161:0/3080267206 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa9cc014070 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.751+0000 7fa9d3fff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa9c80a12a0 0x7fa9c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:17.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.751+0000 7fa9d3fff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa9c80a12a0 0x7fa9c80a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400b2e0 tx=0x7fa9c4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:17.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.754+0000 7fa9d37fe700 1 -- 172.21.15.161:0/3080267206 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa9cc0961d0 con 0x7fa9dc0ffb00 2023-07-14T11:20:17.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.946+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9dc1075c0 con 0x7fa9c80a12a0 2023-07-14T11:20:17.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.959+0000 7fa9d37fe700 1 -- 172.21.15.161:0/3080267206 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422840 (secure 0 0 0) 0x7fa9dc1075c0 con 0x7fa9c80a12a0 2023-07-14T11:20:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:17.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa9c80a12a0 msgr2=0x7fa9c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa9c80a12a0 0x7fa9c80a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400b2e0 tx=0x7fa9c4005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 msgr2=0x7fa9dc197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:17.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 0x7fa9dc197a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e970 tx=0x7fa9cc00ec80 comp rx=0 tx=0).stop 2023-07-14T11:20:17.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 shutdown_connections 2023-07-14T11:20:17.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9dc101680 0x7fa9dc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa9c80a12a0 0x7fa9c80a3750 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0ffb00 0x7fa9dc197a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 --2- 172.21.15.161:0/3080267206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa9dc100cf0 0x7fa9dc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:17.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 >> 172.21.15.161:0/3080267206 conn(0x7fa9dc0fb0e0 msgr2=0x7fa9dc0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:17.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 shutdown_connections 2023-07-14T11:20:17.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:17.964+0000 7fa9e09ff700 1 -- 172.21.15.161:0/3080267206 wait complete. 2023-07-14T11:20:17.971 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:17.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:17 smithi161 ceph-mon[95526]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:17.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:17 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2285566743' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-07-14T11:20:17.986 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-07-14T11:20:16.103870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597217,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":156,"num_read_kb":137,"num_write":303,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334928,"kb_used_data":11504,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606896,"statfs":{"total":1151940427776,"available":1151597461504,"internally_reserved":0,"allocated":11780096,"data_stored":7462234,"data_compressed":253459,"data_compressed_allocated":339968,"data_compressed_original":2179072,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2257,"num_objects":35,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002134"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080953+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:19:47.080953+0000","last_peered":"2023-07-14T11:19:47.080953+0000","last_clean":"2023-07-14T11:19:47.080953+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:19:47.080953+0000","last_undegraded":"2023-07-14T11:19:47.080953+0000","last_fullsized":"2023-07-14T11:19:47.080953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:57.564463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737248+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:19:47.737248+0000","last_peered":"2023-07-14T11:19:47.737248+0000","last_clean":"2023-07-14T11:19:47.737248+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:19:47.737248+0000","last_undegraded":"2023-07-14T11:19:47.737248+0000","last_fullsized":"2023-07-14T11:19:47.737248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:19.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737244+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:19:47.737244+0000","last_peered":"2023-07-14T11:19:47.737244+0000","last_clean":"2023-07-14T11:19:47.737244+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:19:47.737244+0000","last_undegraded":"2023-07-14T11:19:47.737244+0000","last_fullsized":"2023-07-14T11:19:47.737244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:47.796849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968258+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:19:47.968258+0000","last_peered":"2023-07-14T11:19:47.968258+0000","last_clean":"2023-07-14T11:19:47.968258+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:19:47.968258+0000","last_undegraded":"2023-07-14T11:19:47.968258+0000","last_fullsized":"2023-07-14T11:19:47.968258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:18:31.399322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717211+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:14.717211+0000","last_peered":"2023-07-14T11:20:14.717211+0000","last_clean":"2023-07-14T11:20:14.717211+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:14.717211+0000","last_undegraded":"2023-07-14T11:20:14.717211+0000","last_fullsized":"2023-07-14T11:20:14.717211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:45:02.418574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081056+0000","last_change":"2023-07-14T11:19:42.092136+0000","last_active":"2023-07-14T11:19:47.081056+0000","last_peered":"2023-07-14T11:19:47.081056+0000","last_clean":"2023-07-14T11:19:47.081056+0000","last_became_active":"2023-07-14T11:19:42.091820+0000","last_became_peered":"2023-07-14T11:19:42.091820+0000","last_unstale":"2023-07-14T11:19:47.081056+0000","last_undegraded":"2023-07-14T11:19:47.081056+0000","last_fullsized":"2023-07-14T11:19:47.081056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:01.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968087+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:19:47.968087+0000","last_peered":"2023-07-14T11:19:47.968087+0000","last_clean":"2023-07-14T11:19:47.968087+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:19:47.968087+0000","last_undegraded":"2023-07-14T11:19:47.968087+0000","last_fullsized":"2023-07-14T11:19:47.968087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:21.428012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967192+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:19:47.967192+0000","last_peered":"2023-07-14T11:19:47.967192+0000","last_clean":"2023-07-14T11:19:47.967192+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:19:47.967192+0000","last_undegraded":"2023-07-14T11:19:47.967192+0000","last_fullsized":"2023-07-14T11:19:47.967192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:38.965332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081420+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:19:47.081420+0000","last_peered":"2023-07-14T11:19:47.081420+0000","last_clean":"2023-07-14T11:19:47.081420+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:19:47.081420+0000","last_undegraded":"2023-07-14T11:19:47.081420+0000","last_fullsized":"2023-07-14T11:19:47.081420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:51.712868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969101+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:19:47.969101+0000","last_peered":"2023-07-14T11:19:47.969101+0000","last_clean":"2023-07-14T11:19:47.969101+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:19:47.969101+0000","last_undegraded":"2023-07-14T11:19:47.969101+0000","last_fullsized":"2023-07-14T11:19:47.969101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:48:50.962800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968204+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:19:47.968204+0000","last_peered":"2023-07-14T11:19:47.968204+0000","last_clean":"2023-07-14T11:19:47.968204+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:19:47.968204+0000","last_undegraded":"2023-07-14T11:19:47.968204+0000","last_fullsized":"2023-07-14T11:19:47.968204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:47.264465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727198+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:14.727198+0000","last_peered":"2023-07-14T11:20:14.727198+0000","last_clean":"2023-07-14T11:20:14.727198+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:14.727198+0000","last_undegraded":"2023-07-14T11:20:14.727198+0000","last_fullsized":"2023-07-14T11:20:14.727198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:42.155482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736832+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:19:47.736832+0000","last_peered":"2023-07-14T11:19:47.736832+0000","last_clean":"2023-07-14T11:19:47.736832+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:19:47.736832+0000","last_undegraded":"2023-07-14T11:19:47.736832+0000","last_fullsized":"2023-07-14T11:19:47.736832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:26.314511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736956+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:19:47.736956+0000","last_peered":"2023-07-14T11:19:47.736956+0000","last_clean":"2023-07-14T11:19:47.736956+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:19:47.736956+0000","last_undegraded":"2023-07-14T11:19:47.736956+0000","last_fullsized":"2023-07-14T11:19:47.736956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:10:23.738358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081137+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:19:47.081137+0000","last_peered":"2023-07-14T11:19:47.081137+0000","last_clean":"2023-07-14T11:19:47.081137+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:19:47.081137+0000","last_undegraded":"2023-07-14T11:19:47.081137+0000","last_fullsized":"2023-07-14T11:19:47.081137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:51.121664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082305+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:19:47.082305+0000","last_peered":"2023-07-14T11:19:47.082305+0000","last_clean":"2023-07-14T11:19:47.082305+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:19:47.082305+0000","last_undegraded":"2023-07-14T11:19:47.082305+0000","last_fullsized":"2023-07-14T11:19:47.082305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:07:45.552513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197608+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:19:47.197608+0000","last_peered":"2023-07-14T11:19:47.197608+0000","last_clean":"2023-07-14T11:19:47.197608+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:19:47.197608+0000","last_undegraded":"2023-07-14T11:19:47.197608+0000","last_fullsized":"2023-07-14T11:19:47.197608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:44.466407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197036+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:19:47.197036+0000","last_peered":"2023-07-14T11:19:47.197036+0000","last_clean":"2023-07-14T11:19:47.197036+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:19:47.197036+0000","last_undegraded":"2023-07-14T11:19:47.197036+0000","last_fullsized":"2023-07-14T11:19:47.197036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:54.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.719668+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:11.719668+0000","last_peered":"2023-07-14T11:20:11.719668+0000","last_clean":"2023-07-14T11:20:11.719668+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:11.719668+0000","last_undegraded":"2023-07-14T11:20:11.719668+0000","last_fullsized":"2023-07-14T11:20:11.719668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:13:47.991877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967522+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:19:47.967522+0000","last_peered":"2023-07-14T11:19:47.967522+0000","last_clean":"2023-07-14T11:19:47.967522+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:19:47.967522+0000","last_undegraded":"2023-07-14T11:19:47.967522+0000","last_fullsized":"2023-07-14T11:19:47.967522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:21.029051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967474+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:19:47.967474+0000","last_peered":"2023-07-14T11:19:47.967474+0000","last_clean":"2023-07-14T11:19:47.967474+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:19:47.967474+0000","last_undegraded":"2023-07-14T11:19:47.967474+0000","last_fullsized":"2023-07-14T11:19:47.967474+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:10.344354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967640+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:19:47.967640+0000","last_peered":"2023-07-14T11:19:47.967640+0000","last_clean":"2023-07-14T11:19:47.967640+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:19:47.967640+0000","last_undegraded":"2023-07-14T11:19:47.967640+0000","last_fullsized":"2023-07-14T11:19:47.967640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:14.404398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737187+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:19:47.737187+0000","last_peered":"2023-07-14T11:19:47.737187+0000","last_clean":"2023-07-14T11:19:47.737187+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:19:47.737187+0000","last_undegraded":"2023-07-14T11:19:47.737187+0000","last_fullsized":"2023-07-14T11:19:47.737187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:32:25.980256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967578+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:19:47.967578+0000","last_peered":"2023-07-14T11:19:47.967578+0000","last_clean":"2023-07-14T11:19:47.967578+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:19:47.967578+0000","last_undegraded":"2023-07-14T11:19:47.967578+0000","last_fullsized":"2023-07-14T11:19:47.967578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:02:46.721446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718300+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:14.718300+0000","last_peered":"2023-07-14T11:20:14.718300+0000","last_clean":"2023-07-14T11:20:14.718300+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:14.718300+0000","last_undegraded":"2023-07-14T11:20:14.718300+0000","last_fullsized":"2023-07-14T11:20:14.718300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:53:43.048114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967447+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:19:47.967447+0000","last_peered":"2023-07-14T11:19:47.967447+0000","last_clean":"2023-07-14T11:19:47.967447+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:19:47.967447+0000","last_undegraded":"2023-07-14T11:19:47.967447+0000","last_fullsized":"2023-07-14T11:19:47.967447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:38.925420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717638+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:14.717638+0000","last_peered":"2023-07-14T11:20:14.717638+0000","last_clean":"2023-07-14T11:20:14.717638+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:14.717638+0000","last_undegraded":"2023-07-14T11:20:14.717638+0000","last_fullsized":"2023-07-14T11:20:14.717638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:20.321448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968913+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:19:47.968913+0000","last_peered":"2023-07-14T11:19:47.968913+0000","last_clean":"2023-07-14T11:19:47.968913+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:19:47.968913+0000","last_undegraded":"2023-07-14T11:19:47.968913+0000","last_fullsized":"2023-07-14T11:19:47.968913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:18.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197409+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:19:47.197409+0000","last_peered":"2023-07-14T11:19:47.197409+0000","last_clean":"2023-07-14T11:19:47.197409+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:19:47.197409+0000","last_undegraded":"2023-07-14T11:19:47.197409+0000","last_fullsized":"2023-07-14T11:19:47.197409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:17.534514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.678617+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:12.678617+0000","last_peered":"2023-07-14T11:20:12.678617+0000","last_clean":"2023-07-14T11:20:12.678617+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:12.678617+0000","last_undegraded":"2023-07-14T11:20:12.678617+0000","last_fullsized":"2023-07-14T11:20:12.678617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.545637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718422+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:14.718422+0000","last_peered":"2023-07-14T11:20:14.718422+0000","last_clean":"2023-07-14T11:20:14.718422+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:14.718422+0000","last_undegraded":"2023-07-14T11:20:14.718422+0000","last_fullsized":"2023-07-14T11:20:14.718422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:25.009048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197411+0000","last_change":"2023-07-14T11:19:46.096809+0000","last_active":"2023-07-14T11:19:47.197411+0000","last_peered":"2023-07-14T11:19:47.197411+0000","last_clean":"2023-07-14T11:19:47.197411+0000","last_became_active":"2023-07-14T11:19:46.096322+0000","last_became_peered":"2023-07-14T11:19:46.096322+0000","last_unstale":"2023-07-14T11:19:47.197411+0000","last_undegraded":"2023-07-14T11:19:47.197411+0000","last_fullsized":"2023-07-14T11:19:47.197411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:48:14.145380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718924+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:11.718924+0000","last_peered":"2023-07-14T11:20:11.718924+0000","last_clean":"2023-07-14T11:20:11.718924+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:11.718924+0000","last_undegraded":"2023-07-14T11:20:11.718924+0000","last_fullsized":"2023-07-14T11:20:11.718924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:52.081384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968898+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:19:47.968898+0000","last_peered":"2023-07-14T11:19:47.968898+0000","last_clean":"2023-07-14T11:19:47.968898+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:19:47.968898+0000","last_undegraded":"2023-07-14T11:19:47.968898+0000","last_fullsized":"2023-07-14T11:19:47.968898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:04.665047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081037+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:19:47.081037+0000","last_peered":"2023-07-14T11:19:47.081037+0000","last_clean":"2023-07-14T11:19:47.081037+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:19:47.081037+0000","last_undegraded":"2023-07-14T11:19:47.081037+0000","last_fullsized":"2023-07-14T11:19:47.081037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:25.917595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731318+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:14.731318+0000","last_peered":"2023-07-14T11:20:14.731318+0000","last_clean":"2023-07-14T11:20:14.731318+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:14.731318+0000","last_undegraded":"2023-07-14T11:20:14.731318+0000","last_fullsized":"2023-07-14T11:20:14.731318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:39.839896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967040+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:19:47.967040+0000","last_peered":"2023-07-14T11:19:47.967040+0000","last_clean":"2023-07-14T11:19:47.967040+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:19:47.967040+0000","last_undegraded":"2023-07-14T11:19:47.967040+0000","last_fullsized":"2023-07-14T11:19:47.967040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:43.960608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737292+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:19:47.737292+0000","last_peered":"2023-07-14T11:19:47.737292+0000","last_clean":"2023-07-14T11:19:47.737292+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:19:47.737292+0000","last_undegraded":"2023-07-14T11:19:47.737292+0000","last_fullsized":"2023-07-14T11:19:47.737292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:38.334013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967956+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:19:47.967956+0000","last_peered":"2023-07-14T11:19:47.967956+0000","last_clean":"2023-07-14T11:19:47.967956+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:19:47.967956+0000","last_undegraded":"2023-07-14T11:19:47.967956+0000","last_fullsized":"2023-07-14T11:19:47.967956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:16.510067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081653+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:19:47.081653+0000","last_peered":"2023-07-14T11:19:47.081653+0000","last_clean":"2023-07-14T11:19:47.081653+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:19:47.081653+0000","last_undegraded":"2023-07-14T11:19:47.081653+0000","last_fullsized":"2023-07-14T11:19:47.081653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:35.054594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967147+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:19:47.967147+0000","last_peered":"2023-07-14T11:19:47.967147+0000","last_clean":"2023-07-14T11:19:47.967147+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:19:47.967147+0000","last_undegraded":"2023-07-14T11:19:47.967147+0000","last_fullsized":"2023-07-14T11:19:47.967147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:21:05.922286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711077+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:10.711077+0000","last_peered":"2023-07-14T11:20:10.711077+0000","last_clean":"2023-07-14T11:20:10.711077+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:10.711077+0000","last_undegraded":"2023-07-14T11:20:10.711077+0000","last_fullsized":"2023-07-14T11:20:10.711077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:02:01.788231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081629+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:19:47.081629+0000","last_peered":"2023-07-14T11:19:47.081629+0000","last_clean":"2023-07-14T11:19:47.081629+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:19:47.081629+0000","last_undegraded":"2023-07-14T11:19:47.081629+0000","last_fullsized":"2023-07-14T11:19:47.081629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:08.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711857+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:10.711857+0000","last_peered":"2023-07-14T11:20:10.711857+0000","last_clean":"2023-07-14T11:20:10.711857+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:10.711857+0000","last_undegraded":"2023-07-14T11:20:10.711857+0000","last_fullsized":"2023-07-14T11:20:10.711857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:05.009517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737241+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:19:47.737241+0000","last_peered":"2023-07-14T11:19:47.737241+0000","last_clean":"2023-07-14T11:19:47.737241+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:19:47.737241+0000","last_undegraded":"2023-07-14T11:19:47.737241+0000","last_fullsized":"2023-07-14T11:19:47.737241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:30:37.665531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967912+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:19:47.967912+0000","last_peered":"2023-07-14T11:19:47.967912+0000","last_clean":"2023-07-14T11:19:47.967912+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:19:47.967912+0000","last_undegraded":"2023-07-14T11:19:47.967912+0000","last_fullsized":"2023-07-14T11:19:47.967912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:39:26.015436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.710731+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:10.710731+0000","last_peered":"2023-07-14T11:20:10.710731+0000","last_clean":"2023-07-14T11:20:10.710731+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:10.710731+0000","last_undegraded":"2023-07-14T11:20:10.710731+0000","last_fullsized":"2023-07-14T11:20:10.710731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:44.419475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081641+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:19:47.081641+0000","last_peered":"2023-07-14T11:19:47.081641+0000","last_clean":"2023-07-14T11:19:47.081641+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:19:47.081641+0000","last_undegraded":"2023-07-14T11:19:47.081641+0000","last_fullsized":"2023-07-14T11:19:47.081641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:47.740065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711843+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:10.711843+0000","last_peered":"2023-07-14T11:20:10.711843+0000","last_clean":"2023-07-14T11:20:10.711843+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:10.711843+0000","last_undegraded":"2023-07-14T11:20:10.711843+0000","last_fullsized":"2023-07-14T11:20:10.711843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:13:57.943632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717973+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:14.717973+0000","last_peered":"2023-07-14T11:20:14.717973+0000","last_clean":"2023-07-14T11:20:14.717973+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:14.717973+0000","last_undegraded":"2023-07-14T11:20:14.717973+0000","last_fullsized":"2023-07-14T11:20:14.717973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:59.311439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969168+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:19:47.969168+0000","last_peered":"2023-07-14T11:19:47.969168+0000","last_clean":"2023-07-14T11:19:47.969168+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:19:47.969168+0000","last_undegraded":"2023-07-14T11:19:47.969168+0000","last_fullsized":"2023-07-14T11:19:47.969168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:56:28.718616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711234+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:10.711234+0000","last_peered":"2023-07-14T11:20:10.711234+0000","last_clean":"2023-07-14T11:20:10.711234+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:10.711234+0000","last_undegraded":"2023-07-14T11:20:10.711234+0000","last_fullsized":"2023-07-14T11:20:10.711234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:29:12.273324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728223+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:14.728223+0000","last_peered":"2023-07-14T11:20:14.728223+0000","last_clean":"2023-07-14T11:20:14.728223+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:14.728223+0000","last_undegraded":"2023-07-14T11:20:14.728223+0000","last_fullsized":"2023-07-14T11:20:14.728223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:45.374719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080735+0000","last_change":"2023-07-14T11:19:46.097705+0000","last_active":"2023-07-14T11:19:47.080735+0000","last_peered":"2023-07-14T11:19:47.080735+0000","last_clean":"2023-07-14T11:19:47.080735+0000","last_became_active":"2023-07-14T11:19:46.097093+0000","last_became_peered":"2023-07-14T11:19:46.097093+0000","last_unstale":"2023-07-14T11:19:47.080735+0000","last_undegraded":"2023-07-14T11:19:47.080735+0000","last_fullsized":"2023-07-14T11:19:47.080735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:16:13.697886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713460+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:10.713460+0000","last_peered":"2023-07-14T11:20:10.713460+0000","last_clean":"2023-07-14T11:20:10.713460+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:10.713460+0000","last_undegraded":"2023-07-14T11:20:10.713460+0000","last_fullsized":"2023-07-14T11:20:10.713460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:44.692262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968550+0000","last_change":"2023-07-14T11:19:42.077628+0000","last_active":"2023-07-14T11:19:47.968550+0000","last_peered":"2023-07-14T11:19:47.968550+0000","last_clean":"2023-07-14T11:19:47.968550+0000","last_became_active":"2023-07-14T11:19:42.077341+0000","last_became_peered":"2023-07-14T11:19:42.077341+0000","last_unstale":"2023-07-14T11:19:47.968550+0000","last_undegraded":"2023-07-14T11:19:47.968550+0000","last_fullsized":"2023-07-14T11:19:47.968550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:03.185626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082158+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:19:47.082158+0000","last_peered":"2023-07-14T11:19:47.082158+0000","last_clean":"2023-07-14T11:19:47.082158+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:19:47.082158+0000","last_undegraded":"2023-07-14T11:19:47.082158+0000","last_fullsized":"2023-07-14T11:19:47.082158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:07.141091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.680816+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:12.680816+0000","last_peered":"2023-07-14T11:20:12.680816+0000","last_clean":"2023-07-14T11:20:12.680816+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:12.680816+0000","last_undegraded":"2023-07-14T11:20:12.680816+0000","last_fullsized":"2023-07-14T11:20:12.680816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:29.624865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968266+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:19:47.968266+0000","last_peered":"2023-07-14T11:19:47.968266+0000","last_clean":"2023-07-14T11:19:47.968266+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:19:47.968266+0000","last_undegraded":"2023-07-14T11:19:47.968266+0000","last_fullsized":"2023-07-14T11:19:47.968266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:59.325039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728868+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:14.728868+0000","last_peered":"2023-07-14T11:20:14.728868+0000","last_clean":"2023-07-14T11:20:14.728868+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:14.728868+0000","last_undegraded":"2023-07-14T11:20:14.728868+0000","last_fullsized":"2023-07-14T11:20:14.728868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:31.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718779+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:14.718779+0000","last_peered":"2023-07-14T11:20:14.718779+0000","last_clean":"2023-07-14T11:20:14.718779+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:14.718779+0000","last_undegraded":"2023-07-14T11:20:14.718779+0000","last_fullsized":"2023-07-14T11:20:14.718779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:25:31.744731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967979+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:19:47.967979+0000","last_peered":"2023-07-14T11:19:47.967979+0000","last_clean":"2023-07-14T11:19:47.967979+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:19:47.967979+0000","last_undegraded":"2023-07-14T11:19:47.967979+0000","last_fullsized":"2023-07-14T11:19:47.967979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:58.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967884+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:19:47.967884+0000","last_peered":"2023-07-14T11:19:47.967884+0000","last_clean":"2023-07-14T11:19:47.967884+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:19:47.967884+0000","last_undegraded":"2023-07-14T11:19:47.967884+0000","last_fullsized":"2023-07-14T11:19:47.967884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:07.208804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717860+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:14.717860+0000","last_peered":"2023-07-14T11:20:14.717860+0000","last_clean":"2023-07-14T11:20:14.717860+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:14.717860+0000","last_undegraded":"2023-07-14T11:20:14.717860+0000","last_fullsized":"2023-07-14T11:20:14.717860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:15:42.131659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080903+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:19:47.080903+0000","last_peered":"2023-07-14T11:19:47.080903+0000","last_clean":"2023-07-14T11:19:47.080903+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:19:47.080903+0000","last_undegraded":"2023-07-14T11:19:47.080903+0000","last_fullsized":"2023-07-14T11:19:47.080903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:29.469059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T11:19:47.967804+0000","last_change":"2023-07-14T11:20:14.717622+0000","last_active":"2023-07-14T11:20:14.717622+0000","last_peered":"2023-07-14T11:19:47.967804+0000","last_clean":"2023-07-14T11:19:47.967804+0000","last_became_active":"2023-07-14T11:19:46.273092+0000","last_became_peered":"2023-07-14T11:19:46.273092+0000","last_unstale":"2023-07-14T11:19:47.967804+0000","last_undegraded":"2023-07-14T11:19:47.967804+0000","last_fullsized":"2023-07-14T11:19:47.967804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081920+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:19:47.081920+0000","last_peered":"2023-07-14T11:19:47.081920+0000","last_clean":"2023-07-14T11:19:47.081920+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:19:47.081920+0000","last_undegraded":"2023-07-14T11:19:47.081920+0000","last_fullsized":"2023-07-14T11:19:47.081920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:21:18.903408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081118+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:19:47.081118+0000","last_peered":"2023-07-14T11:19:47.081118+0000","last_clean":"2023-07-14T11:19:47.081118+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:19:47.081118+0000","last_undegraded":"2023-07-14T11:19:47.081118+0000","last_fullsized":"2023-07-14T11:19:47.081118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:37.312351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730373+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:14.730373+0000","last_peered":"2023-07-14T11:20:14.730373+0000","last_clean":"2023-07-14T11:20:14.730373+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:14.730373+0000","last_undegraded":"2023-07-14T11:20:14.730373+0000","last_fullsized":"2023-07-14T11:20:14.730373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:10.476499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968436+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:19:47.968436+0000","last_peered":"2023-07-14T11:19:47.968436+0000","last_clean":"2023-07-14T11:19:47.968436+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:19:47.968436+0000","last_undegraded":"2023-07-14T11:19:47.968436+0000","last_fullsized":"2023-07-14T11:19:47.968436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081008+0000","last_change":"2023-07-14T11:19:42.093066+0000","last_active":"2023-07-14T11:19:47.081008+0000","last_peered":"2023-07-14T11:19:47.081008+0000","last_clean":"2023-07-14T11:19:47.081008+0000","last_became_active":"2023-07-14T11:19:42.092719+0000","last_became_peered":"2023-07-14T11:19:42.092719+0000","last_unstale":"2023-07-14T11:19:47.081008+0000","last_undegraded":"2023-07-14T11:19:47.081008+0000","last_fullsized":"2023-07-14T11:19:47.081008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:35:05.018230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.729003+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:14.729003+0000","last_peered":"2023-07-14T11:20:14.729003+0000","last_clean":"2023-07-14T11:20:14.729003+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:14.729003+0000","last_undegraded":"2023-07-14T11:20:14.729003+0000","last_fullsized":"2023-07-14T11:20:14.729003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:29.125385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968186+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:19:47.968186+0000","last_peered":"2023-07-14T11:19:47.968186+0000","last_clean":"2023-07-14T11:19:47.968186+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:19:47.968186+0000","last_undegraded":"2023-07-14T11:19:47.968186+0000","last_fullsized":"2023-07-14T11:19:47.968186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:06.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731196+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:14.731196+0000","last_peered":"2023-07-14T11:20:14.731196+0000","last_clean":"2023-07-14T11:20:14.731196+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:14.731196+0000","last_undegraded":"2023-07-14T11:20:14.731196+0000","last_fullsized":"2023-07-14T11:20:14.731196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:16:34.542159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:49.452867+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:19:49.452867+0000","last_peered":"2023-07-14T11:19:49.452867+0000","last_clean":"2023-07-14T11:19:49.452867+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:19:49.452867+0000","last_undegraded":"2023-07-14T11:19:49.452867+0000","last_fullsized":"2023-07-14T11:19:49.452867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:22.680690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967124+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:19:47.967124+0000","last_peered":"2023-07-14T11:19:47.967124+0000","last_clean":"2023-07-14T11:19:47.967124+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:19:47.967124+0000","last_undegraded":"2023-07-14T11:19:47.967124+0000","last_fullsized":"2023-07-14T11:19:47.967124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:14:39.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:12.676206+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:12.676206+0000","last_peered":"2023-07-14T11:20:12.676206+0000","last_clean":"2023-07-14T11:20:12.676206+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:12.676206+0000","last_undegraded":"2023-07-14T11:20:12.676206+0000","last_fullsized":"2023-07-14T11:20:12.676206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:22:45.207699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968042+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:19:47.968042+0000","last_peered":"2023-07-14T11:19:47.968042+0000","last_clean":"2023-07-14T11:19:47.968042+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:19:47.968042+0000","last_undegraded":"2023-07-14T11:19:47.968042+0000","last_fullsized":"2023-07-14T11:19:47.968042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:23.767120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968619+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:19:47.968619+0000","last_peered":"2023-07-14T11:19:47.968619+0000","last_clean":"2023-07-14T11:19:47.968619+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:19:47.968619+0000","last_undegraded":"2023-07-14T11:19:47.968619+0000","last_fullsized":"2023-07-14T11:19:47.968619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:13.375758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.731070+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:14.731070+0000","last_peered":"2023-07-14T11:20:14.731070+0000","last_clean":"2023-07-14T11:20:14.731070+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:14.731070+0000","last_undegraded":"2023-07-14T11:20:14.731070+0000","last_fullsized":"2023-07-14T11:20:14.731070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:59:07.405076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.717379+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:11.717379+0000","last_peered":"2023-07-14T11:20:11.717379+0000","last_clean":"2023-07-14T11:20:11.717379+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:11.717379+0000","last_undegraded":"2023-07-14T11:20:11.717379+0000","last_fullsized":"2023-07-14T11:20:11.717379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:24.177685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080724+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:19:47.080724+0000","last_peered":"2023-07-14T11:19:47.080724+0000","last_clean":"2023-07-14T11:19:47.080724+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:19:47.080724+0000","last_undegraded":"2023-07-14T11:19:47.080724+0000","last_fullsized":"2023-07-14T11:19:47.080724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:52:43.413334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718394+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:14.718394+0000","last_peered":"2023-07-14T11:20:14.718394+0000","last_clean":"2023-07-14T11:20:14.718394+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:14.718394+0000","last_undegraded":"2023-07-14T11:20:14.718394+0000","last_fullsized":"2023-07-14T11:20:14.718394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:22.172922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730551+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:14.730551+0000","last_peered":"2023-07-14T11:20:14.730551+0000","last_clean":"2023-07-14T11:20:14.730551+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:14.730551+0000","last_undegraded":"2023-07-14T11:20:14.730551+0000","last_fullsized":"2023-07-14T11:20:14.730551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:37.007482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967049+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:19:47.967049+0000","last_peered":"2023-07-14T11:19:47.967049+0000","last_clean":"2023-07-14T11:19:47.967049+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:19:47.967049+0000","last_undegraded":"2023-07-14T11:19:47.967049+0000","last_fullsized":"2023-07-14T11:19:47.967049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:39:36.064722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:11.718705+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:11.718705+0000","last_peered":"2023-07-14T11:20:11.718705+0000","last_clean":"2023-07-14T11:20:11.718705+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:11.718705+0000","last_undegraded":"2023-07-14T11:20:11.718705+0000","last_fullsized":"2023-07-14T11:20:11.718705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:26:49.607683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967586+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:19:47.967586+0000","last_peered":"2023-07-14T11:19:47.967586+0000","last_clean":"2023-07-14T11:19:47.967586+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:19:47.967586+0000","last_undegraded":"2023-07-14T11:19:47.967586+0000","last_fullsized":"2023-07-14T11:19:47.967586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:35:51.028974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.196923+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:19:47.196923+0000","last_peered":"2023-07-14T11:19:47.196923+0000","last_clean":"2023-07-14T11:19:47.196923+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:19:47.196923+0000","last_undegraded":"2023-07-14T11:19:47.196923+0000","last_fullsized":"2023-07-14T11:19:47.196923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:20.885447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.711775+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:10.711775+0000","last_peered":"2023-07-14T11:20:10.711775+0000","last_clean":"2023-07-14T11:20:10.711775+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:10.711775+0000","last_undegraded":"2023-07-14T11:20:10.711775+0000","last_fullsized":"2023-07-14T11:20:10.711775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:36.953428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.737388+0000","last_change":"2023-07-14T11:19:42.085058+0000","last_active":"2023-07-14T11:19:47.737388+0000","last_peered":"2023-07-14T11:19:47.737388+0000","last_clean":"2023-07-14T11:19:47.737388+0000","last_became_active":"2023-07-14T11:19:42.083380+0000","last_became_peered":"2023-07-14T11:19:42.083380+0000","last_unstale":"2023-07-14T11:19:47.737388+0000","last_undegraded":"2023-07-14T11:19:47.737388+0000","last_fullsized":"2023-07-14T11:19:47.737388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:52:52.430115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967817+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:19:47.967817+0000","last_peered":"2023-07-14T11:19:47.967817+0000","last_clean":"2023-07-14T11:19:47.967817+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:19:47.967817+0000","last_undegraded":"2023-07-14T11:19:47.967817+0000","last_fullsized":"2023-07-14T11:19:47.967817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:25:48.734999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717656+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:14.717656+0000","last_peered":"2023-07-14T11:20:14.717656+0000","last_clean":"2023-07-14T11:20:14.717656+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:14.717656+0000","last_undegraded":"2023-07-14T11:20:14.717656+0000","last_fullsized":"2023-07-14T11:20:14.717656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:28.963610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.082443+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:19:47.082443+0000","last_peered":"2023-07-14T11:19:47.082443+0000","last_clean":"2023-07-14T11:19:47.082443+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:19:47.082443+0000","last_undegraded":"2023-07-14T11:19:47.082443+0000","last_fullsized":"2023-07-14T11:19:47.082443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:43.469946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:20:10.713281+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:10.713281+0000","last_peered":"2023-07-14T11:20:10.713281+0000","last_clean":"2023-07-14T11:20:10.713281+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:10.713281+0000","last_undegraded":"2023-07-14T11:20:10.713281+0000","last_fullsized":"2023-07-14T11:20:10.713281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:45:36.543244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080522+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:19:47.080522+0000","last_peered":"2023-07-14T11:19:47.080522+0000","last_clean":"2023-07-14T11:19:47.080522+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:19:47.080522+0000","last_undegraded":"2023-07-14T11:19:47.080522+0000","last_fullsized":"2023-07-14T11:19:47.080522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:44.548171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967752+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:19:47.967752+0000","last_peered":"2023-07-14T11:19:47.967752+0000","last_clean":"2023-07-14T11:19:47.967752+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:19:47.967752+0000","last_undegraded":"2023-07-14T11:19:47.967752+0000","last_fullsized":"2023-07-14T11:19:47.967752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:50:53.130285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197038+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:19:47.197038+0000","last_peered":"2023-07-14T11:19:47.197038+0000","last_clean":"2023-07-14T11:19:47.197038+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:19:47.197038+0000","last_undegraded":"2023-07-14T11:19:47.197038+0000","last_fullsized":"2023-07-14T11:19:47.197038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:15.617687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967360+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:19:47.967360+0000","last_peered":"2023-07-14T11:19:47.967360+0000","last_clean":"2023-07-14T11:19:47.967360+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:19:47.967360+0000","last_undegraded":"2023-07-14T11:19:47.967360+0000","last_fullsized":"2023-07-14T11:19:47.967360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:21.159717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967674+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:19:47.967674+0000","last_peered":"2023-07-14T11:19:47.967674+0000","last_clean":"2023-07-14T11:19:47.967674+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:19:47.967674+0000","last_undegraded":"2023-07-14T11:19:47.967674+0000","last_fullsized":"2023-07-14T11:19:47.967674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:57:50.109303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717280+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:14.717280+0000","last_peered":"2023-07-14T11:20:14.717280+0000","last_clean":"2023-07-14T11:20:14.717280+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:14.717280+0000","last_undegraded":"2023-07-14T11:20:14.717280+0000","last_fullsized":"2023-07-14T11:20:14.717280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:38:55.878246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968641+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:19:47.968641+0000","last_peered":"2023-07-14T11:19:47.968641+0000","last_clean":"2023-07-14T11:19:47.968641+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:19:47.968641+0000","last_undegraded":"2023-07-14T11:19:47.968641+0000","last_fullsized":"2023-07-14T11:19:47.968641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:57.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197577+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:19:47.197577+0000","last_peered":"2023-07-14T11:19:47.197577+0000","last_clean":"2023-07-14T11:19:47.197577+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:19:47.197577+0000","last_undegraded":"2023-07-14T11:19:47.197577+0000","last_fullsized":"2023-07-14T11:19:47.197577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:48.609439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.969031+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:19:47.969031+0000","last_peered":"2023-07-14T11:19:47.969031+0000","last_clean":"2023-07-14T11:19:47.969031+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:19:47.969031+0000","last_undegraded":"2023-07-14T11:19:47.969031+0000","last_fullsized":"2023-07-14T11:19:47.969031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:55.843732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736878+0000","last_change":"2023-07-14T11:19:46.088344+0000","last_active":"2023-07-14T11:19:47.736878+0000","last_peered":"2023-07-14T11:19:47.736878+0000","last_clean":"2023-07-14T11:19:47.736878+0000","last_became_active":"2023-07-14T11:19:46.087922+0000","last_became_peered":"2023-07-14T11:19:46.087922+0000","last_unstale":"2023-07-14T11:19:47.736878+0000","last_undegraded":"2023-07-14T11:19:47.736878+0000","last_fullsized":"2023-07-14T11:19:47.736878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:02:04.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968974+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:19:47.968974+0000","last_peered":"2023-07-14T11:19:47.968974+0000","last_clean":"2023-07-14T11:19:47.968974+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:19:47.968974+0000","last_undegraded":"2023-07-14T11:19:47.968974+0000","last_fullsized":"2023-07-14T11:19:47.968974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:56:35.324459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968655+0000","last_change":"2023-07-14T11:19:42.073286+0000","last_active":"2023-07-14T11:19:47.968655+0000","last_peered":"2023-07-14T11:19:47.968655+0000","last_clean":"2023-07-14T11:19:47.968655+0000","last_became_active":"2023-07-14T11:19:42.072976+0000","last_became_peered":"2023-07-14T11:19:42.072976+0000","last_unstale":"2023-07-14T11:19:47.968655+0000","last_undegraded":"2023-07-14T11:19:47.968655+0000","last_fullsized":"2023-07-14T11:19:47.968655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:37:29.066629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967707+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:19:47.967707+0000","last_peered":"2023-07-14T11:19:47.967707+0000","last_clean":"2023-07-14T11:19:47.967707+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:19:47.967707+0000","last_undegraded":"2023-07-14T11:19:47.967707+0000","last_fullsized":"2023-07-14T11:19:47.967707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:15.123989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.728744+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:14.728744+0000","last_peered":"2023-07-14T11:20:14.728744+0000","last_clean":"2023-07-14T11:20:14.728744+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:14.728744+0000","last_undegraded":"2023-07-14T11:20:14.728744+0000","last_fullsized":"2023-07-14T11:20:14.728744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:03:42.954256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.968714+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:19:47.968714+0000","last_peered":"2023-07-14T11:19:47.968714+0000","last_clean":"2023-07-14T11:19:47.968714+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:19:47.968714+0000","last_undegraded":"2023-07-14T11:19:47.968714+0000","last_fullsized":"2023-07-14T11:19:47.968714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:03.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080574+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:19:47.080574+0000","last_peered":"2023-07-14T11:19:47.080574+0000","last_clean":"2023-07-14T11:19:47.080574+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:19:47.080574+0000","last_undegraded":"2023-07-14T11:19:47.080574+0000","last_fullsized":"2023-07-14T11:19:47.080574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:06.602465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197215+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:19:47.197215+0000","last_peered":"2023-07-14T11:19:47.197215+0000","last_clean":"2023-07-14T11:19:47.197215+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:19:47.197215+0000","last_undegraded":"2023-07-14T11:19:47.197215+0000","last_fullsized":"2023-07-14T11:19:47.197215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:52:37.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197079+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:19:47.197079+0000","last_peered":"2023-07-14T11:19:47.197079+0000","last_clean":"2023-07-14T11:19:47.197079+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:19:47.197079+0000","last_undegraded":"2023-07-14T11:19:47.197079+0000","last_fullsized":"2023-07-14T11:19:47.197079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:26:40.329142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.736992+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:19:47.736992+0000","last_peered":"2023-07-14T11:19:47.736992+0000","last_clean":"2023-07-14T11:19:47.736992+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:19:47.736992+0000","last_undegraded":"2023-07-14T11:19:47.736992+0000","last_fullsized":"2023-07-14T11:19:47.736992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:09:09.081413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.730659+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:14.730659+0000","last_peered":"2023-07-14T11:20:14.730659+0000","last_clean":"2023-07-14T11:20:14.730659+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:14.730659+0000","last_undegraded":"2023-07-14T11:20:14.730659+0000","last_fullsized":"2023-07-14T11:20:14.730659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:18.996495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.081049+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:19:47.081049+0000","last_peered":"2023-07-14T11:19:47.081049+0000","last_clean":"2023-07-14T11:19:47.081049+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:19:47.081049+0000","last_undegraded":"2023-07-14T11:19:47.081049+0000","last_fullsized":"2023-07-14T11:19:47.081049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:56.251387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.197612+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:19:47.197612+0000","last_peered":"2023-07-14T11:19:47.197612+0000","last_clean":"2023-07-14T11:19:47.197612+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:19:47.197612+0000","last_undegraded":"2023-07-14T11:19:47.197612+0000","last_fullsized":"2023-07-14T11:19:47.197612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:32:51.167057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967746+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:19:47.967746+0000","last_peered":"2023-07-14T11:19:47.967746+0000","last_clean":"2023-07-14T11:19:47.967746+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:19:47.967746+0000","last_undegraded":"2023-07-14T11:19:47.967746+0000","last_fullsized":"2023-07-14T11:19:47.967746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:37.267773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-14T11:19:42.068794+0000","last_change":"2023-07-14T11:20:15.683458+0000","last_active":"2023-07-14T11:19:41.086153+0000","last_peered":"2023-07-14T11:19:41.086153+0000","last_clean":"2023-07-14T11:19:41.086153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:19:42.068794+0000","last_undegraded":"2023-07-14T11:19:42.068794+0000","last_fullsized":"2023-07-14T11:19:42.068794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.967293+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:19:47.967293+0000","last_peered":"2023-07-14T11:19:47.967293+0000","last_clean":"2023-07-14T11:19:47.967293+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:19:47.967293+0000","last_undegraded":"2023-07-14T11:19:47.967293+0000","last_fullsized":"2023-07-14T11:19:47.967293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:58:16.557411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.727007+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:14.727007+0000","last_peered":"2023-07-14T11:20:14.727007+0000","last_clean":"2023-07-14T11:20:14.727007+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:14.727007+0000","last_undegraded":"2023-07-14T11:20:14.727007+0000","last_fullsized":"2023-07-14T11:20:14.727007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:33.588223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T11:19:47.080793+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:19:47.080793+0000","last_peered":"2023-07-14T11:19:47.080793+0000","last_clean":"2023-07-14T11:19:47.080793+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:19:47.080793+0000","last_undegraded":"2023-07-14T11:19:47.080793+0000","last_fullsized":"2023-07-14T11:19:47.080793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:57.953734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":121,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":60474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":471801,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.37},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.755},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.78000000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1062815,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.876},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82999999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":467907,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.628},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.1639999999999997,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70299999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.59799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":481467,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93000000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0129999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1062952,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":466940,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.439},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.85099999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.876},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":515148,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.84799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.623},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.752}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":468328,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":1.042},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.55800000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.621},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.64800000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":1.147}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.99099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":471252,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56299999999999994}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467317,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.376}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.5},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.622}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73899999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":467719,"data_compressed":11704,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54000000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1058588,"data_compressed":47401,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.624}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50800000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:17 smithi175 ceph-mon[129081]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:17 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2285566743' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-07-14T11:20:18.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:17 smithi182 ceph-mon[129366]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:18.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:17 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2285566743' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-07-14T11:20:18.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:18 smithi161 ceph-mon[95526]: from='client.24872 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:18.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:18 smithi161 ceph-mon[95526]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.6 KiB/s wr, 18 op/s 2023-07-14T11:20:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:18 smithi175 ceph-mon[129081]: from='client.24872 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:18 smithi175 ceph-mon[129081]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.6 KiB/s wr, 18 op/s 2023-07-14T11:20:19.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:18 smithi182 ceph-mon[129366]: from='client.24872 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:19.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:18 smithi182 ceph-mon[129366]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.6 KiB/s wr, 18 op/s 2023-07-14T11:20:20.996 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:21.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.207+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/206365814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 msgr2=0x7f2c54101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/206365814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 0x7f2c54101140 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009990 tx=0x7f2c44009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/206365814 shutdown_connections 2023-07-14T11:20:21.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/206365814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c54101680 0x7f2c54107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/206365814 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c540ffb00 0x7f2c540fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/206365814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 0x7f2c54101140 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/206365814 >> 172.21.15.161:0/206365814 conn(0x7f2c540fb0e0 msgr2=0x7f2c540fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:21.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/206365814 shutdown_connections 2023-07-14T11:20:21.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.208+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/206365814 wait complete. 2023-07-14T11:20:21.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 Processor -- start 2023-07-14T11:20:21.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 -- start start 2023-07-14T11:20:21.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c540ffb00 0x7f2c54197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 0x7f2c54197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c54101680 0x7f2c5419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5419f590 con 0x7f2c54100cf0 2023-07-14T11:20:21.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5419f6d0 con 0x7f2c540ffb00 2023-07-14T11:20:21.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5c3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5419f860 con 0x7f2c54101680 2023-07-14T11:20:21.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c540ffb00 0x7f2c54197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 0x7f2c54197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5a99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c54101680 0x7f2c5419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c540ffb00 0x7f2c54197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57248/0 (socket says 172.21.15.161:57248) 2023-07-14T11:20:21.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.209+0000 7f2c5a19b700 1 -- 172.21.15.161:0/1239232067 learned_addr learned my addr 172.21.15.161:0/1239232067 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:21.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5a19b700 1 -- 172.21.15.161:0/1239232067 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c54101680 msgr2=0x7f2c5419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5a19b700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c54101680 0x7f2c5419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5a19b700 1 -- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 msgr2=0x7f2c54197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5a19b700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 0x7f2c54197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5a19b700 1 -- 172.21.15.161:0/1239232067 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c440095f0 con 0x7f2c540ffb00 2023-07-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5999a700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 0x7f2c54197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5a19b700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c540ffb00 0x7f2c54197a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000e970 tx=0x7f2c5000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c4b7fe700 1 -- 172.21.15.161:0/1239232067 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50009790 con 0x7f2c540ffb00 2023-07-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5419fb40 con 0x7f2c540ffb00 2023-07-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c4b7fe700 1 -- 172.21.15.161:0/1239232067 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c5000c7e0 con 0x7f2c540ffb00 2023-07-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.210+0000 7f2c4b7fe700 1 -- 172.21.15.161:0/1239232067 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50003c70 con 0x7f2c540ffb00 2023-07-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.211+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c541a0050 con 0x7f2c540ffb00 2023-07-14T11:20:21.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.211+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5410b320 con 0x7f2c540ffb00 2023-07-14T11:20:21.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.213+0000 7f2c4b7fe700 1 -- 172.21.15.161:0/1239232067 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2c5000cc50 con 0x7f2c540ffb00 2023-07-14T11:20:21.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.215+0000 7f2c4b7fe700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2c400a1220 0x7f2c400a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.215+0000 7f2c4b7fe700 1 -- 172.21.15.161:0/1239232067 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2c50014070 con 0x7f2c540ffb00 2023-07-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.215+0000 7f2c5999a700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2c400a1220 0x7f2c400a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.215+0000 7f2c5999a700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2c400a1220 0x7f2c400a36d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b2e0 tx=0x7f2c44005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.217+0000 7f2c4b7fe700 1 -- 172.21.15.161:0/1239232067 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c500968b0 con 0x7f2c540ffb00 2023-07-14T11:20:21.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.411+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c54107cb0 con 0x7f2c400a1220 2023-07-14T11:20:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:21 smithi175 ceph-mon[129081]: pgmap v293: 129 pgs: 2 peering, 127 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 16 op/s 2023-07-14T11:20:21.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.423+0000 7f2c4b7fe700 1 -- 172.21.15.161:0/1239232067 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422716 (secure 0 0 0) 0x7f2c54107cb0 con 0x7f2c400a1220 2023-07-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:21.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2c400a1220 msgr2=0x7f2c400a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2c400a1220 0x7f2c400a36d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b2e0 tx=0x7f2c44005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c540ffb00 msgr2=0x7f2c54197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c540ffb00 0x7f2c54197a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000e970 tx=0x7f2c5000ec80 comp rx=0 tx=0).stop 2023-07-14T11:20:21.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 shutdown_connections 2023-07-14T11:20:21.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2c400a1220 0x7f2c400a36d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c54101680 0x7f2c5419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c540ffb00 0x7f2c54197a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 --2- 172.21.15.161:0/1239232067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54100cf0 0x7f2c54197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.428+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 >> 172.21.15.161:0/1239232067 conn(0x7f2c540fb0e0 msgr2=0x7f2c54106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:21.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.429+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 shutdown_connections 2023-07-14T11:20:21.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.429+0000 7f2c5c3ff700 1 -- 172.21.15.161:0/1239232067 wait complete. 2023-07-14T11:20:21.440 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:21.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:21 smithi161 ceph-mon[95526]: pgmap v293: 129 pgs: 2 peering, 127 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 16 op/s 2023-07-14T11:20:21.450 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-07-14T11:20:20.104895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637256,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":194,"num_read_kb":199,"num_write":325,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264,"ondisk_log_size":264,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335276,"kb_used_data":11852,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606548,"statfs":{"total":1151940427776,"available":1151597105152,"internally_reserved":0,"allocated":12136448,"data_stored":7743322,"data_compressed":309735,"data_compressed_allocated":417792,"data_compressed_original":2334720,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42206,"num_objects":33,"num_object_clones":0,"num_object_copies":101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":58,"num_read_kb":80,"num_write":76,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002159"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726364+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:20:15.726364+0000","last_peered":"2023-07-14T11:20:15.726364+0000","last_clean":"2023-07-14T11:20:15.726364+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:20:15.726364+0000","last_undegraded":"2023-07-14T11:20:15.726364+0000","last_fullsized":"2023-07-14T11:20:15.726364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:25.074131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726699+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:20:15.726699+0000","last_peered":"2023-07-14T11:20:15.726699+0000","last_clean":"2023-07-14T11:20:15.726699+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:20:15.726699+0000","last_undegraded":"2023-07-14T11:20:15.726699+0000","last_fullsized":"2023-07-14T11:20:15.726699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:41.854592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726554+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:20:15.726554+0000","last_peered":"2023-07-14T11:20:15.726554+0000","last_clean":"2023-07-14T11:20:15.726554+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:20:15.726554+0000","last_undegraded":"2023-07-14T11:20:15.726554+0000","last_fullsized":"2023-07-14T11:20:15.726554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:37.342912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726181+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:20:15.726181+0000","last_peered":"2023-07-14T11:20:15.726181+0000","last_clean":"2023-07-14T11:20:15.726181+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:20:15.726181+0000","last_undegraded":"2023-07-14T11:20:15.726181+0000","last_fullsized":"2023-07-14T11:20:15.726181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:46.208987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717211+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:14.717211+0000","last_peered":"2023-07-14T11:20:14.717211+0000","last_clean":"2023-07-14T11:20:14.717211+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:14.717211+0000","last_undegraded":"2023-07-14T11:20:14.717211+0000","last_fullsized":"2023-07-14T11:20:14.717211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:45:02.418574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765846+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:20:15.765846+0000","last_peered":"2023-07-14T11:20:15.765846+0000","last_clean":"2023-07-14T11:20:15.765846+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:20:15.765846+0000","last_undegraded":"2023-07-14T11:20:15.765846+0000","last_fullsized":"2023-07-14T11:20:15.765846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:31:09.497877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725992+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:20:15.725992+0000","last_peered":"2023-07-14T11:20:15.725992+0000","last_clean":"2023-07-14T11:20:15.725992+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:20:15.725992+0000","last_undegraded":"2023-07-14T11:20:15.725992+0000","last_fullsized":"2023-07-14T11:20:15.725992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:18.893918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761828+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:20:15.761828+0000","last_peered":"2023-07-14T11:20:15.761828+0000","last_clean":"2023-07-14T11:20:15.761828+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:20:15.761828+0000","last_undegraded":"2023-07-14T11:20:15.761828+0000","last_fullsized":"2023-07-14T11:20:15.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730008+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:20:15.730008+0000","last_peered":"2023-07-14T11:20:15.730008+0000","last_clean":"2023-07-14T11:20:15.730008+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:20:15.730008+0000","last_undegraded":"2023-07-14T11:20:15.730008+0000","last_fullsized":"2023-07-14T11:20:15.730008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:25:58.920431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726577+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:20:15.726577+0000","last_peered":"2023-07-14T11:20:15.726577+0000","last_clean":"2023-07-14T11:20:15.726577+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:20:15.726577+0000","last_undegraded":"2023-07-14T11:20:15.726577+0000","last_fullsized":"2023-07-14T11:20:15.726577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:29.172086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726169+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:20:15.726169+0000","last_peered":"2023-07-14T11:20:15.726169+0000","last_clean":"2023-07-14T11:20:15.726169+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:20:15.726169+0000","last_undegraded":"2023-07-14T11:20:15.726169+0000","last_fullsized":"2023-07-14T11:20:15.726169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:29.746118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723416+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:15.723416+0000","last_peered":"2023-07-14T11:20:15.723416+0000","last_clean":"2023-07-14T11:20:15.723416+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:15.723416+0000","last_undegraded":"2023-07-14T11:20:15.723416+0000","last_fullsized":"2023-07-14T11:20:15.723416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:57.801423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725956+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:20:15.725956+0000","last_peered":"2023-07-14T11:20:15.725956+0000","last_clean":"2023-07-14T11:20:15.725956+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:20:15.725956+0000","last_undegraded":"2023-07-14T11:20:15.725956+0000","last_fullsized":"2023-07-14T11:20:15.725956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:01.673209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726027+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:20:15.726027+0000","last_peered":"2023-07-14T11:20:15.726027+0000","last_clean":"2023-07-14T11:20:15.726027+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:20:15.726027+0000","last_undegraded":"2023-07-14T11:20:15.726027+0000","last_fullsized":"2023-07-14T11:20:15.726027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:10.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726533+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:20:15.726533+0000","last_peered":"2023-07-14T11:20:15.726533+0000","last_clean":"2023-07-14T11:20:15.726533+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:20:15.726533+0000","last_undegraded":"2023-07-14T11:20:15.726533+0000","last_fullsized":"2023-07-14T11:20:15.726533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:33.733562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763940+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:20:15.763940+0000","last_peered":"2023-07-14T11:20:15.763940+0000","last_clean":"2023-07-14T11:20:15.763940+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:20:15.763940+0000","last_undegraded":"2023-07-14T11:20:15.763940+0000","last_fullsized":"2023-07-14T11:20:15.763940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:28:49.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725800+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:20:15.725800+0000","last_peered":"2023-07-14T11:20:15.725800+0000","last_clean":"2023-07-14T11:20:15.725800+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:20:15.725800+0000","last_undegraded":"2023-07-14T11:20:15.725800+0000","last_fullsized":"2023-07-14T11:20:15.725800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:30.183181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726357+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:20:15.726357+0000","last_peered":"2023-07-14T11:20:15.726357+0000","last_clean":"2023-07-14T11:20:15.726357+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:20:15.726357+0000","last_undegraded":"2023-07-14T11:20:15.726357+0000","last_fullsized":"2023-07-14T11:20:15.726357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:13.424624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763774+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:15.763774+0000","last_peered":"2023-07-14T11:20:15.763774+0000","last_clean":"2023-07-14T11:20:15.763774+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:15.763774+0000","last_undegraded":"2023-07-14T11:20:15.763774+0000","last_fullsized":"2023-07-14T11:20:15.763774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:54.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762089+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:20:15.762089+0000","last_peered":"2023-07-14T11:20:15.762089+0000","last_clean":"2023-07-14T11:20:15.762089+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:20:15.762089+0000","last_undegraded":"2023-07-14T11:20:15.762089+0000","last_fullsized":"2023-07-14T11:20:15.762089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:06.259730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724677+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:20:15.724677+0000","last_peered":"2023-07-14T11:20:15.724677+0000","last_clean":"2023-07-14T11:20:15.724677+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:20:15.724677+0000","last_undegraded":"2023-07-14T11:20:15.724677+0000","last_fullsized":"2023-07-14T11:20:15.724677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:07.076184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762219+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:20:15.762219+0000","last_peered":"2023-07-14T11:20:15.762219+0000","last_clean":"2023-07-14T11:20:15.762219+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:20:15.762219+0000","last_undegraded":"2023-07-14T11:20:15.762219+0000","last_fullsized":"2023-07-14T11:20:15.762219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:30.071523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726815+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:20:15.726815+0000","last_peered":"2023-07-14T11:20:15.726815+0000","last_clean":"2023-07-14T11:20:15.726815+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:20:15.726815+0000","last_undegraded":"2023-07-14T11:20:15.726815+0000","last_fullsized":"2023-07-14T11:20:15.726815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:41:41.007835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724865+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:20:15.724865+0000","last_peered":"2023-07-14T11:20:15.724865+0000","last_clean":"2023-07-14T11:20:15.724865+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:20:15.724865+0000","last_undegraded":"2023-07-14T11:20:15.724865+0000","last_fullsized":"2023-07-14T11:20:15.724865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:56:28.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718300+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:14.718300+0000","last_peered":"2023-07-14T11:20:14.718300+0000","last_clean":"2023-07-14T11:20:14.718300+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:14.718300+0000","last_undegraded":"2023-07-14T11:20:14.718300+0000","last_fullsized":"2023-07-14T11:20:14.718300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:53:43.048114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726665+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:20:15.726665+0000","last_peered":"2023-07-14T11:20:15.726665+0000","last_clean":"2023-07-14T11:20:15.726665+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:20:15.726665+0000","last_undegraded":"2023-07-14T11:20:15.726665+0000","last_fullsized":"2023-07-14T11:20:15.726665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:39.791849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717638+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:14.717638+0000","last_peered":"2023-07-14T11:20:14.717638+0000","last_clean":"2023-07-14T11:20:14.717638+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:14.717638+0000","last_undegraded":"2023-07-14T11:20:14.717638+0000","last_fullsized":"2023-07-14T11:20:14.717638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:20.321448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759435+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:20:15.759435+0000","last_peered":"2023-07-14T11:20:15.759435+0000","last_clean":"2023-07-14T11:20:15.759435+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:20:15.759435+0000","last_undegraded":"2023-07-14T11:20:15.759435+0000","last_fullsized":"2023-07-14T11:20:15.759435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:51.343442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725567+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:20:15.725567+0000","last_peered":"2023-07-14T11:20:15.725567+0000","last_clean":"2023-07-14T11:20:15.725567+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:20:15.725567+0000","last_undegraded":"2023-07-14T11:20:15.725567+0000","last_fullsized":"2023-07-14T11:20:15.725567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:50.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.686170+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:16.686170+0000","last_peered":"2023-07-14T11:20:16.686170+0000","last_clean":"2023-07-14T11:20:16.686170+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:16.686170+0000","last_undegraded":"2023-07-14T11:20:16.686170+0000","last_fullsized":"2023-07-14T11:20:16.686170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:41.232654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718422+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:14.718422+0000","last_peered":"2023-07-14T11:20:14.718422+0000","last_clean":"2023-07-14T11:20:14.718422+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:14.718422+0000","last_undegraded":"2023-07-14T11:20:14.718422+0000","last_fullsized":"2023-07-14T11:20:14.718422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:25.009048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766231+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:20:15.766231+0000","last_peered":"2023-07-14T11:20:15.766231+0000","last_clean":"2023-07-14T11:20:15.766231+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:20:15.766231+0000","last_undegraded":"2023-07-14T11:20:15.766231+0000","last_fullsized":"2023-07-14T11:20:15.766231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:16:19.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730104+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:15.730104+0000","last_peered":"2023-07-14T11:20:15.730104+0000","last_clean":"2023-07-14T11:20:15.730104+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:15.730104+0000","last_undegraded":"2023-07-14T11:20:15.730104+0000","last_fullsized":"2023-07-14T11:20:15.730104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:54:17.106221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759228+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:20:15.759228+0000","last_peered":"2023-07-14T11:20:15.759228+0000","last_clean":"2023-07-14T11:20:15.759228+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:20:15.759228+0000","last_undegraded":"2023-07-14T11:20:15.759228+0000","last_fullsized":"2023-07-14T11:20:15.759228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:47.650473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726249+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:20:15.726249+0000","last_peered":"2023-07-14T11:20:15.726249+0000","last_clean":"2023-07-14T11:20:15.726249+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:20:15.726249+0000","last_undegraded":"2023-07-14T11:20:15.726249+0000","last_fullsized":"2023-07-14T11:20:15.726249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:10.265432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722943+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:15.722943+0000","last_peered":"2023-07-14T11:20:15.722943+0000","last_clean":"2023-07-14T11:20:15.722943+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:15.722943+0000","last_undegraded":"2023-07-14T11:20:15.722943+0000","last_fullsized":"2023-07-14T11:20:15.722943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:48.217979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761668+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:20:15.761668+0000","last_peered":"2023-07-14T11:20:15.761668+0000","last_clean":"2023-07-14T11:20:15.761668+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:20:15.761668+0000","last_undegraded":"2023-07-14T11:20:15.761668+0000","last_fullsized":"2023-07-14T11:20:15.761668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.096155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726651+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:20:15.726651+0000","last_peered":"2023-07-14T11:20:15.726651+0000","last_clean":"2023-07-14T11:20:15.726651+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:20:15.726651+0000","last_undegraded":"2023-07-14T11:20:15.726651+0000","last_fullsized":"2023-07-14T11:20:15.726651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:36.842168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758847+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:20:15.758847+0000","last_peered":"2023-07-14T11:20:15.758847+0000","last_clean":"2023-07-14T11:20:15.758847+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:20:15.758847+0000","last_undegraded":"2023-07-14T11:20:15.758847+0000","last_fullsized":"2023-07-14T11:20:15.758847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:38.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763190+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:20:15.763190+0000","last_peered":"2023-07-14T11:20:15.763190+0000","last_clean":"2023-07-14T11:20:15.763190+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:20:15.763190+0000","last_undegraded":"2023-07-14T11:20:15.763190+0000","last_fullsized":"2023-07-14T11:20:15.763190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:23.015078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761792+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:20:15.761792+0000","last_peered":"2023-07-14T11:20:15.761792+0000","last_clean":"2023-07-14T11:20:15.761792+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:20:15.761792+0000","last_undegraded":"2023-07-14T11:20:15.761792+0000","last_fullsized":"2023-07-14T11:20:15.761792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.645359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729832+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:15.729832+0000","last_peered":"2023-07-14T11:20:15.729832+0000","last_clean":"2023-07-14T11:20:15.729832+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:15.729832+0000","last_undegraded":"2023-07-14T11:20:15.729832+0000","last_fullsized":"2023-07-14T11:20:15.729832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.137485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763325+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:20:15.763325+0000","last_peered":"2023-07-14T11:20:15.763325+0000","last_clean":"2023-07-14T11:20:15.763325+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:20:15.763325+0000","last_undegraded":"2023-07-14T11:20:15.763325+0000","last_fullsized":"2023-07-14T11:20:15.763325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:28.186030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726329+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:15.726329+0000","last_peered":"2023-07-14T11:20:15.726329+0000","last_clean":"2023-07-14T11:20:15.726329+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:15.726329+0000","last_undegraded":"2023-07-14T11:20:15.726329+0000","last_fullsized":"2023-07-14T11:20:15.726329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:38.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726931+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:20:15.726931+0000","last_peered":"2023-07-14T11:20:15.726931+0000","last_clean":"2023-07-14T11:20:15.726931+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:20:15.726931+0000","last_undegraded":"2023-07-14T11:20:15.726931+0000","last_fullsized":"2023-07-14T11:20:15.726931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:58.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725362+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:20:15.725362+0000","last_peered":"2023-07-14T11:20:15.725362+0000","last_clean":"2023-07-14T11:20:15.725362+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:20:15.725362+0000","last_undegraded":"2023-07-14T11:20:15.725362+0000","last_fullsized":"2023-07-14T11:20:15.725362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:29.343315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759436+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:15.759436+0000","last_peered":"2023-07-14T11:20:15.759436+0000","last_clean":"2023-07-14T11:20:15.759436+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:15.759436+0000","last_undegraded":"2023-07-14T11:20:15.759436+0000","last_fullsized":"2023-07-14T11:20:15.759436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:17:07.579078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762929+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:20:15.762929+0000","last_peered":"2023-07-14T11:20:15.762929+0000","last_clean":"2023-07-14T11:20:15.762929+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:20:15.762929+0000","last_undegraded":"2023-07-14T11:20:15.762929+0000","last_fullsized":"2023-07-14T11:20:15.762929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:41.993201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'23","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.690487+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:16.690487+0000","last_peered":"2023-07-14T11:20:16.690487+0000","last_clean":"2023-07-14T11:20:16.690487+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:16.690487+0000","last_undegraded":"2023-07-14T11:20:16.690487+0000","last_fullsized":"2023-07-14T11:20:16.690487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:08:09.309573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43128,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":49,"num_write":24,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717973+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:14.717973+0000","last_peered":"2023-07-14T11:20:14.717973+0000","last_clean":"2023-07-14T11:20:14.717973+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:14.717973+0000","last_undegraded":"2023-07-14T11:20:14.717973+0000","last_fullsized":"2023-07-14T11:20:14.717973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:59.311439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759883+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:20:15.759883+0000","last_peered":"2023-07-14T11:20:15.759883+0000","last_clean":"2023-07-14T11:20:15.759883+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:20:15.759883+0000","last_undegraded":"2023-07-14T11:20:15.759883+0000","last_fullsized":"2023-07-14T11:20:15.759883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:27.460710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726089+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:15.726089+0000","last_peered":"2023-07-14T11:20:15.726089+0000","last_clean":"2023-07-14T11:20:15.726089+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:15.726089+0000","last_undegraded":"2023-07-14T11:20:15.726089+0000","last_fullsized":"2023-07-14T11:20:15.726089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:33:58.333180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722891+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:15.722891+0000","last_peered":"2023-07-14T11:20:15.722891+0000","last_clean":"2023-07-14T11:20:15.722891+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:15.722891+0000","last_undegraded":"2023-07-14T11:20:15.722891+0000","last_fullsized":"2023-07-14T11:20:15.722891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:12.630498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.732150+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:20:15.732150+0000","last_peered":"2023-07-14T11:20:15.732150+0000","last_clean":"2023-07-14T11:20:15.732150+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:20:15.732150+0000","last_undegraded":"2023-07-14T11:20:15.732150+0000","last_fullsized":"2023-07-14T11:20:15.732150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:49.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759053+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:15.759053+0000","last_peered":"2023-07-14T11:20:15.759053+0000","last_clean":"2023-07-14T11:20:15.759053+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:15.759053+0000","last_undegraded":"2023-07-14T11:20:15.759053+0000","last_fullsized":"2023-07-14T11:20:15.759053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:20:45.016996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765761+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:20:15.765761+0000","last_peered":"2023-07-14T11:20:15.765761+0000","last_clean":"2023-07-14T11:20:15.765761+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:20:15.765761+0000","last_undegraded":"2023-07-14T11:20:15.765761+0000","last_fullsized":"2023-07-14T11:20:15.765761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:11.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763313+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:20:15.763313+0000","last_peered":"2023-07-14T11:20:15.763313+0000","last_clean":"2023-07-14T11:20:15.763313+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:20:15.763313+0000","last_undegraded":"2023-07-14T11:20:15.763313+0000","last_fullsized":"2023-07-14T11:20:15.763313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:41.779784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.687593+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:16.687593+0000","last_peered":"2023-07-14T11:20:16.687593+0000","last_clean":"2023-07-14T11:20:16.687593+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:16.687593+0000","last_undegraded":"2023-07-14T11:20:16.687593+0000","last_fullsized":"2023-07-14T11:20:16.687593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:35.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726189+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:20:15.726189+0000","last_peered":"2023-07-14T11:20:15.726189+0000","last_clean":"2023-07-14T11:20:15.726189+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:20:15.726189+0000","last_undegraded":"2023-07-14T11:20:15.726189+0000","last_fullsized":"2023-07-14T11:20:15.726189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:25.466461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723291+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:15.723291+0000","last_peered":"2023-07-14T11:20:15.723291+0000","last_clean":"2023-07-14T11:20:15.723291+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:15.723291+0000","last_undegraded":"2023-07-14T11:20:15.723291+0000","last_fullsized":"2023-07-14T11:20:15.723291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:05.698741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718779+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:14.718779+0000","last_peered":"2023-07-14T11:20:14.718779+0000","last_clean":"2023-07-14T11:20:14.718779+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:14.718779+0000","last_undegraded":"2023-07-14T11:20:14.718779+0000","last_fullsized":"2023-07-14T11:20:14.718779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:25:31.744731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758707+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:20:15.758707+0000","last_peered":"2023-07-14T11:20:15.758707+0000","last_clean":"2023-07-14T11:20:15.758707+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:20:15.758707+0000","last_undegraded":"2023-07-14T11:20:15.758707+0000","last_fullsized":"2023-07-14T11:20:15.758707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:32.059475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758890+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:20:15.758890+0000","last_peered":"2023-07-14T11:20:15.758890+0000","last_clean":"2023-07-14T11:20:15.758890+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:20:15.758890+0000","last_undegraded":"2023-07-14T11:20:15.758890+0000","last_fullsized":"2023-07-14T11:20:15.758890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:51.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717860+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:14.717860+0000","last_peered":"2023-07-14T11:20:14.717860+0000","last_clean":"2023-07-14T11:20:14.717860+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:14.717860+0000","last_undegraded":"2023-07-14T11:20:14.717860+0000","last_fullsized":"2023-07-14T11:20:14.717860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:15:42.131659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725218+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:20:15.725218+0000","last_peered":"2023-07-14T11:20:15.725218+0000","last_clean":"2023-07-14T11:20:15.725218+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:20:15.725218+0000","last_undegraded":"2023-07-14T11:20:15.725218+0000","last_fullsized":"2023-07-14T11:20:15.725218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:42.841223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T11:19:47.967804+0000","last_change":"2023-07-14T11:20:14.717622+0000","last_active":"2023-07-14T11:20:14.717622+0000","last_peered":"2023-07-14T11:19:47.967804+0000","last_clean":"2023-07-14T11:19:47.967804+0000","last_became_active":"2023-07-14T11:19:46.273092+0000","last_became_peered":"2023-07-14T11:19:46.273092+0000","last_unstale":"2023-07-14T11:19:47.967804+0000","last_undegraded":"2023-07-14T11:19:47.967804+0000","last_fullsized":"2023-07-14T11:19:47.967804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764473+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:20:15.764473+0000","last_peered":"2023-07-14T11:20:15.764473+0000","last_clean":"2023-07-14T11:20:15.764473+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:20:15.764473+0000","last_undegraded":"2023-07-14T11:20:15.764473+0000","last_fullsized":"2023-07-14T11:20:15.764473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:06.410233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729586+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:20:15.729586+0000","last_peered":"2023-07-14T11:20:15.729586+0000","last_clean":"2023-07-14T11:20:15.729586+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:20:15.729586+0000","last_undegraded":"2023-07-14T11:20:15.729586+0000","last_fullsized":"2023-07-14T11:20:15.729586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:59.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723225+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:15.723225+0000","last_peered":"2023-07-14T11:20:15.723225+0000","last_clean":"2023-07-14T11:20:15.723225+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:15.723225+0000","last_undegraded":"2023-07-14T11:20:15.723225+0000","last_fullsized":"2023-07-14T11:20:15.723225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:32.064021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726984+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.726984+0000","last_peered":"2023-07-14T11:20:15.726984+0000","last_clean":"2023-07-14T11:20:15.726984+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:20:15.726984+0000","last_undegraded":"2023-07-14T11:20:15.726984+0000","last_fullsized":"2023-07-14T11:20:15.726984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:38:06.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765912+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:20:15.765912+0000","last_peered":"2023-07-14T11:20:15.765912+0000","last_clean":"2023-07-14T11:20:15.765912+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:20:15.765912+0000","last_undegraded":"2023-07-14T11:20:15.765912+0000","last_fullsized":"2023-07-14T11:20:15.765912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:01:52.067140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723192+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:15.723192+0000","last_peered":"2023-07-14T11:20:15.723192+0000","last_clean":"2023-07-14T11:20:15.723192+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:15.723192+0000","last_undegraded":"2023-07-14T11:20:15.723192+0000","last_fullsized":"2023-07-14T11:20:15.723192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:44:48.912804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759017+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:20:15.759017+0000","last_peered":"2023-07-14T11:20:15.759017+0000","last_clean":"2023-07-14T11:20:15.759017+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:20:15.759017+0000","last_undegraded":"2023-07-14T11:20:15.759017+0000","last_fullsized":"2023-07-14T11:20:15.759017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:30:26.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:18.689589+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:18.689589+0000","last_peered":"2023-07-14T11:20:18.689589+0000","last_clean":"2023-07-14T11:20:18.689589+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:18.689589+0000","last_undegraded":"2023-07-14T11:20:18.689589+0000","last_fullsized":"2023-07-14T11:20:18.689589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:54:46.013281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726387+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:20:15.726387+0000","last_peered":"2023-07-14T11:20:15.726387+0000","last_clean":"2023-07-14T11:20:15.726387+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:20:15.726387+0000","last_undegraded":"2023-07-14T11:20:15.726387+0000","last_fullsized":"2023-07-14T11:20:15.726387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:45.551283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761576+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:20:15.761576+0000","last_peered":"2023-07-14T11:20:15.761576+0000","last_clean":"2023-07-14T11:20:15.761576+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:20:15.761576+0000","last_undegraded":"2023-07-14T11:20:15.761576+0000","last_fullsized":"2023-07-14T11:20:15.761576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:35.428242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725019+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:15.725019+0000","last_peered":"2023-07-14T11:20:15.725019+0000","last_clean":"2023-07-14T11:20:15.725019+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:15.725019+0000","last_undegraded":"2023-07-14T11:20:15.725019+0000","last_fullsized":"2023-07-14T11:20:15.725019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:42:08.666520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725963+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:20:15.725963+0000","last_peered":"2023-07-14T11:20:15.725963+0000","last_clean":"2023-07-14T11:20:15.725963+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:20:15.725963+0000","last_undegraded":"2023-07-14T11:20:15.725963+0000","last_fullsized":"2023-07-14T11:20:15.725963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:49.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759565+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:20:15.759565+0000","last_peered":"2023-07-14T11:20:15.759565+0000","last_clean":"2023-07-14T11:20:15.759565+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:20:15.759565+0000","last_undegraded":"2023-07-14T11:20:15.759565+0000","last_fullsized":"2023-07-14T11:20:15.759565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:31.340558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722788+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:15.722788+0000","last_peered":"2023-07-14T11:20:15.722788+0000","last_clean":"2023-07-14T11:20:15.722788+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:15.722788+0000","last_undegraded":"2023-07-14T11:20:15.722788+0000","last_fullsized":"2023-07-14T11:20:15.722788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:40:45.755906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726149+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:15.726149+0000","last_peered":"2023-07-14T11:20:15.726149+0000","last_clean":"2023-07-14T11:20:15.726149+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:15.726149+0000","last_undegraded":"2023-07-14T11:20:15.726149+0000","last_fullsized":"2023-07-14T11:20:15.726149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:29.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725637+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:20:15.725637+0000","last_peered":"2023-07-14T11:20:15.725637+0000","last_clean":"2023-07-14T11:20:15.725637+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:20:15.725637+0000","last_undegraded":"2023-07-14T11:20:15.725637+0000","last_fullsized":"2023-07-14T11:20:15.725637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:47:19.355328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718394+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:14.718394+0000","last_peered":"2023-07-14T11:20:14.718394+0000","last_clean":"2023-07-14T11:20:14.718394+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:14.718394+0000","last_undegraded":"2023-07-14T11:20:14.718394+0000","last_fullsized":"2023-07-14T11:20:14.718394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:22.172922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723369+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:15.723369+0000","last_peered":"2023-07-14T11:20:15.723369+0000","last_clean":"2023-07-14T11:20:15.723369+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:15.723369+0000","last_undegraded":"2023-07-14T11:20:15.723369+0000","last_fullsized":"2023-07-14T11:20:15.723369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:47.388288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761518+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:20:15.761518+0000","last_peered":"2023-07-14T11:20:15.761518+0000","last_clean":"2023-07-14T11:20:15.761518+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:20:15.761518+0000","last_undegraded":"2023-07-14T11:20:15.761518+0000","last_fullsized":"2023-07-14T11:20:15.761518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:28.833104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763956+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:15.763956+0000","last_peered":"2023-07-14T11:20:15.763956+0000","last_clean":"2023-07-14T11:20:15.763956+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:15.763956+0000","last_undegraded":"2023-07-14T11:20:15.763956+0000","last_fullsized":"2023-07-14T11:20:15.763956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:18.046365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762296+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:20:15.762296+0000","last_peered":"2023-07-14T11:20:15.762296+0000","last_clean":"2023-07-14T11:20:15.762296+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:20:15.762296+0000","last_undegraded":"2023-07-14T11:20:15.762296+0000","last_fullsized":"2023-07-14T11:20:15.762296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:04:14.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725262+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:20:15.725262+0000","last_peered":"2023-07-14T11:20:15.725262+0000","last_clean":"2023-07-14T11:20:15.725262+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:20:15.725262+0000","last_undegraded":"2023-07-14T11:20:15.725262+0000","last_fullsized":"2023-07-14T11:20:15.725262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:40:44.958728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725446+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:15.725446+0000","last_peered":"2023-07-14T11:20:15.725446+0000","last_clean":"2023-07-14T11:20:15.725446+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:15.725446+0000","last_undegraded":"2023-07-14T11:20:15.725446+0000","last_fullsized":"2023-07-14T11:20:15.725446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.980346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729830+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:20:15.729830+0000","last_peered":"2023-07-14T11:20:15.729830+0000","last_clean":"2023-07-14T11:20:15.729830+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:20:15.729830+0000","last_undegraded":"2023-07-14T11:20:15.729830+0000","last_fullsized":"2023-07-14T11:20:15.729830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:23.746186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724337+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:20:15.724337+0000","last_peered":"2023-07-14T11:20:15.724337+0000","last_clean":"2023-07-14T11:20:15.724337+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:20:15.724337+0000","last_undegraded":"2023-07-14T11:20:15.724337+0000","last_fullsized":"2023-07-14T11:20:15.724337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:32.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717656+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:14.717656+0000","last_peered":"2023-07-14T11:20:14.717656+0000","last_clean":"2023-07-14T11:20:14.717656+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:14.717656+0000","last_undegraded":"2023-07-14T11:20:14.717656+0000","last_fullsized":"2023-07-14T11:20:14.717656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:28.963610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763590+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:20:15.763590+0000","last_peered":"2023-07-14T11:20:15.763590+0000","last_clean":"2023-07-14T11:20:15.763590+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:20:15.763590+0000","last_undegraded":"2023-07-14T11:20:15.763590+0000","last_fullsized":"2023-07-14T11:20:15.763590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:06:43.200171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761544+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:15.761544+0000","last_peered":"2023-07-14T11:20:15.761544+0000","last_clean":"2023-07-14T11:20:15.761544+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:15.761544+0000","last_undegraded":"2023-07-14T11:20:15.761544+0000","last_fullsized":"2023-07-14T11:20:15.761544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:10:23.505930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:25.389747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725984+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:20:15.725984+0000","last_peered":"2023-07-14T11:20:15.725984+0000","last_clean":"2023-07-14T11:20:15.725984+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:20:15.725984+0000","last_undegraded":"2023-07-14T11:20:15.725984+0000","last_fullsized":"2023-07-14T11:20:15.725984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:28.285910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725493+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:20:15.725493+0000","last_peered":"2023-07-14T11:20:15.725493+0000","last_clean":"2023-07-14T11:20:15.725493+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:20:15.725493+0000","last_undegraded":"2023-07-14T11:20:15.725493+0000","last_fullsized":"2023-07-14T11:20:15.725493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:49.791341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761896+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:20:15.761896+0000","last_peered":"2023-07-14T11:20:15.761896+0000","last_clean":"2023-07-14T11:20:15.761896+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:20:15.761896+0000","last_undegraded":"2023-07-14T11:20:15.761896+0000","last_fullsized":"2023-07-14T11:20:15.761896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:04:06.442323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724354+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:20:15.724354+0000","last_peered":"2023-07-14T11:20:15.724354+0000","last_clean":"2023-07-14T11:20:15.724354+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:20:15.724354+0000","last_undegraded":"2023-07-14T11:20:15.724354+0000","last_fullsized":"2023-07-14T11:20:15.724354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:31.530941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717280+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:14.717280+0000","last_peered":"2023-07-14T11:20:14.717280+0000","last_clean":"2023-07-14T11:20:14.717280+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:14.717280+0000","last_undegraded":"2023-07-14T11:20:14.717280+0000","last_fullsized":"2023-07-14T11:20:14.717280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:38:55.878246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727131+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:20:15.727131+0000","last_peered":"2023-07-14T11:20:15.727131+0000","last_clean":"2023-07-14T11:20:15.727131+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:20:15.727131+0000","last_undegraded":"2023-07-14T11:20:15.727131+0000","last_fullsized":"2023-07-14T11:20:15.727131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:31:54.387366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727120+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:20:15.727120+0000","last_peered":"2023-07-14T11:20:15.727120+0000","last_clean":"2023-07-14T11:20:15.727120+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:20:15.727120+0000","last_undegraded":"2023-07-14T11:20:15.727120+0000","last_fullsized":"2023-07-14T11:20:15.727120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:15.525008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726470+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:20:15.726470+0000","last_peered":"2023-07-14T11:20:15.726470+0000","last_clean":"2023-07-14T11:20:15.726470+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:20:15.726470+0000","last_undegraded":"2023-07-14T11:20:15.726470+0000","last_fullsized":"2023-07-14T11:20:15.726470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:03.965708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729884+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:20:15.729884+0000","last_peered":"2023-07-14T11:20:15.729884+0000","last_clean":"2023-07-14T11:20:15.729884+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:20:15.729884+0000","last_undegraded":"2023-07-14T11:20:15.729884+0000","last_fullsized":"2023-07-14T11:20:15.729884+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:31.118854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759707+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:20:15.759707+0000","last_peered":"2023-07-14T11:20:15.759707+0000","last_clean":"2023-07-14T11:20:15.759707+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:20:15.759707+0000","last_undegraded":"2023-07-14T11:20:15.759707+0000","last_fullsized":"2023-07-14T11:20:15.759707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:35.715875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766177+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:20:15.766177+0000","last_peered":"2023-07-14T11:20:15.766177+0000","last_clean":"2023-07-14T11:20:15.766177+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:20:15.766177+0000","last_undegraded":"2023-07-14T11:20:15.766177+0000","last_fullsized":"2023-07-14T11:20:15.766177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:59.517623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758537+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:20:15.758537+0000","last_peered":"2023-07-14T11:20:15.758537+0000","last_clean":"2023-07-14T11:20:15.758537+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:20:15.758537+0000","last_undegraded":"2023-07-14T11:20:15.758537+0000","last_fullsized":"2023-07-14T11:20:15.758537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.641650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723412+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:15.723412+0000","last_peered":"2023-07-14T11:20:15.723412+0000","last_clean":"2023-07-14T11:20:15.723412+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:15.723412+0000","last_undegraded":"2023-07-14T11:20:15.723412+0000","last_fullsized":"2023-07-14T11:20:15.723412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:35.428375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726258+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:20:15.726258+0000","last_peered":"2023-07-14T11:20:15.726258+0000","last_clean":"2023-07-14T11:20:15.726258+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:20:15.726258+0000","last_undegraded":"2023-07-14T11:20:15.726258+0000","last_fullsized":"2023-07-14T11:20:15.726258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:41.949605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:48.091011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725591+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:20:15.725591+0000","last_peered":"2023-07-14T11:20:15.725591+0000","last_clean":"2023-07-14T11:20:15.725591+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:20:15.725591+0000","last_undegraded":"2023-07-14T11:20:15.725591+0000","last_fullsized":"2023-07-14T11:20:15.725591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:15.490002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726260+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:20:15.726260+0000","last_peered":"2023-07-14T11:20:15.726260+0000","last_clean":"2023-07-14T11:20:15.726260+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:20:15.726260+0000","last_undegraded":"2023-07-14T11:20:15.726260+0000","last_fullsized":"2023-07-14T11:20:15.726260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:53:08.815873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726407+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:20:15.726407+0000","last_peered":"2023-07-14T11:20:15.726407+0000","last_clean":"2023-07-14T11:20:15.726407+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:20:15.726407+0000","last_undegraded":"2023-07-14T11:20:15.726407+0000","last_fullsized":"2023-07-14T11:20:15.726407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:08.673713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723496+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:15.723496+0000","last_peered":"2023-07-14T11:20:15.723496+0000","last_clean":"2023-07-14T11:20:15.723496+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:15.723496+0000","last_undegraded":"2023-07-14T11:20:15.723496+0000","last_fullsized":"2023-07-14T11:20:15.723496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:41:54.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726094+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:20:15.726094+0000","last_peered":"2023-07-14T11:20:15.726094+0000","last_clean":"2023-07-14T11:20:15.726094+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:20:15.726094+0000","last_undegraded":"2023-07-14T11:20:15.726094+0000","last_fullsized":"2023-07-14T11:20:15.726094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:36:46.120312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725900+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:20:15.725900+0000","last_peered":"2023-07-14T11:20:15.725900+0000","last_clean":"2023-07-14T11:20:15.725900+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:20:15.725900+0000","last_undegraded":"2023-07-14T11:20:15.725900+0000","last_fullsized":"2023-07-14T11:20:15.725900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:29.569291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762498+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:20:15.762498+0000","last_peered":"2023-07-14T11:20:15.762498+0000","last_clean":"2023-07-14T11:20:15.762498+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:20:15.762498+0000","last_undegraded":"2023-07-14T11:20:15.762498+0000","last_fullsized":"2023-07-14T11:20:15.762498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:46.020055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-14T11:19:42.068794+0000","last_change":"2023-07-14T11:20:15.683458+0000","last_active":"2023-07-14T11:19:41.086153+0000","last_peered":"2023-07-14T11:19:41.086153+0000","last_clean":"2023-07-14T11:19:41.086153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:19:42.068794+0000","last_undegraded":"2023-07-14T11:19:42.068794+0000","last_fullsized":"2023-07-14T11:19:42.068794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761928+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:20:15.761928+0000","last_peered":"2023-07-14T11:20:15.761928+0000","last_clean":"2023-07-14T11:20:15.761928+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:20:15.761928+0000","last_undegraded":"2023-07-14T11:20:15.761928+0000","last_fullsized":"2023-07-14T11:20:15.761928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:50:15.332355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723311+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:15.723311+0000","last_peered":"2023-07-14T11:20:15.723311+0000","last_clean":"2023-07-14T11:20:15.723311+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:15.723311+0000","last_undegraded":"2023-07-14T11:20:15.723311+0000","last_fullsized":"2023-07-14T11:20:15.723311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:13.493038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725897+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:20:15.725897+0000","last_peered":"2023-07-14T11:20:15.725897+0000","last_clean":"2023-07-14T11:20:15.725897+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:20:15.725897+0000","last_undegraded":"2023-07-14T11:20:15.725897+0000","last_fullsized":"2023-07-14T11:20:15.725897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:20.934793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46642,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":58,"num_read_kb":80,"num_write":142,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":139926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":490869,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.62},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.80100000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.84599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.875}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1105920,"data_stored":1121217,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.377}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.501},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.63500000000000001}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.751},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.86399999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":495201,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":1.012,"15min":1.012},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.748},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.997,"15min":1.0489999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":0.999,"15min":0.999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0880000000000001,"5min":1.264,"15min":1.264},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.56499999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.94699999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.999,"5min":1.073,"15min":1.073},"last":0.999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.84099999999999997,"15min":0.86299999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93100000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.872,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.04},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0409999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.0169999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.151,"15min":1.2090000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99299999999999999,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90100000000000002,"15min":0.91700000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.081,"5min":1.171,"15min":1.171},"last":1.081}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.96799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":481467,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93000000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0129999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1086116,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.373},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.5},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.442}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68999999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.131,"15min":1.131},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486008,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.087}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.04}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.89800000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":515148,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.84799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.623},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.752}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":487396,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.623},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.745}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.80700000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.85899999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.875}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":520790,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73799999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82599999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72499999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486475,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.505}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.63},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.754}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.59999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96699999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.98599999999999999,"15min":1.0169999999999999},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":4.157,"15min":4.157},"last":0.73799999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490883,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.374}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.498}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82499999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.438}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.40100000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47199999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.623}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1081752,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.377},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.53500000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.626}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.747}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":44430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:21.461 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:21.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:21 smithi182 ceph-mon[129366]: pgmap v293: 129 pgs: 2 peering, 127 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 16 op/s 2023-07-14T11:20:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.685+0000 7fac363ff700 1 -- 172.21.15.161:0/2080858356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac30072f70 msgr2=0x7fac300733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.685+0000 7fac363ff700 1 --2- 172.21.15.161:0/2080858356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac30072f70 0x7fac300733e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac240098f0 tx=0x7fac24009c00 comp rx=0 tx=0).stop 2023-07-14T11:20:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.686+0000 7fac363ff700 1 -- 172.21.15.161:0/2080858356 shutdown_connections 2023-07-14T11:20:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.686+0000 7fac363ff700 1 --2- 172.21.15.161:0/2080858356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac3010c0d0 0x7fac3010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.686+0000 7fac363ff700 1 --2- 172.21.15.161:0/2080858356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac30074550 0x7fac300729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.686+0000 7fac363ff700 1 --2- 172.21.15.161:0/2080858356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac30072f70 0x7fac300733e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.686+0000 7fac363ff700 1 -- 172.21.15.161:0/2080858356 >> 172.21.15.161:0/2080858356 conn(0x7fac300fb0e0 msgr2=0x7fac300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.686+0000 7fac363ff700 1 -- 172.21.15.161:0/2080858356 shutdown_connections 2023-07-14T11:20:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.686+0000 7fac363ff700 1 -- 172.21.15.161:0/2080858356 wait complete. 2023-07-14T11:20:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 Processor -- start 2023-07-14T11:20:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 -- start start 2023-07-14T11:20:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac30072f70 0x7fac3019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac30074550 0x7fac3019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac3010c0d0 0x7fac301a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac301a3a20 con 0x7fac3010c0d0 2023-07-14T11:20:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac301a3b60 con 0x7fac30074550 2023-07-14T11:20:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac363ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac301a3cf0 con 0x7fac30072f70 2023-07-14T11:20:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac3499c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac3010c0d0 0x7fac301a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac2ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac30072f70 0x7fac3019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac2ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac30072f70 0x7fac3019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:55040/0 (socket says 172.21.15.161:55040) 2023-07-14T11:20:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac3499c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac3010c0d0 0x7fac301a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37974/0 (socket says 172.21.15.161:37974) 2023-07-14T11:20:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.687+0000 7fac2ffff700 1 -- 172.21.15.161:0/415596759 learned_addr learned my addr 172.21.15.161:0/415596759 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2f7fe700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac30074550 0x7fac3019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2ffff700 1 -- 172.21.15.161:0/415596759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac30074550 msgr2=0x7fac3019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2ffff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac30074550 0x7fac3019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2ffff700 1 -- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac3010c0d0 msgr2=0x7fac301a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2ffff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac3010c0d0 0x7fac301a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2ffff700 1 -- 172.21.15.161:0/415596759 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac24009520 con 0x7fac30072f70 2023-07-14T11:20:21.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac3499c700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac3010c0d0 0x7fac301a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:21.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2ffff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac30072f70 0x7fac3019bea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e9f0 tx=0x7fac2000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2d7fa700 1 -- 172.21.15.161:0/415596759 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20009690 con 0x7fac30072f70 2023-07-14T11:20:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac2d7fa700 1 -- 172.21.15.161:0/415596759 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac2000ce80 con 0x7fac30072f70 2023-07-14T11:20:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.688+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac301a3fd0 con 0x7fac30072f70 2023-07-14T11:20:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.689+0000 7fac2d7fa700 1 -- 172.21.15.161:0/415596759 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20018420 con 0x7fac30072f70 2023-07-14T11:20:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.689+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac301a43f0 con 0x7fac30072f70 2023-07-14T11:20:21.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.690+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac30101c20 con 0x7fac30072f70 2023-07-14T11:20:21.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.692+0000 7fac2d7fa700 1 -- 172.21.15.161:0/415596759 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fac2000c9c0 con 0x7fac30072f70 2023-07-14T11:20:21.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.693+0000 7fac2d7fa700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fac180a1220 0x7fac180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:21.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.693+0000 7fac2d7fa700 1 -- 172.21.15.161:0/415596759 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fac20014070 con 0x7fac30072f70 2023-07-14T11:20:21.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.693+0000 7fac2f7fe700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fac180a1220 0x7fac180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:21.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.694+0000 7fac2f7fe700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fac180a1220 0x7fac180a36d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac24009fd0 tx=0x7fac24019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:21.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.695+0000 7fac2d7fa700 1 -- 172.21.15.161:0/415596759 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fac20096110 con 0x7fac30072f70 2023-07-14T11:20:21.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.890+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac30060950 con 0x7fac180a1220 2023-07-14T11:20:21.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.902+0000 7fac2d7fa700 1 -- 172.21.15.161:0/415596759 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422716 (secure 0 0 0) 0x7fac30060950 con 0x7fac180a1220 2023-07-14T11:20:21.910 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fac180a1220 msgr2=0x7fac180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fac180a1220 0x7fac180a36d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac24009fd0 tx=0x7fac24019040 comp rx=0 tx=0).stop 2023-07-14T11:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac30072f70 msgr2=0x7fac3019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac30072f70 0x7fac3019bea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e9f0 tx=0x7fac2000ed00 comp rx=0 tx=0).stop 2023-07-14T11:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 shutdown_connections 2023-07-14T11:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac3010c0d0 0x7fac301a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fac180a1220 0x7fac180a36d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac30072f70 0x7fac3019bea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 --2- 172.21.15.161:0/415596759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac30074550 0x7fac3019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 >> 172.21.15.161:0/415596759 conn(0x7fac300fb0e0 msgr2=0x7fac300fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 shutdown_connections 2023-07-14T11:20:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:21.907+0000 7fac363ff700 1 -- 172.21.15.161:0/415596759 wait complete. 2023-07-14T11:20:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:21.930 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-07-14T11:20:20.104895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637256,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":194,"num_read_kb":199,"num_write":325,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264,"ondisk_log_size":264,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335276,"kb_used_data":11852,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606548,"statfs":{"total":1151940427776,"available":1151597105152,"internally_reserved":0,"allocated":12136448,"data_stored":7743322,"data_compressed":309735,"data_compressed_allocated":417792,"data_compressed_original":2334720,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42206,"num_objects":33,"num_object_clones":0,"num_object_copies":101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":58,"num_read_kb":80,"num_write":76,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002159"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726364+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:20:15.726364+0000","last_peered":"2023-07-14T11:20:15.726364+0000","last_clean":"2023-07-14T11:20:15.726364+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:20:15.726364+0000","last_undegraded":"2023-07-14T11:20:15.726364+0000","last_fullsized":"2023-07-14T11:20:15.726364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:25.074131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726699+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:20:15.726699+0000","last_peered":"2023-07-14T11:20:15.726699+0000","last_clean":"2023-07-14T11:20:15.726699+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:20:15.726699+0000","last_undegraded":"2023-07-14T11:20:15.726699+0000","last_fullsized":"2023-07-14T11:20:15.726699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:41.854592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726554+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:20:15.726554+0000","last_peered":"2023-07-14T11:20:15.726554+0000","last_clean":"2023-07-14T11:20:15.726554+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:20:15.726554+0000","last_undegraded":"2023-07-14T11:20:15.726554+0000","last_fullsized":"2023-07-14T11:20:15.726554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:37.342912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726181+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:20:15.726181+0000","last_peered":"2023-07-14T11:20:15.726181+0000","last_clean":"2023-07-14T11:20:15.726181+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:20:15.726181+0000","last_undegraded":"2023-07-14T11:20:15.726181+0000","last_fullsized":"2023-07-14T11:20:15.726181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:46.208987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717211+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:14.717211+0000","last_peered":"2023-07-14T11:20:14.717211+0000","last_clean":"2023-07-14T11:20:14.717211+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:14.717211+0000","last_undegraded":"2023-07-14T11:20:14.717211+0000","last_fullsized":"2023-07-14T11:20:14.717211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:45:02.418574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765846+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:20:15.765846+0000","last_peered":"2023-07-14T11:20:15.765846+0000","last_clean":"2023-07-14T11:20:15.765846+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:20:15.765846+0000","last_undegraded":"2023-07-14T11:20:15.765846+0000","last_fullsized":"2023-07-14T11:20:15.765846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:31:09.497877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725992+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:20:15.725992+0000","last_peered":"2023-07-14T11:20:15.725992+0000","last_clean":"2023-07-14T11:20:15.725992+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:20:15.725992+0000","last_undegraded":"2023-07-14T11:20:15.725992+0000","last_fullsized":"2023-07-14T11:20:15.725992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:18.893918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761828+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:20:15.761828+0000","last_peered":"2023-07-14T11:20:15.761828+0000","last_clean":"2023-07-14T11:20:15.761828+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:20:15.761828+0000","last_undegraded":"2023-07-14T11:20:15.761828+0000","last_fullsized":"2023-07-14T11:20:15.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730008+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:20:15.730008+0000","last_peered":"2023-07-14T11:20:15.730008+0000","last_clean":"2023-07-14T11:20:15.730008+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:20:15.730008+0000","last_undegraded":"2023-07-14T11:20:15.730008+0000","last_fullsized":"2023-07-14T11:20:15.730008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:25:58.920431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726577+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:20:15.726577+0000","last_peered":"2023-07-14T11:20:15.726577+0000","last_clean":"2023-07-14T11:20:15.726577+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:20:15.726577+0000","last_undegraded":"2023-07-14T11:20:15.726577+0000","last_fullsized":"2023-07-14T11:20:15.726577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:29.172086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726169+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:20:15.726169+0000","last_peered":"2023-07-14T11:20:15.726169+0000","last_clean":"2023-07-14T11:20:15.726169+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:20:15.726169+0000","last_undegraded":"2023-07-14T11:20:15.726169+0000","last_fullsized":"2023-07-14T11:20:15.726169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:29.746118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723416+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:15.723416+0000","last_peered":"2023-07-14T11:20:15.723416+0000","last_clean":"2023-07-14T11:20:15.723416+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:15.723416+0000","last_undegraded":"2023-07-14T11:20:15.723416+0000","last_fullsized":"2023-07-14T11:20:15.723416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:57.801423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725956+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:20:15.725956+0000","last_peered":"2023-07-14T11:20:15.725956+0000","last_clean":"2023-07-14T11:20:15.725956+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:20:15.725956+0000","last_undegraded":"2023-07-14T11:20:15.725956+0000","last_fullsized":"2023-07-14T11:20:15.725956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:01.673209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726027+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:20:15.726027+0000","last_peered":"2023-07-14T11:20:15.726027+0000","last_clean":"2023-07-14T11:20:15.726027+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:20:15.726027+0000","last_undegraded":"2023-07-14T11:20:15.726027+0000","last_fullsized":"2023-07-14T11:20:15.726027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:10.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726533+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:20:15.726533+0000","last_peered":"2023-07-14T11:20:15.726533+0000","last_clean":"2023-07-14T11:20:15.726533+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:20:15.726533+0000","last_undegraded":"2023-07-14T11:20:15.726533+0000","last_fullsized":"2023-07-14T11:20:15.726533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:33.733562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763940+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:20:15.763940+0000","last_peered":"2023-07-14T11:20:15.763940+0000","last_clean":"2023-07-14T11:20:15.763940+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:20:15.763940+0000","last_undegraded":"2023-07-14T11:20:15.763940+0000","last_fullsized":"2023-07-14T11:20:15.763940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:28:49.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725800+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:20:15.725800+0000","last_peered":"2023-07-14T11:20:15.725800+0000","last_clean":"2023-07-14T11:20:15.725800+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:20:15.725800+0000","last_undegraded":"2023-07-14T11:20:15.725800+0000","last_fullsized":"2023-07-14T11:20:15.725800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:30.183181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726357+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:20:15.726357+0000","last_peered":"2023-07-14T11:20:15.726357+0000","last_clean":"2023-07-14T11:20:15.726357+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:20:15.726357+0000","last_undegraded":"2023-07-14T11:20:15.726357+0000","last_fullsized":"2023-07-14T11:20:15.726357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:13.424624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763774+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:15.763774+0000","last_peered":"2023-07-14T11:20:15.763774+0000","last_clean":"2023-07-14T11:20:15.763774+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:15.763774+0000","last_undegraded":"2023-07-14T11:20:15.763774+0000","last_fullsized":"2023-07-14T11:20:15.763774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:54.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762089+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:20:15.762089+0000","last_peered":"2023-07-14T11:20:15.762089+0000","last_clean":"2023-07-14T11:20:15.762089+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:20:15.762089+0000","last_undegraded":"2023-07-14T11:20:15.762089+0000","last_fullsized":"2023-07-14T11:20:15.762089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:06.259730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724677+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:20:15.724677+0000","last_peered":"2023-07-14T11:20:15.724677+0000","last_clean":"2023-07-14T11:20:15.724677+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:20:15.724677+0000","last_undegraded":"2023-07-14T11:20:15.724677+0000","last_fullsized":"2023-07-14T11:20:15.724677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:07.076184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762219+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:20:15.762219+0000","last_peered":"2023-07-14T11:20:15.762219+0000","last_clean":"2023-07-14T11:20:15.762219+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:20:15.762219+0000","last_undegraded":"2023-07-14T11:20:15.762219+0000","last_fullsized":"2023-07-14T11:20:15.762219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:30.071523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726815+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:20:15.726815+0000","last_peered":"2023-07-14T11:20:15.726815+0000","last_clean":"2023-07-14T11:20:15.726815+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:20:15.726815+0000","last_undegraded":"2023-07-14T11:20:15.726815+0000","last_fullsized":"2023-07-14T11:20:15.726815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:41:41.007835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724865+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:20:15.724865+0000","last_peered":"2023-07-14T11:20:15.724865+0000","last_clean":"2023-07-14T11:20:15.724865+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:20:15.724865+0000","last_undegraded":"2023-07-14T11:20:15.724865+0000","last_fullsized":"2023-07-14T11:20:15.724865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:56:28.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718300+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:14.718300+0000","last_peered":"2023-07-14T11:20:14.718300+0000","last_clean":"2023-07-14T11:20:14.718300+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:14.718300+0000","last_undegraded":"2023-07-14T11:20:14.718300+0000","last_fullsized":"2023-07-14T11:20:14.718300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:53:43.048114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726665+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:20:15.726665+0000","last_peered":"2023-07-14T11:20:15.726665+0000","last_clean":"2023-07-14T11:20:15.726665+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:20:15.726665+0000","last_undegraded":"2023-07-14T11:20:15.726665+0000","last_fullsized":"2023-07-14T11:20:15.726665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:39.791849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717638+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:14.717638+0000","last_peered":"2023-07-14T11:20:14.717638+0000","last_clean":"2023-07-14T11:20:14.717638+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:14.717638+0000","last_undegraded":"2023-07-14T11:20:14.717638+0000","last_fullsized":"2023-07-14T11:20:14.717638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:20.321448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759435+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:20:15.759435+0000","last_peered":"2023-07-14T11:20:15.759435+0000","last_clean":"2023-07-14T11:20:15.759435+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:20:15.759435+0000","last_undegraded":"2023-07-14T11:20:15.759435+0000","last_fullsized":"2023-07-14T11:20:15.759435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:51.343442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725567+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:20:15.725567+0000","last_peered":"2023-07-14T11:20:15.725567+0000","last_clean":"2023-07-14T11:20:15.725567+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:20:15.725567+0000","last_undegraded":"2023-07-14T11:20:15.725567+0000","last_fullsized":"2023-07-14T11:20:15.725567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:50.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.686170+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:16.686170+0000","last_peered":"2023-07-14T11:20:16.686170+0000","last_clean":"2023-07-14T11:20:16.686170+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:16.686170+0000","last_undegraded":"2023-07-14T11:20:16.686170+0000","last_fullsized":"2023-07-14T11:20:16.686170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:41.232654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718422+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:14.718422+0000","last_peered":"2023-07-14T11:20:14.718422+0000","last_clean":"2023-07-14T11:20:14.718422+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:14.718422+0000","last_undegraded":"2023-07-14T11:20:14.718422+0000","last_fullsized":"2023-07-14T11:20:14.718422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:25.009048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766231+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:20:15.766231+0000","last_peered":"2023-07-14T11:20:15.766231+0000","last_clean":"2023-07-14T11:20:15.766231+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:20:15.766231+0000","last_undegraded":"2023-07-14T11:20:15.766231+0000","last_fullsized":"2023-07-14T11:20:15.766231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:16:19.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730104+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:15.730104+0000","last_peered":"2023-07-14T11:20:15.730104+0000","last_clean":"2023-07-14T11:20:15.730104+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:15.730104+0000","last_undegraded":"2023-07-14T11:20:15.730104+0000","last_fullsized":"2023-07-14T11:20:15.730104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:54:17.106221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759228+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:20:15.759228+0000","last_peered":"2023-07-14T11:20:15.759228+0000","last_clean":"2023-07-14T11:20:15.759228+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:20:15.759228+0000","last_undegraded":"2023-07-14T11:20:15.759228+0000","last_fullsized":"2023-07-14T11:20:15.759228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:47.650473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726249+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:20:15.726249+0000","last_peered":"2023-07-14T11:20:15.726249+0000","last_clean":"2023-07-14T11:20:15.726249+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:20:15.726249+0000","last_undegraded":"2023-07-14T11:20:15.726249+0000","last_fullsized":"2023-07-14T11:20:15.726249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:10.265432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722943+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:15.722943+0000","last_peered":"2023-07-14T11:20:15.722943+0000","last_clean":"2023-07-14T11:20:15.722943+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:15.722943+0000","last_undegraded":"2023-07-14T11:20:15.722943+0000","last_fullsized":"2023-07-14T11:20:15.722943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:48.217979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761668+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:20:15.761668+0000","last_peered":"2023-07-14T11:20:15.761668+0000","last_clean":"2023-07-14T11:20:15.761668+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:20:15.761668+0000","last_undegraded":"2023-07-14T11:20:15.761668+0000","last_fullsized":"2023-07-14T11:20:15.761668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.096155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726651+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:20:15.726651+0000","last_peered":"2023-07-14T11:20:15.726651+0000","last_clean":"2023-07-14T11:20:15.726651+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:20:15.726651+0000","last_undegraded":"2023-07-14T11:20:15.726651+0000","last_fullsized":"2023-07-14T11:20:15.726651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:36.842168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758847+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:20:15.758847+0000","last_peered":"2023-07-14T11:20:15.758847+0000","last_clean":"2023-07-14T11:20:15.758847+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:20:15.758847+0000","last_undegraded":"2023-07-14T11:20:15.758847+0000","last_fullsized":"2023-07-14T11:20:15.758847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:38.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763190+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:20:15.763190+0000","last_peered":"2023-07-14T11:20:15.763190+0000","last_clean":"2023-07-14T11:20:15.763190+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:20:15.763190+0000","last_undegraded":"2023-07-14T11:20:15.763190+0000","last_fullsized":"2023-07-14T11:20:15.763190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:23.015078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761792+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:20:15.761792+0000","last_peered":"2023-07-14T11:20:15.761792+0000","last_clean":"2023-07-14T11:20:15.761792+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:20:15.761792+0000","last_undegraded":"2023-07-14T11:20:15.761792+0000","last_fullsized":"2023-07-14T11:20:15.761792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.645359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729832+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:15.729832+0000","last_peered":"2023-07-14T11:20:15.729832+0000","last_clean":"2023-07-14T11:20:15.729832+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:15.729832+0000","last_undegraded":"2023-07-14T11:20:15.729832+0000","last_fullsized":"2023-07-14T11:20:15.729832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.137485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763325+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:20:15.763325+0000","last_peered":"2023-07-14T11:20:15.763325+0000","last_clean":"2023-07-14T11:20:15.763325+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:20:15.763325+0000","last_undegraded":"2023-07-14T11:20:15.763325+0000","last_fullsized":"2023-07-14T11:20:15.763325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:28.186030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726329+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:15.726329+0000","last_peered":"2023-07-14T11:20:15.726329+0000","last_clean":"2023-07-14T11:20:15.726329+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:15.726329+0000","last_undegraded":"2023-07-14T11:20:15.726329+0000","last_fullsized":"2023-07-14T11:20:15.726329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:38.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726931+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:20:15.726931+0000","last_peered":"2023-07-14T11:20:15.726931+0000","last_clean":"2023-07-14T11:20:15.726931+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:20:15.726931+0000","last_undegraded":"2023-07-14T11:20:15.726931+0000","last_fullsized":"2023-07-14T11:20:15.726931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:58.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725362+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:20:15.725362+0000","last_peered":"2023-07-14T11:20:15.725362+0000","last_clean":"2023-07-14T11:20:15.725362+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:20:15.725362+0000","last_undegraded":"2023-07-14T11:20:15.725362+0000","last_fullsized":"2023-07-14T11:20:15.725362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:29.343315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759436+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:15.759436+0000","last_peered":"2023-07-14T11:20:15.759436+0000","last_clean":"2023-07-14T11:20:15.759436+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:15.759436+0000","last_undegraded":"2023-07-14T11:20:15.759436+0000","last_fullsized":"2023-07-14T11:20:15.759436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:17:07.579078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762929+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:20:15.762929+0000","last_peered":"2023-07-14T11:20:15.762929+0000","last_clean":"2023-07-14T11:20:15.762929+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:20:15.762929+0000","last_undegraded":"2023-07-14T11:20:15.762929+0000","last_fullsized":"2023-07-14T11:20:15.762929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:41.993201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'23","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.690487+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:16.690487+0000","last_peered":"2023-07-14T11:20:16.690487+0000","last_clean":"2023-07-14T11:20:16.690487+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:16.690487+0000","last_undegraded":"2023-07-14T11:20:16.690487+0000","last_fullsized":"2023-07-14T11:20:16.690487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:08:09.309573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43128,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":49,"num_write":24,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717973+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:14.717973+0000","last_peered":"2023-07-14T11:20:14.717973+0000","last_clean":"2023-07-14T11:20:14.717973+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:14.717973+0000","last_undegraded":"2023-07-14T11:20:14.717973+0000","last_fullsized":"2023-07-14T11:20:14.717973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:59.311439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759883+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:20:15.759883+0000","last_peered":"2023-07-14T11:20:15.759883+0000","last_clean":"2023-07-14T11:20:15.759883+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:20:15.759883+0000","last_undegraded":"2023-07-14T11:20:15.759883+0000","last_fullsized":"2023-07-14T11:20:15.759883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:27.460710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726089+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:15.726089+0000","last_peered":"2023-07-14T11:20:15.726089+0000","last_clean":"2023-07-14T11:20:15.726089+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:15.726089+0000","last_undegraded":"2023-07-14T11:20:15.726089+0000","last_fullsized":"2023-07-14T11:20:15.726089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:33:58.333180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722891+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:15.722891+0000","last_peered":"2023-07-14T11:20:15.722891+0000","last_clean":"2023-07-14T11:20:15.722891+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:15.722891+0000","last_undegraded":"2023-07-14T11:20:15.722891+0000","last_fullsized":"2023-07-14T11:20:15.722891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:12.630498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.732150+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:20:15.732150+0000","last_peered":"2023-07-14T11:20:15.732150+0000","last_clean":"2023-07-14T11:20:15.732150+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:20:15.732150+0000","last_undegraded":"2023-07-14T11:20:15.732150+0000","last_fullsized":"2023-07-14T11:20:15.732150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:49.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759053+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:15.759053+0000","last_peered":"2023-07-14T11:20:15.759053+0000","last_clean":"2023-07-14T11:20:15.759053+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:15.759053+0000","last_undegraded":"2023-07-14T11:20:15.759053+0000","last_fullsized":"2023-07-14T11:20:15.759053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:20:45.016996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765761+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:20:15.765761+0000","last_peered":"2023-07-14T11:20:15.765761+0000","last_clean":"2023-07-14T11:20:15.765761+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:20:15.765761+0000","last_undegraded":"2023-07-14T11:20:15.765761+0000","last_fullsized":"2023-07-14T11:20:15.765761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:11.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763313+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:20:15.763313+0000","last_peered":"2023-07-14T11:20:15.763313+0000","last_clean":"2023-07-14T11:20:15.763313+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:20:15.763313+0000","last_undegraded":"2023-07-14T11:20:15.763313+0000","last_fullsized":"2023-07-14T11:20:15.763313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:41.779784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.687593+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:16.687593+0000","last_peered":"2023-07-14T11:20:16.687593+0000","last_clean":"2023-07-14T11:20:16.687593+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:16.687593+0000","last_undegraded":"2023-07-14T11:20:16.687593+0000","last_fullsized":"2023-07-14T11:20:16.687593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:35.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726189+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:20:15.726189+0000","last_peered":"2023-07-14T11:20:15.726189+0000","last_clean":"2023-07-14T11:20:15.726189+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:20:15.726189+0000","last_undegraded":"2023-07-14T11:20:15.726189+0000","last_fullsized":"2023-07-14T11:20:15.726189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:25.466461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723291+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:15.723291+0000","last_peered":"2023-07-14T11:20:15.723291+0000","last_clean":"2023-07-14T11:20:15.723291+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:15.723291+0000","last_undegraded":"2023-07-14T11:20:15.723291+0000","last_fullsized":"2023-07-14T11:20:15.723291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:05.698741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718779+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:14.718779+0000","last_peered":"2023-07-14T11:20:14.718779+0000","last_clean":"2023-07-14T11:20:14.718779+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:14.718779+0000","last_undegraded":"2023-07-14T11:20:14.718779+0000","last_fullsized":"2023-07-14T11:20:14.718779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:25:31.744731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758707+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:20:15.758707+0000","last_peered":"2023-07-14T11:20:15.758707+0000","last_clean":"2023-07-14T11:20:15.758707+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:20:15.758707+0000","last_undegraded":"2023-07-14T11:20:15.758707+0000","last_fullsized":"2023-07-14T11:20:15.758707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:32.059475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758890+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:20:15.758890+0000","last_peered":"2023-07-14T11:20:15.758890+0000","last_clean":"2023-07-14T11:20:15.758890+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:20:15.758890+0000","last_undegraded":"2023-07-14T11:20:15.758890+0000","last_fullsized":"2023-07-14T11:20:15.758890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:51.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717860+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:14.717860+0000","last_peered":"2023-07-14T11:20:14.717860+0000","last_clean":"2023-07-14T11:20:14.717860+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:14.717860+0000","last_undegraded":"2023-07-14T11:20:14.717860+0000","last_fullsized":"2023-07-14T11:20:14.717860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:15:42.131659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725218+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:20:15.725218+0000","last_peered":"2023-07-14T11:20:15.725218+0000","last_clean":"2023-07-14T11:20:15.725218+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:20:15.725218+0000","last_undegraded":"2023-07-14T11:20:15.725218+0000","last_fullsized":"2023-07-14T11:20:15.725218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:42.841223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T11:19:47.967804+0000","last_change":"2023-07-14T11:20:14.717622+0000","last_active":"2023-07-14T11:20:14.717622+0000","last_peered":"2023-07-14T11:19:47.967804+0000","last_clean":"2023-07-14T11:19:47.967804+0000","last_became_active":"2023-07-14T11:19:46.273092+0000","last_became_peered":"2023-07-14T11:19:46.273092+0000","last_unstale":"2023-07-14T11:19:47.967804+0000","last_undegraded":"2023-07-14T11:19:47.967804+0000","last_fullsized":"2023-07-14T11:19:47.967804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764473+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:20:15.764473+0000","last_peered":"2023-07-14T11:20:15.764473+0000","last_clean":"2023-07-14T11:20:15.764473+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:20:15.764473+0000","last_undegraded":"2023-07-14T11:20:15.764473+0000","last_fullsized":"2023-07-14T11:20:15.764473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:06.410233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729586+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:20:15.729586+0000","last_peered":"2023-07-14T11:20:15.729586+0000","last_clean":"2023-07-14T11:20:15.729586+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:20:15.729586+0000","last_undegraded":"2023-07-14T11:20:15.729586+0000","last_fullsized":"2023-07-14T11:20:15.729586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:59.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723225+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:15.723225+0000","last_peered":"2023-07-14T11:20:15.723225+0000","last_clean":"2023-07-14T11:20:15.723225+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:15.723225+0000","last_undegraded":"2023-07-14T11:20:15.723225+0000","last_fullsized":"2023-07-14T11:20:15.723225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:32.064021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726984+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.726984+0000","last_peered":"2023-07-14T11:20:15.726984+0000","last_clean":"2023-07-14T11:20:15.726984+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:20:15.726984+0000","last_undegraded":"2023-07-14T11:20:15.726984+0000","last_fullsized":"2023-07-14T11:20:15.726984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:38:06.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765912+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:20:15.765912+0000","last_peered":"2023-07-14T11:20:15.765912+0000","last_clean":"2023-07-14T11:20:15.765912+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:20:15.765912+0000","last_undegraded":"2023-07-14T11:20:15.765912+0000","last_fullsized":"2023-07-14T11:20:15.765912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:01:52.067140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723192+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:15.723192+0000","last_peered":"2023-07-14T11:20:15.723192+0000","last_clean":"2023-07-14T11:20:15.723192+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:15.723192+0000","last_undegraded":"2023-07-14T11:20:15.723192+0000","last_fullsized":"2023-07-14T11:20:15.723192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:44:48.912804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759017+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:20:15.759017+0000","last_peered":"2023-07-14T11:20:15.759017+0000","last_clean":"2023-07-14T11:20:15.759017+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:20:15.759017+0000","last_undegraded":"2023-07-14T11:20:15.759017+0000","last_fullsized":"2023-07-14T11:20:15.759017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:30:26.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:18.689589+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:18.689589+0000","last_peered":"2023-07-14T11:20:18.689589+0000","last_clean":"2023-07-14T11:20:18.689589+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:18.689589+0000","last_undegraded":"2023-07-14T11:20:18.689589+0000","last_fullsized":"2023-07-14T11:20:18.689589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:54:46.013281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726387+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:20:15.726387+0000","last_peered":"2023-07-14T11:20:15.726387+0000","last_clean":"2023-07-14T11:20:15.726387+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:20:15.726387+0000","last_undegraded":"2023-07-14T11:20:15.726387+0000","last_fullsized":"2023-07-14T11:20:15.726387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:45.551283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761576+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:20:15.761576+0000","last_peered":"2023-07-14T11:20:15.761576+0000","last_clean":"2023-07-14T11:20:15.761576+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:20:15.761576+0000","last_undegraded":"2023-07-14T11:20:15.761576+0000","last_fullsized":"2023-07-14T11:20:15.761576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:35.428242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725019+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:15.725019+0000","last_peered":"2023-07-14T11:20:15.725019+0000","last_clean":"2023-07-14T11:20:15.725019+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:15.725019+0000","last_undegraded":"2023-07-14T11:20:15.725019+0000","last_fullsized":"2023-07-14T11:20:15.725019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:42:08.666520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725963+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:20:15.725963+0000","last_peered":"2023-07-14T11:20:15.725963+0000","last_clean":"2023-07-14T11:20:15.725963+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:20:15.725963+0000","last_undegraded":"2023-07-14T11:20:15.725963+0000","last_fullsized":"2023-07-14T11:20:15.725963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:49.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759565+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:20:15.759565+0000","last_peered":"2023-07-14T11:20:15.759565+0000","last_clean":"2023-07-14T11:20:15.759565+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:20:15.759565+0000","last_undegraded":"2023-07-14T11:20:15.759565+0000","last_fullsized":"2023-07-14T11:20:15.759565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:31.340558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722788+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:15.722788+0000","last_peered":"2023-07-14T11:20:15.722788+0000","last_clean":"2023-07-14T11:20:15.722788+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:15.722788+0000","last_undegraded":"2023-07-14T11:20:15.722788+0000","last_fullsized":"2023-07-14T11:20:15.722788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:40:45.755906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726149+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:15.726149+0000","last_peered":"2023-07-14T11:20:15.726149+0000","last_clean":"2023-07-14T11:20:15.726149+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:15.726149+0000","last_undegraded":"2023-07-14T11:20:15.726149+0000","last_fullsized":"2023-07-14T11:20:15.726149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:29.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725637+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:20:15.725637+0000","last_peered":"2023-07-14T11:20:15.725637+0000","last_clean":"2023-07-14T11:20:15.725637+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:20:15.725637+0000","last_undegraded":"2023-07-14T11:20:15.725637+0000","last_fullsized":"2023-07-14T11:20:15.725637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:47:19.355328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.718394+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:14.718394+0000","last_peered":"2023-07-14T11:20:14.718394+0000","last_clean":"2023-07-14T11:20:14.718394+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:14.718394+0000","last_undegraded":"2023-07-14T11:20:14.718394+0000","last_fullsized":"2023-07-14T11:20:14.718394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:22.172922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723369+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:15.723369+0000","last_peered":"2023-07-14T11:20:15.723369+0000","last_clean":"2023-07-14T11:20:15.723369+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:15.723369+0000","last_undegraded":"2023-07-14T11:20:15.723369+0000","last_fullsized":"2023-07-14T11:20:15.723369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:47.388288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761518+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:20:15.761518+0000","last_peered":"2023-07-14T11:20:15.761518+0000","last_clean":"2023-07-14T11:20:15.761518+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:20:15.761518+0000","last_undegraded":"2023-07-14T11:20:15.761518+0000","last_fullsized":"2023-07-14T11:20:15.761518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:28.833104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763956+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:15.763956+0000","last_peered":"2023-07-14T11:20:15.763956+0000","last_clean":"2023-07-14T11:20:15.763956+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:15.763956+0000","last_undegraded":"2023-07-14T11:20:15.763956+0000","last_fullsized":"2023-07-14T11:20:15.763956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:18.046365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762296+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:20:15.762296+0000","last_peered":"2023-07-14T11:20:15.762296+0000","last_clean":"2023-07-14T11:20:15.762296+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:20:15.762296+0000","last_undegraded":"2023-07-14T11:20:15.762296+0000","last_fullsized":"2023-07-14T11:20:15.762296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:04:14.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725262+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:20:15.725262+0000","last_peered":"2023-07-14T11:20:15.725262+0000","last_clean":"2023-07-14T11:20:15.725262+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:20:15.725262+0000","last_undegraded":"2023-07-14T11:20:15.725262+0000","last_fullsized":"2023-07-14T11:20:15.725262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:40:44.958728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725446+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:15.725446+0000","last_peered":"2023-07-14T11:20:15.725446+0000","last_clean":"2023-07-14T11:20:15.725446+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:15.725446+0000","last_undegraded":"2023-07-14T11:20:15.725446+0000","last_fullsized":"2023-07-14T11:20:15.725446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.980346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729830+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:20:15.729830+0000","last_peered":"2023-07-14T11:20:15.729830+0000","last_clean":"2023-07-14T11:20:15.729830+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:20:15.729830+0000","last_undegraded":"2023-07-14T11:20:15.729830+0000","last_fullsized":"2023-07-14T11:20:15.729830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:23.746186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724337+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:20:15.724337+0000","last_peered":"2023-07-14T11:20:15.724337+0000","last_clean":"2023-07-14T11:20:15.724337+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:20:15.724337+0000","last_undegraded":"2023-07-14T11:20:15.724337+0000","last_fullsized":"2023-07-14T11:20:15.724337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:32.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717656+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:14.717656+0000","last_peered":"2023-07-14T11:20:14.717656+0000","last_clean":"2023-07-14T11:20:14.717656+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:14.717656+0000","last_undegraded":"2023-07-14T11:20:14.717656+0000","last_fullsized":"2023-07-14T11:20:14.717656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:28.963610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763590+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:20:15.763590+0000","last_peered":"2023-07-14T11:20:15.763590+0000","last_clean":"2023-07-14T11:20:15.763590+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:20:15.763590+0000","last_undegraded":"2023-07-14T11:20:15.763590+0000","last_fullsized":"2023-07-14T11:20:15.763590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:06:43.200171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761544+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:15.761544+0000","last_peered":"2023-07-14T11:20:15.761544+0000","last_clean":"2023-07-14T11:20:15.761544+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:15.761544+0000","last_undegraded":"2023-07-14T11:20:15.761544+0000","last_fullsized":"2023-07-14T11:20:15.761544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:10:23.505930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:25.389747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725984+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:20:15.725984+0000","last_peered":"2023-07-14T11:20:15.725984+0000","last_clean":"2023-07-14T11:20:15.725984+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:20:15.725984+0000","last_undegraded":"2023-07-14T11:20:15.725984+0000","last_fullsized":"2023-07-14T11:20:15.725984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:28.285910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725493+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:20:15.725493+0000","last_peered":"2023-07-14T11:20:15.725493+0000","last_clean":"2023-07-14T11:20:15.725493+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:20:15.725493+0000","last_undegraded":"2023-07-14T11:20:15.725493+0000","last_fullsized":"2023-07-14T11:20:15.725493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:49.791341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761896+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:20:15.761896+0000","last_peered":"2023-07-14T11:20:15.761896+0000","last_clean":"2023-07-14T11:20:15.761896+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:20:15.761896+0000","last_undegraded":"2023-07-14T11:20:15.761896+0000","last_fullsized":"2023-07-14T11:20:15.761896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:04:06.442323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724354+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:20:15.724354+0000","last_peered":"2023-07-14T11:20:15.724354+0000","last_clean":"2023-07-14T11:20:15.724354+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:20:15.724354+0000","last_undegraded":"2023-07-14T11:20:15.724354+0000","last_fullsized":"2023-07-14T11:20:15.724354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:31.530941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T11:20:14.717280+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:14.717280+0000","last_peered":"2023-07-14T11:20:14.717280+0000","last_clean":"2023-07-14T11:20:14.717280+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:14.717280+0000","last_undegraded":"2023-07-14T11:20:14.717280+0000","last_fullsized":"2023-07-14T11:20:14.717280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:38:55.878246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727131+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:20:15.727131+0000","last_peered":"2023-07-14T11:20:15.727131+0000","last_clean":"2023-07-14T11:20:15.727131+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:20:15.727131+0000","last_undegraded":"2023-07-14T11:20:15.727131+0000","last_fullsized":"2023-07-14T11:20:15.727131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:31:54.387366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727120+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:20:15.727120+0000","last_peered":"2023-07-14T11:20:15.727120+0000","last_clean":"2023-07-14T11:20:15.727120+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:20:15.727120+0000","last_undegraded":"2023-07-14T11:20:15.727120+0000","last_fullsized":"2023-07-14T11:20:15.727120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:15.525008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726470+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:20:15.726470+0000","last_peered":"2023-07-14T11:20:15.726470+0000","last_clean":"2023-07-14T11:20:15.726470+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:20:15.726470+0000","last_undegraded":"2023-07-14T11:20:15.726470+0000","last_fullsized":"2023-07-14T11:20:15.726470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:03.965708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729884+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:20:15.729884+0000","last_peered":"2023-07-14T11:20:15.729884+0000","last_clean":"2023-07-14T11:20:15.729884+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:20:15.729884+0000","last_undegraded":"2023-07-14T11:20:15.729884+0000","last_fullsized":"2023-07-14T11:20:15.729884+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:31.118854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759707+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:20:15.759707+0000","last_peered":"2023-07-14T11:20:15.759707+0000","last_clean":"2023-07-14T11:20:15.759707+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:20:15.759707+0000","last_undegraded":"2023-07-14T11:20:15.759707+0000","last_fullsized":"2023-07-14T11:20:15.759707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:35.715875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766177+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:20:15.766177+0000","last_peered":"2023-07-14T11:20:15.766177+0000","last_clean":"2023-07-14T11:20:15.766177+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:20:15.766177+0000","last_undegraded":"2023-07-14T11:20:15.766177+0000","last_fullsized":"2023-07-14T11:20:15.766177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:59.517623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758537+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:20:15.758537+0000","last_peered":"2023-07-14T11:20:15.758537+0000","last_clean":"2023-07-14T11:20:15.758537+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:20:15.758537+0000","last_undegraded":"2023-07-14T11:20:15.758537+0000","last_fullsized":"2023-07-14T11:20:15.758537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.641650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723412+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:15.723412+0000","last_peered":"2023-07-14T11:20:15.723412+0000","last_clean":"2023-07-14T11:20:15.723412+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:15.723412+0000","last_undegraded":"2023-07-14T11:20:15.723412+0000","last_fullsized":"2023-07-14T11:20:15.723412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:35.428375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726258+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:20:15.726258+0000","last_peered":"2023-07-14T11:20:15.726258+0000","last_clean":"2023-07-14T11:20:15.726258+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:20:15.726258+0000","last_undegraded":"2023-07-14T11:20:15.726258+0000","last_fullsized":"2023-07-14T11:20:15.726258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:41.949605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:48.091011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725591+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:20:15.725591+0000","last_peered":"2023-07-14T11:20:15.725591+0000","last_clean":"2023-07-14T11:20:15.725591+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:20:15.725591+0000","last_undegraded":"2023-07-14T11:20:15.725591+0000","last_fullsized":"2023-07-14T11:20:15.725591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:15.490002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726260+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:20:15.726260+0000","last_peered":"2023-07-14T11:20:15.726260+0000","last_clean":"2023-07-14T11:20:15.726260+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:20:15.726260+0000","last_undegraded":"2023-07-14T11:20:15.726260+0000","last_fullsized":"2023-07-14T11:20:15.726260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:53:08.815873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726407+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:20:15.726407+0000","last_peered":"2023-07-14T11:20:15.726407+0000","last_clean":"2023-07-14T11:20:15.726407+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:20:15.726407+0000","last_undegraded":"2023-07-14T11:20:15.726407+0000","last_fullsized":"2023-07-14T11:20:15.726407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:08.673713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723496+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:15.723496+0000","last_peered":"2023-07-14T11:20:15.723496+0000","last_clean":"2023-07-14T11:20:15.723496+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:15.723496+0000","last_undegraded":"2023-07-14T11:20:15.723496+0000","last_fullsized":"2023-07-14T11:20:15.723496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:41:54.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726094+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:20:15.726094+0000","last_peered":"2023-07-14T11:20:15.726094+0000","last_clean":"2023-07-14T11:20:15.726094+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:20:15.726094+0000","last_undegraded":"2023-07-14T11:20:15.726094+0000","last_fullsized":"2023-07-14T11:20:15.726094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:36:46.120312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725900+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:20:15.725900+0000","last_peered":"2023-07-14T11:20:15.725900+0000","last_clean":"2023-07-14T11:20:15.725900+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:20:15.725900+0000","last_undegraded":"2023-07-14T11:20:15.725900+0000","last_fullsized":"2023-07-14T11:20:15.725900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:29.569291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762498+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:20:15.762498+0000","last_peered":"2023-07-14T11:20:15.762498+0000","last_clean":"2023-07-14T11:20:15.762498+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:20:15.762498+0000","last_undegraded":"2023-07-14T11:20:15.762498+0000","last_fullsized":"2023-07-14T11:20:15.762498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:46.020055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-14T11:19:42.068794+0000","last_change":"2023-07-14T11:20:15.683458+0000","last_active":"2023-07-14T11:19:41.086153+0000","last_peered":"2023-07-14T11:19:41.086153+0000","last_clean":"2023-07-14T11:19:41.086153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:19:42.068794+0000","last_undegraded":"2023-07-14T11:19:42.068794+0000","last_fullsized":"2023-07-14T11:19:42.068794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761928+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:20:15.761928+0000","last_peered":"2023-07-14T11:20:15.761928+0000","last_clean":"2023-07-14T11:20:15.761928+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:20:15.761928+0000","last_undegraded":"2023-07-14T11:20:15.761928+0000","last_fullsized":"2023-07-14T11:20:15.761928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:50:15.332355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723311+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:15.723311+0000","last_peered":"2023-07-14T11:20:15.723311+0000","last_clean":"2023-07-14T11:20:15.723311+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:15.723311+0000","last_undegraded":"2023-07-14T11:20:15.723311+0000","last_fullsized":"2023-07-14T11:20:15.723311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:13.493038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725897+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:20:15.725897+0000","last_peered":"2023-07-14T11:20:15.725897+0000","last_clean":"2023-07-14T11:20:15.725897+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:20:15.725897+0000","last_undegraded":"2023-07-14T11:20:15.725897+0000","last_fullsized":"2023-07-14T11:20:15.725897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:20.934793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46642,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":58,"num_read_kb":80,"num_write":142,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":139926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":490869,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.62},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.80100000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.84599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.875}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1105920,"data_stored":1121217,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.377}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.501},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.63500000000000001}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.751},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.86399999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":495201,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":1.012,"15min":1.012},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.748},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.997,"15min":1.0489999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":0.999,"15min":0.999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0880000000000001,"5min":1.264,"15min":1.264},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.56499999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.94699999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.999,"5min":1.073,"15min":1.073},"last":0.999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.84099999999999997,"15min":0.86299999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93100000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.872,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.04},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0409999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.0169999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.151,"15min":1.2090000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99299999999999999,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90100000000000002,"15min":0.91700000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.081,"5min":1.171,"15min":1.171},"last":1.081}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.96799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":481467,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93000000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0129999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1086116,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.373},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.5},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.442}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68999999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.131,"15min":1.131},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486008,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.087}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.04}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.89800000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":515148,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.84799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.623},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.752}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":487396,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.623},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.745}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.80700000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.85899999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.875}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":520790,"data_compressed":14664,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73799999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82599999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72499999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486475,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.505}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.63},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.754}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.59999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96699999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.98599999999999999,"15min":1.0169999999999999},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":4.157,"15min":4.157},"last":0.73799999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490883,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.374}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.498}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82499999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.438}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.40100000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47199999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.623}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1081752,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.377},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.53500000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.626}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.747}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":44430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:21.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-07-14T11:20:21.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-07-14T11:19:47.967804+0000', 'last_change': '2023-07-14T11:20:14.717622+0000', 'last_active': '2023-07-14T11:20:14.717622+0000', 'last_peered': '2023-07-14T11:19:47.967804+0000', 'last_clean': '2023-07-14T11:19:47.967804+0000', 'last_became_active': '2023-07-14T11:19:46.273092+0000', 'last_became_peered': '2023-07-14T11:19:46.273092+0000', 'last_unstale': '2023-07-14T11:19:47.967804+0000', 'last_undegraded': '2023-07-14T11:19:47.967804+0000', 'last_fullsized': '2023-07-14T11:19:47.967804+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 86, 'last_epoch_clean': 87, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:19:45.066838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:19:45.066838+0000', 'last_clean_scrub_stamp': '2023-07-14T11:19:45.066838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 10, 11, 1], 'acting': [8, 10, 11, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 9, 10, 11], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-07-14T11:20:21.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-07-14T11:20:21.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 89, 'state': 'peering', 'last_fresh': '2023-07-14T11:19:42.068794+0000', 'last_change': '2023-07-14T11:20:15.683458+0000', 'last_active': '2023-07-14T11:19:41.086153+0000', 'last_peered': '2023-07-14T11:19:41.086153+0000', 'last_clean': '2023-07-14T11:19:41.086153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:19:42.068794+0000', 'last_undegraded': '2023-07-14T11:19:42.068794+0000', 'last_fullsized': '2023-07-14T11:19:42.068794+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 83, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:19:41.086153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:19:41.086153+0000', 'last_clean_scrub_stamp': '2023-07-14T11:19:41.086153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 9, 3], 'acting': [8, 9, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-07-14T11:20:21.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-14T11:20:22.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/1619972176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb40ffb00 msgr2=0x7f4eb40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/1619972176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb40ffb00 0x7f4eb40fff10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00098f0 tx=0x7f4eb0009c00 comp rx=0 tx=0).stop 2023-07-14T11:20:22.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/1619972176 shutdown_connections 2023-07-14T11:20:22.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/1619972176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb4101680 0x7f4eb4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/1619972176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb40ffb00 0x7f4eb40fff10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/1619972176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4100cf0 0x7f4eb4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/1619972176 >> 172.21.15.161:0/1619972176 conn(0x7f4eb40fb0e0 msgr2=0x7f4eb40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:22.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/1619972176 shutdown_connections 2023-07-14T11:20:22.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.169+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/1619972176 wait complete. 2023-07-14T11:20:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.170+0000 7f4ebc7ff700 1 Processor -- start 2023-07-14T11:20:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.170+0000 7f4ebc7ff700 1 -- start start 2023-07-14T11:20:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4ebc7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb40ffb00 0x7f4eb4195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4ebc7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb4100cf0 0x7f4eb4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4ebc7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4101680 0x7f4eb419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4ebc7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb419d3a0 con 0x7f4eb4101680 2023-07-14T11:20:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4ebc7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb419d4e0 con 0x7f4eb40ffb00 2023-07-14T11:20:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4ebc7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb419d670 con 0x7f4eb4100cf0 2023-07-14T11:20:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb40ffb00 0x7f4eb4195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4ebad9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4101680 0x7f4eb419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb40ffb00 0x7f4eb4195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57302/0 (socket says 172.21.15.161:57302) 2023-07-14T11:20:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 -- 172.21.15.161:0/3659809970 learned_addr learned my addr 172.21.15.161:0/3659809970 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eb9d9a700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb4100cf0 0x7f4eb4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 -- 172.21.15.161:0/3659809970 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb4100cf0 msgr2=0x7f4eb4195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb4100cf0 0x7f4eb4195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 -- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4101680 msgr2=0x7f4eb419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4101680 0x7f4eb419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.171+0000 7f4eba59b700 1 -- 172.21.15.161:0/3659809970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0009520 con 0x7f4eb40ffb00 2023-07-14T11:20:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.172+0000 7f4eba59b700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb40ffb00 0x7f4eb4195820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009fd0 tx=0x7f4eb000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.172+0000 7f4eab7fe700 1 -- 172.21.15.161:0/3659809970 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb001c070 con 0x7f4eb40ffb00 2023-07-14T11:20:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.172+0000 7f4eab7fe700 1 -- 172.21.15.161:0/3659809970 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb000f7d0 con 0x7f4eb40ffb00 2023-07-14T11:20:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.172+0000 7f4eab7fe700 1 -- 172.21.15.161:0/3659809970 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0003a40 con 0x7f4eb40ffb00 2023-07-14T11:20:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.172+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb419d8f0 con 0x7f4eb40ffb00 2023-07-14T11:20:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.172+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb419dd20 con 0x7f4eb40ffb00 2023-07-14T11:20:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.173+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb410b2c0 con 0x7f4eb40ffb00 2023-07-14T11:20:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.175+0000 7f4eab7fe700 1 -- 172.21.15.161:0/3659809970 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f4eb0020030 con 0x7f4eb40ffb00 2023-07-14T11:20:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.176+0000 7f4eab7fe700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ea00a1220 0x7f4ea00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.176+0000 7f4eab7fe700 1 -- 172.21.15.161:0/3659809970 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4eb00cdda0 con 0x7f4eb40ffb00 2023-07-14T11:20:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.176+0000 7f4eb9d9a700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ea00a1220 0x7f4ea00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.177+0000 7f4eb9d9a700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ea00a1220 0x7f4ea00a36d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009c60 tx=0x7f4ea4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:22.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.181+0000 7f4eab7fe700 1 -- 172.21.15.161:0/3659809970 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4eb00970b0 con 0x7f4eb40ffb00 2023-07-14T11:20:22.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:22 smithi161 ceph-mon[95526]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:22.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:22 smithi175 ceph-mon[129081]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.457+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4eb419e360 con 0x7f4eb40ffb00 2023-07-14T11:20:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.458+0000 7f4eab7fe700 1 -- 172.21.15.161:0/3659809970 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2560 (secure 0 0 0) 0x7f4eb009a6d0 con 0x7f4eb40ffb00 2023-07-14T11:20:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:22.462 INFO:teuthology.orchestra.run.smithi161.stdout:{"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":371,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":90,"num_osds":12,"num_up_osds":12,"osd_up_since":1689333533,"num_in_osds":12,"osd_in_since":1689333516,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":637256,"bytes_used":343363584,"bytes_avail":1151597064192,"bytes_total":1151940427776,"read_bytes_sec":8957,"write_bytes_sec":7165,"read_op_per_sec":5,"write_op_per_sec":6},"fsmap":{"epoch":22,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24503},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24514},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24526},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24533},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24544},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24575}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.161:8443/"}},"servicemap":{"epoch":22,"modified":"2023-07-14T11:20:18.104492+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-07-14T11:20:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:22 smithi182 ceph-mon[129366]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ea00a1220 msgr2=0x7f4ea00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ea00a1220 0x7f4ea00a36d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009c60 tx=0x7f4ea4009260 comp rx=0 tx=0).stop 2023-07-14T11:20:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb40ffb00 msgr2=0x7f4eb4195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb40ffb00 0x7f4eb4195820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009fd0 tx=0x7f4eb000b790 comp rx=0 tx=0).stop 2023-07-14T11:20:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 shutdown_connections 2023-07-14T11:20:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4101680 0x7f4eb419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ea00a1220 0x7f4ea00a36d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb40ffb00 0x7f4eb4195820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 --2- 172.21.15.161:0/3659809970 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4eb4100cf0 0x7f4eb4195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 >> 172.21.15.161:0/3659809970 conn(0x7f4eb40fb0e0 msgr2=0x7f4eb40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 shutdown_connections 2023-07-14T11:20:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.461+0000 7f4ebc7ff700 1 -- 172.21.15.161:0/3659809970 wait complete. 2023-07-14T11:20:22.483 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-14T11:20:22.483 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:22.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/3690860015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64102e70 msgr2=0x7f5d64105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/3690860015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64102e70 0x7f5d64105250 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009990 tx=0x7f5d54009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/3690860015 shutdown_connections 2023-07-14T11:20:22.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/3690860015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d64105790 0x7f5d64107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/3690860015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d64100550 0x7f5d64102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/3690860015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64102e70 0x7f5d64105250 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/3690860015 >> 172.21.15.161:0/3690860015 conn(0x7f5d640f9fd0 msgr2=0x7f5d640fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:22.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/3690860015 shutdown_connections 2023-07-14T11:20:22.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.679+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/3690860015 wait complete. 2023-07-14T11:20:22.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.680+0000 7f5d6b7ff700 1 Processor -- start 2023-07-14T11:20:22.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.680+0000 7f5d6b7ff700 1 -- start start 2023-07-14T11:20:22.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d6b7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d64100550 0x7f5d64197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d6b7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d64102e70 0x7f5d64197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d6b7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64105790 0x7f5d6419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d6b7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6419f590 con 0x7f5d64105790 2023-07-14T11:20:22.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d6b7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6419f6d0 con 0x7f5d64102e70 2023-07-14T11:20:22.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d6b7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6419f860 con 0x7f5d64100550 2023-07-14T11:20:22.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d69d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64105790 0x7f5d6419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d69d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64105790 0x7f5d6419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38014/0 (socket says 172.21.15.161:38014) 2023-07-14T11:20:22.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d69d9c700 1 -- 172.21.15.161:0/2723622886 learned_addr learned my addr 172.21.15.161:0/2723622886 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:22.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d6959b700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d64100550 0x7f5d64197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d68d9a700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d64102e70 0x7f5d64197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d69d9c700 1 -- 172.21.15.161:0/2723622886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d64100550 msgr2=0x7f5d64197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d69d9c700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d64100550 0x7f5d64197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d69d9c700 1 -- 172.21.15.161:0/2723622886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d64102e70 msgr2=0x7f5d64197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.681+0000 7f5d69d9c700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d64102e70 0x7f5d64197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d69d9c700 1 -- 172.21.15.161:0/2723622886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540095f0 con 0x7f5d64105790 2023-07-14T11:20:22.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d68d9a700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d64102e70 0x7f5d64197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:22.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d69d9c700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64105790 0x7f5d6419efc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00ef30 tx=0x7f5d5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:22.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d5a7fc700 1 -- 172.21.15.161:0/2723622886 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c018070 con 0x7f5d64105790 2023-07-14T11:20:22.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d5a7fc700 1 -- 172.21.15.161:0/2723622886 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5c00f040 con 0x7f5d64105790 2023-07-14T11:20:22.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6419fae0 con 0x7f5d64105790 2023-07-14T11:20:22.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d5a7fc700 1 -- 172.21.15.161:0/2723622886 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00cdb0 con 0x7f5d64105790 2023-07-14T11:20:22.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.682+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d641a0040 con 0x7f5d64105790 2023-07-14T11:20:22.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.683+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d64107350 con 0x7f5d64105790 2023-07-14T11:20:22.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.685+0000 7f5d5a7fc700 1 -- 172.21.15.161:0/2723622886 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5d5c003a00 con 0x7f5d64105790 2023-07-14T11:20:22.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.686+0000 7f5d5a7fc700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5d500a12a0 0x7f5d500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:22.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.687+0000 7f5d5a7fc700 1 -- 172.21.15.161:0/2723622886 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5d5c0cc940 con 0x7f5d64105790 2023-07-14T11:20:22.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.687+0000 7f5d6959b700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5d500a12a0 0x7f5d500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:22.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.687+0000 7f5d6959b700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5d500a12a0 0x7f5d500a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009c60 tx=0x7f5d60009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:22.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.691+0000 7f5d5a7fc700 1 -- 172.21.15.161:0/2723622886 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5d5c095c50 con 0x7f5d64105790 2023-07-14T11:20:22.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.887+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d64060950 con 0x7f5d500a12a0 2023-07-14T11:20:22.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.900+0000 7f5d5a7fc700 1 -- 172.21.15.161:0/2723622886 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422777 (secure 0 0 0) 0x7f5d64060950 con 0x7f5d500a12a0 2023-07-14T11:20:22.911 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:22.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.904+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5d500a12a0 msgr2=0x7f5d500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.904+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5d500a12a0 0x7f5d500a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009c60 tx=0x7f5d60009260 comp rx=0 tx=0).stop 2023-07-14T11:20:22.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.904+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64105790 msgr2=0x7f5d6419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.904+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64105790 0x7f5d6419efc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00ef30 tx=0x7f5d5c00c3a0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 shutdown_connections 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d64105790 0x7f5d6419efc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5d500a12a0 0x7f5d500a3750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d64100550 0x7f5d64197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 --2- 172.21.15.161:0/2723622886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d64102e70 0x7f5d64197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 >> 172.21.15.161:0/2723622886 conn(0x7f5d640f9fd0 msgr2=0x7f5d640febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:22.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 shutdown_connections 2023-07-14T11:20:22.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:22.905+0000 7f5d6b7ff700 1 -- 172.21.15.161:0/2723622886 wait complete. 2023-07-14T11:20:22.916 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:22.927 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-07-14T11:20:22.105548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637256,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":194,"num_read_kb":199,"num_write":327,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":423,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335316,"kb_used_data":11892,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606508,"statfs":{"total":1151940427776,"available":1151597064192,"internally_reserved":0,"allocated":12177408,"data_stored":7776609,"data_compressed":318627,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40386,"num_objects":19,"num_object_clones":0,"num_object_copies":59,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":46,"num_read_kb":70,"num_write":53,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002325"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726364+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:20:15.726364+0000","last_peered":"2023-07-14T11:20:15.726364+0000","last_clean":"2023-07-14T11:20:15.726364+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:20:15.726364+0000","last_undegraded":"2023-07-14T11:20:15.726364+0000","last_fullsized":"2023-07-14T11:20:15.726364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:25.074131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726699+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:20:15.726699+0000","last_peered":"2023-07-14T11:20:15.726699+0000","last_clean":"2023-07-14T11:20:15.726699+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:20:15.726699+0000","last_undegraded":"2023-07-14T11:20:15.726699+0000","last_fullsized":"2023-07-14T11:20:15.726699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:41.854592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726554+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:20:15.726554+0000","last_peered":"2023-07-14T11:20:15.726554+0000","last_clean":"2023-07-14T11:20:15.726554+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:20:15.726554+0000","last_undegraded":"2023-07-14T11:20:15.726554+0000","last_fullsized":"2023-07-14T11:20:15.726554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:37.342912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726181+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:20:15.726181+0000","last_peered":"2023-07-14T11:20:15.726181+0000","last_clean":"2023-07-14T11:20:15.726181+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:20:15.726181+0000","last_undegraded":"2023-07-14T11:20:15.726181+0000","last_fullsized":"2023-07-14T11:20:15.726181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:46.208987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728594+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:15.728594+0000","last_peered":"2023-07-14T11:20:15.728594+0000","last_clean":"2023-07-14T11:20:15.728594+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:15.728594+0000","last_undegraded":"2023-07-14T11:20:15.728594+0000","last_fullsized":"2023-07-14T11:20:15.728594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:53:26.823230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765846+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:20:15.765846+0000","last_peered":"2023-07-14T11:20:15.765846+0000","last_clean":"2023-07-14T11:20:15.765846+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:20:15.765846+0000","last_undegraded":"2023-07-14T11:20:15.765846+0000","last_fullsized":"2023-07-14T11:20:15.765846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:31:09.497877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725992+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:20:15.725992+0000","last_peered":"2023-07-14T11:20:15.725992+0000","last_clean":"2023-07-14T11:20:15.725992+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:20:15.725992+0000","last_undegraded":"2023-07-14T11:20:15.725992+0000","last_fullsized":"2023-07-14T11:20:15.725992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:18.893918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761828+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:20:15.761828+0000","last_peered":"2023-07-14T11:20:15.761828+0000","last_clean":"2023-07-14T11:20:15.761828+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:20:15.761828+0000","last_undegraded":"2023-07-14T11:20:15.761828+0000","last_fullsized":"2023-07-14T11:20:15.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730008+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:20:15.730008+0000","last_peered":"2023-07-14T11:20:15.730008+0000","last_clean":"2023-07-14T11:20:15.730008+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:20:15.730008+0000","last_undegraded":"2023-07-14T11:20:15.730008+0000","last_fullsized":"2023-07-14T11:20:15.730008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:25:58.920431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726577+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:20:15.726577+0000","last_peered":"2023-07-14T11:20:15.726577+0000","last_clean":"2023-07-14T11:20:15.726577+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:20:15.726577+0000","last_undegraded":"2023-07-14T11:20:15.726577+0000","last_fullsized":"2023-07-14T11:20:15.726577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:29.172086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726169+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:20:15.726169+0000","last_peered":"2023-07-14T11:20:15.726169+0000","last_clean":"2023-07-14T11:20:15.726169+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:20:15.726169+0000","last_undegraded":"2023-07-14T11:20:15.726169+0000","last_fullsized":"2023-07-14T11:20:15.726169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:29.746118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723416+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:15.723416+0000","last_peered":"2023-07-14T11:20:15.723416+0000","last_clean":"2023-07-14T11:20:15.723416+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:15.723416+0000","last_undegraded":"2023-07-14T11:20:15.723416+0000","last_fullsized":"2023-07-14T11:20:15.723416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:57.801423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725956+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:20:15.725956+0000","last_peered":"2023-07-14T11:20:15.725956+0000","last_clean":"2023-07-14T11:20:15.725956+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:20:15.725956+0000","last_undegraded":"2023-07-14T11:20:15.725956+0000","last_fullsized":"2023-07-14T11:20:15.725956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:01.673209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726027+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:20:15.726027+0000","last_peered":"2023-07-14T11:20:15.726027+0000","last_clean":"2023-07-14T11:20:15.726027+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:20:15.726027+0000","last_undegraded":"2023-07-14T11:20:15.726027+0000","last_fullsized":"2023-07-14T11:20:15.726027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:10.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726533+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:20:15.726533+0000","last_peered":"2023-07-14T11:20:15.726533+0000","last_clean":"2023-07-14T11:20:15.726533+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:20:15.726533+0000","last_undegraded":"2023-07-14T11:20:15.726533+0000","last_fullsized":"2023-07-14T11:20:15.726533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:33.733562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763940+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:20:15.763940+0000","last_peered":"2023-07-14T11:20:15.763940+0000","last_clean":"2023-07-14T11:20:15.763940+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:20:15.763940+0000","last_undegraded":"2023-07-14T11:20:15.763940+0000","last_fullsized":"2023-07-14T11:20:15.763940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:28:49.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725800+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:20:15.725800+0000","last_peered":"2023-07-14T11:20:15.725800+0000","last_clean":"2023-07-14T11:20:15.725800+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:20:15.725800+0000","last_undegraded":"2023-07-14T11:20:15.725800+0000","last_fullsized":"2023-07-14T11:20:15.725800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:30.183181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726357+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:20:15.726357+0000","last_peered":"2023-07-14T11:20:15.726357+0000","last_clean":"2023-07-14T11:20:15.726357+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:20:15.726357+0000","last_undegraded":"2023-07-14T11:20:15.726357+0000","last_fullsized":"2023-07-14T11:20:15.726357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:13.424624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763774+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:15.763774+0000","last_peered":"2023-07-14T11:20:15.763774+0000","last_clean":"2023-07-14T11:20:15.763774+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:15.763774+0000","last_undegraded":"2023-07-14T11:20:15.763774+0000","last_fullsized":"2023-07-14T11:20:15.763774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:54.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762089+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:20:15.762089+0000","last_peered":"2023-07-14T11:20:15.762089+0000","last_clean":"2023-07-14T11:20:15.762089+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:20:15.762089+0000","last_undegraded":"2023-07-14T11:20:15.762089+0000","last_fullsized":"2023-07-14T11:20:15.762089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:06.259730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724677+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:20:15.724677+0000","last_peered":"2023-07-14T11:20:15.724677+0000","last_clean":"2023-07-14T11:20:15.724677+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:20:15.724677+0000","last_undegraded":"2023-07-14T11:20:15.724677+0000","last_fullsized":"2023-07-14T11:20:15.724677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:07.076184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762219+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:20:15.762219+0000","last_peered":"2023-07-14T11:20:15.762219+0000","last_clean":"2023-07-14T11:20:15.762219+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:20:15.762219+0000","last_undegraded":"2023-07-14T11:20:15.762219+0000","last_fullsized":"2023-07-14T11:20:15.762219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:30.071523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726815+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:20:15.726815+0000","last_peered":"2023-07-14T11:20:15.726815+0000","last_clean":"2023-07-14T11:20:15.726815+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:20:15.726815+0000","last_undegraded":"2023-07-14T11:20:15.726815+0000","last_fullsized":"2023-07-14T11:20:15.726815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:41:41.007835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724865+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:20:15.724865+0000","last_peered":"2023-07-14T11:20:15.724865+0000","last_clean":"2023-07-14T11:20:15.724865+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:20:15.724865+0000","last_undegraded":"2023-07-14T11:20:15.724865+0000","last_fullsized":"2023-07-14T11:20:15.724865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:56:28.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729360+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.729360+0000","last_peered":"2023-07-14T11:20:15.729360+0000","last_clean":"2023-07-14T11:20:15.729360+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:15.729360+0000","last_undegraded":"2023-07-14T11:20:15.729360+0000","last_fullsized":"2023-07-14T11:20:15.729360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:54.894820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726665+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:20:15.726665+0000","last_peered":"2023-07-14T11:20:15.726665+0000","last_clean":"2023-07-14T11:20:15.726665+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:20:15.726665+0000","last_undegraded":"2023-07-14T11:20:15.726665+0000","last_fullsized":"2023-07-14T11:20:15.726665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:39.791849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728688+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:15.728688+0000","last_peered":"2023-07-14T11:20:15.728688+0000","last_clean":"2023-07-14T11:20:15.728688+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:15.728688+0000","last_undegraded":"2023-07-14T11:20:15.728688+0000","last_fullsized":"2023-07-14T11:20:15.728688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:23:54.582861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759435+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:20:15.759435+0000","last_peered":"2023-07-14T11:20:15.759435+0000","last_clean":"2023-07-14T11:20:15.759435+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:20:15.759435+0000","last_undegraded":"2023-07-14T11:20:15.759435+0000","last_fullsized":"2023-07-14T11:20:15.759435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:51.343442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725567+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:20:15.725567+0000","last_peered":"2023-07-14T11:20:15.725567+0000","last_clean":"2023-07-14T11:20:15.725567+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:20:15.725567+0000","last_undegraded":"2023-07-14T11:20:15.725567+0000","last_fullsized":"2023-07-14T11:20:15.725567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:50.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.686170+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:16.686170+0000","last_peered":"2023-07-14T11:20:16.686170+0000","last_clean":"2023-07-14T11:20:16.686170+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:16.686170+0000","last_undegraded":"2023-07-14T11:20:16.686170+0000","last_fullsized":"2023-07-14T11:20:16.686170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:41.232654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728839+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:15.728839+0000","last_peered":"2023-07-14T11:20:15.728839+0000","last_clean":"2023-07-14T11:20:15.728839+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:15.728839+0000","last_undegraded":"2023-07-14T11:20:15.728839+0000","last_fullsized":"2023-07-14T11:20:15.728839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:12.092163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766231+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:20:15.766231+0000","last_peered":"2023-07-14T11:20:15.766231+0000","last_clean":"2023-07-14T11:20:15.766231+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:20:15.766231+0000","last_undegraded":"2023-07-14T11:20:15.766231+0000","last_fullsized":"2023-07-14T11:20:15.766231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:16:19.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730104+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:15.730104+0000","last_peered":"2023-07-14T11:20:15.730104+0000","last_clean":"2023-07-14T11:20:15.730104+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:15.730104+0000","last_undegraded":"2023-07-14T11:20:15.730104+0000","last_fullsized":"2023-07-14T11:20:15.730104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:54:17.106221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759228+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:20:15.759228+0000","last_peered":"2023-07-14T11:20:15.759228+0000","last_clean":"2023-07-14T11:20:15.759228+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:20:15.759228+0000","last_undegraded":"2023-07-14T11:20:15.759228+0000","last_fullsized":"2023-07-14T11:20:15.759228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:47.650473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726249+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:20:15.726249+0000","last_peered":"2023-07-14T11:20:15.726249+0000","last_clean":"2023-07-14T11:20:15.726249+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:20:15.726249+0000","last_undegraded":"2023-07-14T11:20:15.726249+0000","last_fullsized":"2023-07-14T11:20:15.726249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:10.265432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722943+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:15.722943+0000","last_peered":"2023-07-14T11:20:15.722943+0000","last_clean":"2023-07-14T11:20:15.722943+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:15.722943+0000","last_undegraded":"2023-07-14T11:20:15.722943+0000","last_fullsized":"2023-07-14T11:20:15.722943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:48.217979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761668+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:20:15.761668+0000","last_peered":"2023-07-14T11:20:15.761668+0000","last_clean":"2023-07-14T11:20:15.761668+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:20:15.761668+0000","last_undegraded":"2023-07-14T11:20:15.761668+0000","last_fullsized":"2023-07-14T11:20:15.761668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.096155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726651+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:20:15.726651+0000","last_peered":"2023-07-14T11:20:15.726651+0000","last_clean":"2023-07-14T11:20:15.726651+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:20:15.726651+0000","last_undegraded":"2023-07-14T11:20:15.726651+0000","last_fullsized":"2023-07-14T11:20:15.726651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:36.842168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758847+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:20:15.758847+0000","last_peered":"2023-07-14T11:20:15.758847+0000","last_clean":"2023-07-14T11:20:15.758847+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:20:15.758847+0000","last_undegraded":"2023-07-14T11:20:15.758847+0000","last_fullsized":"2023-07-14T11:20:15.758847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:38.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763190+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:20:15.763190+0000","last_peered":"2023-07-14T11:20:15.763190+0000","last_clean":"2023-07-14T11:20:15.763190+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:20:15.763190+0000","last_undegraded":"2023-07-14T11:20:15.763190+0000","last_fullsized":"2023-07-14T11:20:15.763190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:23.015078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761792+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:20:15.761792+0000","last_peered":"2023-07-14T11:20:15.761792+0000","last_clean":"2023-07-14T11:20:15.761792+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:20:15.761792+0000","last_undegraded":"2023-07-14T11:20:15.761792+0000","last_fullsized":"2023-07-14T11:20:15.761792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.645359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729832+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:15.729832+0000","last_peered":"2023-07-14T11:20:15.729832+0000","last_clean":"2023-07-14T11:20:15.729832+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:15.729832+0000","last_undegraded":"2023-07-14T11:20:15.729832+0000","last_fullsized":"2023-07-14T11:20:15.729832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.137485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763325+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:20:15.763325+0000","last_peered":"2023-07-14T11:20:15.763325+0000","last_clean":"2023-07-14T11:20:15.763325+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:20:15.763325+0000","last_undegraded":"2023-07-14T11:20:15.763325+0000","last_fullsized":"2023-07-14T11:20:15.763325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:28.186030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726329+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:15.726329+0000","last_peered":"2023-07-14T11:20:15.726329+0000","last_clean":"2023-07-14T11:20:15.726329+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:15.726329+0000","last_undegraded":"2023-07-14T11:20:15.726329+0000","last_fullsized":"2023-07-14T11:20:15.726329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:38.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726931+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:20:15.726931+0000","last_peered":"2023-07-14T11:20:15.726931+0000","last_clean":"2023-07-14T11:20:15.726931+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:20:15.726931+0000","last_undegraded":"2023-07-14T11:20:15.726931+0000","last_fullsized":"2023-07-14T11:20:15.726931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:58.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725362+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:20:15.725362+0000","last_peered":"2023-07-14T11:20:15.725362+0000","last_clean":"2023-07-14T11:20:15.725362+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:20:15.725362+0000","last_undegraded":"2023-07-14T11:20:15.725362+0000","last_fullsized":"2023-07-14T11:20:15.725362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:29.343315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759436+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:15.759436+0000","last_peered":"2023-07-14T11:20:15.759436+0000","last_clean":"2023-07-14T11:20:15.759436+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:15.759436+0000","last_undegraded":"2023-07-14T11:20:15.759436+0000","last_fullsized":"2023-07-14T11:20:15.759436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:17:07.579078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762929+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:20:15.762929+0000","last_peered":"2023-07-14T11:20:15.762929+0000","last_clean":"2023-07-14T11:20:15.762929+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:20:15.762929+0000","last_undegraded":"2023-07-14T11:20:15.762929+0000","last_fullsized":"2023-07-14T11:20:15.762929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:41.993201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'23","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.690487+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:16.690487+0000","last_peered":"2023-07-14T11:20:16.690487+0000","last_clean":"2023-07-14T11:20:16.690487+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:16.690487+0000","last_undegraded":"2023-07-14T11:20:16.690487+0000","last_fullsized":"2023-07-14T11:20:16.690487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:08:09.309573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43128,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":49,"num_write":24,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728912+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:15.728912+0000","last_peered":"2023-07-14T11:20:15.728912+0000","last_clean":"2023-07-14T11:20:15.728912+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:15.728912+0000","last_undegraded":"2023-07-14T11:20:15.728912+0000","last_fullsized":"2023-07-14T11:20:15.728912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:40:20.286989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759883+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:20:15.759883+0000","last_peered":"2023-07-14T11:20:15.759883+0000","last_clean":"2023-07-14T11:20:15.759883+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:20:15.759883+0000","last_undegraded":"2023-07-14T11:20:15.759883+0000","last_fullsized":"2023-07-14T11:20:15.759883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:27.460710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726089+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:15.726089+0000","last_peered":"2023-07-14T11:20:15.726089+0000","last_clean":"2023-07-14T11:20:15.726089+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:15.726089+0000","last_undegraded":"2023-07-14T11:20:15.726089+0000","last_fullsized":"2023-07-14T11:20:15.726089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:33:58.333180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722891+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:15.722891+0000","last_peered":"2023-07-14T11:20:15.722891+0000","last_clean":"2023-07-14T11:20:15.722891+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:15.722891+0000","last_undegraded":"2023-07-14T11:20:15.722891+0000","last_fullsized":"2023-07-14T11:20:15.722891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:12.630498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.732150+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:20:15.732150+0000","last_peered":"2023-07-14T11:20:15.732150+0000","last_clean":"2023-07-14T11:20:15.732150+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:20:15.732150+0000","last_undegraded":"2023-07-14T11:20:15.732150+0000","last_fullsized":"2023-07-14T11:20:15.732150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:49.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759053+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:15.759053+0000","last_peered":"2023-07-14T11:20:15.759053+0000","last_clean":"2023-07-14T11:20:15.759053+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:15.759053+0000","last_undegraded":"2023-07-14T11:20:15.759053+0000","last_fullsized":"2023-07-14T11:20:15.759053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:20:45.016996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765761+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:20:15.765761+0000","last_peered":"2023-07-14T11:20:15.765761+0000","last_clean":"2023-07-14T11:20:15.765761+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:20:15.765761+0000","last_undegraded":"2023-07-14T11:20:15.765761+0000","last_fullsized":"2023-07-14T11:20:15.765761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:11.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763313+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:20:15.763313+0000","last_peered":"2023-07-14T11:20:15.763313+0000","last_clean":"2023-07-14T11:20:15.763313+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:20:15.763313+0000","last_undegraded":"2023-07-14T11:20:15.763313+0000","last_fullsized":"2023-07-14T11:20:15.763313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:41.779784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:16.687593+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:16.687593+0000","last_peered":"2023-07-14T11:20:16.687593+0000","last_clean":"2023-07-14T11:20:16.687593+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:16.687593+0000","last_undegraded":"2023-07-14T11:20:16.687593+0000","last_fullsized":"2023-07-14T11:20:16.687593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:35.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726189+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:20:15.726189+0000","last_peered":"2023-07-14T11:20:15.726189+0000","last_clean":"2023-07-14T11:20:15.726189+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:20:15.726189+0000","last_undegraded":"2023-07-14T11:20:15.726189+0000","last_fullsized":"2023-07-14T11:20:15.726189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:25.466461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723291+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:15.723291+0000","last_peered":"2023-07-14T11:20:15.723291+0000","last_clean":"2023-07-14T11:20:15.723291+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:15.723291+0000","last_undegraded":"2023-07-14T11:20:15.723291+0000","last_fullsized":"2023-07-14T11:20:15.723291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:05.698741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728325+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:15.728325+0000","last_peered":"2023-07-14T11:20:15.728325+0000","last_clean":"2023-07-14T11:20:15.728325+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:15.728325+0000","last_undegraded":"2023-07-14T11:20:15.728325+0000","last_fullsized":"2023-07-14T11:20:15.728325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:03:30.273008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758707+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:20:15.758707+0000","last_peered":"2023-07-14T11:20:15.758707+0000","last_clean":"2023-07-14T11:20:15.758707+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:20:15.758707+0000","last_undegraded":"2023-07-14T11:20:15.758707+0000","last_fullsized":"2023-07-14T11:20:15.758707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:32.059475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758890+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:20:15.758890+0000","last_peered":"2023-07-14T11:20:15.758890+0000","last_clean":"2023-07-14T11:20:15.758890+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:20:15.758890+0000","last_undegraded":"2023-07-14T11:20:15.758890+0000","last_fullsized":"2023-07-14T11:20:15.758890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:51.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729124+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:15.729124+0000","last_peered":"2023-07-14T11:20:15.729124+0000","last_clean":"2023-07-14T11:20:15.729124+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:15.729124+0000","last_undegraded":"2023-07-14T11:20:15.729124+0000","last_fullsized":"2023-07-14T11:20:15.729124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:06.994422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725218+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:20:15.725218+0000","last_peered":"2023-07-14T11:20:15.725218+0000","last_clean":"2023-07-14T11:20:15.725218+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:20:15.725218+0000","last_undegraded":"2023-07-14T11:20:15.725218+0000","last_fullsized":"2023-07-14T11:20:15.725218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:42.841223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.731601+0000","last_change":"2023-07-14T11:20:15.731601+0000","last_active":"2023-07-14T11:20:15.731601+0000","last_peered":"2023-07-14T11:20:15.731601+0000","last_clean":"2023-07-14T11:20:15.731601+0000","last_became_active":"2023-07-14T11:20:15.731121+0000","last_became_peered":"2023-07-14T11:20:15.731121+0000","last_unstale":"2023-07-14T11:20:15.731601+0000","last_undegraded":"2023-07-14T11:20:15.731601+0000","last_fullsized":"2023-07-14T11:20:15.731601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:58.090267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764473+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:20:15.764473+0000","last_peered":"2023-07-14T11:20:15.764473+0000","last_clean":"2023-07-14T11:20:15.764473+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:20:15.764473+0000","last_undegraded":"2023-07-14T11:20:15.764473+0000","last_fullsized":"2023-07-14T11:20:15.764473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:06.410233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729586+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:20:15.729586+0000","last_peered":"2023-07-14T11:20:15.729586+0000","last_clean":"2023-07-14T11:20:15.729586+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:20:15.729586+0000","last_undegraded":"2023-07-14T11:20:15.729586+0000","last_fullsized":"2023-07-14T11:20:15.729586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:59.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723225+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:15.723225+0000","last_peered":"2023-07-14T11:20:15.723225+0000","last_clean":"2023-07-14T11:20:15.723225+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:15.723225+0000","last_undegraded":"2023-07-14T11:20:15.723225+0000","last_fullsized":"2023-07-14T11:20:15.723225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:32.064021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726984+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.726984+0000","last_peered":"2023-07-14T11:20:15.726984+0000","last_clean":"2023-07-14T11:20:15.726984+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:20:15.726984+0000","last_undegraded":"2023-07-14T11:20:15.726984+0000","last_fullsized":"2023-07-14T11:20:15.726984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:38:06.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765912+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:20:15.765912+0000","last_peered":"2023-07-14T11:20:15.765912+0000","last_clean":"2023-07-14T11:20:15.765912+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:20:15.765912+0000","last_undegraded":"2023-07-14T11:20:15.765912+0000","last_fullsized":"2023-07-14T11:20:15.765912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:01:52.067140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723192+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:15.723192+0000","last_peered":"2023-07-14T11:20:15.723192+0000","last_clean":"2023-07-14T11:20:15.723192+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:15.723192+0000","last_undegraded":"2023-07-14T11:20:15.723192+0000","last_fullsized":"2023-07-14T11:20:15.723192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:44:48.912804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759017+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:20:15.759017+0000","last_peered":"2023-07-14T11:20:15.759017+0000","last_clean":"2023-07-14T11:20:15.759017+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:20:15.759017+0000","last_undegraded":"2023-07-14T11:20:15.759017+0000","last_fullsized":"2023-07-14T11:20:15.759017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:30:26.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:18.689589+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:18.689589+0000","last_peered":"2023-07-14T11:20:18.689589+0000","last_clean":"2023-07-14T11:20:18.689589+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:18.689589+0000","last_undegraded":"2023-07-14T11:20:18.689589+0000","last_fullsized":"2023-07-14T11:20:18.689589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:54:46.013281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726387+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:20:15.726387+0000","last_peered":"2023-07-14T11:20:15.726387+0000","last_clean":"2023-07-14T11:20:15.726387+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:20:15.726387+0000","last_undegraded":"2023-07-14T11:20:15.726387+0000","last_fullsized":"2023-07-14T11:20:15.726387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:45.551283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761576+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:20:15.761576+0000","last_peered":"2023-07-14T11:20:15.761576+0000","last_clean":"2023-07-14T11:20:15.761576+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:20:15.761576+0000","last_undegraded":"2023-07-14T11:20:15.761576+0000","last_fullsized":"2023-07-14T11:20:15.761576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:35.428242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725019+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:15.725019+0000","last_peered":"2023-07-14T11:20:15.725019+0000","last_clean":"2023-07-14T11:20:15.725019+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:15.725019+0000","last_undegraded":"2023-07-14T11:20:15.725019+0000","last_fullsized":"2023-07-14T11:20:15.725019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:42:08.666520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725963+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:20:15.725963+0000","last_peered":"2023-07-14T11:20:15.725963+0000","last_clean":"2023-07-14T11:20:15.725963+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:20:15.725963+0000","last_undegraded":"2023-07-14T11:20:15.725963+0000","last_fullsized":"2023-07-14T11:20:15.725963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:49.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759565+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:20:15.759565+0000","last_peered":"2023-07-14T11:20:15.759565+0000","last_clean":"2023-07-14T11:20:15.759565+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:20:15.759565+0000","last_undegraded":"2023-07-14T11:20:15.759565+0000","last_fullsized":"2023-07-14T11:20:15.759565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:31.340558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722788+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:15.722788+0000","last_peered":"2023-07-14T11:20:15.722788+0000","last_clean":"2023-07-14T11:20:15.722788+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:15.722788+0000","last_undegraded":"2023-07-14T11:20:15.722788+0000","last_fullsized":"2023-07-14T11:20:15.722788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:40:45.755906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726149+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:15.726149+0000","last_peered":"2023-07-14T11:20:15.726149+0000","last_clean":"2023-07-14T11:20:15.726149+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:15.726149+0000","last_undegraded":"2023-07-14T11:20:15.726149+0000","last_fullsized":"2023-07-14T11:20:15.726149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:29.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725637+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:20:15.725637+0000","last_peered":"2023-07-14T11:20:15.725637+0000","last_clean":"2023-07-14T11:20:15.725637+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:20:15.725637+0000","last_undegraded":"2023-07-14T11:20:15.725637+0000","last_fullsized":"2023-07-14T11:20:15.725637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:47:19.355328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729053+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:15.729053+0000","last_peered":"2023-07-14T11:20:15.729053+0000","last_clean":"2023-07-14T11:20:15.729053+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:15.729053+0000","last_undegraded":"2023-07-14T11:20:15.729053+0000","last_fullsized":"2023-07-14T11:20:15.729053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:06:22.336930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723369+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:15.723369+0000","last_peered":"2023-07-14T11:20:15.723369+0000","last_clean":"2023-07-14T11:20:15.723369+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:15.723369+0000","last_undegraded":"2023-07-14T11:20:15.723369+0000","last_fullsized":"2023-07-14T11:20:15.723369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:47.388288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761518+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:20:15.761518+0000","last_peered":"2023-07-14T11:20:15.761518+0000","last_clean":"2023-07-14T11:20:15.761518+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:20:15.761518+0000","last_undegraded":"2023-07-14T11:20:15.761518+0000","last_fullsized":"2023-07-14T11:20:15.761518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:28.833104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764010+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:15.764010+0000","last_peered":"2023-07-14T11:20:15.764010+0000","last_clean":"2023-07-14T11:20:15.764010+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:15.764010+0000","last_undegraded":"2023-07-14T11:20:15.764010+0000","last_fullsized":"2023-07-14T11:20:15.764010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:18.046365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762296+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:20:15.762296+0000","last_peered":"2023-07-14T11:20:15.762296+0000","last_clean":"2023-07-14T11:20:15.762296+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:20:15.762296+0000","last_undegraded":"2023-07-14T11:20:15.762296+0000","last_fullsized":"2023-07-14T11:20:15.762296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:04:14.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725262+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:20:15.725262+0000","last_peered":"2023-07-14T11:20:15.725262+0000","last_clean":"2023-07-14T11:20:15.725262+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:20:15.725262+0000","last_undegraded":"2023-07-14T11:20:15.725262+0000","last_fullsized":"2023-07-14T11:20:15.725262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:40:44.958728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725446+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:15.725446+0000","last_peered":"2023-07-14T11:20:15.725446+0000","last_clean":"2023-07-14T11:20:15.725446+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:15.725446+0000","last_undegraded":"2023-07-14T11:20:15.725446+0000","last_fullsized":"2023-07-14T11:20:15.725446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.980346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729830+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:20:15.729830+0000","last_peered":"2023-07-14T11:20:15.729830+0000","last_clean":"2023-07-14T11:20:15.729830+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:20:15.729830+0000","last_undegraded":"2023-07-14T11:20:15.729830+0000","last_fullsized":"2023-07-14T11:20:15.729830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:23.746186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724337+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:20:15.724337+0000","last_peered":"2023-07-14T11:20:15.724337+0000","last_clean":"2023-07-14T11:20:15.724337+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:20:15.724337+0000","last_undegraded":"2023-07-14T11:20:15.724337+0000","last_fullsized":"2023-07-14T11:20:15.724337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:32.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728279+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:15.728279+0000","last_peered":"2023-07-14T11:20:15.728279+0000","last_clean":"2023-07-14T11:20:15.728279+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:15.728279+0000","last_undegraded":"2023-07-14T11:20:15.728279+0000","last_fullsized":"2023-07-14T11:20:15.728279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:32.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763590+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:20:15.763590+0000","last_peered":"2023-07-14T11:20:15.763590+0000","last_clean":"2023-07-14T11:20:15.763590+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:20:15.763590+0000","last_undegraded":"2023-07-14T11:20:15.763590+0000","last_fullsized":"2023-07-14T11:20:15.763590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:06:43.200171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761544+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:15.761544+0000","last_peered":"2023-07-14T11:20:15.761544+0000","last_clean":"2023-07-14T11:20:15.761544+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:15.761544+0000","last_undegraded":"2023-07-14T11:20:15.761544+0000","last_fullsized":"2023-07-14T11:20:15.761544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:10:23.505930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:25.389747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725984+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:20:15.725984+0000","last_peered":"2023-07-14T11:20:15.725984+0000","last_clean":"2023-07-14T11:20:15.725984+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:20:15.725984+0000","last_undegraded":"2023-07-14T11:20:15.725984+0000","last_fullsized":"2023-07-14T11:20:15.725984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:28.285910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725493+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:20:15.725493+0000","last_peered":"2023-07-14T11:20:15.725493+0000","last_clean":"2023-07-14T11:20:15.725493+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:20:15.725493+0000","last_undegraded":"2023-07-14T11:20:15.725493+0000","last_fullsized":"2023-07-14T11:20:15.725493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:49.791341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761896+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:20:15.761896+0000","last_peered":"2023-07-14T11:20:15.761896+0000","last_clean":"2023-07-14T11:20:15.761896+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:20:15.761896+0000","last_undegraded":"2023-07-14T11:20:15.761896+0000","last_fullsized":"2023-07-14T11:20:15.761896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:04:06.442323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724354+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:20:15.724354+0000","last_peered":"2023-07-14T11:20:15.724354+0000","last_clean":"2023-07-14T11:20:15.724354+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:20:15.724354+0000","last_undegraded":"2023-07-14T11:20:15.724354+0000","last_fullsized":"2023-07-14T11:20:15.724354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:31.530941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728516+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:15.728516+0000","last_peered":"2023-07-14T11:20:15.728516+0000","last_clean":"2023-07-14T11:20:15.728516+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:15.728516+0000","last_undegraded":"2023-07-14T11:20:15.728516+0000","last_fullsized":"2023-07-14T11:20:15.728516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:29.709167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727131+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:20:15.727131+0000","last_peered":"2023-07-14T11:20:15.727131+0000","last_clean":"2023-07-14T11:20:15.727131+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:20:15.727131+0000","last_undegraded":"2023-07-14T11:20:15.727131+0000","last_fullsized":"2023-07-14T11:20:15.727131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:31:54.387366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727120+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:20:15.727120+0000","last_peered":"2023-07-14T11:20:15.727120+0000","last_clean":"2023-07-14T11:20:15.727120+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:20:15.727120+0000","last_undegraded":"2023-07-14T11:20:15.727120+0000","last_fullsized":"2023-07-14T11:20:15.727120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:15.525008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726470+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:20:15.726470+0000","last_peered":"2023-07-14T11:20:15.726470+0000","last_clean":"2023-07-14T11:20:15.726470+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:20:15.726470+0000","last_undegraded":"2023-07-14T11:20:15.726470+0000","last_fullsized":"2023-07-14T11:20:15.726470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:03.965708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729884+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:20:15.729884+0000","last_peered":"2023-07-14T11:20:15.729884+0000","last_clean":"2023-07-14T11:20:15.729884+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:20:15.729884+0000","last_undegraded":"2023-07-14T11:20:15.729884+0000","last_fullsized":"2023-07-14T11:20:15.729884+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:31.118854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759707+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:20:15.759707+0000","last_peered":"2023-07-14T11:20:15.759707+0000","last_clean":"2023-07-14T11:20:15.759707+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:20:15.759707+0000","last_undegraded":"2023-07-14T11:20:15.759707+0000","last_fullsized":"2023-07-14T11:20:15.759707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:35.715875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766177+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:20:15.766177+0000","last_peered":"2023-07-14T11:20:15.766177+0000","last_clean":"2023-07-14T11:20:15.766177+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:20:15.766177+0000","last_undegraded":"2023-07-14T11:20:15.766177+0000","last_fullsized":"2023-07-14T11:20:15.766177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:59.517623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758537+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:20:15.758537+0000","last_peered":"2023-07-14T11:20:15.758537+0000","last_clean":"2023-07-14T11:20:15.758537+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:20:15.758537+0000","last_undegraded":"2023-07-14T11:20:15.758537+0000","last_fullsized":"2023-07-14T11:20:15.758537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.641650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723412+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:15.723412+0000","last_peered":"2023-07-14T11:20:15.723412+0000","last_clean":"2023-07-14T11:20:15.723412+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:15.723412+0000","last_undegraded":"2023-07-14T11:20:15.723412+0000","last_fullsized":"2023-07-14T11:20:15.723412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:35.428375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726258+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:20:15.726258+0000","last_peered":"2023-07-14T11:20:15.726258+0000","last_clean":"2023-07-14T11:20:15.726258+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:20:15.726258+0000","last_undegraded":"2023-07-14T11:20:15.726258+0000","last_fullsized":"2023-07-14T11:20:15.726258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:41.949605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:48.091011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725591+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:20:15.725591+0000","last_peered":"2023-07-14T11:20:15.725591+0000","last_clean":"2023-07-14T11:20:15.725591+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:20:15.725591+0000","last_undegraded":"2023-07-14T11:20:15.725591+0000","last_fullsized":"2023-07-14T11:20:15.725591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:15.490002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726260+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:20:15.726260+0000","last_peered":"2023-07-14T11:20:15.726260+0000","last_clean":"2023-07-14T11:20:15.726260+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:20:15.726260+0000","last_undegraded":"2023-07-14T11:20:15.726260+0000","last_fullsized":"2023-07-14T11:20:15.726260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:53:08.815873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726407+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:20:15.726407+0000","last_peered":"2023-07-14T11:20:15.726407+0000","last_clean":"2023-07-14T11:20:15.726407+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:20:15.726407+0000","last_undegraded":"2023-07-14T11:20:15.726407+0000","last_fullsized":"2023-07-14T11:20:15.726407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:08.673713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723496+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:15.723496+0000","last_peered":"2023-07-14T11:20:15.723496+0000","last_clean":"2023-07-14T11:20:15.723496+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:15.723496+0000","last_undegraded":"2023-07-14T11:20:15.723496+0000","last_fullsized":"2023-07-14T11:20:15.723496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:41:54.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726094+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:20:15.726094+0000","last_peered":"2023-07-14T11:20:15.726094+0000","last_clean":"2023-07-14T11:20:15.726094+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:20:15.726094+0000","last_undegraded":"2023-07-14T11:20:15.726094+0000","last_fullsized":"2023-07-14T11:20:15.726094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:36:46.120312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725900+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:20:15.725900+0000","last_peered":"2023-07-14T11:20:15.725900+0000","last_clean":"2023-07-14T11:20:15.725900+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:20:15.725900+0000","last_undegraded":"2023-07-14T11:20:15.725900+0000","last_fullsized":"2023-07-14T11:20:15.725900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:29.569291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762498+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:20:15.762498+0000","last_peered":"2023-07-14T11:20:15.762498+0000","last_clean":"2023-07-14T11:20:15.762498+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:20:15.762498+0000","last_undegraded":"2023-07-14T11:20:15.762498+0000","last_fullsized":"2023-07-14T11:20:15.762498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:46.020055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765762+0000","last_change":"2023-07-14T11:20:15.765762+0000","last_active":"2023-07-14T11:20:15.765762+0000","last_peered":"2023-07-14T11:20:15.765762+0000","last_clean":"2023-07-14T11:20:15.765762+0000","last_became_active":"2023-07-14T11:20:15.765349+0000","last_became_peered":"2023-07-14T11:20:15.765349+0000","last_unstale":"2023-07-14T11:20:15.765762+0000","last_undegraded":"2023-07-14T11:20:15.765762+0000","last_fullsized":"2023-07-14T11:20:15.765762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:51:54.292936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761928+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:20:15.761928+0000","last_peered":"2023-07-14T11:20:15.761928+0000","last_clean":"2023-07-14T11:20:15.761928+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:20:15.761928+0000","last_undegraded":"2023-07-14T11:20:15.761928+0000","last_fullsized":"2023-07-14T11:20:15.761928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:50:15.332355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723311+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:15.723311+0000","last_peered":"2023-07-14T11:20:15.723311+0000","last_clean":"2023-07-14T11:20:15.723311+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:15.723311+0000","last_undegraded":"2023-07-14T11:20:15.723311+0000","last_fullsized":"2023-07-14T11:20:15.723311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:13.493038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725897+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:20:15.725897+0000","last_peered":"2023-07-14T11:20:15.725897+0000","last_clean":"2023-07-14T11:20:15.725897+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:20:15.725897+0000","last_undegraded":"2023-07-14T11:20:15.725897+0000","last_fullsized":"2023-07-14T11:20:15.725897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:20.934793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46642,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":58,"num_read_kb":80,"num_write":144,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":165334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":490869,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.62},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.80100000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.84599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.875}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1105920,"data_stored":1121217,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.377}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.501},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.63500000000000001}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.751},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.86399999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":495201,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":1.012,"15min":1.012},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.752},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.997,"15min":1.0489999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":0.999,"15min":0.999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0880000000000001,"5min":1.264,"15min":1.264},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.56499999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.999,"5min":1.073,"15min":1.073},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.84099999999999997,"15min":0.86299999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93100000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.872,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0409999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.70499999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.151,"15min":1.2090000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99299999999999999,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90100000000000002,"15min":0.91700000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.081,"5min":1.171,"15min":1.171},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.60299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":491025,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.754}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.94499999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":1.002}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.88}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93200000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.319}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.218}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1086116,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.373},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.5},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.442}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68999999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.131,"15min":1.131},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486008,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.087}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.04}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.89800000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":529319,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64900000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.71699999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54400000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.873}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":487396,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.623},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.745}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.80700000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.85899999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.875}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":530348,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81499999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.749}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486475,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.50600000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.59999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96699999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.98599999999999999,"15min":1.0169999999999999},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":4.157,"15min":4.157},"last":0.58899999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490883,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.374}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.498}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82499999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.438}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.40100000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59799999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47199999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.623}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1081752,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.377},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.53500000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.626}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.747}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":55832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:23.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:23 smithi161 ceph-mon[95526]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:23.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:23 smithi161 ceph-mon[95526]: pgmap v294: 129 pgs: 129 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2023-07-14T11:20:23.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:23 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3659809970' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-07-14T11:20:23.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:23 smithi182 ceph-mon[129366]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:23.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:23 smithi182 ceph-mon[129366]: pgmap v294: 129 pgs: 129 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2023-07-14T11:20:23.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:23 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3659809970' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-07-14T11:20:23.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:23 smithi175 ceph-mon[129081]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:23 smithi175 ceph-mon[129081]: pgmap v294: 129 pgs: 129 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2023-07-14T11:20:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:23 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3659809970' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-07-14T11:20:24.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:24 smithi182 ceph-mon[129366]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:24.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:24 smithi161 ceph-mon[95526]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:24.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:24 smithi175 ceph-mon[129081]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:25.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:25 smithi182 ceph-mon[129366]: pgmap v295: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2023-07-14T11:20:25.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:25 smithi161 ceph-mon[95526]: pgmap v295: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2023-07-14T11:20:25.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:25 smithi175 ceph-mon[129081]: pgmap v295: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2023-07-14T11:20:25.939 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:20:26.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.148+0000 7fc49d1ff700 1 -- 172.21.15.161:0/1354325018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 msgr2=0x7fc498107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:26.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.148+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/1354325018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498107f40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b0a0 tx=0x7fc49000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.149+0000 7fc49d1ff700 1 -- 172.21.15.161:0/1354325018 shutdown_connections 2023-07-14T11:20:26.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.149+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/1354325018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498107f40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.149+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/1354325018 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4980ffb00 0x7fc4980fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.149+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/1354325018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498100cf0 0x7fc498101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.149+0000 7fc49d1ff700 1 -- 172.21.15.161:0/1354325018 >> 172.21.15.161:0/1354325018 conn(0x7fc4980fb0e0 msgr2=0x7fc4980fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:26.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.149+0000 7fc49d1ff700 1 -- 172.21.15.161:0/1354325018 shutdown_connections 2023-07-14T11:20:26.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.149+0000 7fc49d1ff700 1 -- 172.21.15.161:0/1354325018 wait complete. 2023-07-14T11:20:26.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 Processor -- start 2023-07-14T11:20:26.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 -- start start 2023-07-14T11:20:26.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4980ffb00 0x7fc4980742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:26.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498100cf0 0x7fc498072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:26.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:26.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980735e0 con 0x7fc498101680 2023-07-14T11:20:26.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc498073750 con 0x7fc498100cf0 2023-07-14T11:20:26.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49d1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49819f930 con 0x7fc4980ffb00 2023-07-14T11:20:26.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38026/0 (socket says 172.21.15.161:38026) 2023-07-14T11:20:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49759e700 1 -- 172.21.15.161:0/2777656111 learned_addr learned my addr 172.21.15.161:0/2777656111 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:26.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc496d9d700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4980ffb00 0x7fc4980742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.150+0000 7fc49659c700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498100cf0 0x7fc498072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49759e700 1 -- 172.21.15.161:0/2777656111 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4980ffb00 msgr2=0x7fc4980742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49759e700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4980ffb00 0x7fc4980742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49759e700 1 -- 172.21.15.161:0/2777656111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498100cf0 msgr2=0x7fc498072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:26.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49759e700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498100cf0 0x7fc498072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49759e700 1 -- 172.21.15.161:0/2777656111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488009520 con 0x7fc498101680 2023-07-14T11:20:26.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49759e700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498072e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc49000d010 tx=0x7fc49000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:26.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc47ffff700 1 -- 172.21.15.161:0/2777656111 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49000e040 con 0x7fc498101680 2023-07-14T11:20:26.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc490009d00 con 0x7fc498101680 2023-07-14T11:20:26.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc47ffff700 1 -- 172.21.15.161:0/2777656111 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc490003e10 con 0x7fc498101680 2023-07-14T11:20:26.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc47ffff700 1 -- 172.21.15.161:0/2777656111 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900052a0 con 0x7fc498101680 2023-07-14T11:20:26.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.151+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49819fdf0 con 0x7fc498101680 2023-07-14T11:20:26.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.152+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc498106fa0 con 0x7fc498101680 2023-07-14T11:20:26.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.154+0000 7fc47ffff700 1 -- 172.21.15.161:0/2777656111 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc49001d480 con 0x7fc498101680 2023-07-14T11:20:26.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.156+0000 7fc47ffff700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc4800a1220 0x7fc4800a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:26.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.156+0000 7fc47ffff700 1 -- 172.21.15.161:0/2777656111 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc4900ce160 con 0x7fc498101680 2023-07-14T11:20:26.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.156+0000 7fc496d9d700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc4800a1220 0x7fc4800a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:26.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.157+0000 7fc496d9d700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc4800a1220 0x7fc4800a36d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc498100b50 tx=0x7fc488009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:26.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.159+0000 7fc47ffff700 1 -- 172.21.15.161:0/2777656111 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc490097470 con 0x7fc498101680 2023-07-14T11:20:26.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.356+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc498060950 con 0x7fc4800a1220 2023-07-14T11:20:26.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.369+0000 7fc47ffff700 1 -- 172.21.15.161:0/2777656111 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422745 (secure 0 0 0) 0x7fc498060950 con 0x7fc4800a1220 2023-07-14T11:20:26.376 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:26.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc4800a1220 msgr2=0x7fc4800a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:26.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc4800a1220 0x7fc4800a36d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc498100b50 tx=0x7fc488009260 comp rx=0 tx=0).stop 2023-07-14T11:20:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 msgr2=0x7fc498072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498072e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc49000d010 tx=0x7fc49000bc80 comp rx=0 tx=0).stop 2023-07-14T11:20:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 shutdown_connections 2023-07-14T11:20:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc498101680 0x7fc498072e40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc4800a1220 0x7fc4800a36d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4980ffb00 0x7fc4980742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 --2- 172.21.15.161:0/2777656111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498100cf0 0x7fc498072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 >> 172.21.15.161:0/2777656111 conn(0x7fc4980fb0e0 msgr2=0x7fc4981040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 shutdown_connections 2023-07-14T11:20:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:26.374+0000 7fc49d1ff700 1 -- 172.21.15.161:0/2777656111 wait complete. 2023-07-14T11:20:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:20:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-07-14T11:20:26.106721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649960,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":225,"num_read_kb":227,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335340,"kb_used_data":11916,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606484,"statfs":{"total":1151940427776,"available":1151597039616,"internally_reserved":0,"allocated":12201984,"data_stored":7802017,"data_compressed":318627,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52743,"num_objects":5,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":69,"num_read_kb":90,"num_write":30,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.385099"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726364+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:20:15.726364+0000","last_peered":"2023-07-14T11:20:15.726364+0000","last_clean":"2023-07-14T11:20:15.726364+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:20:15.726364+0000","last_undegraded":"2023-07-14T11:20:15.726364+0000","last_fullsized":"2023-07-14T11:20:15.726364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:25.074131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726699+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:20:15.726699+0000","last_peered":"2023-07-14T11:20:15.726699+0000","last_clean":"2023-07-14T11:20:15.726699+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:20:15.726699+0000","last_undegraded":"2023-07-14T11:20:15.726699+0000","last_fullsized":"2023-07-14T11:20:15.726699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:41.854592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726554+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:20:15.726554+0000","last_peered":"2023-07-14T11:20:15.726554+0000","last_clean":"2023-07-14T11:20:15.726554+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:20:15.726554+0000","last_undegraded":"2023-07-14T11:20:15.726554+0000","last_fullsized":"2023-07-14T11:20:15.726554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:37.342912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726181+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:20:15.726181+0000","last_peered":"2023-07-14T11:20:15.726181+0000","last_clean":"2023-07-14T11:20:15.726181+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:20:15.726181+0000","last_undegraded":"2023-07-14T11:20:15.726181+0000","last_fullsized":"2023-07-14T11:20:15.726181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:46.208987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728594+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:15.728594+0000","last_peered":"2023-07-14T11:20:15.728594+0000","last_clean":"2023-07-14T11:20:15.728594+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:15.728594+0000","last_undegraded":"2023-07-14T11:20:15.728594+0000","last_fullsized":"2023-07-14T11:20:15.728594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:53:26.823230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765846+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:20:15.765846+0000","last_peered":"2023-07-14T11:20:15.765846+0000","last_clean":"2023-07-14T11:20:15.765846+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:20:15.765846+0000","last_undegraded":"2023-07-14T11:20:15.765846+0000","last_fullsized":"2023-07-14T11:20:15.765846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:31:09.497877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725992+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:20:15.725992+0000","last_peered":"2023-07-14T11:20:15.725992+0000","last_clean":"2023-07-14T11:20:15.725992+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:20:15.725992+0000","last_undegraded":"2023-07-14T11:20:15.725992+0000","last_fullsized":"2023-07-14T11:20:15.725992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:18.893918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761828+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:20:15.761828+0000","last_peered":"2023-07-14T11:20:15.761828+0000","last_clean":"2023-07-14T11:20:15.761828+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:20:15.761828+0000","last_undegraded":"2023-07-14T11:20:15.761828+0000","last_fullsized":"2023-07-14T11:20:15.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730008+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:20:15.730008+0000","last_peered":"2023-07-14T11:20:15.730008+0000","last_clean":"2023-07-14T11:20:15.730008+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:20:15.730008+0000","last_undegraded":"2023-07-14T11:20:15.730008+0000","last_fullsized":"2023-07-14T11:20:15.730008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:25:58.920431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726577+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:20:15.726577+0000","last_peered":"2023-07-14T11:20:15.726577+0000","last_clean":"2023-07-14T11:20:15.726577+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:20:15.726577+0000","last_undegraded":"2023-07-14T11:20:15.726577+0000","last_fullsized":"2023-07-14T11:20:15.726577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:29.172086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726169+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:20:15.726169+0000","last_peered":"2023-07-14T11:20:15.726169+0000","last_clean":"2023-07-14T11:20:15.726169+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:20:15.726169+0000","last_undegraded":"2023-07-14T11:20:15.726169+0000","last_fullsized":"2023-07-14T11:20:15.726169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:29.746118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723416+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:15.723416+0000","last_peered":"2023-07-14T11:20:15.723416+0000","last_clean":"2023-07-14T11:20:15.723416+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:15.723416+0000","last_undegraded":"2023-07-14T11:20:15.723416+0000","last_fullsized":"2023-07-14T11:20:15.723416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:57.801423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725956+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:20:15.725956+0000","last_peered":"2023-07-14T11:20:15.725956+0000","last_clean":"2023-07-14T11:20:15.725956+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:20:15.725956+0000","last_undegraded":"2023-07-14T11:20:15.725956+0000","last_fullsized":"2023-07-14T11:20:15.725956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:01.673209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726027+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:20:15.726027+0000","last_peered":"2023-07-14T11:20:15.726027+0000","last_clean":"2023-07-14T11:20:15.726027+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:20:15.726027+0000","last_undegraded":"2023-07-14T11:20:15.726027+0000","last_fullsized":"2023-07-14T11:20:15.726027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:10.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726533+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:20:15.726533+0000","last_peered":"2023-07-14T11:20:15.726533+0000","last_clean":"2023-07-14T11:20:15.726533+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:20:15.726533+0000","last_undegraded":"2023-07-14T11:20:15.726533+0000","last_fullsized":"2023-07-14T11:20:15.726533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:33.733562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763940+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:20:15.763940+0000","last_peered":"2023-07-14T11:20:15.763940+0000","last_clean":"2023-07-14T11:20:15.763940+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:20:15.763940+0000","last_undegraded":"2023-07-14T11:20:15.763940+0000","last_fullsized":"2023-07-14T11:20:15.763940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:28:49.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725800+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:20:15.725800+0000","last_peered":"2023-07-14T11:20:15.725800+0000","last_clean":"2023-07-14T11:20:15.725800+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:20:15.725800+0000","last_undegraded":"2023-07-14T11:20:15.725800+0000","last_fullsized":"2023-07-14T11:20:15.725800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:30.183181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726357+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:20:15.726357+0000","last_peered":"2023-07-14T11:20:15.726357+0000","last_clean":"2023-07-14T11:20:15.726357+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:20:15.726357+0000","last_undegraded":"2023-07-14T11:20:15.726357+0000","last_fullsized":"2023-07-14T11:20:15.726357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:13.424624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763774+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:15.763774+0000","last_peered":"2023-07-14T11:20:15.763774+0000","last_clean":"2023-07-14T11:20:15.763774+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:15.763774+0000","last_undegraded":"2023-07-14T11:20:15.763774+0000","last_fullsized":"2023-07-14T11:20:15.763774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:54.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762089+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:20:15.762089+0000","last_peered":"2023-07-14T11:20:15.762089+0000","last_clean":"2023-07-14T11:20:15.762089+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:20:15.762089+0000","last_undegraded":"2023-07-14T11:20:15.762089+0000","last_fullsized":"2023-07-14T11:20:15.762089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:06.259730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724677+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:20:15.724677+0000","last_peered":"2023-07-14T11:20:15.724677+0000","last_clean":"2023-07-14T11:20:15.724677+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:20:15.724677+0000","last_undegraded":"2023-07-14T11:20:15.724677+0000","last_fullsized":"2023-07-14T11:20:15.724677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:07.076184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762219+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:20:15.762219+0000","last_peered":"2023-07-14T11:20:15.762219+0000","last_clean":"2023-07-14T11:20:15.762219+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:20:15.762219+0000","last_undegraded":"2023-07-14T11:20:15.762219+0000","last_fullsized":"2023-07-14T11:20:15.762219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:30.071523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726815+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:20:15.726815+0000","last_peered":"2023-07-14T11:20:15.726815+0000","last_clean":"2023-07-14T11:20:15.726815+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:20:15.726815+0000","last_undegraded":"2023-07-14T11:20:15.726815+0000","last_fullsized":"2023-07-14T11:20:15.726815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:41:41.007835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724865+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:20:15.724865+0000","last_peered":"2023-07-14T11:20:15.724865+0000","last_clean":"2023-07-14T11:20:15.724865+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:20:15.724865+0000","last_undegraded":"2023-07-14T11:20:15.724865+0000","last_fullsized":"2023-07-14T11:20:15.724865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:56:28.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729360+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.729360+0000","last_peered":"2023-07-14T11:20:15.729360+0000","last_clean":"2023-07-14T11:20:15.729360+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:15.729360+0000","last_undegraded":"2023-07-14T11:20:15.729360+0000","last_fullsized":"2023-07-14T11:20:15.729360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:54.894820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726665+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:20:15.726665+0000","last_peered":"2023-07-14T11:20:15.726665+0000","last_clean":"2023-07-14T11:20:15.726665+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:20:15.726665+0000","last_undegraded":"2023-07-14T11:20:15.726665+0000","last_fullsized":"2023-07-14T11:20:15.726665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:39.791849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728688+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:15.728688+0000","last_peered":"2023-07-14T11:20:15.728688+0000","last_clean":"2023-07-14T11:20:15.728688+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:15.728688+0000","last_undegraded":"2023-07-14T11:20:15.728688+0000","last_fullsized":"2023-07-14T11:20:15.728688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:23:54.582861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759435+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:20:15.759435+0000","last_peered":"2023-07-14T11:20:15.759435+0000","last_clean":"2023-07-14T11:20:15.759435+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:20:15.759435+0000","last_undegraded":"2023-07-14T11:20:15.759435+0000","last_fullsized":"2023-07-14T11:20:15.759435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:51.343442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725567+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:20:15.725567+0000","last_peered":"2023-07-14T11:20:15.725567+0000","last_clean":"2023-07-14T11:20:15.725567+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:20:15.725567+0000","last_undegraded":"2023-07-14T11:20:15.725567+0000","last_fullsized":"2023-07-14T11:20:15.725567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:50.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:21.693348+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:20:21.693348+0000","last_peered":"2023-07-14T11:20:21.693348+0000","last_clean":"2023-07-14T11:20:21.693348+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:20:21.693348+0000","last_undegraded":"2023-07-14T11:20:21.693348+0000","last_fullsized":"2023-07-14T11:20:21.693348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:41.232654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728839+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:15.728839+0000","last_peered":"2023-07-14T11:20:15.728839+0000","last_clean":"2023-07-14T11:20:15.728839+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:15.728839+0000","last_undegraded":"2023-07-14T11:20:15.728839+0000","last_fullsized":"2023-07-14T11:20:15.728839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:12.092163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766231+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:20:15.766231+0000","last_peered":"2023-07-14T11:20:15.766231+0000","last_clean":"2023-07-14T11:20:15.766231+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:20:15.766231+0000","last_undegraded":"2023-07-14T11:20:15.766231+0000","last_fullsized":"2023-07-14T11:20:15.766231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:16:19.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730104+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:15.730104+0000","last_peered":"2023-07-14T11:20:15.730104+0000","last_clean":"2023-07-14T11:20:15.730104+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:15.730104+0000","last_undegraded":"2023-07-14T11:20:15.730104+0000","last_fullsized":"2023-07-14T11:20:15.730104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:54:17.106221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759228+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:20:15.759228+0000","last_peered":"2023-07-14T11:20:15.759228+0000","last_clean":"2023-07-14T11:20:15.759228+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:20:15.759228+0000","last_undegraded":"2023-07-14T11:20:15.759228+0000","last_fullsized":"2023-07-14T11:20:15.759228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:47.650473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726249+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:20:15.726249+0000","last_peered":"2023-07-14T11:20:15.726249+0000","last_clean":"2023-07-14T11:20:15.726249+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:20:15.726249+0000","last_undegraded":"2023-07-14T11:20:15.726249+0000","last_fullsized":"2023-07-14T11:20:15.726249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:10.265432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722943+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:15.722943+0000","last_peered":"2023-07-14T11:20:15.722943+0000","last_clean":"2023-07-14T11:20:15.722943+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:15.722943+0000","last_undegraded":"2023-07-14T11:20:15.722943+0000","last_fullsized":"2023-07-14T11:20:15.722943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:48.217979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761668+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:20:15.761668+0000","last_peered":"2023-07-14T11:20:15.761668+0000","last_clean":"2023-07-14T11:20:15.761668+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:20:15.761668+0000","last_undegraded":"2023-07-14T11:20:15.761668+0000","last_fullsized":"2023-07-14T11:20:15.761668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.096155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726651+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:20:15.726651+0000","last_peered":"2023-07-14T11:20:15.726651+0000","last_clean":"2023-07-14T11:20:15.726651+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:20:15.726651+0000","last_undegraded":"2023-07-14T11:20:15.726651+0000","last_fullsized":"2023-07-14T11:20:15.726651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:36.842168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758847+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:20:15.758847+0000","last_peered":"2023-07-14T11:20:15.758847+0000","last_clean":"2023-07-14T11:20:15.758847+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:20:15.758847+0000","last_undegraded":"2023-07-14T11:20:15.758847+0000","last_fullsized":"2023-07-14T11:20:15.758847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:38.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763190+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:20:15.763190+0000","last_peered":"2023-07-14T11:20:15.763190+0000","last_clean":"2023-07-14T11:20:15.763190+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:20:15.763190+0000","last_undegraded":"2023-07-14T11:20:15.763190+0000","last_fullsized":"2023-07-14T11:20:15.763190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:23.015078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761792+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:20:15.761792+0000","last_peered":"2023-07-14T11:20:15.761792+0000","last_clean":"2023-07-14T11:20:15.761792+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:20:15.761792+0000","last_undegraded":"2023-07-14T11:20:15.761792+0000","last_fullsized":"2023-07-14T11:20:15.761792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.645359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:20.691549+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:20:20.691549+0000","last_peered":"2023-07-14T11:20:20.691549+0000","last_clean":"2023-07-14T11:20:20.691549+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:20:20.691549+0000","last_undegraded":"2023-07-14T11:20:20.691549+0000","last_fullsized":"2023-07-14T11:20:20.691549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.137485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763325+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:20:15.763325+0000","last_peered":"2023-07-14T11:20:15.763325+0000","last_clean":"2023-07-14T11:20:15.763325+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:20:15.763325+0000","last_undegraded":"2023-07-14T11:20:15.763325+0000","last_fullsized":"2023-07-14T11:20:15.763325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:28.186030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726329+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:15.726329+0000","last_peered":"2023-07-14T11:20:15.726329+0000","last_clean":"2023-07-14T11:20:15.726329+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:15.726329+0000","last_undegraded":"2023-07-14T11:20:15.726329+0000","last_fullsized":"2023-07-14T11:20:15.726329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:38.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726931+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:20:15.726931+0000","last_peered":"2023-07-14T11:20:15.726931+0000","last_clean":"2023-07-14T11:20:15.726931+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:20:15.726931+0000","last_undegraded":"2023-07-14T11:20:15.726931+0000","last_fullsized":"2023-07-14T11:20:15.726931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:58.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725362+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:20:15.725362+0000","last_peered":"2023-07-14T11:20:15.725362+0000","last_clean":"2023-07-14T11:20:15.725362+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:20:15.725362+0000","last_undegraded":"2023-07-14T11:20:15.725362+0000","last_fullsized":"2023-07-14T11:20:15.725362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:29.343315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759436+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:15.759436+0000","last_peered":"2023-07-14T11:20:15.759436+0000","last_clean":"2023-07-14T11:20:15.759436+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:15.759436+0000","last_undegraded":"2023-07-14T11:20:15.759436+0000","last_fullsized":"2023-07-14T11:20:15.759436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:17:07.579078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762929+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:20:15.762929+0000","last_peered":"2023-07-14T11:20:15.762929+0000","last_clean":"2023-07-14T11:20:15.762929+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:20:15.762929+0000","last_undegraded":"2023-07-14T11:20:15.762929+0000","last_fullsized":"2023-07-14T11:20:15.762929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:41.993201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:21.697966+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:20:21.697966+0000","last_peered":"2023-07-14T11:20:21.697966+0000","last_clean":"2023-07-14T11:20:21.697966+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:20:21.697966+0000","last_undegraded":"2023-07-14T11:20:21.697966+0000","last_fullsized":"2023-07-14T11:20:21.697966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:08:09.309573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55832,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":67,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728912+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:15.728912+0000","last_peered":"2023-07-14T11:20:15.728912+0000","last_clean":"2023-07-14T11:20:15.728912+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:15.728912+0000","last_undegraded":"2023-07-14T11:20:15.728912+0000","last_fullsized":"2023-07-14T11:20:15.728912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:40:20.286989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759883+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:20:15.759883+0000","last_peered":"2023-07-14T11:20:15.759883+0000","last_clean":"2023-07-14T11:20:15.759883+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:20:15.759883+0000","last_undegraded":"2023-07-14T11:20:15.759883+0000","last_fullsized":"2023-07-14T11:20:15.759883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:27.460710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726089+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:15.726089+0000","last_peered":"2023-07-14T11:20:15.726089+0000","last_clean":"2023-07-14T11:20:15.726089+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:15.726089+0000","last_undegraded":"2023-07-14T11:20:15.726089+0000","last_fullsized":"2023-07-14T11:20:15.726089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:33:58.333180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722891+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:15.722891+0000","last_peered":"2023-07-14T11:20:15.722891+0000","last_clean":"2023-07-14T11:20:15.722891+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:15.722891+0000","last_undegraded":"2023-07-14T11:20:15.722891+0000","last_fullsized":"2023-07-14T11:20:15.722891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:12.630498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.732150+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:20:15.732150+0000","last_peered":"2023-07-14T11:20:15.732150+0000","last_clean":"2023-07-14T11:20:15.732150+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:20:15.732150+0000","last_undegraded":"2023-07-14T11:20:15.732150+0000","last_fullsized":"2023-07-14T11:20:15.732150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:49.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759053+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:15.759053+0000","last_peered":"2023-07-14T11:20:15.759053+0000","last_clean":"2023-07-14T11:20:15.759053+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:15.759053+0000","last_undegraded":"2023-07-14T11:20:15.759053+0000","last_fullsized":"2023-07-14T11:20:15.759053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:20:45.016996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765761+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:20:15.765761+0000","last_peered":"2023-07-14T11:20:15.765761+0000","last_clean":"2023-07-14T11:20:15.765761+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:20:15.765761+0000","last_undegraded":"2023-07-14T11:20:15.765761+0000","last_fullsized":"2023-07-14T11:20:15.765761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:11.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763313+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:20:15.763313+0000","last_peered":"2023-07-14T11:20:15.763313+0000","last_clean":"2023-07-14T11:20:15.763313+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:20:15.763313+0000","last_undegraded":"2023-07-14T11:20:15.763313+0000","last_fullsized":"2023-07-14T11:20:15.763313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:41.779784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:21.694186+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:20:21.694186+0000","last_peered":"2023-07-14T11:20:21.694186+0000","last_clean":"2023-07-14T11:20:21.694186+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:20:21.694186+0000","last_undegraded":"2023-07-14T11:20:21.694186+0000","last_fullsized":"2023-07-14T11:20:21.694186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:35.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726189+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:20:15.726189+0000","last_peered":"2023-07-14T11:20:15.726189+0000","last_clean":"2023-07-14T11:20:15.726189+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:20:15.726189+0000","last_undegraded":"2023-07-14T11:20:15.726189+0000","last_fullsized":"2023-07-14T11:20:15.726189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:25.466461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723291+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:15.723291+0000","last_peered":"2023-07-14T11:20:15.723291+0000","last_clean":"2023-07-14T11:20:15.723291+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:15.723291+0000","last_undegraded":"2023-07-14T11:20:15.723291+0000","last_fullsized":"2023-07-14T11:20:15.723291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:05.698741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728325+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:15.728325+0000","last_peered":"2023-07-14T11:20:15.728325+0000","last_clean":"2023-07-14T11:20:15.728325+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:15.728325+0000","last_undegraded":"2023-07-14T11:20:15.728325+0000","last_fullsized":"2023-07-14T11:20:15.728325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:03:30.273008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758707+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:20:15.758707+0000","last_peered":"2023-07-14T11:20:15.758707+0000","last_clean":"2023-07-14T11:20:15.758707+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:20:15.758707+0000","last_undegraded":"2023-07-14T11:20:15.758707+0000","last_fullsized":"2023-07-14T11:20:15.758707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:32.059475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758890+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:20:15.758890+0000","last_peered":"2023-07-14T11:20:15.758890+0000","last_clean":"2023-07-14T11:20:15.758890+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:20:15.758890+0000","last_undegraded":"2023-07-14T11:20:15.758890+0000","last_fullsized":"2023-07-14T11:20:15.758890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:51.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729124+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:15.729124+0000","last_peered":"2023-07-14T11:20:15.729124+0000","last_clean":"2023-07-14T11:20:15.729124+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:15.729124+0000","last_undegraded":"2023-07-14T11:20:15.729124+0000","last_fullsized":"2023-07-14T11:20:15.729124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:06.994422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725218+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:20:15.725218+0000","last_peered":"2023-07-14T11:20:15.725218+0000","last_clean":"2023-07-14T11:20:15.725218+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:20:15.725218+0000","last_undegraded":"2023-07-14T11:20:15.725218+0000","last_fullsized":"2023-07-14T11:20:15.725218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:42.841223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.731601+0000","last_change":"2023-07-14T11:20:15.731601+0000","last_active":"2023-07-14T11:20:15.731601+0000","last_peered":"2023-07-14T11:20:15.731601+0000","last_clean":"2023-07-14T11:20:15.731601+0000","last_became_active":"2023-07-14T11:20:15.731121+0000","last_became_peered":"2023-07-14T11:20:15.731121+0000","last_unstale":"2023-07-14T11:20:15.731601+0000","last_undegraded":"2023-07-14T11:20:15.731601+0000","last_fullsized":"2023-07-14T11:20:15.731601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:58.090267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764473+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:20:15.764473+0000","last_peered":"2023-07-14T11:20:15.764473+0000","last_clean":"2023-07-14T11:20:15.764473+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:20:15.764473+0000","last_undegraded":"2023-07-14T11:20:15.764473+0000","last_fullsized":"2023-07-14T11:20:15.764473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:06.410233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729586+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:20:15.729586+0000","last_peered":"2023-07-14T11:20:15.729586+0000","last_clean":"2023-07-14T11:20:15.729586+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:20:15.729586+0000","last_undegraded":"2023-07-14T11:20:15.729586+0000","last_fullsized":"2023-07-14T11:20:15.729586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:59.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723225+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:15.723225+0000","last_peered":"2023-07-14T11:20:15.723225+0000","last_clean":"2023-07-14T11:20:15.723225+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:15.723225+0000","last_undegraded":"2023-07-14T11:20:15.723225+0000","last_fullsized":"2023-07-14T11:20:15.723225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:32.064021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726984+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.726984+0000","last_peered":"2023-07-14T11:20:15.726984+0000","last_clean":"2023-07-14T11:20:15.726984+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:20:15.726984+0000","last_undegraded":"2023-07-14T11:20:15.726984+0000","last_fullsized":"2023-07-14T11:20:15.726984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:38:06.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765912+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:20:15.765912+0000","last_peered":"2023-07-14T11:20:15.765912+0000","last_clean":"2023-07-14T11:20:15.765912+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:20:15.765912+0000","last_undegraded":"2023-07-14T11:20:15.765912+0000","last_fullsized":"2023-07-14T11:20:15.765912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:01:52.067140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723192+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:15.723192+0000","last_peered":"2023-07-14T11:20:15.723192+0000","last_clean":"2023-07-14T11:20:15.723192+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:15.723192+0000","last_undegraded":"2023-07-14T11:20:15.723192+0000","last_fullsized":"2023-07-14T11:20:15.723192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:44:48.912804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759017+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:20:15.759017+0000","last_peered":"2023-07-14T11:20:15.759017+0000","last_clean":"2023-07-14T11:20:15.759017+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:20:15.759017+0000","last_undegraded":"2023-07-14T11:20:15.759017+0000","last_fullsized":"2023-07-14T11:20:15.759017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:30:26.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:23.696426+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:20:23.696426+0000","last_peered":"2023-07-14T11:20:23.696426+0000","last_clean":"2023-07-14T11:20:23.696426+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:20:23.696426+0000","last_undegraded":"2023-07-14T11:20:23.696426+0000","last_fullsized":"2023-07-14T11:20:23.696426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:54:46.013281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726387+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:20:15.726387+0000","last_peered":"2023-07-14T11:20:15.726387+0000","last_clean":"2023-07-14T11:20:15.726387+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:20:15.726387+0000","last_undegraded":"2023-07-14T11:20:15.726387+0000","last_fullsized":"2023-07-14T11:20:15.726387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:45.551283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761576+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:20:15.761576+0000","last_peered":"2023-07-14T11:20:15.761576+0000","last_clean":"2023-07-14T11:20:15.761576+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:20:15.761576+0000","last_undegraded":"2023-07-14T11:20:15.761576+0000","last_fullsized":"2023-07-14T11:20:15.761576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:35.428242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725019+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:15.725019+0000","last_peered":"2023-07-14T11:20:15.725019+0000","last_clean":"2023-07-14T11:20:15.725019+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:15.725019+0000","last_undegraded":"2023-07-14T11:20:15.725019+0000","last_fullsized":"2023-07-14T11:20:15.725019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:42:08.666520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725963+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:20:15.725963+0000","last_peered":"2023-07-14T11:20:15.725963+0000","last_clean":"2023-07-14T11:20:15.725963+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:20:15.725963+0000","last_undegraded":"2023-07-14T11:20:15.725963+0000","last_fullsized":"2023-07-14T11:20:15.725963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:49.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759565+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:20:15.759565+0000","last_peered":"2023-07-14T11:20:15.759565+0000","last_clean":"2023-07-14T11:20:15.759565+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:20:15.759565+0000","last_undegraded":"2023-07-14T11:20:15.759565+0000","last_fullsized":"2023-07-14T11:20:15.759565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:31.340558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722788+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:15.722788+0000","last_peered":"2023-07-14T11:20:15.722788+0000","last_clean":"2023-07-14T11:20:15.722788+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:15.722788+0000","last_undegraded":"2023-07-14T11:20:15.722788+0000","last_fullsized":"2023-07-14T11:20:15.722788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:40:45.755906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726149+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:15.726149+0000","last_peered":"2023-07-14T11:20:15.726149+0000","last_clean":"2023-07-14T11:20:15.726149+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:15.726149+0000","last_undegraded":"2023-07-14T11:20:15.726149+0000","last_fullsized":"2023-07-14T11:20:15.726149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:29.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725637+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:20:15.725637+0000","last_peered":"2023-07-14T11:20:15.725637+0000","last_clean":"2023-07-14T11:20:15.725637+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:20:15.725637+0000","last_undegraded":"2023-07-14T11:20:15.725637+0000","last_fullsized":"2023-07-14T11:20:15.725637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:47:19.355328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729053+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:15.729053+0000","last_peered":"2023-07-14T11:20:15.729053+0000","last_clean":"2023-07-14T11:20:15.729053+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:15.729053+0000","last_undegraded":"2023-07-14T11:20:15.729053+0000","last_fullsized":"2023-07-14T11:20:15.729053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:06:22.336930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723369+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:15.723369+0000","last_peered":"2023-07-14T11:20:15.723369+0000","last_clean":"2023-07-14T11:20:15.723369+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:15.723369+0000","last_undegraded":"2023-07-14T11:20:15.723369+0000","last_fullsized":"2023-07-14T11:20:15.723369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:47.388288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761518+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:20:15.761518+0000","last_peered":"2023-07-14T11:20:15.761518+0000","last_clean":"2023-07-14T11:20:15.761518+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:20:15.761518+0000","last_undegraded":"2023-07-14T11:20:15.761518+0000","last_fullsized":"2023-07-14T11:20:15.761518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:28.833104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764010+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:15.764010+0000","last_peered":"2023-07-14T11:20:15.764010+0000","last_clean":"2023-07-14T11:20:15.764010+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:15.764010+0000","last_undegraded":"2023-07-14T11:20:15.764010+0000","last_fullsized":"2023-07-14T11:20:15.764010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:18.046365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762296+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:20:15.762296+0000","last_peered":"2023-07-14T11:20:15.762296+0000","last_clean":"2023-07-14T11:20:15.762296+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:20:15.762296+0000","last_undegraded":"2023-07-14T11:20:15.762296+0000","last_fullsized":"2023-07-14T11:20:15.762296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:04:14.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725262+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:20:15.725262+0000","last_peered":"2023-07-14T11:20:15.725262+0000","last_clean":"2023-07-14T11:20:15.725262+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:20:15.725262+0000","last_undegraded":"2023-07-14T11:20:15.725262+0000","last_fullsized":"2023-07-14T11:20:15.725262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:40:44.958728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725446+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:15.725446+0000","last_peered":"2023-07-14T11:20:15.725446+0000","last_clean":"2023-07-14T11:20:15.725446+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:15.725446+0000","last_undegraded":"2023-07-14T11:20:15.725446+0000","last_fullsized":"2023-07-14T11:20:15.725446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.980346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729830+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:20:15.729830+0000","last_peered":"2023-07-14T11:20:15.729830+0000","last_clean":"2023-07-14T11:20:15.729830+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:20:15.729830+0000","last_undegraded":"2023-07-14T11:20:15.729830+0000","last_fullsized":"2023-07-14T11:20:15.729830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:23.746186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724337+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:20:15.724337+0000","last_peered":"2023-07-14T11:20:15.724337+0000","last_clean":"2023-07-14T11:20:15.724337+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:20:15.724337+0000","last_undegraded":"2023-07-14T11:20:15.724337+0000","last_fullsized":"2023-07-14T11:20:15.724337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:32.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728279+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:15.728279+0000","last_peered":"2023-07-14T11:20:15.728279+0000","last_clean":"2023-07-14T11:20:15.728279+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:15.728279+0000","last_undegraded":"2023-07-14T11:20:15.728279+0000","last_fullsized":"2023-07-14T11:20:15.728279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:32.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763590+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:20:15.763590+0000","last_peered":"2023-07-14T11:20:15.763590+0000","last_clean":"2023-07-14T11:20:15.763590+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:20:15.763590+0000","last_undegraded":"2023-07-14T11:20:15.763590+0000","last_fullsized":"2023-07-14T11:20:15.763590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:06:43.200171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761544+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:15.761544+0000","last_peered":"2023-07-14T11:20:15.761544+0000","last_clean":"2023-07-14T11:20:15.761544+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:15.761544+0000","last_undegraded":"2023-07-14T11:20:15.761544+0000","last_fullsized":"2023-07-14T11:20:15.761544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:10:23.505930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:25.389747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725984+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:20:15.725984+0000","last_peered":"2023-07-14T11:20:15.725984+0000","last_clean":"2023-07-14T11:20:15.725984+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:20:15.725984+0000","last_undegraded":"2023-07-14T11:20:15.725984+0000","last_fullsized":"2023-07-14T11:20:15.725984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:28.285910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725493+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:20:15.725493+0000","last_peered":"2023-07-14T11:20:15.725493+0000","last_clean":"2023-07-14T11:20:15.725493+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:20:15.725493+0000","last_undegraded":"2023-07-14T11:20:15.725493+0000","last_fullsized":"2023-07-14T11:20:15.725493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:49.791341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761896+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:20:15.761896+0000","last_peered":"2023-07-14T11:20:15.761896+0000","last_clean":"2023-07-14T11:20:15.761896+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:20:15.761896+0000","last_undegraded":"2023-07-14T11:20:15.761896+0000","last_fullsized":"2023-07-14T11:20:15.761896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:04:06.442323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724354+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:20:15.724354+0000","last_peered":"2023-07-14T11:20:15.724354+0000","last_clean":"2023-07-14T11:20:15.724354+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:20:15.724354+0000","last_undegraded":"2023-07-14T11:20:15.724354+0000","last_fullsized":"2023-07-14T11:20:15.724354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:31.530941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728516+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:15.728516+0000","last_peered":"2023-07-14T11:20:15.728516+0000","last_clean":"2023-07-14T11:20:15.728516+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:15.728516+0000","last_undegraded":"2023-07-14T11:20:15.728516+0000","last_fullsized":"2023-07-14T11:20:15.728516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:29.709167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727131+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:20:15.727131+0000","last_peered":"2023-07-14T11:20:15.727131+0000","last_clean":"2023-07-14T11:20:15.727131+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:20:15.727131+0000","last_undegraded":"2023-07-14T11:20:15.727131+0000","last_fullsized":"2023-07-14T11:20:15.727131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:31:54.387366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727120+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:20:15.727120+0000","last_peered":"2023-07-14T11:20:15.727120+0000","last_clean":"2023-07-14T11:20:15.727120+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:20:15.727120+0000","last_undegraded":"2023-07-14T11:20:15.727120+0000","last_fullsized":"2023-07-14T11:20:15.727120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:15.525008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726470+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:20:15.726470+0000","last_peered":"2023-07-14T11:20:15.726470+0000","last_clean":"2023-07-14T11:20:15.726470+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:20:15.726470+0000","last_undegraded":"2023-07-14T11:20:15.726470+0000","last_fullsized":"2023-07-14T11:20:15.726470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:03.965708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729884+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:20:15.729884+0000","last_peered":"2023-07-14T11:20:15.729884+0000","last_clean":"2023-07-14T11:20:15.729884+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:20:15.729884+0000","last_undegraded":"2023-07-14T11:20:15.729884+0000","last_fullsized":"2023-07-14T11:20:15.729884+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:31.118854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759707+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:20:15.759707+0000","last_peered":"2023-07-14T11:20:15.759707+0000","last_clean":"2023-07-14T11:20:15.759707+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:20:15.759707+0000","last_undegraded":"2023-07-14T11:20:15.759707+0000","last_fullsized":"2023-07-14T11:20:15.759707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:35.715875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766177+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:20:15.766177+0000","last_peered":"2023-07-14T11:20:15.766177+0000","last_clean":"2023-07-14T11:20:15.766177+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:20:15.766177+0000","last_undegraded":"2023-07-14T11:20:15.766177+0000","last_fullsized":"2023-07-14T11:20:15.766177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:59.517623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758537+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:20:15.758537+0000","last_peered":"2023-07-14T11:20:15.758537+0000","last_clean":"2023-07-14T11:20:15.758537+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:20:15.758537+0000","last_undegraded":"2023-07-14T11:20:15.758537+0000","last_fullsized":"2023-07-14T11:20:15.758537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.641650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723412+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:15.723412+0000","last_peered":"2023-07-14T11:20:15.723412+0000","last_clean":"2023-07-14T11:20:15.723412+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:15.723412+0000","last_undegraded":"2023-07-14T11:20:15.723412+0000","last_fullsized":"2023-07-14T11:20:15.723412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:35.428375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726258+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:20:15.726258+0000","last_peered":"2023-07-14T11:20:15.726258+0000","last_clean":"2023-07-14T11:20:15.726258+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:20:15.726258+0000","last_undegraded":"2023-07-14T11:20:15.726258+0000","last_fullsized":"2023-07-14T11:20:15.726258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:41.949605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:48.091011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725591+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:20:15.725591+0000","last_peered":"2023-07-14T11:20:15.725591+0000","last_clean":"2023-07-14T11:20:15.725591+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:20:15.725591+0000","last_undegraded":"2023-07-14T11:20:15.725591+0000","last_fullsized":"2023-07-14T11:20:15.725591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:15.490002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726260+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:20:15.726260+0000","last_peered":"2023-07-14T11:20:15.726260+0000","last_clean":"2023-07-14T11:20:15.726260+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:20:15.726260+0000","last_undegraded":"2023-07-14T11:20:15.726260+0000","last_fullsized":"2023-07-14T11:20:15.726260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:53:08.815873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726407+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:20:15.726407+0000","last_peered":"2023-07-14T11:20:15.726407+0000","last_clean":"2023-07-14T11:20:15.726407+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:20:15.726407+0000","last_undegraded":"2023-07-14T11:20:15.726407+0000","last_fullsized":"2023-07-14T11:20:15.726407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:08.673713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723496+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:15.723496+0000","last_peered":"2023-07-14T11:20:15.723496+0000","last_clean":"2023-07-14T11:20:15.723496+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:15.723496+0000","last_undegraded":"2023-07-14T11:20:15.723496+0000","last_fullsized":"2023-07-14T11:20:15.723496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:41:54.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726094+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:20:15.726094+0000","last_peered":"2023-07-14T11:20:15.726094+0000","last_clean":"2023-07-14T11:20:15.726094+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:20:15.726094+0000","last_undegraded":"2023-07-14T11:20:15.726094+0000","last_fullsized":"2023-07-14T11:20:15.726094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:36:46.120312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725900+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:20:15.725900+0000","last_peered":"2023-07-14T11:20:15.725900+0000","last_clean":"2023-07-14T11:20:15.725900+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:20:15.725900+0000","last_undegraded":"2023-07-14T11:20:15.725900+0000","last_fullsized":"2023-07-14T11:20:15.725900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:29.569291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762498+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:20:15.762498+0000","last_peered":"2023-07-14T11:20:15.762498+0000","last_clean":"2023-07-14T11:20:15.762498+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:20:15.762498+0000","last_undegraded":"2023-07-14T11:20:15.762498+0000","last_fullsized":"2023-07-14T11:20:15.762498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:46.020055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765762+0000","last_change":"2023-07-14T11:20:15.765762+0000","last_active":"2023-07-14T11:20:15.765762+0000","last_peered":"2023-07-14T11:20:15.765762+0000","last_clean":"2023-07-14T11:20:15.765762+0000","last_became_active":"2023-07-14T11:20:15.765349+0000","last_became_peered":"2023-07-14T11:20:15.765349+0000","last_unstale":"2023-07-14T11:20:15.765762+0000","last_undegraded":"2023-07-14T11:20:15.765762+0000","last_fullsized":"2023-07-14T11:20:15.765762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:51:54.292936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761928+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:20:15.761928+0000","last_peered":"2023-07-14T11:20:15.761928+0000","last_clean":"2023-07-14T11:20:15.761928+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:20:15.761928+0000","last_undegraded":"2023-07-14T11:20:15.761928+0000","last_fullsized":"2023-07-14T11:20:15.761928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:50:15.332355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723311+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:15.723311+0000","last_peered":"2023-07-14T11:20:15.723311+0000","last_clean":"2023-07-14T11:20:15.723311+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:15.723311+0000","last_undegraded":"2023-07-14T11:20:15.723311+0000","last_fullsized":"2023-07-14T11:20:15.723311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:13.493038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725897+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:20:15.725897+0000","last_peered":"2023-07-14T11:20:15.725897+0000","last_clean":"2023-07-14T11:20:15.725897+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:20:15.725897+0000","last_undegraded":"2023-07-14T11:20:15.725897+0000","last_fullsized":"2023-07-14T11:20:15.725897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:20.934793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59346,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":89,"num_read_kb":108,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":178038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416405,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":490869,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.435},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.81699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.86199999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.755},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.73999999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1118208,"data_stored":1133921,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.373},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.498}]},{"osd":3,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.629},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.72199999999999998}]},{"osd":11,"last update":"Fri Jul 14 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.753},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.80900000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":495201,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":1.012,"15min":1.012},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.752},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.997,"15min":1.0489999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":0.999,"15min":0.999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0880000000000001,"5min":1.264,"15min":1.264},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.56499999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.999,"5min":1.073,"15min":1.073},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.84099999999999997,"15min":0.86299999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93100000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.872,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0409999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.70499999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.151,"15min":1.2090000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99299999999999999,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90100000000000002,"15min":0.91700000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.081,"5min":1.171,"15min":1.171},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.8900000000000001,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.60299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":491025,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.62}]},{"osd":1,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.1680000000000001,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64900000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45900000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":2.032,"15min":2.032},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.879,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93600000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.85999999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.876},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":1.038}]},{"osd":6,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.024,"15min":1.024},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.96699999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.94399999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.0880000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.022},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.0529999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1086116,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.374},{"interface":"front","average":{"1min":0.438,"5min":0.442,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58899999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85499999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.312,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41699999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.498},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.501,"15min":0.501},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.57099999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53600000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.131,"15min":1.131},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.84199999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486008,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.435}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.85899999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.59399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":1.135,"15min":1.135},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.78000000000000003,"15min":0.76800000000000002},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.077,"5min":1.115,"15min":1.115},"last":0.93600000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.82699999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.98999999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.052}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":542023,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.57299999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.749}]},{"osd":2,"last update":"Fri Jul 14 11:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.45900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.42599999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.443},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45900000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42599999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.625},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.47099999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72199999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81999999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":487396,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.998,"15min":0.998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.437,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55900000000000005},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.627},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.95699999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.83499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.996,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.874}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":1.014}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.96999999999999997}]},{"osd":11,"last update":"Fri Jul 14 11:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.054},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.0669999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":530348,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.748,"15min":0.748},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.747,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:19:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:19:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81499999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.749}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486475,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.35499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.60199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.66000000000000003,"15min":0.58999999999999997},"min":{"1min":0.59899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.76800000000000002,"15min":0.73399999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.50600000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.59999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96699999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.98599999999999999,"15min":1.0169999999999999},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":4.157,"15min":4.157},"last":0.58899999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490883,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.748}]},{"osd":3,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.437,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93400000000000005,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82499999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45900000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.47999999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69699999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1081752,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57799999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38200000000000001},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41699999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84899999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.78600000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.75},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.53500000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.52100000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":55832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:20:26.405 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T11:20:26.406 INFO:tasks.cephadm:Setting up client nodes... 2023-07-14T11:20:26.406 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-14T11:20:27.167 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:27.456 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:27 smithi161 ceph-mon[95526]: pgmap v296: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2023-07-14T11:20:27.456 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:27 smithi161 ceph-mon[95526]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:27 smithi182 ceph-mon[129366]: pgmap v296: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2023-07-14T11:20:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:27 smithi182 ceph-mon[129366]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:27 smithi175 ceph-mon[129081]: pgmap v296: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2023-07-14T11:20:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:27 smithi175 ceph-mon[129081]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:20:28.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 -- 172.21.15.161:0/2318539466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6414100db0 msgr2=0x7f6414101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 --2- 172.21.15.161:0/2318539466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6414100db0 0x7f6414101200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6408009990 tx=0x7f6408009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 -- 172.21.15.161:0/2318539466 shutdown_connections 2023-07-14T11:20:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 --2- 172.21.15.161:0/2318539466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6414101740 0x7f6414108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 --2- 172.21.15.161:0/2318539466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64140ffbb0 0x7f64140fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 --2- 172.21.15.161:0/2318539466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6414100db0 0x7f6414101200 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 -- 172.21.15.161:0/2318539466 >> 172.21.15.161:0/2318539466 conn(0x7f64140fb200 msgr2=0x7f64140fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:28.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.597+0000 7f6418c53700 1 -- 172.21.15.161:0/2318539466 shutdown_connections 2023-07-14T11:20:28.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.598+0000 7f6418c53700 1 -- 172.21.15.161:0/2318539466 wait complete. 2023-07-14T11:20:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.598+0000 7f6418c53700 1 Processor -- start 2023-07-14T11:20:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6418c53700 1 -- start start 2023-07-14T11:20:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6418c53700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64140ffbb0 0x7f64140742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6418c53700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6414100db0 0x7f6414072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6418c53700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6414101740 0x7f6414072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6418c53700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64140735c0 con 0x7f64140ffbb0 2023-07-14T11:20:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6418c53700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6414073730 con 0x7f6414101740 2023-07-14T11:20:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6418c53700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641419f8f0 con 0x7f6414100db0 2023-07-14T11:20:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f641259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64140ffbb0 0x7f64140742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f641259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64140ffbb0 0x7f64140742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50696/0 (socket says 172.21.15.161:50696) 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f641259c700 1 -- 172.21.15.161:0/4068945622 learned_addr learned my addr 172.21.15.161:0/4068945622 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6412d9d700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6414101740 0x7f6414072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.599+0000 7f6411d9b700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6414100db0 0x7f6414072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f641259c700 1 -- 172.21.15.161:0/4068945622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6414100db0 msgr2=0x7f6414072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f641259c700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6414100db0 0x7f6414072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f641259c700 1 -- 172.21.15.161:0/4068945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6414101740 msgr2=0x7f6414072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f641259c700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6414101740 0x7f6414072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f641259c700 1 -- 172.21.15.161:0/4068945622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080095f0 con 0x7f64140ffbb0 2023-07-14T11:20:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f6412d9d700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6414101740 0x7f6414072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f641259c700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64140ffbb0 0x7f64140742b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f640400d980 tx=0x7f640400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f64037fe700 1 -- 172.21.15.161:0/4068945622 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400b950 con 0x7f64140ffbb0 2023-07-14T11:20:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641419fbd0 con 0x7f64140ffbb0 2023-07-14T11:20:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f64037fe700 1 -- 172.21.15.161:0/4068945622 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6404007620 con 0x7f64140ffbb0 2023-07-14T11:20:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f64037fe700 1 -- 172.21.15.161:0/4068945622 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400f3c0 con 0x7f64140ffbb0 2023-07-14T11:20:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.600+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64141a0120 con 0x7f64140ffbb0 2023-07-14T11:20:28.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.602+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6414106af0 con 0x7f64140ffbb0 2023-07-14T11:20:28.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.603+0000 7f64037fe700 1 -- 172.21.15.161:0/4068945622 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6404010640 con 0x7f64140ffbb0 2023-07-14T11:20:28.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.605+0000 7f64037fe700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63fc0a11e0 0x7f63fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:28.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.605+0000 7f64037fe700 1 -- 172.21.15.161:0/4068945622 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f64040cccf0 con 0x7f64140ffbb0 2023-07-14T11:20:28.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.605+0000 7f6411d9b700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63fc0a11e0 0x7f63fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:28.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.606+0000 7f6411d9b700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63fc0a11e0 0x7f63fc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f640800b2e0 tx=0x7f6408005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:28.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.609+0000 7f64037fe700 1 -- 172.21.15.161:0/4068945622 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6404095f70 con 0x7f64140ffbb0 2023-07-14T11:20:28.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.881+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f641402c850 con 0x7f64140ffbb0 2023-07-14T11:20:28.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.886+0000 7f64037fe700 1 -- 172.21.15.161:0/4068945622 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6404099590 con 0x7f64140ffbb0 2023-07-14T11:20:28.889 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2023-07-14T11:20:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQB8L7FkCYqfNBAAniViPvpttzkyPi0XwA0d3A== 2023-07-14T11:20:28.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63fc0a11e0 msgr2=0x7f63fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:28.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63fc0a11e0 0x7f63fc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f640800b2e0 tx=0x7f6408005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64140ffbb0 msgr2=0x7f64140742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:28.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64140ffbb0 0x7f64140742b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f640400d980 tx=0x7f640400dc90 comp rx=0 tx=0).stop 2023-07-14T11:20:28.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 shutdown_connections 2023-07-14T11:20:28.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64140ffbb0 0x7f64140742b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63fc0a11e0 0x7f63fc0a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6414101740 0x7f6414072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 --2- 172.21.15.161:0/4068945622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6414100db0 0x7f6414072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:28.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 >> 172.21.15.161:0/4068945622 conn(0x7f64140fb200 msgr2=0x7f64140fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:28.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 shutdown_connections 2023-07-14T11:20:28.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:28.889+0000 7f6418c53700 1 -- 172.21.15.161:0/4068945622 wait complete. 2023-07-14T11:20:29.362 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:20:29.362 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-14T11:20:29.362 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-14T11:20:29.465 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-14T11:20:29.466 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-14T11:20:29.466 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph mgr dump --format=json 2023-07-14T11:20:29.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:29 smithi182 ceph-mon[129366]: pgmap v297: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-07-14T11:20:29.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:29 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4068945622' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T11:20:29.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:29 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4068945622' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T11:20:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:29 smithi161 ceph-mon[95526]: pgmap v297: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-07-14T11:20:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:29 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4068945622' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T11:20:29.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:29 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4068945622' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T11:20:30.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:29 smithi175 ceph-mon[129081]: pgmap v297: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-07-14T11:20:30.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:29 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4068945622' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T11:20:30.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:29 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4068945622' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T11:20:30.231 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:31.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2175714022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc1007a0 msgr2=0x7ff1dc104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:31.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2175714022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc104770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00099c0 tx=0x7ff1d0009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:31.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2175714022 shutdown_connections 2023-07-14T11:20:31.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2175714022 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff1dc104ea0 0x7ff1dc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:31.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2175714022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc0ffe00 0x7ff1dc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:31.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2175714022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc104770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:31.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2175714022 >> 172.21.15.161:0/2175714022 conn(0x7ff1dc0fb660 msgr2=0x7ff1dc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:31.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2175714022 shutdown_connections 2023-07-14T11:20:31.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.732+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2175714022 wait complete. 2023-07-14T11:20:31.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.733+0000 7ff1e1a00700 1 Processor -- start 2023-07-14T11:20:31.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1e1a00700 1 -- start start 2023-07-14T11:20:31.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1e1a00700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff1dc0ffe00 0x7ff1dc195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:31.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1e1a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:31.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1e1a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc104ea0 0x7ff1dc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:31.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1e1a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc19ad10 con 0x7ff1dc104ea0 2023-07-14T11:20:31.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1e1a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc19ae80 con 0x7ff1dc1007a0 2023-07-14T11:20:31.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1e1a00700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc19b040 con 0x7ff1dc0ffe00 2023-07-14T11:20:31.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1da7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:31.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1da7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34772/0 (socket says 172.21.15.161:34772) 2023-07-14T11:20:31.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1da7fc700 1 -- 172.21.15.161:0/2713306467 learned_addr learned my addr 172.21.15.161:0/2713306467 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:31.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1daffd700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff1dc0ffe00 0x7ff1dc195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:31.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.734+0000 7ff1db7fe700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc104ea0 0x7ff1dc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:31.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1da7fc700 1 -- 172.21.15.161:0/2713306467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff1dc0ffe00 msgr2=0x7ff1dc195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:31.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1da7fc700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff1dc0ffe00 0x7ff1dc195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:31.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1da7fc700 1 -- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc104ea0 msgr2=0x7ff1dc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:31.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1da7fc700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc104ea0 0x7ff1dc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:31.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1da7fc700 1 -- 172.21.15.161:0/2713306467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d00095f0 con 0x7ff1dc1007a0 2023-07-14T11:20:31.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1da7fc700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc196290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00098e0 tx=0x7ff1d000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:31.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1e09fe700 1 -- 172.21.15.161:0/2713306467 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d001d070 con 0x7ff1dc1007a0 2023-07-14T11:20:31.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc19b2c0 con 0x7ff1dc1007a0 2023-07-14T11:20:31.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1e09fe700 1 -- 172.21.15.161:0/2713306467 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1d0022470 con 0x7ff1dc1007a0 2023-07-14T11:20:31.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1e09fe700 1 -- 172.21.15.161:0/2713306467 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0017650 con 0x7ff1dc1007a0 2023-07-14T11:20:31.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.735+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc1a2b80 con 0x7ff1dc1007a0 2023-07-14T11:20:31.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.736+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc04e210 con 0x7ff1dc1007a0 2023-07-14T11:20:31.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.738+0000 7ff1e09fe700 1 -- 172.21.15.161:0/2713306467 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff1d0022850 con 0x7ff1dc1007a0 2023-07-14T11:20:31.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.740+0000 7ff1e09fe700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff1c40a11e0 0x7ff1c40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:31.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.740+0000 7ff1e09fe700 1 -- 172.21.15.161:0/2713306467 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff1d00cdc40 con 0x7ff1dc1007a0 2023-07-14T11:20:31.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.740+0000 7ff1daffd700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff1c40a11e0 0x7ff1c40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:31.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.740+0000 7ff1daffd700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff1c40a11e0 0x7ff1c40a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009c60 tx=0x7ff1cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:31.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.742+0000 7ff1e09fe700 1 -- 172.21.15.161:0/2713306467 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff1d0097060 con 0x7ff1dc1007a0 2023-07-14T11:20:31.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:31 smithi182 ceph-mon[129366]: pgmap v298: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-07-14T11:20:31.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.992+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff1dc19b450 con 0x7ff1dc1007a0 2023-07-14T11:20:31.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:31 smithi161 ceph-mon[95526]: pgmap v298: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-07-14T11:20:32.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:31.998+0000 7ff1e09fe700 1 -- 172.21.15.161:0/2713306467 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260132 (secure 0 0 0) 0x7ff1d009a680 con 0x7ff1dc1007a0 2023-07-14T11:20:32.003 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:32.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff1c40a11e0 msgr2=0x7ff1c40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:32.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff1c40a11e0 0x7ff1c40a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009c60 tx=0x7ff1cc009260 comp rx=0 tx=0).stop 2023-07-14T11:20:32.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc1007a0 msgr2=0x7ff1dc196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:32.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc196290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00098e0 tx=0x7ff1d000f690 comp rx=0 tx=0).stop 2023-07-14T11:20:32.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 shutdown_connections 2023-07-14T11:20:32.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1dc104ea0 0x7ff1dc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:32.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff1c40a11e0 0x7ff1c40a3690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:32.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff1dc0ffe00 0x7ff1dc195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:32.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 --2- 172.21.15.161:0/2713306467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1dc1007a0 0x7ff1dc196290 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:32.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 >> 172.21.15.161:0/2713306467 conn(0x7ff1dc0fb660 msgr2=0x7ff1dc0fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:32.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 shutdown_connections 2023-07-14T11:20:32.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:32.003+0000 7ff1e1a00700 1 -- 172.21.15.161:0/2713306467 wait complete. 2023-07-14T11:20:32.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:31 smithi175 ceph-mon[129081]: pgmap v298: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-07-14T11:20:32.421 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":17,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2004305703},{"type":"v1","addr":"172.21.15.161:6801","nonce":2004305703}]},"active_addr":"172.21.15.161:6801/2004305703","active_change":"2023-07-14T11:13:14.005214+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.161:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":142996677}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":3308966068}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":182250477}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":389110447}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":1767569563}]}]} 2023-07-14T11:20:32.427 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-14T11:20:32.428 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-14T11:20:32.428 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd dump --format=json 2023-07-14T11:20:32.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:32 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2713306467' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T11:20:32.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:32 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2713306467' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T11:20:33.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:32 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2713306467' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T11:20:33.188 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:33.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:33 smithi182 ceph-mon[129366]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-07-14T11:20:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:33 smithi161 ceph-mon[95526]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-07-14T11:20:34.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:33 smithi175 ceph-mon[129081]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-07-14T11:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 -- 172.21.15.161:0/3549583071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4470104ea0 msgr2=0x7f447010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 --2- 172.21.15.161:0/3549583071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4470104ea0 0x7f447010ca20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f446800b0d0 tx=0x7f446800b3e0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 -- 172.21.15.161:0/3549583071 shutdown_connections 2023-07-14T11:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 --2- 172.21.15.161:0/3549583071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4470104ea0 0x7f447010ca20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 --2- 172.21.15.161:0/3549583071 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f44700ffe00 0x7f44701001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 --2- 172.21.15.161:0/3549583071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44701007a0 0x7f4470104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 -- 172.21.15.161:0/3549583071 >> 172.21.15.161:0/3549583071 conn(0x7f44700fb660 msgr2=0x7f44700fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 -- 172.21.15.161:0/3549583071 shutdown_connections 2023-07-14T11:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.473+0000 7f4474893700 1 -- 172.21.15.161:0/3549583071 wait complete. 2023-07-14T11:20:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.474+0000 7f4474893700 1 Processor -- start 2023-07-14T11:20:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.474+0000 7f4474893700 1 -- start start 2023-07-14T11:20:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.474+0000 7f4474893700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44700ffe00 0x7f4470072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.474+0000 7f4474893700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44701007a0 0x7f4470072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f4474893700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4470104ea0 0x7f44700732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f4474893700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44701a41d0 con 0x7f44701007a0 2023-07-14T11:20:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f4474893700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44701a4340 con 0x7f44700ffe00 2023-07-14T11:20:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f4474893700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44701a4500 con 0x7f4470104ea0 2023-07-14T11:20:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44701007a0 0x7f4470072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44700ffe00 0x7f4470072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44701007a0 0x7f4470072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50730/0 (socket says 172.21.15.161:50730) 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 -- 172.21.15.161:0/2048112360 learned_addr learned my addr 172.21.15.161:0/2048112360 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446ed9d700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4470104ea0 0x7f44700732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 -- 172.21.15.161:0/2048112360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4470104ea0 msgr2=0x7f44700732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4470104ea0 0x7f44700732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 -- 172.21.15.161:0/2048112360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44700ffe00 msgr2=0x7f4470072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44700ffe00 0x7f4470072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 -- 172.21.15.161:0/2048112360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4468009d00 con 0x7f44701007a0 2023-07-14T11:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446ed9d700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4470104ea0 0x7f44700732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446e59c700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44700ffe00 0x7f4470072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.475+0000 7f446dd9b700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44701007a0 0x7f4470072d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f446400e970 tx=0x7f446400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.476+0000 7f445f7fe700 1 -- 172.21.15.161:0/2048112360 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464009790 con 0x7f44701007a0 2023-07-14T11:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.476+0000 7f445f7fe700 1 -- 172.21.15.161:0/2048112360 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f446400eec0 con 0x7f44701007a0 2023-07-14T11:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.476+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44701a47e0 con 0x7f44701007a0 2023-07-14T11:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.476+0000 7f445f7fe700 1 -- 172.21.15.161:0/2048112360 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464005590 con 0x7f44701007a0 2023-07-14T11:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.476+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44701a4d30 con 0x7f44701007a0 2023-07-14T11:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.477+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447004e210 con 0x7f44701007a0 2023-07-14T11:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.479+0000 7f445f7fe700 1 -- 172.21.15.161:0/2048112360 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f446400cbd0 con 0x7f44701007a0 2023-07-14T11:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.480+0000 7f445f7fe700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44580a1260 0x7f44580a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.480+0000 7f445f7fe700 1 -- 172.21.15.161:0/2048112360 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4464014070 con 0x7f44701007a0 2023-07-14T11:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.480+0000 7f446e59c700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44580a1260 0x7f44580a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.481+0000 7f446e59c700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44580a1260 0x7f44580a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4460007fd0 tx=0x7f44600073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:34.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.484+0000 7f445f7fe700 1 -- 172.21.15.161:0/2048112360 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f44640d1070 con 0x7f44701007a0 2023-07-14T11:20:34.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.681+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44701a50c0 con 0x7f44701007a0 2023-07-14T11:20:34.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.683+0000 7f445f7fe700 1 -- 172.21.15.161:0/2048112360 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24461 (secure 0 0 0) 0x7f4464099c80 con 0x7f44701007a0 2023-07-14T11:20:34.686 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:34.687 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":90,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:20:15.713307+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:20:34.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44580a1260 msgr2=0x7f44580a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:34.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44580a1260 0x7f44580a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4460007fd0 tx=0x7f44600073d0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44701007a0 msgr2=0x7f4470072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:34.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44701007a0 0x7f4470072d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f446400e970 tx=0x7f446400ec80 comp rx=0 tx=0).stop 2023-07-14T11:20:34.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 shutdown_connections 2023-07-14T11:20:34.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f44580a1260 0x7f44580a3710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4470104ea0 0x7f44700732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44700ffe00 0x7f4470072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.685+0000 7f4474893700 1 --2- 172.21.15.161:0/2048112360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44701007a0 0x7f4470072d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:34.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.686+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 >> 172.21.15.161:0/2048112360 conn(0x7f44700fb660 msgr2=0x7f44700fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:34.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.686+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 shutdown_connections 2023-07-14T11:20:34.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:34.686+0000 7f4474893700 1 -- 172.21.15.161:0/2048112360 wait complete. 2023-07-14T11:20:34.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:34 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2048112360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:34.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:34 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2048112360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:35.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:34 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2048112360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:35.181 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-14T11:20:35.181 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd dump --format=json 2023-07-14T11:20:35.906 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:35.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:35 smithi182 ceph-mon[129366]: pgmap v300: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T11:20:35.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:35 smithi161 ceph-mon[95526]: pgmap v300: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T11:20:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:35 smithi175 ceph-mon[129081]: pgmap v300: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T11:20:37.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 -- 172.21.15.161:0/2825423238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 msgr2=0x7fe744104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:37.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 --2- 172.21.15.161:0/2825423238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744104770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe7340099c0 tx=0x7fe734009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 -- 172.21.15.161:0/2825423238 shutdown_connections 2023-07-14T11:20:37.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 --2- 172.21.15.161:0/2825423238 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe744104ea0 0x7fe74410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 --2- 172.21.15.161:0/2825423238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7440ffe00 0x7fe7441001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 --2- 172.21.15.161:0/2825423238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744104770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 -- 172.21.15.161:0/2825423238 >> 172.21.15.161:0/2825423238 conn(0x7fe7440fb660 msgr2=0x7fe7440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:37.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 -- 172.21.15.161:0/2825423238 shutdown_connections 2023-07-14T11:20:37.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.413+0000 7fe74bca1700 1 -- 172.21.15.161:0/2825423238 wait complete. 2023-07-14T11:20:37.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.414+0000 7fe74bca1700 1 Processor -- start 2023-07-14T11:20:37.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.414+0000 7fe74bca1700 1 -- start start 2023-07-14T11:20:37.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.414+0000 7fe74bca1700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7440ffe00 0x7fe744072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:37.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.414+0000 7fe74bca1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:37.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74bca1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe744104ea0 0x7fe7440732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:37.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74bca1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7441a41a0 con 0x7fe7441007a0 2023-07-14T11:20:37.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74bca1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7441a4310 con 0x7fe744104ea0 2023-07-14T11:20:37.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74bca1700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7441a44d0 con 0x7fe7440ffe00 2023-07-14T11:20:37.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:37.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50738/0 (socket says 172.21.15.161:50738) 2023-07-14T11:20:37.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 -- 172.21.15.161:0/1663581513 learned_addr learned my addr 172.21.15.161:0/1663581513 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:37.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe749a3d700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7440ffe00 0x7fe744072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74a23e700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe744104ea0 0x7fe7440732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 -- 172.21.15.161:0/1663581513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7440ffe00 msgr2=0x7fe744072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7440ffe00 0x7fe744072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 -- 172.21.15.161:0/1663581513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe744104ea0 msgr2=0x7fe7440732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe744104ea0 0x7fe7440732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74923c700 1 -- 172.21.15.161:0/1663581513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340095f0 con 0x7fe7441007a0 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe749a3d700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7440ffe00 0x7fe744072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:37.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.415+0000 7fe74a23e700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe744104ea0 0x7fe7440732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:37.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.416+0000 7fe74923c700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744072d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7340098e0 tx=0x7fe73400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:37.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.416+0000 7fe73affd700 1 -- 172.21.15.161:0/1663581513 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73401d070 con 0x7fe7441007a0 2023-07-14T11:20:37.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.416+0000 7fe73affd700 1 -- 172.21.15.161:0/1663581513 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe734022470 con 0x7fe7441007a0 2023-07-14T11:20:37.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.416+0000 7fe73affd700 1 -- 172.21.15.161:0/1663581513 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734017650 con 0x7fe7441007a0 2023-07-14T11:20:37.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.416+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7441a4750 con 0x7fe7441007a0 2023-07-14T11:20:37.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.416+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7441a4b10 con 0x7fe7441007a0 2023-07-14T11:20:37.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.417+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74404e210 con 0x7fe7441007a0 2023-07-14T11:20:37.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.419+0000 7fe73affd700 1 -- 172.21.15.161:0/1663581513 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe734022850 con 0x7fe7441007a0 2023-07-14T11:20:37.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.420+0000 7fe73affd700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7300a1260 0x7fe7300a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:37.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.421+0000 7fe73affd700 1 -- 172.21.15.161:0/1663581513 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe7340cd7c0 con 0x7fe7441007a0 2023-07-14T11:20:37.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.421+0000 7fe749a3d700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7300a1260 0x7fe7300a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:37.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.421+0000 7fe749a3d700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7300a1260 0x7fe7300a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe740011490 tx=0x7fe740009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:37.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.424+0000 7fe73affd700 1 -- 172.21.15.161:0/1663581513 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe734096be0 con 0x7fe7441007a0 2023-07-14T11:20:37.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.621+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe744073d40 con 0x7fe7441007a0 2023-07-14T11:20:37.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.623+0000 7fe73affd700 1 -- 172.21.15.161:0/1663581513 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24461 (secure 0 0 0) 0x7fe73409a200 con 0x7fe7441007a0 2023-07-14T11:20:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:20:37.626 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":90,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:20:15.713307+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:20:37.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7300a1260 msgr2=0x7fe7300a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:37.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7300a1260 0x7fe7300a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe740011490 tx=0x7fe740009260 comp rx=0 tx=0).stop 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 msgr2=0x7fe744072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744072d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7340098e0 tx=0x7fe73400f690 comp rx=0 tx=0).stop 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 shutdown_connections 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7300a1260 0x7fe7300a3710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe744104ea0 0x7fe7440732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7440ffe00 0x7fe744072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 --2- 172.21.15.161:0/1663581513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7441007a0 0x7fe744072d50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:37.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 >> 172.21.15.161:0/1663581513 conn(0x7fe7440fb660 msgr2=0x7fe7440fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:37.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 shutdown_connections 2023-07-14T11:20:37.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:37.626+0000 7fe74bca1700 1 -- 172.21.15.161:0/1663581513 wait complete. 2023-07-14T11:20:37.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:37 smithi182 ceph-mon[129366]: pgmap v301: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:37.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:37 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1663581513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:37.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:37 smithi161 ceph-mon[95526]: pgmap v301: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:37.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1663581513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:38.104 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-14T11:20:38.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-14T11:20:38.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-14T11:20:38.106 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-14T11:20:38.107 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-14T11:20:38.107 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-14T11:20:38.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-14T11:20:38.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-14T11:20:38.110 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-14T11:20:38.110 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-14T11:20:38.111 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-14T11:20:38.111 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-14T11:20:38.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:37 smithi175 ceph-mon[129081]: pgmap v301: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:38.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:37 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1663581513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:20:39.697 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:39.731 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:39.994 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:39 smithi161 ceph-mon[95526]: pgmap v302: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:20:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:40.101 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:40.115 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:40.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:39 smithi175 ceph-mon[129081]: pgmap v302: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:20:40.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:39 smithi182 ceph-mon[129366]: pgmap v302: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:20:40.732 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:40.750 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:40.799 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:40.939 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:40 smithi161 ceph-mon[95526]: pgmap v303: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:40.999 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:41.112 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:41.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:40 smithi175 ceph-mon[129081]: pgmap v303: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:41.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:40 smithi182 ceph-mon[129366]: pgmap v303: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 -- 172.21.15.161:0/3097631704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d000731c0 msgr2=0x7f8d00110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 --2- 172.21.15.161:0/3097631704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d000731c0 0x7f8d00110670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009990 tx=0x7f8cf4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 -- 172.21.15.161:0/3097631704 shutdown_connections 2023-07-14T11:20:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 --2- 172.21.15.161:0/3097631704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d00110bb0 0x7f8d00112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 --2- 172.21.15.161:0/3097631704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d000728b0 0x7f8d00072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 --2- 172.21.15.161:0/3097631704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d000731c0 0x7f8d00110670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 -- 172.21.15.161:0/3097631704 >> 172.21.15.161:0/3097631704 conn(0x7f8d000fb660 msgr2=0x7f8d000fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 -- 172.21.15.161:0/3097631704 shutdown_connections 2023-07-14T11:20:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.869+0000 7f8d04ced700 1 -- 172.21.15.161:0/3097631704 wait complete. 2023-07-14T11:20:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.870+0000 7f8d04ced700 1 Processor -- start 2023-07-14T11:20:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.870+0000 7f8d04ced700 1 -- start start 2023-07-14T11:20:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.870+0000 7f8d04ced700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d000728b0 0x7f8d00108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8d04ced700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d000731c0 0x7f8d00109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8d04ced700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d00110bb0 0x7f8d001b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8d04ced700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d001b0d80 con 0x7f8d000731c0 2023-07-14T11:20:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8d04ced700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d001b0ef0 con 0x7f8d00110bb0 2023-07-14T11:20:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8d04ced700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d001b10b0 con 0x7f8d000728b0 2023-07-14T11:20:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d000728b0 0x7f8d00108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfdd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d000731c0 0x7f8d00109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d000728b0 0x7f8d00108e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58468/0 (socket says 172.21.15.161:58468) 2023-07-14T11:20:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 -- 172.21.15.161:0/1449993699 learned_addr learned my addr 172.21.15.161:0/1449993699 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfed9d700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d00110bb0 0x7f8d001b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 -- 172.21.15.161:0/1449993699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d00110bb0 msgr2=0x7f8d001b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d00110bb0 0x7f8d001b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 -- 172.21.15.161:0/1449993699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d000731c0 msgr2=0x7f8d00109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d000731c0 0x7f8d00109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.871+0000 7f8cfe59c700 1 -- 172.21.15.161:0/1449993699 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40095f0 con 0x7f8d000728b0 2023-07-14T11:20:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.872+0000 7f8cfe59c700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d000728b0 0x7f8d00108e50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000e9a0 tx=0x7f8cf000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.872+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0009790 con 0x7f8d000728b0 2023-07-14T11:20:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.872+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d001b1390 con 0x7f8d000728b0 2023-07-14T11:20:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.872+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cf0010430 con 0x7f8d000728b0 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.872+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0018680 con 0x7f8d000728b0 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.872+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d001b1890 con 0x7f8d000728b0 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.873+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d00116380 con 0x7f8d000728b0 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.875+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8cf00105a0 con 0x7f8d000728b0 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.876+0000 7f8cef7fe700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8ce80a11e0 0x7f8ce80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.876+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8cf0014070 con 0x7f8d000728b0 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.876+0000 7f8cfdd9b700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8ce80a11e0 0x7f8ce80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.877+0000 7f8cef7fe700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] conn(0x7f8ce80ac1f0 0x7f8ce80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.877+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 --> [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce80aed50 con 0x7f8ce80ac1f0 2023-07-14T11:20:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.877+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cf00cd610 con 0x7f8d000728b0 2023-07-14T11:20:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.877+0000 7f8cfed9d700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] conn(0x7f8ce80ac1f0 0x7f8ce80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.877+0000 7f8cfdd9b700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8ce80a11e0 0x7f8ce80a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d00112310 tx=0x7f8cf4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.878+0000 7f8cfed9d700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] conn(0x7f8ce80ac1f0 0x7f8ce80ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.879+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== osd.2 v2:172.21.15.182:6800/868656265 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8ce80aed50 con 0x7f8ce80ac1f0 2023-07-14T11:20:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.897+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 --> [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d0004e210 con 0x7f8ce80ac1f0 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.898+0000 7f8cef7fe700 1 -- 172.21.15.161:0/1449993699 <== osd.2 v2:172.21.15.182:6800/868656265 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8d0004e210 con 0x7f8ce80ac1f0 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.898+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] conn(0x7f8ce80ac1f0 msgr2=0x7f8ce80ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.898+0000 7f8d04ced700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] conn(0x7f8ce80ac1f0 0x7f8ce80ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8ce80a11e0 msgr2=0x7f8ce80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8ce80a11e0 0x7f8ce80a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d00112310 tx=0x7f8cf4005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d000728b0 msgr2=0x7f8d00108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 --2- 172.21.15.161:0/1449993699 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d000728b0 0x7f8d00108e50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000e9a0 tx=0x7f8cf000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8cfed9d700 1 -- 172.21.15.161:0/1449993699 reap_dead start 2023-07-14T11:20:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 shutdown_connections 2023-07-14T11:20:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 >> 172.21.15.161:0/1449993699 conn(0x7f8d000fb660 msgr2=0x7f8d000fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 shutdown_connections 2023-07-14T11:20:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:42.899+0000 7f8d04ced700 1 -- 172.21.15.161:0/1449993699 wait complete. 2023-07-14T11:20:43.091 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:43.091 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:43.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 -- 172.21.15.161:0/1920376805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8104ea0 msgr2=0x7fd2a810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 --2- 172.21.15.161:0/1920376805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8104ea0 0x7fd2a810ca20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000b0d0 tx=0x7fd2a000b3e0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 -- 172.21.15.161:0/1920376805 shutdown_connections 2023-07-14T11:20:43.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 --2- 172.21.15.161:0/1920376805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8104ea0 0x7fd2a810ca20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 --2- 172.21.15.161:0/1920376805 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd2a80ffe00 0x7fd2a81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 --2- 172.21.15.161:0/1920376805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2a81007a0 0x7fd2a8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 -- 172.21.15.161:0/1920376805 >> 172.21.15.161:0/1920376805 conn(0x7fd2a80fb660 msgr2=0x7fd2a80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:43.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 -- 172.21.15.161:0/1920376805 shutdown_connections 2023-07-14T11:20:43.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.656+0000 7fd2adece700 1 -- 172.21.15.161:0/1920376805 wait complete. 2023-07-14T11:20:43.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.657+0000 7fd2adece700 1 Processor -- start 2023-07-14T11:20:43.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.657+0000 7fd2adece700 1 -- start start 2023-07-14T11:20:43.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2adece700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80ffe00 0x7fd2a8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2adece700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2a81007a0 0x7fd2a8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2adece700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd2a8104ea0 0x7fd2a80732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2adece700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81a41d0 con 0x7fd2a80ffe00 2023-07-14T11:20:43.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2adece700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81a4340 con 0x7fd2a81007a0 2023-07-14T11:20:43.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2adece700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81a4500 con 0x7fd2a8104ea0 2023-07-14T11:20:43.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2a81007a0 0x7fd2a8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80ffe00 0x7fd2a8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2a81007a0 0x7fd2a8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:32780/0 (socket says 172.21.15.161:32780) 2023-07-14T11:20:43.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 -- 172.21.15.161:0/2133856958 learned_addr learned my addr 172.21.15.161:0/2133856958 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:43.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a7fff700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd2a8104ea0 0x7fd2a80732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 -- 172.21.15.161:0/2133856958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd2a8104ea0 msgr2=0x7fd2a80732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd2a8104ea0 0x7fd2a80732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 -- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80ffe00 msgr2=0x7fd2a8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80ffe00 0x7fd2a8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.658+0000 7fd2a6ffd700 1 -- 172.21.15.161:0/2133856958 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c009520 con 0x7fd2a81007a0 2023-07-14T11:20:43.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.659+0000 7fd2a6ffd700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2a81007a0 0x7fd2a8072d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00ea30 tx=0x7fd29c00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:43.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.659+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00cbd0 con 0x7fd2a81007a0 2023-07-14T11:20:43.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.659+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd29c00eed0 con 0x7fd2a81007a0 2023-07-14T11:20:43.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.659+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c003d20 con 0x7fd2a81007a0 2023-07-14T11:20:43.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.659+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a0009d00 con 0x7fd2a81007a0 2023-07-14T11:20:43.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.659+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a81a4b00 con 0x7fd2a81007a0 2023-07-14T11:20:43.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.660+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2a804e210 con 0x7fd2a81007a0 2023-07-14T11:20:43.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.662+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd29c003e80 con 0x7fd2a81007a0 2023-07-14T11:20:43.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.664+0000 7fd2a4ff9700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd2900a11e0 0x7fd2900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.664+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd29c014070 con 0x7fd2a81007a0 2023-07-14T11:20:43.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.664+0000 7fd2a77fe700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd2900a11e0 0x7fd2900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.664+0000 7fd2a4ff9700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] conn(0x7fd2900ac1f0 0x7fd2900ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:43.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.664+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 --> [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2900aed50 con 0x7fd2900ac1f0 2023-07-14T11:20:43.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.664+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd29c0cd4c0 con 0x7fd2a81007a0 2023-07-14T11:20:43.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.664+0000 7fd2a7fff700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] conn(0x7fd2900ac1f0 0x7fd2900ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:43.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.665+0000 7fd2a77fe700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd2900a11e0 0x7fd2900a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd298009c60 tx=0x7fd298009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:43.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.665+0000 7fd2a7fff700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] conn(0x7fd2900ac1f0 0x7fd2900ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:43.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.666+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== osd.3 v2:172.21.15.161:6810/2962874030 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd2900aed50 con 0x7fd2900ac1f0 2023-07-14T11:20:43.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.682+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 --> [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2a810fd90 con 0x7fd2900ac1f0 2023-07-14T11:20:43.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2a4ff9700 1 -- 172.21.15.161:0/2133856958 <== osd.3 v2:172.21.15.161:6810/2962874030 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd2a000baa0 con 0x7fd2900ac1f0 2023-07-14T11:20:43.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] conn(0x7fd2900ac1f0 msgr2=0x7fd2900ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] conn(0x7fd2900ac1f0 0x7fd2900ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:43.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd2900a11e0 msgr2=0x7fd2900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd2900a11e0 0x7fd2900a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd298009c60 tx=0x7fd298009260 comp rx=0 tx=0).stop 2023-07-14T11:20:43.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2a81007a0 msgr2=0x7fd2a8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:43.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 --2- 172.21.15.161:0/2133856958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2a81007a0 0x7fd2a8072d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00ea30 tx=0x7fd29c00ed40 comp rx=0 tx=0).stop 2023-07-14T11:20:43.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2a7fff700 1 -- 172.21.15.161:0/2133856958 reap_dead start 2023-07-14T11:20:43.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 shutdown_connections 2023-07-14T11:20:43.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.683+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 >> 172.21.15.161:0/2133856958 conn(0x7fd2a80fb660 msgr2=0x7fd2a80fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:43.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.684+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 shutdown_connections 2023-07-14T11:20:43.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:43.684+0000 7fd2adece700 1 -- 172.21.15.161:0/2133856958 wait complete. 2023-07-14T11:20:43.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:43 smithi182 ceph-mon[129366]: pgmap v304: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:43 smithi161 ceph-mon[95526]: pgmap v304: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:43 smithi175 ceph-mon[129081]: pgmap v304: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:20:44.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:44 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:20:44.965 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:44 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:20:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:44 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:20:45.289 INFO:teuthology.orchestra.run.smithi161.stdout:77309411394 2023-07-14T11:20:45.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-14T11:20:45.598 INFO:teuthology.orchestra.run.smithi161.stdout:111669149756 2023-07-14T11:20:45.598 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-14T11:20:45.923 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:45 smithi161 ceph-mon[95526]: pgmap v305: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:20:45.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:45 smithi182 ceph-mon[129366]: pgmap v305: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:20:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:45 smithi175 ceph-mon[129081]: pgmap v305: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:20:46.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.016+0000 7fe03c79a700 1 -- 172.21.15.161:0/2888510476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0340731c0 msgr2=0x7fe034110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:46.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.016+0000 7fe03c79a700 1 --2- 172.21.15.161:0/2888510476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0340731c0 0x7fe034110670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe024009990 tx=0x7fe024009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.016+0000 7fe03c79a700 1 -- 172.21.15.161:0/2888510476 shutdown_connections 2023-07-14T11:20:46.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.016+0000 7fe03c79a700 1 --2- 172.21.15.161:0/2888510476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034110bb0 0x7fe034112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.016+0000 7fe03c79a700 1 --2- 172.21.15.161:0/2888510476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe0340728b0 0x7fe034072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.016+0000 7fe03c79a700 1 --2- 172.21.15.161:0/2888510476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0340731c0 0x7fe034110670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.016+0000 7fe03c79a700 1 -- 172.21.15.161:0/2888510476 >> 172.21.15.161:0/2888510476 conn(0x7fe0340fb660 msgr2=0x7fe0340fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:46.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.017+0000 7fe03c79a700 1 -- 172.21.15.161:0/2888510476 shutdown_connections 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.017+0000 7fe03c79a700 1 -- 172.21.15.161:0/2888510476 wait complete. 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 Processor -- start 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 -- start start 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0340728b0 0x7fe03419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe0340731c0 0x7fe03419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe034110bb0 0x7fe0341a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341a1220 con 0x7fe034110bb0 2023-07-14T11:20:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341a1390 con 0x7fe0340728b0 2023-07-14T11:20:46.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03c79a700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341a1550 con 0x7fe0340731c0 2023-07-14T11:20:46.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.018+0000 7fe03a536700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0340728b0 0x7fe03419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:46.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0340728b0 0x7fe03419c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:32798/0 (socket says 172.21.15.161:32798) 2023-07-14T11:20:46.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 -- 172.21.15.161:0/3992789512 learned_addr learned my addr 172.21.15.161:0/3992789512 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:46.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03ad37700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe034110bb0 0x7fe0341a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:46.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe039d35700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe0340731c0 0x7fe03419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:46.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 -- 172.21.15.161:0/3992789512 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe0340731c0 msgr2=0x7fe03419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:46.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe0340731c0 0x7fe03419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 -- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe034110bb0 msgr2=0x7fe0341a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:46.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe034110bb0 0x7fe0341a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 -- 172.21.15.161:0/3992789512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240095f0 con 0x7fe0340728b0 2023-07-14T11:20:46.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.019+0000 7fe03a536700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0340728b0 0x7fe03419c260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe03000e9a0 tx=0x7fe03000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:46.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.020+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030009790 con 0x7fe0340728b0 2023-07-14T11:20:46.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.020+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe030010430 con 0x7fe0340728b0 2023-07-14T11:20:46.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.020+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0341a1830 con 0x7fe0340728b0 2023-07-14T11:20:46.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.020+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030018680 con 0x7fe0340728b0 2023-07-14T11:20:46.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.020+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0341a9090 con 0x7fe0340728b0 2023-07-14T11:20:46.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.021+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe034116330 con 0x7fe0340728b0 2023-07-14T11:20:46.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.024+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe0300105a0 con 0x7fe0340728b0 2023-07-14T11:20:46.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.026+0000 7fe02b7fe700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe0200a11e0 0x7fe0200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:46.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.026+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe030014070 con 0x7fe0340728b0 2023-07-14T11:20:46.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.026+0000 7fe039d35700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe0200a11e0 0x7fe0200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:46.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.026+0000 7fe02b7fe700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] conn(0x7fe0200ac1f0 0x7fe0200ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:46.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.027+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 --> [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0200aed50 con 0x7fe0200ac1f0 2023-07-14T11:20:46.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.027+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0300cd720 con 0x7fe0340728b0 2023-07-14T11:20:46.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.027+0000 7fe03ad37700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] conn(0x7fe0200ac1f0 0x7fe0200ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:46.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.027+0000 7fe039d35700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe0200a11e0 0x7fe0200a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe02400b2e0 tx=0x7fe024005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:46.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.028+0000 7fe03ad37700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] conn(0x7fe0200ac1f0 0x7fe0200ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:46.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.028+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== osd.7 v2:172.21.15.175:6816/4058697071 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe0200aed50 con 0x7fe0200ac1f0 2023-07-14T11:20:46.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.048+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 --> [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe03404e210 con 0x7fe0200ac1f0 2023-07-14T11:20:46.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe02b7fe700 1 -- 172.21.15.161:0/3992789512 <== osd.7 v2:172.21.15.175:6816/4058697071 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe03404e210 con 0x7fe0200ac1f0 2023-07-14T11:20:46.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] conn(0x7fe0200ac1f0 msgr2=0x7fe0200ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:46.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] conn(0x7fe0200ac1f0 0x7fe0200ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe0200a11e0 msgr2=0x7fe0200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:46.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe0200a11e0 0x7fe0200a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe02400b2e0 tx=0x7fe024005960 comp rx=0 tx=0).stop 2023-07-14T11:20:46.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0340728b0 msgr2=0x7fe03419c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:46.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 --2- 172.21.15.161:0/3992789512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0340728b0 0x7fe03419c260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe03000e9a0 tx=0x7fe03000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:20:46.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03ad37700 1 -- 172.21.15.161:0/3992789512 reap_dead start 2023-07-14T11:20:46.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 shutdown_connections 2023-07-14T11:20:46.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.049+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 >> 172.21.15.161:0/3992789512 conn(0x7fe0340fb660 msgr2=0x7fe0340fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:46.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.050+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 shutdown_connections 2023-07-14T11:20:46.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:46.050+0000 7fe03c79a700 1 -- 172.21.15.161:0/3992789512 wait complete. 2023-07-14T11:20:47.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 -- 172.21.15.161:0/1841247313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb4104ea0 msgr2=0x7f5bb410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:47.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 --2- 172.21.15.161:0/1841247313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb4104ea0 0x7f5bb410ca20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00b0d0 tx=0x7f5bac00b3e0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 -- 172.21.15.161:0/1841247313 shutdown_connections 2023-07-14T11:20:47.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 --2- 172.21.15.161:0/1841247313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb4104ea0 0x7f5bb410ca20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 --2- 172.21.15.161:0/1841247313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5bb40ffe00 0x7f5bb41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 --2- 172.21.15.161:0/1841247313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb41007a0 0x7f5bb4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 -- 172.21.15.161:0/1841247313 >> 172.21.15.161:0/1841247313 conn(0x7f5bb40fb660 msgr2=0x7f5bb40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:47.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.142+0000 7f5bb9a40700 1 -- 172.21.15.161:0/1841247313 shutdown_connections 2023-07-14T11:20:47.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.143+0000 7f5bb9a40700 1 -- 172.21.15.161:0/1841247313 wait complete. 2023-07-14T11:20:47.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 Processor -- start 2023-07-14T11:20:47.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 -- start start 2023-07-14T11:20:47.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb40ffe00 0x7f5bb4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:47.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb41007a0 0x7f5bb41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:47.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5bb4104ea0 0x7f5bb419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb419cf40 con 0x7f5bb40ffe00 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb419d0b0 con 0x7f5bb41007a0 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb9a40700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb419d270 con 0x7f5bb4104ea0 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb40ffe00 0x7f5bb4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb40ffe00 0x7f5bb4197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40388/0 (socket says 172.21.15.161:40388) 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.144+0000 7f5bb2ffd700 1 -- 172.21.15.161:0/4060605832 learned_addr learned my addr 172.21.15.161:0/4060605832 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb27fc700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb41007a0 0x7f5bb41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:47.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb37fe700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5bb4104ea0 0x7f5bb419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:47.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb37fe700 1 -- 172.21.15.161:0/4060605832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb41007a0 msgr2=0x7f5bb41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:47.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb37fe700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb41007a0 0x7f5bb41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb37fe700 1 -- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb40ffe00 msgr2=0x7f5bb4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:47.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb37fe700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb40ffe00 0x7f5bb4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb37fe700 1 -- 172.21.15.161:0/4060605832 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac009d00 con 0x7f5bb4104ea0 2023-07-14T11:20:47.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb27fc700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb41007a0 0x7f5bb41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:47.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb2ffd700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb40ffe00 0x7f5bb4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:47.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb37fe700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5bb4104ea0 0x7f5bb419c930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009fd0 tx=0x7f5bac0077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:47.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00e040 con 0x7f5bb4104ea0 2023-07-14T11:20:47.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb419d4f0 con 0x7f5bb4104ea0 2023-07-14T11:20:47.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bac00bcb0 con 0x7f5bb4104ea0 2023-07-14T11:20:47.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.145+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac005840 con 0x7f5bb4104ea0 2023-07-14T11:20:47.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.146+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb41a4db0 con 0x7f5bb4104ea0 2023-07-14T11:20:47.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.146+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5bb419d7d0 con 0x7f5bb4104ea0 2023-07-14T11:20:47.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.148+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5bac004a50 con 0x7f5bb4104ea0 2023-07-14T11:20:47.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.150+0000 7f5bb8a3e700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5b9c0a11e0 0x7f5b9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:47.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.150+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5bac0ce270 con 0x7f5bb4104ea0 2023-07-14T11:20:47.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.150+0000 7f5bb2ffd700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5b9c0a11e0 0x7f5b9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:47.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.150+0000 7f5bb8a3e700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] conn(0x7f5b9c0ac0c0 0x7f5b9c0ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:47.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.150+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 --> [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b9c0aec20 con 0x7f5b9c0ac0c0 2023-07-14T11:20:47.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.150+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bac0ce5e0 con 0x7f5bb4104ea0 2023-07-14T11:20:47.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.150+0000 7f5bb27fc700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] conn(0x7f5b9c0ac0c0 0x7f5b9c0ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:47.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.151+0000 7f5bb2ffd700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5b9c0a11e0 0x7f5b9c0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4011550 tx=0x7f5ba4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:47.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.151+0000 7f5bb27fc700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] conn(0x7f5b9c0ac0c0 0x7f5b9c0ae550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:47.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.152+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== osd.8 v2:172.21.15.182:6816/3827127 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5b9c0aec20 con 0x7f5b9c0ac0c0 2023-07-14T11:20:47.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.168+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 --> [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5bb410fd90 con 0x7f5b9c0ac0c0 2023-07-14T11:20:47.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb8a3e700 1 -- 172.21.15.161:0/4060605832 <== osd.8 v2:172.21.15.182:6816/3827127 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5ba8009d00 con 0x7f5b9c0ac0c0 2023-07-14T11:20:47.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] conn(0x7f5b9c0ac0c0 msgr2=0x7f5b9c0ae550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:47.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] conn(0x7f5b9c0ac0c0 0x7f5b9c0ae550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5b9c0a11e0 msgr2=0x7f5b9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:47.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5b9c0a11e0 0x7f5b9c0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4011550 tx=0x7f5ba4009260 comp rx=0 tx=0).stop 2023-07-14T11:20:47.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5bb4104ea0 msgr2=0x7f5bb419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:47.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 --2- 172.21.15.161:0/4060605832 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5bb4104ea0 0x7f5bb419c930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009fd0 tx=0x7f5bac0077f0 comp rx=0 tx=0).stop 2023-07-14T11:20:47.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb37fe700 1 -- 172.21.15.161:0/4060605832 reap_dead start 2023-07-14T11:20:47.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 shutdown_connections 2023-07-14T11:20:47.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 >> 172.21.15.161:0/4060605832 conn(0x7f5bb40fb660 msgr2=0x7f5bb40fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:47.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 shutdown_connections 2023-07-14T11:20:47.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:47.169+0000 7f5bb9a40700 1 -- 172.21.15.161:0/4060605832 wait complete. 2023-07-14T11:20:47.863 INFO:teuthology.orchestra.run.smithi161.stdout:223338299434 2023-07-14T11:20:47.863 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-14T11:20:47.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:47 smithi182 ceph-mon[129366]: pgmap v306: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:20:47.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:47 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:47.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:47 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:47.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:47 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:47.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:47 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:47.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:47 smithi161 ceph-mon[95526]: pgmap v306: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:20:47.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:47 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:47.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:47 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:47.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:47 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:47.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:47 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:47 smithi175 ceph-mon[129081]: pgmap v306: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:20:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:47 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:48.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:47 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:48.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:47 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:48.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:47 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:20:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 -- 172.21.15.161:0/1120362803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c81007a0 msgr2=0x7f15c8104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 --2- 172.21.15.161:0/1120362803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c81007a0 0x7f15c8104770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f15b40099c0 tx=0x7f15b4009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 -- 172.21.15.161:0/1120362803 shutdown_connections 2023-07-14T11:20:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 --2- 172.21.15.161:0/1120362803 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15c8104ea0 0x7f15c810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 --2- 172.21.15.161:0/1120362803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c80ffe00 0x7f15c81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 --2- 172.21.15.161:0/1120362803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c81007a0 0x7f15c8104770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 -- 172.21.15.161:0/1120362803 >> 172.21.15.161:0/1120362803 conn(0x7f15c80fb660 msgr2=0x7f15c80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 -- 172.21.15.161:0/1120362803 shutdown_connections 2023-07-14T11:20:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.210+0000 7f15cf16a700 1 -- 172.21.15.161:0/1120362803 wait complete. 2023-07-14T11:20:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.211+0000 7f15cf16a700 1 Processor -- start 2023-07-14T11:20:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.211+0000 7f15cf16a700 1 -- start start 2023-07-14T11:20:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.211+0000 7f15cf16a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15c80ffe00 0x7f15c819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.211+0000 7f15cf16a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c81007a0 0x7f15c819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15cf16a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8104ea0 0x7f15c8195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15cf16a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c8196160 con 0x7f15c8104ea0 2023-07-14T11:20:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15cf16a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c81962d0 con 0x7f15c81007a0 2023-07-14T11:20:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15cf16a700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c8196490 con 0x7f15c80ffe00 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15c80ffe00 0x7f15c819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15c80ffe00 0x7f15c819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:42284/0 (socket says 172.21.15.161:42284) 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 -- 172.21.15.161:0/4156879038 learned_addr learned my addr 172.21.15.161:0/4156879038 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15cd707700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8104ea0 0x7f15c8195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15c7fff700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c81007a0 0x7f15c819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 -- 172.21.15.161:0/4156879038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c81007a0 msgr2=0x7f15c819bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c81007a0 0x7f15c819bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 -- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8104ea0 msgr2=0x7f15c8195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8104ea0 0x7f15c8195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 -- 172.21.15.161:0/4156879038 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40095f0 con 0x7f15c80ffe00 2023-07-14T11:20:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.212+0000 7f15ccf06700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15c80ffe00 0x7f15c819ba20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00e9a0 tx=0x7f15bc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.213+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc009790 con 0x7f15c80ffe00 2023-07-14T11:20:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.213+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c8196770 con 0x7f15c80ffe00 2023-07-14T11:20:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.213+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15bc010430 con 0x7f15c80ffe00 2023-07-14T11:20:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.213+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc018680 con 0x7f15c80ffe00 2023-07-14T11:20:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.213+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c81a2a10 con 0x7f15c80ffe00 2023-07-14T11:20:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.214+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15c804e210 con 0x7f15c80ffe00 2023-07-14T11:20:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.216+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f15bc0105a0 con 0x7f15c80ffe00 2023-07-14T11:20:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.217+0000 7f15c5ffb700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f15b80a1260 0x7f15b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.217+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f15bc014070 con 0x7f15c80ffe00 2023-07-14T11:20:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.218+0000 7f15c7fff700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f15b80a1260 0x7f15b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.218+0000 7f15c5ffb700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] conn(0x7f15b80ac270 0x7f15b80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.218+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 --> [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15b80aedd0 con 0x7f15b80ac270 2023-07-14T11:20:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.218+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15bc0cd5b0 con 0x7f15c80ffe00 2023-07-14T11:20:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.218+0000 7f15cd707700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] conn(0x7f15b80ac270 0x7f15b80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.218+0000 7f15c7fff700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f15b80a1260 0x7f15b80a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009fd0 tx=0x7f15b4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.219+0000 7f15cd707700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] conn(0x7f15b80ac270 0x7f15b80ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.219+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== osd.9 v2:172.21.15.161:6826/565147230 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f15b80aedd0 con 0x7f15b80ac270 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.238+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 --> [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15c80fca50 con 0x7f15b80ac270 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.238+0000 7f15c5ffb700 1 -- 172.21.15.161:0/4156879038 <== osd.9 v2:172.21.15.161:6826/565147230 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f15c80fca50 con 0x7f15b80ac270 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] conn(0x7f15b80ac270 msgr2=0x7f15b80ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] conn(0x7f15b80ac270 0x7f15b80ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f15b80a1260 msgr2=0x7f15b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f15b80a1260 0x7f15b80a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009fd0 tx=0x7f15b4005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15c80ffe00 msgr2=0x7f15c819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 --2- 172.21.15.161:0/4156879038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15c80ffe00 0x7f15c819ba20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00e9a0 tx=0x7f15bc00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:20:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cd707700 1 -- 172.21.15.161:0/4156879038 reap_dead start 2023-07-14T11:20:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 shutdown_connections 2023-07-14T11:20:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 >> 172.21.15.161:0/4156879038 conn(0x7f15c80fb660 msgr2=0x7f15c80fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 shutdown_connections 2023-07-14T11:20:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.239+0000 7f15cf16a700 1 -- 172.21.15.161:0/4156879038 wait complete. 2023-07-14T11:20:48.869 INFO:teuthology.orchestra.run.smithi161.stdout:253403070502 2023-07-14T11:20:48.870 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-14T11:20:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 -- 172.21.15.161:0/675526837 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba40ffe00 msgr2=0x7f9ba41001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 --2- 172.21.15.161:0/675526837 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba40ffe00 0x7f9ba41001d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009920 tx=0x7f9ba0009c30 comp rx=0 tx=0).stop 2023-07-14T11:20:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 -- 172.21.15.161:0/675526837 shutdown_connections 2023-07-14T11:20:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 --2- 172.21.15.161:0/675526837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba4104ea0 0x7f9ba410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 --2- 172.21.15.161:0/675526837 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba40ffe00 0x7f9ba41001d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 --2- 172.21.15.161:0/675526837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba41007a0 0x7f9ba4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 -- 172.21.15.161:0/675526837 >> 172.21.15.161:0/675526837 conn(0x7f9ba40fb660 msgr2=0x7f9ba40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 -- 172.21.15.161:0/675526837 shutdown_connections 2023-07-14T11:20:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.236+0000 7f9bac05a700 1 -- 172.21.15.161:0/675526837 wait complete. 2023-07-14T11:20:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 Processor -- start 2023-07-14T11:20:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 -- start start 2023-07-14T11:20:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba40ffe00 0x7f9ba4195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41007a0 0x7f9ba4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba4104ea0 0x7f9ba419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419ad00 con 0x7f9ba41007a0 2023-07-14T11:20:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419ae70 con 0x7f9ba40ffe00 2023-07-14T11:20:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.238+0000 7f9bac05a700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419b030 con 0x7f9ba4104ea0 2023-07-14T11:20:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba95f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41007a0 0x7f9ba4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba9df6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba40ffe00 0x7f9ba4195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba95f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41007a0 0x7f9ba4196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52176/0 (socket says 172.21.15.161:52176) 2023-07-14T11:20:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9baa5f7700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba4104ea0 0x7f9ba419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba95f5700 1 -- 172.21.15.161:0/3890308472 learned_addr learned my addr 172.21.15.161:0/3890308472 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba9df6700 1 -- 172.21.15.161:0/3890308472 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba4104ea0 msgr2=0x7f9ba419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba9df6700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba4104ea0 0x7f9ba419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba9df6700 1 -- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41007a0 msgr2=0x7f9ba4196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba9df6700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41007a0 0x7f9ba4196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba9df6700 1 -- 172.21.15.161:0/3890308472 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c009d00 con 0x7f9ba40ffe00 2023-07-14T11:20:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9ba95f5700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41007a0 0x7f9ba4196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:49.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.239+0000 7f9baa5f7700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9ba4104ea0 0x7f9ba419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:49.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.240+0000 7f9ba9df6700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba40ffe00 0x7f9ba4195d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009230 tx=0x7f9ba000fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.240+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0020070 con 0x7f9ba40ffe00 2023-07-14T11:20:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.240+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba0009520 con 0x7f9ba40ffe00 2023-07-14T11:20:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.240+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ba001bd50 con 0x7f9ba40ffe00 2023-07-14T11:20:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.240+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00052a0 con 0x7f9ba40ffe00 2023-07-14T11:20:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.240+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba419b530 con 0x7f9ba40ffe00 2023-07-14T11:20:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.241+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ba410fdb0 con 0x7f9ba40ffe00 2023-07-14T11:20:49.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.243+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9ba0024070 con 0x7f9ba40ffe00 2023-07-14T11:20:49.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.245+0000 7f9b9affd700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b900a1260 0x7f9b900a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.245+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9ba00d24d0 con 0x7f9ba40ffe00 2023-07-14T11:20:49.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.245+0000 7f9ba95f5700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b900a1260 0x7f9b900a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.245+0000 7f9b9affd700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] conn(0x7f9b900ac270 0x7f9b900ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.246+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 --> [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b900aedd0 con 0x7f9b900ac270 2023-07-14T11:20:49.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.246+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ba00d2840 con 0x7f9ba40ffe00 2023-07-14T11:20:49.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.246+0000 7f9baa5f7700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] conn(0x7f9b900ac270 0x7f9b900ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.246+0000 7f9ba95f5700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b900a1260 0x7f9b900a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4197360 tx=0x7f9b94009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.247+0000 7f9baa5f7700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] conn(0x7f9b900ac270 0x7f9b900ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.247+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== osd.1 v2:172.21.15.175:6800/3382092746 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9b900aedd0 con 0x7f9b900ac270 2023-07-14T11:20:49.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.276+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 --> [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ba404e210 con 0x7f9b900ac270 2023-07-14T11:20:49.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.276+0000 7f9b9affd700 1 -- 172.21.15.161:0/3890308472 <== osd.1 v2:172.21.15.175:6800/3382092746 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9ba404e210 con 0x7f9b900ac270 2023-07-14T11:20:49.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] conn(0x7f9b900ac270 msgr2=0x7f9b900ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] conn(0x7f9b900ac270 0x7f9b900ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b900a1260 msgr2=0x7f9b900a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f9b900a1260 0x7f9b900a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4197360 tx=0x7f9b94009260 comp rx=0 tx=0).stop 2023-07-14T11:20:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba40ffe00 msgr2=0x7f9ba4195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 --2- 172.21.15.161:0/3890308472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba40ffe00 0x7f9ba4195d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009230 tx=0x7f9ba000fa20 comp rx=0 tx=0).stop 2023-07-14T11:20:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9baa5f7700 1 -- 172.21.15.161:0/3890308472 reap_dead start 2023-07-14T11:20:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 shutdown_connections 2023-07-14T11:20:49.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 >> 172.21.15.161:0/3890308472 conn(0x7f9ba40fb660 msgr2=0x7f9ba40fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:49.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 shutdown_connections 2023-07-14T11:20:49.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.277+0000 7f9bac05a700 1 -- 172.21.15.161:0/3890308472 wait complete. 2023-07-14T11:20:49.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- 172.21.15.161:0/1442435297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40100df0 msgr2=0x7fed401011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 --2- 172.21.15.161:0/1442435297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40100df0 0x7fed401011c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed300098f0 tx=0x7fed30009c00 comp rx=0 tx=0).stop 2023-07-14T11:20:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- 172.21.15.161:0/1442435297 shutdown_connections 2023-07-14T11:20:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 --2- 172.21.15.161:0/1442435297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed4010a140 0x7fed4010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 --2- 172.21.15.161:0/1442435297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40100df0 0x7fed401011c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 --2- 172.21.15.161:0/1442435297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed40101700 0x7fed40109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- 172.21.15.161:0/1442435297 >> 172.21.15.161:0/1442435297 conn(0x7fed40075b70 msgr2=0x7fed40075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- 172.21.15.161:0/1442435297 shutdown_connections 2023-07-14T11:20:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- 172.21.15.161:0/1442435297 wait complete. 2023-07-14T11:20:49.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 Processor -- start 2023-07-14T11:20:49.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- start start 2023-07-14T11:20:49.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed40100df0 0x7fed400fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40101700 0x7fed400fef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed4010a140 0x7fed400ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed40199cd0 con 0x7fed4010a140 2023-07-14T11:20:49.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed401a86a0 con 0x7fed40100df0 2023-07-14T11:20:49.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed401a8860 con 0x7fed40101700 2023-07-14T11:20:49.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40101700 0x7fed400fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed40100df0 0x7fed400fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed4010a140 0x7fed400ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40101700 0x7fed400fef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:42326/0 (socket says 172.21.15.161:42326) 2023-07-14T11:20:49.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 -- 172.21.15.161:0/2149832176 learned_addr learned my addr 172.21.15.161:0/2149832176 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:49.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed40100df0 0x7fed400fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33836/0 (socket says 172.21.15.161:33836) 2023-07-14T11:20:49.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 -- 172.21.15.161:0/2149832176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed40100df0 msgr2=0x7fed400fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed40100df0 0x7fed400fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 -- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed4010a140 msgr2=0x7fed400ff4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed4010a140 0x7fed400ff4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 -- 172.21.15.161:0/2149832176 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed30009520 con 0x7fed40101700 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3effd700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40101700 0x7fed400fef90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed3400e9c0 tx=0x7fed3400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400cbd0 con 0x7fed40101700 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed3400eeb0 con 0x7fed40101700 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34005580 con 0x7fed40101700 2023-07-14T11:20:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed401a8b40 con 0x7fed40101700 2023-07-14T11:20:49.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.314+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed401a8ff0 con 0x7fed40101700 2023-07-14T11:20:49.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.320+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed4010fd70 con 0x7fed40101700 2023-07-14T11:20:49.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fed34004750 con 0x7fed40101700 2023-07-14T11:20:49.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3cff9700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fed280a1260 0x7fed280a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed34014070 con 0x7fed40101700 2023-07-14T11:20:49.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3f7fe700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fed280a1260 0x7fed280a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3cff9700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] conn(0x7fed280ac270 0x7fed280ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 --> [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed280aedd0 con 0x7fed280ac270 2023-07-14T11:20:49.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed340cd8e0 con 0x7fed40101700 2023-07-14T11:20:49.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3ffff700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] conn(0x7fed280ac270 0x7fed280ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3f7fe700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fed280a1260 0x7fed280a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c010 tx=0x7fed30019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.321+0000 7fed3ffff700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] conn(0x7fed280ac270 0x7fed280ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.327+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== osd.6 v2:172.21.15.161:6818/2715861815 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fed280aedd0 con 0x7fed280ac270 2023-07-14T11:20:49.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.348+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 --> [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed4004e210 con 0x7fed280ac270 2023-07-14T11:20:49.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed3cff9700 1 -- 172.21.15.161:0/2149832176 <== osd.6 v2:172.21.15.161:6818/2715861815 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fed4004e210 con 0x7fed280ac270 2023-07-14T11:20:49.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] conn(0x7fed280ac270 msgr2=0x7fed280ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] conn(0x7fed280ac270 0x7fed280ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fed280a1260 msgr2=0x7fed280a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fed280a1260 0x7fed280a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c010 tx=0x7fed30019040 comp rx=0 tx=0).stop 2023-07-14T11:20:49.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40101700 msgr2=0x7fed400fef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 --2- 172.21.15.161:0/2149832176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed40101700 0x7fed400fef90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed3400e9c0 tx=0x7fed3400ecd0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed3ffff700 1 -- 172.21.15.161:0/2149832176 reap_dead start 2023-07-14T11:20:49.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 shutdown_connections 2023-07-14T11:20:49.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 >> 172.21.15.161:0/2149832176 conn(0x7fed40075b70 msgr2=0x7fed40101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:49.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 shutdown_connections 2023-07-14T11:20:49.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.349+0000 7fed4612c700 1 -- 172.21.15.161:0/2149832176 wait complete. 2023-07-14T11:20:49.954 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:49 smithi161 ceph-mon[95526]: pgmap v307: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:20:49.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:49 smithi182 ceph-mon[129366]: pgmap v307: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:20:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 -- 172.21.15.161:0/1465658704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 msgr2=0x7fe3e0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 --2- 172.21.15.161:0/1465658704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 0x7fe3e0104770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40099c0 tx=0x7fe3d4009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 -- 172.21.15.161:0/1465658704 shutdown_connections 2023-07-14T11:20:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 --2- 172.21.15.161:0/1465658704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e0104ea0 0x7fe3e010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 --2- 172.21.15.161:0/1465658704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe3e00ffe00 0x7fe3e01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 --2- 172.21.15.161:0/1465658704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 0x7fe3e0104770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 -- 172.21.15.161:0/1465658704 >> 172.21.15.161:0/1465658704 conn(0x7fe3e00fb660 msgr2=0x7fe3e00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 -- 172.21.15.161:0/1465658704 shutdown_connections 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.904+0000 7fe3e528b700 1 -- 172.21.15.161:0/1465658704 wait complete. 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.905+0000 7fe3e528b700 1 Processor -- start 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.905+0000 7fe3e528b700 1 -- start start 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.905+0000 7fe3e528b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe3e00ffe00 0x7fe3e0197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.905+0000 7fe3e528b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 0x7fe3e0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3e528b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e0104ea0 0x7fe3e019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3e528b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019cde0 con 0x7fe3e01007a0 2023-07-14T11:20:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3e528b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019cf50 con 0x7fe3e0104ea0 2023-07-14T11:20:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3e528b700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019d110 con 0x7fe3e00ffe00 2023-07-14T11:20:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3de7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 0x7fe3e0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3de7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 0x7fe3e0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52216/0 (socket says 172.21.15.161:52216) 2023-07-14T11:20:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3de7fc700 1 -- 172.21.15.161:0/1616334054 learned_addr learned my addr 172.21.15.161:0/1616334054 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3deffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe3e00ffe00 0x7fe3e0197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3df7fe700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e0104ea0 0x7fe3e019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3deffd700 1 -- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e0104ea0 msgr2=0x7fe3e019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3deffd700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e0104ea0 0x7fe3e019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3deffd700 1 -- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 msgr2=0x7fe3e0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3deffd700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 0x7fe3e0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3deffd700 1 -- 172.21.15.161:0/1616334054 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d40095f0 con 0x7fe3e00ffe00 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3de7fc700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e01007a0 0x7fe3e0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.906+0000 7fe3deffd700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe3e00ffe00 0x7fe3e0197e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e9f0 tx=0x7fe3d000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.907+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000c9c0 con 0x7fe3e00ffe00 2023-07-14T11:20:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.907+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d0010430 con 0x7fe3e00ffe00 2023-07-14T11:20:49.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.907+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e019d3f0 con 0x7fe3e00ffe00 2023-07-14T11:20:49.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.907+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0018680 con 0x7fe3e00ffe00 2023-07-14T11:20:49.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.907+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e01a4c50 con 0x7fe3e00ffe00 2023-07-14T11:20:49.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.908+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3e010fd60 con 0x7fe3e00ffe00 2023-07-14T11:20:49.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.910+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe3d0010860 con 0x7fe3e00ffe00 2023-07-14T11:20:49.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.911+0000 7fe3c7fff700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3c80a1260 0x7fe3c80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.911+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe3d0014070 con 0x7fe3e00ffe00 2023-07-14T11:20:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.911+0000 7fe3de7fc700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3c80a1260 0x7fe3c80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.912+0000 7fe3c7fff700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] conn(0x7fe3c80ac270 0x7fe3c80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.912+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 --> [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3c80aedd0 con 0x7fe3c80ac270 2023-07-14T11:20:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.912+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d00cd000 con 0x7fe3e00ffe00 2023-07-14T11:20:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.912+0000 7fe3de7fc700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3c80a1260 0x7fe3c80a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0199440 tx=0x7fe3d4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.912+0000 7fe3df7fe700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] conn(0x7fe3c80ac270 0x7fe3c80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.912+0000 7fe3df7fe700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] conn(0x7fe3c80ac270 0x7fe3c80ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.913+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== osd.10 v2:172.21.15.175:6824/4263786893 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe3c80aedd0 con 0x7fe3c80ac270 2023-07-14T11:20:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.930+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 --> [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe3e004e210 con 0x7fe3c80ac270 2023-07-14T11:20:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.930+0000 7fe3c7fff700 1 -- 172.21.15.161:0/1616334054 <== osd.10 v2:172.21.15.175:6824/4263786893 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe3e004e210 con 0x7fe3c80ac270 2023-07-14T11:20:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] conn(0x7fe3c80ac270 msgr2=0x7fe3c80ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] conn(0x7fe3c80ac270 0x7fe3c80ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3c80a1260 msgr2=0x7fe3c80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3c80a1260 0x7fe3c80a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0199440 tx=0x7fe3d4005fb0 comp rx=0 tx=0).stop 2023-07-14T11:20:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe3e00ffe00 msgr2=0x7fe3e0197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 --2- 172.21.15.161:0/1616334054 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe3e00ffe00 0x7fe3e0197e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e9f0 tx=0x7fe3d000ed00 comp rx=0 tx=0).stop 2023-07-14T11:20:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3df7fe700 1 -- 172.21.15.161:0/1616334054 reap_dead start 2023-07-14T11:20:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 shutdown_connections 2023-07-14T11:20:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 >> 172.21.15.161:0/1616334054 conn(0x7fe3e00fb660 msgr2=0x7fe3e00fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 shutdown_connections 2023-07-14T11:20:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:49.931+0000 7fe3e528b700 1 -- 172.21.15.161:0/1616334054 wait complete. 2023-07-14T11:20:50.129 INFO:teuthology.orchestra.run.smithi161.stdout:201863462960 2023-07-14T11:20:50.129 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-14T11:20:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:49 smithi175 ceph-mon[129081]: pgmap v307: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:20:50.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 -- 172.21.15.161:0/1028419829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810a5b0 msgr2=0x7fa80810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 --2- 172.21.15.161:0/1028419829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810a5b0 0x7fa80810aa20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009990 tx=0x7fa7f8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 -- 172.21.15.161:0/1028419829 shutdown_connections 2023-07-14T11:20:50.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 --2- 172.21.15.161:0/1028419829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa80810af60 0x7fa8081173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 --2- 172.21.15.161:0/1028419829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa80810f530 0x7fa80810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 --2- 172.21.15.161:0/1028419829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810a5b0 0x7fa80810aa20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 -- 172.21.15.161:0/1028419829 >> 172.21.15.161:0/1028419829 conn(0x7fa80806aad0 msgr2=0x7fa80806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:50.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 -- 172.21.15.161:0/1028419829 shutdown_connections 2023-07-14T11:20:50.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.880+0000 7fa80db31700 1 -- 172.21.15.161:0/1028419829 wait complete. 2023-07-14T11:20:50.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 Processor -- start 2023-07-14T11:20:50.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 -- start start 2023-07-14T11:20:50.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa80810a5b0 0x7fa8081a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810af60 0x7fa8081a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa80810f530 0x7fa8081a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8081a5600 con 0x7fa80810af60 2023-07-14T11:20:50.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8081a5770 con 0x7fa80810f530 2023-07-14T11:20:50.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.882+0000 7fa80db31700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8081a5930 con 0x7fa80810a5b0 2023-07-14T11:20:50.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa80810a5b0 0x7fa8081a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa80810a5b0 0x7fa8081a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:42300/0 (socket says 172.21.15.161:42300) 2023-07-14T11:20:50.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 -- 172.21.15.161:0/3150046721 learned_addr learned my addr 172.21.15.161:0/3150046721 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa807fff700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa80810f530 0x7fa8081a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 -- 172.21.15.161:0/3150046721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa80810f530 msgr2=0x7fa8081a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa80810f530 0x7fa8081a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa806ffd700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810af60 0x7fa8081a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 -- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810af60 msgr2=0x7fa8081a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810af60 0x7fa8081a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa8077fe700 1 -- 172.21.15.161:0/3150046721 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f80095f0 con 0x7fa80810a5b0 2023-07-14T11:20:50.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa807fff700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa80810f530 0x7fa8081a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.883+0000 7fa806ffd700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa80810af60 0x7fa8081a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.884+0000 7fa8077fe700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa80810a5b0 0x7fa8081a0640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e970 tx=0x7fa7f000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.884+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0009790 con 0x7fa80810a5b0 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.884+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7f00044e0 con 0x7fa80810a5b0 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.884+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8081a5c10 con 0x7fa80810a5b0 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.884+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8081ad470 con 0x7fa80810a5b0 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.884+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0003e30 con 0x7fa80810a5b0 2023-07-14T11:20:50.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.885+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7e8000ff0 con 0x7fa80810a5b0 2023-07-14T11:20:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.887+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa7f0004020 con 0x7fa80810a5b0 2023-07-14T11:20:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.889+0000 7fa804ff9700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7f40a1260 0x7fa7f40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.890+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa7f0014070 con 0x7fa80810a5b0 2023-07-14T11:20:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.890+0000 7fa806ffd700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7f40a1260 0x7fa7f40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.890+0000 7fa804ff9700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] conn(0x7fa7f40ac270 0x7fa7f40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.890+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 --> [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7f40aedd0 con 0x7fa7f40ac270 2023-07-14T11:20:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.890+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7f00cf400 con 0x7fa80810a5b0 2023-07-14T11:20:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.890+0000 7fa807fff700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] conn(0x7fa7f40ac270 0x7fa7f40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.891+0000 7fa806ffd700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7f40a1260 0x7fa7f40a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8005cd0 tx=0x7fa7f8005c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.891+0000 7fa807fff700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] conn(0x7fa7f40ac270 0x7fa7f40ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.892+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== osd.0 v2:172.21.15.161:6802/3629732803 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa7f40aedd0 con 0x7fa7f40ac270 2023-07-14T11:20:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.909+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 --> [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7e8002d70 con 0x7fa7f40ac270 2023-07-14T11:20:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.909+0000 7fa804ff9700 1 -- 172.21.15.161:0/3150046721 <== osd.0 v2:172.21.15.161:6802/3629732803 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa7e8002d70 con 0x7fa7f40ac270 2023-07-14T11:20:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] conn(0x7fa7f40ac270 msgr2=0x7fa7f40ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] conn(0x7fa7f40ac270 0x7fa7f40ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7f40a1260 msgr2=0x7fa7f40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7f40a1260 0x7fa7f40a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8005cd0 tx=0x7fa7f8005c60 comp rx=0 tx=0).stop 2023-07-14T11:20:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa80810a5b0 msgr2=0x7fa8081a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 --2- 172.21.15.161:0/3150046721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa80810a5b0 0x7fa8081a0640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e970 tx=0x7fa7f000ed30 comp rx=0 tx=0).stop 2023-07-14T11:20:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa807fff700 1 -- 172.21.15.161:0/3150046721 reap_dead start 2023-07-14T11:20:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 shutdown_connections 2023-07-14T11:20:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 >> 172.21.15.161:0/3150046721 conn(0x7fa80806aad0 msgr2=0x7fa80810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 shutdown_connections 2023-07-14T11:20:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:48.910+0000 7fa80db31700 1 -- 172.21.15.161:0/3150046721 wait complete. 2023-07-14T11:20:50.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.490+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/2727412189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac104ea0 msgr2=0x7f3bac10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.490+0000 7f3bb0e2a700 1 --2- 172.21.15.161:0/2727412189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac104ea0 0x7f3bac10ca20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b0a0 tx=0x7f3ba400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.490+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/2727412189 shutdown_connections 2023-07-14T11:20:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.490+0000 7f3bb0e2a700 1 --2- 172.21.15.161:0/2727412189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac104ea0 0x7f3bac10ca20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.490+0000 7f3bb0e2a700 1 --2- 172.21.15.161:0/2727412189 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3bac0ffe00 0x7f3bac1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.490+0000 7f3bb0e2a700 1 --2- 172.21.15.161:0/2727412189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bac1007a0 0x7f3bac104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.491+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/2727412189 >> 172.21.15.161:0/2727412189 conn(0x7f3bac0fb660 msgr2=0x7f3bac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.491+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/2727412189 shutdown_connections 2023-07-14T11:20:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.491+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/2727412189 wait complete. 2023-07-14T11:20:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 Processor -- start 2023-07-14T11:20:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 -- start start 2023-07-14T11:20:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac0ffe00 0x7f3bac197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bac1007a0 0x7f3bac1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3bac104ea0 0x7f3bac19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac19cf20 con 0x7f3bac1007a0 2023-07-14T11:20:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac19d090 con 0x7f3bac0ffe00 2023-07-14T11:20:50.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.492+0000 7f3bb0e2a700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac19d250 con 0x7f3bac104ea0 2023-07-14T11:20:50.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac0ffe00 0x7f3bac197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac0ffe00 0x7f3bac197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33876/0 (socket says 172.21.15.161:33876) 2023-07-14T11:20:50.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 -- 172.21.15.161:0/1585531641 learned_addr learned my addr 172.21.15.161:0/1585531641 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:50.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3ba9d9b700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bac1007a0 0x7f3bac1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baad9d700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3bac104ea0 0x7f3bac19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 -- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3bac104ea0 msgr2=0x7f3bac19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3bac104ea0 0x7f3bac19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 -- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bac1007a0 msgr2=0x7f3bac1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bac1007a0 0x7f3bac1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 -- 172.21.15.161:0/1585531641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4009d00 con 0x7f3bac0ffe00 2023-07-14T11:20:50.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baad9d700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3bac104ea0 0x7f3bac19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:50.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3baa59c700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac0ffe00 0x7f3bac197f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00e9a0 tx=0x7f3b9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.493+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c009790 con 0x7f3bac0ffe00 2023-07-14T11:20:50.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.494+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b9c00cbd0 con 0x7f3bac0ffe00 2023-07-14T11:20:50.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.494+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac19d530 con 0x7f3bac0ffe00 2023-07-14T11:20:50.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.494+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c018680 con 0x7f3bac0ffe00 2023-07-14T11:20:50.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.494+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac1a4d90 con 0x7f3bac0ffe00 2023-07-14T11:20:50.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.494+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3bac04e210 con 0x7f3bac0ffe00 2023-07-14T11:20:50.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.497+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3b9c00cd40 con 0x7f3bac0ffe00 2023-07-14T11:20:50.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.498+0000 7f3b9b7fe700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b940a11e0 0x7f3b940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.498+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3b9c014070 con 0x7f3bac0ffe00 2023-07-14T11:20:50.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.498+0000 7f3ba9d9b700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b940a11e0 0x7f3b940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.499+0000 7f3b9b7fe700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] conn(0x7f3b940ac1f0 0x7f3b940ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.499+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 --> [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b940aed50 con 0x7f3b940ac1f0 2023-07-14T11:20:50.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.499+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b9c0cd7b0 con 0x7f3bac0ffe00 2023-07-14T11:20:50.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.499+0000 7f3baad9d700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] conn(0x7f3b940ac1f0 0x7f3b940ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.499+0000 7f3ba9d9b700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b940a11e0 0x7f3b940a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bac199580 tx=0x7f3ba0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.500+0000 7f3baad9d700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] conn(0x7f3b940ac1f0 0x7f3b940ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.500+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== osd.5 v2:172.21.15.182:6808/4034252274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3b940aed50 con 0x7f3b940ac1f0 2023-07-14T11:20:50.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.517+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 --> [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3bac10fda0 con 0x7f3b940ac1f0 2023-07-14T11:20:50.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.517+0000 7f3b9b7fe700 1 -- 172.21.15.161:0/1585531641 <== osd.5 v2:172.21.15.182:6808/4034252274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3ba4009420 con 0x7f3b940ac1f0 2023-07-14T11:20:50.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] conn(0x7f3b940ac1f0 msgr2=0x7f3b940ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] conn(0x7f3b940ac1f0 0x7f3b940ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b940a11e0 msgr2=0x7f3b940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b940a11e0 0x7f3b940a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bac199580 tx=0x7f3ba0009260 comp rx=0 tx=0).stop 2023-07-14T11:20:50.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac0ffe00 msgr2=0x7f3bac197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 --2- 172.21.15.161:0/1585531641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bac0ffe00 0x7f3bac197f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00e9a0 tx=0x7f3b9c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3baad9d700 1 -- 172.21.15.161:0/1585531641 reap_dead start 2023-07-14T11:20:50.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 shutdown_connections 2023-07-14T11:20:50.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 >> 172.21.15.161:0/1585531641 conn(0x7f3bac0fb660 msgr2=0x7f3bac0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:50.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 shutdown_connections 2023-07-14T11:20:50.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.518+0000 7f3bb0e2a700 1 -- 172.21.15.161:0/1585531641 wait complete. 2023-07-14T11:20:50.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 -- 172.21.15.161:0/1951123306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 msgr2=0x7fa140104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 --2- 172.21.15.161:0/1951123306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 0x7fa140104770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1300099c0 tx=0x7fa130009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 -- 172.21.15.161:0/1951123306 shutdown_connections 2023-07-14T11:20:50.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 --2- 172.21.15.161:0/1951123306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa140104ea0 0x7fa14010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 --2- 172.21.15.161:0/1951123306 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa1400ffe00 0x7fa1401001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 --2- 172.21.15.161:0/1951123306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 0x7fa140104770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 -- 172.21.15.161:0/1951123306 >> 172.21.15.161:0/1951123306 conn(0x7fa1400fb660 msgr2=0x7fa1400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:50.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 -- 172.21.15.161:0/1951123306 shutdown_connections 2023-07-14T11:20:50.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.982+0000 7fa148157700 1 -- 172.21.15.161:0/1951123306 wait complete. 2023-07-14T11:20:50.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.983+0000 7fa148157700 1 Processor -- start 2023-07-14T11:20:50.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.983+0000 7fa148157700 1 -- start start 2023-07-14T11:20:50.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa148157700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1400ffe00 0x7fa140072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa148157700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 0x7fa140072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa148157700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa140104ea0 0x7fa1400732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa148157700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401a41a0 con 0x7fa1401007a0 2023-07-14T11:20:50.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa148157700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401a4310 con 0x7fa1400ffe00 2023-07-14T11:20:50.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa148157700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401a44d0 con 0x7fa140104ea0 2023-07-14T11:20:50.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa1456f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 0x7fa140072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1400ffe00 0x7fa140072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1400ffe00 0x7fa140072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33886/0 (socket says 172.21.15.161:33886) 2023-07-14T11:20:50.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa1456f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 0x7fa140072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52244/0 (socket says 172.21.15.161:52244) 2023-07-14T11:20:50.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 -- 172.21.15.161:0/3710162152 learned_addr learned my addr 172.21.15.161:0/3710162152 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:50.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa1466f4700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa140104ea0 0x7fa1400732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 -- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa140104ea0 msgr2=0x7fa1400732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa140104ea0 0x7fa1400732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 -- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 msgr2=0x7fa140072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:50.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 0x7fa140072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:50.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa145ef3700 1 -- 172.21.15.161:0/3710162152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300095f0 con 0x7fa1400ffe00 2023-07-14T11:20:50.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.984+0000 7fa1456f2700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1401007a0 0x7fa140072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:50.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.985+0000 7fa145ef3700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1400ffe00 0x7fa140072810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9a0 tx=0x7fa13c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.985+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c009790 con 0x7fa1400ffe00 2023-07-14T11:20:50.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.985+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1401a47b0 con 0x7fa1400ffe00 2023-07-14T11:20:50.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.985+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa13c010430 con 0x7fa1400ffe00 2023-07-14T11:20:50.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.985+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c018680 con 0x7fa1400ffe00 2023-07-14T11:20:50.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.985+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401a4cd0 con 0x7fa1400ffe00 2023-07-14T11:20:50.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.986+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa14010fd50 con 0x7fa1400ffe00 2023-07-14T11:20:50.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.988+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa13c0105a0 con 0x7fa1400ffe00 2023-07-14T11:20:50.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.989+0000 7fa136ffd700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa12c0a11e0 0x7fa12c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.989+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa13c014070 con 0x7fa1400ffe00 2023-07-14T11:20:50.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.989+0000 7fa1456f2700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa12c0a11e0 0x7fa12c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.990+0000 7fa136ffd700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] conn(0x7fa12c0ac1f0 0x7fa12c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:50.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.990+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 --> [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa12c0aed50 con 0x7fa12c0ac1f0 2023-07-14T11:20:50.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.990+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa13c0cd610 con 0x7fa1400ffe00 2023-07-14T11:20:50.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.990+0000 7fa1466f4700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] conn(0x7fa12c0ac1f0 0x7fa12c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:50.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.990+0000 7fa1456f2700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa12c0a11e0 0x7fa12c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa1300098e0 tx=0x7fa13001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.991+0000 7fa1466f4700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] conn(0x7fa12c0ac1f0 0x7fa12c0ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:50.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:50.991+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== osd.11 v2:172.21.15.182:6824/2744013300 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa12c0aed50 con 0x7fa12c0ac1f0 2023-07-14T11:20:51.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.012+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 --> [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa14004e210 con 0x7fa12c0ac1f0 2023-07-14T11:20:51.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa136ffd700 1 -- 172.21.15.161:0/3710162152 <== osd.11 v2:172.21.15.182:6824/2744013300 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa14004e210 con 0x7fa12c0ac1f0 2023-07-14T11:20:51.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] conn(0x7fa12c0ac1f0 msgr2=0x7fa12c0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] conn(0x7fa12c0ac1f0 0x7fa12c0ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa12c0a11e0 msgr2=0x7fa12c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa12c0a11e0 0x7fa12c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa1300098e0 tx=0x7fa13001a040 comp rx=0 tx=0).stop 2023-07-14T11:20:51.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1400ffe00 msgr2=0x7fa140072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 --2- 172.21.15.161:0/3710162152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1400ffe00 0x7fa140072810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9a0 tx=0x7fa13c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa1466f4700 1 -- 172.21.15.161:0/3710162152 reap_dead start 2023-07-14T11:20:51.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 shutdown_connections 2023-07-14T11:20:51.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.013+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 >> 172.21.15.161:0/3710162152 conn(0x7fa1400fb660 msgr2=0x7fa1400fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:51.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.014+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 shutdown_connections 2023-07-14T11:20:51.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.014+0000 7fa148157700 1 -- 172.21.15.161:0/3710162152 wait complete. 2023-07-14T11:20:51.041 INFO:teuthology.orchestra.run.smithi161.stdout:287762808866 2023-07-14T11:20:51.041 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-14T11:20:51.591 INFO:teuthology.orchestra.run.smithi161.stdout:309237645341 2023-07-14T11:20:51.592 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-14T11:20:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 -- 172.21.15.161:0/1707270202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00ffe00 msgr2=0x7f64d01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 --2- 172.21.15.161:0/1707270202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00ffe00 0x7f64d01001d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64c40099c0 tx=0x7f64c4009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 -- 172.21.15.161:0/1707270202 shutdown_connections 2023-07-14T11:20:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 --2- 172.21.15.161:0/1707270202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00ffe00 0x7f64d01001d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 --2- 172.21.15.161:0/1707270202 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64d0104ea0 0x7f64d010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 --2- 172.21.15.161:0/1707270202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64d01007a0 0x7f64d0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 -- 172.21.15.161:0/1707270202 >> 172.21.15.161:0/1707270202 conn(0x7f64d00fb660 msgr2=0x7f64d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 -- 172.21.15.161:0/1707270202 shutdown_connections 2023-07-14T11:20:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.631+0000 7f64d5eb9700 1 -- 172.21.15.161:0/1707270202 wait complete. 2023-07-14T11:20:51.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.633+0000 7f64d5eb9700 1 Processor -- start 2023-07-14T11:20:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.633+0000 7f64d5eb9700 1 -- start start 2023-07-14T11:20:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.633+0000 7f64d5eb9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64d00ffe00 0x7f64d019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.633+0000 7f64d5eb9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d01007a0 0x7f64d019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:51.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64d5eb9700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64d0104ea0 0x7f64d0195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:51.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64d5eb9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0196140 con 0x7f64d01007a0 2023-07-14T11:20:51.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64d5eb9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d01962b0 con 0x7f64d00ffe00 2023-07-14T11:20:51.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64d5eb9700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0196470 con 0x7f64d0104ea0 2023-07-14T11:20:51.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64ceffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d01007a0 0x7f64d019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:51.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64cf7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64d00ffe00 0x7f64d019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:51.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64cf7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64d00ffe00 0x7f64d019ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33910/0 (socket says 172.21.15.161:33910) 2023-07-14T11:20:51.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64ceffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d01007a0 0x7f64d019bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52254/0 (socket says 172.21.15.161:52254) 2023-07-14T11:20:51.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64cf7fe700 1 -- 172.21.15.161:0/3269667396 learned_addr learned my addr 172.21.15.161:0/3269667396 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:51.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.634+0000 7f64cffff700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64d0104ea0 0x7f64d0195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:51.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64cf7fe700 1 -- 172.21.15.161:0/3269667396 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64d0104ea0 msgr2=0x7f64d0195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64cf7fe700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64d0104ea0 0x7f64d0195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64cf7fe700 1 -- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d01007a0 msgr2=0x7f64d019bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64cf7fe700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d01007a0 0x7f64d019bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64cf7fe700 1 -- 172.21.15.161:0/3269667396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40095f0 con 0x7f64d00ffe00 2023-07-14T11:20:51.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64ceffd700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d01007a0 0x7f64d019bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:51.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64cf7fe700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64d00ffe00 0x7f64d019ba50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64c400b580 tx=0x7f64c400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:51.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c401d070 con 0x7f64d00ffe00 2023-07-14T11:20:51.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64c4022470 con 0x7f64d00ffe00 2023-07-14T11:20:51.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d01966f0 con 0x7f64d00ffe00 2023-07-14T11:20:51.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.635+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4017650 con 0x7f64d00ffe00 2023-07-14T11:20:51.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.636+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d01a2a40 con 0x7f64d00ffe00 2023-07-14T11:20:51.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.636+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64d010fd50 con 0x7f64d00ffe00 2023-07-14T11:20:51.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.639+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f64c40225e0 con 0x7f64d00ffe00 2023-07-14T11:20:51.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.641+0000 7f64ccff9700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64b80a1260 0x7f64b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:51.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.641+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f64c40ce360 con 0x7f64d00ffe00 2023-07-14T11:20:51.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.641+0000 7f64ceffd700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64b80a1260 0x7f64b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:51.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.642+0000 7f64ccff9700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] conn(0x7f64b80ac270 0x7f64b80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:51.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.642+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 --> [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64b80aedd0 con 0x7f64b80ac270 2023-07-14T11:20:51.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.642+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64c40ce6d0 con 0x7f64d00ffe00 2023-07-14T11:20:51.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.642+0000 7f64cffff700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] conn(0x7f64b80ac270 0x7f64b80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:51.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.642+0000 7f64ceffd700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64b80a1260 0x7f64b80a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f64d00febe0 tx=0x7f64c0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:51.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.643+0000 7f64cffff700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] conn(0x7f64b80ac270 0x7f64b80ae700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:51.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.643+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== osd.4 v2:172.21.15.175:6808/1496586319 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f64b80aedd0 con 0x7f64b80ac270 2023-07-14T11:20:51.647 INFO:teuthology.orchestra.run.smithi161.stdout:339302416411 2023-07-14T11:20:51.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-14T11:20:51.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.661+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 --> [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64d00fcad0 con 0x7f64b80ac270 2023-07-14T11:20:51.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.661+0000 7f64ccff9700 1 -- 172.21.15.161:0/3269667396 <== osd.4 v2:172.21.15.175:6808/1496586319 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f64d00fcad0 con 0x7f64b80ac270 2023-07-14T11:20:51.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] conn(0x7f64b80ac270 msgr2=0x7f64b80ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] conn(0x7f64b80ac270 0x7f64b80ae700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64b80a1260 msgr2=0x7f64b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64b80a1260 0x7f64b80a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f64d00febe0 tx=0x7f64c0009260 comp rx=0 tx=0).stop 2023-07-14T11:20:51.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64d00ffe00 msgr2=0x7f64d019ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:51.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 --2- 172.21.15.161:0/3269667396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64d00ffe00 0x7f64d019ba50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64c400b580 tx=0x7f64c400bfd0 comp rx=0 tx=0).stop 2023-07-14T11:20:51.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64cffff700 1 -- 172.21.15.161:0/3269667396 reap_dead start 2023-07-14T11:20:51.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 shutdown_connections 2023-07-14T11:20:51.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.662+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 >> 172.21.15.161:0/3269667396 conn(0x7f64d00fb660 msgr2=0x7f64d00fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:51.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.663+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 shutdown_connections 2023-07-14T11:20:51.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:51.663+0000 7f64d5eb9700 1 -- 172.21.15.161:0/3269667396 wait complete. 2023-07-14T11:20:51.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:51 smithi182 ceph-mon[129366]: pgmap v308: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-07-14T11:20:51.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:51 smithi161 ceph-mon[95526]: pgmap v308: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-07-14T11:20:52.127 INFO:teuthology.orchestra.run.smithi161.stdout:167503724597 2023-07-14T11:20:52.128 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-14T11:20:52.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:51 smithi175 ceph-mon[129081]: pgmap v308: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-07-14T11:20:52.332 INFO:teuthology.orchestra.run.smithi161.stdout:55834574917 2023-07-14T11:20:52.333 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-14T11:20:52.354 INFO:teuthology.orchestra.run.smithi161.stdout:146028888122 2023-07-14T11:20:52.355 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-14T11:20:52.874 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:52.993 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:52.993 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:34359738444 2023-07-14T11:20:53.035 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-14T11:20:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:53.902 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:53.948 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:53.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:53 smithi182 ceph-mon[129366]: pgmap v309: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:20:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:53 smithi161 ceph-mon[95526]: pgmap v309: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:20:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:53 smithi175 ceph-mon[129081]: pgmap v309: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:20:54.180 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:55.317 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:55.869 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:55 smithi161 ceph-mon[95526]: pgmap v310: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:20:55.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:55 smithi182 ceph-mon[129366]: pgmap v310: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:20:56.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:55 smithi175 ceph-mon[129081]: pgmap v310: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:20:56.321 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:57.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:57 smithi161 ceph-mon[95526]: pgmap v311: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:20:58.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:57 smithi175 ceph-mon[129081]: pgmap v311: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:20:58.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:57 smithi182 ceph-mon[129366]: pgmap v311: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:20:58.584 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:58.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 -- 172.21.15.161:0/1311581395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c1007a0 msgr2=0x7f0f9c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:58.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/1311581395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c1007a0 0x7f0f9c104770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f900099c0 tx=0x7f0f90009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:58.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 -- 172.21.15.161:0/1311581395 shutdown_connections 2023-07-14T11:20:58.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/1311581395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f9c104ea0 0x7f0f9c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:58.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/1311581395 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0f9c0ffe00 0x7f0f9c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:58.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/1311581395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c1007a0 0x7f0f9c104770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:58.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 -- 172.21.15.161:0/1311581395 >> 172.21.15.161:0/1311581395 conn(0x7f0f9c0fb660 msgr2=0x7f0f9c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:58.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 -- 172.21.15.161:0/1311581395 shutdown_connections 2023-07-14T11:20:58.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.842+0000 7f0fa0bab700 1 -- 172.21.15.161:0/1311581395 wait complete. 2023-07-14T11:20:58.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.843+0000 7f0fa0bab700 1 Processor -- start 2023-07-14T11:20:58.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0fa0bab700 1 -- start start 2023-07-14T11:20:58.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0fa0bab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c0ffe00 0x7f0f9c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:58.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0fa0bab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f9c1007a0 0x7f0f9c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:58.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0fa0bab700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0f9c104ea0 0x7f0f9c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:58.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0fa0bab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c19cf10 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0fa0bab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c19d080 con 0x7f0f9c1007a0 2023-07-14T11:20:58.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0fa0bab700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c19d240 con 0x7f0f9c104ea0 2023-07-14T11:20:58.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c0ffe00 0x7f0f9c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:58.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c0ffe00 0x7f0f9c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58994/0 (socket says 172.21.15.161:58994) 2023-07-14T11:20:58.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0f9a59c700 1 -- 172.21.15.161:0/3765694707 learned_addr learned my addr 172.21.15.161:0/3765694707 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:58.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0f9ad9d700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0f9c104ea0 0x7f0f9c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:58.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.844+0000 7f0f99d9b700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f9c1007a0 0x7f0f9c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:58.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f9a59c700 1 -- 172.21.15.161:0/3765694707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0f9c104ea0 msgr2=0x7f0f9c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:58.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f9a59c700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0f9c104ea0 0x7f0f9c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:58.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f9a59c700 1 -- 172.21.15.161:0/3765694707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f9c1007a0 msgr2=0x7f0f9c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:58.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f9a59c700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f9c1007a0 0x7f0f9c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:58.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f9a59c700 1 -- 172.21.15.161:0/3765694707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f900095f0 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f9a59c700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c0ffe00 0x7f0f9c197f50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d950 tx=0x7f0f8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:58.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f8b7fe700 1 -- 172.21.15.161:0/3765694707 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00f890 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c19d520 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f8b7fe700 1 -- 172.21.15.161:0/3765694707 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f8c010460 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0f8b7fe700 1 -- 172.21.15.161:0/3765694707 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00e610 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.845+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c1a4d80 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.847+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c04e210 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.848+0000 7f0f8b7fe700 1 -- 172.21.15.161:0/3765694707 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0f8c0108d0 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.851+0000 7f0f8b7fe700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0f840a11e0 0x7f0f840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:58.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.851+0000 7f0f8b7fe700 1 -- 172.21.15.161:0/3765694707 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f8c0ccbf0 con 0x7f0f9c0ffe00 2023-07-14T11:20:58.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.851+0000 7f0f99d9b700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0f840a11e0 0x7f0f840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:58.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.852+0000 7f0f99d9b700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0f840a11e0 0x7f0f840a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f900098e0 tx=0x7f0f90000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:58.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:58.855+0000 7f0f8b7fe700 1 -- 172.21.15.161:0/3765694707 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0f8c095e70 con 0x7f0f9c0ffe00 2023-07-14T11:20:59.075 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:20:59.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.100+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0f9c1a4f10 con 0x7f0f9c0ffe00 2023-07-14T11:20:59.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.101+0000 7f0f8b7fe700 1 -- 172.21.15.161:0/3765694707 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f8c016090 con 0x7f0f9c0ffe00 2023-07-14T11:20:59.105 INFO:teuthology.orchestra.run.smithi161.stdout:201863462962 2023-07-14T11:20:59.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0f840a11e0 msgr2=0x7f0f840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0f840a11e0 0x7f0f840a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f900098e0 tx=0x7f0f90000bc0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c0ffe00 msgr2=0x7f0f9c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c0ffe00 0x7f0f9c197f50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d950 tx=0x7f0f8c00dc60 comp rx=0 tx=0).stop 2023-07-14T11:20:59.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 shutdown_connections 2023-07-14T11:20:59.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9c0ffe00 0x7f0f9c197f50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f0f840a11e0 0x7f0f840a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0f9c104ea0 0x7f0f9c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 --2- 172.21.15.161:0/3765694707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f9c1007a0 0x7f0f9c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 >> 172.21.15.161:0/3765694707 conn(0x7f0f9c0fb660 msgr2=0x7f0f9c0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:59.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 shutdown_connections 2023-07-14T11:20:59.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.103+0000 7f0fa0bab700 1 -- 172.21.15.161:0/3765694707 wait complete. 2023-07-14T11:20:59.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.231+0000 7f687792c700 1 -- 172.21.15.161:0/4117965235 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 msgr2=0x7f6870100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.231+0000 7f687792c700 1 --2- 172.21.15.161:0/4117965235 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870100200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f686c0099c0 tx=0x7f686c009cd0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.232+0000 7f687792c700 1 -- 172.21.15.161:0/4117965235 shutdown_connections 2023-07-14T11:20:59.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.232+0000 7f687792c700 1 --2- 172.21.15.161:0/4117965235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6870104ec0 0x7f687010ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.232+0000 7f687792c700 1 --2- 172.21.15.161:0/4117965235 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870100200 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.232+0000 7f687792c700 1 --2- 172.21.15.161:0/4117965235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68701007d0 0x7f6870104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.232+0000 7f687792c700 1 -- 172.21.15.161:0/4117965235 >> 172.21.15.161:0/4117965235 conn(0x7f6870077b70 msgr2=0x7f6870077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:59.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.232+0000 7f687792c700 1 -- 172.21.15.161:0/4117965235 shutdown_connections 2023-07-14T11:20:59.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.232+0000 7f687792c700 1 -- 172.21.15.161:0/4117965235 wait complete. 2023-07-14T11:20:59.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 Processor -- start 2023-07-14T11:20:59.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 -- start start 2023-07-14T11:20:59.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68701007d0 0x7f68701983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6870104ec0 0x7f687019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019ce70 con 0x7f6870104ec0 2023-07-14T11:20:59.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019cfe0 con 0x7f68701007d0 2023-07-14T11:20:59.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.233+0000 7f687792c700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019d1a0 con 0x7f68700ffe30 2023-07-14T11:20:59.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36948/0 (socket says 172.21.15.161:36948) 2023-07-14T11:20:59.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 -- 172.21.15.161:0/2486140602 learned_addr learned my addr 172.21.15.161:0/2486140602 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:59.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f6874ec7700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68701007d0 0x7f68701983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f6875ec9700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6870104ec0 0x7f687019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 -- 172.21.15.161:0/2486140602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68701007d0 msgr2=0x7f68701983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68701007d0 0x7f68701983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 -- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6870104ec0 msgr2=0x7f687019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6870104ec0 0x7f687019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 -- 172.21.15.161:0/2486140602 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6860009520 con 0x7f68700ffe30 2023-07-14T11:20:59.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f6875ec9700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6870104ec0 0x7f687019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:20:59.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.234+0000 7f68756c8700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870197eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f686c00b580 tx=0x7f686c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:59.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.235+0000 7f68667fc700 1 -- 172.21.15.161:0/2486140602 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c01d070 con 0x7f68700ffe30 2023-07-14T11:20:59.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.235+0000 7f68667fc700 1 -- 172.21.15.161:0/2486140602 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f686c005050 con 0x7f68700ffe30 2023-07-14T11:20:59.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.235+0000 7f68667fc700 1 -- 172.21.15.161:0/2486140602 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c004a40 con 0x7f68700ffe30 2023-07-14T11:20:59.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.235+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c0095f0 con 0x7f68700ffe30 2023-07-14T11:20:59.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.235+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687019d650 con 0x7f68700ffe30 2023-07-14T11:20:59.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.236+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687004e210 con 0x7f68700ffe30 2023-07-14T11:20:59.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.238+0000 7f68667fc700 1 -- 172.21.15.161:0/2486140602 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f686c0051c0 con 0x7f68700ffe30 2023-07-14T11:20:59.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.239+0000 7f68667fc700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f685c0a11e0 0x7f685c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.240+0000 7f68667fc700 1 -- 172.21.15.161:0/2486140602 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f686c0ce090 con 0x7f68700ffe30 2023-07-14T11:20:59.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.240+0000 7f6874ec7700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f685c0a11e0 0x7f685c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.240+0000 7f6874ec7700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f685c0a11e0 0x7f685c0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f68701994d0 tx=0x7f6860009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:59.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.241+0000 7f68667fc700 1 -- 172.21.15.161:0/2486140602 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f686c0973a0 con 0x7f68700ffe30 2023-07-14T11:20:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.452+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f6870199210 con 0x7f68700ffe30 2023-07-14T11:20:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.452+0000 7f68667fc700 1 -- 172.21.15.161:0/2486140602 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f686c09a9c0 con 0x7f68700ffe30 2023-07-14T11:20:59.456 INFO:teuthology.orchestra.run.smithi161.stdout:253403070504 2023-07-14T11:20:59.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f685c0a11e0 msgr2=0x7f685c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f685c0a11e0 0x7f685c0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f68701994d0 tx=0x7f6860009260 comp rx=0 tx=0).stop 2023-07-14T11:20:59.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 msgr2=0x7f6870197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870197eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f686c00b580 tx=0x7f686c00bfd0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 shutdown_connections 2023-07-14T11:20:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6870104ec0 0x7f687019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f685c0a11e0 0x7f685c0a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68700ffe30 0x7f6870197eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 --2- 172.21.15.161:0/2486140602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68701007d0 0x7f68701983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.455+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 >> 172.21.15.161:0/2486140602 conn(0x7f6870077b70 msgr2=0x7f687010bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:59.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.456+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 shutdown_connections 2023-07-14T11:20:59.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.456+0000 7f687792c700 1 -- 172.21.15.161:0/2486140602 wait complete. 2023-07-14T11:20:59.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.636+0000 7f022cce7700 1 -- 172.21.15.161:0/1466811592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280728b0 msgr2=0x7f0228072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.636+0000 7f022cce7700 1 --2- 172.21.15.161:0/1466811592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280728b0 0x7f0228072c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f02180098f0 tx=0x7f0218009c00 comp rx=0 tx=0).stop 2023-07-14T11:20:59.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.637+0000 7f022cce7700 1 -- 172.21.15.161:0/1466811592 shutdown_connections 2023-07-14T11:20:59.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.637+0000 7f022cce7700 1 --2- 172.21.15.161:0/1466811592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f022810c3e0 0x7f022810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.637+0000 7f022cce7700 1 --2- 172.21.15.161:0/1466811592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280728b0 0x7f0228072c80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.637+0000 7f022cce7700 1 --2- 172.21.15.161:0/1466811592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02280731c0 0x7f022810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.637+0000 7f022cce7700 1 -- 172.21.15.161:0/1466811592 >> 172.21.15.161:0/1466811592 conn(0x7f02280fb660 msgr2=0x7f02280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.637+0000 7f022cce7700 1 -- 172.21.15.161:0/1466811592 shutdown_connections 2023-07-14T11:20:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.637+0000 7f022cce7700 1 -- 172.21.15.161:0/1466811592 wait complete. 2023-07-14T11:20:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.638+0000 7f022cce7700 1 Processor -- start 2023-07-14T11:20:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.639+0000 7f022cce7700 1 -- start start 2023-07-14T11:20:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f022cce7700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f02280728b0 0x7f0228197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f022cce7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280731c0 0x7f02281983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f022cce7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022810c3e0 0x7f022819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f022cce7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819ceb0 con 0x7f022810c3e0 2023-07-14T11:20:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f022cce7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819d020 con 0x7f02280731c0 2023-07-14T11:20:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f022cce7700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819d1e0 con 0x7f02280728b0 2023-07-14T11:20:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f0226d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022810c3e0 0x7f022819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.640+0000 7f0226d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022810c3e0 0x7f022819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59022/0 (socket says 172.21.15.161:59022) 2023-07-14T11:20:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f0226d9d700 1 -- 172.21.15.161:0/2707785923 learned_addr learned my addr 172.21.15.161:0/2707785923 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f0225d9b700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280731c0 0x7f02281983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f022659c700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f02280728b0 0x7f0228197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f022659c700 1 -- 172.21.15.161:0/2707785923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280731c0 msgr2=0x7f02281983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f022659c700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280731c0 0x7f02281983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f022659c700 1 -- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022810c3e0 msgr2=0x7f022819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f022659c700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022810c3e0 0x7f022819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f022659c700 1 -- 172.21.15.161:0/2707785923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218009520 con 0x7f02280728b0 2023-07-14T11:20:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.641+0000 7f0226d9d700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022810c3e0 0x7f022819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:59.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.645+0000 7f022659c700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f02280728b0 0x7f0228197e60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0218009810 tx=0x7f021800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:59.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.646+0000 7f02177fe700 1 -- 172.21.15.161:0/2707785923 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021801c070 con 0x7f02280728b0 2023-07-14T11:20:59.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.646+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022819d460 con 0x7f02280728b0 2023-07-14T11:20:59.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.646+0000 7f02177fe700 1 -- 172.21.15.161:0/2707785923 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0218005040 con 0x7f02280728b0 2023-07-14T11:20:59.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.646+0000 7f02177fe700 1 -- 172.21.15.161:0/2707785923 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218003970 con 0x7f02280728b0 2023-07-14T11:20:59.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.646+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02280ffc00 con 0x7f02280728b0 2023-07-14T11:20:59.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.647+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022804e210 con 0x7f02280728b0 2023-07-14T11:20:59.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.649+0000 7f02177fe700 1 -- 172.21.15.161:0/2707785923 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0218020030 con 0x7f02280728b0 2023-07-14T11:20:59.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.650+0000 7f02177fe700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f02100a1260 0x7f02100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.651+0000 7f02177fe700 1 -- 172.21.15.161:0/2707785923 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f02180cdf60 con 0x7f02280728b0 2023-07-14T11:20:59.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.651+0000 7f0225d9b700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f02100a1260 0x7f02100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.651+0000 7f0225d9b700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f02100a1260 0x7f02100a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0228199480 tx=0x7f021c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:59.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.652+0000 7f02177fe700 1 -- 172.21.15.161:0/2707785923 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f02180972d0 con 0x7f02280728b0 2023-07-14T11:20:59.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:59 smithi161 ceph-mon[95526]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:20:59.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:59 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3765694707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T11:20:59.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:20:59 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2486140602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T11:20:59.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.874+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f02280ffe20 con 0x7f02280728b0 2023-07-14T11:20:59.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.874+0000 7f02177fe700 1 -- 172.21.15.161:0/2707785923 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f021809a8f0 con 0x7f02280728b0 2023-07-14T11:20:59.879 INFO:teuthology.orchestra.run.smithi161.stdout:111669149759 2023-07-14T11:20:59.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f02100a1260 msgr2=0x7f02100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f02100a1260 0x7f02100a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0228199480 tx=0x7f021c009260 comp rx=0 tx=0).stop 2023-07-14T11:20:59.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f02280728b0 msgr2=0x7f0228197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f02280728b0 0x7f0228197e60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0218009810 tx=0x7f021800fa00 comp rx=0 tx=0).stop 2023-07-14T11:20:59.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 shutdown_connections 2023-07-14T11:20:59.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022810c3e0 0x7f022819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f02100a1260 0x7f02100a3710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f02280728b0 0x7f0228197e60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 --2- 172.21.15.161:0/2707785923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02280731c0 0x7f02281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 >> 172.21.15.161:0/2707785923 conn(0x7f02280fb660 msgr2=0x7f022810e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:59.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 shutdown_connections 2023-07-14T11:20:59.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.877+0000 7f022cce7700 1 -- 172.21.15.161:0/2707785923 wait complete. 2023-07-14T11:20:59.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.938+0000 7f64ad16f700 1 -- 172.21.15.161:0/593377288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a8104ea0 msgr2=0x7f64a810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 --2- 172.21.15.161:0/593377288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a8104ea0 0x7f64a810ca20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64a000b0d0 tx=0x7f64a000b3e0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 -- 172.21.15.161:0/593377288 shutdown_connections 2023-07-14T11:20:59.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 --2- 172.21.15.161:0/593377288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a8104ea0 0x7f64a810ca20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 --2- 172.21.15.161:0/593377288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 0x7f64a81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 --2- 172.21.15.161:0/593377288 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64a81007a0 0x7f64a8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 -- 172.21.15.161:0/593377288 >> 172.21.15.161:0/593377288 conn(0x7f64a80fb660 msgr2=0x7f64a80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:20:59.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 -- 172.21.15.161:0/593377288 shutdown_connections 2023-07-14T11:20:59.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.939+0000 7f64ad16f700 1 -- 172.21.15.161:0/593377288 wait complete. 2023-07-14T11:20:59.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.940+0000 7f64ad16f700 1 Processor -- start 2023-07-14T11:20:59.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.940+0000 7f64ad16f700 1 -- start start 2023-07-14T11:20:59.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64ad16f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 0x7f64a8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64ad16f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a81007a0 0x7f64a81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64ad16f700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64a8104ea0 0x7f64a819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64ad16f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a819cf40 con 0x7f64a81007a0 2023-07-14T11:20:59.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64ad16f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a819d0b0 con 0x7f64a80ffe00 2023-07-14T11:20:59.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64ad16f700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a819d270 con 0x7f64a8104ea0 2023-07-14T11:20:59.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64a6d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 0x7f64a8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64a6d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 0x7f64a8197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56230/0 (socket says 172.21.15.161:56230) 2023-07-14T11:20:59.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64a6d9d700 1 -- 172.21.15.161:0/176108967 learned_addr learned my addr 172.21.15.161:0/176108967 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:20:59.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64a659c700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a81007a0 0x7f64a81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.941+0000 7f64a759e700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64a8104ea0 0x7f64a819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64a6d9d700 1 -- 172.21.15.161:0/176108967 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64a8104ea0 msgr2=0x7f64a819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64a6d9d700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64a8104ea0 0x7f64a819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64a6d9d700 1 -- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a81007a0 msgr2=0x7f64a81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:20:59.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64a6d9d700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a81007a0 0x7f64a81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:20:59.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64a6d9d700 1 -- 172.21.15.161:0/176108967 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a0009d00 con 0x7f64a80ffe00 2023-07-14T11:20:59.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64a759e700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64a8104ea0 0x7f64a819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:20:59.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64a6d9d700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 0x7f64a8197f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f649800e9a0 tx=0x7f649800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:59.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f648ffff700 1 -- 172.21.15.161:0/176108967 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498009790 con 0x7f64a80ffe00 2023-07-14T11:20:59.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a819d550 con 0x7f64a80ffe00 2023-07-14T11:20:59.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f648ffff700 1 -- 172.21.15.161:0/176108967 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6498010430 con 0x7f64a80ffe00 2023-07-14T11:20:59.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f648ffff700 1 -- 172.21.15.161:0/176108967 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498018680 con 0x7f64a80ffe00 2023-07-14T11:20:59.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.942+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a81a4db0 con 0x7f64a80ffe00 2023-07-14T11:20:59.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.943+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a804e210 con 0x7f64a80ffe00 2023-07-14T11:20:59.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.946+0000 7f648ffff700 1 -- 172.21.15.161:0/176108967 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f64980105a0 con 0x7f64a80ffe00 2023-07-14T11:20:59.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.947+0000 7f648ffff700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64900a11e0 0x7f64900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:20:59.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.947+0000 7f648ffff700 1 -- 172.21.15.161:0/176108967 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6498014070 con 0x7f64a80ffe00 2023-07-14T11:20:59.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.947+0000 7f64a659c700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64900a11e0 0x7f64900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:20:59.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.948+0000 7f64a659c700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64900a11e0 0x7f64900a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64a81995a0 tx=0x7f649c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:20:59.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:20:59.949+0000 7f648ffff700 1 -- 172.21.15.161:0/176108967 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6498096220 con 0x7f64a80ffe00 2023-07-14T11:21:00.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:59 smithi175 ceph-mon[129081]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:00.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:59 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3765694707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T11:21:00.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:20:59 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2486140602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T11:21:00.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.183+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f64a81a5040 con 0x7f64a80ffe00 2023-07-14T11:21:00.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.183+0000 7f648ffff700 1 -- 172.21.15.161:0/176108967 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6498099840 con 0x7f64a80ffe00 2023-07-14T11:21:00.188 INFO:teuthology.orchestra.run.smithi161.stdout:339302416412 2023-07-14T11:21:00.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64900a11e0 msgr2=0x7f64900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:00.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64900a11e0 0x7f64900a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64a81995a0 tx=0x7f649c009260 comp rx=0 tx=0).stop 2023-07-14T11:21:00.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 msgr2=0x7f64a8197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:00.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 0x7f64a8197f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f649800e9a0 tx=0x7f649800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 shutdown_connections 2023-07-14T11:21:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64900a11e0 0x7f64900a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64a8104ea0 0x7f64a819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a80ffe00 0x7f64a8197f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 --2- 172.21.15.161:0/176108967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a81007a0 0x7f64a81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.186+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 >> 172.21.15.161:0/176108967 conn(0x7f64a80fb660 msgr2=0x7f64a80fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.187+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 shutdown_connections 2023-07-14T11:21:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.187+0000 7f64ad16f700 1 -- 172.21.15.161:0/176108967 wait complete. 2023-07-14T11:21:00.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:59 smithi182 ceph-mon[129366]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:00.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:59 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3765694707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T11:21:00.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:20:59 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2486140602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T11:21:00.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462960 got 201863462962 for osd.6 2023-07-14T11:21:00.622 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:00.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:00 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2707785923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T11:21:00.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:00 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/176108967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T11:21:00.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 -- 172.21.15.161:0/2082714951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c104ea0 msgr2=0x7f2f8c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:00.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 --2- 172.21.15.161:0/2082714951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c104ea0 0x7f2f8c10ca20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400b0d0 tx=0x7f2f8400b3e0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 -- 172.21.15.161:0/2082714951 shutdown_connections 2023-07-14T11:21:00.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 --2- 172.21.15.161:0/2082714951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c104ea0 0x7f2f8c10ca20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 --2- 172.21.15.161:0/2082714951 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2f8c0ffe00 0x7f2f8c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 --2- 172.21.15.161:0/2082714951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f8c1007a0 0x7f2f8c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:00.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 -- 172.21.15.161:0/2082714951 >> 172.21.15.161:0/2082714951 conn(0x7f2f8c0fb660 msgr2=0x7f2f8c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:00.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 -- 172.21.15.161:0/2082714951 shutdown_connections 2023-07-14T11:21:00.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.993+0000 7f2f9204b700 1 -- 172.21.15.161:0/2082714951 wait complete. 2023-07-14T11:21:01.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.994+0000 7f2f9204b700 1 Processor -- start 2023-07-14T11:21:01.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f9204b700 1 -- start start 2023-07-14T11:21:01.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f9204b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f8c0ffe00 0x7f2f8c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f9204b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c1007a0 0x7f2f8c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f9204b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2f8c104ea0 0x7f2f8c1a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f9204b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c073640 con 0x7f2f8c1007a0 2023-07-14T11:21:01.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f9204b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c1a4470 con 0x7f2f8c0ffe00 2023-07-14T11:21:01.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f9204b700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c1a45e0 con 0x7f2f8c104ea0 2023-07-14T11:21:01.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f8affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c1007a0 0x7f2f8c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f8affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c1007a0 0x7f2f8c072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59058/0 (socket says 172.21.15.161:59058) 2023-07-14T11:21:01.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f8affd700 1 -- 172.21.15.161:0/695681361 learned_addr learned my addr 172.21.15.161:0/695681361 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:01.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f8b7fe700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f8c0ffe00 0x7f2f8c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.995+0000 7f2f8bfff700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2f8c104ea0 0x7f2f8c1a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f8affd700 1 -- 172.21.15.161:0/695681361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2f8c104ea0 msgr2=0x7f2f8c1a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f8affd700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2f8c104ea0 0x7f2f8c1a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f8affd700 1 -- 172.21.15.161:0/695681361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f8c0ffe00 msgr2=0x7f2f8c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f8affd700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f8c0ffe00 0x7f2f8c072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f8affd700 1 -- 172.21.15.161:0/695681361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84009d00 con 0x7f2f8c1007a0 2023-07-14T11:21:01.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f8b7fe700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f8c0ffe00 0x7f2f8c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:01.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f8affd700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c1007a0 0x7f2f8c072d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000d980 tx=0x7f2f8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:01.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f88ff9700 1 -- 172.21.15.161:0/695681361 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000f890 con 0x7f2f8c1007a0 2023-07-14T11:21:01.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c1a4860 con 0x7f2f8c1007a0 2023-07-14T11:21:01.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f88ff9700 1 -- 172.21.15.161:0/695681361 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f80010460 con 0x7f2f8c1007a0 2023-07-14T11:21:01.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f88ff9700 1 -- 172.21.15.161:0/695681361 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000e610 con 0x7f2f8c1007a0 2023-07-14T11:21:01.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.996+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c1a4db0 con 0x7f2f8c1007a0 2023-07-14T11:21:01.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:00.998+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c04e210 con 0x7f2f8c1007a0 2023-07-14T11:21:01.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.000+0000 7f2f88ff9700 1 -- 172.21.15.161:0/695681361 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2f800105d0 con 0x7f2f8c1007a0 2023-07-14T11:21:01.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.001+0000 7f2f88ff9700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f740a1260 0x7f2f740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.002+0000 7f2f88ff9700 1 -- 172.21.15.161:0/695681361 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f800cbff0 con 0x7f2f8c1007a0 2023-07-14T11:21:01.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.002+0000 7f2f8b7fe700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f740a1260 0x7f2f740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.002+0000 7f2f8b7fe700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f740a1260 0x7f2f740a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c011490 tx=0x7f2f7c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:01.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.005+0000 7f2f88ff9700 1 -- 172.21.15.161:0/695681361 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2f800952d0 con 0x7f2f8c1007a0 2023-07-14T11:21:01.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:00 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2707785923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T11:21:01.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:00 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/176108967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T11:21:01.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:00 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2707785923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T11:21:01.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:00 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/176108967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T11:21:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.293+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f2f8c073d40 con 0x7f2f8c1007a0 2023-07-14T11:21:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.294+0000 7f2f88ff9700 1 -- 172.21.15.161:0/695681361 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f80020090 con 0x7f2f8c1007a0 2023-07-14T11:21:01.300 INFO:teuthology.orchestra.run.smithi161.stdout:287762808868 2023-07-14T11:21:01.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.301+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f740a1260 msgr2=0x7f2f740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.301+0000 7f2f9204b700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f740a1260 0x7f2f740a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c011490 tx=0x7f2f7c009260 comp rx=0 tx=0).stop 2023-07-14T11:21:01.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.301+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c1007a0 msgr2=0x7f2f8c072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.301+0000 7f2f9204b700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c1007a0 0x7f2f8c072d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000d980 tx=0x7f2f8000dc90 comp rx=0 tx=0).stop 2023-07-14T11:21:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.302+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 shutdown_connections 2023-07-14T11:21:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.302+0000 7f2f9204b700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2f740a1260 0x7f2f740a3710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.302+0000 7f2f9204b700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2f8c104ea0 0x7f2f8c1a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.302+0000 7f2f9204b700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f8c0ffe00 0x7f2f8c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.302+0000 7f2f9204b700 1 --2- 172.21.15.161:0/695681361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f8c1007a0 0x7f2f8c072d50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.302+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 >> 172.21.15.161:0/695681361 conn(0x7f2f8c0fb660 msgr2=0x7f2f8c0fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:01.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.303+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 shutdown_connections 2023-07-14T11:21:01.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.303+0000 7f2f9204b700 1 -- 172.21.15.161:0/695681361 wait complete. 2023-07-14T11:21:01.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 -- 172.21.15.161:0/2313419714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631c1007a0 msgr2=0x7f631c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 --2- 172.21.15.161:0/2313419714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631c1007a0 0x7f631c104770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f630c009990 tx=0x7f630c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 -- 172.21.15.161:0/2313419714 shutdown_connections 2023-07-14T11:21:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 --2- 172.21.15.161:0/2313419714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 0x7f631c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 --2- 172.21.15.161:0/2313419714 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f631c104ea0 0x7f631c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 --2- 172.21.15.161:0/2313419714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631c1007a0 0x7f631c104770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 -- 172.21.15.161:0/2313419714 >> 172.21.15.161:0/2313419714 conn(0x7f631c0fb660 msgr2=0x7f631c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:01.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 -- 172.21.15.161:0/2313419714 shutdown_connections 2023-07-14T11:21:01.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.371+0000 7f6323731700 1 -- 172.21.15.161:0/2313419714 wait complete. 2023-07-14T11:21:01.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.374+0000 7f6323731700 1 Processor -- start 2023-07-14T11:21:01.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.375+0000 7f6323731700 1 -- start start 2023-07-14T11:21:01.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.375+0000 7f6323731700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 0x7f631c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.375+0000 7f6323731700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631c1007a0 0x7f631c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.375+0000 7f6323731700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f631c104ea0 0x7f631c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.375+0000 7f6323731700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c19cf30 con 0x7f631c0ffe00 2023-07-14T11:21:01.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.375+0000 7f6323731700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c19d0a0 con 0x7f631c1007a0 2023-07-14T11:21:01.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.375+0000 7f6323731700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c19d260 con 0x7f631c104ea0 2023-07-14T11:21:01.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.379+0000 7f63214cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 0x7f631c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.379+0000 7f63214cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 0x7f631c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59068/0 (socket says 172.21.15.161:59068) 2023-07-14T11:21:01.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.379+0000 7f63214cd700 1 -- 172.21.15.161:0/1794608466 learned_addr learned my addr 172.21.15.161:0/1794608466 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:01.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.380+0000 7f63214cd700 1 -- 172.21.15.161:0/1794608466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f631c104ea0 msgr2=0x7f631c19c920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T11:21:01.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.380+0000 7f63214cd700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f631c104ea0 0x7f631c19c920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.380+0000 7f63214cd700 1 -- 172.21.15.161:0/1794608466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631c1007a0 msgr2=0x7f631c1984b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T11:21:01.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.380+0000 7f63214cd700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631c1007a0 0x7f631c1984b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.380+0000 7f63214cd700 1 -- 172.21.15.161:0/1794608466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c0095f0 con 0x7f631c0ffe00 2023-07-14T11:21:01.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.381+0000 7f63214cd700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 0x7f631c197f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f631800e9a0 tx=0x7f631800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:01.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.383+0000 7f63127fc700 1 -- 172.21.15.161:0/1794608466 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800cbd0 con 0x7f631c0ffe00 2023-07-14T11:21:01.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.384+0000 7f63127fc700 1 -- 172.21.15.161:0/1794608466 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f631800cd30 con 0x7f631c0ffe00 2023-07-14T11:21:01.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.384+0000 7f63127fc700 1 -- 172.21.15.161:0/1794608466 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318018710 con 0x7f631c0ffe00 2023-07-14T11:21:01.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.385+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c19d540 con 0x7f631c0ffe00 2023-07-14T11:21:01.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.385+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c06a100 con 0x7f631c0ffe00 2023-07-14T11:21:01.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.392+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c04e210 con 0x7f631c0ffe00 2023-07-14T11:21:01.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.394+0000 7f63127fc700 1 -- 172.21.15.161:0/1794608466 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6318010430 con 0x7f631c0ffe00 2023-07-14T11:21:01.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.397+0000 7f63127fc700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63080a13e0 0x7f63080a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.397+0000 7f63127fc700 1 -- 172.21.15.161:0/1794608466 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6318014070 con 0x7f631c0ffe00 2023-07-14T11:21:01.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.399+0000 7f6320ccc700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63080a13e0 0x7f63080a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.400+0000 7f6320ccc700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63080a13e0 0x7f63080a3890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f630c0098e0 tx=0x7f630c005bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:01.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.407+0000 7f63127fc700 1 -- 172.21.15.161:0/1794608466 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6318096520 con 0x7f631c0ffe00 2023-07-14T11:21:01.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 -- 172.21.15.161:0/59543825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0728b0 msgr2=0x7feb4c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 --2- 172.21.15.161:0/59543825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0728b0 0x7feb4c072c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009920 tx=0x7feb3c009c30 comp rx=0 tx=0).stop 2023-07-14T11:21:01.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 -- 172.21.15.161:0/59543825 shutdown_connections 2023-07-14T11:21:01.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 --2- 172.21.15.161:0/59543825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb4c110bb0 0x7feb4c112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 --2- 172.21.15.161:0/59543825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0728b0 0x7feb4c072c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 --2- 172.21.15.161:0/59543825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c0731c0 0x7feb4c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 -- 172.21.15.161:0/59543825 >> 172.21.15.161:0/59543825 conn(0x7feb4c0fb660 msgr2=0x7feb4c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:01.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 -- 172.21.15.161:0/59543825 shutdown_connections 2023-07-14T11:21:01.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.455+0000 7feb51ea8700 1 -- 172.21.15.161:0/59543825 wait complete. 2023-07-14T11:21:01.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.456+0000 7feb51ea8700 1 Processor -- start 2023-07-14T11:21:01.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.457+0000 7feb51ea8700 1 -- start start 2023-07-14T11:21:01.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.457+0000 7feb51ea8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb4c0728b0 0x7feb4c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.457+0000 7feb51ea8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0731c0 0x7feb4c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.457+0000 7feb51ea8700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c110bb0 0x7feb4c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.457+0000 7feb51ea8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1a1370 con 0x7feb4c0728b0 2023-07-14T11:21:01.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.457+0000 7feb51ea8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1a14e0 con 0x7feb4c0731c0 2023-07-14T11:21:01.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.457+0000 7feb51ea8700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1a16a0 con 0x7feb4c110bb0 2023-07-14T11:21:01.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c110bb0 0x7feb4c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c110bb0 0x7feb4c1a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:37042/0 (socket says 172.21.15.161:37042) 2023-07-14T11:21:01.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 -- 172.21.15.161:0/3259443068 learned_addr learned my addr 172.21.15.161:0/3259443068 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:01.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4b7fe700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb4c0728b0 0x7feb4c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb42ffd700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0731c0 0x7feb4c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 -- 172.21.15.161:0/3259443068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0731c0 msgr2=0x7feb4c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0731c0 0x7feb4c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 -- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb4c0728b0 msgr2=0x7feb4c19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb4c0728b0 0x7feb4c19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb4bfff700 1 -- 172.21.15.161:0/3259443068 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c009520 con 0x7feb4c110bb0 2023-07-14T11:21:01.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.458+0000 7feb42ffd700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0731c0 0x7feb4c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:01.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.459+0000 7feb4bfff700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c110bb0 0x7feb4c1a0d60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb4400ef90 tx=0x7feb4400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:01.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.459+0000 7feb497fa700 1 -- 172.21.15.161:0/3259443068 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44018070 con 0x7feb4c110bb0 2023-07-14T11:21:01.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.459+0000 7feb497fa700 1 -- 172.21.15.161:0/3259443068 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb4400f040 con 0x7feb4c110bb0 2023-07-14T11:21:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.459+0000 7feb497fa700 1 -- 172.21.15.161:0/3259443068 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44003e00 con 0x7feb4c110bb0 2023-07-14T11:21:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.459+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c1a1920 con 0x7feb4c110bb0 2023-07-14T11:21:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.459+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c1a91e0 con 0x7feb4c110bb0 2023-07-14T11:21:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.461+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c103090 con 0x7feb4c110bb0 2023-07-14T11:21:01.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.464+0000 7feb497fa700 1 -- 172.21.15.161:0/3259443068 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7feb4400ca50 con 0x7feb4c110bb0 2023-07-14T11:21:01.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.467+0000 7feb497fa700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7feb2c0a11e0 0x7feb2c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:01.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.467+0000 7feb497fa700 1 -- 172.21.15.161:0/3259443068 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb440cc9f0 con 0x7feb4c110bb0 2023-07-14T11:21:01.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.468+0000 7feb497fa700 1 -- 172.21.15.161:0/3259443068 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feb44098e20 con 0x7feb4c110bb0 2023-07-14T11:21:01.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.469+0000 7feb4b7fe700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7feb2c0a11e0 0x7feb2c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:01.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.473+0000 7feb4b7fe700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7feb2c0a11e0 0x7feb2c0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c010 tx=0x7feb3c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:01.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.656+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f631c06a3e0 con 0x7f631c0ffe00 2023-07-14T11:21:01.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.656+0000 7f63127fc700 1 -- 172.21.15.161:0/1794608466 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6318099b40 con 0x7f631c0ffe00 2023-07-14T11:21:01.661 INFO:teuthology.orchestra.run.smithi161.stdout:146028888124 2023-07-14T11:21:01.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63080a13e0 msgr2=0x7f63080a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63080a13e0 0x7f63080a3890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f630c0098e0 tx=0x7f630c005bc0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 msgr2=0x7f631c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 0x7f631c197f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f631800e9a0 tx=0x7f631800ed60 comp rx=0 tx=0).stop 2023-07-14T11:21:01.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 shutdown_connections 2023-07-14T11:21:01.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f631c0ffe00 0x7f631c197f70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63080a13e0 0x7f63080a3890 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f631c104ea0 0x7f631c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 --2- 172.21.15.161:0/1794608466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631c1007a0 0x7f631c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 >> 172.21.15.161:0/1794608466 conn(0x7f631c0fb660 msgr2=0x7f631c0fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:01.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 shutdown_connections 2023-07-14T11:21:01.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.660+0000 7f6323731700 1 -- 172.21.15.161:0/1794608466 wait complete. 2023-07-14T11:21:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.705+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7feb4c04e210 con 0x7feb4c110bb0 2023-07-14T11:21:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.706+0000 7feb497fa700 1 -- 172.21.15.161:0/3259443068 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7feb440990b0 con 0x7feb4c110bb0 2023-07-14T11:21:01.710 INFO:teuthology.orchestra.run.smithi161.stdout:309237645343 2023-07-14T11:21:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7feb2c0a11e0 msgr2=0x7feb2c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7feb2c0a11e0 0x7feb2c0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c010 tx=0x7feb3c019040 comp rx=0 tx=0).stop 2023-07-14T11:21:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c110bb0 msgr2=0x7feb4c1a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c110bb0 0x7feb4c1a0d60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb4400ef90 tx=0x7feb4400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 shutdown_connections 2023-07-14T11:21:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb4c0728b0 0x7feb4c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7feb2c0a11e0 0x7feb2c0a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feb4c110bb0 0x7feb4c1a0d60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 --2- 172.21.15.161:0/3259443068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb4c0731c0 0x7feb4c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 >> 172.21.15.161:0/3259443068 conn(0x7feb4c0fb660 msgr2=0x7feb4c0fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 shutdown_connections 2023-07-14T11:21:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.709+0000 7feb51ea8700 1 -- 172.21.15.161:0/3259443068 wait complete. 2023-07-14T11:21:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:01 smithi161 ceph-mon[95526]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/695681361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T11:21:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2199286439' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T11:21:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1794608466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T11:21:01.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3259443068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T11:21:02.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 -- 172.21.15.161:0/1788931895 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f38140731c0 msgr2=0x7f3814110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 --2- 172.21.15.161:0/1788931895 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f38140731c0 0x7f3814110670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0099c0 tx=0x7f37fc009cd0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 -- 172.21.15.161:0/1788931895 shutdown_connections 2023-07-14T11:21:02.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 --2- 172.21.15.161:0/1788931895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 0x7f3814072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 --2- 172.21.15.161:0/1788931895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814110bb0 0x7f3814112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 --2- 172.21.15.161:0/1788931895 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f38140731c0 0x7f3814110670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 -- 172.21.15.161:0/1788931895 >> 172.21.15.161:0/1788931895 conn(0x7f38140fb660 msgr2=0x7f38140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 -- 172.21.15.161:0/1788931895 shutdown_connections 2023-07-14T11:21:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.787+0000 7f3818c73700 1 -- 172.21.15.161:0/1788931895 wait complete. 2023-07-14T11:21:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.788+0000 7f3818c73700 1 Processor -- start 2023-07-14T11:21:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.789+0000 7f3818c73700 1 -- start start 2023-07-14T11:21:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.789+0000 7f3818c73700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 0x7f381419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.789+0000 7f3818c73700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140731c0 0x7f381419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.789+0000 7f3818c73700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3814110bb0 0x7f38141a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.789+0000 7f3818c73700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a1310 con 0x7f38140728b0 2023-07-14T11:21:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3818c73700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a1480 con 0x7f38140731c0 2023-07-14T11:21:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3818c73700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a1640 con 0x7f3814110bb0 2023-07-14T11:21:02.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f381259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 0x7f381419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:02.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3809d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140731c0 0x7f381419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:02.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f381259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 0x7f381419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59104/0 (socket says 172.21.15.161:59104) 2023-07-14T11:21:02.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f381259c700 1 -- 172.21.15.161:0/848762230 learned_addr learned my addr 172.21.15.161:0/848762230 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:02.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3812d9d700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3814110bb0 0x7f38141a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:02.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3809d9b700 1 -- 172.21.15.161:0/848762230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3814110bb0 msgr2=0x7f38141a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3809d9b700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3814110bb0 0x7f38141a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3809d9b700 1 -- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 msgr2=0x7f381419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3809d9b700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 0x7f381419c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f3809d9b700 1 -- 172.21.15.161:0/848762230 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0095f0 con 0x7f38140731c0 2023-07-14T11:21:02.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.790+0000 7f381259c700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 0x7f381419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:02.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.791+0000 7f3809d9b700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140731c0 0x7f381419c890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0098e0 tx=0x7f37fc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:02.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.791+0000 7f380bfff700 1 -- 172.21.15.161:0/848762230 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc01d070 con 0x7f38140731c0 2023-07-14T11:21:02.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.791+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38141a18c0 con 0x7f38140731c0 2023-07-14T11:21:02.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.791+0000 7f380bfff700 1 -- 172.21.15.161:0/848762230 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37fc005510 con 0x7f38140731c0 2023-07-14T11:21:02.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.791+0000 7f380bfff700 1 -- 172.21.15.161:0/848762230 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc004b30 con 0x7f38140731c0 2023-07-14T11:21:02.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.791+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381406a100 con 0x7f38140731c0 2023-07-14T11:21:02.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.793+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814103090 con 0x7f38140731c0 2023-07-14T11:21:02.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.794+0000 7f380bfff700 1 -- 172.21.15.161:0/848762230 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f37fc00fb80 con 0x7f38140731c0 2023-07-14T11:21:02.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.796+0000 7f380bfff700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f38000a11e0 0x7f38000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.796+0000 7f380bfff700 1 -- 172.21.15.161:0/848762230 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f37fc0cdb80 con 0x7f38140731c0 2023-07-14T11:21:02.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.797+0000 7f381259c700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f38000a11e0 0x7f38000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:02.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.798+0000 7f381259c700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f38000a11e0 0x7f38000a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f380400c530 tx=0x7f380400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:02.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.800+0000 7f380bfff700 1 -- 172.21.15.161:0/848762230 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f37fc096e90 con 0x7f38140731c0 2023-07-14T11:21:02.062 INFO:teuthology.orchestra.run.smithi161.stdout:77309411397 2023-07-14T11:21:02.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.015+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3814102a70 con 0x7f38140731c0 2023-07-14T11:21:02.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.015+0000 7f380bfff700 1 -- 172.21.15.161:0/848762230 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37fc026090 con 0x7f38140731c0 2023-07-14T11:21:02.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f38000a11e0 msgr2=0x7f38000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f38000a11e0 0x7f38000a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f380400c530 tx=0x7f380400b040 comp rx=0 tx=0).stop 2023-07-14T11:21:02.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140731c0 msgr2=0x7f381419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140731c0 0x7f381419c890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0098e0 tx=0x7f37fc00f690 comp rx=0 tx=0).stop 2023-07-14T11:21:02.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 shutdown_connections 2023-07-14T11:21:02.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38140728b0 0x7f381419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f38000a11e0 0x7f38000a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3814110bb0 0x7f38141a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 --2- 172.21.15.161:0/848762230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38140731c0 0x7f381419c890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 >> 172.21.15.161:0/848762230 conn(0x7f38140fb660 msgr2=0x7f38140fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:02.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 shutdown_connections 2023-07-14T11:21:02.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:02.018+0000 7f3818c73700 1 -- 172.21.15.161:0/848762230 wait complete. 2023-07-14T11:21:02.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:01 smithi175 ceph-mon[129081]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:02.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/695681361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T11:21:02.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2199286439' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T11:21:02.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1794608466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T11:21:02.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3259443068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T11:21:02.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:01 smithi182 ceph-mon[129366]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:02.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/695681361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T11:21:02.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2199286439' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T11:21:02.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1794608466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T11:21:02.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3259443068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T11:21:02.435 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808868 for osd.9 2023-07-14T11:21:02.435 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:02.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 -- 172.21.15.161:0/1924806979 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff520100df0 msgr2=0x7ff5201011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 --2- 172.21.15.161:0/1924806979 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff520100df0 0x7ff5201011c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff510009990 tx=0x7ff510009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 -- 172.21.15.161:0/1924806979 shutdown_connections 2023-07-14T11:21:02.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 --2- 172.21.15.161:0/1924806979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff52010a140 0x7ff52010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 --2- 172.21.15.161:0/1924806979 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff520100df0 0x7ff5201011c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 --2- 172.21.15.161:0/1924806979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520101700 0x7ff520109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 -- 172.21.15.161:0/1924806979 >> 172.21.15.161:0/1924806979 conn(0x7ff520075b70 msgr2=0x7ff520075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:02.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 -- 172.21.15.161:0/1924806979 shutdown_connections 2023-07-14T11:21:02.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.231+0000 7ff525d43700 1 -- 172.21.15.161:0/1924806979 wait complete. 2023-07-14T11:21:02.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.233+0000 7ff525d43700 1 Processor -- start 2023-07-14T11:21:02.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.233+0000 7ff525d43700 1 -- start start 2023-07-14T11:21:02.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff525d43700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff520100df0 0x7ff5200fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff525d43700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff520101700 0x7ff5200fef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff525d43700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52010a140 0x7ff5200ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff525d43700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201a4190 con 0x7ff52010a140 2023-07-14T11:21:02.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff525d43700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201a4300 con 0x7ff520100df0 2023-07-14T11:21:02.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff525d43700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201a44c0 con 0x7ff520101700 2023-07-14T11:21:02.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff51f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff520100df0 0x7ff5200fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:02.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff51f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff520100df0 0x7ff5200fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56252/0 (socket says 172.21.15.161:56252) 2023-07-14T11:21:02.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.234+0000 7ff51f7fe700 1 -- 172.21.15.161:0/2199286439 learned_addr learned my addr 172.21.15.161:0/2199286439 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:02.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51f7fe700 1 -- 172.21.15.161:0/2199286439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff520101700 msgr2=0x7ff5200fef90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:21:02.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51f7fe700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff520101700 0x7ff5200fef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51f7fe700 1 -- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52010a140 msgr2=0x7ff5200ff510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:21:02.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51f7fe700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52010a140 0x7ff5200ff510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51f7fe700 1 -- 172.21.15.161:0/2199286439 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100095f0 con 0x7ff520100df0 2023-07-14T11:21:02.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51f7fe700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff520100df0 0x7ff5200fea50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff510006010 tx=0x7ff510003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:02.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51cff9700 1 -- 172.21.15.161:0/2199286439 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51001d070 con 0x7ff520100df0 2023-07-14T11:21:02.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5201a4740 con 0x7ff520100df0 2023-07-14T11:21:02.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51cff9700 1 -- 172.21.15.161:0/2199286439 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff510003e60 con 0x7ff520100df0 2023-07-14T11:21:02.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff51cff9700 1 -- 172.21.15.161:0/2199286439 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100173f0 con 0x7ff520100df0 2023-07-14T11:21:02.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.235+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5201a4c30 con 0x7ff520100df0 2023-07-14T11:21:02.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.236+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520104a30 con 0x7ff520100df0 2023-07-14T11:21:02.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.241+0000 7ff51cff9700 1 -- 172.21.15.161:0/2199286439 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff51000f560 con 0x7ff520100df0 2023-07-14T11:21:02.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.243+0000 7ff51cff9700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5080a1260 0x7ff5080a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:02.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.243+0000 7ff51cff9700 1 -- 172.21.15.161:0/2199286439 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff5100ce5a0 con 0x7ff520100df0 2023-07-14T11:21:02.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.243+0000 7ff51effd700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5080a1260 0x7ff5080a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:02.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.243+0000 7ff51cff9700 1 -- 172.21.15.161:0/2199286439 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff5100ce9e0 con 0x7ff520100df0 2023-07-14T11:21:02.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.244+0000 7ff51effd700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5080a1260 0x7ff5080a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff520100240 tx=0x7ff514006c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:02.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.549+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff52004e210 con 0x7ff520100df0 2023-07-14T11:21:02.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.550+0000 7ff51cff9700 1 -- 172.21.15.161:0/2199286439 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff52004e210 con 0x7ff520100df0 2023-07-14T11:21:02.571 INFO:teuthology.orchestra.run.smithi161.stdout:223338299437 2023-07-14T11:21:02.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5080a1260 msgr2=0x7ff5080a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5080a1260 0x7ff5080a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff520100240 tx=0x7ff514006c60 comp rx=0 tx=0).stop 2023-07-14T11:21:02.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff520100df0 msgr2=0x7ff5200fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:02.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff520100df0 0x7ff5200fea50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff510006010 tx=0x7ff510003b80 comp rx=0 tx=0).stop 2023-07-14T11:21:02.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 shutdown_connections 2023-07-14T11:21:02.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52010a140 0x7ff5200ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5080a1260 0x7ff5080a3710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff520100df0 0x7ff5200fea50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 --2- 172.21.15.161:0/2199286439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff520101700 0x7ff5200fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:02.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 >> 172.21.15.161:0/2199286439 conn(0x7ff520075b70 msgr2=0x7ff520101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:02.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 shutdown_connections 2023-07-14T11:21:02.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:01.554+0000 7ff525d43700 1 -- 172.21.15.161:0/2199286439 wait complete. 2023-07-14T11:21:02.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:02 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/848762230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T11:21:02.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:02 smithi161 ceph-mon[95526]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2023-07-14T11:21:03.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:02 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/848762230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T11:21:03.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:02 smithi175 ceph-mon[129081]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2023-07-14T11:21:03.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:02 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/848762230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T11:21:03.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:02 smithi182 ceph-mon[129366]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2023-07-14T11:21:03.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299437 for osd.7 2023-07-14T11:21:03.540 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:03.908 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149759 for osd.3 2023-07-14T11:21:03.908 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:04.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416411 got 339302416412 for osd.11 2023-07-14T11:21:04.242 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:04.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411397 for osd.2 2023-07-14T11:21:04.362 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:04.416 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070502 got 253403070504 for osd.8 2023-07-14T11:21:04.417 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:04.477 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645341 got 309237645343 for osd.10 2023-07-14T11:21:04.478 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:04.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.601+0000 7f90579f3700 1 -- 172.21.15.161:0/3012997343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90500731c0 msgr2=0x7f9050110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.601+0000 7f90579f3700 1 --2- 172.21.15.161:0/3012997343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90500731c0 0x7f9050110670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f904c0098f0 tx=0x7f904c009c00 comp rx=0 tx=0).stop 2023-07-14T11:21:04.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.602+0000 7f90579f3700 1 -- 172.21.15.161:0/3012997343 shutdown_connections 2023-07-14T11:21:04.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.602+0000 7f90579f3700 1 --2- 172.21.15.161:0/3012997343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9050110bb0 0x7f9050112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.602+0000 7f90579f3700 1 --2- 172.21.15.161:0/3012997343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 0x7f9050072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.602+0000 7f90579f3700 1 --2- 172.21.15.161:0/3012997343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90500731c0 0x7f9050110670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.602+0000 7f90579f3700 1 -- 172.21.15.161:0/3012997343 >> 172.21.15.161:0/3012997343 conn(0x7f90500fb660 msgr2=0x7f90500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:04.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.603+0000 7f90579f3700 1 -- 172.21.15.161:0/3012997343 shutdown_connections 2023-07-14T11:21:04.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.603+0000 7f90579f3700 1 -- 172.21.15.161:0/3012997343 wait complete. 2023-07-14T11:21:04.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.604+0000 7f90579f3700 1 Processor -- start 2023-07-14T11:21:04.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.604+0000 7f90579f3700 1 -- start start 2023-07-14T11:21:04.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.605+0000 7f90579f3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 0x7f905019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.605+0000 7f90579f3700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90500731c0 0x7f905019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.605+0000 7f90579f3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9050110bb0 0x7f90501a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.605+0000 7f90579f3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90501a1320 con 0x7f9050110bb0 2023-07-14T11:21:04.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f90579f3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90501a1490 con 0x7f90500728b0 2023-07-14T11:21:04.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f90579f3700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90501a1650 con 0x7f90500731c0 2023-07-14T11:21:04.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 0x7f905019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f9054f8e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90500731c0 0x7f905019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 0x7f905019c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56326/0 (socket says 172.21.15.161:56326) 2023-07-14T11:21:04.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 -- 172.21.15.161:0/647626811 learned_addr learned my addr 172.21.15.161:0/647626811 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:04.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f9055f90700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9050110bb0 0x7f90501a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888124 for osd.4 2023-07-14T11:21:04.613 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:04.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 -- 172.21.15.161:0/647626811 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90500731c0 msgr2=0x7f905019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90500731c0 0x7f905019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 -- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9050110bb0 msgr2=0x7f90501a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9050110bb0 0x7f90501a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.606+0000 7f905578f700 1 -- 172.21.15.161:0/647626811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c009520 con 0x7f90500728b0 2023-07-14T11:21:04.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.607+0000 7f905578f700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 0x7f905019c360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f904000b550 tx=0x7f904000b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:04.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.607+0000 7f90467fc700 1 -- 172.21.15.161:0/647626811 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040011830 con 0x7f90500728b0 2023-07-14T11:21:04.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.607+0000 7f90467fc700 1 -- 172.21.15.161:0/647626811 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9040011e50 con 0x7f90500728b0 2023-07-14T11:21:04.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.607+0000 7f90467fc700 1 -- 172.21.15.161:0/647626811 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000f2e0 con 0x7f90500728b0 2023-07-14T11:21:04.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.607+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90501a1930 con 0x7f90500728b0 2023-07-14T11:21:04.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.607+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90501a9190 con 0x7f90500728b0 2023-07-14T11:21:04.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.608+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9050103090 con 0x7f90500728b0 2023-07-14T11:21:04.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.611+0000 7f90467fc700 1 -- 172.21.15.161:0/647626811 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9040011990 con 0x7f90500728b0 2023-07-14T11:21:04.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.612+0000 7f90467fc700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f903c0a1260 0x7f903c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.612+0000 7f90467fc700 1 -- 172.21.15.161:0/647626811 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f90400cccd0 con 0x7f90500728b0 2023-07-14T11:21:04.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.612+0000 7f9054f8e700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f903c0a1260 0x7f903c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.613+0000 7f9054f8e700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f903c0a1260 0x7f903c0a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f904c009fd0 tx=0x7f904c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:04.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.614+0000 7f90467fc700 1 -- 172.21.15.161:0/647626811 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9040096060 con 0x7f90500728b0 2023-07-14T11:21:04.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.849+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f905004e210 con 0x7f90500728b0 2023-07-14T11:21:04.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.850+0000 7f90467fc700 1 -- 172.21.15.161:0/647626811 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9040015090 con 0x7f90500728b0 2023-07-14T11:21:04.854 INFO:teuthology.orchestra.run.smithi161.stdout:167503724600 2023-07-14T11:21:04.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.852+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f903c0a1260 msgr2=0x7f903c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.852+0000 7f90579f3700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f903c0a1260 0x7f903c0a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f904c009fd0 tx=0x7f904c019040 comp rx=0 tx=0).stop 2023-07-14T11:21:04.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.852+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 msgr2=0x7f905019c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.852+0000 7f90579f3700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 0x7f905019c360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f904000b550 tx=0x7f904000b860 comp rx=0 tx=0).stop 2023-07-14T11:21:04.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 shutdown_connections 2023-07-14T11:21:04.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9050110bb0 0x7f90501a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f903c0a1260 0x7f903c0a3710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90500728b0 0x7f905019c360 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 --2- 172.21.15.161:0/647626811 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90500731c0 0x7f905019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 >> 172.21.15.161:0/647626811 conn(0x7f90500fb660 msgr2=0x7f90500fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:04.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 shutdown_connections 2023-07-14T11:21:04.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.853+0000 7f90579f3700 1 -- 172.21.15.161:0/647626811 wait complete. 2023-07-14T11:21:04.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.955+0000 7fa78df3f700 1 -- 172.21.15.161:0/1905964305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa788104ea0 msgr2=0x7fa78810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.955+0000 7fa78df3f700 1 --2- 172.21.15.161:0/1905964305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa788104ea0 0x7fa78810ca20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa78000b0d0 tx=0x7fa78000b3e0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.956+0000 7fa78df3f700 1 -- 172.21.15.161:0/1905964305 shutdown_connections 2023-07-14T11:21:04.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.956+0000 7fa78df3f700 1 --2- 172.21.15.161:0/1905964305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7880ffe00 0x7fa7881001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.956+0000 7fa78df3f700 1 --2- 172.21.15.161:0/1905964305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa788104ea0 0x7fa78810ca20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.956+0000 7fa78df3f700 1 --2- 172.21.15.161:0/1905964305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7881007a0 0x7fa788104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.956+0000 7fa78df3f700 1 -- 172.21.15.161:0/1905964305 >> 172.21.15.161:0/1905964305 conn(0x7fa7880fb660 msgr2=0x7fa7880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:04.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.956+0000 7fa78df3f700 1 -- 172.21.15.161:0/1905964305 shutdown_connections 2023-07-14T11:21:04.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.956+0000 7fa78df3f700 1 -- 172.21.15.161:0/1905964305 wait complete. 2023-07-14T11:21:04.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 Processor -- start 2023-07-14T11:21:04.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 -- start start 2023-07-14T11:21:04.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7880ffe00 0x7fa788197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7881007a0 0x7fa7881984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa788104ea0 0x7fa78819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78819cf30 con 0x7fa788104ea0 2023-07-14T11:21:04.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78819d0a0 con 0x7fa7881007a0 2023-07-14T11:21:04.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.957+0000 7fa78df3f700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78819d260 con 0x7fa7880ffe00 2023-07-14T11:21:04.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7880ffe00 0x7fa788197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7880ffe00 0x7fa788197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:37096/0 (socket says 172.21.15.161:37096) 2023-07-14T11:21:04.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa787fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa788104ea0 0x7fa78819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 -- 172.21.15.161:0/3087389457 learned_addr learned my addr 172.21.15.161:0/3087389457 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:04.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa786ffd700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7881007a0 0x7fa7881984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 -- 172.21.15.161:0/3087389457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7881007a0 msgr2=0x7fa7881984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7881007a0 0x7fa7881984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 -- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa788104ea0 msgr2=0x7fa78819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:04.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa788104ea0 0x7fa78819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:04.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa7877fe700 1 -- 172.21.15.161:0/3087389457 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780009d00 con 0x7fa7880ffe00 2023-07-14T11:21:04.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.958+0000 7fa787fff700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa788104ea0 0x7fa78819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:04.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.959+0000 7fa7877fe700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7880ffe00 0x7fa788197f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e9a0 tx=0x7fa77800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:04.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.959+0000 7fa784ff9700 1 -- 172.21.15.161:0/3087389457 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778009790 con 0x7fa7880ffe00 2023-07-14T11:21:04.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.959+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78819d540 con 0x7fa7880ffe00 2023-07-14T11:21:04.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.959+0000 7fa784ff9700 1 -- 172.21.15.161:0/3087389457 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa778010430 con 0x7fa7880ffe00 2023-07-14T11:21:04.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.959+0000 7fa784ff9700 1 -- 172.21.15.161:0/3087389457 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778018680 con 0x7fa7880ffe00 2023-07-14T11:21:04.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.959+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7881a4da0 con 0x7fa7880ffe00 2023-07-14T11:21:04.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.960+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78804e210 con 0x7fa7880ffe00 2023-07-14T11:21:04.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.964+0000 7fa784ff9700 1 -- 172.21.15.161:0/3087389457 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa7780105a0 con 0x7fa7880ffe00 2023-07-14T11:21:04.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.965+0000 7fa784ff9700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7700a11e0 0x7fa7700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:04.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.966+0000 7fa784ff9700 1 -- 172.21.15.161:0/3087389457 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa778014070 con 0x7fa7880ffe00 2023-07-14T11:21:04.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.966+0000 7fa786ffd700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7700a11e0 0x7fa7700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:04.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.966+0000 7fa784ff9700 1 -- 172.21.15.161:0/3087389457 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa778096220 con 0x7fa7880ffe00 2023-07-14T11:21:04.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:04.967+0000 7fa786ffd700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7700a11e0 0x7fa7700a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa788199590 tx=0x7fa77c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:05.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.216+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa7881992d0 con 0x7fa7880ffe00 2023-07-14T11:21:05.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.216+0000 7fa784ff9700 1 -- 172.21.15.161:0/3087389457 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa778099840 con 0x7fa7880ffe00 2023-07-14T11:21:05.220 INFO:teuthology.orchestra.run.smithi161.stdout:55834574920 2023-07-14T11:21:05.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7700a11e0 msgr2=0x7fa7700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:05.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7700a11e0 0x7fa7700a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa788199590 tx=0x7fa77c009260 comp rx=0 tx=0).stop 2023-07-14T11:21:05.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7880ffe00 msgr2=0x7fa788197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:05.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7880ffe00 0x7fa788197f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e9a0 tx=0x7fa77800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 shutdown_connections 2023-07-14T11:21:05.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa788104ea0 0x7fa78819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa7700a11e0 0x7fa7700a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7880ffe00 0x7fa788197f70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 --2- 172.21.15.161:0/3087389457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7881007a0 0x7fa7881984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 >> 172.21.15.161:0/3087389457 conn(0x7fa7880fb660 msgr2=0x7fa7880fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:05.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 shutdown_connections 2023-07-14T11:21:05.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.219+0000 7fa78df3f700 1 -- 172.21.15.161:0/3087389457 wait complete. 2023-07-14T11:21:05.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2382222297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f80ffe00 msgr2=0x7f00f81001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:05.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2382222297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f80ffe00 0x7f00f81001d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0099c0 tx=0x7f00ec009cd0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2382222297 shutdown_connections 2023-07-14T11:21:05.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2382222297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f8104ea0 0x7f00f810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2382222297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f80ffe00 0x7f00f81001d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2382222297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f81007a0 0x7f00f8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2382222297 >> 172.21.15.161:0/2382222297 conn(0x7f00f80fb660 msgr2=0x7f00f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:05.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2382222297 shutdown_connections 2023-07-14T11:21:05.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.336+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2382222297 wait complete. 2023-07-14T11:21:05.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 Processor -- start 2023-07-14T11:21:05.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 -- start start 2023-07-14T11:21:05.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f80ffe00 0x7f00f819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:05.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f81007a0 0x7f00f819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:05.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f8104ea0 0x7f00f8195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:05.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8196150 con 0x7f00f81007a0 2023-07-14T11:21:05.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81962c0 con 0x7f00f80ffe00 2023-07-14T11:21:05.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.338+0000 7f00fe3ce700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8196480 con 0x7f00f8104ea0 2023-07-14T11:21:05.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f81007a0 0x7f00f819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:05.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00fc96b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f8104ea0 0x7f00f8195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:05.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f81007a0 0x7f00f819bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59158/0 (socket says 172.21.15.161:59158) 2023-07-14T11:21:05.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 -- 172.21.15.161:0/2095170704 learned_addr learned my addr 172.21.15.161:0/2095170704 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:05.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f7fff700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f80ffe00 0x7f00f819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:05.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 -- 172.21.15.161:0/2095170704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f8104ea0 msgr2=0x7f00f8195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:05.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f8104ea0 0x7f00f8195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 -- 172.21.15.161:0/2095170704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f80ffe00 msgr2=0x7f00f819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:05.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f80ffe00 0x7f00f819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 -- 172.21.15.161:0/2095170704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec0095f0 con 0x7f00f81007a0 2023-07-14T11:21:05.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00fc96b700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f8104ea0 0x7f00f8195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:05.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.339+0000 7f00f77fe700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f81007a0 0x7f00f819bfa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f00f0007f10 tx=0x7f00f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:05.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.340+0000 7f00f57fa700 1 -- 172.21.15.161:0/2095170704 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000dd20 con 0x7f00f81007a0 2023-07-14T11:21:05.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.340+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f8196760 con 0x7f00f81007a0 2023-07-14T11:21:05.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.340+0000 7f00f57fa700 1 -- 172.21.15.161:0/2095170704 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f000f040 con 0x7f00f81007a0 2023-07-14T11:21:05.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.340+0000 7f00f57fa700 1 -- 172.21.15.161:0/2095170704 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00125b0 con 0x7f00f81007a0 2023-07-14T11:21:05.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.340+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f8072910 con 0x7f00f81007a0 2023-07-14T11:21:05.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.341+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f804e210 con 0x7f00f81007a0 2023-07-14T11:21:05.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.348+0000 7f00f57fa700 1 -- 172.21.15.161:0/2095170704 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f00f00133f0 con 0x7f00f81007a0 2023-07-14T11:21:05.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.350+0000 7f00f57fa700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00e00a11e0 0x7f00e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:05.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.350+0000 7f00f57fa700 1 -- 172.21.15.161:0/2095170704 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f00f00ccfc0 con 0x7f00f81007a0 2023-07-14T11:21:05.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.350+0000 7f00f7fff700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00e00a11e0 0x7f00e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:05.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.350+0000 7f00f57fa700 1 -- 172.21.15.161:0/2095170704 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f00f00cd400 con 0x7f00f81007a0 2023-07-14T11:21:05.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.351+0000 7f00f7fff700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00e00a11e0 0x7f00e00a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006010 tx=0x7f00ec009f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:05.358 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:05 smithi161 ceph-mon[95526]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T11:21:05.359 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:05.359 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:05 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:05.359 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:05 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/647626811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T11:21:05.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:05 smithi182 ceph-mon[129366]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T11:21:05.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:05.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:05 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:05.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:05 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/647626811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T11:21:05.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.582+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f00f8072bf0 con 0x7f00f81007a0 2023-07-14T11:21:05.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.583+0000 7f00f57fa700 1 -- 172.21.15.161:0/2095170704 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00f0096240 con 0x7f00f81007a0 2023-07-14T11:21:05.587 INFO:teuthology.orchestra.run.smithi161.stdout:34359738447 2023-07-14T11:21:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00e00a11e0 msgr2=0x7f00e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00e00a11e0 0x7f00e00a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006010 tx=0x7f00ec009f10 comp rx=0 tx=0).stop 2023-07-14T11:21:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f81007a0 msgr2=0x7f00f819bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f81007a0 0x7f00f819bfa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f00f0007f10 tx=0x7f00f000d3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 shutdown_connections 2023-07-14T11:21:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00e00a11e0 0x7f00e00a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00f8104ea0 0x7f00f8195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f80ffe00 0x7f00f819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 --2- 172.21.15.161:0/2095170704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f81007a0 0x7f00f819bfa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:05.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 >> 172.21.15.161:0/2095170704 conn(0x7f00f80fb660 msgr2=0x7f00f8103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:05.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 shutdown_connections 2023-07-14T11:21:05.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:05.587+0000 7f00fe3ce700 1 -- 172.21.15.161:0/2095170704 wait complete. 2023-07-14T11:21:05.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:05 smithi175 ceph-mon[129081]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T11:21:05.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:05.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:05 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:05.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:05 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/647626811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T11:21:05.719 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574920 for osd.1 2023-07-14T11:21:05.719 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:06.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724597 got 167503724600 for osd.5 2023-07-14T11:21:06.235 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:06.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738447 for osd.0 2023-07-14T11:21:06.250 DEBUG:teuthology.parallel:result is None 2023-07-14T11:21:06.250 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-14T11:21:06.251 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph pg dump --format=json 2023-07-14T11:21:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:06 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3087389457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T11:21:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:06 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2095170704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T11:21:06.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:21:06.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:21:06.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:06 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3087389457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T11:21:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:06 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2095170704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T11:21:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:21:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:21:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:06.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:06 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3087389457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T11:21:06.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:06 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2095170704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T11:21:06.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:21:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:21:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' 2023-07-14T11:21:07.015 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:07.871 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:07 smithi161 ceph-mon[95526]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:08.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:07 smithi175 ceph-mon[129081]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:08.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:07 smithi182 ceph-mon[129366]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:08.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 -- 172.21.15.161:0/4075589411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 msgr2=0x7ff57c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:08.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 --2- 172.21.15.161:0/4075589411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 0x7ff57c10ca20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff57400b0d0 tx=0x7ff57400b3e0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 -- 172.21.15.161:0/4075589411 shutdown_connections 2023-07-14T11:21:08.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 --2- 172.21.15.161:0/4075589411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 0x7ff57c10ca20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 --2- 172.21.15.161:0/4075589411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c0ffe00 0x7ff57c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 --2- 172.21.15.161:0/4075589411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c1007a0 0x7ff57c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 -- 172.21.15.161:0/4075589411 >> 172.21.15.161:0/4075589411 conn(0x7ff57c0fb660 msgr2=0x7ff57c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:08.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 -- 172.21.15.161:0/4075589411 shutdown_connections 2023-07-14T11:21:08.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.532+0000 7ff581d16700 1 -- 172.21.15.161:0/4075589411 wait complete. 2023-07-14T11:21:08.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 Processor -- start 2023-07-14T11:21:08.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 -- start start 2023-07-14T11:21:08.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c0ffe00 0x7ff57c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:08.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c1007a0 0x7ff57c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:08.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 0x7ff57c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:08.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1a41d0 con 0x7ff57c104ea0 2023-07-14T11:21:08.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1a4340 con 0x7ff57c1007a0 2023-07-14T11:21:08.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.533+0000 7ff581d16700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1a4500 con 0x7ff57c0ffe00 2023-07-14T11:21:08.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c0ffe00 0x7ff57c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:08.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c0ffe00 0x7ff57c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45698/0 (socket says 172.21.15.161:45698) 2023-07-14T11:21:08.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 -- 172.21.15.161:0/3697241323 learned_addr learned my addr 172.21.15.161:0/3697241323 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:08.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 0x7ff57c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:08.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57affd700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c1007a0 0x7ff57c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:08.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 -- 172.21.15.161:0/3697241323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c1007a0 msgr2=0x7ff57c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:08.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c1007a0 0x7ff57c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 -- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 msgr2=0x7ff57c0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 0x7ff57c0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 -- 172.21.15.161:0/3697241323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574009d00 con 0x7ff57c0ffe00 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57bfff700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 0x7ff57c0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.534+0000 7ff57b7fe700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c0ffe00 0x7ff57c072810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00d980 tx=0x7ff56c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.535+0000 7ff578ff9700 1 -- 172.21.15.161:0/3697241323 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c004220 con 0x7ff57c0ffe00 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.535+0000 7ff578ff9700 1 -- 172.21.15.161:0/3697241323 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff56c007640 con 0x7ff57c0ffe00 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.535+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c1a47e0 con 0x7ff57c0ffe00 2023-07-14T11:21:08.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.535+0000 7ff578ff9700 1 -- 172.21.15.161:0/3697241323 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c005690 con 0x7ff57c0ffe00 2023-07-14T11:21:08.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.535+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c1a4ce0 con 0x7ff57c0ffe00 2023-07-14T11:21:08.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.536+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c04e210 con 0x7ff57c0ffe00 2023-07-14T11:21:08.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.538+0000 7ff578ff9700 1 -- 172.21.15.161:0/3697241323 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff56c004750 con 0x7ff57c0ffe00 2023-07-14T11:21:08.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.539+0000 7ff578ff9700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5640a11e0 0x7ff5640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:08.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.540+0000 7ff578ff9700 1 -- 172.21.15.161:0/3697241323 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff56c0cd0d0 con 0x7ff57c0ffe00 2023-07-14T11:21:08.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.540+0000 7ff57affd700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5640a11e0 0x7ff5640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:08.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.540+0000 7ff57affd700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5640a11e0 0x7ff5640a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074000 tx=0x7ff570009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:08.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.541+0000 7ff578ff9700 1 -- 172.21.15.161:0/3697241323 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff56c096350 con 0x7ff57c0ffe00 2023-07-14T11:21:08.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.733+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff57c0fc790 con 0x7ff5640a11e0 2023-07-14T11:21:08.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.747+0000 7ff578ff9700 1 -- 172.21.15.161:0/3697241323 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427221 (secure 0 0 0) 0x7ff57c0fc790 con 0x7ff5640a11e0 2023-07-14T11:21:08.756 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:08.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5640a11e0 msgr2=0x7ff5640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:08.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5640a11e0 0x7ff5640a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074000 tx=0x7ff570009260 comp rx=0 tx=0).stop 2023-07-14T11:21:08.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c0ffe00 msgr2=0x7ff57c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:08.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c0ffe00 0x7ff57c072810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00d980 tx=0x7ff56c00dc90 comp rx=0 tx=0).stop 2023-07-14T11:21:08.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 shutdown_connections 2023-07-14T11:21:08.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff57c104ea0 0x7ff57c0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff5640a11e0 0x7ff5640a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff57c0ffe00 0x7ff57c072810 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 --2- 172.21.15.161:0/3697241323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c1007a0 0x7ff57c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:08.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 >> 172.21.15.161:0/3697241323 conn(0x7ff57c0fb660 msgr2=0x7ff57c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:08.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 shutdown_connections 2023-07-14T11:21:08.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:08.753+0000 7ff581d16700 1 -- 172.21.15.161:0/3697241323 wait complete. 2023-07-14T11:21:08.760 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:21:09.206 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2023-07-14T11:21:08.118821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651098,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":490,"num_read_kb":359,"num_write":334,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335352,"kb_used_data":11928,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606472,"statfs":{"total":1151940427776,"available":1151597027328,"internally_reserved":0,"allocated":12214272,"data_stored":7818135,"data_compressed":318627,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003228"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726364+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:20:15.726364+0000","last_peered":"2023-07-14T11:20:15.726364+0000","last_clean":"2023-07-14T11:20:15.726364+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:20:15.726364+0000","last_undegraded":"2023-07-14T11:20:15.726364+0000","last_fullsized":"2023-07-14T11:20:15.726364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:25.074131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726699+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:20:15.726699+0000","last_peered":"2023-07-14T11:20:15.726699+0000","last_clean":"2023-07-14T11:20:15.726699+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:20:15.726699+0000","last_undegraded":"2023-07-14T11:20:15.726699+0000","last_fullsized":"2023-07-14T11:20:15.726699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:41.854592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726554+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:20:15.726554+0000","last_peered":"2023-07-14T11:20:15.726554+0000","last_clean":"2023-07-14T11:20:15.726554+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:20:15.726554+0000","last_undegraded":"2023-07-14T11:20:15.726554+0000","last_fullsized":"2023-07-14T11:20:15.726554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:37.342912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726181+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:20:15.726181+0000","last_peered":"2023-07-14T11:20:15.726181+0000","last_clean":"2023-07-14T11:20:15.726181+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:20:15.726181+0000","last_undegraded":"2023-07-14T11:20:15.726181+0000","last_fullsized":"2023-07-14T11:20:15.726181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:46.208987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728594+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:15.728594+0000","last_peered":"2023-07-14T11:20:15.728594+0000","last_clean":"2023-07-14T11:20:15.728594+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:15.728594+0000","last_undegraded":"2023-07-14T11:20:15.728594+0000","last_fullsized":"2023-07-14T11:20:15.728594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:53:26.823230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765846+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:20:15.765846+0000","last_peered":"2023-07-14T11:20:15.765846+0000","last_clean":"2023-07-14T11:20:15.765846+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:20:15.765846+0000","last_undegraded":"2023-07-14T11:20:15.765846+0000","last_fullsized":"2023-07-14T11:20:15.765846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:31:09.497877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725992+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:20:15.725992+0000","last_peered":"2023-07-14T11:20:15.725992+0000","last_clean":"2023-07-14T11:20:15.725992+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:20:15.725992+0000","last_undegraded":"2023-07-14T11:20:15.725992+0000","last_fullsized":"2023-07-14T11:20:15.725992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:18.893918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761828+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:20:15.761828+0000","last_peered":"2023-07-14T11:20:15.761828+0000","last_clean":"2023-07-14T11:20:15.761828+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:20:15.761828+0000","last_undegraded":"2023-07-14T11:20:15.761828+0000","last_fullsized":"2023-07-14T11:20:15.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730008+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:20:15.730008+0000","last_peered":"2023-07-14T11:20:15.730008+0000","last_clean":"2023-07-14T11:20:15.730008+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:20:15.730008+0000","last_undegraded":"2023-07-14T11:20:15.730008+0000","last_fullsized":"2023-07-14T11:20:15.730008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:25:58.920431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726577+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:20:15.726577+0000","last_peered":"2023-07-14T11:20:15.726577+0000","last_clean":"2023-07-14T11:20:15.726577+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:20:15.726577+0000","last_undegraded":"2023-07-14T11:20:15.726577+0000","last_fullsized":"2023-07-14T11:20:15.726577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:29.172086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726169+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:20:15.726169+0000","last_peered":"2023-07-14T11:20:15.726169+0000","last_clean":"2023-07-14T11:20:15.726169+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:20:15.726169+0000","last_undegraded":"2023-07-14T11:20:15.726169+0000","last_fullsized":"2023-07-14T11:20:15.726169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:29.746118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723416+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:15.723416+0000","last_peered":"2023-07-14T11:20:15.723416+0000","last_clean":"2023-07-14T11:20:15.723416+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:15.723416+0000","last_undegraded":"2023-07-14T11:20:15.723416+0000","last_fullsized":"2023-07-14T11:20:15.723416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:57.801423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725956+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:20:15.725956+0000","last_peered":"2023-07-14T11:20:15.725956+0000","last_clean":"2023-07-14T11:20:15.725956+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:20:15.725956+0000","last_undegraded":"2023-07-14T11:20:15.725956+0000","last_fullsized":"2023-07-14T11:20:15.725956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:01.673209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726027+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:20:15.726027+0000","last_peered":"2023-07-14T11:20:15.726027+0000","last_clean":"2023-07-14T11:20:15.726027+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:20:15.726027+0000","last_undegraded":"2023-07-14T11:20:15.726027+0000","last_fullsized":"2023-07-14T11:20:15.726027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:10.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726533+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:20:15.726533+0000","last_peered":"2023-07-14T11:20:15.726533+0000","last_clean":"2023-07-14T11:20:15.726533+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:20:15.726533+0000","last_undegraded":"2023-07-14T11:20:15.726533+0000","last_fullsized":"2023-07-14T11:20:15.726533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:33.733562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763940+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:20:15.763940+0000","last_peered":"2023-07-14T11:20:15.763940+0000","last_clean":"2023-07-14T11:20:15.763940+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:20:15.763940+0000","last_undegraded":"2023-07-14T11:20:15.763940+0000","last_fullsized":"2023-07-14T11:20:15.763940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:28:49.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725800+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:20:15.725800+0000","last_peered":"2023-07-14T11:20:15.725800+0000","last_clean":"2023-07-14T11:20:15.725800+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:20:15.725800+0000","last_undegraded":"2023-07-14T11:20:15.725800+0000","last_fullsized":"2023-07-14T11:20:15.725800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:30.183181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726357+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:20:15.726357+0000","last_peered":"2023-07-14T11:20:15.726357+0000","last_clean":"2023-07-14T11:20:15.726357+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:20:15.726357+0000","last_undegraded":"2023-07-14T11:20:15.726357+0000","last_fullsized":"2023-07-14T11:20:15.726357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:13.424624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763774+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:15.763774+0000","last_peered":"2023-07-14T11:20:15.763774+0000","last_clean":"2023-07-14T11:20:15.763774+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:15.763774+0000","last_undegraded":"2023-07-14T11:20:15.763774+0000","last_fullsized":"2023-07-14T11:20:15.763774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:54.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762089+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:20:15.762089+0000","last_peered":"2023-07-14T11:20:15.762089+0000","last_clean":"2023-07-14T11:20:15.762089+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:20:15.762089+0000","last_undegraded":"2023-07-14T11:20:15.762089+0000","last_fullsized":"2023-07-14T11:20:15.762089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:06.259730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724677+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:20:15.724677+0000","last_peered":"2023-07-14T11:20:15.724677+0000","last_clean":"2023-07-14T11:20:15.724677+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:20:15.724677+0000","last_undegraded":"2023-07-14T11:20:15.724677+0000","last_fullsized":"2023-07-14T11:20:15.724677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:07.076184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762219+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:20:15.762219+0000","last_peered":"2023-07-14T11:20:15.762219+0000","last_clean":"2023-07-14T11:20:15.762219+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:20:15.762219+0000","last_undegraded":"2023-07-14T11:20:15.762219+0000","last_fullsized":"2023-07-14T11:20:15.762219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:30.071523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726815+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:20:15.726815+0000","last_peered":"2023-07-14T11:20:15.726815+0000","last_clean":"2023-07-14T11:20:15.726815+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:20:15.726815+0000","last_undegraded":"2023-07-14T11:20:15.726815+0000","last_fullsized":"2023-07-14T11:20:15.726815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:41:41.007835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724865+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:20:15.724865+0000","last_peered":"2023-07-14T11:20:15.724865+0000","last_clean":"2023-07-14T11:20:15.724865+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:20:15.724865+0000","last_undegraded":"2023-07-14T11:20:15.724865+0000","last_fullsized":"2023-07-14T11:20:15.724865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:56:28.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729360+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.729360+0000","last_peered":"2023-07-14T11:20:15.729360+0000","last_clean":"2023-07-14T11:20:15.729360+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:15.729360+0000","last_undegraded":"2023-07-14T11:20:15.729360+0000","last_fullsized":"2023-07-14T11:20:15.729360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:54.894820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726665+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:20:15.726665+0000","last_peered":"2023-07-14T11:20:15.726665+0000","last_clean":"2023-07-14T11:20:15.726665+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:20:15.726665+0000","last_undegraded":"2023-07-14T11:20:15.726665+0000","last_fullsized":"2023-07-14T11:20:15.726665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:39.791849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728688+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:15.728688+0000","last_peered":"2023-07-14T11:20:15.728688+0000","last_clean":"2023-07-14T11:20:15.728688+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:15.728688+0000","last_undegraded":"2023-07-14T11:20:15.728688+0000","last_fullsized":"2023-07-14T11:20:15.728688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:23:54.582861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759435+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:20:15.759435+0000","last_peered":"2023-07-14T11:20:15.759435+0000","last_clean":"2023-07-14T11:20:15.759435+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:20:15.759435+0000","last_undegraded":"2023-07-14T11:20:15.759435+0000","last_fullsized":"2023-07-14T11:20:15.759435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:51.343442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725567+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:20:15.725567+0000","last_peered":"2023-07-14T11:20:15.725567+0000","last_clean":"2023-07-14T11:20:15.725567+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:20:15.725567+0000","last_undegraded":"2023-07-14T11:20:15.725567+0000","last_fullsized":"2023-07-14T11:20:15.725567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:50.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:06.761128+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:21:06.761128+0000","last_peered":"2023-07-14T11:21:06.761128+0000","last_clean":"2023-07-14T11:21:06.761128+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:21:06.761128+0000","last_undegraded":"2023-07-14T11:21:06.761128+0000","last_fullsized":"2023-07-14T11:21:06.761128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:41.232654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728839+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:15.728839+0000","last_peered":"2023-07-14T11:20:15.728839+0000","last_clean":"2023-07-14T11:20:15.728839+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:15.728839+0000","last_undegraded":"2023-07-14T11:20:15.728839+0000","last_fullsized":"2023-07-14T11:20:15.728839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:12.092163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766231+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:20:15.766231+0000","last_peered":"2023-07-14T11:20:15.766231+0000","last_clean":"2023-07-14T11:20:15.766231+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:20:15.766231+0000","last_undegraded":"2023-07-14T11:20:15.766231+0000","last_fullsized":"2023-07-14T11:20:15.766231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:16:19.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730104+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:15.730104+0000","last_peered":"2023-07-14T11:20:15.730104+0000","last_clean":"2023-07-14T11:20:15.730104+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:15.730104+0000","last_undegraded":"2023-07-14T11:20:15.730104+0000","last_fullsized":"2023-07-14T11:20:15.730104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:54:17.106221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759228+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:20:15.759228+0000","last_peered":"2023-07-14T11:20:15.759228+0000","last_clean":"2023-07-14T11:20:15.759228+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:20:15.759228+0000","last_undegraded":"2023-07-14T11:20:15.759228+0000","last_fullsized":"2023-07-14T11:20:15.759228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:47.650473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726249+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:20:15.726249+0000","last_peered":"2023-07-14T11:20:15.726249+0000","last_clean":"2023-07-14T11:20:15.726249+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:20:15.726249+0000","last_undegraded":"2023-07-14T11:20:15.726249+0000","last_fullsized":"2023-07-14T11:20:15.726249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:10.265432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722943+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:15.722943+0000","last_peered":"2023-07-14T11:20:15.722943+0000","last_clean":"2023-07-14T11:20:15.722943+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:15.722943+0000","last_undegraded":"2023-07-14T11:20:15.722943+0000","last_fullsized":"2023-07-14T11:20:15.722943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:48.217979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761668+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:20:15.761668+0000","last_peered":"2023-07-14T11:20:15.761668+0000","last_clean":"2023-07-14T11:20:15.761668+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:20:15.761668+0000","last_undegraded":"2023-07-14T11:20:15.761668+0000","last_fullsized":"2023-07-14T11:20:15.761668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.096155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726651+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:20:15.726651+0000","last_peered":"2023-07-14T11:20:15.726651+0000","last_clean":"2023-07-14T11:20:15.726651+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:20:15.726651+0000","last_undegraded":"2023-07-14T11:20:15.726651+0000","last_fullsized":"2023-07-14T11:20:15.726651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:36.842168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758847+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:20:15.758847+0000","last_peered":"2023-07-14T11:20:15.758847+0000","last_clean":"2023-07-14T11:20:15.758847+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:20:15.758847+0000","last_undegraded":"2023-07-14T11:20:15.758847+0000","last_fullsized":"2023-07-14T11:20:15.758847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:38.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763190+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:20:15.763190+0000","last_peered":"2023-07-14T11:20:15.763190+0000","last_clean":"2023-07-14T11:20:15.763190+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:20:15.763190+0000","last_undegraded":"2023-07-14T11:20:15.763190+0000","last_fullsized":"2023-07-14T11:20:15.763190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:23.015078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761792+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:20:15.761792+0000","last_peered":"2023-07-14T11:20:15.761792+0000","last_clean":"2023-07-14T11:20:15.761792+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:20:15.761792+0000","last_undegraded":"2023-07-14T11:20:15.761792+0000","last_fullsized":"2023-07-14T11:20:15.761792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.645359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:05.758700+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:21:05.758700+0000","last_peered":"2023-07-14T11:21:05.758700+0000","last_clean":"2023-07-14T11:21:05.758700+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:21:05.758700+0000","last_undegraded":"2023-07-14T11:21:05.758700+0000","last_fullsized":"2023-07-14T11:21:05.758700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.137485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763325+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:20:15.763325+0000","last_peered":"2023-07-14T11:20:15.763325+0000","last_clean":"2023-07-14T11:20:15.763325+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:20:15.763325+0000","last_undegraded":"2023-07-14T11:20:15.763325+0000","last_fullsized":"2023-07-14T11:20:15.763325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:28.186030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726329+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:15.726329+0000","last_peered":"2023-07-14T11:20:15.726329+0000","last_clean":"2023-07-14T11:20:15.726329+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:15.726329+0000","last_undegraded":"2023-07-14T11:20:15.726329+0000","last_fullsized":"2023-07-14T11:20:15.726329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:38.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726931+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:20:15.726931+0000","last_peered":"2023-07-14T11:20:15.726931+0000","last_clean":"2023-07-14T11:20:15.726931+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:20:15.726931+0000","last_undegraded":"2023-07-14T11:20:15.726931+0000","last_fullsized":"2023-07-14T11:20:15.726931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:58.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725362+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:20:15.725362+0000","last_peered":"2023-07-14T11:20:15.725362+0000","last_clean":"2023-07-14T11:20:15.725362+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:20:15.725362+0000","last_undegraded":"2023-07-14T11:20:15.725362+0000","last_fullsized":"2023-07-14T11:20:15.725362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:29.343315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759436+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:15.759436+0000","last_peered":"2023-07-14T11:20:15.759436+0000","last_clean":"2023-07-14T11:20:15.759436+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:15.759436+0000","last_undegraded":"2023-07-14T11:20:15.759436+0000","last_fullsized":"2023-07-14T11:20:15.759436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:17:07.579078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762929+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:20:15.762929+0000","last_peered":"2023-07-14T11:20:15.762929+0000","last_clean":"2023-07-14T11:20:15.762929+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:20:15.762929+0000","last_undegraded":"2023-07-14T11:20:15.762929+0000","last_fullsized":"2023-07-14T11:20:15.762929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:41.993201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":173,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:06.760309+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:21:06.760309+0000","last_peered":"2023-07-14T11:21:06.760309+0000","last_clean":"2023-07-14T11:21:06.760309+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:21:06.760309+0000","last_undegraded":"2023-07-14T11:21:06.760309+0000","last_fullsized":"2023-07-14T11:21:06.760309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:08:09.309573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56970,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":117,"num_read_kb":114,"num_write":31,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728912+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:15.728912+0000","last_peered":"2023-07-14T11:20:15.728912+0000","last_clean":"2023-07-14T11:20:15.728912+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:15.728912+0000","last_undegraded":"2023-07-14T11:20:15.728912+0000","last_fullsized":"2023-07-14T11:20:15.728912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:40:20.286989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759883+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:20:15.759883+0000","last_peered":"2023-07-14T11:20:15.759883+0000","last_clean":"2023-07-14T11:20:15.759883+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:20:15.759883+0000","last_undegraded":"2023-07-14T11:20:15.759883+0000","last_fullsized":"2023-07-14T11:20:15.759883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:27.460710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726089+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:15.726089+0000","last_peered":"2023-07-14T11:20:15.726089+0000","last_clean":"2023-07-14T11:20:15.726089+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:15.726089+0000","last_undegraded":"2023-07-14T11:20:15.726089+0000","last_fullsized":"2023-07-14T11:20:15.726089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:33:58.333180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722891+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:15.722891+0000","last_peered":"2023-07-14T11:20:15.722891+0000","last_clean":"2023-07-14T11:20:15.722891+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:15.722891+0000","last_undegraded":"2023-07-14T11:20:15.722891+0000","last_fullsized":"2023-07-14T11:20:15.722891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:12.630498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.732150+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:20:15.732150+0000","last_peered":"2023-07-14T11:20:15.732150+0000","last_clean":"2023-07-14T11:20:15.732150+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:20:15.732150+0000","last_undegraded":"2023-07-14T11:20:15.732150+0000","last_fullsized":"2023-07-14T11:20:15.732150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:49.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759053+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:15.759053+0000","last_peered":"2023-07-14T11:20:15.759053+0000","last_clean":"2023-07-14T11:20:15.759053+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:15.759053+0000","last_undegraded":"2023-07-14T11:20:15.759053+0000","last_fullsized":"2023-07-14T11:20:15.759053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:20:45.016996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765761+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:20:15.765761+0000","last_peered":"2023-07-14T11:20:15.765761+0000","last_clean":"2023-07-14T11:20:15.765761+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:20:15.765761+0000","last_undegraded":"2023-07-14T11:20:15.765761+0000","last_fullsized":"2023-07-14T11:20:15.765761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:11.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763313+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:20:15.763313+0000","last_peered":"2023-07-14T11:20:15.763313+0000","last_clean":"2023-07-14T11:20:15.763313+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:20:15.763313+0000","last_undegraded":"2023-07-14T11:20:15.763313+0000","last_fullsized":"2023-07-14T11:20:15.763313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:41.779784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":86,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:05.758868+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:21:05.758868+0000","last_peered":"2023-07-14T11:21:05.758868+0000","last_clean":"2023-07-14T11:21:05.758868+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:21:05.758868+0000","last_undegraded":"2023-07-14T11:21:05.758868+0000","last_fullsized":"2023-07-14T11:21:05.758868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:35.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726189+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:20:15.726189+0000","last_peered":"2023-07-14T11:20:15.726189+0000","last_clean":"2023-07-14T11:20:15.726189+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:20:15.726189+0000","last_undegraded":"2023-07-14T11:20:15.726189+0000","last_fullsized":"2023-07-14T11:20:15.726189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:25.466461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723291+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:15.723291+0000","last_peered":"2023-07-14T11:20:15.723291+0000","last_clean":"2023-07-14T11:20:15.723291+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:15.723291+0000","last_undegraded":"2023-07-14T11:20:15.723291+0000","last_fullsized":"2023-07-14T11:20:15.723291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:05.698741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728325+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:15.728325+0000","last_peered":"2023-07-14T11:20:15.728325+0000","last_clean":"2023-07-14T11:20:15.728325+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:15.728325+0000","last_undegraded":"2023-07-14T11:20:15.728325+0000","last_fullsized":"2023-07-14T11:20:15.728325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:03:30.273008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758707+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:20:15.758707+0000","last_peered":"2023-07-14T11:20:15.758707+0000","last_clean":"2023-07-14T11:20:15.758707+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:20:15.758707+0000","last_undegraded":"2023-07-14T11:20:15.758707+0000","last_fullsized":"2023-07-14T11:20:15.758707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:32.059475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758890+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:20:15.758890+0000","last_peered":"2023-07-14T11:20:15.758890+0000","last_clean":"2023-07-14T11:20:15.758890+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:20:15.758890+0000","last_undegraded":"2023-07-14T11:20:15.758890+0000","last_fullsized":"2023-07-14T11:20:15.758890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:51.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729124+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:15.729124+0000","last_peered":"2023-07-14T11:20:15.729124+0000","last_clean":"2023-07-14T11:20:15.729124+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:15.729124+0000","last_undegraded":"2023-07-14T11:20:15.729124+0000","last_fullsized":"2023-07-14T11:20:15.729124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:06.994422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725218+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:20:15.725218+0000","last_peered":"2023-07-14T11:20:15.725218+0000","last_clean":"2023-07-14T11:20:15.725218+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:20:15.725218+0000","last_undegraded":"2023-07-14T11:20:15.725218+0000","last_fullsized":"2023-07-14T11:20:15.725218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:42.841223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.731601+0000","last_change":"2023-07-14T11:20:15.731601+0000","last_active":"2023-07-14T11:20:15.731601+0000","last_peered":"2023-07-14T11:20:15.731601+0000","last_clean":"2023-07-14T11:20:15.731601+0000","last_became_active":"2023-07-14T11:20:15.731121+0000","last_became_peered":"2023-07-14T11:20:15.731121+0000","last_unstale":"2023-07-14T11:20:15.731601+0000","last_undegraded":"2023-07-14T11:20:15.731601+0000","last_fullsized":"2023-07-14T11:20:15.731601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:58.090267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764473+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:20:15.764473+0000","last_peered":"2023-07-14T11:20:15.764473+0000","last_clean":"2023-07-14T11:20:15.764473+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:20:15.764473+0000","last_undegraded":"2023-07-14T11:20:15.764473+0000","last_fullsized":"2023-07-14T11:20:15.764473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:06.410233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729586+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:20:15.729586+0000","last_peered":"2023-07-14T11:20:15.729586+0000","last_clean":"2023-07-14T11:20:15.729586+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:20:15.729586+0000","last_undegraded":"2023-07-14T11:20:15.729586+0000","last_fullsized":"2023-07-14T11:20:15.729586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:59.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723225+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:15.723225+0000","last_peered":"2023-07-14T11:20:15.723225+0000","last_clean":"2023-07-14T11:20:15.723225+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:15.723225+0000","last_undegraded":"2023-07-14T11:20:15.723225+0000","last_fullsized":"2023-07-14T11:20:15.723225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:32.064021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726984+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.726984+0000","last_peered":"2023-07-14T11:20:15.726984+0000","last_clean":"2023-07-14T11:20:15.726984+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:20:15.726984+0000","last_undegraded":"2023-07-14T11:20:15.726984+0000","last_fullsized":"2023-07-14T11:20:15.726984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:38:06.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765912+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:20:15.765912+0000","last_peered":"2023-07-14T11:20:15.765912+0000","last_clean":"2023-07-14T11:20:15.765912+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:20:15.765912+0000","last_undegraded":"2023-07-14T11:20:15.765912+0000","last_fullsized":"2023-07-14T11:20:15.765912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:01:52.067140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723192+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:15.723192+0000","last_peered":"2023-07-14T11:20:15.723192+0000","last_clean":"2023-07-14T11:20:15.723192+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:15.723192+0000","last_undegraded":"2023-07-14T11:20:15.723192+0000","last_fullsized":"2023-07-14T11:20:15.723192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:44:48.912804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759017+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:20:15.759017+0000","last_peered":"2023-07-14T11:20:15.759017+0000","last_clean":"2023-07-14T11:20:15.759017+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:20:15.759017+0000","last_undegraded":"2023-07-14T11:20:15.759017+0000","last_fullsized":"2023-07-14T11:20:15.759017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:30:26.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":83,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:03.754939+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:21:03.754939+0000","last_peered":"2023-07-14T11:21:03.754939+0000","last_clean":"2023-07-14T11:21:03.754939+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:21:03.754939+0000","last_undegraded":"2023-07-14T11:21:03.754939+0000","last_fullsized":"2023-07-14T11:21:03.754939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:54:46.013281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726387+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:20:15.726387+0000","last_peered":"2023-07-14T11:20:15.726387+0000","last_clean":"2023-07-14T11:20:15.726387+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:20:15.726387+0000","last_undegraded":"2023-07-14T11:20:15.726387+0000","last_fullsized":"2023-07-14T11:20:15.726387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:45.551283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761576+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:20:15.761576+0000","last_peered":"2023-07-14T11:20:15.761576+0000","last_clean":"2023-07-14T11:20:15.761576+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:20:15.761576+0000","last_undegraded":"2023-07-14T11:20:15.761576+0000","last_fullsized":"2023-07-14T11:20:15.761576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:35.428242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725019+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:15.725019+0000","last_peered":"2023-07-14T11:20:15.725019+0000","last_clean":"2023-07-14T11:20:15.725019+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:15.725019+0000","last_undegraded":"2023-07-14T11:20:15.725019+0000","last_fullsized":"2023-07-14T11:20:15.725019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:42:08.666520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725963+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:20:15.725963+0000","last_peered":"2023-07-14T11:20:15.725963+0000","last_clean":"2023-07-14T11:20:15.725963+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:20:15.725963+0000","last_undegraded":"2023-07-14T11:20:15.725963+0000","last_fullsized":"2023-07-14T11:20:15.725963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:49.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759565+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:20:15.759565+0000","last_peered":"2023-07-14T11:20:15.759565+0000","last_clean":"2023-07-14T11:20:15.759565+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:20:15.759565+0000","last_undegraded":"2023-07-14T11:20:15.759565+0000","last_fullsized":"2023-07-14T11:20:15.759565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:31.340558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722788+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:15.722788+0000","last_peered":"2023-07-14T11:20:15.722788+0000","last_clean":"2023-07-14T11:20:15.722788+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:15.722788+0000","last_undegraded":"2023-07-14T11:20:15.722788+0000","last_fullsized":"2023-07-14T11:20:15.722788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:40:45.755906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726149+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:15.726149+0000","last_peered":"2023-07-14T11:20:15.726149+0000","last_clean":"2023-07-14T11:20:15.726149+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:15.726149+0000","last_undegraded":"2023-07-14T11:20:15.726149+0000","last_fullsized":"2023-07-14T11:20:15.726149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:29.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725637+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:20:15.725637+0000","last_peered":"2023-07-14T11:20:15.725637+0000","last_clean":"2023-07-14T11:20:15.725637+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:20:15.725637+0000","last_undegraded":"2023-07-14T11:20:15.725637+0000","last_fullsized":"2023-07-14T11:20:15.725637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:47:19.355328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729053+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:15.729053+0000","last_peered":"2023-07-14T11:20:15.729053+0000","last_clean":"2023-07-14T11:20:15.729053+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:15.729053+0000","last_undegraded":"2023-07-14T11:20:15.729053+0000","last_fullsized":"2023-07-14T11:20:15.729053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:06:22.336930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723369+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:15.723369+0000","last_peered":"2023-07-14T11:20:15.723369+0000","last_clean":"2023-07-14T11:20:15.723369+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:15.723369+0000","last_undegraded":"2023-07-14T11:20:15.723369+0000","last_fullsized":"2023-07-14T11:20:15.723369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:47.388288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761518+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:20:15.761518+0000","last_peered":"2023-07-14T11:20:15.761518+0000","last_clean":"2023-07-14T11:20:15.761518+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:20:15.761518+0000","last_undegraded":"2023-07-14T11:20:15.761518+0000","last_fullsized":"2023-07-14T11:20:15.761518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:28.833104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764010+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:15.764010+0000","last_peered":"2023-07-14T11:20:15.764010+0000","last_clean":"2023-07-14T11:20:15.764010+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:15.764010+0000","last_undegraded":"2023-07-14T11:20:15.764010+0000","last_fullsized":"2023-07-14T11:20:15.764010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:18.046365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762296+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:20:15.762296+0000","last_peered":"2023-07-14T11:20:15.762296+0000","last_clean":"2023-07-14T11:20:15.762296+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:20:15.762296+0000","last_undegraded":"2023-07-14T11:20:15.762296+0000","last_fullsized":"2023-07-14T11:20:15.762296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:04:14.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725262+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:20:15.725262+0000","last_peered":"2023-07-14T11:20:15.725262+0000","last_clean":"2023-07-14T11:20:15.725262+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:20:15.725262+0000","last_undegraded":"2023-07-14T11:20:15.725262+0000","last_fullsized":"2023-07-14T11:20:15.725262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:40:44.958728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725446+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:15.725446+0000","last_peered":"2023-07-14T11:20:15.725446+0000","last_clean":"2023-07-14T11:20:15.725446+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:15.725446+0000","last_undegraded":"2023-07-14T11:20:15.725446+0000","last_fullsized":"2023-07-14T11:20:15.725446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.980346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729830+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:20:15.729830+0000","last_peered":"2023-07-14T11:20:15.729830+0000","last_clean":"2023-07-14T11:20:15.729830+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:20:15.729830+0000","last_undegraded":"2023-07-14T11:20:15.729830+0000","last_fullsized":"2023-07-14T11:20:15.729830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:23.746186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724337+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:20:15.724337+0000","last_peered":"2023-07-14T11:20:15.724337+0000","last_clean":"2023-07-14T11:20:15.724337+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:20:15.724337+0000","last_undegraded":"2023-07-14T11:20:15.724337+0000","last_fullsized":"2023-07-14T11:20:15.724337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:32.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728279+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:15.728279+0000","last_peered":"2023-07-14T11:20:15.728279+0000","last_clean":"2023-07-14T11:20:15.728279+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:15.728279+0000","last_undegraded":"2023-07-14T11:20:15.728279+0000","last_fullsized":"2023-07-14T11:20:15.728279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:32.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763590+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:20:15.763590+0000","last_peered":"2023-07-14T11:20:15.763590+0000","last_clean":"2023-07-14T11:20:15.763590+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:20:15.763590+0000","last_undegraded":"2023-07-14T11:20:15.763590+0000","last_fullsized":"2023-07-14T11:20:15.763590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:06:43.200171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761544+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:15.761544+0000","last_peered":"2023-07-14T11:20:15.761544+0000","last_clean":"2023-07-14T11:20:15.761544+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:15.761544+0000","last_undegraded":"2023-07-14T11:20:15.761544+0000","last_fullsized":"2023-07-14T11:20:15.761544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:10:23.505930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:25.389747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725984+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:20:15.725984+0000","last_peered":"2023-07-14T11:20:15.725984+0000","last_clean":"2023-07-14T11:20:15.725984+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:20:15.725984+0000","last_undegraded":"2023-07-14T11:20:15.725984+0000","last_fullsized":"2023-07-14T11:20:15.725984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:28.285910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725493+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:20:15.725493+0000","last_peered":"2023-07-14T11:20:15.725493+0000","last_clean":"2023-07-14T11:20:15.725493+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:20:15.725493+0000","last_undegraded":"2023-07-14T11:20:15.725493+0000","last_fullsized":"2023-07-14T11:20:15.725493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:49.791341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761896+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:20:15.761896+0000","last_peered":"2023-07-14T11:20:15.761896+0000","last_clean":"2023-07-14T11:20:15.761896+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:20:15.761896+0000","last_undegraded":"2023-07-14T11:20:15.761896+0000","last_fullsized":"2023-07-14T11:20:15.761896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:04:06.442323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724354+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:20:15.724354+0000","last_peered":"2023-07-14T11:20:15.724354+0000","last_clean":"2023-07-14T11:20:15.724354+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:20:15.724354+0000","last_undegraded":"2023-07-14T11:20:15.724354+0000","last_fullsized":"2023-07-14T11:20:15.724354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:31.530941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728516+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:15.728516+0000","last_peered":"2023-07-14T11:20:15.728516+0000","last_clean":"2023-07-14T11:20:15.728516+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:15.728516+0000","last_undegraded":"2023-07-14T11:20:15.728516+0000","last_fullsized":"2023-07-14T11:20:15.728516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:29.709167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727131+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:20:15.727131+0000","last_peered":"2023-07-14T11:20:15.727131+0000","last_clean":"2023-07-14T11:20:15.727131+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:20:15.727131+0000","last_undegraded":"2023-07-14T11:20:15.727131+0000","last_fullsized":"2023-07-14T11:20:15.727131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:31:54.387366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727120+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:20:15.727120+0000","last_peered":"2023-07-14T11:20:15.727120+0000","last_clean":"2023-07-14T11:20:15.727120+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:20:15.727120+0000","last_undegraded":"2023-07-14T11:20:15.727120+0000","last_fullsized":"2023-07-14T11:20:15.727120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:15.525008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726470+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:20:15.726470+0000","last_peered":"2023-07-14T11:20:15.726470+0000","last_clean":"2023-07-14T11:20:15.726470+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:20:15.726470+0000","last_undegraded":"2023-07-14T11:20:15.726470+0000","last_fullsized":"2023-07-14T11:20:15.726470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:03.965708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729884+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:20:15.729884+0000","last_peered":"2023-07-14T11:20:15.729884+0000","last_clean":"2023-07-14T11:20:15.729884+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:20:15.729884+0000","last_undegraded":"2023-07-14T11:20:15.729884+0000","last_fullsized":"2023-07-14T11:20:15.729884+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:31.118854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759707+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:20:15.759707+0000","last_peered":"2023-07-14T11:20:15.759707+0000","last_clean":"2023-07-14T11:20:15.759707+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:20:15.759707+0000","last_undegraded":"2023-07-14T11:20:15.759707+0000","last_fullsized":"2023-07-14T11:20:15.759707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:35.715875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766177+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:20:15.766177+0000","last_peered":"2023-07-14T11:20:15.766177+0000","last_clean":"2023-07-14T11:20:15.766177+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:20:15.766177+0000","last_undegraded":"2023-07-14T11:20:15.766177+0000","last_fullsized":"2023-07-14T11:20:15.766177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:59.517623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758537+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:20:15.758537+0000","last_peered":"2023-07-14T11:20:15.758537+0000","last_clean":"2023-07-14T11:20:15.758537+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:20:15.758537+0000","last_undegraded":"2023-07-14T11:20:15.758537+0000","last_fullsized":"2023-07-14T11:20:15.758537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.641650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723412+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:15.723412+0000","last_peered":"2023-07-14T11:20:15.723412+0000","last_clean":"2023-07-14T11:20:15.723412+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:15.723412+0000","last_undegraded":"2023-07-14T11:20:15.723412+0000","last_fullsized":"2023-07-14T11:20:15.723412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:35.428375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726258+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:20:15.726258+0000","last_peered":"2023-07-14T11:20:15.726258+0000","last_clean":"2023-07-14T11:20:15.726258+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:20:15.726258+0000","last_undegraded":"2023-07-14T11:20:15.726258+0000","last_fullsized":"2023-07-14T11:20:15.726258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:41.949605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:48.091011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725591+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:20:15.725591+0000","last_peered":"2023-07-14T11:20:15.725591+0000","last_clean":"2023-07-14T11:20:15.725591+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:20:15.725591+0000","last_undegraded":"2023-07-14T11:20:15.725591+0000","last_fullsized":"2023-07-14T11:20:15.725591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:15.490002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726260+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:20:15.726260+0000","last_peered":"2023-07-14T11:20:15.726260+0000","last_clean":"2023-07-14T11:20:15.726260+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:20:15.726260+0000","last_undegraded":"2023-07-14T11:20:15.726260+0000","last_fullsized":"2023-07-14T11:20:15.726260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:53:08.815873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726407+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:20:15.726407+0000","last_peered":"2023-07-14T11:20:15.726407+0000","last_clean":"2023-07-14T11:20:15.726407+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:20:15.726407+0000","last_undegraded":"2023-07-14T11:20:15.726407+0000","last_fullsized":"2023-07-14T11:20:15.726407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:08.673713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723496+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:15.723496+0000","last_peered":"2023-07-14T11:20:15.723496+0000","last_clean":"2023-07-14T11:20:15.723496+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:15.723496+0000","last_undegraded":"2023-07-14T11:20:15.723496+0000","last_fullsized":"2023-07-14T11:20:15.723496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:41:54.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726094+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:20:15.726094+0000","last_peered":"2023-07-14T11:20:15.726094+0000","last_clean":"2023-07-14T11:20:15.726094+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:20:15.726094+0000","last_undegraded":"2023-07-14T11:20:15.726094+0000","last_fullsized":"2023-07-14T11:20:15.726094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:36:46.120312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725900+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:20:15.725900+0000","last_peered":"2023-07-14T11:20:15.725900+0000","last_clean":"2023-07-14T11:20:15.725900+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:20:15.725900+0000","last_undegraded":"2023-07-14T11:20:15.725900+0000","last_fullsized":"2023-07-14T11:20:15.725900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:29.569291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762498+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:20:15.762498+0000","last_peered":"2023-07-14T11:20:15.762498+0000","last_clean":"2023-07-14T11:20:15.762498+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:20:15.762498+0000","last_undegraded":"2023-07-14T11:20:15.762498+0000","last_fullsized":"2023-07-14T11:20:15.762498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:46.020055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765762+0000","last_change":"2023-07-14T11:20:15.765762+0000","last_active":"2023-07-14T11:20:15.765762+0000","last_peered":"2023-07-14T11:20:15.765762+0000","last_clean":"2023-07-14T11:20:15.765762+0000","last_became_active":"2023-07-14T11:20:15.765349+0000","last_became_peered":"2023-07-14T11:20:15.765349+0000","last_unstale":"2023-07-14T11:20:15.765762+0000","last_undegraded":"2023-07-14T11:20:15.765762+0000","last_fullsized":"2023-07-14T11:20:15.765762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:51:54.292936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761928+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:20:15.761928+0000","last_peered":"2023-07-14T11:20:15.761928+0000","last_clean":"2023-07-14T11:20:15.761928+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:20:15.761928+0000","last_undegraded":"2023-07-14T11:20:15.761928+0000","last_fullsized":"2023-07-14T11:20:15.761928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:50:15.332355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723311+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:15.723311+0000","last_peered":"2023-07-14T11:20:15.723311+0000","last_clean":"2023-07-14T11:20:15.723311+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:15.723311+0000","last_undegraded":"2023-07-14T11:20:15.723311+0000","last_fullsized":"2023-07-14T11:20:15.723311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:13.493038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725897+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:20:15.725897+0000","last_peered":"2023-07-14T11:20:15.725897+0000","last_clean":"2023-07-14T11:20:15.725897+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:20:15.725897+0000","last_undegraded":"2023-07-14T11:20:15.725897+0000","last_fullsized":"2023-07-14T11:20:15.725897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:20.934793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60484,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":354,"num_read_kb":240,"num_write":151,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":181452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":490869,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82899999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80700000000000005,"5min":1.097,"15min":1.097},"last":0.629}]},{"osd":2,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.30599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.62,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46400000000000002,"15min":0.45700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97899999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":1.042,"15min":1.042},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83099999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.379}]},{"osd":6,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.41199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.873,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78300000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53200000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.999,"5min":1.159,"15min":1.159},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.85099999999999998,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56799999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.871,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.046,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.92300000000000004,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.92900000000000005,"5min":1.456,"15min":1.456},"last":0.75}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1118208,"data_stored":1135059,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":5.9969999999999999,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":6.0090000000000003,"5min":6.0090000000000003,"15min":6.0090000000000003},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.753,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.68899999999999995,"15min":0.70299999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85199999999999998,"5min":2.46,"15min":2.46},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86599999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.79300000000000004,"5min":1.349,"15min":1.349},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74199999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.373,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.625}]},{"osd":5,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.78000000000000003,"15min":0.79700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81299999999999994,"5min":2.431,"15min":2.431},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.81899999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.80200000000000005,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88700000000000001,"5min":1.044,"15min":1.044},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95299999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.90200000000000002,"5min":1.01,"15min":1.01},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.88900000000000001,"15min":0.90900000000000003},"min":{"1min":0.64900000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.92300000000000004,"5min":2.536,"15min":2.536},"last":0.75900000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.85099999999999998,"15min":0.85899999999999999},"min":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97899999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.93700000000000006,"15min":0.96299999999999997},"min":{"1min":0.63500000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0009999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Jul 14 11:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86799999999999999,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95999999999999996,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.70299999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.054},"last":0.69099999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":495201,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":1.012,"15min":1.012},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.997,"15min":1.0489999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":0.999,"15min":0.999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0880000000000001,"5min":1.264,"15min":1.264},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.56499999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.999,"5min":1.073,"15min":1.073},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.84099999999999997,"15min":0.86299999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93100000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.872,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0409999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.64800000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.151,"15min":1.2090000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99299999999999999,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90100000000000002,"15min":0.91700000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.081,"5min":1.171,"15min":1.171},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.872},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.93100000000000005,"15min":0.94499999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1319999999999999,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.66100000000000003}]},{"osd":11,"last update":"Fri Jul 14 11:20:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.752},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.78000000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":491025,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.52400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.45500000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.48499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78500000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.748}]},{"osd":2,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.34499999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64700000000000002,"15min":0.67300000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.876,"5min":2.032,"15min":2.032},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.51400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.48199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.65200000000000002,"15min":0.628},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83599999999999997}]},{"osd":5,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.56499999999999995,"15min":0.54800000000000004},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.91100000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.58699999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.53500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.61299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":1.018}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.008}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.71599999999999997,"15min":0.69599999999999995},"min":{"1min":0.54900000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.0580000000000001}]},{"osd":11,"last update":"Fri Jul 14 11:20:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.995},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":1.032}]}]},{"osd":7,"up_from":52,"seq":223338299438,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1086116,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"min":{"1min":0.32800000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.38},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.44900000000000001,"15min":0.44500000000000001},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65100000000000002,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79400000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.70799999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.438}]},{"osd":5,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.873,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85999999999999999,"5min":1.149,"15min":1.149},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.80000000000000004,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.57599999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.72599999999999998,"15min":0.69299999999999995},"min":{"1min":0.52900000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68999999999999995,"15min":0.67300000000000004},"min":{"1min":0.53600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85199999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51700000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.131,"15min":1.131},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.77000000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.88400000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89100000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59499999999999997}]},{"osd":11,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70999999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486008,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.499}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":0.85299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.52600000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.79900000000000004,"15min":0.77400000000000002},"min":{"1min":0.65600000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0880000000000001,"5min":1.115,"15min":1.115},"last":0.82299999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.83299999999999996,"15min":0.80900000000000005},"min":{"1min":0.61499999999999999,"5min":0.495,"15min":0.495},"max":{"1min":2.3069999999999999,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.81200000000000006,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.413,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.90000000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.70899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.4450000000000001,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.93100000000000005,"15min":0.92500000000000004},"min":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.93300000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:20:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":543161,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.57299999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":4.077}]},{"osd":2,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57299999999999995,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.59099999999999997,"15min":0.51700000000000002},"min":{"1min":0.52300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89300000000000002,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70199999999999996,"15min":0.64900000000000002},"min":{"1min":0.44700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":4.016,"5min":4.016,"15min":4.016},"last":4.016}]},{"osd":4,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.40999999999999998},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.497,"15min":0.46500000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66100000000000003,"15min":0.625},"min":{"1min":0.55600000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.51500000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":4.1219999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":4.1459999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88900000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84499999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71199999999999997,"15min":0.68400000000000005},"min":{"1min":0.56399999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97999999999999998,"5min":1.036,"15min":1.036},"last":0.97699999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.93500000000000005}]},{"osd":10,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96399999999999997,"5min":1.014,"15min":1.014},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.95199999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":1}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":487396,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.998,"15min":0.998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.437,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.66900000000000004,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.9169999999999998,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.65900000000000003,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.2060000000000004,"5min":5.2060000000000004,"15min":5.2060000000000004},"last":0.441}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93400000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.70099999999999996,"15min":0.63200000000000001},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.435},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.755},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.72699999999999998,"15min":0.66100000000000003},"min":{"1min":0.56299999999999994,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.2489999999999997,"5min":5.2489999999999997,"15min":5.2489999999999997},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.351,"5min":5.351,"15min":5.351},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 11:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.86799999999999999,"15min":0.871},"min":{"1min":0.61699999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":5.2839999999999998,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0029999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.63600000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1299999999999999,"5min":1.611,"15min":1.611},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0760000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.65600000000000003}]},{"osd":11,"last update":"Fri Jul 14 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.85399999999999998,"15min":0.81599999999999995},"min":{"1min":0.628,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"last":0.874},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.88600000000000001,"15min":0.84399999999999997},"min":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":0.77800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":544190,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":1.012,"15min":1.012},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.746},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57899999999999996,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.46999999999999997,"15min":0.44800000000000001},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.439,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83499999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.93300000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.47199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65100000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Jul 14 11:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72399999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486475,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.52800000000000002,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.4649999999999999,"5min":7.4649999999999999,"15min":7.4649999999999999},"last":0.5},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.54700000000000004,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":6.4809999999999999,"5min":6.4809999999999999,"15min":6.4809999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.71699999999999997,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.7329999999999997,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.73799999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":6.335,"5min":6.335,"15min":6.335},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60399999999999998,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.69999999999999996,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"last":0.77100000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67600000000000005,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.60599999999999998},"min":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89900000000000002,"5min":1.089,"15min":1.089},"last":0.90600000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.75900000000000001,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.8570000000000002,"5min":6.8570000000000002,"15min":6.8570000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.52300000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.749}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:20:31 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.58099999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.032,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.878},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.96799999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.59999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96699999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.98599999999999999,"15min":1.0169999999999999},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":4.157,"15min":4.157},"last":0.94999999999999996}]},{"osd":10,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.97199999999999998,"15min":1.0029999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1259999999999999,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.119,"5min":1.218,"15min":1.218},"last":0.93500000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":6.8120000000000003,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.85999999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490883,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.25,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56000000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.753},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.3090000000000002,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.624},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79100000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59499999999999997,"15min":0.56200000000000006},"min":{"1min":0.55700000000000005,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82499999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.49199999999999999,"15min":0.47699999999999998},"min":{"1min":0.35799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72099999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.52400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.755,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76900000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80900000000000005}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75700000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67800000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73899999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1081752,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":6.6680000000000001,"5min":6.6680000000000001,"15min":6.6680000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.53700000000000003,"15min":0.48799999999999999},"min":{"1min":0.35799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.877,"15min":0.877},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.75,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.38700000000000001,"15min":0.35799999999999998},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.66800000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.442},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.38800000000000001},"min":{"1min":0.33600000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.69699999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.61799999999999999,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":3.3759999999999999,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.626,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57999999999999996,"15min":0.54200000000000004},"min":{"1min":0.47699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81200000000000006,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.57299999999999995,"15min":0.53800000000000003},"min":{"1min":0.44500000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.53500000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.50800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.038,"15min":1.038},"last":0.52600000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85599999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81299999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48599999999999999}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.751},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70299999999999996}]},{"osd":10,"last update":"Fri Jul 14 11:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85899999999999999,"5min":1.073,"15min":1.073},"last":0.80700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:21:09.217 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph pg dump --format=json 2023-07-14T11:21:09.843 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:09.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:09 smithi161 ceph-mon[95526]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:10.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:09 smithi175 ceph-mon[129081]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:10.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:09 smithi182 ceph-mon[129366]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:10.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:10 smithi161 ceph-mon[95526]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:21:11.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:10 smithi175 ceph-mon[129081]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:21:11.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:10 smithi182 ceph-mon[129366]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:21:11.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 -- 172.21.15.161:0/546270578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c0ffe00 msgr2=0x7f328c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:11.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 --2- 172.21.15.161:0/546270578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c0ffe00 0x7f328c1001d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32880099c0 tx=0x7f3288009cd0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 -- 172.21.15.161:0/546270578 shutdown_connections 2023-07-14T11:21:11.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 --2- 172.21.15.161:0/546270578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c0ffe00 0x7f328c1001d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 --2- 172.21.15.161:0/546270578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f328c104ea0 0x7f328c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 --2- 172.21.15.161:0/546270578 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f328c1007a0 0x7f328c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 -- 172.21.15.161:0/546270578 >> 172.21.15.161:0/546270578 conn(0x7f328c0fb660 msgr2=0x7f328c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:11.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 -- 172.21.15.161:0/546270578 shutdown_connections 2023-07-14T11:21:11.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.340+0000 7f3293f4d700 1 -- 172.21.15.161:0/546270578 wait complete. 2023-07-14T11:21:11.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.341+0000 7f3293f4d700 1 Processor -- start 2023-07-14T11:21:11.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.341+0000 7f3293f4d700 1 -- start start 2023-07-14T11:21:11.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f3293f4d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f328c0ffe00 0x7f328c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:11.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f3293f4d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f328c1007a0 0x7f328c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:11.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f3293f4d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c104ea0 0x7f328c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:11.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f3293f4d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1a40b0 con 0x7f328c104ea0 2023-07-14T11:21:11.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f3293f4d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1a4220 con 0x7f328c1007a0 2023-07-14T11:21:11.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f3293f4d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1a43e0 con 0x7f328c0ffe00 2023-07-14T11:21:11.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f32924ea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c104ea0 0x7f328c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:11.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f3291ce9700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f328c0ffe00 0x7f328c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:11.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f32924ea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c104ea0 0x7f328c0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40870/0 (socket says 172.21.15.161:40870) 2023-07-14T11:21:11.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f32924ea700 1 -- 172.21.15.161:0/1406761631 learned_addr learned my addr 172.21.15.161:0/1406761631 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:11.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.342+0000 7f32914e8700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f328c1007a0 0x7f328c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:11.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3291ce9700 1 -- 172.21.15.161:0/1406761631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f328c1007a0 msgr2=0x7f328c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:11.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3291ce9700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f328c1007a0 0x7f328c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3291ce9700 1 -- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c104ea0 msgr2=0x7f328c0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:11.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3291ce9700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c104ea0 0x7f328c0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3291ce9700 1 -- 172.21.15.161:0/1406761631 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32880095f0 con 0x7f328c0ffe00 2023-07-14T11:21:11.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f32924ea700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c104ea0 0x7f328c0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:11.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3291ce9700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f328c0ffe00 0x7f328c072810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f32880098e0 tx=0x7f328800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:11.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3282ffd700 1 -- 172.21.15.161:0/1406761631 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328801d070 con 0x7f328c0ffe00 2023-07-14T11:21:11.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c1a4660 con 0x7f328c0ffe00 2023-07-14T11:21:11.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3282ffd700 1 -- 172.21.15.161:0/1406761631 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3288005050 con 0x7f328c0ffe00 2023-07-14T11:21:11.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3282ffd700 1 -- 172.21.15.161:0/1406761631 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288004970 con 0x7f328c0ffe00 2023-07-14T11:21:11.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.343+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c1a4ad0 con 0x7f328c0ffe00 2023-07-14T11:21:11.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.344+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c04e210 con 0x7f328c0ffe00 2023-07-14T11:21:11.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.346+0000 7f3282ffd700 1 -- 172.21.15.161:0/1406761631 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3288005470 con 0x7f328c0ffe00 2023-07-14T11:21:11.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.348+0000 7f3282ffd700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f32780a1260 0x7f32780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:11.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.348+0000 7f3282ffd700 1 -- 172.21.15.161:0/1406761631 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32880cdb20 con 0x7f328c0ffe00 2023-07-14T11:21:11.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.348+0000 7f32914e8700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f32780a1260 0x7f32780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:11.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.349+0000 7f32914e8700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f32780a1260 0x7f32780a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f328c074000 tx=0x7f327c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:11.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.350+0000 7f3282ffd700 1 -- 172.21.15.161:0/1406761631 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3288096e30 con 0x7f328c0ffe00 2023-07-14T11:21:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.542+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f328c0fc860 con 0x7f32780a1260 2023-07-14T11:21:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.554+0000 7f3282ffd700 1 -- 172.21.15.161:0/1406761631 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427269 (secure 0 0 0) 0x7f328c0fc860 con 0x7f32780a1260 2023-07-14T11:21:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:11.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f32780a1260 msgr2=0x7f32780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:11.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f32780a1260 0x7f32780a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f328c074000 tx=0x7f327c009260 comp rx=0 tx=0).stop 2023-07-14T11:21:11.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f328c0ffe00 msgr2=0x7f328c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:11.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f328c0ffe00 0x7f328c072810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f32880098e0 tx=0x7f328800f690 comp rx=0 tx=0).stop 2023-07-14T11:21:11.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 shutdown_connections 2023-07-14T11:21:11.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c104ea0 0x7f328c0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f32780a1260 0x7f32780a3710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f328c0ffe00 0x7f328c072810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 --2- 172.21.15.161:0/1406761631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f328c1007a0 0x7f328c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:11.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 >> 172.21.15.161:0/1406761631 conn(0x7f328c0fb660 msgr2=0x7f328c0fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:11.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.561+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 shutdown_connections 2023-07-14T11:21:11.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:11.562+0000 7f3293f4d700 1 -- 172.21.15.161:0/1406761631 wait complete. 2023-07-14T11:21:11.571 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:21:11.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:11 smithi161 ceph-mon[95526]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:11.935 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2023-07-14T11:21:10.119367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651098,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":495,"num_read_kb":364,"num_write":334,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335352,"kb_used_data":11928,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606472,"statfs":{"total":1151940427776,"available":1151597027328,"internally_reserved":0,"allocated":12214272,"data_stored":7818135,"data_compressed":318627,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003157"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726364+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:20:15.726364+0000","last_peered":"2023-07-14T11:20:15.726364+0000","last_clean":"2023-07-14T11:20:15.726364+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:20:15.726364+0000","last_undegraded":"2023-07-14T11:20:15.726364+0000","last_fullsized":"2023-07-14T11:20:15.726364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:25.074131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726699+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:20:15.726699+0000","last_peered":"2023-07-14T11:20:15.726699+0000","last_clean":"2023-07-14T11:20:15.726699+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:20:15.726699+0000","last_undegraded":"2023-07-14T11:20:15.726699+0000","last_fullsized":"2023-07-14T11:20:15.726699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:41.854592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726554+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:20:15.726554+0000","last_peered":"2023-07-14T11:20:15.726554+0000","last_clean":"2023-07-14T11:20:15.726554+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:20:15.726554+0000","last_undegraded":"2023-07-14T11:20:15.726554+0000","last_fullsized":"2023-07-14T11:20:15.726554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:37.342912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726181+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:20:15.726181+0000","last_peered":"2023-07-14T11:20:15.726181+0000","last_clean":"2023-07-14T11:20:15.726181+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:20:15.726181+0000","last_undegraded":"2023-07-14T11:20:15.726181+0000","last_fullsized":"2023-07-14T11:20:15.726181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:46.208987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728594+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:20:15.728594+0000","last_peered":"2023-07-14T11:20:15.728594+0000","last_clean":"2023-07-14T11:20:15.728594+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:20:15.728594+0000","last_undegraded":"2023-07-14T11:20:15.728594+0000","last_fullsized":"2023-07-14T11:20:15.728594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:53:26.823230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765846+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:20:15.765846+0000","last_peered":"2023-07-14T11:20:15.765846+0000","last_clean":"2023-07-14T11:20:15.765846+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:20:15.765846+0000","last_undegraded":"2023-07-14T11:20:15.765846+0000","last_fullsized":"2023-07-14T11:20:15.765846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:31:09.497877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725992+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:20:15.725992+0000","last_peered":"2023-07-14T11:20:15.725992+0000","last_clean":"2023-07-14T11:20:15.725992+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:20:15.725992+0000","last_undegraded":"2023-07-14T11:20:15.725992+0000","last_fullsized":"2023-07-14T11:20:15.725992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:18.893918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761828+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:20:15.761828+0000","last_peered":"2023-07-14T11:20:15.761828+0000","last_clean":"2023-07-14T11:20:15.761828+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:20:15.761828+0000","last_undegraded":"2023-07-14T11:20:15.761828+0000","last_fullsized":"2023-07-14T11:20:15.761828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.028545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724766+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:20:15.724766+0000","last_peered":"2023-07-14T11:20:15.724766+0000","last_clean":"2023-07-14T11:20:15.724766+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:20:15.724766+0000","last_undegraded":"2023-07-14T11:20:15.724766+0000","last_fullsized":"2023-07-14T11:20:15.724766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:36:41.060858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730008+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:20:15.730008+0000","last_peered":"2023-07-14T11:20:15.730008+0000","last_clean":"2023-07-14T11:20:15.730008+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:20:15.730008+0000","last_undegraded":"2023-07-14T11:20:15.730008+0000","last_fullsized":"2023-07-14T11:20:15.730008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:25:58.920431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726577+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:20:15.726577+0000","last_peered":"2023-07-14T11:20:15.726577+0000","last_clean":"2023-07-14T11:20:15.726577+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:20:15.726577+0000","last_undegraded":"2023-07-14T11:20:15.726577+0000","last_fullsized":"2023-07-14T11:20:15.726577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:29.172086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726169+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:20:15.726169+0000","last_peered":"2023-07-14T11:20:15.726169+0000","last_clean":"2023-07-14T11:20:15.726169+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:20:15.726169+0000","last_undegraded":"2023-07-14T11:20:15.726169+0000","last_fullsized":"2023-07-14T11:20:15.726169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:29.746118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723416+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:20:15.723416+0000","last_peered":"2023-07-14T11:20:15.723416+0000","last_clean":"2023-07-14T11:20:15.723416+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:20:15.723416+0000","last_undegraded":"2023-07-14T11:20:15.723416+0000","last_fullsized":"2023-07-14T11:20:15.723416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:57.801423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725956+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:20:15.725956+0000","last_peered":"2023-07-14T11:20:15.725956+0000","last_clean":"2023-07-14T11:20:15.725956+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:20:15.725956+0000","last_undegraded":"2023-07-14T11:20:15.725956+0000","last_fullsized":"2023-07-14T11:20:15.725956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:01.673209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726027+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:20:15.726027+0000","last_peered":"2023-07-14T11:20:15.726027+0000","last_clean":"2023-07-14T11:20:15.726027+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:20:15.726027+0000","last_undegraded":"2023-07-14T11:20:15.726027+0000","last_fullsized":"2023-07-14T11:20:15.726027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:10.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723654+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:20:15.723654+0000","last_peered":"2023-07-14T11:20:15.723654+0000","last_clean":"2023-07-14T11:20:15.723654+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:20:15.723654+0000","last_undegraded":"2023-07-14T11:20:15.723654+0000","last_fullsized":"2023-07-14T11:20:15.723654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:53:36.294135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726533+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:20:15.726533+0000","last_peered":"2023-07-14T11:20:15.726533+0000","last_clean":"2023-07-14T11:20:15.726533+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:20:15.726533+0000","last_undegraded":"2023-07-14T11:20:15.726533+0000","last_fullsized":"2023-07-14T11:20:15.726533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:33.733562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763940+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:20:15.763940+0000","last_peered":"2023-07-14T11:20:15.763940+0000","last_clean":"2023-07-14T11:20:15.763940+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:20:15.763940+0000","last_undegraded":"2023-07-14T11:20:15.763940+0000","last_fullsized":"2023-07-14T11:20:15.763940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:28:49.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725800+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:20:15.725800+0000","last_peered":"2023-07-14T11:20:15.725800+0000","last_clean":"2023-07-14T11:20:15.725800+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:20:15.725800+0000","last_undegraded":"2023-07-14T11:20:15.725800+0000","last_fullsized":"2023-07-14T11:20:15.725800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:30.183181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726357+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:20:15.726357+0000","last_peered":"2023-07-14T11:20:15.726357+0000","last_clean":"2023-07-14T11:20:15.726357+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:20:15.726357+0000","last_undegraded":"2023-07-14T11:20:15.726357+0000","last_fullsized":"2023-07-14T11:20:15.726357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:13.424624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763774+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:20:15.763774+0000","last_peered":"2023-07-14T11:20:15.763774+0000","last_clean":"2023-07-14T11:20:15.763774+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:20:15.763774+0000","last_undegraded":"2023-07-14T11:20:15.763774+0000","last_fullsized":"2023-07-14T11:20:15.763774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:54.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762089+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:20:15.762089+0000","last_peered":"2023-07-14T11:20:15.762089+0000","last_clean":"2023-07-14T11:20:15.762089+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:20:15.762089+0000","last_undegraded":"2023-07-14T11:20:15.762089+0000","last_fullsized":"2023-07-14T11:20:15.762089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:06.259730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724677+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:20:15.724677+0000","last_peered":"2023-07-14T11:20:15.724677+0000","last_clean":"2023-07-14T11:20:15.724677+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:20:15.724677+0000","last_undegraded":"2023-07-14T11:20:15.724677+0000","last_fullsized":"2023-07-14T11:20:15.724677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:07.076184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762219+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:20:15.762219+0000","last_peered":"2023-07-14T11:20:15.762219+0000","last_clean":"2023-07-14T11:20:15.762219+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:20:15.762219+0000","last_undegraded":"2023-07-14T11:20:15.762219+0000","last_fullsized":"2023-07-14T11:20:15.762219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:30.071523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726815+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:20:15.726815+0000","last_peered":"2023-07-14T11:20:15.726815+0000","last_clean":"2023-07-14T11:20:15.726815+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:20:15.726815+0000","last_undegraded":"2023-07-14T11:20:15.726815+0000","last_fullsized":"2023-07-14T11:20:15.726815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:41:41.007835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724865+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:20:15.724865+0000","last_peered":"2023-07-14T11:20:15.724865+0000","last_clean":"2023-07-14T11:20:15.724865+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:20:15.724865+0000","last_undegraded":"2023-07-14T11:20:15.724865+0000","last_fullsized":"2023-07-14T11:20:15.724865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:56:28.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729360+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.729360+0000","last_peered":"2023-07-14T11:20:15.729360+0000","last_clean":"2023-07-14T11:20:15.729360+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:20:15.729360+0000","last_undegraded":"2023-07-14T11:20:15.729360+0000","last_fullsized":"2023-07-14T11:20:15.729360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:54.894820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724215+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:20:15.724215+0000","last_peered":"2023-07-14T11:20:15.724215+0000","last_clean":"2023-07-14T11:20:15.724215+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:20:15.724215+0000","last_undegraded":"2023-07-14T11:20:15.724215+0000","last_fullsized":"2023-07-14T11:20:15.724215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:16.677928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726665+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:20:15.726665+0000","last_peered":"2023-07-14T11:20:15.726665+0000","last_clean":"2023-07-14T11:20:15.726665+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:20:15.726665+0000","last_undegraded":"2023-07-14T11:20:15.726665+0000","last_fullsized":"2023-07-14T11:20:15.726665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:39.791849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728688+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:20:15.728688+0000","last_peered":"2023-07-14T11:20:15.728688+0000","last_clean":"2023-07-14T11:20:15.728688+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:20:15.728688+0000","last_undegraded":"2023-07-14T11:20:15.728688+0000","last_fullsized":"2023-07-14T11:20:15.728688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:23:54.582861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759435+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:20:15.759435+0000","last_peered":"2023-07-14T11:20:15.759435+0000","last_clean":"2023-07-14T11:20:15.759435+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:20:15.759435+0000","last_undegraded":"2023-07-14T11:20:15.759435+0000","last_fullsized":"2023-07-14T11:20:15.759435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:51.343442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725567+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:20:15.725567+0000","last_peered":"2023-07-14T11:20:15.725567+0000","last_clean":"2023-07-14T11:20:15.725567+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:20:15.725567+0000","last_undegraded":"2023-07-14T11:20:15.725567+0000","last_fullsized":"2023-07-14T11:20:15.725567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:50.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:06.761128+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:21:06.761128+0000","last_peered":"2023-07-14T11:21:06.761128+0000","last_clean":"2023-07-14T11:21:06.761128+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:21:06.761128+0000","last_undegraded":"2023-07-14T11:21:06.761128+0000","last_fullsized":"2023-07-14T11:21:06.761128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:41.232654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728839+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:20:15.728839+0000","last_peered":"2023-07-14T11:20:15.728839+0000","last_clean":"2023-07-14T11:20:15.728839+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:20:15.728839+0000","last_undegraded":"2023-07-14T11:20:15.728839+0000","last_fullsized":"2023-07-14T11:20:15.728839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:12.092163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766231+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:20:15.766231+0000","last_peered":"2023-07-14T11:20:15.766231+0000","last_clean":"2023-07-14T11:20:15.766231+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:20:15.766231+0000","last_undegraded":"2023-07-14T11:20:15.766231+0000","last_fullsized":"2023-07-14T11:20:15.766231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:16:19.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.730104+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:20:15.730104+0000","last_peered":"2023-07-14T11:20:15.730104+0000","last_clean":"2023-07-14T11:20:15.730104+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:20:15.730104+0000","last_undegraded":"2023-07-14T11:20:15.730104+0000","last_fullsized":"2023-07-14T11:20:15.730104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:54:17.106221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759228+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:20:15.759228+0000","last_peered":"2023-07-14T11:20:15.759228+0000","last_clean":"2023-07-14T11:20:15.759228+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:20:15.759228+0000","last_undegraded":"2023-07-14T11:20:15.759228+0000","last_fullsized":"2023-07-14T11:20:15.759228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:47:47.650473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726249+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:20:15.726249+0000","last_peered":"2023-07-14T11:20:15.726249+0000","last_clean":"2023-07-14T11:20:15.726249+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:20:15.726249+0000","last_undegraded":"2023-07-14T11:20:15.726249+0000","last_fullsized":"2023-07-14T11:20:15.726249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:10.265432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722943+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:20:15.722943+0000","last_peered":"2023-07-14T11:20:15.722943+0000","last_clean":"2023-07-14T11:20:15.722943+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:20:15.722943+0000","last_undegraded":"2023-07-14T11:20:15.722943+0000","last_fullsized":"2023-07-14T11:20:15.722943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:48.217979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761668+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:20:15.761668+0000","last_peered":"2023-07-14T11:20:15.761668+0000","last_clean":"2023-07-14T11:20:15.761668+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:20:15.761668+0000","last_undegraded":"2023-07-14T11:20:15.761668+0000","last_fullsized":"2023-07-14T11:20:15.761668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.096155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726651+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:20:15.726651+0000","last_peered":"2023-07-14T11:20:15.726651+0000","last_clean":"2023-07-14T11:20:15.726651+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:20:15.726651+0000","last_undegraded":"2023-07-14T11:20:15.726651+0000","last_fullsized":"2023-07-14T11:20:15.726651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:36.842168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758847+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:20:15.758847+0000","last_peered":"2023-07-14T11:20:15.758847+0000","last_clean":"2023-07-14T11:20:15.758847+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:20:15.758847+0000","last_undegraded":"2023-07-14T11:20:15.758847+0000","last_fullsized":"2023-07-14T11:20:15.758847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:38.403836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763190+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:20:15.763190+0000","last_peered":"2023-07-14T11:20:15.763190+0000","last_clean":"2023-07-14T11:20:15.763190+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:20:15.763190+0000","last_undegraded":"2023-07-14T11:20:15.763190+0000","last_fullsized":"2023-07-14T11:20:15.763190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:23.015078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761792+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:20:15.761792+0000","last_peered":"2023-07-14T11:20:15.761792+0000","last_clean":"2023-07-14T11:20:15.761792+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:20:15.761792+0000","last_undegraded":"2023-07-14T11:20:15.761792+0000","last_fullsized":"2023-07-14T11:20:15.761792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.645359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:05.758700+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:21:05.758700+0000","last_peered":"2023-07-14T11:21:05.758700+0000","last_clean":"2023-07-14T11:21:05.758700+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:21:05.758700+0000","last_undegraded":"2023-07-14T11:21:05.758700+0000","last_fullsized":"2023-07-14T11:21:05.758700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.137485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725168+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:20:15.725168+0000","last_peered":"2023-07-14T11:20:15.725168+0000","last_clean":"2023-07-14T11:20:15.725168+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:20:15.725168+0000","last_undegraded":"2023-07-14T11:20:15.725168+0000","last_fullsized":"2023-07-14T11:20:15.725168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:27:02.322268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763325+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:20:15.763325+0000","last_peered":"2023-07-14T11:20:15.763325+0000","last_clean":"2023-07-14T11:20:15.763325+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:20:15.763325+0000","last_undegraded":"2023-07-14T11:20:15.763325+0000","last_fullsized":"2023-07-14T11:20:15.763325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:28.186030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726329+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:20:15.726329+0000","last_peered":"2023-07-14T11:20:15.726329+0000","last_clean":"2023-07-14T11:20:15.726329+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:20:15.726329+0000","last_undegraded":"2023-07-14T11:20:15.726329+0000","last_fullsized":"2023-07-14T11:20:15.726329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:38.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726931+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:20:15.726931+0000","last_peered":"2023-07-14T11:20:15.726931+0000","last_clean":"2023-07-14T11:20:15.726931+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:20:15.726931+0000","last_undegraded":"2023-07-14T11:20:15.726931+0000","last_fullsized":"2023-07-14T11:20:15.726931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:58.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725362+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:20:15.725362+0000","last_peered":"2023-07-14T11:20:15.725362+0000","last_clean":"2023-07-14T11:20:15.725362+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:20:15.725362+0000","last_undegraded":"2023-07-14T11:20:15.725362+0000","last_fullsized":"2023-07-14T11:20:15.725362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:29.343315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759436+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:20:15.759436+0000","last_peered":"2023-07-14T11:20:15.759436+0000","last_clean":"2023-07-14T11:20:15.759436+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:20:15.759436+0000","last_undegraded":"2023-07-14T11:20:15.759436+0000","last_fullsized":"2023-07-14T11:20:15.759436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:17:07.579078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724381+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:20:15.724381+0000","last_peered":"2023-07-14T11:20:15.724381+0000","last_clean":"2023-07-14T11:20:15.724381+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:20:15.724381+0000","last_undegraded":"2023-07-14T11:20:15.724381+0000","last_fullsized":"2023-07-14T11:20:15.724381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:16:27.614204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762929+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:20:15.762929+0000","last_peered":"2023-07-14T11:20:15.762929+0000","last_clean":"2023-07-14T11:20:15.762929+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:20:15.762929+0000","last_undegraded":"2023-07-14T11:20:15.762929+0000","last_fullsized":"2023-07-14T11:20:15.762929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:41.993201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":173,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:06.760309+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:21:06.760309+0000","last_peered":"2023-07-14T11:21:06.760309+0000","last_clean":"2023-07-14T11:21:06.760309+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:21:06.760309+0000","last_undegraded":"2023-07-14T11:21:06.760309+0000","last_fullsized":"2023-07-14T11:21:06.760309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:08:09.309573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56970,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":117,"num_read_kb":114,"num_write":31,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728912+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:20:15.728912+0000","last_peered":"2023-07-14T11:20:15.728912+0000","last_clean":"2023-07-14T11:20:15.728912+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:20:15.728912+0000","last_undegraded":"2023-07-14T11:20:15.728912+0000","last_fullsized":"2023-07-14T11:20:15.728912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:40:20.286989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759883+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:20:15.759883+0000","last_peered":"2023-07-14T11:20:15.759883+0000","last_clean":"2023-07-14T11:20:15.759883+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:20:15.759883+0000","last_undegraded":"2023-07-14T11:20:15.759883+0000","last_fullsized":"2023-07-14T11:20:15.759883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:27.460710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726089+0000","last_change":"2023-07-14T11:19:41.114085+0000","last_active":"2023-07-14T11:20:15.726089+0000","last_peered":"2023-07-14T11:20:15.726089+0000","last_clean":"2023-07-14T11:20:15.726089+0000","last_became_active":"2023-07-14T11:19:41.113633+0000","last_became_peered":"2023-07-14T11:19:41.113633+0000","last_unstale":"2023-07-14T11:20:15.726089+0000","last_undegraded":"2023-07-14T11:20:15.726089+0000","last_fullsized":"2023-07-14T11:20:15.726089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:33:58.333180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722891+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:20:15.722891+0000","last_peered":"2023-07-14T11:20:15.722891+0000","last_clean":"2023-07-14T11:20:15.722891+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:20:15.722891+0000","last_undegraded":"2023-07-14T11:20:15.722891+0000","last_fullsized":"2023-07-14T11:20:15.722891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:12.630498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.732150+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:20:15.732150+0000","last_peered":"2023-07-14T11:20:15.732150+0000","last_clean":"2023-07-14T11:20:15.732150+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:20:15.732150+0000","last_undegraded":"2023-07-14T11:20:15.732150+0000","last_fullsized":"2023-07-14T11:20:15.732150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:49.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759053+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:20:15.759053+0000","last_peered":"2023-07-14T11:20:15.759053+0000","last_clean":"2023-07-14T11:20:15.759053+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:20:15.759053+0000","last_undegraded":"2023-07-14T11:20:15.759053+0000","last_fullsized":"2023-07-14T11:20:15.759053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:20:45.016996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765761+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:20:15.765761+0000","last_peered":"2023-07-14T11:20:15.765761+0000","last_clean":"2023-07-14T11:20:15.765761+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:20:15.765761+0000","last_undegraded":"2023-07-14T11:20:15.765761+0000","last_fullsized":"2023-07-14T11:20:15.765761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:51:11.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763313+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:20:15.763313+0000","last_peered":"2023-07-14T11:20:15.763313+0000","last_clean":"2023-07-14T11:20:15.763313+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:20:15.763313+0000","last_undegraded":"2023-07-14T11:20:15.763313+0000","last_fullsized":"2023-07-14T11:20:15.763313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:41.779784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":86,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:05.758868+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:21:05.758868+0000","last_peered":"2023-07-14T11:21:05.758868+0000","last_clean":"2023-07-14T11:21:05.758868+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:21:05.758868+0000","last_undegraded":"2023-07-14T11:21:05.758868+0000","last_fullsized":"2023-07-14T11:21:05.758868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:35.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726189+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:20:15.726189+0000","last_peered":"2023-07-14T11:20:15.726189+0000","last_clean":"2023-07-14T11:20:15.726189+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:20:15.726189+0000","last_undegraded":"2023-07-14T11:20:15.726189+0000","last_fullsized":"2023-07-14T11:20:15.726189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:25.466461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723291+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:20:15.723291+0000","last_peered":"2023-07-14T11:20:15.723291+0000","last_clean":"2023-07-14T11:20:15.723291+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:20:15.723291+0000","last_undegraded":"2023-07-14T11:20:15.723291+0000","last_fullsized":"2023-07-14T11:20:15.723291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:05.698741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728325+0000","last_change":"2023-07-14T11:19:41.112192+0000","last_active":"2023-07-14T11:20:15.728325+0000","last_peered":"2023-07-14T11:20:15.728325+0000","last_clean":"2023-07-14T11:20:15.728325+0000","last_became_active":"2023-07-14T11:19:41.111882+0000","last_became_peered":"2023-07-14T11:19:41.111882+0000","last_unstale":"2023-07-14T11:20:15.728325+0000","last_undegraded":"2023-07-14T11:20:15.728325+0000","last_fullsized":"2023-07-14T11:20:15.728325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:03:30.273008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758707+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:20:15.758707+0000","last_peered":"2023-07-14T11:20:15.758707+0000","last_clean":"2023-07-14T11:20:15.758707+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:20:15.758707+0000","last_undegraded":"2023-07-14T11:20:15.758707+0000","last_fullsized":"2023-07-14T11:20:15.758707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:26:32.059475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758890+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:20:15.758890+0000","last_peered":"2023-07-14T11:20:15.758890+0000","last_clean":"2023-07-14T11:20:15.758890+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:20:15.758890+0000","last_undegraded":"2023-07-14T11:20:15.758890+0000","last_fullsized":"2023-07-14T11:20:15.758890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:51.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729124+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:20:15.729124+0000","last_peered":"2023-07-14T11:20:15.729124+0000","last_clean":"2023-07-14T11:20:15.729124+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:20:15.729124+0000","last_undegraded":"2023-07-14T11:20:15.729124+0000","last_fullsized":"2023-07-14T11:20:15.729124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:06.994422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725218+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:20:15.725218+0000","last_peered":"2023-07-14T11:20:15.725218+0000","last_clean":"2023-07-14T11:20:15.725218+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:20:15.725218+0000","last_undegraded":"2023-07-14T11:20:15.725218+0000","last_fullsized":"2023-07-14T11:20:15.725218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:42.841223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.731601+0000","last_change":"2023-07-14T11:20:15.731601+0000","last_active":"2023-07-14T11:20:15.731601+0000","last_peered":"2023-07-14T11:20:15.731601+0000","last_clean":"2023-07-14T11:20:15.731601+0000","last_became_active":"2023-07-14T11:20:15.731121+0000","last_became_peered":"2023-07-14T11:20:15.731121+0000","last_unstale":"2023-07-14T11:20:15.731601+0000","last_undegraded":"2023-07-14T11:20:15.731601+0000","last_fullsized":"2023-07-14T11:20:15.731601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:58.090267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764473+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:20:15.764473+0000","last_peered":"2023-07-14T11:20:15.764473+0000","last_clean":"2023-07-14T11:20:15.764473+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:20:15.764473+0000","last_undegraded":"2023-07-14T11:20:15.764473+0000","last_fullsized":"2023-07-14T11:20:15.764473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:06.410233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729586+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:20:15.729586+0000","last_peered":"2023-07-14T11:20:15.729586+0000","last_clean":"2023-07-14T11:20:15.729586+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:20:15.729586+0000","last_undegraded":"2023-07-14T11:20:15.729586+0000","last_fullsized":"2023-07-14T11:20:15.729586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:59.409139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723225+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:20:15.723225+0000","last_peered":"2023-07-14T11:20:15.723225+0000","last_clean":"2023-07-14T11:20:15.723225+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:20:15.723225+0000","last_undegraded":"2023-07-14T11:20:15.723225+0000","last_fullsized":"2023-07-14T11:20:15.723225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:32.064021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726984+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:20:15.726984+0000","last_peered":"2023-07-14T11:20:15.726984+0000","last_clean":"2023-07-14T11:20:15.726984+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:20:15.726984+0000","last_undegraded":"2023-07-14T11:20:15.726984+0000","last_fullsized":"2023-07-14T11:20:15.726984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:38:06.530216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765912+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:20:15.765912+0000","last_peered":"2023-07-14T11:20:15.765912+0000","last_clean":"2023-07-14T11:20:15.765912+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:20:15.765912+0000","last_undegraded":"2023-07-14T11:20:15.765912+0000","last_fullsized":"2023-07-14T11:20:15.765912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:01:52.067140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723192+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:20:15.723192+0000","last_peered":"2023-07-14T11:20:15.723192+0000","last_clean":"2023-07-14T11:20:15.723192+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:20:15.723192+0000","last_undegraded":"2023-07-14T11:20:15.723192+0000","last_fullsized":"2023-07-14T11:20:15.723192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:44:48.912804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759017+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:20:15.759017+0000","last_peered":"2023-07-14T11:20:15.759017+0000","last_clean":"2023-07-14T11:20:15.759017+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:20:15.759017+0000","last_undegraded":"2023-07-14T11:20:15.759017+0000","last_fullsized":"2023-07-14T11:20:15.759017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:30:26.429533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:21:08.762993+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:21:08.762993+0000","last_peered":"2023-07-14T11:21:08.762993+0000","last_clean":"2023-07-14T11:21:08.762993+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:21:08.762993+0000","last_undegraded":"2023-07-14T11:21:08.762993+0000","last_fullsized":"2023-07-14T11:21:08.762993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:54:46.013281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726387+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:20:15.726387+0000","last_peered":"2023-07-14T11:20:15.726387+0000","last_clean":"2023-07-14T11:20:15.726387+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:20:15.726387+0000","last_undegraded":"2023-07-14T11:20:15.726387+0000","last_fullsized":"2023-07-14T11:20:15.726387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:45.551283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761576+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:20:15.761576+0000","last_peered":"2023-07-14T11:20:15.761576+0000","last_clean":"2023-07-14T11:20:15.761576+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:20:15.761576+0000","last_undegraded":"2023-07-14T11:20:15.761576+0000","last_fullsized":"2023-07-14T11:20:15.761576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:11:35.428242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725019+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:20:15.725019+0000","last_peered":"2023-07-14T11:20:15.725019+0000","last_clean":"2023-07-14T11:20:15.725019+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:20:15.725019+0000","last_undegraded":"2023-07-14T11:20:15.725019+0000","last_fullsized":"2023-07-14T11:20:15.725019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:42:08.666520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725963+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:20:15.725963+0000","last_peered":"2023-07-14T11:20:15.725963+0000","last_clean":"2023-07-14T11:20:15.725963+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:20:15.725963+0000","last_undegraded":"2023-07-14T11:20:15.725963+0000","last_fullsized":"2023-07-14T11:20:15.725963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:49.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759565+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:20:15.759565+0000","last_peered":"2023-07-14T11:20:15.759565+0000","last_clean":"2023-07-14T11:20:15.759565+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:20:15.759565+0000","last_undegraded":"2023-07-14T11:20:15.759565+0000","last_fullsized":"2023-07-14T11:20:15.759565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:31.340558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.722788+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:20:15.722788+0000","last_peered":"2023-07-14T11:20:15.722788+0000","last_clean":"2023-07-14T11:20:15.722788+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:20:15.722788+0000","last_undegraded":"2023-07-14T11:20:15.722788+0000","last_fullsized":"2023-07-14T11:20:15.722788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:40:45.755906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726149+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:20:15.726149+0000","last_peered":"2023-07-14T11:20:15.726149+0000","last_clean":"2023-07-14T11:20:15.726149+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:20:15.726149+0000","last_undegraded":"2023-07-14T11:20:15.726149+0000","last_fullsized":"2023-07-14T11:20:15.726149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:05:29.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725637+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:20:15.725637+0000","last_peered":"2023-07-14T11:20:15.725637+0000","last_clean":"2023-07-14T11:20:15.725637+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:20:15.725637+0000","last_undegraded":"2023-07-14T11:20:15.725637+0000","last_fullsized":"2023-07-14T11:20:15.725637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:47:19.355328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729053+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:20:15.729053+0000","last_peered":"2023-07-14T11:20:15.729053+0000","last_clean":"2023-07-14T11:20:15.729053+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:20:15.729053+0000","last_undegraded":"2023-07-14T11:20:15.729053+0000","last_fullsized":"2023-07-14T11:20:15.729053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:06:22.336930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723369+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:20:15.723369+0000","last_peered":"2023-07-14T11:20:15.723369+0000","last_clean":"2023-07-14T11:20:15.723369+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:20:15.723369+0000","last_undegraded":"2023-07-14T11:20:15.723369+0000","last_fullsized":"2023-07-14T11:20:15.723369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:47.388288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761518+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:20:15.761518+0000","last_peered":"2023-07-14T11:20:15.761518+0000","last_clean":"2023-07-14T11:20:15.761518+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:20:15.761518+0000","last_undegraded":"2023-07-14T11:20:15.761518+0000","last_fullsized":"2023-07-14T11:20:15.761518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:28.833104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.764010+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:20:15.764010+0000","last_peered":"2023-07-14T11:20:15.764010+0000","last_clean":"2023-07-14T11:20:15.764010+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:20:15.764010+0000","last_undegraded":"2023-07-14T11:20:15.764010+0000","last_fullsized":"2023-07-14T11:20:15.764010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:12:18.046365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762296+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:20:15.762296+0000","last_peered":"2023-07-14T11:20:15.762296+0000","last_clean":"2023-07-14T11:20:15.762296+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:20:15.762296+0000","last_undegraded":"2023-07-14T11:20:15.762296+0000","last_fullsized":"2023-07-14T11:20:15.762296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:04:14.446719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725262+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:20:15.725262+0000","last_peered":"2023-07-14T11:20:15.725262+0000","last_clean":"2023-07-14T11:20:15.725262+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:20:15.725262+0000","last_undegraded":"2023-07-14T11:20:15.725262+0000","last_fullsized":"2023-07-14T11:20:15.725262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:40:44.958728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725446+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:20:15.725446+0000","last_peered":"2023-07-14T11:20:15.725446+0000","last_clean":"2023-07-14T11:20:15.725446+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:20:15.725446+0000","last_undegraded":"2023-07-14T11:20:15.725446+0000","last_fullsized":"2023-07-14T11:20:15.725446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.980346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729830+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:20:15.729830+0000","last_peered":"2023-07-14T11:20:15.729830+0000","last_clean":"2023-07-14T11:20:15.729830+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:20:15.729830+0000","last_undegraded":"2023-07-14T11:20:15.729830+0000","last_fullsized":"2023-07-14T11:20:15.729830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:23.746186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724337+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:20:15.724337+0000","last_peered":"2023-07-14T11:20:15.724337+0000","last_clean":"2023-07-14T11:20:15.724337+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:20:15.724337+0000","last_undegraded":"2023-07-14T11:20:15.724337+0000","last_fullsized":"2023-07-14T11:20:15.724337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:32.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723475+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:20:15.723475+0000","last_peered":"2023-07-14T11:20:15.723475+0000","last_clean":"2023-07-14T11:20:15.723475+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:20:15.723475+0000","last_undegraded":"2023-07-14T11:20:15.723475+0000","last_fullsized":"2023-07-14T11:20:15.723475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:58.127654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728279+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:20:15.728279+0000","last_peered":"2023-07-14T11:20:15.728279+0000","last_clean":"2023-07-14T11:20:15.728279+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:20:15.728279+0000","last_undegraded":"2023-07-14T11:20:15.728279+0000","last_fullsized":"2023-07-14T11:20:15.728279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:00:32.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.763590+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:20:15.763590+0000","last_peered":"2023-07-14T11:20:15.763590+0000","last_clean":"2023-07-14T11:20:15.763590+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:20:15.763590+0000","last_undegraded":"2023-07-14T11:20:15.763590+0000","last_fullsized":"2023-07-14T11:20:15.763590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:06:43.200171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761544+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:20:15.761544+0000","last_peered":"2023-07-14T11:20:15.761544+0000","last_clean":"2023-07-14T11:20:15.761544+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:20:15.761544+0000","last_undegraded":"2023-07-14T11:20:15.761544+0000","last_fullsized":"2023-07-14T11:20:15.761544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:10:23.505930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:25.389747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725984+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:20:15.725984+0000","last_peered":"2023-07-14T11:20:15.725984+0000","last_clean":"2023-07-14T11:20:15.725984+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:20:15.725984+0000","last_undegraded":"2023-07-14T11:20:15.725984+0000","last_fullsized":"2023-07-14T11:20:15.725984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:28.285910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725493+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:20:15.725493+0000","last_peered":"2023-07-14T11:20:15.725493+0000","last_clean":"2023-07-14T11:20:15.725493+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:20:15.725493+0000","last_undegraded":"2023-07-14T11:20:15.725493+0000","last_fullsized":"2023-07-14T11:20:15.725493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:49.791341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761896+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:20:15.761896+0000","last_peered":"2023-07-14T11:20:15.761896+0000","last_clean":"2023-07-14T11:20:15.761896+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:20:15.761896+0000","last_undegraded":"2023-07-14T11:20:15.761896+0000","last_fullsized":"2023-07-14T11:20:15.761896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:04:06.442323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724354+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:20:15.724354+0000","last_peered":"2023-07-14T11:20:15.724354+0000","last_clean":"2023-07-14T11:20:15.724354+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:20:15.724354+0000","last_undegraded":"2023-07-14T11:20:15.724354+0000","last_fullsized":"2023-07-14T11:20:15.724354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:45:31.530941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.728516+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:20:15.728516+0000","last_peered":"2023-07-14T11:20:15.728516+0000","last_clean":"2023-07-14T11:20:15.728516+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:20:15.728516+0000","last_undegraded":"2023-07-14T11:20:15.728516+0000","last_fullsized":"2023-07-14T11:20:15.728516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:29.709167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727131+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:20:15.727131+0000","last_peered":"2023-07-14T11:20:15.727131+0000","last_clean":"2023-07-14T11:20:15.727131+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:20:15.727131+0000","last_undegraded":"2023-07-14T11:20:15.727131+0000","last_fullsized":"2023-07-14T11:20:15.727131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:31:54.387366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723797+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:20:15.723797+0000","last_peered":"2023-07-14T11:20:15.723797+0000","last_clean":"2023-07-14T11:20:15.723797+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:20:15.723797+0000","last_undegraded":"2023-07-14T11:20:15.723797+0000","last_fullsized":"2023-07-14T11:20:15.723797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:50:00.530059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.727120+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:20:15.727120+0000","last_peered":"2023-07-14T11:20:15.727120+0000","last_clean":"2023-07-14T11:20:15.727120+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:20:15.727120+0000","last_undegraded":"2023-07-14T11:20:15.727120+0000","last_fullsized":"2023-07-14T11:20:15.727120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:48:15.525008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726470+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:20:15.726470+0000","last_peered":"2023-07-14T11:20:15.726470+0000","last_clean":"2023-07-14T11:20:15.726470+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:20:15.726470+0000","last_undegraded":"2023-07-14T11:20:15.726470+0000","last_fullsized":"2023-07-14T11:20:15.726470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:03.965708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.729884+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:20:15.729884+0000","last_peered":"2023-07-14T11:20:15.729884+0000","last_clean":"2023-07-14T11:20:15.729884+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:20:15.729884+0000","last_undegraded":"2023-07-14T11:20:15.729884+0000","last_fullsized":"2023-07-14T11:20:15.729884+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:31.118854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.759707+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:20:15.759707+0000","last_peered":"2023-07-14T11:20:15.759707+0000","last_clean":"2023-07-14T11:20:15.759707+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:20:15.759707+0000","last_undegraded":"2023-07-14T11:20:15.759707+0000","last_fullsized":"2023-07-14T11:20:15.759707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:35.715875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.766177+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:20:15.766177+0000","last_peered":"2023-07-14T11:20:15.766177+0000","last_clean":"2023-07-14T11:20:15.766177+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:20:15.766177+0000","last_undegraded":"2023-07-14T11:20:15.766177+0000","last_fullsized":"2023-07-14T11:20:15.766177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:59.517623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.758537+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:20:15.758537+0000","last_peered":"2023-07-14T11:20:15.758537+0000","last_clean":"2023-07-14T11:20:15.758537+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:20:15.758537+0000","last_undegraded":"2023-07-14T11:20:15.758537+0000","last_fullsized":"2023-07-14T11:20:15.758537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:56.641650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723412+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:20:15.723412+0000","last_peered":"2023-07-14T11:20:15.723412+0000","last_clean":"2023-07-14T11:20:15.723412+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:20:15.723412+0000","last_undegraded":"2023-07-14T11:20:15.723412+0000","last_fullsized":"2023-07-14T11:20:15.723412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:35.428375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726258+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:20:15.726258+0000","last_peered":"2023-07-14T11:20:15.726258+0000","last_clean":"2023-07-14T11:20:15.726258+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:20:15.726258+0000","last_undegraded":"2023-07-14T11:20:15.726258+0000","last_fullsized":"2023-07-14T11:20:15.726258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:41.949605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725148+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:20:15.725148+0000","last_peered":"2023-07-14T11:20:15.725148+0000","last_clean":"2023-07-14T11:20:15.725148+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:20:15.725148+0000","last_undegraded":"2023-07-14T11:20:15.725148+0000","last_fullsized":"2023-07-14T11:20:15.725148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:48.091011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725591+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:20:15.725591+0000","last_peered":"2023-07-14T11:20:15.725591+0000","last_clean":"2023-07-14T11:20:15.725591+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:20:15.725591+0000","last_undegraded":"2023-07-14T11:20:15.725591+0000","last_fullsized":"2023-07-14T11:20:15.725591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:15.490002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726260+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:20:15.726260+0000","last_peered":"2023-07-14T11:20:15.726260+0000","last_clean":"2023-07-14T11:20:15.726260+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:20:15.726260+0000","last_undegraded":"2023-07-14T11:20:15.726260+0000","last_fullsized":"2023-07-14T11:20:15.726260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:53:08.815873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726407+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:20:15.726407+0000","last_peered":"2023-07-14T11:20:15.726407+0000","last_clean":"2023-07-14T11:20:15.726407+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:20:15.726407+0000","last_undegraded":"2023-07-14T11:20:15.726407+0000","last_fullsized":"2023-07-14T11:20:15.726407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:39:08.673713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723496+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:20:15.723496+0000","last_peered":"2023-07-14T11:20:15.723496+0000","last_clean":"2023-07-14T11:20:15.723496+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:20:15.723496+0000","last_undegraded":"2023-07-14T11:20:15.723496+0000","last_fullsized":"2023-07-14T11:20:15.723496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:41:54.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.726094+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:20:15.726094+0000","last_peered":"2023-07-14T11:20:15.726094+0000","last_clean":"2023-07-14T11:20:15.726094+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:20:15.726094+0000","last_undegraded":"2023-07-14T11:20:15.726094+0000","last_fullsized":"2023-07-14T11:20:15.726094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:36:46.120312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725900+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:20:15.725900+0000","last_peered":"2023-07-14T11:20:15.725900+0000","last_clean":"2023-07-14T11:20:15.725900+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:20:15.725900+0000","last_undegraded":"2023-07-14T11:20:15.725900+0000","last_fullsized":"2023-07-14T11:20:15.725900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:29.569291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.762498+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:20:15.762498+0000","last_peered":"2023-07-14T11:20:15.762498+0000","last_clean":"2023-07-14T11:20:15.762498+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:20:15.762498+0000","last_undegraded":"2023-07-14T11:20:15.762498+0000","last_fullsized":"2023-07-14T11:20:15.762498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:34:46.020055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.765762+0000","last_change":"2023-07-14T11:20:15.765762+0000","last_active":"2023-07-14T11:20:15.765762+0000","last_peered":"2023-07-14T11:20:15.765762+0000","last_clean":"2023-07-14T11:20:15.765762+0000","last_became_active":"2023-07-14T11:20:15.765349+0000","last_became_peered":"2023-07-14T11:20:15.765349+0000","last_unstale":"2023-07-14T11:20:15.765762+0000","last_undegraded":"2023-07-14T11:20:15.765762+0000","last_fullsized":"2023-07-14T11:20:15.765762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:51:54.292936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.761928+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:20:15.761928+0000","last_peered":"2023-07-14T11:20:15.761928+0000","last_clean":"2023-07-14T11:20:15.761928+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:20:15.761928+0000","last_undegraded":"2023-07-14T11:20:15.761928+0000","last_fullsized":"2023-07-14T11:20:15.761928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:50:15.332355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.724152+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:20:15.724152+0000","last_peered":"2023-07-14T11:20:15.724152+0000","last_clean":"2023-07-14T11:20:15.724152+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:20:15.724152+0000","last_undegraded":"2023-07-14T11:20:15.724152+0000","last_fullsized":"2023-07-14T11:20:15.724152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:17.416943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.723311+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:20:15.723311+0000","last_peered":"2023-07-14T11:20:15.723311+0000","last_clean":"2023-07-14T11:20:15.723311+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:20:15.723311+0000","last_undegraded":"2023-07-14T11:20:15.723311+0000","last_fullsized":"2023-07-14T11:20:15.723311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:13.493038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T11:20:15.725897+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:20:15.725897+0000","last_peered":"2023-07-14T11:20:15.725897+0000","last_clean":"2023-07-14T11:20:15.725897+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:20:15.725897+0000","last_undegraded":"2023-07-14T11:20:15.725897+0000","last_fullsized":"2023-07-14T11:20:15.725897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:20.934793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60484,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":359,"num_read_kb":245,"num_write":151,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":181452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":490869,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82899999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.753},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80700000000000005,"5min":1.097,"15min":1.097},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.30599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.62,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46400000000000002,"15min":0.45700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97899999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":1.042,"15min":1.042},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83099999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.41199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.873,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78300000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53200000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.999,"5min":1.159,"15min":1.159},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.85099999999999998,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.76600000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.67600000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.871,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.046,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.92300000000000004,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.92900000000000005,"5min":1.456,"15min":1.456},"last":0.83499999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1118208,"data_stored":1135059,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":5.9969999999999999,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":6.0090000000000003,"5min":6.0090000000000003,"15min":6.0090000000000003},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.753,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.68899999999999995,"15min":0.70299999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85199999999999998,"5min":2.46,"15min":2.46},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.752,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86599999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.79300000000000004,"5min":1.349,"15min":1.349},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74199999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.373,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.625}]},{"osd":5,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.78000000000000003,"15min":0.79700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81299999999999994,"5min":2.431,"15min":2.431},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.81899999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.80200000000000005,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88700000000000001,"5min":1.044,"15min":1.044},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95299999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.90200000000000002,"5min":1.01,"15min":1.01},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.88900000000000001,"15min":0.90900000000000003},"min":{"1min":0.64900000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.92300000000000004,"5min":2.536,"15min":2.536},"last":0.75900000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.85099999999999998,"15min":0.85899999999999999},"min":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97899999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.93700000000000006,"15min":0.96299999999999997},"min":{"1min":0.63500000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0009999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Jul 14 11:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86799999999999999,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95999999999999996,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.70299999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.054},"last":0.69099999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":495201,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":1.012,"15min":1.012},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.997,"15min":1.0489999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":0.999,"15min":0.999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0880000000000001,"5min":1.264,"15min":1.264},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.56499999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.999,"5min":1.073,"15min":1.073},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.84099999999999997,"15min":0.86299999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93100000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.872,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0409999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.64800000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.151,"15min":1.2090000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99299999999999999,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90100000000000002,"15min":0.91700000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.081,"5min":1.171,"15min":1.171},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.872},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.93100000000000005,"15min":0.94499999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1319999999999999,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.66100000000000003}]},{"osd":11,"last update":"Fri Jul 14 11:20:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.752},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.78000000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":491025,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.52400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.45500000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.48499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78500000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.748}]},{"osd":2,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.34499999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64700000000000002,"15min":0.67300000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.876,"5min":2.032,"15min":2.032},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.51400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.48199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.65200000000000002,"15min":0.628},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83599999999999997}]},{"osd":5,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.56499999999999995,"15min":0.54800000000000004},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.91100000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.58699999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.53500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.61299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":1.018}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.008}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.71599999999999997,"15min":0.69599999999999995},"min":{"1min":0.54900000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.0580000000000001}]},{"osd":11,"last update":"Fri Jul 14 11:20:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.995},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":1.032}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1086116,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"min":{"1min":0.32800000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.44900000000000001,"15min":0.44500000000000001},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65100000000000002,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79400000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.308,"15min":0.308},"max":{"1min":0.70799999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.873,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85999999999999999,"5min":1.149,"15min":1.149},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.80000000000000004,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.86599999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.72599999999999998,"15min":0.69299999999999995},"min":{"1min":0.52900000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68999999999999995,"15min":0.67300000000000004},"min":{"1min":0.53600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85199999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80300000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.131,"15min":1.131},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.995}]},{"osd":10,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.88400000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89100000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.70599999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.97699999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486008,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.46000000000000002},"min":{"1min":0.36899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":0.871},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40100000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64700000000000002},"min":{"1min":0.59399999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":1.016,"15min":1.016},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81799999999999995,"15min":0.876},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":1.258}]},{"osd":2,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0069999999999999,"5min":1.071,"15min":1.071},"last":1.3540000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.1000000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.52600000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.79900000000000004,"15min":0.77400000000000002},"min":{"1min":0.65600000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0880000000000001,"5min":1.115,"15min":1.115},"last":1.2769999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.83299999999999996,"15min":0.80900000000000005},"min":{"1min":0.61499999999999999,"5min":0.495,"15min":0.495},"max":{"1min":2.3069999999999999,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.81200000000000006,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.413,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.0620000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.034,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.80600000000000005}]},{"osd":10,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.70899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.4450000000000001,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.93100000000000005,"15min":0.92500000000000004},"min":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.3999999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:20:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.234},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.3029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":543161,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.57299999999999995},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":4.077}]},{"osd":2,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57299999999999995,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.59099999999999997,"15min":0.51700000000000002},"min":{"1min":0.52300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89300000000000002,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70199999999999996,"15min":0.64900000000000002},"min":{"1min":0.44700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":4.016,"5min":4.016,"15min":4.016},"last":4.016}]},{"osd":4,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.40999999999999998},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.497,"15min":0.46500000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66100000000000003,"15min":0.625},"min":{"1min":0.55600000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.51500000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":4.1219999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":4.1459999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88900000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84499999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71199999999999997,"15min":0.68400000000000005},"min":{"1min":0.56399999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97999999999999998,"5min":1.036,"15min":1.036},"last":0.97699999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.93500000000000005}]},{"osd":10,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96399999999999997,"5min":1.014,"15min":1.014},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.95199999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":1}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":487396,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.998,"15min":0.998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.437,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.66900000000000004,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.9169999999999998,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.65900000000000003,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.2060000000000004,"5min":5.2060000000000004,"15min":5.2060000000000004},"last":0.746}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93400000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.70099999999999996,"15min":0.63200000000000001},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.435},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.72699999999999998,"15min":0.66100000000000003},"min":{"1min":0.56299999999999994,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.2489999999999997,"5min":5.2489999999999997,"15min":5.2489999999999997},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0129999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1240000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.351,"5min":5.351,"15min":5.351},"last":0.92800000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.86799999999999999,"15min":0.871},"min":{"1min":0.61699999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":5.2839999999999998,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0029999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.68400000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.90200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.036,"5min":1.577,"15min":1.577},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.147,"5min":1.538,"15min":1.538},"last":0.91200000000000003}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1299999999999999,"5min":1.611,"15min":1.611},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0760000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.71699999999999997}]},{"osd":11,"last update":"Fri Jul 14 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.85399999999999998,"15min":0.81599999999999995},"min":{"1min":0.628,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.88600000000000001,"15min":0.84399999999999997},"min":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":0.85999999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":544190,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":1.012,"15min":1.012},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.746},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57899999999999996,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.46999999999999997,"15min":0.44800000000000001},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.439,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83499999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.93300000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.47199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65100000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Jul 14 11:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Jul 14 11:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72399999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486475,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.52800000000000002,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.4649999999999999,"5min":7.4649999999999999,"15min":7.4649999999999999},"last":0.5},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.54700000000000004,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":6.4809999999999999,"5min":6.4809999999999999,"15min":6.4809999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.71699999999999997,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.7329999999999997,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.73799999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":6.335,"5min":6.335,"15min":6.335},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60399999999999998,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.69999999999999996,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"last":0.77100000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67600000000000005,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.60599999999999998},"min":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89900000000000002,"5min":1.089,"15min":1.089},"last":0.90600000000000003}]},{"osd":5,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.75900000000000001,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.8570000000000002,"5min":6.8570000000000002,"15min":6.8570000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.52300000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.749}]},{"osd":6,"last update":"Fri Jul 14 11:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:20:31 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.58099999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.032,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.878},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.96799999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.59999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96699999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.98599999999999999,"15min":1.0169999999999999},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":4.157,"15min":4.157},"last":0.94999999999999996}]},{"osd":10,"last update":"Fri Jul 14 11:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.97199999999999998,"15min":1.0029999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1259999999999999,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.119,"5min":1.218,"15min":1.218},"last":0.93500000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":6.8120000000000003,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.85999999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490883,"data_compressed":17628,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.25,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56000000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.753},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.3090000000000002,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.624},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79100000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59499999999999997,"15min":0.56200000000000006},"min":{"1min":0.55700000000000005,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82499999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.49199999999999999,"15min":0.47699999999999998},"min":{"1min":0.35799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72099999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.52400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.755,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76900000000000002}]},{"osd":9,"last update":"Fri Jul 14 11:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80900000000000005}]},{"osd":10,"last update":"Fri Jul 14 11:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75700000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67800000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73899999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1081752,"data_compressed":53325,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":6.6680000000000001,"5min":6.6680000000000001,"15min":6.6680000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.53700000000000003,"15min":0.48799999999999999},"min":{"1min":0.35799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.877,"15min":0.877},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.75,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.38700000000000001,"15min":0.35799999999999998},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.66800000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.442},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.38800000000000001},"min":{"1min":0.33600000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.69699999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.61799999999999999,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":3.3759999999999999,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.626,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57999999999999996,"15min":0.54200000000000004},"min":{"1min":0.47699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81200000000000006,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.57299999999999995,"15min":0.53800000000000003},"min":{"1min":0.44500000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.53500000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.50800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.038,"15min":1.038},"last":0.52600000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85599999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81299999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48599999999999999}]},{"osd":9,"last update":"Fri Jul 14 11:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.751},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70299999999999996}]},{"osd":10,"last update":"Fri Jul 14 11:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85899999999999999,"5min":1.073,"15min":1.073},"last":0.80700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:21:11.944 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-14T11:21:11.945 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-14T11:21:11.945 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-14T11:21:11.945 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph health --format=json 2023-07-14T11:21:12.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:11 smithi175 ceph-mon[129081]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:12.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:11 smithi182 ceph-mon[129366]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:12.673 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:12.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:12 smithi161 ceph-mon[95526]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:21:13.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:12 smithi175 ceph-mon[129081]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:21:13.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:12 smithi182 ceph-mon[129366]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:21:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:13 smithi161 ceph-mon[95526]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:13 smithi175 ceph-mon[129081]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:13 smithi182 ceph-mon[129366]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:14.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.387+0000 7f3034472700 1 -- 172.21.15.161:0/381395559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0728b0 msgr2=0x7f302c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:14.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.387+0000 7f3034472700 1 --2- 172.21.15.161:0/381395559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0728b0 0x7f302c072c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3028009990 tx=0x7f3028009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.388+0000 7f3034472700 1 -- 172.21.15.161:0/381395559 shutdown_connections 2023-07-14T11:21:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.388+0000 7f3034472700 1 --2- 172.21.15.161:0/381395559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0728b0 0x7f302c072c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.388+0000 7f3034472700 1 --2- 172.21.15.161:0/381395559 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f302c10c3e0 0x7f302c10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.388+0000 7f3034472700 1 --2- 172.21.15.161:0/381395559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c0731c0 0x7f302c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.388+0000 7f3034472700 1 -- 172.21.15.161:0/381395559 >> 172.21.15.161:0/381395559 conn(0x7f302c0fb660 msgr2=0x7f302c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.388+0000 7f3034472700 1 -- 172.21.15.161:0/381395559 shutdown_connections 2023-07-14T11:21:14.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.388+0000 7f3034472700 1 -- 172.21.15.161:0/381395559 wait complete. 2023-07-14T11:21:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.389+0000 7f3034472700 1 Processor -- start 2023-07-14T11:21:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.389+0000 7f3034472700 1 -- start start 2023-07-14T11:21:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3034472700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f302c0728b0 0x7f302c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3034472700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0731c0 0x7f302c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3034472700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c10c3e0 0x7f302c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3034472700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c19cdd0 con 0x7f302c0731c0 2023-07-14T11:21:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3034472700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c19cf40 con 0x7f302c10c3e0 2023-07-14T11:21:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3034472700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c19d100 con 0x7f302c0728b0 2023-07-14T11:21:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3031a0d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0731c0 0x7f302c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3031a0d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0731c0 0x7f302c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40900/0 (socket says 172.21.15.161:40900) 2023-07-14T11:21:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3031a0d700 1 -- 172.21.15.161:0/1569250408 learned_addr learned my addr 172.21.15.161:0/1569250408 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f303220e700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f302c0728b0 0x7f302c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.390+0000 7f3032a0f700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c10c3e0 0x7f302c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:14.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3031a0d700 1 -- 172.21.15.161:0/1569250408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f302c0728b0 msgr2=0x7f302c197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:14.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3031a0d700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f302c0728b0 0x7f302c197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3031a0d700 1 -- 172.21.15.161:0/1569250408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c10c3e0 msgr2=0x7f302c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:14.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3031a0d700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c10c3e0 0x7f302c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3031a0d700 1 -- 172.21.15.161:0/1569250408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280095f0 con 0x7f302c0731c0 2023-07-14T11:21:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3031a0d700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0731c0 0x7f302c1982c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f301c00e9a0 tx=0x7f301c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f30237fe700 1 -- 172.21.15.161:0/1569250408 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c009790 con 0x7f302c0731c0 2023-07-14T11:21:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f30237fe700 1 -- 172.21.15.161:0/1569250408 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f301c010430 con 0x7f302c0731c0 2023-07-14T11:21:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c19d3e0 con 0x7f302c0731c0 2023-07-14T11:21:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f30237fe700 1 -- 172.21.15.161:0/1569250408 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c018680 con 0x7f302c0731c0 2023-07-14T11:21:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.391+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c1a4bb0 con 0x7f302c0731c0 2023-07-14T11:21:14.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.392+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c106d00 con 0x7f302c0731c0 2023-07-14T11:21:14.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.394+0000 7f30237fe700 1 -- 172.21.15.161:0/1569250408 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f301c0105a0 con 0x7f302c0731c0 2023-07-14T11:21:14.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.396+0000 7f30237fe700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f30180a11e0 0x7f30180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:14.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.396+0000 7f30237fe700 1 -- 172.21.15.161:0/1569250408 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f301c014070 con 0x7f302c0731c0 2023-07-14T11:21:14.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.396+0000 7f303220e700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f30180a11e0 0x7f30180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:14.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.397+0000 7f303220e700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f30180a11e0 0x7f30180a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f302800b2e0 tx=0x7f3028005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:14.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.399+0000 7f30237fe700 1 -- 172.21.15.161:0/1569250408 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f301c051790 con 0x7f302c0731c0 2023-07-14T11:21:14.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.637+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f302c04e210 con 0x7f302c0731c0 2023-07-14T11:21:14.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.638+0000 7f30237fe700 1 -- 172.21.15.161:0/1569250408 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f301c0d0030 con 0x7f302c0731c0 2023-07-14T11:21:14.640 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:14.641 INFO:teuthology.orchestra.run.smithi161.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-14T11:21:14.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f30180a11e0 msgr2=0x7f30180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:14.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f30180a11e0 0x7f30180a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f302800b2e0 tx=0x7f3028005fd0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0731c0 msgr2=0x7f302c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:14.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0731c0 0x7f302c1982c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f301c00e9a0 tx=0x7f301c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 shutdown_connections 2023-07-14T11:21:14.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f30180a11e0 0x7f30180a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c10c3e0 0x7f302c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f302c0728b0 0x7f302c197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 --2- 172.21.15.161:0/1569250408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f302c0731c0 0x7f302c1982c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:14.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 >> 172.21.15.161:0/1569250408 conn(0x7f302c0fb660 msgr2=0x7f302c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:14.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.640+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 shutdown_connections 2023-07-14T11:21:14.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:14.641+0000 7f3034472700 1 -- 172.21.15.161:0/1569250408 wait complete. 2023-07-14T11:21:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:14 smithi161 ceph-mon[95526]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:14 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1569250408' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T11:21:15.162 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-14T11:21:15.163 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:14 smithi175 ceph-mon[129081]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:14 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1569250408' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T11:21:15.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:14 smithi182 ceph-mon[129366]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:15.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:14 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1569250408' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T11:21:15.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 -- 172.21.15.161:0/3903734097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 msgr2=0x7f90b80ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/3903734097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 0x7f90b80ffdf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f90a80098f0 tx=0x7f90a8009c00 comp rx=0 tx=0).stop 2023-07-14T11:21:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 -- 172.21.15.161:0/3903734097 shutdown_connections 2023-07-14T11:21:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/3903734097 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90b8101560 0x7f90b8107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/3903734097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 0x7f90b80ffdf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/3903734097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8100bd0 0x7f90b8101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 -- 172.21.15.161:0/3903734097 >> 172.21.15.161:0/3903734097 conn(0x7f90b80fafe0 msgr2=0x7f90b80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:15.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 -- 172.21.15.161:0/3903734097 shutdown_connections 2023-07-14T11:21:15.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.365+0000 7f90bcd9d700 1 -- 172.21.15.161:0/3903734097 wait complete. 2023-07-14T11:21:15.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 Processor -- start 2023-07-14T11:21:15.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 -- start start 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 0x7f90b8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90b8100bd0 0x7f90b8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8101560 0x7f90b819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b819f300 con 0x7f90b8101560 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b819f440 con 0x7f90b80ff9e0 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.366+0000 7f90bcd9d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b819f5d0 con 0x7f90b8100bd0 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8101560 0x7f90b819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 0x7f90b8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8101560 0x7f90b819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40916/0 (socket says 172.21.15.161:40916) 2023-07-14T11:21:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 -- 172.21.15.161:0/177338161 learned_addr learned my addr 172.21.15.161:0/177338161 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b6ffd700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90b8100bd0 0x7f90b8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 -- 172.21.15.161:0/177338161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90b8100bd0 msgr2=0x7f90b8197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90b8100bd0 0x7f90b8197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 -- 172.21.15.161:0/177338161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 msgr2=0x7f90b8197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 0x7f90b8197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 -- 172.21.15.161:0/177338161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0009d00 con 0x7f90b8101560 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b77fe700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 0x7f90b8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b7fff700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8101560 0x7f90b819ed30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f90b000e3c0 tx=0x7f90b000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b4ff9700 1 -- 172.21.15.161:0/177338161 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00090d0 con 0x7f90b8101560 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a8009520 con 0x7f90b8101560 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.367+0000 7f90b4ff9700 1 -- 172.21.15.161:0/177338161 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90b000f040 con 0x7f90b8101560 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.368+0000 7f90b4ff9700 1 -- 172.21.15.161:0/177338161 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0007c20 con 0x7f90b8101560 2023-07-14T11:21:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.368+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b819fc30 con 0x7f90b8101560 2023-07-14T11:21:15.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.369+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b810b1b0 con 0x7f90b8101560 2023-07-14T11:21:15.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.371+0000 7f90b4ff9700 1 -- 172.21.15.161:0/177338161 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f90b0003a00 con 0x7f90b8101560 2023-07-14T11:21:15.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.372+0000 7f90b4ff9700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90a00a12e0 0x7f90a00a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.372+0000 7f90b4ff9700 1 -- 172.21.15.161:0/177338161 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f90b00cd060 con 0x7f90b8101560 2023-07-14T11:21:15.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.372+0000 7f90b77fe700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90a00a12e0 0x7f90a00a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.373+0000 7f90b77fe700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90a00a12e0 0x7f90a00a3790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90a80098c0 tx=0x7f90a8019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:15.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.374+0000 7f90b4ff9700 1 -- 172.21.15.161:0/177338161 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f90b0096370 con 0x7f90b8101560 2023-07-14T11:21:15.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.574+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90b81a0270 con 0x7f90b8101560 2023-07-14T11:21:15.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.575+0000 7f90b4ff9700 1 -- 172.21.15.161:0/177338161 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24461 (secure 0 0 0) 0x7f90b0099990 con 0x7f90b8101560 2023-07-14T11:21:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:15.579 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":90,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:20:15.713307+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:15.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90a00a12e0 msgr2=0x7f90a00a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90a00a12e0 0x7f90a00a3790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90a80098c0 tx=0x7f90a8019040 comp rx=0 tx=0).stop 2023-07-14T11:21:15.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8101560 msgr2=0x7f90b819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8101560 0x7f90b819ed30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f90b000e3c0 tx=0x7f90b000e6d0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 shutdown_connections 2023-07-14T11:21:15.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90b8101560 0x7f90b819ed30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f90a00a12e0 0x7f90a00a3790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b80ff9e0 0x7f90b8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 --2- 172.21.15.161:0/177338161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f90b8100bd0 0x7f90b8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 >> 172.21.15.161:0/177338161 conn(0x7f90b80fafe0 msgr2=0x7f90b80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:15.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 shutdown_connections 2023-07-14T11:21:15.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.578+0000 7f90bcd9d700 1 -- 172.21.15.161:0/177338161 wait complete. 2023-07-14T11:21:15.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:21:15.601 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:21:15.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 -- 172.21.15.161:0/1865161965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42480ffaf0 msgr2=0x7f42480fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 --2- 172.21.15.161:0/1865161965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42480ffaf0 0x7f42480fff00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4244009990 tx=0x7f4244009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 -- 172.21.15.161:0/1865161965 shutdown_connections 2023-07-14T11:21:15.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 --2- 172.21.15.161:0/1865161965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42480ffaf0 0x7f42480fff00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 --2- 172.21.15.161:0/1865161965 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4248101670 0x7f4248107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 --2- 172.21.15.161:0/1865161965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4248100ce0 0x7f4248101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 -- 172.21.15.161:0/1865161965 >> 172.21.15.161:0/1865161965 conn(0x7f42480fb130 msgr2=0x7f42480fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:15.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 -- 172.21.15.161:0/1865161965 shutdown_connections 2023-07-14T11:21:15.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.805+0000 7f424ffff700 1 -- 172.21.15.161:0/1865161965 wait complete. 2023-07-14T11:21:15.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.806+0000 7f424ffff700 1 Processor -- start 2023-07-14T11:21:15.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.806+0000 7f424ffff700 1 -- start start 2023-07-14T11:21:15.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42480ffaf0 0x7f4248197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4248100ce0 0x7f4248197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248101670 0x7f424819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424819f5e0 con 0x7f4248101670 2023-07-14T11:21:15.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424819f720 con 0x7f42480ffaf0 2023-07-14T11:21:15.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424ffff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424819f8b0 con 0x7f4248100ce0 2023-07-14T11:21:15.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248101670 0x7f424819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248101670 0x7f424819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40934/0 (socket says 172.21.15.161:40934) 2023-07-14T11:21:15.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 -- 172.21.15.161:0/2524724501 learned_addr learned my addr 172.21.15.161:0/2524724501 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:15.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424dd9b700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42480ffaf0 0x7f4248197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424d59a700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4248100ce0 0x7f4248197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 -- 172.21.15.161:0/2524724501 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4248100ce0 msgr2=0x7f4248197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4248100ce0 0x7f4248197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 -- 172.21.15.161:0/2524724501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42480ffaf0 msgr2=0x7f4248197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42480ffaf0 0x7f4248197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.807+0000 7f424e59c700 1 -- 172.21.15.161:0/2524724501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440095f0 con 0x7f4248101670 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.808+0000 7f424e59c700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248101670 0x7f424819f010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f424000ef30 tx=0x7f424000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.808+0000 7f423effd700 1 -- 172.21.15.161:0/2524724501 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240018070 con 0x7f4248101670 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.808+0000 7f423effd700 1 -- 172.21.15.161:0/2524724501 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f424000f040 con 0x7f4248101670 2023-07-14T11:21:15.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.808+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424819fb30 con 0x7f4248101670 2023-07-14T11:21:15.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.808+0000 7f423effd700 1 -- 172.21.15.161:0/2524724501 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424000cd00 con 0x7f4248101670 2023-07-14T11:21:15.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.808+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424819fff0 con 0x7f4248101670 2023-07-14T11:21:15.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.809+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42481921d0 con 0x7f4248101670 2023-07-14T11:21:15.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.811+0000 7f423effd700 1 -- 172.21.15.161:0/2524724501 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f4240003a00 con 0x7f4248101670 2023-07-14T11:21:15.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.814+0000 7f423effd700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42340a11e0 0x7f42340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:15.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.814+0000 7f423effd700 1 -- 172.21.15.161:0/2524724501 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f42400cc9a0 con 0x7f4248101670 2023-07-14T11:21:15.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.814+0000 7f424dd9b700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42340a11e0 0x7f42340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:15.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.814+0000 7f424dd9b700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42340a11e0 0x7f42340a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f424400b2e0 tx=0x7f4244005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:15.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:15.817+0000 7f423effd700 1 -- 172.21.15.161:0/2524724501 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4240095c70 con 0x7f4248101670 2023-07-14T11:21:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-07-14T11:21:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-14T11:21:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:15 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/177338161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:15 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-07-14T11:21:16.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.020+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f424802ca60 con 0x7f4248101670 2023-07-14T11:21:16.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.020+0000 7f423effd700 1 -- 172.21.15.161:0/2524724501 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4240099290 con 0x7f4248101670 2023-07-14T11:21:16.023 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:21:16.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42340a11e0 msgr2=0x7f42340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42340a11e0 0x7f42340a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f424400b2e0 tx=0x7f4244005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248101670 msgr2=0x7f424819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248101670 0x7f424819f010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f424000ef30 tx=0x7f424000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 shutdown_connections 2023-07-14T11:21:16.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248101670 0x7f424819f010 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42340a11e0 0x7f42340a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42480ffaf0 0x7f4248197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 --2- 172.21.15.161:0/2524724501 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4248100ce0 0x7f4248197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 >> 172.21.15.161:0/2524724501 conn(0x7f42480fb130 msgr2=0x7f42481065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:16.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 shutdown_connections 2023-07-14T11:21:16.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.023+0000 7f424ffff700 1 -- 172.21.15.161:0/2524724501 wait complete. 2023-07-14T11:21:16.044 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:21:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-07-14T11:21:16.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-14T11:21:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:15 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/177338161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:15 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-07-14T11:21:16.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-07-14T11:21:16.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-14T11:21:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:15 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/177338161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:15 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-07-14T11:21:16.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 -- 172.21.15.161:0/4146823177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1190101660 msgr2=0x7f1190107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 --2- 172.21.15.161:0/4146823177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1190101660 0x7f1190107f20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f118800b0a0 tx=0x7f118800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 -- 172.21.15.161:0/4146823177 shutdown_connections 2023-07-14T11:21:16.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 --2- 172.21.15.161:0/4146823177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1190101660 0x7f1190107f20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 --2- 172.21.15.161:0/4146823177 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f11900ffae0 0x7f11900ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 --2- 172.21.15.161:0/4146823177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1190100cd0 0x7f1190101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 -- 172.21.15.161:0/4146823177 >> 172.21.15.161:0/4146823177 conn(0x7f11900fb0e0 msgr2=0x7f11900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:16.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 -- 172.21.15.161:0/4146823177 shutdown_connections 2023-07-14T11:21:16.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.243+0000 7f11955ff700 1 -- 172.21.15.161:0/4146823177 wait complete. 2023-07-14T11:21:16.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 Processor -- start 2023-07-14T11:21:16.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 -- start start 2023-07-14T11:21:16.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11900ffae0 0x7f1190197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1190100cd0 0x7f11901980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1190101660 0x7f119019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019f730 con 0x7f11900ffae0 2023-07-14T11:21:16.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019f870 con 0x7f1190100cd0 2023-07-14T11:21:16.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.244+0000 7f11955ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019fa00 con 0x7f1190101660 2023-07-14T11:21:16.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11900ffae0 0x7f1190197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11900ffae0 0x7f1190197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40952/0 (socket says 172.21.15.161:40952) 2023-07-14T11:21:16.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 -- 172.21.15.161:0/877969644 learned_addr learned my addr 172.21.15.161:0/877969644 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:16.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1190100cd0 0x7f11901980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118f7fe700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1190101660 0x7f119019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 -- 172.21.15.161:0/877969644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1190101660 msgr2=0x7f119019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1190101660 0x7f119019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 -- 172.21.15.161:0/877969644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1190100cd0 msgr2=0x7f11901980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1190100cd0 0x7f11901980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 -- 172.21.15.161:0/877969644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188009d00 con 0x7f11900ffae0 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118e7fc700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1190100cd0 0x7f11901980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:16.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.245+0000 7f118effd700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11900ffae0 0x7f1190197bb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f118000e970 tx=0x7f118000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:16.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.246+0000 7f1177fff700 1 -- 172.21.15.161:0/877969644 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180009790 con 0x7f11900ffae0 2023-07-14T11:21:16.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.246+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119019fce0 con 0x7f11900ffae0 2023-07-14T11:21:16.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.246+0000 7f1177fff700 1 -- 172.21.15.161:0/877969644 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f118000c7e0 con 0x7f11900ffae0 2023-07-14T11:21:16.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.246+0000 7f1177fff700 1 -- 172.21.15.161:0/877969644 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180003c70 con 0x7f11900ffae0 2023-07-14T11:21:16.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.246+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901a0190 con 0x7f11900ffae0 2023-07-14T11:21:16.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.247+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119010b310 con 0x7f11900ffae0 2023-07-14T11:21:16.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.249+0000 7f1177fff700 1 -- 172.21.15.161:0/877969644 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f118000cc50 con 0x7f11900ffae0 2023-07-14T11:21:16.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.250+0000 7f1177fff700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11780a1260 0x7f11780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.250+0000 7f1177fff700 1 -- 172.21.15.161:0/877969644 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1180014070 con 0x7f11900ffae0 2023-07-14T11:21:16.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.250+0000 7f118e7fc700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11780a1260 0x7f11780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.251+0000 7f118e7fc700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11780a1260 0x7f11780a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1184009fd0 tx=0x7f1184009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:16.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.254+0000 7f1177fff700 1 -- 172.21.15.161:0/877969644 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f11800961d0 con 0x7f11900ffae0 2023-07-14T11:21:16.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.453+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f119010b500 con 0x7f11900ffae0 2023-07-14T11:21:16.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.454+0000 7f1177fff700 1 -- 172.21.15.161:0/877969644 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f119010b500 con 0x7f11900ffae0 2023-07-14T11:21:16.457 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11780a1260 msgr2=0x7f11780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11780a1260 0x7f11780a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1184009fd0 tx=0x7f1184009260 comp rx=0 tx=0).stop 2023-07-14T11:21:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11900ffae0 msgr2=0x7f1190197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11900ffae0 0x7f1190197bb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f118000e970 tx=0x7f118000ec80 comp rx=0 tx=0).stop 2023-07-14T11:21:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 shutdown_connections 2023-07-14T11:21:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11900ffae0 0x7f1190197bb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11780a1260 0x7f11780a3710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1190101660 0x7f119019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 --2- 172.21.15.161:0/877969644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1190100cd0 0x7f11901980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 >> 172.21.15.161:0/877969644 conn(0x7f11900fb0e0 msgr2=0x7f1190106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:16.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 shutdown_connections 2023-07-14T11:21:16.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.456+0000 7f11955ff700 1 -- 172.21.15.161:0/877969644 wait complete. 2023-07-14T11:21:16.477 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:21:16.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 -- 172.21.15.161:0/4000698822 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f86900ffae0 msgr2=0x7f86900ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 --2- 172.21.15.161:0/4000698822 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f86900ffae0 0x7f86900ffef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8680009f90 tx=0x7f868000b330 comp rx=0 tx=0).stop 2023-07-14T11:21:16.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 -- 172.21.15.161:0/4000698822 shutdown_connections 2023-07-14T11:21:16.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 --2- 172.21.15.161:0/4000698822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8690101660 0x7f8690107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 --2- 172.21.15.161:0/4000698822 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f86900ffae0 0x7f86900ffef0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 --2- 172.21.15.161:0/4000698822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 0x7f8690101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 -- 172.21.15.161:0/4000698822 >> 172.21.15.161:0/4000698822 conn(0x7f86900fb0e0 msgr2=0x7f86900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:16.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 -- 172.21.15.161:0/4000698822 shutdown_connections 2023-07-14T11:21:16.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.676+0000 7f86949ff700 1 -- 172.21.15.161:0/4000698822 wait complete. 2023-07-14T11:21:16.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.677+0000 7f86949ff700 1 Processor -- start 2023-07-14T11:21:16.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.677+0000 7f86949ff700 1 -- start start 2023-07-14T11:21:16.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f86949ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900ffae0 0x7f86900742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f86949ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 0x7f8690072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f86949ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8690101660 0x7f8690072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f86949ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900735e0 con 0x7f8690100cd0 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f86949ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8690073750 con 0x7f86900ffae0 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f86949ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869019f7f0 con 0x7f8690101660 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 0x7f8690072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f868ed9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8690101660 0x7f8690072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 0x7f8690072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40974/0 (socket says 172.21.15.161:40974) 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 -- 172.21.15.161:0/733203387 learned_addr learned my addr 172.21.15.161:0/733203387 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:16.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f868e59c700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900ffae0 0x7f86900742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 -- 172.21.15.161:0/733203387 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8690101660 msgr2=0x7f8690072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8690101660 0x7f8690072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 -- 172.21.15.161:0/733203387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900ffae0 msgr2=0x7f86900742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900ffae0 0x7f86900742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.678+0000 7f8687fff700 1 -- 172.21.15.161:0/733203387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680009c40 con 0x7f8690100cd0 2023-07-14T11:21:16.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.679+0000 7f8687fff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 0x7f8690072900 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9c0 tx=0x7f867800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:16.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.679+0000 7f86877fe700 1 -- 172.21.15.161:0/733203387 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678004a30 con 0x7f8690100cd0 2023-07-14T11:21:16.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.679+0000 7f86877fe700 1 -- 172.21.15.161:0/733203387 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f867800cc80 con 0x7f8690100cd0 2023-07-14T11:21:16.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.679+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869019fad0 con 0x7f8690100cd0 2023-07-14T11:21:16.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.679+0000 7f86877fe700 1 -- 172.21.15.161:0/733203387 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780054e0 con 0x7f8690100cd0 2023-07-14T11:21:16.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.679+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86901a0030 con 0x7f8690100cd0 2023-07-14T11:21:16.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.680+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86901070c0 con 0x7f8690100cd0 2023-07-14T11:21:16.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.682+0000 7f86877fe700 1 -- 172.21.15.161:0/733203387 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8678004b90 con 0x7f8690100cd0 2023-07-14T11:21:16.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.684+0000 7f86877fe700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f867c0a1260 0x7f867c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:16.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.685+0000 7f86877fe700 1 -- 172.21.15.161:0/733203387 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8678014070 con 0x7f8690100cd0 2023-07-14T11:21:16.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.685+0000 7f868e59c700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f867c0a1260 0x7f867c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:16.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.685+0000 7f868e59c700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f867c0a1260 0x7f867c0a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8680009790 tx=0x7f86800096c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:16.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.688+0000 7f86877fe700 1 -- 172.21.15.161:0/733203387 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f86780960e0 con 0x7f8690100cd0 2023-07-14T11:21:16.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.893+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f869002ca70 con 0x7f8690100cd0 2023-07-14T11:21:16.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.894+0000 7f86877fe700 1 -- 172.21.15.161:0/733203387 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8678099700 con 0x7f8690100cd0 2023-07-14T11:21:16.896 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:21:16.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f867c0a1260 msgr2=0x7f867c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f867c0a1260 0x7f867c0a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8680009790 tx=0x7f86800096c0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 msgr2=0x7f8690072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:16.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 0x7f8690072900 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9c0 tx=0x7f867800ecd0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 shutdown_connections 2023-07-14T11:21:16.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f867c0a1260 0x7f867c0a3710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8690101660 0x7f8690072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900ffae0 0x7f86900742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 --2- 172.21.15.161:0/733203387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8690100cd0 0x7f8690072900 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:16.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 >> 172.21.15.161:0/733203387 conn(0x7f86900fb0e0 msgr2=0x7f8690104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:16.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 shutdown_connections 2023-07-14T11:21:16.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:16.896+0000 7f86949ff700 1 -- 172.21.15.161:0/733203387 wait complete. 2023-07-14T11:21:16.918 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:21:17.031 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:16 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-14T11:21:17.031 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:16 smithi161 ceph-mon[95526]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T11:21:17.031 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:16 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2524724501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:17.031 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:16 smithi161 ceph-mon[95526]: pgmap v322: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:17.031 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:16 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/877969644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:17.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:16 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-14T11:21:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:16 smithi175 ceph-mon[129081]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T11:21:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:16 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2524724501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:16 smithi175 ceph-mon[129081]: pgmap v322: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:17.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:16 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/877969644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:17.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 -- 172.21.15.161:0/1009940294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4101670 msgr2=0x7fbfb4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/1009940294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4101670 0x7fbfb4107f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00b0a0 tx=0x7fbfac00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 -- 172.21.15.161:0/1009940294 shutdown_connections 2023-07-14T11:21:17.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/1009940294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4101670 0x7fbfb4107f30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/1009940294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb40ffaf0 0x7fbfb40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/1009940294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfb4100ce0 0x7fbfb4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 -- 172.21.15.161:0/1009940294 >> 172.21.15.161:0/1009940294 conn(0x7fbfb40fb130 msgr2=0x7fbfb40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:17.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 -- 172.21.15.161:0/1009940294 shutdown_connections 2023-07-14T11:21:17.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.172+0000 7fbfba7ff700 1 -- 172.21.15.161:0/1009940294 wait complete. 2023-07-14T11:21:17.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 Processor -- start 2023-07-14T11:21:17.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 -- start start 2023-07-14T11:21:17.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfb40ffaf0 0x7fbfb40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4100ce0 0x7fbfb4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb4101670 0x7fbfb4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40735e0 con 0x7fbfb4100ce0 2023-07-14T11:21:17.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4073750 con 0x7fbfb4101670 2023-07-14T11:21:17.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfba7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb419f9c0 con 0x7fbfb40ffaf0 2023-07-14T11:21:17.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfb37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4100ce0 0x7fbfb4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfb37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4100ce0 0x7fbfb4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41002/0 (socket says 172.21.15.161:41002) 2023-07-14T11:21:17.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfb37fe700 1 -- 172.21.15.161:0/3151625476 learned_addr learned my addr 172.21.15.161:0/3151625476 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfb8d9c700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb4101670 0x7fbfb4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.174+0000 7fbfb3fff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfb40ffaf0 0x7fbfb40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb37fe700 1 -- 172.21.15.161:0/3151625476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfb40ffaf0 msgr2=0x7fbfb40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb37fe700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfb40ffaf0 0x7fbfb40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb37fe700 1 -- 172.21.15.161:0/3151625476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb4101670 msgr2=0x7fbfb4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb37fe700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb4101670 0x7fbfb4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb37fe700 1 -- 172.21.15.161:0/3151625476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8009520 con 0x7fbfb4100ce0 2023-07-14T11:21:17.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb37fe700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4100ce0 0x7fbfb4072900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800eb30 tx=0x7fbfa800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:17.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb17fa700 1 -- 172.21.15.161:0/3151625476 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8009880 con 0x7fbfb4100ce0 2023-07-14T11:21:17.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac009d00 con 0x7fbfb4100ce0 2023-07-14T11:21:17.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb17fa700 1 -- 172.21.15.161:0/3151625476 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfa80044e0 con 0x7fbfb4100ce0 2023-07-14T11:21:17.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfb17fa700 1 -- 172.21.15.161:0/3151625476 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8003d80 con 0x7fbfb4100ce0 2023-07-14T11:21:17.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.175+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb41a0040 con 0x7fbfb4100ce0 2023-07-14T11:21:17.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.176+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4106980 con 0x7fbfb4100ce0 2023-07-14T11:21:17.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.178+0000 7fbfb17fa700 1 -- 172.21.15.161:0/3151625476 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fbfa8003f20 con 0x7fbfb4100ce0 2023-07-14T11:21:17.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.180+0000 7fbfb17fa700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbf9c0a11e0 0x7fbf9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.180+0000 7fbfb17fa700 1 -- 172.21.15.161:0/3151625476 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbfa8014070 con 0x7fbfb4100ce0 2023-07-14T11:21:17.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.180+0000 7fbfb3fff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbf9c0a11e0 0x7fbf9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.181+0000 7fbfb3fff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbf9c0a11e0 0x7fbf9c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4100b40 tx=0x7fbfa4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:17.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.183+0000 7fbfb17fa700 1 -- 172.21.15.161:0/3151625476 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbfa80965d0 con 0x7fbfb4100ce0 2023-07-14T11:21:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:16 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-14T11:21:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:16 smithi182 ceph-mon[129366]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T11:21:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:16 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2524724501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:17.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:16 smithi182 ceph-mon[129366]: pgmap v322: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:17.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:16 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/877969644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:17.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.381+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbfb402ca50 con 0x7fbfb4100ce0 2023-07-14T11:21:17.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.381+0000 7fbfb17fa700 1 -- 172.21.15.161:0/3151625476 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbfa8099bf0 con 0x7fbfb4100ce0 2023-07-14T11:21:17.384 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:17.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.383+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbf9c0a11e0 msgr2=0x7fbf9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.383+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbf9c0a11e0 0x7fbf9c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4100b40 tx=0x7fbfa4009260 comp rx=0 tx=0).stop 2023-07-14T11:21:17.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4100ce0 msgr2=0x7fbfb4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4100ce0 0x7fbfb4072900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800eb30 tx=0x7fbfa800ee40 comp rx=0 tx=0).stop 2023-07-14T11:21:17.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 shutdown_connections 2023-07-14T11:21:17.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbf9c0a11e0 0x7fbf9c0a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfb4101670 0x7fbfb4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfb40ffaf0 0x7fbfb40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 --2- 172.21.15.161:0/3151625476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb4100ce0 0x7fbfb4072900 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 >> 172.21.15.161:0/3151625476 conn(0x7fbfb40fb130 msgr2=0x7fbfb40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:17.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 shutdown_connections 2023-07-14T11:21:17.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.384+0000 7fbfba7ff700 1 -- 172.21.15.161:0/3151625476 wait complete. 2023-07-14T11:21:17.405 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:21:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 -- 172.21.15.161:0/1804580447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa80ffb00 msgr2=0x7f1fa80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 --2- 172.21.15.161:0/1804580447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa80ffb00 0x7f1fa80fff10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009990 tx=0x7f1f9c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 -- 172.21.15.161:0/1804580447 shutdown_connections 2023-07-14T11:21:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 --2- 172.21.15.161:0/1804580447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa80ffb00 0x7f1fa80fff10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 --2- 172.21.15.161:0/1804580447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1fa8101680 0x7f1fa8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 --2- 172.21.15.161:0/1804580447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa8100cf0 0x7f1fa8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 -- 172.21.15.161:0/1804580447 >> 172.21.15.161:0/1804580447 conn(0x7f1fa80fb0e0 msgr2=0x7f1fa80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 -- 172.21.15.161:0/1804580447 shutdown_connections 2023-07-14T11:21:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.602+0000 7f1faebff700 1 -- 172.21.15.161:0/1804580447 wait complete. 2023-07-14T11:21:17.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.603+0000 7f1faebff700 1 Processor -- start 2023-07-14T11:21:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.603+0000 7f1faebff700 1 -- start start 2023-07-14T11:21:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1faebff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa80ffb00 0x7f1fa80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1faebff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8100cf0 0x7f1fa8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1faebff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1fa8101680 0x7f1fa8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1faebff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80735e0 con 0x7f1fa8100cf0 2023-07-14T11:21:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1faebff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8073750 con 0x7f1fa80ffb00 2023-07-14T11:21:17.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1faebff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa819f8a0 con 0x7f1fa8101680 2023-07-14T11:21:17.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8100cf0 0x7f1fa8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8100cf0 0x7f1fa8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41022/0 (socket says 172.21.15.161:41022) 2023-07-14T11:21:17.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 -- 172.21.15.161:0/1703072275 learned_addr learned my addr 172.21.15.161:0/1703072275 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:17.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fac99b700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa80ffb00 0x7f1fa80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fad19c700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1fa8101680 0x7f1fa8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 -- 172.21.15.161:0/1703072275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1fa8101680 msgr2=0x7f1fa8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1fa8101680 0x7f1fa8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 -- 172.21.15.161:0/1703072275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa80ffb00 msgr2=0x7f1fa80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa80ffb00 0x7f1fa80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.604+0000 7f1fa7fff700 1 -- 172.21.15.161:0/1703072275 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0095f0 con 0x7f1fa8100cf0 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.605+0000 7f1fa7fff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8100cf0 0x7f1fa8072900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e9a0 tx=0x7f1f9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.605+0000 7f1fa5ffb700 1 -- 172.21.15.161:0/1703072275 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94009790 con 0x7f1fa8100cf0 2023-07-14T11:21:17.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.605+0000 7f1fa5ffb700 1 -- 172.21.15.161:0/1703072275 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f9400cbd0 con 0x7f1fa8100cf0 2023-07-14T11:21:17.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.605+0000 7f1fa5ffb700 1 -- 172.21.15.161:0/1703072275 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94018680 con 0x7f1fa8100cf0 2023-07-14T11:21:17.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.605+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa819fb80 con 0x7f1fa8100cf0 2023-07-14T11:21:17.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.605+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa81a0040 con 0x7f1fa8100cf0 2023-07-14T11:21:17.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.606+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa810b320 con 0x7f1fa8100cf0 2023-07-14T11:21:17.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.608+0000 7f1fa5ffb700 1 -- 172.21.15.161:0/1703072275 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1f9400cd40 con 0x7f1fa8100cf0 2023-07-14T11:21:17.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.610+0000 7f1fa5ffb700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f980a12a0 0x7f1f980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:17.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.610+0000 7f1fa5ffb700 1 -- 172.21.15.161:0/1703072275 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1f94014070 con 0x7f1fa8100cf0 2023-07-14T11:21:17.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.610+0000 7f1fac99b700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f980a12a0 0x7f1f980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:17.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.611+0000 7f1fac99b700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f980a12a0 0x7f1f980a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b2e0 tx=0x7f1f9c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:17.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.614+0000 7f1fa5ffb700 1 -- 172.21.15.161:0/1703072275 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1f94051790 con 0x7f1fa8100cf0 2023-07-14T11:21:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.858+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fa81069a0 con 0x7f1fa8100cf0 2023-07-14T11:21:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.859+0000 7f1fa5ffb700 1 -- 172.21.15.161:0/1703072275 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f1f940962b0 con 0x7f1fa8100cf0 2023-07-14T11:21:17.862 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:17.862 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:21:17.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f980a12a0 msgr2=0x7f1f980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f980a12a0 0x7f1f980a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b2e0 tx=0x7f1f9c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8100cf0 msgr2=0x7f1fa8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8100cf0 0x7f1fa8072900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e9a0 tx=0x7f1f9400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 shutdown_connections 2023-07-14T11:21:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f980a12a0 0x7f1f980a3750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1fa8101680 0x7f1fa8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa80ffb00 0x7f1fa80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 --2- 172.21.15.161:0/1703072275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8100cf0 0x7f1fa8072900 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 >> 172.21.15.161:0/1703072275 conn(0x7f1fa80fb0e0 msgr2=0x7f1fa80fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 shutdown_connections 2023-07-14T11:21:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:17.862+0000 7f1faebff700 1 -- 172.21.15.161:0/1703072275 wait complete. 2023-07-14T11:21:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:21:17.884 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:18.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 -- 172.21.15.161:0/982068848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 msgr2=0x7f78a0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 --2- 172.21.15.161:0/982068848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 0x7f78a0101140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f78900098f0 tx=0x7f7890009c00 comp rx=0 tx=0).stop 2023-07-14T11:21:18.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 -- 172.21.15.161:0/982068848 shutdown_connections 2023-07-14T11:21:18.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 --2- 172.21.15.161:0/982068848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78a00ffb00 0x7f78a00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 --2- 172.21.15.161:0/982068848 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a0101680 0x7f78a0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 --2- 172.21.15.161:0/982068848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 0x7f78a0101140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 -- 172.21.15.161:0/982068848 >> 172.21.15.161:0/982068848 conn(0x7f78a00fb0e0 msgr2=0x7f78a00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:18.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 -- 172.21.15.161:0/982068848 shutdown_connections 2023-07-14T11:21:18.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.081+0000 7f78a87ff700 1 -- 172.21.15.161:0/982068848 wait complete. 2023-07-14T11:21:18.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.083+0000 7f78a87ff700 1 Processor -- start 2023-07-14T11:21:18.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.083+0000 7f78a87ff700 1 -- start start 2023-07-14T11:21:18.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.083+0000 7f78a87ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a00ffb00 0x7f78a0193650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.083+0000 7f78a87ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 0x7f78a0193b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.083+0000 7f78a87ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78a0101680 0x7f78a019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a87ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a019b1d0 con 0x7f78a0101680 2023-07-14T11:21:18.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a87ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a019b310 con 0x7f78a0100cf0 2023-07-14T11:21:18.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a87ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a019b4a0 con 0x7f78a00ffb00 2023-07-14T11:21:18.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a00ffb00 0x7f78a0193650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a00ffb00 0x7f78a0193650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45846/0 (socket says 172.21.15.161:45846) 2023-07-14T11:21:18.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 -- 172.21.15.161:0/3800700651 learned_addr learned my addr 172.21.15.161:0/3800700651 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:18.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a6d9c700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78a0101680 0x7f78a019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a5d9a700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 0x7f78a0193b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 -- 172.21.15.161:0/3800700651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 msgr2=0x7f78a0193b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 0x7f78a0193b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 -- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78a0101680 msgr2=0x7f78a019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78a0101680 0x7f78a019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a659b700 1 -- 172.21.15.161:0/3800700651 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c0095f0 con 0x7f78a00ffb00 2023-07-14T11:21:18.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.084+0000 7f78a5d9a700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 0x7f78a0193b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:21:18.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.085+0000 7f78a659b700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a00ffb00 0x7f78a0193650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f789c00ebf0 tx=0x7f789c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:18.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.085+0000 7f78977fe700 1 -- 172.21.15.161:0/3800700651 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c009a10 con 0x7f78a00ffb00 2023-07-14T11:21:18.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.085+0000 7f78977fe700 1 -- 172.21.15.161:0/3800700651 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789c00cd50 con 0x7f78a00ffb00 2023-07-14T11:21:18.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.085+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7898009c40 con 0x7f78a00ffb00 2023-07-14T11:21:18.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.085+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7890009520 con 0x7f78a00ffb00 2023-07-14T11:21:18.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.085+0000 7f78977fe700 1 -- 172.21.15.161:0/3800700651 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c018880 con 0x7f78a00ffb00 2023-07-14T11:21:18.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.086+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7884005320 con 0x7f78a00ffb00 2023-07-14T11:21:18.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.088+0000 7f78977fe700 1 -- 172.21.15.161:0/3800700651 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f789c0189e0 con 0x7f78a00ffb00 2023-07-14T11:21:18.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.090+0000 7f78977fe700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f788c0a5630 0x7f788c0a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.090+0000 7f78977fe700 1 -- 172.21.15.161:0/3800700651 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f789c014070 con 0x7f78a00ffb00 2023-07-14T11:21:18.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.090+0000 7f78a5d9a700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f788c0a5630 0x7f788c0a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.091+0000 7f78a5d9a700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f788c0a5630 0x7f788c0a7ae0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f789000c010 tx=0x7f7890009f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:18.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.092+0000 7f78977fe700 1 -- 172.21.15.161:0/3800700651 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f789c096970 con 0x7f78a00ffb00 2023-07-14T11:21:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:17 smithi175 ceph-mon[129081]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T11:21:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:17 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/733203387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:18.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:17 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3151625476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:18.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:17 smithi182 ceph-mon[129366]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T11:21:18.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:17 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/733203387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:18.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:17 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3151625476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:18.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:17 smithi161 ceph-mon[95526]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T11:21:18.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:17 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/733203387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:18.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:17 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3151625476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.293+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7884005190 con 0x7f78a00ffb00 2023-07-14T11:21:18.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.295+0000 7f78977fe700 1 -- 172.21.15.161:0/3800700651 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7f789c099f90 con 0x7f78a00ffb00 2023-07-14T11:21:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:18.298 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:18.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f788c0a5630 msgr2=0x7f788c0a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f788c0a5630 0x7f788c0a7ae0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f789000c010 tx=0x7f7890009f50 comp rx=0 tx=0).stop 2023-07-14T11:21:18.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a00ffb00 msgr2=0x7f78a0193650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a00ffb00 0x7f78a0193650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f789c00ebf0 tx=0x7f789c00c2d0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 shutdown_connections 2023-07-14T11:21:18.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78a0101680 0x7f78a019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f788c0a5630 0x7f788c0a7ae0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78a00ffb00 0x7f78a0193650 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 --2- 172.21.15.161:0/3800700651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0100cf0 0x7f78a0193b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 >> 172.21.15.161:0/3800700651 conn(0x7f78a00fb0e0 msgr2=0x7f78a01041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:18.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 shutdown_connections 2023-07-14T11:21:18.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.297+0000 7f78a87ff700 1 -- 172.21.15.161:0/3800700651 wait complete. 2023-07-14T11:21:18.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:21:18.320 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:21:18.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.517+0000 7f65145ff700 1 -- 172.21.15.161:0/1739237433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c107980 msgr2=0x7f650c109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.517+0000 7f65145ff700 1 --2- 172.21.15.161:0/1739237433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c107980 0x7f650c109e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f650400b0a0 tx=0x7f650400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.517+0000 7f65145ff700 1 -- 172.21.15.161:0/1739237433 shutdown_connections 2023-07-14T11:21:18.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.517+0000 7f65145ff700 1 --2- 172.21.15.161:0/1739237433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c107980 0x7f650c109e00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.517+0000 7f65145ff700 1 --2- 172.21.15.161:0/1739237433 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f650c0744f0 0x7f650c072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.517+0000 7f65145ff700 1 --2- 172.21.15.161:0/1739237433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c072f10 0x7f650c073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.517+0000 7f65145ff700 1 -- 172.21.15.161:0/1739237433 >> 172.21.15.161:0/1739237433 conn(0x7f650c0fb130 msgr2=0x7f650c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:18.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.518+0000 7f65145ff700 1 -- 172.21.15.161:0/1739237433 shutdown_connections 2023-07-14T11:21:18.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.518+0000 7f65145ff700 1 -- 172.21.15.161:0/1739237433 wait complete. 2023-07-14T11:21:18.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.518+0000 7f65145ff700 1 Processor -- start 2023-07-14T11:21:18.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f65145ff700 1 -- start start 2023-07-14T11:21:18.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f65145ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f650c072f10 0x7f650c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f65145ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c0744f0 0x7f650c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f65145ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c107980 0x7f650c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f65145ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c19f670 con 0x7f650c107980 2023-07-14T11:21:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f65145ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c19f7b0 con 0x7f650c0744f0 2023-07-14T11:21:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f65145ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c19f940 con 0x7f650c072f10 2023-07-14T11:21:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f6512b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c107980 0x7f650c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f6512b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c107980 0x7f650c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57514/0 (socket says 172.21.15.161:57514) 2023-07-14T11:21:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f6512b9c700 1 -- 172.21.15.161:0/3006947799 learned_addr learned my addr 172.21.15.161:0/3006947799 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f6511b9a700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c0744f0 0x7f650c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.519+0000 7f651239b700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f650c072f10 0x7f650c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f6512b9c700 1 -- 172.21.15.161:0/3006947799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f650c072f10 msgr2=0x7f650c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f6512b9c700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f650c072f10 0x7f650c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f6512b9c700 1 -- 172.21.15.161:0/3006947799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c0744f0 msgr2=0x7f650c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f6512b9c700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c0744f0 0x7f650c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f6512b9c700 1 -- 172.21.15.161:0/3006947799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc009520 con 0x7f650c107980 2023-07-14T11:21:18.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f651239b700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f650c072f10 0x7f650c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:18.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f6512b9c700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c107980 0x7f650c19f0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6504009fd0 tx=0x7f650400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:18.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f65037fe700 1 -- 172.21.15.161:0/3006947799 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400e040 con 0x7f650c107980 2023-07-14T11:21:18.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f65037fe700 1 -- 172.21.15.161:0/3006947799 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65040094a0 con 0x7f650c107980 2023-07-14T11:21:18.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6504009d00 con 0x7f650c107980 2023-07-14T11:21:18.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f65037fe700 1 -- 172.21.15.161:0/3006947799 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650401b6c0 con 0x7f650c107980 2023-07-14T11:21:18.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.520+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c19fe50 con 0x7f650c107980 2023-07-14T11:21:18.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.521+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c1060c0 con 0x7f650c107980 2023-07-14T11:21:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.523+0000 7f65037fe700 1 -- 172.21.15.161:0/3006947799 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f650401d9c0 con 0x7f650c107980 2023-07-14T11:21:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.524+0000 7f65037fe700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64f80a1260 0x7f64f80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.525+0000 7f65037fe700 1 -- 172.21.15.161:0/3006947799 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f65040ce180 con 0x7f650c107980 2023-07-14T11:21:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.525+0000 7f651239b700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64f80a1260 0x7f64f80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.525+0000 7f651239b700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64f80a1260 0x7f64f80a3710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6508009cb0 tx=0x7f6508009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:18.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.529+0000 7f65037fe700 1 -- 172.21.15.161:0/3006947799 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6504097450 con 0x7f650c107980 2023-07-14T11:21:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.727+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f650c02ca70 con 0x7f650c107980 2023-07-14T11:21:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.728+0000 7f65037fe700 1 -- 172.21.15.161:0/3006947799 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f650401a3c0 con 0x7f650c107980 2023-07-14T11:21:18.731 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:21:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64f80a1260 msgr2=0x7f64f80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64f80a1260 0x7f64f80a3710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6508009cb0 tx=0x7f6508009260 comp rx=0 tx=0).stop 2023-07-14T11:21:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c107980 msgr2=0x7f650c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c107980 0x7f650c19f0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6504009fd0 tx=0x7f650400bd90 comp rx=0 tx=0).stop 2023-07-14T11:21:18.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 shutdown_connections 2023-07-14T11:21:18.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f650c107980 0x7f650c19f0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f64f80a1260 0x7f64f80a3710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f650c072f10 0x7f650c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 --2- 172.21.15.161:0/3006947799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f650c0744f0 0x7f650c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 >> 172.21.15.161:0/3006947799 conn(0x7f650c0fb130 msgr2=0x7f650c109bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:18.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 shutdown_connections 2023-07-14T11:21:18.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.731+0000 7f65145ff700 1 -- 172.21.15.161:0/3006947799 wait complete. 2023-07-14T11:21:18.752 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:21:18.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 -- 172.21.15.161:0/2821701996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c0ffaf0 msgr2=0x7f8b4c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2821701996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c0ffaf0 0x7f8b4c0fff00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8b480098f0 tx=0x7f8b48009c00 comp rx=0 tx=0).stop 2023-07-14T11:21:18.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 -- 172.21.15.161:0/2821701996 shutdown_connections 2023-07-14T11:21:18.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2821701996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c101670 0x7f8b4c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2821701996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c0ffaf0 0x7f8b4c0fff00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2821701996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b4c100ce0 0x7f8b4c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 -- 172.21.15.161:0/2821701996 >> 172.21.15.161:0/2821701996 conn(0x7f8b4c0fb130 msgr2=0x7f8b4c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:18.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 -- 172.21.15.161:0/2821701996 shutdown_connections 2023-07-14T11:21:18.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.953+0000 7f8b539ff700 1 -- 172.21.15.161:0/2821701996 wait complete. 2023-07-14T11:21:18.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.954+0000 7f8b539ff700 1 Processor -- start 2023-07-14T11:21:18.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.954+0000 7f8b539ff700 1 -- start start 2023-07-14T11:21:18.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.954+0000 7f8b539ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b4c0ffaf0 0x7f8b4c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.954+0000 7f8b539ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c100ce0 0x7f8b4c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b539ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c101670 0x7f8b4c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b539ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19f620 con 0x7f8b4c100ce0 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b539ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19f760 con 0x7f8b4c0ffaf0 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b539ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19f8f0 con 0x7f8b4c101670 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c100ce0 0x7f8b4c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b5179b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b4c0ffaf0 0x7f8b4c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c100ce0 0x7f8b4c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57524/0 (socket says 172.21.15.161:57524) 2023-07-14T11:21:18.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 -- 172.21.15.161:0/2823377748 learned_addr learned my addr 172.21.15.161:0/2823377748 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:18.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b51f9c700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c101670 0x7f8b4c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 -- 172.21.15.161:0/2823377748 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c101670 msgr2=0x7f8b4c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c101670 0x7f8b4c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 -- 172.21.15.161:0/2823377748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b4c0ffaf0 msgr2=0x7f8b4c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:18.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b4c0ffaf0 0x7f8b4c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:18.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 -- 172.21.15.161:0/2823377748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b44009d00 con 0x7f8b4c100ce0 2023-07-14T11:21:18.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b5179b700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b4c0ffaf0 0x7f8b4c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:18.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b51f9c700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c101670 0x7f8b4c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:18.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.955+0000 7f8b50f9a700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c100ce0 0x7f8b4c197fe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9a0 tx=0x7f8b3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:18.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.956+0000 7f8b427fc700 1 -- 172.21.15.161:0/2823377748 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c009790 con 0x7f8b4c100ce0 2023-07-14T11:21:18.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.956+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b48009520 con 0x7f8b4c100ce0 2023-07-14T11:21:18.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.956+0000 7f8b427fc700 1 -- 172.21.15.161:0/2823377748 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3c010430 con 0x7f8b4c100ce0 2023-07-14T11:21:18.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.956+0000 7f8b427fc700 1 -- 172.21.15.161:0/2823377748 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c018680 con 0x7f8b4c100ce0 2023-07-14T11:21:18.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.956+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c19ff50 con 0x7f8b4c100ce0 2023-07-14T11:21:18.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.957+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c10b2b0 con 0x7f8b4c100ce0 2023-07-14T11:21:18.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.959+0000 7f8b427fc700 1 -- 172.21.15.161:0/2823377748 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8b3c0105a0 con 0x7f8b4c100ce0 2023-07-14T11:21:18.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.960+0000 7f8b427fc700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b380a11e0 0x7f8b380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:18.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.960+0000 7f8b427fc700 1 -- 172.21.15.161:0/2823377748 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8b3c014070 con 0x7f8b4c100ce0 2023-07-14T11:21:18.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.960+0000 7f8b5179b700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b380a11e0 0x7f8b380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:18.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.961+0000 7f8b5179b700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b380a11e0 0x7f8b380a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009fd0 tx=0x7f8b48019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:18.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:18.964+0000 7f8b427fc700 1 -- 172.21.15.161:0/2823377748 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8b3c051790 con 0x7f8b4c100ce0 2023-07-14T11:21:19.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.160+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b4c1a0560 con 0x7f8b4c100ce0 2023-07-14T11:21:19.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.161+0000 7f8b427fc700 1 -- 172.21.15.161:0/2823377748 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b3c096240 con 0x7f8b4c100ce0 2023-07-14T11:21:19.163 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:19.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b380a11e0 msgr2=0x7f8b380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b380a11e0 0x7f8b380a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009fd0 tx=0x7f8b48019040 comp rx=0 tx=0).stop 2023-07-14T11:21:19.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c100ce0 msgr2=0x7f8b4c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c100ce0 0x7f8b4c197fe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9a0 tx=0x7f8b3c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 shutdown_connections 2023-07-14T11:21:19.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b380a11e0 0x7f8b380a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b4c101670 0x7f8b4c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b4c0ffaf0 0x7f8b4c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 --2- 172.21.15.161:0/2823377748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c100ce0 0x7f8b4c197fe0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.163+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 >> 172.21.15.161:0/2823377748 conn(0x7f8b4c0fb130 msgr2=0x7f8b4c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:19.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.164+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 shutdown_connections 2023-07-14T11:21:19.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.164+0000 7f8b539ff700 1 -- 172.21.15.161:0/2823377748 wait complete. 2023-07-14T11:21:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:18 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1703072275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:18 smithi175 ceph-mon[129081]: pgmap v324: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:21:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:18 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3800700651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:19.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:18 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3006947799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:19.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:18 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1703072275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:19.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:18 smithi161 ceph-mon[95526]: pgmap v324: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:21:19.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:18 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3800700651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:19.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:18 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3006947799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:19.185 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:21:19.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:18 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1703072275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:19.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:18 smithi182 ceph-mon[129366]: pgmap v324: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:21:19.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:18 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3800700651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:19.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:18 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3006947799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3779936694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d4100ce0 msgr2=0x7fe7d4101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3779936694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d4100ce0 0x7fe7d4101130 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009990 tx=0x7fe7c8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3779936694 shutdown_connections 2023-07-14T11:21:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3779936694 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7d4101670 0x7fe7d4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3779936694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d40ffaf0 0x7fe7d40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3779936694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d4100ce0 0x7fe7d4101130 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3779936694 >> 172.21.15.161:0/3779936694 conn(0x7fe7d40fb130 msgr2=0x7fe7d40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3779936694 shutdown_connections 2023-07-14T11:21:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.383+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3779936694 wait complete. 2023-07-14T11:21:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 Processor -- start 2023-07-14T11:21:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 -- start start 2023-07-14T11:21:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d40ffaf0 0x7fe7d40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100ce0 0x7fe7d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7d4101670 0x7fe7d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d40735e0 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4073750 con 0x7fe7d4100ce0 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.384+0000 7fe7d93ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d419f8a0 con 0x7fe7d4101670 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d40ffaf0 0x7fe7d40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100ce0 0x7fe7d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100ce0 0x7fe7d4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46676/0 (socket says 172.21.15.161:46676) 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d40ffaf0 0x7fe7d40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57540/0 (socket says 172.21.15.161:57540) 2023-07-14T11:21:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d27fc700 1 -- 172.21.15.161:0/3225211513 learned_addr learned my addr 172.21.15.161:0/3225211513 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d37fe700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7d4101670 0x7fe7d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d2ffd700 1 -- 172.21.15.161:0/3225211513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7d4101670 msgr2=0x7fe7d4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d2ffd700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7d4101670 0x7fe7d4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d2ffd700 1 -- 172.21.15.161:0/3225211513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100ce0 msgr2=0x7fe7d4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d2ffd700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100ce0 0x7fe7d4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.385+0000 7fe7d2ffd700 1 -- 172.21.15.161:0/3225211513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c4009520 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.386+0000 7fe7d2ffd700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d40ffaf0 0x7fe7d40742c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400e9c0 tx=0x7fe7c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:19.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.386+0000 7fe7bbfff700 1 -- 172.21.15.161:0/3225211513 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c400cb70 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.386+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c80095f0 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.386+0000 7fe7bbfff700 1 -- 172.21.15.161:0/3225211513 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7c400ee60 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.386+0000 7fe7bbfff700 1 -- 172.21.15.161:0/3225211513 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4018680 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.386+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d419ff00 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.387+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d410b310 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.389+0000 7fe7bbfff700 1 -- 172.21.15.161:0/3225211513 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe7c4010430 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.390+0000 7fe7bbfff700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7bc0a11e0 0x7fe7bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.391+0000 7fe7bbfff700 1 -- 172.21.15.161:0/3225211513 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe7c4014070 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.391+0000 7fe7d27fc700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7bc0a11e0 0x7fe7bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.391+0000 7fe7d27fc700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7bc0a11e0 0x7fe7bc0a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b2e0 tx=0x7fe7c8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:19.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.394+0000 7fe7bbfff700 1 -- 172.21.15.161:0/3225211513 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe7c4096470 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.594+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7d41069f0 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.595+0000 7fe7bbfff700 1 -- 172.21.15.161:0/3225211513 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7c400ce60 con 0x7fe7d40ffaf0 2023-07-14T11:21:19.597 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:21:19.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7bc0a11e0 msgr2=0x7fe7bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7bc0a11e0 0x7fe7bc0a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b2e0 tx=0x7fe7c8005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d40ffaf0 msgr2=0x7fe7d40742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d40ffaf0 0x7fe7d40742c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400e9c0 tx=0x7fe7c400ecd0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 shutdown_connections 2023-07-14T11:21:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d40ffaf0 0x7fe7d40742c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe7bc0a11e0 0x7fe7bc0a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe7d4101670 0x7fe7d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 --2- 172.21.15.161:0/3225211513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100ce0 0x7fe7d4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 >> 172.21.15.161:0/3225211513 conn(0x7fe7d40fb130 msgr2=0x7fe7d40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 shutdown_connections 2023-07-14T11:21:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.597+0000 7fe7d93ff700 1 -- 172.21.15.161:0/3225211513 wait complete. 2023-07-14T11:21:19.618 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:21:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 -- 172.21.15.161:0/604212615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8100ce0 msgr2=0x7f5be8101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 --2- 172.21.15.161:0/604212615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8100ce0 0x7f5be8101130 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009990 tx=0x7f5bd4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 -- 172.21.15.161:0/604212615 shutdown_connections 2023-07-14T11:21:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 --2- 172.21.15.161:0/604212615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5be8101670 0x7f5be8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 --2- 172.21.15.161:0/604212615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 0x7f5be80fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 --2- 172.21.15.161:0/604212615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8100ce0 0x7f5be8101130 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 -- 172.21.15.161:0/604212615 >> 172.21.15.161:0/604212615 conn(0x7f5be80fb130 msgr2=0x7f5be80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:19.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 -- 172.21.15.161:0/604212615 shutdown_connections 2023-07-14T11:21:19.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.820+0000 7f5beefff700 1 -- 172.21.15.161:0/604212615 wait complete. 2023-07-14T11:21:19.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 Processor -- start 2023-07-14T11:21:19.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 -- start start 2023-07-14T11:21:19.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 0x7f5be8195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5be8100ce0 0x7f5be8195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8101670 0x7f5be819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be819d3c0 con 0x7f5be8101670 2023-07-14T11:21:19.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be819d500 con 0x7f5be80ffaf0 2023-07-14T11:21:19.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.821+0000 7f5beefff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be819d690 con 0x7f5be8100ce0 2023-07-14T11:21:19.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5bed59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8101670 0x7f5be819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 0x7f5be8195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 0x7f5be8195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46692/0 (socket says 172.21.15.161:46692) 2023-07-14T11:21:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5be7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5be8100ce0 0x7f5be8195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 -- 172.21.15.161:0/113610603 learned_addr learned my addr 172.21.15.161:0/113610603 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 -- 172.21.15.161:0/113610603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5be8100ce0 msgr2=0x7f5be8195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5be8100ce0 0x7f5be8195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 -- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8101670 msgr2=0x7f5be819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8101670 0x7f5be819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 -- 172.21.15.161:0/113610603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40095f0 con 0x7f5be80ffaf0 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5bed59c700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8101670 0x7f5be819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5be7fff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5be8100ce0 0x7f5be8195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.822+0000 7f5becd9b700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 0x7f5be8195840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e970 tx=0x7f5bdc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.823+0000 7f5be5ffb700 1 -- 172.21.15.161:0/113610603 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc009790 con 0x7f5be80ffaf0 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.823+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be819d970 con 0x7f5be80ffaf0 2023-07-14T11:21:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.823+0000 7f5be5ffb700 1 -- 172.21.15.161:0/113610603 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bdc00c7e0 con 0x7f5be80ffaf0 2023-07-14T11:21:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.823+0000 7f5be5ffb700 1 -- 172.21.15.161:0/113610603 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc003c70 con 0x7f5be80ffaf0 2023-07-14T11:21:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.823+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be819ded0 con 0x7f5be80ffaf0 2023-07-14T11:21:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.824+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be8106a20 con 0x7f5be80ffaf0 2023-07-14T11:21:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.826+0000 7f5be5ffb700 1 -- 172.21.15.161:0/113610603 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5bdc00cc50 con 0x7f5be80ffaf0 2023-07-14T11:21:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.827+0000 7f5be5ffb700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5bd80a11e0 0x7f5bd80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.827+0000 7f5be5ffb700 1 -- 172.21.15.161:0/113610603 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5bdc014070 con 0x7f5be80ffaf0 2023-07-14T11:21:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.827+0000 7f5be7fff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5bd80a11e0 0x7f5bd80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.828+0000 7f5be7fff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5bd80a11e0 0x7f5bd80a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b2e0 tx=0x7f5bd4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:19.829+0000 7f5be5ffb700 1 -- 172.21.15.161:0/113610603 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5bdc096380 con 0x7f5be80ffaf0 2023-07-14T11:21:19.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:19 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2823377748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:19.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:19 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3225211513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:20.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.029+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5be802cb00 con 0x7f5be80ffaf0 2023-07-14T11:21:20.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.030+0000 7f5be5ffb700 1 -- 172.21.15.161:0/113610603 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5bdc003dd0 con 0x7f5be80ffaf0 2023-07-14T11:21:20.032 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:20.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5bd80a11e0 msgr2=0x7f5bd80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5bd80a11e0 0x7f5bd80a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b2e0 tx=0x7f5bd4005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 msgr2=0x7f5be8195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 0x7f5be8195840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e970 tx=0x7f5bdc00ec80 comp rx=0 tx=0).stop 2023-07-14T11:21:20.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 shutdown_connections 2023-07-14T11:21:20.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be8101670 0x7f5be819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5bd80a11e0 0x7f5bd80a3690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5be80ffaf0 0x7f5be8195840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 --2- 172.21.15.161:0/113610603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5be8100ce0 0x7f5be8195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 >> 172.21.15.161:0/113610603 conn(0x7f5be80fb130 msgr2=0x7f5be80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 shutdown_connections 2023-07-14T11:21:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.032+0000 7f5beefff700 1 -- 172.21.15.161:0/113610603 wait complete. 2023-07-14T11:21:20.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:21:20.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:19 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2823377748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:20.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:19 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3225211513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:20.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:19 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2823377748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:20.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:19 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3225211513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:20.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 -- 172.21.15.161:0/1426305413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 msgr2=0x7f5c5c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 --2- 172.21.15.161:0/1426305413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0fff10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009990 tx=0x7f5c4c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 -- 172.21.15.161:0/1426305413 shutdown_connections 2023-07-14T11:21:20.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 --2- 172.21.15.161:0/1426305413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0fff10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 --2- 172.21.15.161:0/1426305413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c5c101680 0x7f5c5c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 --2- 172.21.15.161:0/1426305413 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c5c100cf0 0x7f5c5c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 -- 172.21.15.161:0/1426305413 >> 172.21.15.161:0/1426305413 conn(0x7f5c5c0fb0e0 msgr2=0x7f5c5c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:20.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 -- 172.21.15.161:0/1426305413 shutdown_connections 2023-07-14T11:21:20.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.250+0000 7f5c629ff700 1 -- 172.21.15.161:0/1426305413 wait complete. 2023-07-14T11:21:20.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.251+0000 7f5c629ff700 1 Processor -- start 2023-07-14T11:21:20.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.251+0000 7f5c629ff700 1 -- start start 2023-07-14T11:21:20.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c629ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c629ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c5c100cf0 0x7f5c5c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c629ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c5c101680 0x7f5c5c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c629ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c0735e0 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c629ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c073750 con 0x7f5c5c101680 2023-07-14T11:21:20.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c629ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c19f880 con 0x7f5c5c100cf0 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57590/0 (socket says 172.21.15.161:57590) 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3446180842 learned_addr learned my addr 172.21.15.161:0/3446180842 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5b7fe700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c5c100cf0 0x7f5c5c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c60f9c700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c5c101680 0x7f5c5c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3446180842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c5c100cf0 msgr2=0x7f5c5c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c5c100cf0 0x7f5c5c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3446180842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c5c101680 msgr2=0x7f5c5c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c5c101680 0x7f5c5c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.252+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3446180842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c0095f0 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.253+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0742c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c005740 tx=0x7f5c4c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:20.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.253+0000 7f5c597fa700 1 -- 172.21.15.161:0/3446180842 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c01d070 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.253+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5c19fb00 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.253+0000 7f5c597fa700 1 -- 172.21.15.161:0/3446180842 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c4c021410 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.253+0000 7f5c597fa700 1 -- 172.21.15.161:0/3446180842 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c017650 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.253+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c19ff30 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.254+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5c107050 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.256+0000 7f5c597fa700 1 -- 172.21.15.161:0/3446180842 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5c4c021580 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.257+0000 7f5c597fa700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c440a1220 0x7f5c440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.257+0000 7f5c597fa700 1 -- 172.21.15.161:0/3446180842 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5c4c0ce850 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.258+0000 7f5c5b7fe700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c440a1220 0x7f5c440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.258+0000 7f5c5b7fe700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c440a1220 0x7f5c440a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009c60 tx=0x7f5c50009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:20.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.259+0000 7f5c597fa700 1 -- 172.21.15.161:0/3446180842 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c4c097bb0 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.501+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c5c02c850 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.502+0000 7f5c597fa700 1 -- 172.21.15.161:0/3446180842 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f5c4c09b1d0 con 0x7f5c5c0ffb00 2023-07-14T11:21:20.505 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:20.505 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:21:20.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c440a1220 msgr2=0x7f5c440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c440a1220 0x7f5c440a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009c60 tx=0x7f5c50009260 comp rx=0 tx=0).stop 2023-07-14T11:21:20.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 msgr2=0x7f5c5c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0742c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c005740 tx=0x7f5c4c005770 comp rx=0 tx=0).stop 2023-07-14T11:21:20.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 shutdown_connections 2023-07-14T11:21:20.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c5c0ffb00 0x7f5c5c0742c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c440a1220 0x7f5c440a36d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c5c101680 0x7f5c5c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 --2- 172.21.15.161:0/3446180842 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c5c100cf0 0x7f5c5c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 >> 172.21.15.161:0/3446180842 conn(0x7f5c5c0fb0e0 msgr2=0x7f5c5c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:20.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 shutdown_connections 2023-07-14T11:21:20.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.505+0000 7f5c629ff700 1 -- 172.21.15.161:0/3446180842 wait complete. 2023-07-14T11:21:20.509 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:21:20.526 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:20.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 -- 172.21.15.161:0/3065872076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618101680 msgr2=0x7f7618107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 --2- 172.21.15.161:0/3065872076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618101680 0x7f7618107f40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f761000b0a0 tx=0x7f761000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 -- 172.21.15.161:0/3065872076 shutdown_connections 2023-07-14T11:21:20.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 --2- 172.21.15.161:0/3065872076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618101680 0x7f7618107f40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 --2- 172.21.15.161:0/3065872076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180ffb00 0x7f76180fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 --2- 172.21.15.161:0/3065872076 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7618100cf0 0x7f7618101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 -- 172.21.15.161:0/3065872076 >> 172.21.15.161:0/3065872076 conn(0x7f76180fb0e0 msgr2=0x7f76180fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:20.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 -- 172.21.15.161:0/3065872076 shutdown_connections 2023-07-14T11:21:20.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.721+0000 7f761cdff700 1 -- 172.21.15.161:0/3065872076 wait complete. 2023-07-14T11:21:20.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.722+0000 7f761cdff700 1 Processor -- start 2023-07-14T11:21:20.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.722+0000 7f761cdff700 1 -- start start 2023-07-14T11:21:20.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f761cdff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180ffb00 0x7f76180742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f761cdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618100cf0 0x7f7618072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f761cdff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7618101680 0x7f7618072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f761cdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76180735e0 con 0x7f7618100cf0 2023-07-14T11:21:20.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f761cdff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618073750 con 0x7f76180ffb00 2023-07-14T11:21:20.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f761cdff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761819f9a0 con 0x7f7618101680 2023-07-14T11:21:20.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618100cf0 0x7f7618072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618100cf0 0x7f7618072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57610/0 (socket says 172.21.15.161:57610) 2023-07-14T11:21:20.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 -- 172.21.15.161:0/2708199077 learned_addr learned my addr 172.21.15.161:0/2708199077 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:20.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f761659c700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180ffb00 0x7f76180742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7616d9d700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7618101680 0x7f7618072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 -- 172.21.15.161:0/2708199077 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7618101680 msgr2=0x7f7618072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7618101680 0x7f7618072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 -- 172.21.15.161:0/2708199077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180ffb00 msgr2=0x7f76180742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180ffb00 0x7f76180742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.723+0000 7f7615d9b700 1 -- 172.21.15.161:0/2708199077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7608009520 con 0x7f7618100cf0 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.724+0000 7f7615d9b700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618100cf0 0x7f7618072900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f760c00e9a0 tx=0x7f760c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.724+0000 7f76077fe700 1 -- 172.21.15.161:0/2708199077 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c009790 con 0x7f7618100cf0 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.724+0000 7f76077fe700 1 -- 172.21.15.161:0/2708199077 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f760c00cbd0 con 0x7f7618100cf0 2023-07-14T11:21:20.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.724+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7610009d00 con 0x7f7618100cf0 2023-07-14T11:21:20.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.724+0000 7f76077fe700 1 -- 172.21.15.161:0/2708199077 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c018680 con 0x7f7618100cf0 2023-07-14T11:21:20.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.724+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76181a0050 con 0x7f7618100cf0 2023-07-14T11:21:20.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.725+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761810b330 con 0x7f7618100cf0 2023-07-14T11:21:20.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.727+0000 7f76077fe700 1 -- 172.21.15.161:0/2708199077 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f760c00cd40 con 0x7f7618100cf0 2023-07-14T11:21:20.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.728+0000 7f76077fe700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76000a12a0 0x7f76000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:20.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.729+0000 7f76077fe700 1 -- 172.21.15.161:0/2708199077 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f760c014070 con 0x7f7618100cf0 2023-07-14T11:21:20.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.729+0000 7f761659c700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76000a12a0 0x7f76000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:20.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.729+0000 7f761659c700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76000a12a0 0x7f76000a3750 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7618100b50 tx=0x7f7608009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:20.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.732+0000 7f76077fe700 1 -- 172.21.15.161:0/2708199077 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f760c051790 con 0x7f7618100cf0 2023-07-14T11:21:20.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.934+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f761810b550 con 0x7f7618100cf0 2023-07-14T11:21:20.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.936+0000 7f76077fe700 1 -- 172.21.15.161:0/2708199077 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7f760c0962b0 con 0x7f7618100cf0 2023-07-14T11:21:20.939 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:20.939 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:20.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76000a12a0 msgr2=0x7f76000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76000a12a0 0x7f76000a3750 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7618100b50 tx=0x7f7608009260 comp rx=0 tx=0).stop 2023-07-14T11:21:20.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618100cf0 msgr2=0x7f7618072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618100cf0 0x7f7618072900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f760c00e9a0 tx=0x7f760c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 shutdown_connections 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f76000a12a0 0x7f76000a3750 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7618101680 0x7f7618072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76180ffb00 0x7f76180742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 --2- 172.21.15.161:0/2708199077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7618100cf0 0x7f7618072900 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 >> 172.21.15.161:0/2708199077 conn(0x7f76180fb0e0 msgr2=0x7f76180fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:20.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 shutdown_connections 2023-07-14T11:21:20.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:20.939+0000 7f761cdff700 1 -- 172.21.15.161:0/2708199077 wait complete. 2023-07-14T11:21:20.958 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:20 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/113610603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:20.958 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:20 smithi161 ceph-mon[95526]: pgmap v325: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:20.958 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:20 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3446180842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:20.960 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:21:21.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 -- 172.21.15.161:0/4206454989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11600ffb00 msgr2=0x7f11600fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 --2- 172.21.15.161:0/4206454989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11600ffb00 0x7f11600fff10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1150009990 tx=0x7f1150009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 -- 172.21.15.161:0/4206454989 shutdown_connections 2023-07-14T11:21:21.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 --2- 172.21.15.161:0/4206454989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11600ffb00 0x7f11600fff10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 --2- 172.21.15.161:0/4206454989 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1160101680 0x7f1160107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 --2- 172.21.15.161:0/4206454989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1160100cf0 0x7f1160101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 -- 172.21.15.161:0/4206454989 >> 172.21.15.161:0/4206454989 conn(0x7f11600fb0e0 msgr2=0x7f11600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 -- 172.21.15.161:0/4206454989 shutdown_connections 2023-07-14T11:21:21.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.157+0000 7f11649ff700 1 -- 172.21.15.161:0/4206454989 wait complete. 2023-07-14T11:21:21.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 Processor -- start 2023-07-14T11:21:21.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 -- start start 2023-07-14T11:21:21.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11600ffb00 0x7f1160195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1160100cf0 0x7f1160195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1160101680 0x7f116019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019d3c0 con 0x7f1160101680 2023-07-14T11:21:21.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019d500 con 0x7f11600ffb00 2023-07-14T11:21:21.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.158+0000 7f11649ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019d690 con 0x7f1160100cf0 2023-07-14T11:21:21.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1160101680 0x7f116019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1160101680 0x7f116019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57618/0 (socket says 172.21.15.161:57618) 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 -- 172.21.15.161:0/4237012504 learned_addr learned my addr 172.21.15.161:0/4237012504 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115e59c700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11600ffb00 0x7f1160195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f1157fff700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1160100cf0 0x7f1160195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 -- 172.21.15.161:0/4237012504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1160100cf0 msgr2=0x7f1160195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1160100cf0 0x7f1160195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 -- 172.21.15.161:0/4237012504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11600ffb00 msgr2=0x7f1160195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11600ffb00 0x7f1160195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 -- 172.21.15.161:0/4237012504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500095f0 con 0x7f1160101680 2023-07-14T11:21:21.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115e59c700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11600ffb00 0x7f1160195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.159+0000 7f115ed9d700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1160101680 0x7f116019cdf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1158007ee0 tx=0x7f115800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.160+0000 7f11577fe700 1 -- 172.21.15.161:0/4237012504 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158017070 con 0x7f1160101680 2023-07-14T11:21:21.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.160+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116019d970 con 0x7f1160101680 2023-07-14T11:21:21.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.160+0000 7f11577fe700 1 -- 172.21.15.161:0/4237012504 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115800f040 con 0x7f1160101680 2023-07-14T11:21:21.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.160+0000 7f11577fe700 1 -- 172.21.15.161:0/4237012504 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158005550 con 0x7f1160101680 2023-07-14T11:21:21.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.160+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116019de80 con 0x7f1160101680 2023-07-14T11:21:21.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.161+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116010b320 con 0x7f1160101680 2023-07-14T11:21:21.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.163+0000 7f11577fe700 1 -- 172.21.15.161:0/4237012504 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f115800db20 con 0x7f1160101680 2023-07-14T11:21:21.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.164+0000 7f11577fe700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f114c0a1220 0x7f114c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.164+0000 7f11577fe700 1 -- 172.21.15.161:0/4237012504 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f11580cc4d0 con 0x7f1160101680 2023-07-14T11:21:21.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.164+0000 7f115e59c700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f114c0a1220 0x7f114c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.165+0000 7f115e59c700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f114c0a1220 0x7f114c0a36d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f115000b2e0 tx=0x7f1150005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:20 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/113610603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:21.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:20 smithi175 ceph-mon[129081]: pgmap v325: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:21.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:20 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3446180842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:21.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.168+0000 7f11577fe700 1 -- 172.21.15.161:0/4237012504 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1158095770 con 0x7f1160101680 2023-07-14T11:21:21.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:20 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/113610603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:21.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:20 smithi182 ceph-mon[129366]: pgmap v325: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:21.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:20 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3446180842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:21.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.413+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11601069e0 con 0x7f1160101680 2023-07-14T11:21:21.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.414+0000 7f11577fe700 1 -- 172.21.15.161:0/4237012504 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f1158098d90 con 0x7f1160101680 2023-07-14T11:21:21.416 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:21.417 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:21:21.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f114c0a1220 msgr2=0x7f114c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f114c0a1220 0x7f114c0a36d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f115000b2e0 tx=0x7f1150005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1160101680 msgr2=0x7f116019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1160101680 0x7f116019cdf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1158007ee0 tx=0x7f115800d3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 shutdown_connections 2023-07-14T11:21:21.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1160101680 0x7f116019cdf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f114c0a1220 0x7f114c0a36d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11600ffb00 0x7f1160195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 --2- 172.21.15.161:0/4237012504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1160100cf0 0x7f1160195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 >> 172.21.15.161:0/4237012504 conn(0x7f11600fb0e0 msgr2=0x7f11600fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 shutdown_connections 2023-07-14T11:21:21.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.416+0000 7f11649ff700 1 -- 172.21.15.161:0/4237012504 wait complete. 2023-07-14T11:21:21.420 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:21:21.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T11:19:41.650966+0000', 'modified': '2023-07-14T11:20:12.668182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24575, 'mds_1': 24544, 'mds_2': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/4098726991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 4098726991}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 4098726991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.182:6835/1383844371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.182:6834', 'nonce': 1383844371}, {'type': 'v1', 'addr': '172.21.15.182:6835', 'nonce': 1383844371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.182:6837/1326049792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.182:6836', 'nonce': 1326049792}, {'type': 'v1', 'addr': '172.21.15.182:6837', 'nonce': 1326049792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.175:6837/4186191407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 4186191407}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 4186191407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.182:6839/392865750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.182:6838', 'nonce': 392865750}, {'type': 'v1', 'addr': '172.21.15.182:6839', 'nonce': 392865750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6839/2144664422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 2144664422}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 2144664422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T11:21:21.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T11:21:21.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-14T11:21:21.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 -- 172.21.15.161:0/1359989674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648101660 msgr2=0x7f8648107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 --2- 172.21.15.161:0/1359989674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648101660 0x7f8648107f20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f864000b0a0 tx=0x7f864000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 -- 172.21.15.161:0/1359989674 shutdown_connections 2023-07-14T11:21:21.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 --2- 172.21.15.161:0/1359989674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648101660 0x7f8648107f20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 --2- 172.21.15.161:0/1359989674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86480ffae0 0x7f86480ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 --2- 172.21.15.161:0/1359989674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8648100cd0 0x7f8648101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 -- 172.21.15.161:0/1359989674 >> 172.21.15.161:0/1359989674 conn(0x7f86480fb0e0 msgr2=0x7f86480fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 -- 172.21.15.161:0/1359989674 shutdown_connections 2023-07-14T11:21:21.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.635+0000 7f864f3ff700 1 -- 172.21.15.161:0/1359989674 wait complete. 2023-07-14T11:21:21.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.636+0000 7f864f3ff700 1 Processor -- start 2023-07-14T11:21:21.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.636+0000 7f864f3ff700 1 -- start start 2023-07-14T11:21:21.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864f3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480ffae0 0x7f8648197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864f3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8648100cd0 0x7f86481980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864f3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8648101660 0x7f864819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864f3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864819f6f0 con 0x7f86480ffae0 2023-07-14T11:21:21.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864f3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864819f830 con 0x7f8648101660 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864f3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864819f9c0 con 0x7f8648100cd0 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480ffae0 0x7f8648197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480ffae0 0x7f8648197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57634/0 (socket says 172.21.15.161:57634) 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 -- 172.21.15.161:0/2729572558 learned_addr learned my addr 172.21.15.161:0/2729572558 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864c99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8648100cd0 0x7f86481980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d99c700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8648101660 0x7f864819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 -- 172.21.15.161:0/2729572558 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8648100cd0 msgr2=0x7f86481980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8648100cd0 0x7f86481980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 -- 172.21.15.161:0/2729572558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8648101660 msgr2=0x7f864819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8648101660 0x7f864819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.637+0000 7f864d19b700 1 -- 172.21.15.161:0/2729572558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8644009520 con 0x7f86480ffae0 2023-07-14T11:21:21.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.638+0000 7f864d19b700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480ffae0 0x7f8648197b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f864400ea50 tx=0x7f864400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.638+0000 7f863e7fc700 1 -- 172.21.15.161:0/2729572558 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400cbd0 con 0x7f86480ffae0 2023-07-14T11:21:21.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.638+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8638009520 con 0x7f86480ffae0 2023-07-14T11:21:21.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.638+0000 7f863e7fc700 1 -- 172.21.15.161:0/2729572558 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864400cd30 con 0x7f86480ffae0 2023-07-14T11:21:21.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.638+0000 7f863e7fc700 1 -- 172.21.15.161:0/2729572558 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644018680 con 0x7f86480ffae0 2023-07-14T11:21:21.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.638+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640009d00 con 0x7f86480ffae0 2023-07-14T11:21:21.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.641+0000 7f863e7fc700 1 -- 172.21.15.161:0/2729572558 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8644010430 con 0x7f86480ffae0 2023-07-14T11:21:21.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.642+0000 7f8662c69b80 1 Processor -- start 2023-07-14T11:21:21.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f863e7fc700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f86340a11e0 0x7f86340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f8662c69b80 1 -- start start 2023-07-14T11:21:21.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f863e7fc700 1 -- 172.21.15.161:0/2729572558 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8644014070 con 0x7f86480ffae0 2023-07-14T11:21:21.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f864d99c700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f86340a11e0 0x7f86340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db08fc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db08fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db08fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f8662c69b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db0801c60 con 0x562db08fc480 2023-07-14T11:21:21.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f8662c69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db08549f0 con 0x562db08fcdd0 2023-07-14T11:21:21.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.643+0000 7f8662c69b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db084de80 con 0x562db08fdc20 2023-07-14T11:21:21.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db08fc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db08fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864c99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db08fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db08fe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46012/0 (socket says 172.21.15.161:46012) 2023-07-14T11:21:21.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d99c700 1 -- 172.21.15.161:0/2744293697 learned_addr learned my addr 172.21.15.161:0/2744293697 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d99c700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f86340a11e0 0x7f86340a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f864000b780 tx=0x7f8640006210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d19b700 1 -- 172.21.15.161:0/2744293697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 msgr2=0x562db08fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d19b700 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db08fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d19b700 1 -- 172.21.15.161:0/2744293697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 msgr2=0x562db08fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d19b700 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db08fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d19b700 1 -- 172.21.15.161:0/2744293697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db067bd90 con 0x562db08fc480 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864c99a700 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db08fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f864d19b700 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db08fc890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f86440ca910 tx=0x7f8644099c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d02c4c9cd9a865 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f86337fe700 1 -- 172.21.15.161:0/2744293697 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864409a210 con 0x562db08fc480 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f86337fe700 1 -- 172.21.15.161:0/2744293697 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864409e240 con 0x562db08fc480 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.644+0000 7f86337fe700 1 -- 172.21.15.161:0/2744293697 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440a21d0 con 0x562db08fc480 2023-07-14T11:21:21.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 -- 172.21.15.161:0/2744293697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 msgr2=0x562db08fc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db08fc890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f86440ca910 tx=0x7f8644099c70 comp rx=0 tx=0).stop 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 -- 172.21.15.161:0/2744293697 shutdown_connections 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db08fc890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db08fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 --2- 172.21.15.161:0/2744293697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db08fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 -- 172.21.15.161:0/2744293697 >> 172.21.15.161:0/2744293697 conn(0x562db08f1d50 msgr2=0x562db08f7180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 -- 172.21.15.161:0/2744293697 shutdown_connections 2023-07-14T11:21:21.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.645+0000 7f8662c69b80 1 -- 172.21.15.161:0/2744293697 wait complete. 2023-07-14T11:21:21.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 -- start start 2023-07-14T11:21:21.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 Processor -- start 2023-07-14T11:21:21.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fc480 0x562db0906190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db09066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db090d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db07eda70 con 0x562db08fdc20 2023-07-14T11:21:21.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db07ed770 con 0x562db08fcdd0 2023-07-14T11:21:21.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.646+0000 7f8662c69b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db06d18d0 con 0x562db08fc480 2023-07-14T11:21:21.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f8662c69b80 10 client.0 _ll_register_callbacks cb 0x562db07eb710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:21.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db090d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864c99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db09066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db090d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57650/0 (socket says 172.21.15.161:57650) 2023-07-14T11:21:21.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 -- 172.21.15.161:0/1686642380 learned_addr learned my addr 172.21.15.161:0/1686642380 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d19b700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fc480 0x562db0906190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 -- 172.21.15.161:0/1686642380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fc480 msgr2=0x562db0906190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fc480 0x562db0906190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 -- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 msgr2=0x562db09066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db09066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d99c700 1 -- 172.21.15.161:0/1686642380 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8638009a70 con 0x562db08fdc20 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864c99a700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db09066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.647+0000 7f864d19b700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fc480 0x562db0906190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f864d99c700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db090d710 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8640009fd0 tx=0x7f86400099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b140c3d560fc22d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8630ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8630ff9700 1 -- 172.21.15.161:0/1686642380 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864000e040 con 0x562db08fdc20 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8630ff9700 1 -- 172.21.15.161:0/1686642380 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f864001c850 con 0x562db08fdc20 2023-07-14T11:21:21.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562db07a74a0 con 0x562db08fdc20 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8630ff9700 1 -- 172.21.15.161:0/1686642380 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8640003e60 con 0x562db08fdc20 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8662c69b80 10 client.15138 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f8640009d00 con 0x562db08fdc20 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8630ff9700 1 -- 172.21.15.161:0/1686642380 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562db07a74a0 con 0x562db08fdc20 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8630ff9700 1 -- 172.21.15.161:0/1686642380 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8640005680 con 0x562db08fdc20 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.648+0000 7f8662c69b80 10 client.15138 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.649+0000 7f8662c69b80 10 client.15138 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T11:21:21.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.649+0000 7f8662c69b80 20 client.15138 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.649+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db09112f0 0x562db0910700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.649+0000 7f8662c69b80 4 client.15138 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:21:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.649+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562db0846e60 con 0x562db09112f0 2023-07-14T11:21:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.649+0000 7f864d19b700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db09112f0 0x562db0910700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.650+0000 7f864d19b700 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db09112f0 0x562db0910700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12d1af3b3d486ea8 server_cookie=6e291549ddc6bea2 in_seq=0 out_seq=0 2023-07-14T11:21:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.650+0000 7f8630ff9700 10 client.15138 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8630ff9700 1 -- 172.21.15.161:0/1686642380 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562db0846e60 con 0x562db09112f0 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8630ff9700 10 client.15138 handle_command_reply: tid=0 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 client.15138 shutdown 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 msgr2=0x562db090d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db090d710 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8640009fd0 tx=0x7f86400099e0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 shutdown_connections 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db09112f0 0x562db0910700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db090d710 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fc480 0x562db0906190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1686642380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db09066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 >> 172.21.15.161:0/1686642380 conn(0x562db08f1d50 msgr2=0x562db08f6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8630ff9700 0 client.15138 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:21.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 shutdown_connections 2023-07-14T11:21:21.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 1 -- 172.21.15.161:0/1686642380 wait complete. 2023-07-14T11:21:21.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.651+0000 7f8662c69b80 20 client.15138 trim_cache size 0 max 16384 2023-07-14T11:21:21.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.669+0000 7f8662c69b80 1 Processor -- start 2023-07-14T11:21:21.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.669+0000 7f8662c69b80 1 -- start start 2023-07-14T11:21:21.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fc480 0x562db08f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fcdd0 0x562db090d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db0904e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f8662c69b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db0907380 con 0x562db08fdc20 2023-07-14T11:21:21.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f8662c69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db0904790 con 0x562db08fc480 2023-07-14T11:21:21.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f8662c69b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db081c790 con 0x562db08fcdd0 2023-07-14T11:21:21.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db0904e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fc480 0x562db08f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db0904e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57652/0 (socket says 172.21.15.161:57652) 2023-07-14T11:21:21.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d99c700 1 -- 172.21.15.161:0/1408833293 learned_addr learned my addr 172.21.15.161:0/1408833293 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864c99a700 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fcdd0 0x562db090d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d19b700 1 -- 172.21.15.161:0/1408833293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fcdd0 msgr2=0x562db090d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d19b700 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fcdd0 0x562db090d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d19b700 1 -- 172.21.15.161:0/1408833293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 msgr2=0x562db0904e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d19b700 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db0904e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d19b700 1 -- 172.21.15.161:0/1408833293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db08f35d0 con 0x562db08fc480 2023-07-14T11:21:21.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.670+0000 7f864d99c700 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db0904e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.671+0000 7f864d19b700 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fc480 0x562db08f6860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f864400c5b0 tx=0x7f864400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a787dfeaf4e76d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.671+0000 7f8633fff700 1 -- 172.21.15.161:0/1408833293 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440b6070 con 0x562db08fc480 2023-07-14T11:21:21.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.671+0000 7f8633fff700 1 -- 172.21.15.161:0/1408833293 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864409a210 con 0x562db08fc480 2023-07-14T11:21:21.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.671+0000 7f8633fff700 1 -- 172.21.15.161:0/1408833293 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644099960 con 0x562db08fc480 2023-07-14T11:21:21.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.671+0000 7f8662c69b80 1 -- 172.21.15.161:0/1408833293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fc480 msgr2=0x562db08f6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.671+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fc480 0x562db08f6860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f864400c5b0 tx=0x7f864400eef0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 -- 172.21.15.161:0/1408833293 shutdown_connections 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fdc20 0x562db0904e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fc480 0x562db08f6860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 --2- 172.21.15.161:0/1408833293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fcdd0 0x562db090d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 -- 172.21.15.161:0/1408833293 >> 172.21.15.161:0/1408833293 conn(0x562db08f1d50 msgr2=0x562db0904080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 -- 172.21.15.161:0/1408833293 shutdown_connections 2023-07-14T11:21:21.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 -- 172.21.15.161:0/1408833293 wait complete. 2023-07-14T11:21:21.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 -- start start 2023-07-14T11:21:21.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.672+0000 7f8662c69b80 1 Processor -- start 2023-07-14T11:21:21.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db090a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db090ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f8662c69b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db090bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f8662c69b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db090c330 con 0x562db08fc480 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f8662c69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db090c4a0 con 0x562db08fcdd0 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f8662c69b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db090c660 con 0x562db08fdc20 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f8662c69b80 10 client.0 _ll_register_callbacks cb 0x562db07eb710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db090a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db090a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57662/0 (socket says 172.21.15.161:57662) 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 -- 172.21.15.161:0/3742468551 learned_addr learned my addr 172.21.15.161:0/3742468551 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864c99a700 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db090ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d99c700 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db090bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 -- 172.21.15.161:0/3742468551 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 msgr2=0x562db090bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db090bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 -- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 msgr2=0x562db090ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db090ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.673+0000 7f864d19b700 1 -- 172.21.15.161:0/3742468551 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640009d00 con 0x562db08fc480 2023-07-14T11:21:21.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f864d19b700 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db090a8a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f86440a2130 tx=0x7f8644005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfb95aed4533689 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8630ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8630ff9700 1 -- 172.21.15.161:0/3742468551 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440b6070 con 0x562db08fc480 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8630ff9700 1 -- 172.21.15.161:0/3742468551 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8644099700 con 0x562db08fc480 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f864409e240 con 0x562db08fc480 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8630ff9700 1 -- 172.21.15.161:0/3742468551 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86440998c0 con 0x562db08fc480 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8662c69b80 10 client.15144 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x562db067bd90 con 0x562db08fc480 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8630ff9700 1 -- 172.21.15.161:0/3742468551 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8644066020 con 0x562db08fc480 2023-07-14T11:21:21.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.674+0000 7f8630ff9700 1 -- 172.21.15.161:0/3742468551 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f864401eb90 con 0x562db08fc480 2023-07-14T11:21:21.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.675+0000 7f8662c69b80 10 client.15144 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:21.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.675+0000 7f8662c69b80 10 client.15144 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T11:21:21.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.675+0000 7f8662c69b80 20 client.15144 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:21.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.675+0000 7f8662c69b80 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db096a3e0 0x562db0967160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.675+0000 7f8662c69b80 4 client.15144 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-07-14T11:21:21.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.675+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562db0809000 con 0x562db096a3e0 2023-07-14T11:21:21.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.675+0000 7f864c99a700 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db096a3e0 0x562db0967160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f864c99a700 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db096a3e0 0x562db0967160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff4fda4d29ce9aed server_cookie=f80ed7494c78a854 in_seq=0 out_seq=0 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8630ff9700 10 client.15144 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8630ff9700 1 -- 172.21.15.161:0/3742468551 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562db0809000 con 0x562db096a3e0 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8630ff9700 10 client.15144 handle_command_reply: tid=0 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 client.15144 shutdown 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 msgr2=0x562db090a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db090a8a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f86440a2130 tx=0x7f8644005b60 comp rx=0 tx=0).stop 2023-07-14T11:21:21.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 shutdown_connections 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562db096a3e0 0x562db0967160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562db08fc480 0x562db090a8a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562db08fdc20 0x562db090bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8630ff9700 0 client.15144 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 --2- 172.21.15.161:0/3742468551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562db08fcdd0 0x562db090ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 >> 172.21.15.161:0/3742468551 conn(0x562db08f1d50 msgr2=0x562db08f6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 shutdown_connections 2023-07-14T11:21:21.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 1 -- 172.21.15.161:0/3742468551 wait complete. 2023-07-14T11:21:21.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.676+0000 7f8662c69b80 20 client.15144 trim_cache size 0 max 16384 2023-07-14T11:21:21.687 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:21:21.687 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:21:21.687 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 2, 2023-07-14T11:21:21.687 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24533, 2023-07-14T11:21:21.687 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2023-07-14T11:21:21.688 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2023-07-14T11:21:21.688 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2023-07-14T11:21:21.688 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 69.967658208000003, 2023-07-14T11:21:21.688 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 22, 2023-07-14T11:21:21.688 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 92, 2023-07-14T11:21:21.688 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T11:21:21.688 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 116.449867365 2023-07-14T11:21:21.689 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:21:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f86340a11e0 msgr2=0x7f86340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f86340a11e0 0x7f86340a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f864000b780 tx=0x7f8640006210 comp rx=0 tx=0).stop 2023-07-14T11:21:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480ffae0 msgr2=0x7f8648197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480ffae0 0x7f8648197b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f864400ea50 tx=0x7f864400ed60 comp rx=0 tx=0).stop 2023-07-14T11:21:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 shutdown_connections 2023-07-14T11:21:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480ffae0 0x7f8648197b70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f86340a11e0 0x7f86340a3690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8648101660 0x7f864819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 --2- 172.21.15.161:0/2729572558 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8648100cd0 0x7f86481980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 >> 172.21.15.161:0/2729572558 conn(0x7f86480fb0e0 msgr2=0x7f8648106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 shutdown_connections 2023-07-14T11:21:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.677+0000 7f864f3ff700 1 -- 172.21.15.161:0/2729572558 wait complete. 2023-07-14T11:21:21.698 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T11:21:21.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 -- 172.21.15.161:0/3123494026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90072f10 msgr2=0x7f7d90073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 --2- 172.21.15.161:0/3123494026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90072f10 0x7f7d90073380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009990 tx=0x7f7d7c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 -- 172.21.15.161:0/3123494026 shutdown_connections 2023-07-14T11:21:21.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 --2- 172.21.15.161:0/3123494026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90107980 0x7f7d90109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 --2- 172.21.15.161:0/3123494026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d900744f0 0x7f7d90072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 --2- 172.21.15.161:0/3123494026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90072f10 0x7f7d90073380 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 -- 172.21.15.161:0/3123494026 >> 172.21.15.161:0/3123494026 conn(0x7f7d900fb130 msgr2=0x7f7d900fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 -- 172.21.15.161:0/3123494026 shutdown_connections 2023-07-14T11:21:21.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.895+0000 7f7d96bff700 1 -- 172.21.15.161:0/3123494026 wait complete. 2023-07-14T11:21:21.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.896+0000 7f7d96bff700 1 Processor -- start 2023-07-14T11:21:21.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.896+0000 7f7d96bff700 1 -- start start 2023-07-14T11:21:21.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d96bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90072f10 0x7f7d90197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d96bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d900744f0 0x7f7d90197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d96bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90107980 0x7f7d9019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d96bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9019f5b0 con 0x7f7d90107980 2023-07-14T11:21:21.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d96bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9019f6f0 con 0x7f7d90072f10 2023-07-14T11:21:21.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d96bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9019f880 con 0x7f7d900744f0 2023-07-14T11:21:21.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9519c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90107980 0x7f7d9019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9499b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90072f10 0x7f7d90197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d900744f0 0x7f7d90197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9499b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90072f10 0x7f7d90197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46818/0 (socket says 172.21.15.161:46818) 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9519c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90107980 0x7f7d9019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57692/0 (socket says 172.21.15.161:57692) 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9499b700 1 -- 172.21.15.161:0/2521930307 learned_addr learned my addr 172.21.15.161:0/2521930307 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9519c700 1 -- 172.21.15.161:0/2521930307 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d900744f0 msgr2=0x7f7d90197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9519c700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d900744f0 0x7f7d90197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9519c700 1 -- 172.21.15.161:0/2521930307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90072f10 msgr2=0x7f7d90197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9519c700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90072f10 0x7f7d90197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.897+0000 7f7d9519c700 1 -- 172.21.15.161:0/2521930307 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0095f0 con 0x7f7d90107980 2023-07-14T11:21:21.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.898+0000 7f7d9519c700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90107980 0x7f7d9019efe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800efc0 tx=0x7f7d8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.898+0000 7f7d8dffb700 1 -- 172.21.15.161:0/2521930307 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88018070 con 0x7f7d90107980 2023-07-14T11:21:21.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.898+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9019fb00 con 0x7f7d90107980 2023-07-14T11:21:21.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.898+0000 7f7d8dffb700 1 -- 172.21.15.161:0/2521930307 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d8800f040 con 0x7f7d90107980 2023-07-14T11:21:21.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.898+0000 7f7d8dffb700 1 -- 172.21.15.161:0/2521930307 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800cd40 con 0x7f7d90107980 2023-07-14T11:21:21.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.898+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d901a0010 con 0x7f7d90107980 2023-07-14T11:21:21.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.901+0000 7f7d8dffb700 1 -- 172.21.15.161:0/2521930307 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7d88003a00 con 0x7f7d90107980 2023-07-14T11:21:21.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.902+0000 7f7daa607b80 1 Processor -- start 2023-07-14T11:21:21.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.902+0000 7f7daa607b80 1 -- start start 2023-07-14T11:21:21.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.902+0000 7f7d8dffb700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d800a1260 0x7f7d800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7d8dffb700 1 -- 172.21.15.161:0/2521930307 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7d880cc980 con 0x7f7d90107980 2023-07-14T11:21:21.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b3c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b3d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d800a1260 0x7f7d800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7daa607b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885a41c60 con 0x561885b3cdd0 2023-07-14T11:21:21.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7daa607b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885a949f0 con 0x561885b3dc20 2023-07-14T11:21:21.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7daa607b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885a8de80 con 0x561885b3c480 2023-07-14T11:21:21.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b3d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b3d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57696/0 (socket says 172.21.15.161:57696) 2023-07-14T11:21:21.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7d8ffff700 1 -- 172.21.15.161:0/3513143185 learned_addr learned my addr 172.21.15.161:0/3513143185 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7d9499b700 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b3c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.903+0000 7f7d9519c700 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b3e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d8ffff700 1 -- 172.21.15.161:0/3513143185 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 msgr2=0x561885b3c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b3c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d8ffff700 1 -- 172.21.15.161:0/3513143185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 msgr2=0x561885b3e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b3e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d8ffff700 1 -- 172.21.15.161:0/3513143185 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618858bbd90 con 0x561885b3cdd0 2023-07-14T11:21:21.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d9519c700 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d800a1260 0x7f7d800a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00b680 tx=0x7f7d7c00b870 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b3d690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c015000 tx=0x7f7d7c029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3c78df2f58c38e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d7affd700 1 -- 172.21.15.161:0/3513143185 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c005760 con 0x561885b3cdd0 2023-07-14T11:21:21.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d7affd700 1 -- 172.21.15.161:0/3513143185 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d7c00ba80 con 0x561885b3cdd0 2023-07-14T11:21:21.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.904+0000 7f7d7affd700 1 -- 172.21.15.161:0/3513143185 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c029950 con 0x561885b3cdd0 2023-07-14T11:21:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 -- 172.21.15.161:0/3513143185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 msgr2=0x561885b3d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b3d690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c015000 tx=0x7f7d7c029000 comp rx=0 tx=0).stop 2023-07-14T11:21:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 -- 172.21.15.161:0/3513143185 shutdown_connections 2023-07-14T11:21:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b3c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3513143185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b3d690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 -- 172.21.15.161:0/3513143185 >> 172.21.15.161:0/3513143185 conn(0x561885b31d50 msgr2=0x561885b37180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 -- 172.21.15.161:0/3513143185 shutdown_connections 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 -- 172.21.15.161:0/3513143185 wait complete. 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.905+0000 7f7daa607b80 1 -- start start 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 1 Processor -- start 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885a2da70 con 0x561885b3cdd0 2023-07-14T11:21:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885a2d770 con 0x561885b3dc20 2023-07-14T11:21:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618859118d0 con 0x561885b3c480 2023-07-14T11:21:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7daa607b80 10 client.0 _ll_register_callbacks cb 0x561885a2b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57700/0 (socket says 172.21.15.161:57700) 2023-07-14T11:21:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7d9499b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.906+0000 7f7d8ffff700 1 -- 172.21.15.161:0/2473785969 learned_addr learned my addr 172.21.15.161:0/2473785969 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d9519c700 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d8ffff700 1 -- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 msgr2=0x561885b460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d8ffff700 1 -- 172.21.15.161:0/2473785969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 msgr2=0x561885b4d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d8ffff700 1 -- 172.21.15.161:0/2473785969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d88084040 con 0x561885b3cdd0 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d9499b700 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b465f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00bc30 tx=0x7f7d7c0295a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23528a1172ad51d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d6ffff700 1 -- 172.21.15.161:0/2473785969 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c004b70 con 0x561885b3cdd0 2023-07-14T11:21:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618859e74a0 con 0x561885b3cdd0 2023-07-14T11:21:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.907+0000 7f7d6ffff700 1 -- 172.21.15.161:0/2473785969 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7d7c03a020 con 0x561885b3cdd0 2023-07-14T11:21:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7d6ffff700 1 -- 172.21.15.161:0/2473785969 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d7c004cd0 con 0x561885b3cdd0 2023-07-14T11:21:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 10 client.15156 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f7d7c020040 con 0x561885b3cdd0 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7d6ffff700 1 -- 172.21.15.161:0/2473785969 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618859e74a0 con 0x561885b3cdd0 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7d6ffff700 1 -- 172.21.15.161:0/2473785969 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d7c00fa50 con 0x561885b3cdd0 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 10 client.15156 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 10 client.15156 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 20 client.15156 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885b51450 0x561885b50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 4 client.15156 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:21:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.908+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561885a86e60 con 0x561885b51450 2023-07-14T11:21:21.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.909+0000 7f7d9499b700 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885b51450 0x561885b50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.909+0000 7f7d9499b700 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885b51450 0x561885b50560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91938a1914531718 server_cookie=71f5ab64a7b9b5b0 in_seq=0 out_seq=0 2023-07-14T11:21:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.909+0000 7f7d6ffff700 10 client.15156 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.910+0000 7f7d6ffff700 1 -- 172.21.15.161:0/2473785969 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561885a86e60 con 0x561885b51450 2023-07-14T11:21:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.910+0000 7f7d6ffff700 10 client.15156 handle_command_reply: tid=0 2023-07-14T11:21:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.910+0000 7f7daa607b80 1 client.15156 shutdown 2023-07-14T11:21:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 msgr2=0x561885b465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b465f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00bc30 tx=0x7f7d7c0295a0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 shutdown_connections 2023-07-14T11:21:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885b51450 0x561885b50560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3c480 0x561885b460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7d6ffff700 0 client.15156 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 --2- 172.21.15.161:0/2473785969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3cdd0 0x561885b465f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 >> 172.21.15.161:0/2473785969 conn(0x561885b31d50 msgr2=0x561885b36860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 shutdown_connections 2023-07-14T11:21:21.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 1 -- 172.21.15.161:0/2473785969 wait complete. 2023-07-14T11:21:21.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.911+0000 7f7daa607b80 20 client.15156 trim_cache size 0 max 16384 2023-07-14T11:21:21.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 Processor -- start 2023-07-14T11:21:21.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 -- start start 2023-07-14T11:21:21.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3c480 0x561885b50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3dc20 0x561885b36860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885b38660 con 0x561885b3dc20 2023-07-14T11:21:21.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885b52140 con 0x561885b3c480 2023-07-14T11:21:21.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.929+0000 7f7daa607b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885a86e60 con 0x561885b3cdd0 2023-07-14T11:21:21.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3dc20 0x561885b36860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3dc20 0x561885b36860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57710/0 (socket says 172.21.15.161:57710) 2023-07-14T11:21:21.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 -- 172.21.15.161:0/1318910467 learned_addr learned my addr 172.21.15.161:0/1318910467 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9499b700 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3c480 0x561885b50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 -- 172.21.15.161:0/1318910467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 msgr2=0x561885b4d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 -- 172.21.15.161:0/1318910467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3c480 msgr2=0x561885b50560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3c480 0x561885b50560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 -- 172.21.15.161:0/1318910467 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618858a4c10 con 0x561885b3dc20 2023-07-14T11:21:21.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9499b700 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3c480 0x561885b50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:21.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d9519c700 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3dc20 0x561885b36860 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d880998a0 tx=0x7f7d88099540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d079c2cc11e4194d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d7b7fe700 1 -- 172.21.15.161:0/1318910467 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8809d940 con 0x561885b3dc20 2023-07-14T11:21:21.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d7b7fe700 1 -- 172.21.15.161:0/1318910467 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d880a1cd0 con 0x561885b3dc20 2023-07-14T11:21:21.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.930+0000 7f7d7b7fe700 1 -- 172.21.15.161:0/1318910467 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88098e00 con 0x561885b3dc20 2023-07-14T11:21:21.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 -- 172.21.15.161:0/1318910467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3dc20 msgr2=0x561885b36860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3dc20 0x561885b36860 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d880998a0 tx=0x7f7d88099540 comp rx=0 tx=0).stop 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 -- 172.21.15.161:0/1318910467 shutdown_connections 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3dc20 0x561885b36860 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3c480 0x561885b50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 --2- 172.21.15.161:0/1318910467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 -- 172.21.15.161:0/1318910467 >> 172.21.15.161:0/1318910467 conn(0x561885b31d50 msgr2=0x561885b44dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 -- 172.21.15.161:0/1318910467 shutdown_connections 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.931+0000 7f7daa607b80 1 -- 172.21.15.161:0/1318910467 wait complete. 2023-07-14T11:21:21.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 -- start start 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 Processor -- start 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3c480 0x561885b4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885b4c430 con 0x561885b3c480 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885b4c5a0 con 0x561885b3dc20 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885b4c760 con 0x561885b3cdd0 2023-07-14T11:21:21.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7daa607b80 10 client.0 _ll_register_callbacks cb 0x7f7d8800cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:21.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7d9499b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3c480 0x561885b4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7d9499b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3c480 0x561885b4a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57714/0 (socket says 172.21.15.161:57714) 2023-07-14T11:21:21.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7d9499b700 1 -- 172.21.15.161:0/3638350663 learned_addr learned my addr 172.21.15.161:0/3638350663 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:21.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.932+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d9519c700 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d9499b700 1 -- 172.21.15.161:0/3638350663 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 msgr2=0x561885b4aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d9499b700 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d9499b700 1 -- 172.21.15.161:0/3638350663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 msgr2=0x561885b4bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d9499b700 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d9499b700 1 -- 172.21.15.161:0/3638350663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d840120f0 con 0x561885b3c480 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d9499b700 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3c480 0x561885b4a9a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561885b46c40 tx=0x7f7d8401f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54621ab0980565ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d6ffff700 1 -- 172.21.15.161:0/3638350663 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84017030 con 0x561885b3c480 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7d6ffff700 1 -- 172.21.15.161:0/3638350663 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7d8401fa40 con 0x561885b3c480 2023-07-14T11:21:21.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.933+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d7c00ba80 con 0x561885b3c480 2023-07-14T11:21:21.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7d6ffff700 1 -- 172.21.15.161:0/3638350663 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d8401fc00 con 0x561885b3c480 2023-07-14T11:21:21.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 10 client.15162 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:21.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f7d901a0010 con 0x561885b3c480 2023-07-14T11:21:21.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7d6ffff700 1 -- 172.21.15.161:0/3638350663 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d7c00ba80 con 0x561885b3c480 2023-07-14T11:21:21.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7d6ffff700 1 -- 172.21.15.161:0/3638350663 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d84005380 con 0x561885b3c480 2023-07-14T11:21:21.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:21.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 10 client.15162 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T11:21:21.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 20 client.15162 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:21.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885baa640 0x561885ba6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:21.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 4 client.15162 mds_command: new command op to 24544 tid=0[{"prefix": "status"}] 2023-07-14T11:21:21.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.934+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561885a49000 con 0x561885baa640 2023-07-14T11:21:21.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885baa640 0x561885ba6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:21.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7d8ffff700 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885baa640 0x561885ba6fb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=272ed24d4926dcbf server_cookie=e3b571c2abad256e in_seq=0 out_seq=0 2023-07-14T11:21:21.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7d6ffff700 10 client.15162 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:21.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7d6ffff700 1 -- 172.21.15.161:0/3638350663 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561885a49000 con 0x561885baa640 2023-07-14T11:21:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7d6ffff700 10 client.15162 handle_command_reply: tid=0 2023-07-14T11:21:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 client.15162 shutdown 2023-07-14T11:21:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3c480 msgr2=0x561885b4a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3c480 0x561885b4a9a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561885b46c40 tx=0x7f7d8401f6e0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 shutdown_connections 2023-07-14T11:21:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561885baa640 0x561885ba6fb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561885b3c480 0x561885b4a9a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561885b3dc20 0x561885b4bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7d6ffff700 0 client.15162 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:21.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 --2- 172.21.15.161:0/3638350663 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561885b3cdd0 0x561885b4aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.935+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 >> 172.21.15.161:0/3638350663 conn(0x561885b31d50 msgr2=0x561885b50560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 shutdown_connections 2023-07-14T11:21:21.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7daa607b80 1 -- 172.21.15.161:0/3638350663 wait complete. 2023-07-14T11:21:21.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7daa607b80 20 client.15162 trim_cache size 0 max 16384 2023-07-14T11:21:21.947 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:21:21.947 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:21:21.947 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 1, 2023-07-14T11:21:21.947 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24544, 2023-07-14T11:21:21.947 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2023-07-14T11:21:21.947 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2023-07-14T11:21:21.947 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2023-07-14T11:21:21.948 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 71.233427535000004, 2023-07-14T11:21:21.948 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 22, 2023-07-14T11:21:21.948 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 92, 2023-07-14T11:21:21.948 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T11:21:21.948 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 114.512168722 2023-07-14T11:21:21.948 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:21:21.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d800a1260 msgr2=0x7f7d800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d800a1260 0x7f7d800a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00b680 tx=0x7f7d7c00b870 comp rx=0 tx=0).stop 2023-07-14T11:21:21.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90107980 msgr2=0x7f7d9019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:21.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90107980 0x7f7d9019efe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800efc0 tx=0x7f7d8800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 shutdown_connections 2023-07-14T11:21:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d90107980 0x7f7d9019efe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d800a1260 0x7f7d800a3710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90072f10 0x7f7d90197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 --2- 172.21.15.161:0/2521930307 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d900744f0 0x7f7d90197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 >> 172.21.15.161:0/2521930307 conn(0x7f7d900fb130 msgr2=0x7f7d900fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 shutdown_connections 2023-07-14T11:21:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:21.936+0000 7f7d96bff700 1 -- 172.21.15.161:0/2521930307 wait complete. 2023-07-14T11:21:21.954 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:21 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2708199077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:21.954 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:21 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4237012504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:21.956 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T11:21:22.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 -- 172.21.15.161:0/1692784375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc0ffaf0 msgr2=0x7f84cc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 --2- 172.21.15.161:0/1692784375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc0ffaf0 0x7f84cc0fff00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009990 tx=0x7f84c8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 -- 172.21.15.161:0/1692784375 shutdown_connections 2023-07-14T11:21:22.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 --2- 172.21.15.161:0/1692784375 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84cc101670 0x7f84cc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 --2- 172.21.15.161:0/1692784375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc0ffaf0 0x7f84cc0fff00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 --2- 172.21.15.161:0/1692784375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc100ce0 0x7f84cc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 -- 172.21.15.161:0/1692784375 >> 172.21.15.161:0/1692784375 conn(0x7f84cc0fb130 msgr2=0x7f84cc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:22.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 -- 172.21.15.161:0/1692784375 shutdown_connections 2023-07-14T11:21:22.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.155+0000 7f84d45ff700 1 -- 172.21.15.161:0/1692784375 wait complete. 2023-07-14T11:21:22.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.156+0000 7f84d45ff700 1 Processor -- start 2023-07-14T11:21:22.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.156+0000 7f84d45ff700 1 -- start start 2023-07-14T11:21:22.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d45ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84cc0ffaf0 0x7f84cc195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d45ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc100ce0 0x7f84cc195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d45ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc101670 0x7f84cc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d45ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc19d3b0 con 0x7f84cc101670 2023-07-14T11:21:22.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d45ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc19d4f0 con 0x7f84cc100ce0 2023-07-14T11:21:22.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d45ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc19d680 con 0x7f84cc0ffaf0 2023-07-14T11:21:22.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d2b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc101670 0x7f84cc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d2b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc101670 0x7f84cc19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57728/0 (socket says 172.21.15.161:57728) 2023-07-14T11:21:22.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d2b9c700 1 -- 172.21.15.161:0/3221479819 learned_addr learned my addr 172.21.15.161:0/3221479819 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:22.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d239b700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84cc0ffaf0 0x7f84cc195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc100ce0 0x7f84cc195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d2b9c700 1 -- 172.21.15.161:0/3221479819 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84cc0ffaf0 msgr2=0x7f84cc195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.157+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84cc0ffaf0 0x7f84cc195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84d2b9c700 1 -- 172.21.15.161:0/3221479819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc100ce0 msgr2=0x7f84cc195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc100ce0 0x7f84cc195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84d2b9c700 1 -- 172.21.15.161:0/3221479819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80095f0 con 0x7f84cc101670 2023-07-14T11:21:22.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc101670 0x7f84cc19cde0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ef30 tx=0x7f84c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:22.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84c37fe700 1 -- 172.21.15.161:0/3221479819 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4018070 con 0x7f84cc101670 2023-07-14T11:21:22.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc19d900 con 0x7f84cc101670 2023-07-14T11:21:22.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84c37fe700 1 -- 172.21.15.161:0/3221479819 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c400f040 con 0x7f84cc101670 2023-07-14T11:21:22.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84c37fe700 1 -- 172.21.15.161:0/3221479819 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400cd00 con 0x7f84cc101670 2023-07-14T11:21:22.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.158+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc19de60 con 0x7f84cc101670 2023-07-14T11:21:22.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.161+0000 7f84c37fe700 1 -- 172.21.15.161:0/3221479819 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f84c4003a00 con 0x7f84cc101670 2023-07-14T11:21:22.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.162+0000 7f84e7fa5b80 1 Processor -- start 2023-07-14T11:21:22.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84c37fe700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f84b80a11e0 0x7f84b80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84c37fe700 1 -- 172.21.15.161:0/3221479819 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f84c40cc930 con 0x7f84cc101670 2023-07-14T11:21:22.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84e7fa5b80 1 -- start start 2023-07-14T11:21:22.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f84b80a11e0 0x7f84b80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3898b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3898c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e3898d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e38890c60 con 0x558e3898bdd0 2023-07-14T11:21:22.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e388e39f0 con 0x558e3898cc20 2023-07-14T11:21:22.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.163+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e388dce80 con 0x558e3898b480 2023-07-14T11:21:22.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3898c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d239b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3898b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3898c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57732/0 (socket says 172.21.15.161:57732) 2023-07-14T11:21:22.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 -- 172.21.15.161:0/1235450612 learned_addr learned my addr 172.21.15.161:0/1235450612 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:22.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d239b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3898b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46126/0 (socket says 172.21.15.161:46126) 2023-07-14T11:21:22.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e3898d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f84b80a11e0 0x7f84b80a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009cb0 tx=0x7f84bc011720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:22.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 -- 172.21.15.161:0/1235450612 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 msgr2=0x558e3898b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3898b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 -- 172.21.15.161:0/1235450612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 msgr2=0x558e3898d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e3898d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 -- 172.21.15.161:0/1235450612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e3870ad90 con 0x558e3898bdd0 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e3898d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.164+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3898c690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84bc011f90 tx=0x7f84bc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feb801f7ca4f310 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84c0bfe700 1 -- 172.21.15.161:0/1235450612 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc01dd40 con 0x558e3898bdd0 2023-07-14T11:21:22.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84c0bfe700 1 -- 172.21.15.161:0/1235450612 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84bc0258a0 con 0x558e3898bdd0 2023-07-14T11:21:22.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84c0bfe700 1 -- 172.21.15.161:0/1235450612 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00f730 con 0x558e3898bdd0 2023-07-14T11:21:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:21 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2708199077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:21 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4237012504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:22.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/1235450612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 msgr2=0x558e3898c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3898c690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84bc011f90 tx=0x7f84bc01b000 comp rx=0 tx=0).stop 2023-07-14T11:21:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/1235450612 shutdown_connections 2023-07-14T11:21:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e3898d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3898b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/1235450612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3898c690 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/1235450612 >> 172.21.15.161:0/1235450612 conn(0x558e38980d50 msgr2=0x558e38986180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:22.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.165+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/1235450612 shutdown_connections 2023-07-14T11:21:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.166+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/1235450612 wait complete. 2023-07-14T11:21:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.166+0000 7f84e7fa5b80 1 -- start start 2023-07-14T11:21:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.166+0000 7f84e7fa5b80 1 Processor -- start 2023-07-14T11:21:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898b480 0x558e389950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e389955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898cc20 0x558e3899c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3887ca70 con 0x558e3898b480 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3887c770 con 0x558e3898bdd0 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e387608d0 con 0x558e3898cc20 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84e7fa5b80 10 client.0 _ll_register_callbacks cb 0x558e3887a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d239b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898b480 0x558e389950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e389955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e389955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46874/0 (socket says 172.21.15.161:46874) 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 -- 172.21.15.161:0/960083522 learned_addr learned my addr 172.21.15.161:0/960083522 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:22.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d239b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898b480 0x558e389950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57748/0 (socket says 172.21.15.161:57748) 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898cc20 0x558e3899c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 -- 172.21.15.161:0/960083522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898cc20 msgr2=0x558e3899c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898cc20 0x558e3899c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 -- 172.21.15.161:0/960083522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898b480 msgr2=0x558e389950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898b480 0x558e389950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.167+0000 7f84d1b9a700 1 -- 172.21.15.161:0/960083522 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc19de60 con 0x558e3898bdd0 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e389955f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84bc01dfd0 tx=0x7f84bc025a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe76c42054d703b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:22.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:21:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84b5ffb700 1 -- 172.21.15.161:0/960083522 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0253e0 con 0x558e3898bdd0 2023-07-14T11:21:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e388364a0 con 0x558e3898bdd0 2023-07-14T11:21:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84b5ffb700 1 -- 172.21.15.161:0/960083522 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f84bc025540 con 0x558e3898bdd0 2023-07-14T11:21:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84b5ffb700 1 -- 172.21.15.161:0/960083522 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84bc025700 con 0x558e3898bdd0 2023-07-14T11:21:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84e7fa5b80 10 client.25211 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.168+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f84bc011cd0 con 0x558e3898bdd0 2023-07-14T11:21:22.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84b5ffb700 1 -- 172.21.15.161:0/960083522 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e388364a0 con 0x558e3898bdd0 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84b5ffb700 1 -- 172.21.15.161:0/960083522 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84bc026440 con 0x558e3898bdd0 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84e7fa5b80 10 client.25211 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84e7fa5b80 10 client.25211 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84e7fa5b80 20 client.25211 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389a0320 0x558e3899f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84e7fa5b80 4 client.25211 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e388d5e60 con 0x558e389a0320 2023-07-14T11:21:22.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.169+0000 7f84d239b700 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389a0320 0x558e3899f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.170+0000 7f84b5ffb700 10 client.25211 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.170+0000 7f84d239b700 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389a0320 0x558e3899f730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48252a925f889673 server_cookie=2b7231a67e6b3d7e in_seq=0 out_seq=0 2023-07-14T11:21:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84b5ffb700 1 -- 172.21.15.161:0/960083522 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e388d5e60 con 0x558e389a0320 2023-07-14T11:21:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84b5ffb700 10 client.25211 handle_command_reply: tid=0 2023-07-14T11:21:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 client.25211 shutdown 2023-07-14T11:21:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 msgr2=0x558e389955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e389955f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84bc01dfd0 tx=0x7f84bc025a10 comp rx=0 tx=0).stop 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 shutdown_connections 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389a0320 0x558e3899f730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898b480 0x558e389950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898cc20 0x558e3899c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/960083522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e389955f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84b5ffb700 0 client.25211 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 >> 172.21.15.161:0/960083522 conn(0x558e38980d50 msgr2=0x558e38985860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:22.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 shutdown_connections 2023-07-14T11:21:22.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/960083522 wait complete. 2023-07-14T11:21:22.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.171+0000 7f84e7fa5b80 20 client.25211 trim_cache size 0 max 16384 2023-07-14T11:21:22.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.189+0000 7f84e7fa5b80 1 Processor -- start 2023-07-14T11:21:22.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.189+0000 7f84e7fa5b80 1 -- start start 2023-07-14T11:21:22.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3899cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3899c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e38985860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e388d5e60 con 0x558e3898bdd0 2023-07-14T11:21:22.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e388364a0 con 0x558e3898cc20 2023-07-14T11:21:22.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e388ab790 con 0x558e3898b480 2023-07-14T11:21:22.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3899c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3899c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57754/0 (socket says 172.21.15.161:57754) 2023-07-14T11:21:22.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 -- 172.21.15.161:0/471043993 learned_addr learned my addr 172.21.15.161:0/471043993 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:22.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d239b700 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3899cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e38985860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 -- 172.21.15.161:0/471043993 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 msgr2=0x558e3899cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3899cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 -- 172.21.15.161:0/471043993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 msgr2=0x558e38985860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e38985860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.190+0000 7f84d1b9a700 1 -- 172.21.15.161:0/471043993 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e386f3c10 con 0x558e3898bdd0 2023-07-14T11:21:22.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3899c5f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009b70 tx=0x7f84bc025e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f485a7451358dfb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:22.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84c13ff700 1 -- 172.21.15.161:0/471043993 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc021070 con 0x558e3898bdd0 2023-07-14T11:21:22.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84c13ff700 1 -- 172.21.15.161:0/471043993 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84bc026440 con 0x558e3898bdd0 2023-07-14T11:21:22.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84c13ff700 1 -- 172.21.15.161:0/471043993 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0056f0 con 0x558e3898bdd0 2023-07-14T11:21:22.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/471043993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 msgr2=0x558e3899c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3899c5f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009b70 tx=0x7f84bc025e90 comp rx=0 tx=0).stop 2023-07-14T11:21:22.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/471043993 shutdown_connections 2023-07-14T11:21:22.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898cc20 0x558e38985860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.191+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e3899cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.192+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/471043993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898bdd0 0x558e3899c5f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.192+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/471043993 >> 172.21.15.161:0/471043993 conn(0x558e38980d50 msgr2=0x558e389a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:22.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.192+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/471043993 shutdown_connections 2023-07-14T11:21:22.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.192+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/471043993 wait complete. 2023-07-14T11:21:22.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.192+0000 7f84e7fa5b80 1 -- start start 2023-07-14T11:21:22.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.192+0000 7f84e7fa5b80 1 Processor -- start 2023-07-14T11:21:22.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e389977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e38997d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84e7fa5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898cc20 0x558e3899ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3899b080 con 0x558e3898cc20 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3899b1f0 con 0x558e3898bdd0 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84e7fa5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3899b3b0 con 0x558e3898b480 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84e7fa5b80 10 client.0 _ll_register_callbacks cb 0x7f84c4009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d2b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898cc20 0x558e3899ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d1b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e38997d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d2b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898cc20 0x558e3899ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57758/0 (socket says 172.21.15.161:57758) 2023-07-14T11:21:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d2b9c700 1 -- 172.21.15.161:0/3361172784 learned_addr learned my addr 172.21.15.161:0/3361172784 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d239b700 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e389977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d1b9a700 1 -- 172.21.15.161:0/3361172784 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 msgr2=0x558e389977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e389977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d1b9a700 1 -- 172.21.15.161:0/3361172784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898cc20 msgr2=0x558e3899ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898cc20 0x558e3899ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d1b9a700 1 -- 172.21.15.161:0/3361172784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc011cd0 con 0x558e3898bdd0 2023-07-14T11:21:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.193+0000 7f84d2b9c700 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898cc20 0x558e3899ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84d1b9a700 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e38997d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84bc005360 tx=0x7f84bc026bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38a7b40631e438 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:21:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84b5ffb700 1 -- 172.21.15.161:0/3361172784 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc021050 con 0x558e3898bdd0 2023-07-14T11:21:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84bc0258a0 con 0x558e3898bdd0 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84b5ffb700 1 -- 172.21.15.161:0/3361172784 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f84bc0255f0 con 0x558e3898bdd0 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84b5ffb700 1 -- 172.21.15.161:0/3361172784 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84bc0265b0 con 0x558e3898bdd0 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84e7fa5b80 10 client.25217 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558e38731200 con 0x558e3898bdd0 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84b5ffb700 1 -- 172.21.15.161:0/3361172784 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84bc0258a0 con 0x558e3898bdd0 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84b5ffb700 1 -- 172.21.15.161:0/3361172784 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84bc003cf0 con 0x558e3898bdd0 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.194+0000 7f84e7fa5b80 10 client.25217 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84e7fa5b80 10 client.25217 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84e7fa5b80 20 client.25217 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389f9200 0x558e389f6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84e7fa5b80 4 client.25217 mds_command: new command op to 24575 tid=0[{"prefix": "status"}] 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558e38898000 con 0x558e389f9200 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84d239b700 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389f9200 0x558e389f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84b5ffb700 10 client.25217 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.195+0000 7f84d239b700 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389f9200 0x558e389f6190 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7382f22865c677f0 server_cookie=3cb91e25c7f0ea92 in_seq=0 out_seq=0 2023-07-14T11:21:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84b5ffb700 1 -- 172.21.15.161:0/3361172784 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558e38898000 con 0x558e389f9200 2023-07-14T11:21:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84b5ffb700 10 client.25217 handle_command_reply: tid=0 2023-07-14T11:21:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 client.25217 shutdown 2023-07-14T11:21:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 msgr2=0x558e38997d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e38997d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84bc005360 tx=0x7f84bc026bc0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 shutdown_connections 2023-07-14T11:21:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558e389f9200 0x558e389f6190 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e3898cc20 0x558e3899ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84b5ffb700 0 client.25217 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558e3898b480 0x558e389977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 --2- 172.21.15.161:0/3361172784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558e3898bdd0 0x558e38997d00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 >> 172.21.15.161:0/3361172784 conn(0x558e38980d50 msgr2=0x558e38985860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:22.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 shutdown_connections 2023-07-14T11:21:22.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 1 -- 172.21.15.161:0/3361172784 wait complete. 2023-07-14T11:21:22.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.196+0000 7f84e7fa5b80 20 client.25217 trim_cache size 0 max 16384 2023-07-14T11:21:22.210 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:21:22.210 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:21:22.211 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2023-07-14T11:21:22.211 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24575, 2023-07-14T11:21:22.211 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2023-07-14T11:21:22.211 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2023-07-14T11:21:22.211 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2023-07-14T11:21:22.211 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 100.10037215600001, 2023-07-14T11:21:22.211 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 22, 2023-07-14T11:21:22.212 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 92, 2023-07-14T11:21:22.212 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T11:21:22.212 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 106.517415544 2023-07-14T11:21:22.212 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:21:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f84b80a11e0 msgr2=0x7f84b80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f84b80a11e0 0x7f84b80a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009cb0 tx=0x7f84bc011720 comp rx=0 tx=0).stop 2023-07-14T11:21:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc101670 msgr2=0x7f84cc19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc101670 0x7f84cc19cde0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ef30 tx=0x7f84c400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 shutdown_connections 2023-07-14T11:21:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84cc101670 0x7f84cc19cde0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f84b80a11e0 0x7f84b80a3690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84cc0ffaf0 0x7f84cc195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 --2- 172.21.15.161:0/3221479819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84cc100ce0 0x7f84cc195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 >> 172.21.15.161:0/3221479819 conn(0x7f84cc0fb130 msgr2=0x7f84cc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 shutdown_connections 2023-07-14T11:21:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:22.197+0000 7f84d45ff700 1 -- 172.21.15.161:0/3221479819 wait complete. 2023-07-14T11:21:22.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:21 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2708199077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:22.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:21 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4237012504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:22.217 INFO:tasks.cephadm:Setup complete, yielding 2023-07-14T11:21:22.218 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T11:21:22.227 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T11:21:22.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- bash -c 'ceph orch status' 2023-07-14T11:21:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:23.941 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:23 smithi161 ceph-mon[95526]: pgmap v326: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:23.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:23 smithi182 ceph-mon[129366]: pgmap v326: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:23 smithi175 ceph-mon[129081]: pgmap v326: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:24.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.567+0000 7f8e65940700 1 -- 172.21.15.161:0/813643081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e600ffb90 msgr2=0x7f8e600fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:24.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.567+0000 7f8e65940700 1 --2- 172.21.15.161:0/813643081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e600ffb90 0x7f8e600fffa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009990 tx=0x7f8e50009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.567+0000 7f8e65940700 1 -- 172.21.15.161:0/813643081 shutdown_connections 2023-07-14T11:21:24.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.567+0000 7f8e65940700 1 --2- 172.21.15.161:0/813643081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e600ffb90 0x7f8e600fffa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.567+0000 7f8e65940700 1 --2- 172.21.15.161:0/813643081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e60101720 0x7f8e60107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.567+0000 7f8e65940700 1 --2- 172.21.15.161:0/813643081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8e60100d90 0x7f8e601011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.567+0000 7f8e65940700 1 -- 172.21.15.161:0/813643081 >> 172.21.15.161:0/813643081 conn(0x7f8e600fb1a0 msgr2=0x7f8e600fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:24.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.568+0000 7f8e65940700 1 -- 172.21.15.161:0/813643081 shutdown_connections 2023-07-14T11:21:24.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.568+0000 7f8e65940700 1 -- 172.21.15.161:0/813643081 wait complete. 2023-07-14T11:21:24.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 Processor -- start 2023-07-14T11:21:24.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 -- start start 2023-07-14T11:21:24.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e600ffb90 0x7f8e60197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:24.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e60100d90 0x7f8e60198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:24.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8e60101720 0x7f8e6019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:24.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6019f660 con 0x7f8e60100d90 2023-07-14T11:21:24.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6019f7a0 con 0x7f8e600ffb90 2023-07-14T11:21:24.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e65940700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6019f930 con 0x7f8e60101720 2023-07-14T11:21:24.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e5e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e60100d90 0x7f8e60198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:24.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e5e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e60100d90 0x7f8e60198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57770/0 (socket says 172.21.15.161:57770) 2023-07-14T11:21:24.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e5e7fc700 1 -- 172.21.15.161:0/3372229273 learned_addr learned my addr 172.21.15.161:0/3372229273 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:24.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.569+0000 7f8e5effd700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e600ffb90 0x7f8e60197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:24.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5f7fe700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8e60101720 0x7f8e6019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:24.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5e7fc700 1 -- 172.21.15.161:0/3372229273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8e60101720 msgr2=0x7f8e6019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:24.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5e7fc700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8e60101720 0x7f8e6019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5e7fc700 1 -- 172.21.15.161:0/3372229273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e600ffb90 msgr2=0x7f8e60197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:24.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5e7fc700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e600ffb90 0x7f8e60197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5e7fc700 1 -- 172.21.15.161:0/3372229273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e500095f0 con 0x7f8e60100d90 2023-07-14T11:21:24.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5f7fe700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8e60101720 0x7f8e6019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:24.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e5e7fc700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e60100d90 0x7f8e60198010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400e9d0 tx=0x7f8e5400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:24.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e6493e700 1 -- 172.21.15.161:0/3372229273 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54009790 con 0x7f8e60100d90 2023-07-14T11:21:24.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6019fc10 con 0x7f8e60100d90 2023-07-14T11:21:24.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e6493e700 1 -- 172.21.15.161:0/3372229273 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5400cbd0 con 0x7f8e60100d90 2023-07-14T11:21:24.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.570+0000 7f8e6493e700 1 -- 172.21.15.161:0/3372229273 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54018680 con 0x7f8e60100d90 2023-07-14T11:21:24.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.571+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e601a0160 con 0x7f8e60100d90 2023-07-14T11:21:24.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.572+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e60192210 con 0x7f8e60100d90 2023-07-14T11:21:24.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.574+0000 7f8e6493e700 1 -- 172.21.15.161:0/3372229273 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8e5400cd40 con 0x7f8e60100d90 2023-07-14T11:21:24.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.575+0000 7f8e6493e700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8e480a11e0 0x7f8e480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:24.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.575+0000 7f8e6493e700 1 -- 172.21.15.161:0/3372229273 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8e54014070 con 0x7f8e60100d90 2023-07-14T11:21:24.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.575+0000 7f8e5effd700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8e480a11e0 0x7f8e480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:24.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.576+0000 7f8e5effd700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8e480a11e0 0x7f8e480a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000b2e0 tx=0x7f8e50005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:24.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.577+0000 7f8e6493e700 1 -- 172.21.15.161:0/3372229273 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8e54096a00 con 0x7f8e60100d90 2023-07-14T11:21:24.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.793+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e60060950 con 0x7f8e480a11e0 2023-07-14T11:21:24.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.794+0000 7f8e6493e700 1 -- 172.21.15.161:0/3372229273 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f8e60060950 con 0x7f8e480a11e0 2023-07-14T11:21:24.796 INFO:teuthology.orchestra.run.smithi161.stdout:Backend: cephadm 2023-07-14T11:21:24.797 INFO:teuthology.orchestra.run.smithi161.stdout:Available: Yes 2023-07-14T11:21:24.797 INFO:teuthology.orchestra.run.smithi161.stdout:Paused: No 2023-07-14T11:21:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.796+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8e480a11e0 msgr2=0x7f8e480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.796+0000 7f8e65940700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8e480a11e0 0x7f8e480a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000b2e0 tx=0x7f8e50005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.796+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e60100d90 msgr2=0x7f8e60198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.796+0000 7f8e65940700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e60100d90 0x7f8e60198010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400e9d0 tx=0x7f8e5400ece0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 shutdown_connections 2023-07-14T11:21:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8e480a11e0 0x7f8e480a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8e60101720 0x7f8e6019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e600ffb90 0x7f8e60197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 --2- 172.21.15.161:0/3372229273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e60100d90 0x7f8e60198010 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 >> 172.21.15.161:0/3372229273 conn(0x7f8e600fb1a0 msgr2=0x7f8e601066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 shutdown_connections 2023-07-14T11:21:24.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:24.797+0000 7f8e65940700 1 -- 172.21.15.161:0/3372229273 wait complete. 2023-07-14T11:21:25.247 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-14T11:21:25.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:25 smithi182 ceph-mon[129366]: pgmap v327: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:25.995 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:25 smithi161 ceph-mon[95526]: pgmap v327: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:26.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:25 smithi175 ceph-mon[129081]: pgmap v327: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2023-07-14T11:21:26.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:26 smithi161 ceph-mon[95526]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:26.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:26 smithi182 ceph-mon[129366]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:26 smithi175 ceph-mon[129081]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:27.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1162949172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8101720 msgr2=0x7f6ba8107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:27.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1162949172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8101720 0x7f6ba8107fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000b0a0 tx=0x7f6ba000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1162949172 shutdown_connections 2023-07-14T11:21:27.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1162949172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8101720 0x7f6ba8107fe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1162949172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ba80ffb90 0x7f6ba80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1162949172 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ba8100d90 0x7f6ba81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1162949172 >> 172.21.15.161:0/1162949172 conn(0x7f6ba80fb1a0 msgr2=0x7f6ba80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:27.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1162949172 shutdown_connections 2023-07-14T11:21:27.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.530+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1162949172 wait complete. 2023-07-14T11:21:27.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.531+0000 7f6bafc9a700 1 Processor -- start 2023-07-14T11:21:27.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.531+0000 7f6bafc9a700 1 -- start start 2023-07-14T11:21:27.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bafc9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ba80ffb90 0x7f6ba80742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:27.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bafc9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8100d90 0x7f6ba8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:27.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bafc9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ba8101720 0x7f6ba8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:27.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bafc9a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80735c0 con 0x7f6ba8100d90 2023-07-14T11:21:27.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bafc9a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8073730 con 0x7f6ba8101720 2023-07-14T11:21:27.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bafc9a700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819f9e0 con 0x7f6ba80ffb90 2023-07-14T11:21:27.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bad235700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8100d90 0x7f6ba8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:27.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bad235700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8100d90 0x7f6ba8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57790/0 (socket says 172.21.15.161:57790) 2023-07-14T11:21:27.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bad235700 1 -- 172.21.15.161:0/1625151222 learned_addr learned my addr 172.21.15.161:0/1625151222 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:27.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bada36700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ba80ffb90 0x7f6ba80742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:27.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.532+0000 7f6bae237700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ba8101720 0x7f6ba8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bad235700 1 -- 172.21.15.161:0/1625151222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ba80ffb90 msgr2=0x7f6ba80742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bad235700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ba80ffb90 0x7f6ba80742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bad235700 1 -- 172.21.15.161:0/1625151222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ba8101720 msgr2=0x7f6ba8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bad235700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ba8101720 0x7f6ba8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bad235700 1 -- 172.21.15.161:0/1625151222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0009d00 con 0x7f6ba8100d90 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bada36700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ba80ffb90 0x7f6ba80742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bae237700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ba8101720 0x7f6ba8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:27.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bad235700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8100d90 0x7f6ba8072900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9a0 tx=0x7f6b9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:27.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6b9effd700 1 -- 172.21.15.161:0/1625151222 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98009790 con 0x7f6ba8100d90 2023-07-14T11:21:27.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba819fcc0 con 0x7f6ba8100d90 2023-07-14T11:21:27.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6b9effd700 1 -- 172.21.15.161:0/1625151222 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b98010430 con 0x7f6ba8100d90 2023-07-14T11:21:27.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6b9effd700 1 -- 172.21.15.161:0/1625151222 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98018680 con 0x7f6ba8100d90 2023-07-14T11:21:27.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.533+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba81a01c0 con 0x7f6ba8100d90 2023-07-14T11:21:27.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.534+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba8107110 con 0x7f6ba8100d90 2023-07-14T11:21:27.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.536+0000 7f6b9effd700 1 -- 172.21.15.161:0/1625151222 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6b980105a0 con 0x7f6ba8100d90 2023-07-14T11:21:27.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.538+0000 7f6b9effd700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6b940a11e0 0x7f6b940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:27.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.538+0000 7f6b9effd700 1 -- 172.21.15.161:0/1625151222 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6b98014070 con 0x7f6ba8100d90 2023-07-14T11:21:27.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.538+0000 7f6bada36700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6b940a11e0 0x7f6b940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:27.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.538+0000 7f6bada36700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6b940a11e0 0x7f6b940a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8100bf0 tx=0x7f6ba4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:27.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.541+0000 7f6b9effd700 1 -- 172.21.15.161:0/1625151222 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6b98051790 con 0x7f6ba8100d90 2023-07-14T11:21:27.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.749+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ba8060950 con 0x7f6b940a11e0 2023-07-14T11:21:27.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.761+0000 7f6b9effd700 1 -- 172.21.15.161:0/1625151222 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f6ba8060950 con 0x7f6b940a11e0 2023-07-14T11:21:27.764 INFO:teuthology.orchestra.run.smithi161.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-14T11:21:27.764 INFO:teuthology.orchestra.run.smithi161.stdout:mds.a smithi161 running (114s) 22s ago 114s 13.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 a5edc65f8c88 2023-07-14T11:21:27.764 INFO:teuthology.orchestra.run.smithi161.stdout:mds.b smithi175 running (112s) 40s ago 112s 30.4M - 18.0.0-4757-g3fc0089c afd12e232ec0 aeddc20983dd 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.c smithi182 running (2m) 40s ago 2m 18.5M - 18.0.0-4757-g3fc0089c afd12e232ec0 c3bd4257c7ad 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.d smithi161 running (2m) 22s ago 2m 13.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 6f02dba4a649 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.e smithi175 running (2m) 40s ago 2m 16.2M - 18.0.0-4757-g3fc0089c afd12e232ec0 4f90790fb869 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.f smithi182 running (2m) 40s ago 2m 19.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 8f57f0ad8be0 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.g smithi161 running (117s) 22s ago 117s 13.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 34a8f12f858b 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.h smithi175 running (2m) 40s ago 2m 18.2M - 18.0.0-4757-g3fc0089c afd12e232ec0 cdafe6308568 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.i smithi182 running (2m) 40s ago 2m 20.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 f74abdfc7b1e 2023-07-14T11:21:27.765 INFO:teuthology.orchestra.run.smithi161.stdout:mds.j smithi161 running (2m) 22s ago 2m 13.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 d210d876b8a2 2023-07-14T11:21:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:mds.k smithi175 running (2m) 40s ago 2m 17.4M - 18.0.0-4757-g3fc0089c afd12e232ec0 27b662b4f2ee 2023-07-14T11:21:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:mds.l smithi182 running (2m) 40s ago 2m 16.4M - 18.0.0-4757-g3fc0089c afd12e232ec0 037286ff5c42 2023-07-14T11:21:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:mgr.x smithi161 *:9283,8765 running (9m) 22s ago 9m 521M - 18.0.0-4757-g3fc0089c afd12e232ec0 c3a8ad3e3e77 2023-07-14T11:21:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:mgr.y smithi175 *:8443,8765 running (7m) 40s ago 7m 448M - 18.0.0-4757-g3fc0089c afd12e232ec0 6979a9212f0e 2023-07-14T11:21:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:mgr.z smithi182 *:8443,8765 running (7m) 40s ago 7m 446M - 18.0.0-4757-g3fc0089c afd12e232ec0 9e68936b6858 2023-07-14T11:21:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:mon.a smithi161 running (9m) 22s ago 9m 93.7M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 0ecdc1bf6ec7 2023-07-14T11:21:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:mon.b smithi175 running (7m) 40s ago 7m 65.7M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 fcb95897c17b 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:mon.c smithi182 running (7m) 40s ago 7m 65.2M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 a08ba3aa5161 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:osd.0 smithi161 running (6m) 22s ago 6m 71.5M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 06119d929cdb 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:osd.1 smithi175 running (6m) 40s ago 6m 76.7M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 72d414751497 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:osd.2 smithi182 running (6m) 40s ago 6m 73.6M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 0c3fafe22c68 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:osd.3 smithi161 running (5m) 22s ago 5m 65.5M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 e1034d80be5b 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:osd.4 smithi175 running (5m) 40s ago 5m 62.7M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 bb3958424b3e 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:osd.5 smithi182 running (4m) 40s ago 4m 70.0M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 282d6d7149cb 2023-07-14T11:21:27.767 INFO:teuthology.orchestra.run.smithi161.stdout:osd.6 smithi161 running (4m) 22s ago 4m 61.7M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 2cf9959890c8 2023-07-14T11:21:27.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd.7 smithi175 running (4m) 40s ago 4m 68.1M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 b87abc613ab4 2023-07-14T11:21:27.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd.8 smithi182 running (3m) 40s ago 3m 65.5M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 41197ad96ffc 2023-07-14T11:21:27.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd.9 smithi161 running (3m) 22s ago 3m 60.7M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 55453f2137a3 2023-07-14T11:21:27.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd.10 smithi175 running (3m) 40s ago 3m 61.4M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 352f4922e5dc 2023-07-14T11:21:27.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd.11 smithi182 running (2m) 40s ago 2m 55.5M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 b6289d3768a1 2023-07-14T11:21:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6b940a11e0 msgr2=0x7f6b940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6b940a11e0 0x7f6b940a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8100bf0 tx=0x7f6ba4009260 comp rx=0 tx=0).stop 2023-07-14T11:21:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8100d90 msgr2=0x7f6ba8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8100d90 0x7f6ba8072900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9a0 tx=0x7f6b9800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 shutdown_connections 2023-07-14T11:21:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6b940a11e0 0x7f6b940a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ba8101720 0x7f6ba8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ba80ffb90 0x7f6ba80742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 --2- 172.21.15.161:0/1625151222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ba8100d90 0x7f6ba8072900 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 >> 172.21.15.161:0/1625151222 conn(0x7f6ba80fb1a0 msgr2=0x7f6ba8104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 shutdown_connections 2023-07-14T11:21:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:27.764+0000 7f6bafc9a700 1 -- 172.21.15.161:0/1625151222 wait complete. 2023-07-14T11:21:27.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:27 smithi182 ceph-mon[129366]: pgmap v328: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 keys/s, 1 objects/s recovering 2023-07-14T11:21:27.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:27 smithi161 ceph-mon[95526]: pgmap v328: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 keys/s, 1 objects/s recovering 2023-07-14T11:21:28.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:27 smithi175 ceph-mon[129081]: pgmap v328: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 keys/s, 1 objects/s recovering 2023-07-14T11:21:28.267 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-14T11:21:28.996 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:29.905 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:29 smithi161 ceph-mon[95526]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:29.905 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:29 smithi161 ceph-mon[95526]: pgmap v329: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2023-07-14T11:21:30.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:29 smithi175 ceph-mon[129081]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:30.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:29 smithi175 ceph-mon[129081]: pgmap v329: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2023-07-14T11:21:30.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:29 smithi182 ceph-mon[129366]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:30.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:29 smithi182 ceph-mon[129366]: pgmap v329: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2023-07-14T11:21:30.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 -- 172.21.15.161:0/3037518913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c100d90 msgr2=0x7f8b5c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:30.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 --2- 172.21.15.161:0/3037518913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c100d90 0x7f8b5c1011e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009990 tx=0x7f8b44009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 -- 172.21.15.161:0/3037518913 shutdown_connections 2023-07-14T11:21:30.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 --2- 172.21.15.161:0/3037518913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b5c101720 0x7f8b5c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 --2- 172.21.15.161:0/3037518913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b5c0ffb90 0x7f8b5c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 --2- 172.21.15.161:0/3037518913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c100d90 0x7f8b5c1011e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 -- 172.21.15.161:0/3037518913 >> 172.21.15.161:0/3037518913 conn(0x7f8b5c0fb1a0 msgr2=0x7f8b5c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:30.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 -- 172.21.15.161:0/3037518913 shutdown_connections 2023-07-14T11:21:30.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.524+0000 7f8b61663700 1 -- 172.21.15.161:0/3037518913 wait complete. 2023-07-14T11:21:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.525+0000 7f8b61663700 1 Processor -- start 2023-07-14T11:21:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.525+0000 7f8b61663700 1 -- start start 2023-07-14T11:21:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b61663700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c0ffb90 0x7f8b5c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b61663700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b5c100d90 0x7f8b5c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b61663700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b5c101720 0x7f8b5c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b61663700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c19f620 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b61663700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c19f760 con 0x7f8b5c101720 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b61663700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c19f8f0 con 0x7f8b5c100d90 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b5affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c0ffb90 0x7f8b5c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b5affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c0ffb90 0x7f8b5c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34898/0 (socket says 172.21.15.161:34898) 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b5affd700 1 -- 172.21.15.161:0/756768080 learned_addr learned my addr 172.21.15.161:0/756768080 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b53fff700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b5c100d90 0x7f8b5c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b5b7fe700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b5c101720 0x7f8b5c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b5affd700 1 -- 172.21.15.161:0/756768080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b5c100d90 msgr2=0x7f8b5c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b5affd700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b5c100d90 0x7f8b5c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.526+0000 7f8b5affd700 1 -- 172.21.15.161:0/756768080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b5c101720 msgr2=0x7f8b5c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b5affd700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b5c101720 0x7f8b5c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b5affd700 1 -- 172.21.15.161:0/756768080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440095f0 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b5affd700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c0ffb90 0x7f8b5c197a90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00e9a0 tx=0x7f8b4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b58ff9700 1 -- 172.21.15.161:0/756768080 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c009790 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c19fbd0 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b58ff9700 1 -- 172.21.15.161:0/756768080 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4c0044e0 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b58ff9700 1 -- 172.21.15.161:0/756768080 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c003d20 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.527+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c1a00d0 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.528+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c10b3a0 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.531+0000 7f8b58ff9700 1 -- 172.21.15.161:0/756768080 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8b4c003e80 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.532+0000 7f8b58ff9700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b480a11e0 0x7f8b480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:30.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.532+0000 7f8b58ff9700 1 -- 172.21.15.161:0/756768080 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8b4c014070 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.532+0000 7f8b53fff700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b480a11e0 0x7f8b480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:30.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.533+0000 7f8b53fff700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b480a11e0 0x7f8b480a3690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400b2e0 tx=0x7f8b44005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:30.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.536+0000 7f8b58ff9700 1 -- 172.21.15.161:0/756768080 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8b4c096330 con 0x7f8b5c0ffb90 2023-07-14T11:21:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.742+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b5c107d80 con 0x7f8b480a11e0 2023-07-14T11:21:30.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.748+0000 7f8b58ff9700 1 -- 172.21.15.161:0/756768080 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f8b5c107d80 con 0x7f8b480a11e0 2023-07-14T11:21:30.751 INFO:teuthology.orchestra.run.smithi161.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-14T11:21:30.751 INFO:teuthology.orchestra.run.smithi161.stdout:mds.all 12/12 43s ago 2m smithi161=a;smithi161=d;smithi161=g;smithi161=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi182=c;smithi182=f;smithi182=i;smithi182=l;count:12 2023-07-14T11:21:30.751 INFO:teuthology.orchestra.run.smithi161.stdout:mgr 3/3 43s ago 7m smithi161=x;smithi175=y;smithi182=z;count:3 2023-07-14T11:21:30.751 INFO:teuthology.orchestra.run.smithi161.stdout:mon 3/3 43s ago 7m smithi161:172.21.15.161=a;smithi175:172.21.15.175=b;smithi182:172.21.15.182=c;count:3 2023-07-14T11:21:30.751 INFO:teuthology.orchestra.run.smithi161.stdout:osd 12 43s ago - 2023-07-14T11:21:30.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.750+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b480a11e0 msgr2=0x7f8b480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:30.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.750+0000 7f8b61663700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b480a11e0 0x7f8b480a3690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400b2e0 tx=0x7f8b44005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.750+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c0ffb90 msgr2=0x7f8b5c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:30.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.750+0000 7f8b61663700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c0ffb90 0x7f8b5c197a90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00e9a0 tx=0x7f8b4c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 shutdown_connections 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b5c0ffb90 0x7f8b5c197a90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f8b480a11e0 0x7f8b480a3690 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b5c101720 0x7f8b5c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 --2- 172.21.15.161:0/756768080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b5c100d90 0x7f8b5c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 >> 172.21.15.161:0/756768080 conn(0x7f8b5c0fb1a0 msgr2=0x7f8b5c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 shutdown_connections 2023-07-14T11:21:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:30.751+0000 7f8b61663700 1 -- 172.21.15.161:0/756768080 wait complete. 2023-07-14T11:21:31.291 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-14T11:21:31.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:31 smithi161 ceph-mon[95526]: pgmap v330: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T11:21:32.016 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:32.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:31 smithi175 ceph-mon[129081]: pgmap v330: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T11:21:32.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:31 smithi182 ceph-mon[129366]: pgmap v330: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T11:21:32.923 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:32 smithi161 ceph-mon[95526]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:33.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:32 smithi175 ceph-mon[129081]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:33.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:32 smithi182 ceph-mon[129366]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:33.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.524+0000 7f4279c87700 1 -- 172.21.15.161:0/2551620769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 msgr2=0x7f4274103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:33.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.524+0000 7f4279c87700 1 --2- 172.21.15.161:0/2551620769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 0x7f4274103410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4268009990 tx=0x7f4268009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.525+0000 7f4279c87700 1 -- 172.21.15.161:0/2551620769 shutdown_connections 2023-07-14T11:21:33.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.525+0000 7f4279c87700 1 --2- 172.21.15.161:0/2551620769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4274103950 0x7f427410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.525+0000 7f4279c87700 1 --2- 172.21.15.161:0/2551620769 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4274101dc0 0x7f42741021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.525+0000 7f4279c87700 1 --2- 172.21.15.161:0/2551620769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 0x7f4274103410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.525+0000 7f4279c87700 1 -- 172.21.15.161:0/2551620769 >> 172.21.15.161:0/2551620769 conn(0x7f42740fd410 msgr2=0x7f42740ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:33.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.525+0000 7f4279c87700 1 -- 172.21.15.161:0/2551620769 shutdown_connections 2023-07-14T11:21:33.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.525+0000 7f4279c87700 1 -- 172.21.15.161:0/2551620769 wait complete. 2023-07-14T11:21:33.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.526+0000 7f4279c87700 1 Processor -- start 2023-07-14T11:21:33.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.526+0000 7f4279c87700 1 -- start start 2023-07-14T11:21:33.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4279c87700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4274101dc0 0x7f4274197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:33.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4279c87700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 0x7f4274198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:33.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4279c87700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4274103950 0x7f427419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:33.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4279c87700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427419f670 con 0x7f4274102fc0 2023-07-14T11:21:33.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4279c87700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427419f7b0 con 0x7f4274103950 2023-07-14T11:21:33.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4279c87700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427419f940 con 0x7f4274101dc0 2023-07-14T11:21:33.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4272ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 0x7f4274198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:33.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4272ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 0x7f4274198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34914/0 (socket says 172.21.15.161:34914) 2023-07-14T11:21:33.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f42737fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4274101dc0 0x7f4274197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:33.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4272ffd700 1 -- 172.21.15.161:0/1213120976 learned_addr learned my addr 172.21.15.161:0/1213120976 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:33.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4273fff700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4274103950 0x7f427419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:33.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f42737fe700 1 -- 172.21.15.161:0/1213120976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4274103950 msgr2=0x7f427419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:33.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f42737fe700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4274103950 0x7f427419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f42737fe700 1 -- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 msgr2=0x7f4274198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:33.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f42737fe700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 0x7f4274198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f42737fe700 1 -- 172.21.15.161:0/1213120976 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680095f0 con 0x7f4274101dc0 2023-07-14T11:21:33.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.527+0000 7f4273fff700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4274103950 0x7f427419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:33.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.528+0000 7f42737fe700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4274101dc0 0x7f4274197ae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f426400e9f0 tx=0x7f426400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:33.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.528+0000 7f4270ff9700 1 -- 172.21.15.161:0/1213120976 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426400eee0 con 0x7f4274101dc0 2023-07-14T11:21:33.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.528+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427419fc20 con 0x7f4274101dc0 2023-07-14T11:21:33.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.528+0000 7f4270ff9700 1 -- 172.21.15.161:0/1213120976 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42640044e0 con 0x7f4274101dc0 2023-07-14T11:21:33.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.528+0000 7f4270ff9700 1 -- 172.21.15.161:0/1213120976 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264003c70 con 0x7f4274101dc0 2023-07-14T11:21:33.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.528+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42741a0170 con 0x7f4274101dc0 2023-07-14T11:21:33.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.529+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427410d5d0 con 0x7f4274101dc0 2023-07-14T11:21:33.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.531+0000 7f4270ff9700 1 -- 172.21.15.161:0/1213120976 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f426400ccd0 con 0x7f4274101dc0 2023-07-14T11:21:33.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.533+0000 7f4270ff9700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f425c0a1260 0x7f425c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:33.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.533+0000 7f4270ff9700 1 -- 172.21.15.161:0/1213120976 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4264014070 con 0x7f4274101dc0 2023-07-14T11:21:33.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.533+0000 7f4272ffd700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f425c0a1260 0x7f425c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:33.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.534+0000 7f4272ffd700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f425c0a1260 0x7f425c0a3710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4268006010 tx=0x7f426800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:33.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.535+0000 7f4270ff9700 1 -- 172.21.15.161:0/1213120976 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f42640963d0 con 0x7f4274101dc0 2023-07-14T11:21:33.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.738+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4274060950 con 0x7f425c0a1260 2023-07-14T11:21:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.740+0000 7f4270ff9700 1 -- 172.21.15.161:0/1213120976 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f4274060950 con 0x7f425c0a1260 2023-07-14T11:21:33.742 INFO:teuthology.orchestra.run.smithi161.stdout:HOST ADDR LABELS STATUS 2023-07-14T11:21:33.743 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161 172.21.15.161 2023-07-14T11:21:33.743 INFO:teuthology.orchestra.run.smithi161.stdout:smithi175 172.21.15.175 2023-07-14T11:21:33.743 INFO:teuthology.orchestra.run.smithi161.stdout:smithi182 172.21.15.182 2023-07-14T11:21:33.743 INFO:teuthology.orchestra.run.smithi161.stdout:3 hosts in cluster 2023-07-14T11:21:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f425c0a1260 msgr2=0x7f425c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f425c0a1260 0x7f425c0a3710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4268006010 tx=0x7f426800b260 comp rx=0 tx=0).stop 2023-07-14T11:21:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4274101dc0 msgr2=0x7f4274197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4274101dc0 0x7f4274197ae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f426400e9f0 tx=0x7f426400ed00 comp rx=0 tx=0).stop 2023-07-14T11:21:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 shutdown_connections 2023-07-14T11:21:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f425c0a1260 0x7f425c0a3710 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4274103950 0x7f427419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4274101dc0 0x7f4274197ae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 --2- 172.21.15.161:0/1213120976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274102fc0 0x7f4274198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 >> 172.21.15.161:0/1213120976 conn(0x7f42740fd410 msgr2=0x7f42740ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 shutdown_connections 2023-07-14T11:21:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:33.742+0000 7f4279c87700 1 -- 172.21.15.161:0/1213120976 wait complete. 2023-07-14T11:21:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:33 smithi161 ceph-mon[95526]: pgmap v331: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:34.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:33 smithi175 ceph-mon[129081]: pgmap v331: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:34.206 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-14T11:21:34.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:33 smithi182 ceph-mon[129366]: pgmap v331: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:35.900 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:35 smithi161 ceph-mon[95526]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:35.901 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:35 smithi161 ceph-mon[95526]: pgmap v332: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:35 smithi175 ceph-mon[129081]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:36.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:35 smithi175 ceph-mon[129081]: pgmap v332: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:36.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:35 smithi182 ceph-mon[129366]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:36.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:35 smithi182 ceph-mon[129366]: pgmap v332: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 -- 172.21.15.161:0/2505942629 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608100d90 msgr2=0x7f26081011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 --2- 172.21.15.161:0/2505942629 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608100d90 0x7f26081011e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009990 tx=0x7f25f4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 -- 172.21.15.161:0/2505942629 shutdown_connections 2023-07-14T11:21:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 --2- 172.21.15.161:0/2505942629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608101720 0x7f2608107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 --2- 172.21.15.161:0/2505942629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26080ffb90 0x7f26080fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 --2- 172.21.15.161:0/2505942629 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608100d90 0x7f26081011e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 -- 172.21.15.161:0/2505942629 >> 172.21.15.161:0/2505942629 conn(0x7f26080fb1a0 msgr2=0x7f26080fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:36.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 -- 172.21.15.161:0/2505942629 shutdown_connections 2023-07-14T11:21:36.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.465+0000 7f260ef01700 1 -- 172.21.15.161:0/2505942629 wait complete. 2023-07-14T11:21:36.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.467+0000 7f260ef01700 1 Processor -- start 2023-07-14T11:21:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.467+0000 7f260ef01700 1 -- start start 2023-07-14T11:21:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.467+0000 7f260ef01700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26080ffb90 0x7f26080742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.467+0000 7f260ef01700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608100d90 0x7f2608072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260ef01700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608101720 0x7f2608072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260ef01700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26080735c0 con 0x7f2608100d90 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260ef01700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608073730 con 0x7f26080ffb90 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260ef01700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260819f970 con 0x7f2608101720 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f2607fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608100d90 0x7f2608072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f2607fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608100d90 0x7f2608072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34920/0 (socket says 172.21.15.161:34920) 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f2607fff700 1 -- 172.21.15.161:0/3380590094 learned_addr learned my addr 172.21.15.161:0/3380590094 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260cc9d700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26080ffb90 0x7f26080742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260d49e700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608101720 0x7f2608072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260cc9d700 1 -- 172.21.15.161:0/3380590094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608101720 msgr2=0x7f2608072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260cc9d700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608101720 0x7f2608072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260cc9d700 1 -- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608100d90 msgr2=0x7f2608072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260cc9d700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608100d90 0x7f2608072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260cc9d700 1 -- 172.21.15.161:0/3380590094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40095f0 con 0x7f26080ffb90 2023-07-14T11:21:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f260d49e700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608101720 0x7f2608072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.468+0000 7f2607fff700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608100d90 0x7f2608072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.469+0000 7f260cc9d700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26080ffb90 0x7f26080742b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9c0 tx=0x7f25fc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:36.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.469+0000 7f2605ffb700 1 -- 172.21.15.161:0/3380590094 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00cbd0 con 0x7f26080ffb90 2023-07-14T11:21:36.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.469+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260819fc50 con 0x7f26080ffb90 2023-07-14T11:21:36.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.469+0000 7f2605ffb700 1 -- 172.21.15.161:0/3380590094 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25fc00eeb0 con 0x7f26080ffb90 2023-07-14T11:21:36.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.469+0000 7f2605ffb700 1 -- 172.21.15.161:0/3380590094 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc003c70 con 0x7f26080ffb90 2023-07-14T11:21:36.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.469+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26081a0150 con 0x7f26080ffb90 2023-07-14T11:21:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.470+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26081071e0 con 0x7f26080ffb90 2023-07-14T11:21:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.472+0000 7f2605ffb700 1 -- 172.21.15.161:0/3380590094 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f25fc003dd0 con 0x7f26080ffb90 2023-07-14T11:21:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.473+0000 7f2605ffb700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25f80a11e0 0x7f25f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.474+0000 7f2605ffb700 1 -- 172.21.15.161:0/3380590094 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f25fc014070 con 0x7f26080ffb90 2023-07-14T11:21:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.474+0000 7f2607fff700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25f80a11e0 0x7f25f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.474+0000 7f2607fff700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25f80a11e0 0x7f25f80a3690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b2e0 tx=0x7f25f4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.475+0000 7f2605ffb700 1 -- 172.21.15.161:0/3380590094 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f25fc0964c0 con 0x7f26080ffb90 2023-07-14T11:21:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.679+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2608060950 con 0x7f25f80a11e0 2023-07-14T11:21:36.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.681+0000 7f2605ffb700 1 -- 172.21.15.161:0/3380590094 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f2608060950 con 0x7f25f80a11e0 2023-07-14T11:21:36.684 INFO:teuthology.orchestra.run.smithi161.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-14T11:21:36.684 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 3m ago LVM detected, locked 2023-07-14T11:21:36.684 INFO:teuthology.orchestra.run.smithi161.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 3m ago LVM detected, locked 2023-07-14T11:21:36.684 INFO:teuthology.orchestra.run.smithi161.stdout:smithi182 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161005T400BGN 372G No 2m ago LVM detected, locked 2023-07-14T11:21:36.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25f80a11e0 msgr2=0x7f25f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:36.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25f80a11e0 0x7f25f80a3690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b2e0 tx=0x7f25f4005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26080ffb90 msgr2=0x7f26080742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26080ffb90 0x7f26080742b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9c0 tx=0x7f25fc00ecd0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 shutdown_connections 2023-07-14T11:21:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25f80a11e0 0x7f25f80a3690 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2608101720 0x7f2608072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26080ffb90 0x7f26080742b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 --2- 172.21.15.161:0/3380590094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608100d90 0x7f2608072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 >> 172.21.15.161:0/3380590094 conn(0x7f26080fb1a0 msgr2=0x7f2608104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 shutdown_connections 2023-07-14T11:21:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:36.684+0000 7f260ef01700 1 -- 172.21.15.161:0/3380590094 wait complete. 2023-07-14T11:21:37.130 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T11:21:37.144 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T11:21:37.144 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-14T11:21:37.888 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:37.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:37 smithi161 ceph-mon[95526]: pgmap v333: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:37.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:37 smithi161 ceph-mon[95526]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:38.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:37 smithi175 ceph-mon[129081]: pgmap v333: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:38.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:37 smithi175 ceph-mon[129081]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:38.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:37 smithi182 ceph-mon[129366]: pgmap v333: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:38.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:37 smithi182 ceph-mon[129366]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:21:39.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:38 smithi175 ceph-mon[129081]: pgmap v334: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:39.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:38 smithi182 ceph-mon[129366]: pgmap v334: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:39.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:38 smithi161 ceph-mon[95526]: pgmap v334: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.487+0000 7f980ccc4700 1 -- 172.21.15.161:0/4012006093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9808100d90 msgr2=0x7f98081011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.487+0000 7f980ccc4700 1 --2- 172.21.15.161:0/4012006093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9808100d90 0x7f98081011e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009990 tx=0x7f97fc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.488+0000 7f980ccc4700 1 -- 172.21.15.161:0/4012006093 shutdown_connections 2023-07-14T11:21:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.488+0000 7f980ccc4700 1 --2- 172.21.15.161:0/4012006093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 0x7f9808107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.488+0000 7f980ccc4700 1 --2- 172.21.15.161:0/4012006093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f98080ffb90 0x7f98080fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.488+0000 7f980ccc4700 1 --2- 172.21.15.161:0/4012006093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9808100d90 0x7f98081011e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.488+0000 7f980ccc4700 1 -- 172.21.15.161:0/4012006093 >> 172.21.15.161:0/4012006093 conn(0x7f98080fb1a0 msgr2=0x7f98080fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.488+0000 7f980ccc4700 1 -- 172.21.15.161:0/4012006093 shutdown_connections 2023-07-14T11:21:39.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.488+0000 7f980ccc4700 1 -- 172.21.15.161:0/4012006093 wait complete. 2023-07-14T11:21:39.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 Processor -- start 2023-07-14T11:21:39.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 -- start start 2023-07-14T11:21:39.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98080ffb90 0x7f9808197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9808100d90 0x7f9808198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 0x7f980819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819f660 con 0x7f9808101720 2023-07-14T11:21:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819f7a0 con 0x7f98080ffb90 2023-07-14T11:21:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.489+0000 7f980ccc4700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819f930 con 0x7f9808100d90 2023-07-14T11:21:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 0x7f980819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 0x7f980819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39206/0 (socket says 172.21.15.161:39206) 2023-07-14T11:21:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 -- 172.21.15.161:0/82658367 learned_addr learned my addr 172.21.15.161:0/82658367 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9805d9b700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9808100d90 0x7f9808198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f980659c700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98080ffb90 0x7f9808197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 -- 172.21.15.161:0/82658367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9808100d90 msgr2=0x7f9808198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9808100d90 0x7f9808198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 -- 172.21.15.161:0/82658367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98080ffb90 msgr2=0x7f9808197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98080ffb90 0x7f9808197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 -- 172.21.15.161:0/82658367 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800009c40 con 0x7f9808101720 2023-07-14T11:21:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9805d9b700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9808100d90 0x7f9808198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f9806d9d700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 0x7f980819f090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f980000c3a0 tx=0x7f980000c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.490+0000 7f97f77fe700 1 -- 172.21.15.161:0/82658367 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800017070 con 0x7f9808101720 2023-07-14T11:21:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.491+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc0095f0 con 0x7f9808101720 2023-07-14T11:21:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.491+0000 7f97f77fe700 1 -- 172.21.15.161:0/82658367 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980000f040 con 0x7f9808101720 2023-07-14T11:21:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.491+0000 7f97f77fe700 1 -- 172.21.15.161:0/82658367 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980000e050 con 0x7f9808101720 2023-07-14T11:21:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.491+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980819fed0 con 0x7f9808101720 2023-07-14T11:21:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.492+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9808192260 con 0x7f9808101720 2023-07-14T11:21:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.494+0000 7f97f77fe700 1 -- 172.21.15.161:0/82658367 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f98000044c0 con 0x7f9808101720 2023-07-14T11:21:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.495+0000 7f97f77fe700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97f00a11e0 0x7f97f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.495+0000 7f97f77fe700 1 -- 172.21.15.161:0/82658367 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f98000cc780 con 0x7f9808101720 2023-07-14T11:21:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.495+0000 7f980659c700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97f00a11e0 0x7f97f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.496+0000 7f980659c700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97f00a11e0 0x7f97f00a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9808100bf0 tx=0x7f97f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:39.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.499+0000 7f97f77fe700 1 -- 172.21.15.161:0/82658367 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f98000959c0 con 0x7f9808101720 2023-07-14T11:21:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.735+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f980802c850 con 0x7f9808101720 2023-07-14T11:21:39.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.735+0000 7f97f77fe700 1 -- 172.21.15.161:0/82658367 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2913 (secure 0 0 0) 0x7f9800098fe0 con 0x7f9808101720 2023-07-14T11:21:39.738 INFO:teuthology.orchestra.run.smithi161.stdout:e22 2023-07-14T11:21:39.739 INFO:teuthology.orchestra.run.smithi161.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T11:21:39.739 INFO:teuthology.orchestra.run.smithi161.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:21:39.739 INFO:teuthology.orchestra.run.smithi161.stdout:legacy client fscid: 1 2023-07-14T11:21:39.739 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:39.739 INFO:teuthology.orchestra.run.smithi161.stdout:Filesystem 'cephfs' (1) 2023-07-14T11:21:39.739 INFO:teuthology.orchestra.run.smithi161.stdout:fs_name cephfs 2023-07-14T11:21:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 22 2023-07-14T11:21:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-14T11:21:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:created 2023-07-14T11:19:41.650966+0000 2023-07-14T11:21:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:modified 2023-07-14T11:20:12.668182+0000 2023-07-14T11:21:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:tableserver 0 2023-07-14T11:21:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:root 0 2023-07-14T11:21:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:session_timeout 300 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:session_autoclose 300 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:max_file_size 1099511627776 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:max_xattr_size 65536 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:required_client_features {} 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:last_failure 0 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:last_failure_osd_epoch 0 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:21:39.741 INFO:teuthology.orchestra.run.smithi161.stdout:max_mds 3 2023-07-14T11:21:39.742 INFO:teuthology.orchestra.run.smithi161.stdout:in 0,1,2 2023-07-14T11:21:39.742 INFO:teuthology.orchestra.run.smithi161.stdout:up {0=24575,1=24544,2=24533} 2023-07-14T11:21:39.742 INFO:teuthology.orchestra.run.smithi161.stdout:failed 2023-07-14T11:21:39.742 INFO:teuthology.orchestra.run.smithi161.stdout:damaged 2023-07-14T11:21:39.742 INFO:teuthology.orchestra.run.smithi161.stdout:stopped 2023-07-14T11:21:39.742 INFO:teuthology.orchestra.run.smithi161.stdout:data_pools [3,4] 2023-07-14T11:21:39.742 INFO:teuthology.orchestra.run.smithi161.stdout:metadata_pool 2 2023-07-14T11:21:39.743 INFO:teuthology.orchestra.run.smithi161.stdout:inline_data disabled 2023-07-14T11:21:39.743 INFO:teuthology.orchestra.run.smithi161.stdout:balancer 2023-07-14T11:21:39.743 INFO:teuthology.orchestra.run.smithi161.stdout:bal_rank_mask -1 2023-07-14T11:21:39.743 INFO:teuthology.orchestra.run.smithi161.stdout:standby_count_wanted 0 2023-07-14T11:21:39.743 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b{0:24575} state up:active seq 3 addr [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.743 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c{0:24526} state up:standby-replay seq 1 addr [v2:172.21.15.182:6836/1326049792,v1:172.21.15.182:6837/1326049792] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.i{1:24544} state up:active seq 12 addr [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.f{1:24514} state up:standby-replay seq 1 addr [v2:172.21.15.182:6834/1383844371,v1:172.21.15.182:6835/1383844371] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.h{2:24533} state up:active seq 13 addr [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.e{2:24503} state up:standby-replay seq 1 addr [v2:172.21.15.175:6834/4098726991,v1:172.21.15.175:6835/4098726991] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout:Standby daemons: 2023-07-14T11:21:39.745 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:39.745 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.j{-1:14442} state up:standby seq 1 addr [v2:172.21.15.161:6834/2631325720,v1:172.21.15.161:6835/2631325720] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.745 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.g{-1:14514} state up:standby seq 1 addr [v2:172.21.15.161:6838/1651269993,v1:172.21.15.161:6839/1651269993] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.745 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a{-1:14526} state up:standby seq 1 addr [v2:172.21.15.161:6840/208844218,v1:172.21.15.161:6841/208844218] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.745 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.l{-1:24457} state up:standby seq 1 addr [v2:172.21.15.182:6832/2580899731,v1:172.21.15.182:6833/2580899731] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.745 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.k{-1:24467} state up:standby seq 1 addr [v2:172.21.15.175:6832/3899543425,v1:172.21.15.175:6833/3899543425] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.745 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d{-1:24490} state up:standby seq 1 addr [v2:172.21.15.161:6836/4200056265,v1:172.21.15.161:6837/4200056265] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T11:21:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97f00a11e0 msgr2=0x7f97f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97f00a11e0 0x7f97f00a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9808100bf0 tx=0x7f97f8009260 comp rx=0 tx=0).stop 2023-07-14T11:21:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 msgr2=0x7f980819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 0x7f980819f090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f980000c3a0 tx=0x7f980000c6b0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 shutdown_connections 2023-07-14T11:21:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9808101720 0x7f980819f090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f97f00a11e0 0x7f97f00a3690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98080ffb90 0x7f9808197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 --2- 172.21.15.161:0/82658367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9808100d90 0x7f9808198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 >> 172.21.15.161:0/82658367 conn(0x7f98080fb1a0 msgr2=0x7f9808106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 shutdown_connections 2023-07-14T11:21:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:39.738+0000 7f980ccc4700 1 -- 172.21.15.161:0/82658367 wait complete. 2023-07-14T11:21:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:21:40.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:39 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/82658367' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T11:21:40.170 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:39 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/82658367' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T11:21:40.180 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-14T11:21:40.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:39 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/82658367' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T11:21:40.904 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:41.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:40 smithi175 ceph-mon[129081]: pgmap v335: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:41.185 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:40 smithi161 ceph-mon[95526]: pgmap v335: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:41.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:40 smithi182 ceph-mon[129366]: pgmap v335: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:42.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 -- 172.21.15.161:0/3494123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 msgr2=0x7f113c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:42.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 --2- 172.21.15.161:0/3494123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 0x7f113c0fffa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f112c009990 tx=0x7f112c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 -- 172.21.15.161:0/3494123140 shutdown_connections 2023-07-14T11:21:42.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 --2- 172.21.15.161:0/3494123140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c101720 0x7f113c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 --2- 172.21.15.161:0/3494123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 0x7f113c0fffa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 --2- 172.21.15.161:0/3494123140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113c100d90 0x7f113c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 -- 172.21.15.161:0/3494123140 >> 172.21.15.161:0/3494123140 conn(0x7f113c0fb1a0 msgr2=0x7f113c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:42.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 -- 172.21.15.161:0/3494123140 shutdown_connections 2023-07-14T11:21:42.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.428+0000 7f1141582700 1 -- 172.21.15.161:0/3494123140 wait complete. 2023-07-14T11:21:42.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.429+0000 7f1141582700 1 Processor -- start 2023-07-14T11:21:42.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.429+0000 7f1141582700 1 -- start start 2023-07-14T11:21:42.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f1141582700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 0x7f113c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:42.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f1141582700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100d90 0x7f113c195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:42.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f1141582700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113c101720 0x7f113c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:42.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f1141582700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c19d4c0 con 0x7f113c100d90 2023-07-14T11:21:42.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f1141582700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c19d600 con 0x7f113c101720 2023-07-14T11:21:42.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f1141582700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c19d790 con 0x7f113c0ffb90 2023-07-14T11:21:42.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100d90 0x7f113c195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:42.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100d90 0x7f113c195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39226/0 (socket says 172.21.15.161:39226) 2023-07-14T11:21:42.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 -- 172.21.15.161:0/2075327996 learned_addr learned my addr 172.21.15.161:0/2075327996 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:42.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113affd700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 0x7f113c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:42.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113b7fe700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113c101720 0x7f113c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:42.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 -- 172.21.15.161:0/2075327996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 msgr2=0x7f113c195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:42.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 0x7f113c195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 -- 172.21.15.161:0/2075327996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113c101720 msgr2=0x7f113c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:42.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113c101720 0x7f113c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.430+0000 7f113a7fc700 1 -- 172.21.15.161:0/2075327996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0095f0 con 0x7f113c100d90 2023-07-14T11:21:42.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.431+0000 7f113a7fc700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100d90 0x7f113c195e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f113000e9f0 tx=0x7f113000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:42.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.431+0000 7f1123fff700 1 -- 172.21.15.161:0/2075327996 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300097b0 con 0x7f113c100d90 2023-07-14T11:21:42.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.431+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113c19da70 con 0x7f113c100d90 2023-07-14T11:21:42.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.431+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113c19df70 con 0x7f113c100d90 2023-07-14T11:21:42.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.431+0000 7f1123fff700 1 -- 172.21.15.161:0/2075327996 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f113000ee90 con 0x7f113c100d90 2023-07-14T11:21:42.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.431+0000 7f1123fff700 1 -- 172.21.15.161:0/2075327996 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300054b0 con 0x7f113c100d90 2023-07-14T11:21:42.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.432+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1128005320 con 0x7f113c100d90 2023-07-14T11:21:42.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.434+0000 7f1123fff700 1 -- 172.21.15.161:0/2075327996 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1130005630 con 0x7f113c100d90 2023-07-14T11:21:42.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.435+0000 7f1123fff700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11240a5670 0x7f11240a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:42.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.436+0000 7f1123fff700 1 -- 172.21.15.161:0/2075327996 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1130014070 con 0x7f113c100d90 2023-07-14T11:21:42.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.436+0000 7f113affd700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11240a5670 0x7f11240a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:42.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.436+0000 7f113affd700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11240a5670 0x7f11240a7b20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f112c000c00 tx=0x7f112c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:42.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.437+0000 7f1123fff700 1 -- 172.21.15.161:0/2075327996 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1130099d10 con 0x7f113c100d90 2023-07-14T11:21:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.633+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f1128005cc0 con 0x7f113c100d90 2023-07-14T11:21:42.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.634+0000 7f1123fff700 1 -- 172.21.15.161:0/2075327996 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5669 (secure 0 0 0) 0x7f11300998a0 con 0x7f113c100d90 2023-07-14T11:21:42.638 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 92 2023-07-14T11:21:42.638 INFO:teuthology.orchestra.run.smithi161.stdout:fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c 2023-07-14T11:21:42.638 INFO:teuthology.orchestra.run.smithi161.stdout:created 2023-07-14T11:12:16.073598+0000 2023-07-14T11:21:42.638 INFO:teuthology.orchestra.run.smithi161.stdout:modified 2023-07-14T11:21:16.743617+0000 2023-07-14T11:21:42.638 INFO:teuthology.orchestra.run.smithi161.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-14T11:21:42.638 INFO:teuthology.orchestra.run.smithi161.stdout:crush_version 27 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:full_ratio 0.95 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:backfillfull_ratio 0.9 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:nearfull_ratio 0.85 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:require_min_compat_client luminous 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:min_compat_client luminous 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:require_osd_release reef 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:stretch_mode_enabled false 2023-07-14T11:21:42.639 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-14T11:21:42.640 INFO:teuthology.orchestra.run.smithi161.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-14T11:21:42.640 INFO:teuthology.orchestra.run.smithi161.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-07-14T11:21:42.640 INFO:teuthology.orchestra.run.smithi161.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-07-14T11:21:42.640 INFO:teuthology.orchestra.run.smithi161.stdout:max_osd 12 2023-07-14T11:21:42.640 INFO:teuthology.orchestra.run.smithi161.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6802/3629732803,v1:172.21.15.161:6803/3629732803] [v2:172.21.15.161:6804/3629732803,v1:172.21.15.161:6805/3629732803] exists,up 606d353c-d690-4ba5-b3d5-2283accaf2e5 2023-07-14T11:21:42.640 INFO:teuthology.orchestra.run.smithi161.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/3382092746,v1:172.21.15.175:6801/3382092746] [v2:172.21.15.175:6802/3382092746,v1:172.21.15.175:6803/3382092746] exists,up 4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660 2023-07-14T11:21:42.641 INFO:teuthology.orchestra.run.smithi161.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.182:6800/868656265,v1:172.21.15.182:6801/868656265] [v2:172.21.15.182:6802/868656265,v1:172.21.15.182:6803/868656265] exists,up 32cfe220-ef88-4ab3-a40f-57786bafcf56 2023-07-14T11:21:42.641 INFO:teuthology.orchestra.run.smithi161.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6810/2962874030,v1:172.21.15.161:6811/2962874030] [v2:172.21.15.161:6812/2962874030,v1:172.21.15.161:6813/2962874030] exists,up 24ac073f-086c-4626-8080-a901f73a1397 2023-07-14T11:21:42.641 INFO:teuthology.orchestra.run.smithi161.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/1496586319,v1:172.21.15.175:6809/1496586319] [v2:172.21.15.175:6810/1496586319,v1:172.21.15.175:6811/1496586319] exists,up f4e7501f-63ff-4f01-ade2-136ab260607c 2023-07-14T11:21:42.641 INFO:teuthology.orchestra.run.smithi161.stdout:osd.5 up in weight 1 up_from 39 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.182:6808/4034252274,v1:172.21.15.182:6809/4034252274] [v2:172.21.15.182:6810/4034252274,v1:172.21.15.182:6811/4034252274] exists,up 228b77ca-2772-40c5-8556-030d1b432280 2023-07-14T11:21:42.641 INFO:teuthology.orchestra.run.smithi161.stdout:osd.6 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6818/2715861815,v1:172.21.15.161:6819/2715861815] [v2:172.21.15.161:6820/2715861815,v1:172.21.15.161:6821/2715861815] exists,up bcacbde1-0008-440a-bbe7-07275b13f16a 2023-07-14T11:21:42.641 INFO:teuthology.orchestra.run.smithi161.stdout:osd.7 up in weight 1 up_from 52 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6816/4058697071,v1:172.21.15.175:6817/4058697071] [v2:172.21.15.175:6818/4058697071,v1:172.21.15.175:6819/4058697071] exists,up d6669cd4-11cc-4457-8775-f11dbeb79ec4 2023-07-14T11:21:42.641 INFO:teuthology.orchestra.run.smithi161.stdout:osd.8 up in weight 1 up_from 59 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.182:6816/3827127,v1:172.21.15.182:6817/3827127] [v2:172.21.15.182:6818/3827127,v1:172.21.15.182:6819/3827127] exists,up 56d1548f-b2d9-44bb-914a-9dbfdcd39b6d 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6826/565147230,v1:172.21.15.161:6827/565147230] [v2:172.21.15.161:6828/565147230,v1:172.21.15.161:6829/565147230] exists,up 016c1241-9eaf-48e6-b54a-4cfc973aa73f 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:osd.10 up in weight 1 up_from 72 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6824/4263786893,v1:172.21.15.175:6825/4263786893] [v2:172.21.15.175:6826/4263786893,v1:172.21.15.175:6827/4263786893] exists,up 2755e32a-49c5-4aaa-ad39-861f12b47889 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:osd.11 up in weight 1 up_from 79 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.182:6824/2744013300,v1:172.21.15.182:6825/2744013300] [v2:172.21.15.182:6826/2744013300,v1:172.21.15.182:6827/2744013300] exists,up 4d1d4d3c-b2d7-41e9-ae67-771e27332ebb 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 2.10 [8,11] 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 2.13 [2,11] 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 3.3 [4,9] 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 3.6 [3,7] 2023-07-14T11:21:42.642 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 3.c [3,9] 2023-07-14T11:21:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 3.13 [4,9] 2023-07-14T11:21:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 3.28 [5,8] 2023-07-14T11:21:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 3.3a [4,6] 2023-07-14T11:21:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 4.2 [9,1] 2023-07-14T11:21:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 4.b [9,8] 2023-07-14T11:21:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 4.14 [9,8] 2023-07-14T11:21:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:pg_upmap_items 4.1c [5,10] 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:0/871206111 expires 2023-07-15T11:13:14.005108+0000 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:6801/3292405426 expires 2023-07-15T11:13:14.005108+0000 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:6800/3292405426 expires 2023-07-15T11:13:14.005108+0000 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:0/629268496 expires 2023-07-15T11:12:47.212143+0000 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:0/832098793 expires 2023-07-15T11:13:14.005108+0000 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:6800/4131434778 expires 2023-07-15T11:12:47.212143+0000 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:6801/4131434778 expires 2023-07-15T11:12:47.212143+0000 2023-07-14T11:21:42.644 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:0/3333551885 expires 2023-07-15T11:12:47.212143+0000 2023-07-14T11:21:42.645 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:0/1691634661 expires 2023-07-15T11:12:47.212143+0000 2023-07-14T11:21:42.645 INFO:teuthology.orchestra.run.smithi161.stdout:blocklist 172.21.15.161:0/3118869366 expires 2023-07-15T11:13:14.005108+0000 2023-07-14T11:21:42.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11240a5670 msgr2=0x7f11240a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11240a5670 0x7f11240a7b20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f112c000c00 tx=0x7f112c00b260 comp rx=0 tx=0).stop 2023-07-14T11:21:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100d90 msgr2=0x7f113c195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100d90 0x7f113c195e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f113000e9f0 tx=0x7f113000ed00 comp rx=0 tx=0).stop 2023-07-14T11:21:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 shutdown_connections 2023-07-14T11:21:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f11240a5670 0x7f11240a7b20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113c101720 0x7f113c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f113c0ffb90 0x7f113c195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 --2- 172.21.15.161:0/2075327996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100d90 0x7f113c195e70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:42.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.636+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 >> 172.21.15.161:0/2075327996 conn(0x7f113c0fb1a0 msgr2=0x7f113c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:42.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.637+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 shutdown_connections 2023-07-14T11:21:42.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:42.637+0000 7f1141582700 1 -- 172.21.15.161:0/2075327996 wait complete. 2023-07-14T11:21:43.202 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-14T11:21:43.214 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:43.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 -- 172.21.15.161:0/856483993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce01079c0 msgr2=0x7f4ce0109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/856483993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce01079c0 0x7f4ce0109e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009f90 tx=0x7f4cd800b330 comp rx=0 tx=0).stop 2023-07-14T11:21:43.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 -- 172.21.15.161:0/856483993 shutdown_connections 2023-07-14T11:21:43.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/856483993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce0074550 0x7f4ce00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/856483993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce01079c0 0x7f4ce0109e40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/856483993 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4ce0072f70 0x7f4ce00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 -- 172.21.15.161:0/856483993 >> 172.21.15.161:0/856483993 conn(0x7f4ce00fb0e0 msgr2=0x7f4ce00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:43.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 -- 172.21.15.161:0/856483993 shutdown_connections 2023-07-14T11:21:43.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.412+0000 7f4ce79ff700 1 -- 172.21.15.161:0/856483993 wait complete. 2023-07-14T11:21:43.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.413+0000 7f4ce79ff700 1 Processor -- start 2023-07-14T11:21:43.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.413+0000 7f4ce79ff700 1 -- start start 2023-07-14T11:21:43.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce79ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4ce0072f70 0x7f4ce0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce79ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce0074550 0x7f4ce01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce79ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce01079c0 0x7f4ce019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce79ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce019f6f0 con 0x7f4ce01079c0 2023-07-14T11:21:43.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce79ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce019f830 con 0x7f4ce0074550 2023-07-14T11:21:43.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce79ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce019f9c0 con 0x7f4ce0072f70 2023-07-14T11:21:43.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce5f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce01079c0 0x7f4ce019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce579b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4ce0072f70 0x7f4ce0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce5f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce01079c0 0x7f4ce019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39242/0 (socket says 172.21.15.161:39242) 2023-07-14T11:21:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce5f9c700 1 -- 172.21.15.161:0/1567766708 learned_addr learned my addr 172.21.15.161:0/1567766708 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce579b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4ce0072f70 0x7f4ce0197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50178/0 (socket says 172.21.15.161:50178) 2023-07-14T11:21:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.414+0000 7f4ce4f9a700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce0074550 0x7f4ce01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce5f9c700 1 -- 172.21.15.161:0/1567766708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4ce0072f70 msgr2=0x7f4ce0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce5f9c700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4ce0072f70 0x7f4ce0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce5f9c700 1 -- 172.21.15.161:0/1567766708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce0074550 msgr2=0x7f4ce01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce5f9c700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce0074550 0x7f4ce01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce5f9c700 1 -- 172.21.15.161:0/1567766708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd8009c40 con 0x7f4ce01079c0 2023-07-14T11:21:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce5f9c700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce01079c0 0x7f4ce019f120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009f60 tx=0x7f4cd8004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4cd67fc700 1 -- 172.21.15.161:0/1567766708 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800e040 con 0x7f4ce01079c0 2023-07-14T11:21:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4cd67fc700 1 -- 172.21.15.161:0/1567766708 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd800bbf0 con 0x7f4ce01079c0 2023-07-14T11:21:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce019fc40 con 0x7f4ce01079c0 2023-07-14T11:21:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4cd67fc700 1 -- 172.21.15.161:0/1567766708 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80052a0 con 0x7f4ce01079c0 2023-07-14T11:21:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.415+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce01a0070 con 0x7f4ce01079c0 2023-07-14T11:21:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.416+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce0106780 con 0x7f4ce01079c0 2023-07-14T11:21:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.418+0000 7f4cd67fc700 1 -- 172.21.15.161:0/1567766708 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f4cd801d470 con 0x7f4ce01079c0 2023-07-14T11:21:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.420+0000 7f4cd67fc700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ccc0a1220 0x7f4ccc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.421+0000 7f4cd67fc700 1 -- 172.21.15.161:0/1567766708 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4cd80ce3e0 con 0x7f4ce01079c0 2023-07-14T11:21:43.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.421+0000 7f4ce579b700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ccc0a1220 0x7f4ccc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.421+0000 7f4ce579b700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ccc0a1220 0x7f4ccc0a36d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0074920 tx=0x7f4cdc009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.425+0000 7f4cd67fc700 1 -- 172.21.15.161:0/1567766708 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4cd8097710 con 0x7f4ce01079c0 2023-07-14T11:21:43.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:43 smithi182 ceph-mon[129366]: pgmap v336: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:43.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:43 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2075327996' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T11:21:43.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:43 smithi161 ceph-mon[95526]: pgmap v336: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:43.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:43 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2075327996' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T11:21:43.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.626+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ce002caa0 con 0x7f4ce01079c0 2023-07-14T11:21:43.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.628+0000 7f4cd67fc700 1 -- 172.21.15.161:0/1567766708 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7f4cd809ad30 con 0x7f4ce01079c0 2023-07-14T11:21:43.632 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:43.632 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:43.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ccc0a1220 msgr2=0x7f4ccc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ccc0a1220 0x7f4ccc0a36d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0074920 tx=0x7f4cdc009310 comp rx=0 tx=0).stop 2023-07-14T11:21:43.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce01079c0 msgr2=0x7f4ce019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce01079c0 0x7f4ce019f120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009f60 tx=0x7f4cd8004c60 comp rx=0 tx=0).stop 2023-07-14T11:21:43.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 shutdown_connections 2023-07-14T11:21:43.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ce01079c0 0x7f4ce019f120 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f4ccc0a1220 0x7f4ccc0a36d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4ce0072f70 0x7f4ce0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 --2- 172.21.15.161:0/1567766708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce0074550 0x7f4ce01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 >> 172.21.15.161:0/1567766708 conn(0x7f4ce00fb0e0 msgr2=0x7f4ce01038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:43.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 shutdown_connections 2023-07-14T11:21:43.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.630+0000 7f4ce79ff700 1 -- 172.21.15.161:0/1567766708 wait complete. 2023-07-14T11:21:43.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:21:43.653 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:21:43.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:43 smithi175 ceph-mon[129081]: pgmap v336: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:21:43.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:43 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2075327996' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T11:21:43.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 -- 172.21.15.161:0/3154769777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 msgr2=0x7f00d40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/3154769777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 0x7f00d40fff00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00d00098f0 tx=0x7f00d0009c00 comp rx=0 tx=0).stop 2023-07-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 -- 172.21.15.161:0/3154769777 shutdown_connections 2023-07-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/3154769777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4101670 0x7f00d4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/3154769777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 0x7f00d40fff00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/3154769777 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00d4100ce0 0x7f00d4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 -- 172.21.15.161:0/3154769777 >> 172.21.15.161:0/3154769777 conn(0x7f00d40fb130 msgr2=0x7f00d40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.852+0000 7f00dc3ff700 1 -- 172.21.15.161:0/3154769777 shutdown_connections 2023-07-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.853+0000 7f00dc3ff700 1 -- 172.21.15.161:0/3154769777 wait complete. 2023-07-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.853+0000 7f00dc3ff700 1 Processor -- start 2023-07-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00dc3ff700 1 -- start start 2023-07-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00dc3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 0x7f00d40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00dc3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4100ce0 0x7f00d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00dc3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00d4101670 0x7f00d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00dc3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40735e0 con 0x7f00d4100ce0 2023-07-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00dc3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4073750 con 0x7f00d40ffaf0 2023-07-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00dc3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419f930 con 0x7f00d4101670 2023-07-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00d999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4100ce0 0x7f00d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00d999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4100ce0 0x7f00d4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39248/0 (socket says 172.21.15.161:39248) 2023-07-14T11:21:43.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00da19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 0x7f00d40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00d999a700 1 -- 172.21.15.161:0/574679931 learned_addr learned my addr 172.21.15.161:0/574679931 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:43.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.854+0000 7f00da99c700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00d4101670 0x7f00d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00d999a700 1 -- 172.21.15.161:0/574679931 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00d4101670 msgr2=0x7f00d4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00d999a700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00d4101670 0x7f00d4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00d999a700 1 -- 172.21.15.161:0/574679931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 msgr2=0x7f00d40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:43.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00d999a700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 0x7f00d40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:43.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00d999a700 1 -- 172.21.15.161:0/574679931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009520 con 0x7f00d4100ce0 2023-07-14T11:21:43.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00d999a700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4100ce0 0x7f00d4072900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f00c400e9c0 tx=0x7f00c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:43.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00cb7fe700 1 -- 172.21.15.161:0/574679931 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4004a30 con 0x7f00d4100ce0 2023-07-14T11:21:43.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00cb7fe700 1 -- 172.21.15.161:0/574679931 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00c400cc80 con 0x7f00d4100ce0 2023-07-14T11:21:43.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d419fc10 con 0x7f00d4100ce0 2023-07-14T11:21:43.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00cb7fe700 1 -- 172.21.15.161:0/574679931 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40054e0 con 0x7f00d4100ce0 2023-07-14T11:21:43.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.855+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d41a0170 con 0x7f00d4100ce0 2023-07-14T11:21:43.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.856+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d410b310 con 0x7f00d4100ce0 2023-07-14T11:21:43.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.858+0000 7f00cb7fe700 1 -- 172.21.15.161:0/574679931 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f00c4004b90 con 0x7f00d4100ce0 2023-07-14T11:21:43.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.859+0000 7f00cb7fe700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00c00a1260 0x7f00c00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:43.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.860+0000 7f00cb7fe700 1 -- 172.21.15.161:0/574679931 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f00c4014070 con 0x7f00d4100ce0 2023-07-14T11:21:43.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.860+0000 7f00da19b700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00c00a1260 0x7f00c00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:43.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.860+0000 7f00da19b700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00c00a1260 0x7f00c00a3710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009fd0 tx=0x7f00d0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:43.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:43.863+0000 7f00cb7fe700 1 -- 172.21.15.161:0/574679931 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f00c40960e0 con 0x7f00d4100ce0 2023-07-14T11:21:44.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.063+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00d4106a70 con 0x7f00d4100ce0 2023-07-14T11:21:44.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.064+0000 7f00cb7fe700 1 -- 172.21.15.161:0/574679931 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00c4099700 con 0x7f00d4100ce0 2023-07-14T11:21:44.068 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:21:44.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00c00a1260 msgr2=0x7f00c00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00c00a1260 0x7f00c00a3710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009fd0 tx=0x7f00d0019040 comp rx=0 tx=0).stop 2023-07-14T11:21:44.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4100ce0 msgr2=0x7f00d4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4100ce0 0x7f00d4072900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f00c400e9c0 tx=0x7f00c400ecd0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 shutdown_connections 2023-07-14T11:21:44.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f00c00a1260 0x7f00c00a3710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f00d4101670 0x7f00d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40ffaf0 0x7f00d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 --2- 172.21.15.161:0/574679931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00d4100ce0 0x7f00d4072900 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.066+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 >> 172.21.15.161:0/574679931 conn(0x7f00d40fb130 msgr2=0x7f00d40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:44.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.067+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 shutdown_connections 2023-07-14T11:21:44.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.067+0000 7f00dc3ff700 1 -- 172.21.15.161:0/574679931 wait complete. 2023-07-14T11:21:44.089 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:21:44.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.315+0000 7f4260dff700 1 -- 172.21.15.161:0/327743899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 msgr2=0x7f425c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.315+0000 7f4260dff700 1 --2- 172.21.15.161:0/327743899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c107f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f425400b0a0 tx=0x7f425400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.316+0000 7f4260dff700 1 -- 172.21.15.161:0/327743899 shutdown_connections 2023-07-14T11:21:44.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.316+0000 7f4260dff700 1 --2- 172.21.15.161:0/327743899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c107f20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.316+0000 7f4260dff700 1 --2- 172.21.15.161:0/327743899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 0x7f425c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.316+0000 7f4260dff700 1 --2- 172.21.15.161:0/327743899 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f425c100cd0 0x7f425c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.316+0000 7f4260dff700 1 -- 172.21.15.161:0/327743899 >> 172.21.15.161:0/327743899 conn(0x7f425c0fb0e0 msgr2=0x7f425c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:44.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.316+0000 7f4260dff700 1 -- 172.21.15.161:0/327743899 shutdown_connections 2023-07-14T11:21:44.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.316+0000 7f4260dff700 1 -- 172.21.15.161:0/327743899 wait complete. 2023-07-14T11:21:44.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 Processor -- start 2023-07-14T11:21:44.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 -- start start 2023-07-14T11:21:44.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 0x7f425c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f425c100cd0 0x7f425c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c19f730 con 0x7f425c101660 2023-07-14T11:21:44.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c19f870 con 0x7f425c0ffae0 2023-07-14T11:21:44.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f4260dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c19fa00 con 0x7f425c100cd0 2023-07-14T11:21:44.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f425ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f425a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 0x7f425c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.317+0000 7f425ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c19f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39272/0 (socket says 172.21.15.161:39272) 2023-07-14T11:21:44.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 0x7f425c197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45858/0 (socket says 172.21.15.161:45858) 2023-07-14T11:21:44.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425ad9d700 1 -- 172.21.15.161:0/1815264085 learned_addr learned my addr 172.21.15.161:0/1815264085 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:44.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f4259d9b700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f425c100cd0 0x7f425c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425a59c700 1 -- 172.21.15.161:0/1815264085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f425c100cd0 msgr2=0x7f425c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425a59c700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f425c100cd0 0x7f425c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425a59c700 1 -- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 msgr2=0x7f425c19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425a59c700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425a59c700 1 -- 172.21.15.161:0/1815264085 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4254009d00 con 0x7f425c0ffae0 2023-07-14T11:21:44.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425ad9d700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:44.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f425a59c700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 0x7f425c197bb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f424c00e9c0 tx=0x7f424c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:44.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f424b7fe700 1 -- 172.21.15.161:0/1815264085 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c004220 con 0x7f425c0ffae0 2023-07-14T11:21:44.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f424b7fe700 1 -- 172.21.15.161:0/1815264085 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f424c004380 con 0x7f425c0ffae0 2023-07-14T11:21:44.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c19fce0 con 0x7f425c0ffae0 2023-07-14T11:21:44.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.318+0000 7f424b7fe700 1 -- 172.21.15.161:0/1815264085 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c003be0 con 0x7f425c0ffae0 2023-07-14T11:21:44.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.319+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c1a0190 con 0x7f425c0ffae0 2023-07-14T11:21:44.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.320+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c10b380 con 0x7f425c0ffae0 2023-07-14T11:21:44.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.321+0000 7f424b7fe700 1 -- 172.21.15.161:0/1815264085 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f424c003d40 con 0x7f425c0ffae0 2023-07-14T11:21:44.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.323+0000 7f424b7fe700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42440a1260 0x7f42440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.323+0000 7f424b7fe700 1 -- 172.21.15.161:0/1815264085 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f424c014070 con 0x7f425c0ffae0 2023-07-14T11:21:44.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.323+0000 7f4259d9b700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42440a1260 0x7f42440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.324+0000 7f4259d9b700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42440a1260 0x7f42440a3710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4250009be0 tx=0x7f4250009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:44.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.326+0000 7f424b7fe700 1 -- 172.21.15.161:0/1815264085 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f424c096280 con 0x7f425c0ffae0 2023-07-14T11:21:44.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:44 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1567766708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:44.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:44 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/574679931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:44.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:44 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1567766708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:44.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:44 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/574679931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:44.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.527+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f425c10b510 con 0x7f425c0ffae0 2023-07-14T11:21:44.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.528+0000 7f424b7fe700 1 -- 172.21.15.161:0/1815264085 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f424c0998a0 con 0x7f425c0ffae0 2023-07-14T11:21:44.532 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:44.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42440a1260 msgr2=0x7f42440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42440a1260 0x7f42440a3710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4250009be0 tx=0x7f4250009260 comp rx=0 tx=0).stop 2023-07-14T11:21:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 msgr2=0x7f425c197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 0x7f425c197bb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f424c00e9c0 tx=0x7f424c00ecd0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 shutdown_connections 2023-07-14T11:21:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f425c101660 0x7f425c19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f42440a1260 0x7f42440a3710 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c0ffae0 0x7f425c197bb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 --2- 172.21.15.161:0/1815264085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f425c100cd0 0x7f425c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 >> 172.21.15.161:0/1815264085 conn(0x7f425c0fb0e0 msgr2=0x7f425c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 shutdown_connections 2023-07-14T11:21:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.531+0000 7f4260dff700 1 -- 172.21.15.161:0/1815264085 wait complete. 2023-07-14T11:21:44.554 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:21:44.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:44 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1567766708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:44.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:44 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/574679931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 -- 172.21.15.161:0/1035549928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8100cd0 msgr2=0x7f58e8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 --2- 172.21.15.161:0/1035549928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8100cd0 0x7f58e8101120 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009990 tx=0x7f58dc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 -- 172.21.15.161:0/1035549928 shutdown_connections 2023-07-14T11:21:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 --2- 172.21.15.161:0/1035549928 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f58e8101660 0x7f58e8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 --2- 172.21.15.161:0/1035549928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58e80ffae0 0x7f58e80ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 --2- 172.21.15.161:0/1035549928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8100cd0 0x7f58e8101120 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 -- 172.21.15.161:0/1035549928 >> 172.21.15.161:0/1035549928 conn(0x7f58e80fb0e0 msgr2=0x7f58e80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:44.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.751+0000 7f58ecfff700 1 -- 172.21.15.161:0/1035549928 shutdown_connections 2023-07-14T11:21:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.752+0000 7f58ecfff700 1 -- 172.21.15.161:0/1035549928 wait complete. 2023-07-14T11:21:44.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.752+0000 7f58ecfff700 1 Processor -- start 2023-07-14T11:21:44.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58ecfff700 1 -- start start 2023-07-14T11:21:44.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58ecfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58e80ffae0 0x7f58e8195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58ecfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f58e8100cd0 0x7f58e8195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58ecfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8101660 0x7f58e819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58ecfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e819d400 con 0x7f58e8101660 2023-07-14T11:21:44.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58ecfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e819d540 con 0x7f58e80ffae0 2023-07-14T11:21:44.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58ecfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e819d6d0 con 0x7f58e8100cd0 2023-07-14T11:21:44.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58e6d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8101660 0x7f58e819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58e6d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8101660 0x7f58e819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39282/0 (socket says 172.21.15.161:39282) 2023-07-14T11:21:44.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58e6d9d700 1 -- 172.21.15.161:0/3043003557 learned_addr learned my addr 172.21.15.161:0/3043003557 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:44.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58e5d9b700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f58e8100cd0 0x7f58e8195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.753+0000 7f58e659c700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58e80ffae0 0x7f58e8195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58e6d9d700 1 -- 172.21.15.161:0/3043003557 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f58e8100cd0 msgr2=0x7f58e8195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58e6d9d700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f58e8100cd0 0x7f58e8195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58e6d9d700 1 -- 172.21.15.161:0/3043003557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58e80ffae0 msgr2=0x7f58e8195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58e6d9d700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58e80ffae0 0x7f58e8195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58e6d9d700 1 -- 172.21.15.161:0/3043003557 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0095f0 con 0x7f58e8101660 2023-07-14T11:21:44.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58e5d9b700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f58e8100cd0 0x7f58e8195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:44.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58e6d9d700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8101660 0x7f58e819ce30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f58e000ef30 tx=0x7f58e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:44.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58d77fe700 1 -- 172.21.15.161:0/3043003557 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0018070 con 0x7f58e8101660 2023-07-14T11:21:44.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58d77fe700 1 -- 172.21.15.161:0/3043003557 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e000f040 con 0x7f58e8101660 2023-07-14T11:21:44.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e819d950 con 0x7f58e8101660 2023-07-14T11:21:44.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58d77fe700 1 -- 172.21.15.161:0/3043003557 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e000cdb0 con 0x7f58e8101660 2023-07-14T11:21:44.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.754+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e819de60 con 0x7f58e8101660 2023-07-14T11:21:44.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.755+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e810b300 con 0x7f58e8101660 2023-07-14T11:21:44.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.757+0000 7f58d77fe700 1 -- 172.21.15.161:0/3043003557 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f58e0003a00 con 0x7f58e8101660 2023-07-14T11:21:44.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.759+0000 7f58d77fe700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f58d00a11e0 0x7f58d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:44.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.759+0000 7f58d77fe700 1 -- 172.21.15.161:0/3043003557 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f58e00cca00 con 0x7f58e8101660 2023-07-14T11:21:44.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.759+0000 7f58e659c700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f58d00a11e0 0x7f58d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:44.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.759+0000 7f58e659c700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f58d00a11e0 0x7f58d00a3690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58e8100b30 tx=0x7f58d8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:44.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.762+0000 7f58d77fe700 1 -- 172.21.15.161:0/3043003557 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f58e0095cd0 con 0x7f58e8101660 2023-07-14T11:21:44.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.962+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58e81069e0 con 0x7f58e8101660 2023-07-14T11:21:44.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.962+0000 7f58d77fe700 1 -- 172.21.15.161:0/3043003557 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58e00992f0 con 0x7f58e8101660 2023-07-14T11:21:44.966 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:21:44.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f58d00a11e0 msgr2=0x7f58d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f58d00a11e0 0x7f58d00a3690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58e8100b30 tx=0x7f58d8009260 comp rx=0 tx=0).stop 2023-07-14T11:21:44.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8101660 msgr2=0x7f58e819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:44.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8101660 0x7f58e819ce30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f58e000ef30 tx=0x7f58e000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 shutdown_connections 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e8101660 0x7f58e819ce30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f58d00a11e0 0x7f58d00a3690 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58e80ffae0 0x7f58e8195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 --2- 172.21.15.161:0/3043003557 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f58e8100cd0 0x7f58e8195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 >> 172.21.15.161:0/3043003557 conn(0x7f58e80fb0e0 msgr2=0x7f58e80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.964+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 shutdown_connections 2023-07-14T11:21:44.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:44.965+0000 7f58ecfff700 1 -- 172.21.15.161:0/3043003557 wait complete. 2023-07-14T11:21:44.987 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:21:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.185+0000 7f41131ff700 1 -- 172.21.15.161:0/1487055351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c0ffb00 msgr2=0x7f410c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.185+0000 7f41131ff700 1 --2- 172.21.15.161:0/1487055351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c0ffb00 0x7f410c0fff10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f41000098f0 tx=0x7f4100009c00 comp rx=0 tx=0).stop 2023-07-14T11:21:45.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.186+0000 7f41131ff700 1 -- 172.21.15.161:0/1487055351 shutdown_connections 2023-07-14T11:21:45.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.186+0000 7f41131ff700 1 --2- 172.21.15.161:0/1487055351 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f410c101680 0x7f410c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.186+0000 7f41131ff700 1 --2- 172.21.15.161:0/1487055351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c0ffb00 0x7f410c0fff10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.186+0000 7f41131ff700 1 --2- 172.21.15.161:0/1487055351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c100cf0 0x7f410c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.186+0000 7f41131ff700 1 -- 172.21.15.161:0/1487055351 >> 172.21.15.161:0/1487055351 conn(0x7f410c0fb120 msgr2=0x7f410c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:45.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.186+0000 7f41131ff700 1 -- 172.21.15.161:0/1487055351 shutdown_connections 2023-07-14T11:21:45.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.186+0000 7f41131ff700 1 -- 172.21.15.161:0/1487055351 wait complete. 2023-07-14T11:21:45.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 Processor -- start 2023-07-14T11:21:45.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 -- start start 2023-07-14T11:21:45.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0ffb00 0x7f410c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f410c100cf0 0x7f410c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c101680 0x7f410c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0735e0 con 0x7f410c0ffb00 2023-07-14T11:21:45.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c073750 con 0x7f410c101680 2023-07-14T11:21:45.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f41131ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19f930 con 0x7f410c100cf0 2023-07-14T11:21:45.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f4110f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0ffb00 0x7f410c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f4110f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0ffb00 0x7f410c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39306/0 (socket says 172.21.15.161:39306) 2023-07-14T11:21:45.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.187+0000 7f4110f9b700 1 -- 172.21.15.161:0/1747526258 learned_addr learned my addr 172.21.15.161:0/1747526258 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:45.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f410bfff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f410c100cf0 0x7f410c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f411179c700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c101680 0x7f410c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4110f9b700 1 -- 172.21.15.161:0/1747526258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f410c100cf0 msgr2=0x7f410c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4110f9b700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f410c100cf0 0x7f410c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4110f9b700 1 -- 172.21.15.161:0/1747526258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c101680 msgr2=0x7f410c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4110f9b700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c101680 0x7f410c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4110f9b700 1 -- 172.21.15.161:0/1747526258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f80095f0 con 0x7f410c0ffb00 2023-07-14T11:21:45.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f411179c700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c101680 0x7f410c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:45.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f410bfff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f410c100cf0 0x7f410c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:45.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4110f9b700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0ffb00 0x7f410c0742c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4100009fd0 tx=0x7f410000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:45.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410001c070 con 0x7f410c0ffb00 2023-07-14T11:21:45.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4100009520 con 0x7f410c0ffb00 2023-07-14T11:21:45.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4100005040 con 0x7f410c0ffb00 2023-07-14T11:21:45.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.188+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100003b80 con 0x7f410c0ffb00 2023-07-14T11:21:45.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.189+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c19fea0 con 0x7f410c0ffb00 2023-07-14T11:21:45.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.189+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c10b320 con 0x7f410c0ffb00 2023-07-14T11:21:45.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.192+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f4100020030 con 0x7f410c0ffb00 2023-07-14T11:21:45.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.193+0000 7f4109ffb700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f40fc0a11e0 0x7f40fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.193+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f41000cdf40 con 0x7f410c0ffb00 2023-07-14T11:21:45.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.193+0000 7f410bfff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f40fc0a11e0 0x7f40fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.194+0000 7f410bfff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f40fc0a11e0 0x7f40fc0a3690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f40f8005fd0 tx=0x7f40f8009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:45.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.201+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4100097270 con 0x7f410c0ffb00 2023-07-14T11:21:45.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.211+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f41000051b0 con 0x7f410c0ffb00 2023-07-14T11:21:45.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:45 smithi161 ceph-mon[95526]: pgmap v337: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:45.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:45 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1815264085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:45.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:45 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3043003557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:45.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.405+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f410c106aa0 con 0x7f410c0ffb00 2023-07-14T11:21:45.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.406+0000 7f4109ffb700 1 -- 172.21.15.161:0/1747526258 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f41000871d0 con 0x7f410c0ffb00 2023-07-14T11:21:45.409 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:45.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f40fc0a11e0 msgr2=0x7f40fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f40fc0a11e0 0x7f40fc0a3690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f40f8005fd0 tx=0x7f40f8009310 comp rx=0 tx=0).stop 2023-07-14T11:21:45.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0ffb00 msgr2=0x7f410c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0ffb00 0x7f410c0742c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4100009fd0 tx=0x7f410000b5b0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 shutdown_connections 2023-07-14T11:21:45.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0ffb00 0x7f410c0742c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f40fc0a11e0 0x7f40fc0a3690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f410c101680 0x7f410c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 --2- 172.21.15.161:0/1747526258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f410c100cf0 0x7f410c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 >> 172.21.15.161:0/1747526258 conn(0x7f410c0fb120 msgr2=0x7f410c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:45.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 shutdown_connections 2023-07-14T11:21:45.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.408+0000 7f41131ff700 1 -- 172.21.15.161:0/1747526258 wait complete. 2023-07-14T11:21:45.430 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:21:45.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:45 smithi182 ceph-mon[129366]: pgmap v337: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:45.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:45 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1815264085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:45.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:45 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3043003557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:45.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 -- 172.21.15.161:0/85922092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 msgr2=0x7f3ca0107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/85922092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca0107bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800b0a0 tx=0x7f3c9800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 -- 172.21.15.161:0/85922092 shutdown_connections 2023-07-14T11:21:45.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/85922092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca0107bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/85922092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0100590 0x7f3ca0102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/85922092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ca0102eb0 0x7f3ca0105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 -- 172.21.15.161:0/85922092 >> 172.21.15.161:0/85922092 conn(0x7f3ca00f9fd0 msgr2=0x7f3ca00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:45.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 -- 172.21.15.161:0/85922092 shutdown_connections 2023-07-14T11:21:45.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.626+0000 7f3ca53ff700 1 -- 172.21.15.161:0/85922092 wait complete. 2023-07-14T11:21:45.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.627+0000 7f3ca53ff700 1 Processor -- start 2023-07-14T11:21:45.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.627+0000 7f3ca53ff700 1 -- start start 2023-07-14T11:21:45.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3ca53ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0100590 0x7f3ca0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3ca53ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ca0102eb0 0x7f3ca0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3ca53ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3ca53ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca019f630 con 0x7f3ca01057d0 2023-07-14T11:21:45.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3ca53ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca019f770 con 0x7f3ca0100590 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3ca53ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca019f900 con 0x7f3ca0102eb0 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0100590 0x7f3ca0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39326/0 (socket says 172.21.15.161:39326) 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 -- 172.21.15.161:0/3120890886 learned_addr learned my addr 172.21.15.161:0/3120890886 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c97fff700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ca0102eb0 0x7f3ca0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 -- 172.21.15.161:0/3120890886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ca0102eb0 msgr2=0x7f3ca0197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ca0102eb0 0x7f3ca0197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 -- 172.21.15.161:0/3120890886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0100590 msgr2=0x7f3ca0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0100590 0x7f3ca0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.628+0000 7f3c9f7fe700 1 -- 172.21.15.161:0/3120890886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98009d00 con 0x7f3ca01057d0 2023-07-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.629+0000 7f3c9f7fe700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca019f060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800b780 tx=0x7f3c9800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.629+0000 7f3c9cff9700 1 -- 172.21.15.161:0/3120890886 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800e040 con 0x7f3ca01057d0 2023-07-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.629+0000 7f3c9cff9700 1 -- 172.21.15.161:0/3120890886 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c980094a0 con 0x7f3ca01057d0 2023-07-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.629+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca019fb80 con 0x7f3ca01057d0 2023-07-14T11:21:45.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.629+0000 7f3c9cff9700 1 -- 172.21.15.161:0/3120890886 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98005390 con 0x7f3ca01057d0 2023-07-14T11:21:45.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.629+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca019ffb0 con 0x7f3ca01057d0 2023-07-14T11:21:45.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.630+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca0107340 con 0x7f3ca01057d0 2023-07-14T11:21:45.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.632+0000 7f3c9cff9700 1 -- 172.21.15.161:0/3120890886 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3c9801d470 con 0x7f3ca01057d0 2023-07-14T11:21:45.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.633+0000 7f3c9cff9700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3c8c0a11a0 0x7f3c8c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:45.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.633+0000 7f3c9cff9700 1 -- 172.21.15.161:0/3120890886 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3c980ce3c0 con 0x7f3ca01057d0 2023-07-14T11:21:45.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.634+0000 7f3c9effd700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3c8c0a11a0 0x7f3c8c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:45.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.634+0000 7f3c9effd700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3c8c0a11a0 0x7f3c8c0a3650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c90007c60 tx=0x7f3c900073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:45.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.637+0000 7f3c9cff9700 1 -- 172.21.15.161:0/3120890886 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3c98097850 con 0x7f3ca01057d0 2023-07-14T11:21:45.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:45 smithi175 ceph-mon[129081]: pgmap v337: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:21:45.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:45 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1815264085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:45.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:45 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3043003557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.880+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ca002c850 con 0x7f3ca01057d0 2023-07-14T11:21:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.881+0000 7f3c9cff9700 1 -- 172.21.15.161:0/3120890886 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f3c9809ae70 con 0x7f3ca01057d0 2023-07-14T11:21:45.885 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:45.885 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:21:45.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.883+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3c8c0a11a0 msgr2=0x7f3c8c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.883+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3c8c0a11a0 0x7f3c8c0a3650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c90007c60 tx=0x7f3c900073d0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.883+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 msgr2=0x7f3ca019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:45.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.883+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca019f060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800b780 tx=0x7f3c9800bd60 comp rx=0 tx=0).stop 2023-07-14T11:21:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.883+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 shutdown_connections 2023-07-14T11:21:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.883+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ca01057d0 0x7f3ca019f060 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.883+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3c8c0a11a0 0x7f3c8c0a3650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.884+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0100590 0x7f3ca0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.884+0000 7f3ca53ff700 1 --2- 172.21.15.161:0/3120890886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3ca0102eb0 0x7f3ca0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.884+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 >> 172.21.15.161:0/3120890886 conn(0x7f3ca00f9fd0 msgr2=0x7f3ca00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.884+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 shutdown_connections 2023-07-14T11:21:45.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:45.884+0000 7f3ca53ff700 1 -- 172.21.15.161:0/3120890886 wait complete. 2023-07-14T11:21:45.889 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:21:45.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:46.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.103+0000 7fb7395ff700 1 -- 172.21.15.161:0/2471838514 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 msgr2=0x7fb734105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.103+0000 7fb7395ff700 1 --2- 172.21.15.161:0/2471838514 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734105300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098f0 tx=0x7fb728009c00 comp rx=0 tx=0).stop 2023-07-14T11:21:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.104+0000 7fb7395ff700 1 -- 172.21.15.161:0/2471838514 shutdown_connections 2023-07-14T11:21:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.104+0000 7fb7395ff700 1 --2- 172.21.15.161:0/2471838514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb734100600 0x7fb7341029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.104+0000 7fb7395ff700 1 --2- 172.21.15.161:0/2471838514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734105840 0x7fb734107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.104+0000 7fb7395ff700 1 --2- 172.21.15.161:0/2471838514 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734105300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.104+0000 7fb7395ff700 1 -- 172.21.15.161:0/2471838514 >> 172.21.15.161:0/2471838514 conn(0x7fb7340fa000 msgr2=0x7fb7340fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.104+0000 7fb7395ff700 1 -- 172.21.15.161:0/2471838514 shutdown_connections 2023-07-14T11:21:46.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.104+0000 7fb7395ff700 1 -- 172.21.15.161:0/2471838514 wait complete. 2023-07-14T11:21:46.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 Processor -- start 2023-07-14T11:21:46.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 -- start start 2023-07-14T11:21:46.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734100600 0x7fb734197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb734105840 0x7fb73419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73419f5a0 con 0x7fb734105840 2023-07-14T11:21:46.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73419f6e0 con 0x7fb734100600 2023-07-14T11:21:46.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.105+0000 7fb7395ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73419f870 con 0x7fb734102f20 2023-07-14T11:21:46.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb732ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734100600 0x7fb734197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7337fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb734105840 0x7fb73419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb732ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734100600 0x7fb734197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45926/0 (socket says 172.21.15.161:45926) 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb732ffd700 1 -- 172.21.15.161:0/1792031859 learned_addr learned my addr 172.21.15.161:0/1792031859 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50306/0 (socket says 172.21.15.161:50306) 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 -- 172.21.15.161:0/1792031859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734100600 msgr2=0x7fb734197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734100600 0x7fb734197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 -- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb734105840 msgr2=0x7fb73419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb734105840 0x7fb73419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 -- 172.21.15.161:0/1792031859 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240095f0 con 0x7fb734102f20 2023-07-14T11:21:46.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7337fe700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb734105840 0x7fb73419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:46.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb732ffd700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734100600 0x7fb734197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:46.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb7327fc700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734197f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb728009230 tx=0x7fb72800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:46.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb71bfff700 1 -- 172.21.15.161:0/1792031859 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72801c070 con 0x7fb734102f20 2023-07-14T11:21:46.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.106+0000 7fb71bfff700 1 -- 172.21.15.161:0/1792031859 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb72800f7d0 con 0x7fb734102f20 2023-07-14T11:21:46.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.107+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb728009520 con 0x7fb734102f20 2023-07-14T11:21:46.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.107+0000 7fb71bfff700 1 -- 172.21.15.161:0/1792031859 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728003a00 con 0x7fb734102f20 2023-07-14T11:21:46.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.107+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73419fd80 con 0x7fb734102f20 2023-07-14T11:21:46.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.108+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7341073b0 con 0x7fb734102f20 2023-07-14T11:21:46.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.110+0000 7fb71bfff700 1 -- 172.21.15.161:0/1792031859 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb728020030 con 0x7fb734102f20 2023-07-14T11:21:46.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.111+0000 7fb71bfff700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb71c0a11a0 0x7fb71c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.111+0000 7fb71bfff700 1 -- 172.21.15.161:0/1792031859 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb7280cdd80 con 0x7fb734102f20 2023-07-14T11:21:46.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.111+0000 7fb732ffd700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb71c0a11a0 0x7fb71c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.112+0000 7fb732ffd700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb71c0a11a0 0x7fb71c0a3650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb724005d10 tx=0x7fb724009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:46.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.113+0000 7fb71bfff700 1 -- 172.21.15.161:0/1792031859 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb7280971b0 con 0x7fb734102f20 2023-07-14T11:21:46.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.318+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb73402ca20 con 0x7fb734102f20 2023-07-14T11:21:46.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.321+0000 7fb71bfff700 1 -- 172.21.15.161:0/1792031859 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7fb72809a7d0 con 0x7fb734102f20 2023-07-14T11:21:46.324 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:46.324 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:46.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb71c0a11a0 msgr2=0x7fb71c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb71c0a11a0 0x7fb71c0a3650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb724005d10 tx=0x7fb724009310 comp rx=0 tx=0).stop 2023-07-14T11:21:46.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 msgr2=0x7fb734197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734197f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb728009230 tx=0x7fb72800b790 comp rx=0 tx=0).stop 2023-07-14T11:21:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 shutdown_connections 2023-07-14T11:21:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb734105840 0x7fb73419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fb71c0a11a0 0x7fb71c0a3650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb734100600 0x7fb734197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 --2- 172.21.15.161:0/1792031859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb734102f20 0x7fb734197f60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 >> 172.21.15.161:0/1792031859 conn(0x7fb7340fa000 msgr2=0x7fb734101260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:46.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 shutdown_connections 2023-07-14T11:21:46.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.323+0000 7fb7395ff700 1 -- 172.21.15.161:0/1792031859 wait complete. 2023-07-14T11:21:46.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:21:46.346 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:21:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:46 smithi182 ceph-mon[129366]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-14T11:21:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:46 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1747526258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:46 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3120890886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:46.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:46 smithi161 ceph-mon[95526]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-14T11:21:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:46 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1747526258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:46 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3120890886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:46.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.545+0000 7f91825ff700 1 -- 172.21.15.161:0/3989411817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 msgr2=0x7f917c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.545+0000 7f91825ff700 1 --2- 172.21.15.161:0/3989411817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c0fff00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f916c009990 tx=0x7f916c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.546+0000 7f91825ff700 1 -- 172.21.15.161:0/3989411817 shutdown_connections 2023-07-14T11:21:46.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.546+0000 7f91825ff700 1 --2- 172.21.15.161:0/3989411817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c0fff00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.546+0000 7f91825ff700 1 --2- 172.21.15.161:0/3989411817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f917c101670 0x7f917c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.546+0000 7f91825ff700 1 --2- 172.21.15.161:0/3989411817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f917c100ce0 0x7f917c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.546+0000 7f91825ff700 1 -- 172.21.15.161:0/3989411817 >> 172.21.15.161:0/3989411817 conn(0x7f917c0fb130 msgr2=0x7f917c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:46.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.546+0000 7f91825ff700 1 -- 172.21.15.161:0/3989411817 shutdown_connections 2023-07-14T11:21:46.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.546+0000 7f91825ff700 1 -- 172.21.15.161:0/3989411817 wait complete. 2023-07-14T11:21:46.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 Processor -- start 2023-07-14T11:21:46.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 -- start start 2023-07-14T11:21:46.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f917c100ce0 0x7f917c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f917c101670 0x7f917c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c19f620 con 0x7f917c0ffaf0 2023-07-14T11:21:46.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c19f760 con 0x7f917c101670 2023-07-14T11:21:46.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f91825ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c19f8f0 con 0x7f917c100ce0 2023-07-14T11:21:46.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f917bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f917bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39368/0 (socket says 172.21.15.161:39368) 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.547+0000 7f917bfff700 1 -- 172.21.15.161:0/2712857647 learned_addr learned my addr 172.21.15.161:0/2712857647 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f917b7fe700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f917c100ce0 0x7f917c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f9180b9c700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f917c101670 0x7f917c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f917bfff700 1 -- 172.21.15.161:0/2712857647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f917c100ce0 msgr2=0x7f917c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f917bfff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f917c100ce0 0x7f917c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f917bfff700 1 -- 172.21.15.161:0/2712857647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f917c101670 msgr2=0x7f917c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f917bfff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f917c101670 0x7f917c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f917bfff700 1 -- 172.21.15.161:0/2712857647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0095f0 con 0x7f917c0ffaf0 2023-07-14T11:21:46.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f917bfff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c197aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f916c005740 tx=0x7f916c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:46.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f91797fa700 1 -- 172.21.15.161:0/2712857647 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c01d070 con 0x7f917c0ffaf0 2023-07-14T11:21:46.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f91797fa700 1 -- 172.21.15.161:0/2712857647 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916c021410 con 0x7f917c0ffaf0 2023-07-14T11:21:46.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c19fb70 con 0x7f917c0ffaf0 2023-07-14T11:21:46.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.548+0000 7f91797fa700 1 -- 172.21.15.161:0/2712857647 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c017650 con 0x7f917c0ffaf0 2023-07-14T11:21:46.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.549+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c19fff0 con 0x7f917c0ffaf0 2023-07-14T11:21:46.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.550+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917c1921d0 con 0x7f917c0ffaf0 2023-07-14T11:21:46.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.551+0000 7f91797fa700 1 -- 172.21.15.161:0/2712857647 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f916c021580 con 0x7f917c0ffaf0 2023-07-14T11:21:46.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.553+0000 7f91797fa700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91640a11d0 0x7f91640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.553+0000 7f91797fa700 1 -- 172.21.15.161:0/2712857647 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f916c0cdd60 con 0x7f917c0ffaf0 2023-07-14T11:21:46.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.553+0000 7f917b7fe700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91640a11d0 0x7f91640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.554+0000 7f917b7fe700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91640a11d0 0x7f91640a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9170009c60 tx=0x7f9170009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:46.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.556+0000 7f91797fa700 1 -- 172.21.15.161:0/2712857647 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f916c097190 con 0x7f917c0ffaf0 2023-07-14T11:21:46.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:46 smithi175 ceph-mon[129081]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-14T11:21:46.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:46 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1747526258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:46.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:46 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3120890886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:46.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.752+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f917c0611c0 con 0x7f917c0ffaf0 2023-07-14T11:21:46.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.753+0000 7f91797fa700 1 -- 172.21.15.161:0/2712857647 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f916c09a7b0 con 0x7f917c0ffaf0 2023-07-14T11:21:46.757 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:21:46.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91640a11d0 msgr2=0x7f91640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91640a11d0 0x7f91640a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9170009c60 tx=0x7f9170009260 comp rx=0 tx=0).stop 2023-07-14T11:21:46.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 msgr2=0x7f917c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c197aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f916c005740 tx=0x7f916c005770 comp rx=0 tx=0).stop 2023-07-14T11:21:46.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 shutdown_connections 2023-07-14T11:21:46.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f917c0ffaf0 0x7f917c197aa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f91640a11d0 0x7f91640a3680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f917c101670 0x7f917c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 --2- 172.21.15.161:0/2712857647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f917c100ce0 0x7f917c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 >> 172.21.15.161:0/2712857647 conn(0x7f917c0fb130 msgr2=0x7f917c1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:46.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 shutdown_connections 2023-07-14T11:21:46.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.755+0000 7f91825ff700 1 -- 172.21.15.161:0/2712857647 wait complete. 2023-07-14T11:21:46.777 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:21:46.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 -- 172.21.15.161:0/2582479965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa494100ce0 msgr2=0x7fa494101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/2582479965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa494100ce0 0x7fa494101130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa484009990 tx=0x7fa484009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 -- 172.21.15.161:0/2582479965 shutdown_connections 2023-07-14T11:21:46.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/2582479965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa494101670 0x7fa494107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/2582479965 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa4940ffaf0 0x7fa4940fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/2582479965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa494100ce0 0x7fa494101130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 -- 172.21.15.161:0/2582479965 >> 172.21.15.161:0/2582479965 conn(0x7fa4940fb130 msgr2=0x7fa4940fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:46.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.977+0000 7fa49c7ff700 1 -- 172.21.15.161:0/2582479965 shutdown_connections 2023-07-14T11:21:46.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.978+0000 7fa49c7ff700 1 -- 172.21.15.161:0/2582479965 wait complete. 2023-07-14T11:21:46.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.978+0000 7fa49c7ff700 1 Processor -- start 2023-07-14T11:21:46.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49c7ff700 1 -- start start 2023-07-14T11:21:46.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49c7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4940ffaf0 0x7fa4940742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49c7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa494100ce0 0x7fa494072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49c7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa494101670 0x7fa494072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49c7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941996e0 con 0x7fa4940ffaf0 2023-07-14T11:21:46.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49c7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494073380 con 0x7fa494101670 2023-07-14T11:21:46.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49c7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494073540 con 0x7fa494100ce0 2023-07-14T11:21:46.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49a59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4940ffaf0 0x7fa4940742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49a59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4940ffaf0 0x7fa4940742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39394/0 (socket says 172.21.15.161:39394) 2023-07-14T11:21:46.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49a59b700 1 -- 172.21.15.161:0/3660138037 learned_addr learned my addr 172.21.15.161:0/3660138037 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:46.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa499d9a700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa494100ce0 0x7fa494072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49ad9c700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa494101670 0x7fa494072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49a59b700 1 -- 172.21.15.161:0/3660138037 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa494100ce0 msgr2=0x7fa494072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.979+0000 7fa49a59b700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa494100ce0 0x7fa494072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa49a59b700 1 -- 172.21.15.161:0/3660138037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa494101670 msgr2=0x7fa494072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa49a59b700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa494101670 0x7fa494072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa49a59b700 1 -- 172.21.15.161:0/3660138037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840095f0 con 0x7fa4940ffaf0 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa49ad9c700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa494101670 0x7fa494072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa499d9a700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa494100ce0 0x7fa494072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:46.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa49a59b700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4940ffaf0 0x7fa4940742c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d980 tx=0x7fa49000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:46.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa48b7fe700 1 -- 172.21.15.161:0/3660138037 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49000de60 con 0x7fa4940ffaf0 2023-07-14T11:21:46.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa48b7fe700 1 -- 172.21.15.161:0/3660138037 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4900044e0 con 0x7fa4940ffaf0 2023-07-14T11:21:46.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa48b7fe700 1 -- 172.21.15.161:0/3660138037 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490003be0 con 0x7fa4940ffaf0 2023-07-14T11:21:46.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4941a4090 con 0x7fa4940ffaf0 2023-07-14T11:21:46.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.980+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4941a44b0 con 0x7fa4940ffaf0 2023-07-14T11:21:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.981+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49410b310 con 0x7fa4940ffaf0 2023-07-14T11:21:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.983+0000 7fa48b7fe700 1 -- 172.21.15.161:0/3660138037 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa490009bb0 con 0x7fa4940ffaf0 2023-07-14T11:21:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.984+0000 7fa48b7fe700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa4800a11d0 0x7fa4800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.984+0000 7fa48b7fe700 1 -- 172.21.15.161:0/3660138037 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa4900cc9f0 con 0x7fa4940ffaf0 2023-07-14T11:21:46.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.985+0000 7fa499d9a700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa4800a11d0 0x7fa4800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:46.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.985+0000 7fa499d9a700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa4800a11d0 0x7fa4800a3680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa48400b2e0 tx=0x7fa484005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:46.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:46.988+0000 7fa48b7fe700 1 -- 172.21.15.161:0/3660138037 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa490095d90 con 0x7fa4940ffaf0 2023-07-14T11:21:47.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.191+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa49410b4a0 con 0x7fa4940ffaf0 2023-07-14T11:21:47.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.192+0000 7fa48b7fe700 1 -- 172.21.15.161:0/3660138037 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa490009940 con 0x7fa4940ffaf0 2023-07-14T11:21:47.195 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:47.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa4800a11d0 msgr2=0x7fa4800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa4800a11d0 0x7fa4800a3680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa48400b2e0 tx=0x7fa484005fd0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4940ffaf0 msgr2=0x7fa4940742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4940ffaf0 0x7fa4940742c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d980 tx=0x7fa49000dd40 comp rx=0 tx=0).stop 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 shutdown_connections 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4940ffaf0 0x7fa4940742c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa4800a11d0 0x7fa4800a3680 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa494101670 0x7fa494072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 --2- 172.21.15.161:0/3660138037 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa494100ce0 0x7fa494072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 >> 172.21.15.161:0/3660138037 conn(0x7fa4940fb130 msgr2=0x7fa4940fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:47.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 shutdown_connections 2023-07-14T11:21:47.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.194+0000 7fa49c7ff700 1 -- 172.21.15.161:0/3660138037 wait complete. 2023-07-14T11:21:47.217 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:21:47.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 -- 172.21.15.161:0/1414270105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c072f70 msgr2=0x7f478c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 --2- 172.21.15.161:0/1414270105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c072f70 0x7f478c0733e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f477c009990 tx=0x7f477c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 -- 172.21.15.161:0/1414270105 shutdown_connections 2023-07-14T11:21:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 --2- 172.21.15.161:0/1414270105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f478c1079b0 0x7f478c109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 --2- 172.21.15.161:0/1414270105 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f478c074550 0x7f478c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 --2- 172.21.15.161:0/1414270105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c072f70 0x7f478c0733e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 -- 172.21.15.161:0/1414270105 >> 172.21.15.161:0/1414270105 conn(0x7f478c0fb130 msgr2=0x7f478c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:47.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 -- 172.21.15.161:0/1414270105 shutdown_connections 2023-07-14T11:21:47.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.415+0000 7f4793bff700 1 -- 172.21.15.161:0/1414270105 wait complete. 2023-07-14T11:21:47.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 Processor -- start 2023-07-14T11:21:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 -- start start 2023-07-14T11:21:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f478c072f70 0x7f478c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f478c074550 0x7f478c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c1079b0 0x7f478c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19f650 con 0x7f478c1079b0 2023-07-14T11:21:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19f790 con 0x7f478c074550 2023-07-14T11:21:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.416+0000 7f4793bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19f920 con 0x7f478c072f70 2023-07-14T11:21:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479219c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c1079b0 0x7f478c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479199b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f478c072f70 0x7f478c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479119a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f478c074550 0x7f478c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479219c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c1079b0 0x7f478c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39424/0 (socket says 172.21.15.161:39424) 2023-07-14T11:21:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479219c700 1 -- 172.21.15.161:0/1821319176 learned_addr learned my addr 172.21.15.161:0/1821319176 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479119a700 1 -- 172.21.15.161:0/1821319176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f478c072f70 msgr2=0x7f478c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479119a700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f478c072f70 0x7f478c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479119a700 1 -- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c1079b0 msgr2=0x7f478c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479119a700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c1079b0 0x7f478c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479119a700 1 -- 172.21.15.161:0/1821319176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0095f0 con 0x7f478c074550 2023-07-14T11:21:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479199b700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f478c072f70 0x7f478c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479219c700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c1079b0 0x7f478c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f479119a700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f478c074550 0x7f478c198010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f477c00b680 tx=0x7f477c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.417+0000 7f4782ffd700 1 -- 172.21.15.161:0/1821319176 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c01d070 con 0x7f478c074550 2023-07-14T11:21:47.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.418+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c19fba0 con 0x7f478c074550 2023-07-14T11:21:47.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.418+0000 7f4782ffd700 1 -- 172.21.15.161:0/1821319176 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477c021410 con 0x7f478c074550 2023-07-14T11:21:47.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.418+0000 7f4782ffd700 1 -- 172.21.15.161:0/1821319176 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c021ea0 con 0x7f478c074550 2023-07-14T11:21:47.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.418+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c19ffc0 con 0x7f478c074550 2023-07-14T11:21:47.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.418+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c106060 con 0x7f478c074550 2023-07-14T11:21:47.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.420+0000 7f4782ffd700 1 -- 172.21.15.161:0/1821319176 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f477c0217f0 con 0x7f478c074550 2023-07-14T11:21:47.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.422+0000 7f4782ffd700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f47780a1150 0x7f47780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.422+0000 7f4782ffd700 1 -- 172.21.15.161:0/1821319176 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f477c0ce2a0 con 0x7f478c074550 2023-07-14T11:21:47.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.422+0000 7f479199b700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f47780a1150 0x7f47780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.423+0000 7f479199b700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f47780a1150 0x7f47780a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4788005950 tx=0x7f478800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:47.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.424+0000 7f4782ffd700 1 -- 172.21.15.161:0/1821319176 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f477c0976d0 con 0x7f478c074550 2023-07-14T11:21:47.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:47 smithi182 ceph-mon[129366]: pgmap v338: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:47.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:47 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1792031859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:47.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:47 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2712857647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:47.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:47 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3660138037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:47 smithi161 ceph-mon[95526]: pgmap v338: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:47 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1792031859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:47 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2712857647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:47.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:47 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3660138037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.624+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f478c02c9e0 con 0x7f478c074550 2023-07-14T11:21:47.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.625+0000 7f4782ffd700 1 -- 172.21.15.161:0/1821319176 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f477c09acf0 con 0x7f478c074550 2023-07-14T11:21:47.628 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:21:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f47780a1150 msgr2=0x7f47780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f47780a1150 0x7f47780a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4788005950 tx=0x7f478800b410 comp rx=0 tx=0).stop 2023-07-14T11:21:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f478c074550 msgr2=0x7f478c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f478c074550 0x7f478c198010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f477c00b680 tx=0x7f477c005740 comp rx=0 tx=0).stop 2023-07-14T11:21:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 shutdown_connections 2023-07-14T11:21:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c1079b0 0x7f478c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f47780a1150 0x7f47780a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f478c072f70 0x7f478c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 --2- 172.21.15.161:0/1821319176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f478c074550 0x7f478c198010 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 >> 172.21.15.161:0/1821319176 conn(0x7f478c0fb130 msgr2=0x7f478c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 shutdown_connections 2023-07-14T11:21:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.627+0000 7f4793bff700 1 -- 172.21.15.161:0/1821319176 wait complete. 2023-07-14T11:21:47.650 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:21:47.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:47 smithi175 ceph-mon[129081]: pgmap v338: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T11:21:47.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:47 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1792031859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:47.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:47 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2712857647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:47.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:47 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3660138037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:47.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.869+0000 7f6d77dff700 1 -- 172.21.15.161:0/3722852016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 msgr2=0x7f6d700fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.869+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3722852016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 0x7f6d700fde20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009990 tx=0x7f6d6c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.870+0000 7f6d77dff700 1 -- 172.21.15.161:0/3722852016 shutdown_connections 2023-07-14T11:21:47.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.870+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3722852016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d700ff470 0x7f6d700ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.870+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3722852016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 0x7f6d700fde20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.870+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3722852016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d700feac0 0x7f6d700fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.870+0000 7f6d77dff700 1 -- 172.21.15.161:0/3722852016 >> 172.21.15.161:0/3722852016 conn(0x7f6d700f9340 msgr2=0x7f6d700fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:47.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.870+0000 7f6d77dff700 1 -- 172.21.15.161:0/3722852016 shutdown_connections 2023-07-14T11:21:47.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.870+0000 7f6d77dff700 1 -- 172.21.15.161:0/3722852016 wait complete. 2023-07-14T11:21:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 Processor -- start 2023-07-14T11:21:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 -- start start 2023-07-14T11:21:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 0x7f6d70197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d700feac0 0x7f6d70197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d700ff470 0x7f6d7019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019f5f0 con 0x7f6d700ff470 2023-07-14T11:21:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019f730 con 0x7f6d700feac0 2023-07-14T11:21:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d77dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019f8c0 con 0x7f6d700fda10 2023-07-14T11:21:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d75b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 0x7f6d70197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d7639c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d700ff470 0x7f6d7019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d75b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 0x7f6d70197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52238/0 (socket says 172.21.15.161:52238) 2023-07-14T11:21:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d75b9b700 1 -- 172.21.15.161:0/3006269837 learned_addr learned my addr 172.21.15.161:0/3006269837 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.871+0000 7f6d7539a700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d700feac0 0x7f6d70197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d7639c700 1 -- 172.21.15.161:0/3006269837 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 msgr2=0x7f6d70197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d7639c700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 0x7f6d70197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d7639c700 1 -- 172.21.15.161:0/3006269837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d700feac0 msgr2=0x7f6d70197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d7639c700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d700feac0 0x7f6d70197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d7639c700 1 -- 172.21.15.161:0/3006269837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c0095f0 con 0x7f6d700ff470 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d7639c700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d700ff470 0x7f6d7019f020 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800efc0 tx=0x7f6d6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:47.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d66ffd700 1 -- 172.21.15.161:0/3006269837 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68018070 con 0x7f6d700ff470 2023-07-14T11:21:47.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7019fb40 con 0x7f6d700ff470 2023-07-14T11:21:47.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d66ffd700 1 -- 172.21.15.161:0/3006269837 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6800f040 con 0x7f6d700ff470 2023-07-14T11:21:47.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d66ffd700 1 -- 172.21.15.161:0/3006269837 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6800cd40 con 0x7f6d700ff470 2023-07-14T11:21:47.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.872+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d701a00a0 con 0x7f6d700ff470 2023-07-14T11:21:47.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.873+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d70109df0 con 0x7f6d700ff470 2023-07-14T11:21:47.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.875+0000 7f6d66ffd700 1 -- 172.21.15.161:0/3006269837 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6d68003a00 con 0x7f6d700ff470 2023-07-14T11:21:47.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.877+0000 7f6d66ffd700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d5c0a1150 0x7f6d5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:47.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.877+0000 7f6d66ffd700 1 -- 172.21.15.161:0/3006269837 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6d680cc830 con 0x7f6d700ff470 2023-07-14T11:21:47.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.877+0000 7f6d75b9b700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d5c0a1150 0x7f6d5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:47.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.878+0000 7f6d75b9b700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d5c0a1150 0x7f6d5c0a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c006010 tx=0x7f6d6c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:47.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:47.881+0000 7f6d66ffd700 1 -- 172.21.15.161:0/3006269837 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6d68095c60 con 0x7f6d700ff470 2023-07-14T11:21:48.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.077+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6d70061120 con 0x7f6d700ff470 2023-07-14T11:21:48.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.077+0000 7f6d66ffd700 1 -- 172.21.15.161:0/3006269837 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6d68099280 con 0x7f6d700ff470 2023-07-14T11:21:48.081 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:48.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d5c0a1150 msgr2=0x7f6d5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d5c0a1150 0x7f6d5c0a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c006010 tx=0x7f6d6c00b260 comp rx=0 tx=0).stop 2023-07-14T11:21:48.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d700ff470 msgr2=0x7f6d7019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d700ff470 0x7f6d7019f020 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800efc0 tx=0x7f6d6800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 shutdown_connections 2023-07-14T11:21:48.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d700ff470 0x7f6d7019f020 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6d5c0a1150 0x7f6d5c0a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6d700fda10 0x7f6d70197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 --2- 172.21.15.161:0/3006269837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d700feac0 0x7f6d70197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 >> 172.21.15.161:0/3006269837 conn(0x7f6d700f9340 msgr2=0x7f6d700fb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:48.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 shutdown_connections 2023-07-14T11:21:48.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.080+0000 7f6d77dff700 1 -- 172.21.15.161:0/3006269837 wait complete. 2023-07-14T11:21:48.102 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:21:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.299+0000 7f07ad7ff700 1 -- 172.21.15.161:0/3537027987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a8101680 msgr2=0x7f07a8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.299+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/3537027987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a8101680 0x7f07a8107f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f07a000b0a0 tx=0x7f07a000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.299+0000 7f07ad7ff700 1 -- 172.21.15.161:0/3537027987 shutdown_connections 2023-07-14T11:21:48.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.299+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/3537027987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a8101680 0x7f07a8107f40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.299+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/3537027987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07a80ffb00 0x7f07a80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.299+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/3537027987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8100cf0 0x7f07a8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.299+0000 7f07ad7ff700 1 -- 172.21.15.161:0/3537027987 >> 172.21.15.161:0/3537027987 conn(0x7f07a80fb0e0 msgr2=0x7f07a80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:48.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.300+0000 7f07ad7ff700 1 -- 172.21.15.161:0/3537027987 shutdown_connections 2023-07-14T11:21:48.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.300+0000 7f07ad7ff700 1 -- 172.21.15.161:0/3537027987 wait complete. 2023-07-14T11:21:48.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.301+0000 7f07ad7ff700 1 Processor -- start 2023-07-14T11:21:48.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.301+0000 7f07ad7ff700 1 -- start start 2023-07-14T11:21:48.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07ad7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a80ffb00 0x7f07a80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07ad7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07a8100cf0 0x7f07a8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07ad7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8101680 0x7f07a8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07ad7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80735e0 con 0x7f07a8101680 2023-07-14T11:21:48.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07ad7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a8073750 con 0x7f07a80ffb00 2023-07-14T11:21:48.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07ad7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819f920 con 0x7f07a8100cf0 2023-07-14T11:21:48.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8101680 0x7f07a8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8101680 0x7f07a8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48262/0 (socket says 172.21.15.161:48262) 2023-07-14T11:21:48.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 -- 172.21.15.161:0/2334126957 learned_addr learned my addr 172.21.15.161:0/2334126957 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:48.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a67fc700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07a8100cf0 0x7f07a8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a6ffd700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a80ffb00 0x7f07a80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 -- 172.21.15.161:0/2334126957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07a8100cf0 msgr2=0x7f07a8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07a8100cf0 0x7f07a8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 -- 172.21.15.161:0/2334126957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a80ffb00 msgr2=0x7f07a80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a80ffb00 0x7f07a80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.302+0000 7f07a77fe700 1 -- 172.21.15.161:0/2334126957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798009520 con 0x7f07a8101680 2023-07-14T11:21:48.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.303+0000 7f07a77fe700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8101680 0x7f07a8072e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f07a000b780 tx=0x7f07a000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:48.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.303+0000 7f078ffff700 1 -- 172.21.15.161:0/2334126957 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000e040 con 0x7f07a8101680 2023-07-14T11:21:48.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.303+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a0009d00 con 0x7f07a8101680 2023-07-14T11:21:48.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.303+0000 7f078ffff700 1 -- 172.21.15.161:0/2334126957 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a0004c60 con 0x7f07a8101680 2023-07-14T11:21:48.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.303+0000 7f078ffff700 1 -- 172.21.15.161:0/2334126957 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a001b6b0 con 0x7f07a8101680 2023-07-14T11:21:48.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.303+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a819fee0 con 0x7f07a8101680 2023-07-14T11:21:48.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.304+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a8106fd0 con 0x7f07a8101680 2023-07-14T11:21:48.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.306+0000 7f078ffff700 1 -- 172.21.15.161:0/2334126957 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f07a001d480 con 0x7f07a8101680 2023-07-14T11:21:48.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.307+0000 7f078ffff700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f07900a11a0 0x7f07900a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.307+0000 7f078ffff700 1 -- 172.21.15.161:0/2334126957 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f07a00ce3e0 con 0x7f07a8101680 2023-07-14T11:21:48.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.308+0000 7f07a6ffd700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f07900a11a0 0x7f07900a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.308+0000 7f07a6ffd700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f07900a11a0 0x7f07900a3650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f07a8100b50 tx=0x7f0798009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:48.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.311+0000 7f078ffff700 1 -- 172.21.15.161:0/2334126957 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f07a0097810 con 0x7f07a8101680 2023-07-14T11:21:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:48 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1821319176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:48 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3006269837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:48.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.554+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07a802cce0 con 0x7f07a8101680 2023-07-14T11:21:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.555+0000 7f078ffff700 1 -- 172.21.15.161:0/2334126957 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f07a009ae30 con 0x7f07a8101680 2023-07-14T11:21:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:48.559 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:21:48.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.557+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f07900a11a0 msgr2=0x7f07900a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.557+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f07900a11a0 0x7f07900a3650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f07a8100b50 tx=0x7f0798009260 comp rx=0 tx=0).stop 2023-07-14T11:21:48.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.557+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8101680 msgr2=0x7f07a8072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.557+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8101680 0x7f07a8072e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f07a000b780 tx=0x7f07a000bd00 comp rx=0 tx=0).stop 2023-07-14T11:21:48.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 shutdown_connections 2023-07-14T11:21:48.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07a8101680 0x7f07a8072e40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f07900a11a0 0x7f07900a3650 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a80ffb00 0x7f07a80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 --2- 172.21.15.161:0/2334126957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07a8100cf0 0x7f07a8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 >> 172.21.15.161:0/2334126957 conn(0x7f07a80fb0e0 msgr2=0x7f07a8104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:48.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 shutdown_connections 2023-07-14T11:21:48.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.558+0000 7f07ad7ff700 1 -- 172.21.15.161:0/2334126957 wait complete. 2023-07-14T11:21:48.565 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:21:48.580 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:48.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:48 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1821319176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:48 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3006269837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:48.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:48 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1821319176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:48.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:48 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3006269837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:48.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 -- 172.21.15.161:0/3074176113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 msgr2=0x7fc764101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 --2- 172.21.15.161:0/3074176113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764101140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc758009990 tx=0x7fc758009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 -- 172.21.15.161:0/3074176113 shutdown_connections 2023-07-14T11:21:48.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 --2- 172.21.15.161:0/3074176113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc764101680 0x7fc764107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 --2- 172.21.15.161:0/3074176113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc7640ffb00 0x7fc7640fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 --2- 172.21.15.161:0/3074176113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764101140 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 -- 172.21.15.161:0/3074176113 >> 172.21.15.161:0/3074176113 conn(0x7fc7640fb0e0 msgr2=0x7fc7640fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:48.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 -- 172.21.15.161:0/3074176113 shutdown_connections 2023-07-14T11:21:48.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.777+0000 7fc769dff700 1 -- 172.21.15.161:0/3074176113 wait complete. 2023-07-14T11:21:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.778+0000 7fc769dff700 1 Processor -- start 2023-07-14T11:21:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.778+0000 7fc769dff700 1 -- start start 2023-07-14T11:21:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc769dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7640ffb00 0x7fc764195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc769dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc769dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc764101680 0x7fc76419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc769dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76419d3a0 con 0x7fc764100cf0 2023-07-14T11:21:48.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc769dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76419d4e0 con 0x7fc7640ffb00 2023-07-14T11:21:48.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc769dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76419d670 con 0x7fc764101680 2023-07-14T11:21:48.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48284/0 (socket says 172.21.15.161:48284) 2023-07-14T11:21:48.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 -- 172.21.15.161:0/1748068757 learned_addr learned my addr 172.21.15.161:0/1748068757 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc7637fe700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7640ffb00 0x7fc764195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc763fff700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc764101680 0x7fc76419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 -- 172.21.15.161:0/1748068757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc764101680 msgr2=0x7fc76419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc764101680 0x7fc76419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 -- 172.21.15.161:0/1748068757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7640ffb00 msgr2=0x7fc764195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7640ffb00 0x7fc764195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc762ffd700 1 -- 172.21.15.161:0/1748068757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7580095f0 con 0x7fc764100cf0 2023-07-14T11:21:48.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.779+0000 7fc7637fe700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7640ffb00 0x7fc764195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:48.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.780+0000 7fc762ffd700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764195d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc75800c010 tx=0x7fc758004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:48.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.780+0000 7fc760ff9700 1 -- 172.21.15.161:0/1748068757 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75801d070 con 0x7fc764100cf0 2023-07-14T11:21:48.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.780+0000 7fc760ff9700 1 -- 172.21.15.161:0/1748068757 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7580054f0 con 0x7fc764100cf0 2023-07-14T11:21:48.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.780+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76419d8f0 con 0x7fc764100cf0 2023-07-14T11:21:48.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.780+0000 7fc760ff9700 1 -- 172.21.15.161:0/1748068757 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc758004ca0 con 0x7fc764100cf0 2023-07-14T11:21:48.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.780+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76419dd10 con 0x7fc764100cf0 2023-07-14T11:21:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.781+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc764106ff0 con 0x7fc764100cf0 2023-07-14T11:21:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.783+0000 7fc760ff9700 1 -- 172.21.15.161:0/1748068757 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc75800fbc0 con 0x7fc764100cf0 2023-07-14T11:21:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.784+0000 7fc760ff9700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc74c0a1220 0x7fc74c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.784+0000 7fc760ff9700 1 -- 172.21.15.161:0/1748068757 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc7580cdf40 con 0x7fc764100cf0 2023-07-14T11:21:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.785+0000 7fc7637fe700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc74c0a1220 0x7fc74c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.785+0000 7fc7637fe700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc74c0a1220 0x7fc74c0a36d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc764100b50 tx=0x7fc754007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.788+0000 7fc760ff9700 1 -- 172.21.15.161:0/1748068757 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc75805a790 con 0x7fc764100cf0 2023-07-14T11:21:48.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.989+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc76402c850 con 0x7fc764100cf0 2023-07-14T11:21:48.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.990+0000 7fc760ff9700 1 -- 172.21.15.161:0/1748068757 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7fc758097370 con 0x7fc764100cf0 2023-07-14T11:21:48.994 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:48.995 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:48.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc74c0a1220 msgr2=0x7fc74c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc74c0a1220 0x7fc74c0a36d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc764100b50 tx=0x7fc754007400 comp rx=0 tx=0).stop 2023-07-14T11:21:48.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 msgr2=0x7fc764195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:48.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764195d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc75800c010 tx=0x7fc758004730 comp rx=0 tx=0).stop 2023-07-14T11:21:48.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 shutdown_connections 2023-07-14T11:21:48.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fc74c0a1220 0x7fc74c0a36d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc764101680 0x7fc76419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7640ffb00 0x7fc764195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 --2- 172.21.15.161:0/1748068757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc764100cf0 0x7fc764195d60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:48.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 >> 172.21.15.161:0/1748068757 conn(0x7fc7640fb0e0 msgr2=0x7fc764104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:48.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 shutdown_connections 2023-07-14T11:21:48.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:48.993+0000 7fc769dff700 1 -- 172.21.15.161:0/1748068757 wait complete. 2023-07-14T11:21:49.016 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T11:19:41.650966+0000', 'modified': '2023-07-14T11:20:12.668182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24575, 'mds_1': 24544, 'mds_2': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/4098726991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 4098726991}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 4098726991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.182:6835/1383844371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.182:6834', 'nonce': 1383844371}, {'type': 'v1', 'addr': '172.21.15.182:6835', 'nonce': 1383844371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.182:6837/1326049792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.182:6836', 'nonce': 1326049792}, {'type': 'v1', 'addr': '172.21.15.182:6837', 'nonce': 1326049792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.175:6837/4186191407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 4186191407}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 4186191407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.182:6839/392865750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.182:6838', 'nonce': 392865750}, {'type': 'v1', 'addr': '172.21.15.182:6839', 'nonce': 392865750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6839/2144664422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 2144664422}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 2144664422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T11:21:49.016 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T11:21:49.016 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-14T11:21:49.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 -- 172.21.15.161:0/3889234750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af40ffae0 msgr2=0x7f6af40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/3889234750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af40ffae0 0x7f6af40ffef0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009990 tx=0x7f6af0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 -- 172.21.15.161:0/3889234750 shutdown_connections 2023-07-14T11:21:49.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/3889234750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af40ffae0 0x7f6af40ffef0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/3889234750 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6af4101660 0x7f6af4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/3889234750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af4100cd0 0x7f6af4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 -- 172.21.15.161:0/3889234750 >> 172.21.15.161:0/3889234750 conn(0x7f6af40fb0e0 msgr2=0x7f6af40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 -- 172.21.15.161:0/3889234750 shutdown_connections 2023-07-14T11:21:49.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.217+0000 7f6afb3ff700 1 -- 172.21.15.161:0/3889234750 wait complete. 2023-07-14T11:21:49.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.218+0000 7f6afb3ff700 1 Processor -- start 2023-07-14T11:21:49.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6afb3ff700 1 -- start start 2023-07-14T11:21:49.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6afb3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af40ffae0 0x7f6af4195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6afb3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af4100cd0 0x7f6af4195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6afb3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6af4101660 0x7f6af419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6afb3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af419d410 con 0x7f6af4100cd0 2023-07-14T11:21:49.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6afb3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af419d550 con 0x7f6af40ffae0 2023-07-14T11:21:49.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6afb3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af419d6e0 con 0x7f6af4101660 2023-07-14T11:21:49.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.219+0000 7f6af899a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af4100cd0 0x7f6af4195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af999c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6af4101660 0x7f6af419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af919b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af40ffae0 0x7f6af4195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af4100cd0 0x7f6af4195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48308/0 (socket says 172.21.15.161:48308) 2023-07-14T11:21:49.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 -- 172.21.15.161:0/116417157 learned_addr learned my addr 172.21.15.161:0/116417157 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 -- 172.21.15.161:0/116417157 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6af4101660 msgr2=0x7f6af419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6af4101660 0x7f6af419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 -- 172.21.15.161:0/116417157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af40ffae0 msgr2=0x7f6af4195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af40ffae0 0x7f6af4195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 -- 172.21.15.161:0/116417157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00095f0 con 0x7f6af4100cd0 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af999c700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6af4101660 0x7f6af419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af919b700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af40ffae0 0x7f6af4195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6af899a700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af4100cd0 0x7f6af4195dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e9a0 tx=0x7f6ae400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6aea7fc700 1 -- 172.21.15.161:0/116417157 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4009790 con 0x7f6af4100cd0 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6aea7fc700 1 -- 172.21.15.161:0/116417157 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae4010430 con 0x7f6af4100cd0 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af419d9c0 con 0x7f6af4100cd0 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.220+0000 7f6aea7fc700 1 -- 172.21.15.161:0/116417157 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4018680 con 0x7f6af4100cd0 2023-07-14T11:21:49.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.221+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af419ded0 con 0x7f6af4100cd0 2023-07-14T11:21:49.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.224+0000 7f6aea7fc700 1 -- 172.21.15.161:0/116417157 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6ae40105a0 con 0x7f6af4100cd0 2023-07-14T11:21:49.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 Processor -- start 2023-07-14T11:21:49.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 -- start start 2023-07-14T11:21:49.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6aea7fc700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6ae00a1150 0x7f6ae00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7bf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe6c4c60 con 0x5642fe7c0c20 2023-07-14T11:21:49.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe7179f0 con 0x5642fe7bfdd0 2023-07-14T11:21:49.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe710e80 con 0x5642fe7bf480 2023-07-14T11:21:49.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.225+0000 7f6aea7fc700 1 -- 172.21.15.161:0/116417157 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6ae4014070 con 0x7f6af4100cd0 2023-07-14T11:21:49.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af999c700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6ae00a1150 0x7f6ae00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af919b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7bf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af899a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af919b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7bf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52300/0 (socket says 172.21.15.161:52300) 2023-07-14T11:21:49.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af919b700 1 -- 172.21.15.161:0/3638138999 learned_addr learned my addr 172.21.15.161:0/3638138999 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af999c700 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af899a700 1 -- 172.21.15.161:0/3638138999 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 msgr2=0x5642fe7bf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af899a700 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7bf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af899a700 1 -- 172.21.15.161:0/3638138999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 msgr2=0x5642fe7c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af899a700 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af899a700 1 -- 172.21.15.161:0/3638138999 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642fe53ed90 con 0x5642fe7bfdd0 2023-07-14T11:21:49.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af919b700 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7bf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af999c700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6ae00a1150 0x7f6ae00a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007f40 tx=0x7f6aec007580 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6af899a700 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c0690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e970 tx=0x7f6ae4099930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad39a6e8934a064 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.226+0000 7f6adf7fe700 1 -- 172.21.15.161:0/3638138999 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae409df10 con 0x5642fe7bfdd0 2023-07-14T11:21:49.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6adf7fe700 1 -- 172.21.15.161:0/3638138999 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae40a2330 con 0x5642fe7bfdd0 2023-07-14T11:21:49.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6adf7fe700 1 -- 172.21.15.161:0/3638138999 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40a1ea0 con 0x5642fe7bfdd0 2023-07-14T11:21:49.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/3638138999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 msgr2=0x5642fe7c0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c0690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e970 tx=0x7f6ae4099930 comp rx=0 tx=0).stop 2023-07-14T11:21:49.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/3638138999 shutdown_connections 2023-07-14T11:21:49.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7bf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/3638138999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c0690 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/3638138999 >> 172.21.15.161:0/3638138999 conn(0x5642fe7b4d50 msgr2=0x5642fe7ba180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/3638138999 shutdown_connections 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.227+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/3638138999 wait complete. 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.228+0000 7f6b0ece7b80 1 -- start start 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.228+0000 7f6b0ece7b80 1 Processor -- start 2023-07-14T11:21:49.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.228+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe6b0a70 con 0x5642fe7c0c20 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe6b0770 con 0x5642fe7bfdd0 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe5948d0 con 0x5642fe7bf480 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6b0ece7b80 10 client.0 _ll_register_callbacks cb 0x5642fe6ae710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af899a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af999c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af899a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35298/0 (socket says 172.21.15.161:35298) 2023-07-14T11:21:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af899a700 1 -- 172.21.15.161:0/1089395653 learned_addr learned my addr 172.21.15.161:0/1089395653 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af919b700 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af999c700 1 -- 172.21.15.161:0/1089395653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 msgr2=0x5642fe7c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af999c700 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af999c700 1 -- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 msgr2=0x5642fe7c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af999c700 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af999c700 1 -- 172.21.15.161:0/1089395653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af4108850 con 0x5642fe7c0c20 2023-07-14T11:21:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.229+0000 7f6af899a700 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6af999c700 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0730 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007fd0 tx=0x7f6aec009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9385ed70ad3996e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6adcff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6adcff9700 1 -- 172.21.15.161:0/1089395653 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0046b0 con 0x5642fe7c0c20 2023-07-14T11:21:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642fe66a4a0 con 0x5642fe7c0c20 2023-07-14T11:21:49.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6adcff9700 1 -- 172.21.15.161:0/1089395653 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6aec004810 con 0x5642fe7c0c20 2023-07-14T11:21:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6adcff9700 1 -- 172.21.15.161:0/1089395653 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aec01b450 con 0x5642fe7c0c20 2023-07-14T11:21:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6b0ece7b80 10 client.15312 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.230+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f6af419ded0 con 0x5642fe7c0c20 2023-07-14T11:21:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6adcff9700 1 -- 172.21.15.161:0/1089395653 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642fe66a4a0 con 0x5642fe7c0c20 2023-07-14T11:21:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6adcff9700 1 -- 172.21.15.161:0/1089395653 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6aec024510 con 0x5642fe7c0c20 2023-07-14T11:21:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6b0ece7b80 10 client.15312 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:49.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6b0ece7b80 10 client.15312 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T11:21:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6b0ece7b80 20 client.15312 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe7d4480 0x5642fe7d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6b0ece7b80 4 client.15312 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:21:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642fe709e60 con 0x5642fe7d4480 2023-07-14T11:21:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.231+0000 7f6af919b700 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe7d4480 0x5642fe7d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.232+0000 7f6adcff9700 10 client.15312 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:49.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.232+0000 7f6af919b700 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe7d4480 0x5642fe7d3830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d997eddf24a790cb server_cookie=c225ab9de5b88ba9 in_seq=0 out_seq=0 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.232+0000 7f6adcff9700 1 -- 172.21.15.161:0/1089395653 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5642fe709e60 con 0x5642fe7d4480 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.232+0000 7f6adcff9700 10 client.15312 handle_command_reply: tid=0 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.232+0000 7f6b0ece7b80 1 client.15312 shutdown 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 msgr2=0x5642fe7d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0730 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007fd0 tx=0x7f6aec009100 comp rx=0 tx=0).stop 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 shutdown_connections 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe7d4480 0x5642fe7d3830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0730 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bf480 0x5642fe7c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/1089395653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7bfdd0 0x5642fe7c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6adcff9700 0 client.15312 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 >> 172.21.15.161:0/1089395653 conn(0x5642fe7b4d50 msgr2=0x5642fe7b9860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 shutdown_connections 2023-07-14T11:21:49.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/1089395653 wait complete. 2023-07-14T11:21:49.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.233+0000 7f6b0ece7b80 20 client.15312 trim_cache size 0 max 16384 2023-07-14T11:21:49.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.251+0000 7f6b0ece7b80 1 Processor -- start 2023-07-14T11:21:49.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.251+0000 7f6b0ece7b80 1 -- start start 2023-07-14T11:21:49.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe6df7c0 con 0x5642fe7bf480 2023-07-14T11:21:49.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe5a0e10 con 0x5642fe7c0c20 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe6acd50 con 0x5642fe7bfdd0 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7d3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48340/0 (socket says 172.21.15.161:48340) 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 -- 172.21.15.161:0/201211412 learned_addr learned my addr 172.21.15.161:0/201211412 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af899a700 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af999c700 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 -- 172.21.15.161:0/201211412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 msgr2=0x5642fe7d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7d3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 -- 172.21.15.161:0/201211412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 msgr2=0x5642fe7d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af919b700 1 -- 172.21.15.161:0/201211412 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642fe7c12c0 con 0x5642fe7bf480 2023-07-14T11:21:49.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.252+0000 7f6af999c700 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:49.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.253+0000 7f6af919b700 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7d3830 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b620 tx=0x7f6af0005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204652975097c90d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.253+0000 7f6adffff700 1 -- 172.21.15.161:0/201211412 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af001d040 con 0x5642fe7bf480 2023-07-14T11:21:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.253+0000 7f6adffff700 1 -- 172.21.15.161:0/201211412 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af001e040 con 0x5642fe7bf480 2023-07-14T11:21:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.253+0000 7f6adffff700 1 -- 172.21.15.161:0/201211412 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0004760 con 0x5642fe7bf480 2023-07-14T11:21:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/201211412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 msgr2=0x5642fe7d3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7d3830 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b620 tx=0x7f6af0005fb0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/201211412 shutdown_connections 2023-07-14T11:21:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7d3830 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe7d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/201211412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/201211412 >> 172.21.15.161:0/201211412 conn(0x5642fe7b4d50 msgr2=0x5642fe7c7050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/201211412 shutdown_connections 2023-07-14T11:21:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/201211412 wait complete. 2023-07-14T11:21:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 -- start start 2023-07-14T11:21:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.254+0000 7f6b0ece7b80 1 Processor -- start 2023-07-14T11:21:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6b0ece7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe828300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe7cc280 con 0x5642fe7bf480 2023-07-14T11:21:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe828840 con 0x5642fe7c0c20 2023-07-14T11:21:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6b0ece7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fe8289d0 con 0x5642fe7bfdd0 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6b0ece7b80 10 client.0 _ll_register_callbacks cb 0x7f6aec019f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48356/0 (socket says 172.21.15.161:48356) 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 -- 172.21.15.161:0/2657563325 learned_addr learned my addr 172.21.15.161:0/2657563325 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af999c700 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe828300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af899a700 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 -- 172.21.15.161:0/2657563325 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 msgr2=0x5642fe7cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 -- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 msgr2=0x5642fe828300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe828300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.255+0000 7f6af919b700 1 -- 172.21.15.161:0/2657563325 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af000bce0 con 0x5642fe7bf480 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6af919b700 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7cb670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6af002b040 tx=0x7f6af0005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe7134017a918b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6adcff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6adcff9700 1 -- 172.21.15.161:0/2657563325 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af001d040 con 0x5642fe7bf480 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ae40a2330 con 0x5642fe7bf480 2023-07-14T11:21:49.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6adcff9700 1 -- 172.21.15.161:0/2657563325 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6af0029210 con 0x5642fe7bf480 2023-07-14T11:21:49.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6adcff9700 1 -- 172.21.15.161:0/2657563325 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af00293d0 con 0x5642fe7bf480 2023-07-14T11:21:49.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6b0ece7b80 10 client.15318 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:49.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5642fe527c10 con 0x5642fe7bf480 2023-07-14T11:21:49.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.256+0000 7f6adcff9700 1 -- 172.21.15.161:0/2657563325 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae40a2330 con 0x5642fe7bf480 2023-07-14T11:21:49.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6adcff9700 1 -- 172.21.15.161:0/2657563325 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6af003a030 con 0x5642fe7bf480 2023-07-14T11:21:49.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6b0ece7b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:49.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6b0ece7b80 10 client.15318 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T11:21:49.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6b0ece7b80 20 client.15318 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:49.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe82d550 0x5642fe829ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6b0ece7b80 4 client.15318 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-07-14T11:21:49.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5642fe6cc000 con 0x5642fe82d550 2023-07-14T11:21:49.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6af899a700 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe82d550 0x5642fe829ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6af899a700 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe82d550 0x5642fe829ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=577606f42dae807d server_cookie=151c037fefe3da4e in_seq=0 out_seq=0 2023-07-14T11:21:49.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.257+0000 7f6adcff9700 10 client.15318 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:49.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6adcff9700 1 -- 172.21.15.161:0/2657563325 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5642fe6cc000 con 0x5642fe82d550 2023-07-14T11:21:49.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6adcff9700 10 client.15318 handle_command_reply: tid=0 2023-07-14T11:21:49.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 client.15318 shutdown 2023-07-14T11:21:49.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 msgr2=0x5642fe7cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7cb670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6af002b040 tx=0x7f6af0005220 comp rx=0 tx=0).stop 2023-07-14T11:21:49.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 shutdown_connections 2023-07-14T11:21:49.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5642fe82d550 0x5642fe829ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642fe7bf480 0x5642fe7cb670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642fe7c0c20 0x5642fe828300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6adcff9700 0 client.15318 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:21:49.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 --2- 172.21.15.161:0/2657563325 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5642fe7bfdd0 0x5642fe7cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 >> 172.21.15.161:0/2657563325 conn(0x5642fe7b4d50 msgr2=0x5642fe7d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 shutdown_connections 2023-07-14T11:21:49.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 1 -- 172.21.15.161:0/2657563325 wait complete. 2023-07-14T11:21:49.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.258+0000 7f6b0ece7b80 20 client.15318 trim_cache size 0 max 16384 2023-07-14T11:21:49.271 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:21:49.271 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:21:49.271 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 2, 2023-07-14T11:21:49.271 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24533, 2023-07-14T11:21:49.271 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2023-07-14T11:21:49.271 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2023-07-14T11:21:49.272 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2023-07-14T11:21:49.272 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 97.550506354999996, 2023-07-14T11:21:49.272 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 22, 2023-07-14T11:21:49.272 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 92, 2023-07-14T11:21:49.272 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T11:21:49.272 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 144.03271577199999 2023-07-14T11:21:49.272 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:21:49.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6ae00a1150 msgr2=0x7f6ae00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6ae00a1150 0x7f6ae00a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007f40 tx=0x7f6aec007580 comp rx=0 tx=0).stop 2023-07-14T11:21:49.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af4100cd0 msgr2=0x7f6af4195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af4100cd0 0x7f6af4195dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e9a0 tx=0x7f6ae400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 shutdown_connections 2023-07-14T11:21:49.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6ae00a1150 0x7f6ae00a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6af4101660 0x7f6af419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af40ffae0 0x7f6af4195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 --2- 172.21.15.161:0/116417157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6af4100cd0 0x7f6af4195dd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 >> 172.21.15.161:0/116417157 conn(0x7f6af40fb0e0 msgr2=0x7f6af40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 shutdown_connections 2023-07-14T11:21:49.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.259+0000 7f6afb3ff700 1 -- 172.21.15.161:0/116417157 wait complete. 2023-07-14T11:21:49.278 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:49 smithi161 ceph-mon[95526]: pgmap v339: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T11:21:49.278 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2334126957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:49.278 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1748068757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:49.280 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T11:21:49.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 -- 172.21.15.161:0/2574139159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 msgr2=0x7f173c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 --2- 172.21.15.161:0/2574139159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 0x7f173c107f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f173400b0a0 tx=0x7f173400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 -- 172.21.15.161:0/2574139159 shutdown_connections 2023-07-14T11:21:49.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 --2- 172.21.15.161:0/2574139159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 0x7f173c107f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 --2- 172.21.15.161:0/2574139159 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f173c0ffae0 0x7f173c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 --2- 172.21.15.161:0/2574139159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f173c100cd0 0x7f173c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 -- 172.21.15.161:0/2574139159 >> 172.21.15.161:0/2574139159 conn(0x7f173c0fb0e0 msgr2=0x7f173c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 -- 172.21.15.161:0/2574139159 shutdown_connections 2023-07-14T11:21:49.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.476+0000 7f17433ff700 1 -- 172.21.15.161:0/2574139159 wait complete. 2023-07-14T11:21:49.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 Processor -- start 2023-07-14T11:21:49.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 -- start start 2023-07-14T11:21:49.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f173c0ffae0 0x7f173c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f173c100cd0 0x7f173c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 0x7f173c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0735e0 con 0x7f173c101660 2023-07-14T11:21:49.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c073750 con 0x7f173c0ffae0 2023-07-14T11:21:49.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.477+0000 7f17433ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c19f990 con 0x7f173c100cd0 2023-07-14T11:21:49.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174199c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 0x7f173c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174119b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f173c0ffae0 0x7f173c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174099a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f173c100cd0 0x7f173c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174119b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f173c0ffae0 0x7f173c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35340/0 (socket says 172.21.15.161:35340) 2023-07-14T11:21:49.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174119b700 1 -- 172.21.15.161:0/2742265830 learned_addr learned my addr 172.21.15.161:0/2742265830 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174199c700 1 -- 172.21.15.161:0/2742265830 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f173c100cd0 msgr2=0x7f173c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174199c700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f173c100cd0 0x7f173c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174199c700 1 -- 172.21.15.161:0/2742265830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f173c0ffae0 msgr2=0x7f173c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174199c700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f173c0ffae0 0x7f173c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174199c700 1 -- 172.21.15.161:0/2742265830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734009d00 con 0x7f173c101660 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174099a700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f173c100cd0 0x7f173c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f174199c700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 0x7f173c072e40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f173400b070 tx=0x7f173400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.478+0000 7f17327fc700 1 -- 172.21.15.161:0/2742265830 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400e040 con 0x7f173c101660 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.479+0000 7f17327fc700 1 -- 172.21.15.161:0/2742265830 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1734009480 con 0x7f173c101660 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.479+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c19fc10 con 0x7f173c101660 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.479+0000 7f17327fc700 1 -- 172.21.15.161:0/2742265830 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17340052a0 con 0x7f173c101660 2023-07-14T11:21:49.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.479+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c19ffe0 con 0x7f173c101660 2023-07-14T11:21:49.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.483+0000 7f17327fc700 1 -- 172.21.15.161:0/2742265830 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f173401d480 con 0x7f173c101660 2023-07-14T11:21:49.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.484+0000 7f1756c29b80 1 Processor -- start 2023-07-14T11:21:49.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.484+0000 7f1756c29b80 1 -- start start 2023-07-14T11:21:49.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.484+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3d480 0x55ae7ce3d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.484+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce3e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f1756c29b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cd42c60 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f1756c29b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cd959f0 con 0x55ae7ce3d480 2023-07-14T11:21:49.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f1756c29b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cd8ee80 con 0x55ae7ce3ec20 2023-07-14T11:21:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce3e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f17327fc700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17280a1150 0x7f17280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce3e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48388/0 (socket says 172.21.15.161:48388) 2023-07-14T11:21:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 -- 172.21.15.161:0/3961810024 learned_addr learned my addr 172.21.15.161:0/3961810024 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174099a700 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3d480 0x55ae7ce3d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f17327fc700 1 -- 172.21.15.161:0/2742265830 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f17340cdf00 con 0x7f173c101660 2023-07-14T11:21:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174099a700 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174199c700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17280a1150 0x7f17280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 -- 172.21.15.161:0/3961810024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 msgr2=0x55ae7ce3f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce3f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 -- 172.21.15.161:0/3961810024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3d480 msgr2=0x55ae7ce3d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3d480 0x55ae7ce3d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174119b700 1 -- 172.21.15.161:0/3961810024 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae7cbbcd90 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.485+0000 7f174099a700 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce3f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.486+0000 7f174119b700 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce3e690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1738009df0 tx=0x7f1738011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b85819e860d686 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.486+0000 7f174199c700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17280a1150 0x7f17280a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17340062a0 tx=0x7f1734009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.486+0000 7f17277fe700 1 -- 172.21.15.161:0/3961810024 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738004230 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.486+0000 7f17277fe700 1 -- 172.21.15.161:0/3961810024 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1738004390 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.486+0000 7f17277fe700 1 -- 172.21.15.161:0/3961810024 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738019860 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 -- 172.21.15.161:0/3961810024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 msgr2=0x55ae7ce3e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce3e690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1738009df0 tx=0x7f1738011720 comp rx=0 tx=0).stop 2023-07-14T11:21:49.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 -- 172.21.15.161:0/3961810024 shutdown_connections 2023-07-14T11:21:49.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce3f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3d480 0x55ae7ce3d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 --2- 172.21.15.161:0/3961810024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce3e690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 -- 172.21.15.161:0/3961810024 >> 172.21.15.161:0/3961810024 conn(0x55ae7ce32d50 msgr2=0x55ae7ce38180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 -- 172.21.15.161:0/3961810024 shutdown_connections 2023-07-14T11:21:49.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 -- 172.21.15.161:0/3961810024 wait complete. 2023-07-14T11:21:49.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 -- start start 2023-07-14T11:21:49.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.487+0000 7f1756c29b80 1 Processor -- start 2023-07-14T11:21:49.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce47600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f1756c29b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cd2ea70 con 0x55ae7ce3d480 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f1756c29b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cd2e770 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f1756c29b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cc128d0 con 0x55ae7ce3ec20 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f1756c29b80 10 client.0 _ll_register_callbacks cb 0x55ae7cd2c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f174099a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f174099a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48404/0 (socket says 172.21.15.161:48404) 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f174099a700 1 -- 172.21.15.161:0/1389016649 learned_addr learned my addr 172.21.15.161:0/1389016649 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f174119b700 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce47600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.488+0000 7f174099a700 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f174099a700 1 -- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 msgr2=0x55ae7ce4e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f174099a700 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f174099a700 1 -- 172.21.15.161:0/1389016649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 msgr2=0x55ae7ce47600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f174099a700 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce47600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f174099a700 1 -- 172.21.15.161:0/1389016649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1738009a50 con 0x55ae7ce3d480 2023-07-14T11:21:49.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f174099a700 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce470c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f172c00f460 tx=0x7f172c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d47cf94961e2df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f1724ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:49.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f1724ff9700 1 -- 172.21.15.161:0/1389016649 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0130f0 con 0x55ae7ce3d480 2023-07-14T11:21:49.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae7cce84a0 con 0x55ae7ce3d480 2023-07-14T11:21:49.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f1724ff9700 1 -- 172.21.15.161:0/1389016649 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f172c013250 con 0x55ae7ce3d480 2023-07-14T11:21:49.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f1724ff9700 1 -- 172.21.15.161:0/1389016649 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f172c013410 con 0x55ae7ce3d480 2023-07-14T11:21:49.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f1756c29b80 10 client.15330 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:49.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.489+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ae7cba5c10 con 0x55ae7ce3d480 2023-07-14T11:21:49.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.490+0000 7f1724ff9700 1 -- 172.21.15.161:0/1389016649 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae7cce84a0 con 0x55ae7ce3d480 2023-07-14T11:21:49.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.490+0000 7f1724ff9700 1 -- 172.21.15.161:0/1389016649 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f172c0115f0 con 0x55ae7ce3d480 2023-07-14T11:21:49.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.490+0000 7f1756c29b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:49.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.490+0000 7f1756c29b80 10 client.15330 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T11:21:49.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.490+0000 7f1756c29b80 20 client.15330 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:49.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.491+0000 7f1756c29b80 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ce52350 0x55ae7ce517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.491+0000 7f1756c29b80 4 client.15330 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:21:49.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.491+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae7cd87e60 con 0x55ae7ce52350 2023-07-14T11:21:49.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.491+0000 7f174119b700 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ce52350 0x55ae7ce517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.491+0000 7f1724ff9700 10 client.15330 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:49.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.491+0000 7f174119b700 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ce52350 0x55ae7ce517a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c92e9b4b00feec1 server_cookie=371295ebee17d602 in_seq=0 out_seq=0 2023-07-14T11:21:49.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1724ff9700 1 -- 172.21.15.161:0/1389016649 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ae7cd87e60 con 0x55ae7ce52350 2023-07-14T11:21:49.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1724ff9700 10 client.15330 handle_command_reply: tid=0 2023-07-14T11:21:49.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 client.15330 shutdown 2023-07-14T11:21:49.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 msgr2=0x55ae7ce470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce470c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f172c00f460 tx=0x7f172c0047c0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 shutdown_connections 2023-07-14T11:21:49.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ce52350 0x55ae7ce517a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce470c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 --2- 172.21.15.161:0/1389016649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce47600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1724ff9700 0 client.15330 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:49.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 >> 172.21.15.161:0/1389016649 conn(0x55ae7ce32d50 msgr2=0x55ae7ce37860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.492+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 shutdown_connections 2023-07-14T11:21:49.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.493+0000 7f1756c29b80 1 -- 172.21.15.161:0/1389016649 wait complete. 2023-07-14T11:21:49.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.493+0000 7f1756c29b80 20 client.15330 trim_cache size 0 max 16384 2023-07-14T11:21:49.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.510+0000 7f1756c29b80 1 Processor -- start 2023-07-14T11:21:49.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f1756c29b80 1 -- start start 2023-07-14T11:21:49.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce44f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f1756c29b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cc1ede0 con 0x55ae7ce3d480 2023-07-14T11:21:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f1756c29b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cd4d480 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f1756c29b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7cd2ad50 con 0x55ae7ce3ec20 2023-07-14T11:21:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f174099a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f174099a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48416/0 (socket says 172.21.15.161:48416) 2023-07-14T11:21:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f174099a700 1 -- 172.21.15.161:0/2245961142 learned_addr learned my addr 172.21.15.161:0/2245961142 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.511+0000 7f174119b700 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f174099a700 1 -- 172.21.15.161:0/2245961142 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 msgr2=0x55ae7ce44f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f174099a700 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce44f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f174099a700 1 -- 172.21.15.161:0/2245961142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 msgr2=0x55ae7ce4e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f174099a700 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f174099a700 1 -- 172.21.15.161:0/2245961142 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae7cbe3200 con 0x55ae7ce3d480 2023-07-14T11:21:49.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f174099a700 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce517a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f172c005940 tx=0x7f172c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c251796adcb1f018 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f1727fff700 1 -- 172.21.15.161:0/2245961142 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c011c20 con 0x55ae7ce3d480 2023-07-14T11:21:49.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f1727fff700 1 -- 172.21.15.161:0/2245961142 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f172c0115f0 con 0x55ae7ce3d480 2023-07-14T11:21:49.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.512+0000 7f1727fff700 1 -- 172.21.15.161:0/2245961142 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c020f10 con 0x55ae7ce3d480 2023-07-14T11:21:49.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 -- 172.21.15.161:0/2245961142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 msgr2=0x55ae7ce517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce517a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f172c005940 tx=0x7f172c011e60 comp rx=0 tx=0).stop 2023-07-14T11:21:49.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 -- 172.21.15.161:0/2245961142 shutdown_connections 2023-07-14T11:21:49.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3d480 0x55ae7ce517a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce44f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 --2- 172.21.15.161:0/2245961142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 -- 172.21.15.161:0/2245961142 >> 172.21.15.161:0/2245961142 conn(0x55ae7ce32d50 msgr2=0x55ae7ce45e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 -- 172.21.15.161:0/2245961142 shutdown_connections 2023-07-14T11:21:49.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 -- 172.21.15.161:0/2245961142 wait complete. 2023-07-14T11:21:49.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 -- start start 2023-07-14T11:21:49.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.513+0000 7f1756c29b80 1 Processor -- start 2023-07-14T11:21:49.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3d480 0x55ae7ce49ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f1756c29b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f1756c29b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7ce4d560 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f1756c29b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7ce4d6d0 con 0x55ae7ce3ec20 2023-07-14T11:21:49.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f1756c29b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae7ce4d890 con 0x55ae7ce3d480 2023-07-14T11:21:49.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f1756c29b80 10 client.0 _ll_register_callbacks cb 0x7f172c013d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:49.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48424/0 (socket says 172.21.15.161:48424) 2023-07-14T11:21:49.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 -- 172.21.15.161:0/153800231 learned_addr learned my addr 172.21.15.161:0/153800231 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174099a700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3d480 0x55ae7ce49ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174099a700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 -- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3d480 msgr2=0x55ae7ce49ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3d480 0x55ae7ce49ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 -- 172.21.15.161:0/153800231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ec20 msgr2=0x55ae7ce4d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174119b700 1 -- 172.21.15.161:0/153800231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae7cba5c10 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.514+0000 7f174099a700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:49.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f174119b700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4a000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1738004360 tx=0x7f17380099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ddc2fac2e28dad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1724ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:21:49.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1724ff9700 1 -- 172.21.15.161:0/153800231 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738029040 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1738004390 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1724ff9700 1 -- 172.21.15.161:0/153800231 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1738003680 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1724ff9700 1 -- 172.21.15.161:0/153800231 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f173802a620 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1756c29b80 10 client.15336 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:49.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ae7cbbb260 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.515+0000 7f1724ff9700 1 -- 172.21.15.161:0/153800231 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1738004390 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f1724ff9700 1 -- 172.21.15.161:0/153800231 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1738011440 con 0x55ae7ce3ddd0 2023-07-14T11:21:49.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f1756c29b80 10 client.15336 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:49.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f1756c29b80 10 client.15336 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T11:21:49.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f1756c29b80 20 client.15336 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:49.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f1756c29b80 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ceabb80 0x55ae7cea84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f1756c29b80 4 client.15336 mds_command: new command op to 24544 tid=0[{"prefix": "status"}] 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae7cd4a000 con 0x55ae7ceabb80 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.516+0000 7f174099a700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ceabb80 0x55ae7cea84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f174099a700 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ceabb80 0x55ae7cea84f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef0ca162103f93ab server_cookie=4510392c43f35828 in_seq=0 out_seq=0 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1724ff9700 10 client.15336 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1724ff9700 1 -- 172.21.15.161:0/153800231 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55ae7cd4a000 con 0x55ae7ceabb80 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1724ff9700 10 client.15336 handle_command_reply: tid=0 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 client.15336 shutdown 2023-07-14T11:21:49.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 msgr2=0x55ae7ce4a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4a000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1738004360 tx=0x7f17380099a0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 shutdown_connections 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55ae7ceabb80 0x55ae7cea84f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae7ce3ec20 0x55ae7ce4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ae7ce3d480 0x55ae7ce49ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1724ff9700 0 client.15336 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 --2- 172.21.15.161:0/153800231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae7ce3ddd0 0x55ae7ce4a000 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 >> 172.21.15.161:0/153800231 conn(0x55ae7ce32d50 msgr2=0x55ae7ce44f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 shutdown_connections 2023-07-14T11:21:49.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.517+0000 7f1756c29b80 1 -- 172.21.15.161:0/153800231 wait complete. 2023-07-14T11:21:49.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f1756c29b80 20 client.15336 trim_cache size 0 max 16384 2023-07-14T11:21:49.530 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:21:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:21:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 1, 2023-07-14T11:21:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24544, 2023-07-14T11:21:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2023-07-14T11:21:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2023-07-14T11:21:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2023-07-14T11:21:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 98.81619336, 2023-07-14T11:21:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 22, 2023-07-14T11:21:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 92, 2023-07-14T11:21:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T11:21:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 142.09493447700001 2023-07-14T11:21:49.532 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:21:49.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17280a1150 msgr2=0x7f17280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17280a1150 0x7f17280a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17340062a0 tx=0x7f1734009040 comp rx=0 tx=0).stop 2023-07-14T11:21:49.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 msgr2=0x7f173c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 0x7f173c072e40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f173400b070 tx=0x7f173400bc80 comp rx=0 tx=0).stop 2023-07-14T11:21:49.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 shutdown_connections 2023-07-14T11:21:49.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f173c101660 0x7f173c072e40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17280a1150 0x7f17280a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f173c0ffae0 0x7f173c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 --2- 172.21.15.161:0/2742265830 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f173c100cd0 0x7f173c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 >> 172.21.15.161:0/2742265830 conn(0x7f173c0fb0e0 msgr2=0x7f173c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 shutdown_connections 2023-07-14T11:21:49.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.518+0000 7f17433ff700 1 -- 172.21.15.161:0/2742265830 wait complete. 2023-07-14T11:21:49.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T11:21:49.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:49 smithi175 ceph-mon[129081]: pgmap v339: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T11:21:49.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:49 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2334126957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:49.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:49 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1748068757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:49.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:49 smithi182 ceph-mon[129366]: pgmap v339: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T11:21:49.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:49 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2334126957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:21:49.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:49 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1748068757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:49.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 -- 172.21.15.161:0/2339945969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 msgr2=0x7f3a0010e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 --2- 172.21.15.161:0/2339945969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 0x7f3a0010e4a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b0a0 tx=0x7f39f800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 -- 172.21.15.161:0/2339945969 shutdown_connections 2023-07-14T11:21:49.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 --2- 172.21.15.161:0/2339945969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 0x7f3a0010e4a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 --2- 172.21.15.161:0/2339945969 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3a00074550 0x7f3a000729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 --2- 172.21.15.161:0/2339945969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 0x7f3a000733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 -- 172.21.15.161:0/2339945969 >> 172.21.15.161:0/2339945969 conn(0x7f3a000fb130 msgr2=0x7f3a000fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 -- 172.21.15.161:0/2339945969 shutdown_connections 2023-07-14T11:21:49.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.735+0000 7f3a063ff700 1 -- 172.21.15.161:0/2339945969 wait complete. 2023-07-14T11:21:49.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.736+0000 7f3a063ff700 1 Processor -- start 2023-07-14T11:21:49.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.736+0000 7f3a063ff700 1 -- start start 2023-07-14T11:21:49.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a063ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 0x7f3a0019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a063ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3a00074550 0x7f3a0019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a063ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 0x7f3a001a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a063ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001a3a70 con 0x7f3a0010c0c0 2023-07-14T11:21:49.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a063ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001a3bb0 con 0x7f3a00072f70 2023-07-14T11:21:49.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a063ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001a3d40 con 0x7f3a00074550 2023-07-14T11:21:49.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 0x7f3a0019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a0499c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 0x7f3a001a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 0x7f3a0019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35382/0 (socket says 172.21.15.161:35382) 2023-07-14T11:21:49.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 -- 172.21.15.161:0/578018868 learned_addr learned my addr 172.21.15.161:0/578018868 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3a00074550 0x7f3a0019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 -- 172.21.15.161:0/578018868 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3a00074550 msgr2=0x7f3a0019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3a00074550 0x7f3a0019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 -- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 msgr2=0x7f3a001a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 0x7f3a001a34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f39fffff700 1 -- 172.21.15.161:0/578018868 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f8009d00 con 0x7f3a00072f70 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.737+0000 7f3a0499c700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 0x7f3a001a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.738+0000 7f39fffff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 0x7f3a0019bef0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39f000e970 tx=0x7f39f000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.738+0000 7f39fd7fa700 1 -- 172.21.15.161:0/578018868 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0009790 con 0x7f3a00072f70 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.738+0000 7f39fd7fa700 1 -- 172.21.15.161:0/578018868 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f000c7e0 con 0x7f3a00072f70 2023-07-14T11:21:49.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.738+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a001a3fc0 con 0x7f3a00072f70 2023-07-14T11:21:49.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.738+0000 7f39fd7fa700 1 -- 172.21.15.161:0/578018868 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00054e0 con 0x7f3a00072f70 2023-07-14T11:21:49.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.738+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a001a4440 con 0x7f3a00072f70 2023-07-14T11:21:49.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.742+0000 7f39fd7fa700 1 -- 172.21.15.161:0/578018868 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f39f000c950 con 0x7f3a00072f70 2023-07-14T11:21:49.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.742+0000 7f3a19c7bb80 1 Processor -- start 2023-07-14T11:21:49.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.742+0000 7f3a19c7bb80 1 -- start start 2023-07-14T11:21:49.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a653b3c60 con 0x558a654ae480 2023-07-14T11:21:49.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a654069f0 con 0x558a654afc20 2023-07-14T11:21:49.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a653ffe80 con 0x558a654aedd0 2023-07-14T11:21:49.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a0499c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a0499c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48458/0 (socket says 172.21.15.161:48458) 2023-07-14T11:21:49.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f3a0499c700 1 -- 172.21.15.161:0/3105533294 learned_addr learned my addr 172.21.15.161:0/3105533294 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f39fffff700 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.743+0000 7f39fd7fa700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f39e80a1150 0x7f39e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f3a0499c700 1 -- 172.21.15.161:0/3105533294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 msgr2=0x558a654af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f3a0499c700 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f39fd7fa700 1 -- 172.21.15.161:0/578018868 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f39f0014070 con 0x7f3a00072f70 2023-07-14T11:21:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f3a0499c700 1 -- 172.21.15.161:0/3105533294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 msgr2=0x558a654b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f3a0499c700 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f3a0499c700 1 -- 172.21.15.161:0/3105533294 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a6522dd90 con 0x558a654ae480 2023-07-14T11:21:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f3a0499c700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f39e80a1150 0x7f39e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f3a0499c700 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654ae890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b0a0 tx=0x7f39f8009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dddb17f93ea2f4a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f39e67fc700 1 -- 172.21.15.161:0/3105533294 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800e040 con 0x558a654ae480 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f39e67fc700 1 -- 172.21.15.161:0/3105533294 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f800be90 con 0x558a654ae480 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.744+0000 7f39e67fc700 1 -- 172.21.15.161:0/3105533294 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80058f0 con 0x558a654ae480 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/3105533294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 msgr2=0x558a654ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654ae890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b0a0 tx=0x7f39f8009340 comp rx=0 tx=0).stop 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a0499c700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f39e80a1150 0x7f39e80a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f39f802b000 tx=0x7f39f802c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/3105533294 shutdown_connections 2023-07-14T11:21:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654ae890 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/3105533294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/3105533294 >> 172.21.15.161:0/3105533294 conn(0x558a654a3d50 msgr2=0x558a654a9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/3105533294 shutdown_connections 2023-07-14T11:21:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.745+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/3105533294 wait complete. 2023-07-14T11:21:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.746+0000 7f3a19c7bb80 1 -- start start 2023-07-14T11:21:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.746+0000 7f3a19c7bb80 1 Processor -- start 2023-07-14T11:21:49.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.746+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.746+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654aedd0 0x558a654b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.746+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654afc20 0x558a654bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a6539fa70 con 0x558a654afc20 2023-07-14T11:21:49.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a6539f770 con 0x558a654aedd0 2023-07-14T11:21:49.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a652838d0 con 0x558a654ae480 2023-07-14T11:21:49.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f3a19c7bb80 10 client.0 _ll_register_callbacks cb 0x558a6539d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:49.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39fffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654aedd0 0x558a654b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654aedd0 0x558a654b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35402/0 (socket says 172.21.15.161:35402) 2023-07-14T11:21:49.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39fffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52384/0 (socket says 172.21.15.161:52384) 2023-07-14T11:21:49.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 -- 172.21.15.161:0/2529416965 learned_addr learned my addr 172.21.15.161:0/2529416965 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f3a0499c700 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654afc20 0x558a654bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 -- 172.21.15.161:0/2529416965 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 msgr2=0x558a654b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 -- 172.21.15.161:0/2529416965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654afc20 msgr2=0x558a654bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654afc20 0x558a654bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 -- 172.21.15.161:0/2529416965 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0010ee50 con 0x558a654aedd0 2023-07-14T11:21:49.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39fffff700 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654aedd0 0x558a654b8650 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f39f40117c0 tx=0x7f39f4011ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9841790903deff52 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.747+0000 7f39df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f39df7fe700 1 -- 172.21.15.161:0/2529416965 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4016070 con 0x558a654aedd0 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a653594a0 con 0x558a654aedd0 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f39df7fe700 1 -- 172.21.15.161:0/2529416965 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f39f4004300 con 0x558a654aedd0 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f39df7fe700 1 -- 172.21.15.161:0/2529416965 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f40044c0 con 0x558a654aedd0 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f3a19c7bb80 10 client.25382 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558a65216c10 con 0x558a654aedd0 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f39df7fe700 1 -- 172.21.15.161:0/2529416965 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a653594a0 con 0x558a654aedd0 2023-07-14T11:21:49.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f39df7fe700 1 -- 172.21.15.161:0/2529416965 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39f400f830 con 0x558a654aedd0 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f3a19c7bb80 10 client.25382 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.748+0000 7f3a19c7bb80 10 client.25382 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.749+0000 7f3a19c7bb80 20 client.25382 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.749+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a654c3420 0x558a654c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.749+0000 7f3a19c7bb80 4 client.25382 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.749+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a653f8e60 con 0x558a654c3420 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.749+0000 7f39fffff700 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a654c3420 0x558a654c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.749+0000 7f39df7fe700 10 client.25382 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:49.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.749+0000 7f39fffff700 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a654c3420 0x558a654c27f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd230f181cfc750 server_cookie=ef1f44bb9f39981d in_seq=0 out_seq=0 2023-07-14T11:21:49.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f39df7fe700 1 -- 172.21.15.161:0/2529416965 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a653f8e60 con 0x558a654c3420 2023-07-14T11:21:49.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f39df7fe700 10 client.25382 handle_command_reply: tid=0 2023-07-14T11:21:49.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 client.25382 shutdown 2023-07-14T11:21:49.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654aedd0 msgr2=0x558a654b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654aedd0 0x558a654b8650 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f39f40117c0 tx=0x7f39f4011ad0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 shutdown_connections 2023-07-14T11:21:49.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a654c3420 0x558a654c27f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654afc20 0x558a654bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f39df7fe700 0 client.25382 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:49.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/2529416965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654aedd0 0x558a654b8650 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.750+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 >> 172.21.15.161:0/2529416965 conn(0x558a654a3d50 msgr2=0x558a654a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.751+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 shutdown_connections 2023-07-14T11:21:49.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.751+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/2529416965 wait complete. 2023-07-14T11:21:49.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.751+0000 7f3a19c7bb80 20 client.25382 trim_cache size 0 max 16384 2023-07-14T11:21:49.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.768+0000 7f3a19c7bb80 1 Processor -- start 2023-07-14T11:21:49.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a19c7bb80 1 -- start start 2023-07-14T11:21:49.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a654b7ce0 con 0x558a654aedd0 2023-07-14T11:21:49.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a653ce790 con 0x558a654afc20 2023-07-14T11:21:49.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a653be480 con 0x558a654ae480 2023-07-14T11:21:49.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48488/0 (socket says 172.21.15.161:48488) 2023-07-14T11:21:49.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f39ff7fe700 1 -- 172.21.15.161:0/629636712 learned_addr learned my addr 172.21.15.161:0/629636712 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f39fffff700 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f3a0499c700 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f39ff7fe700 1 -- 172.21.15.161:0/629636712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 msgr2=0x558a654bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.769+0000 7f39ff7fe700 1 -- 172.21.15.161:0/629636712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 msgr2=0x558a654b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.770+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.770+0000 7f39ff7fe700 1 -- 172.21.15.161:0/629636712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a654b02c0 con 0x558a654aedd0 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.770+0000 7f3a0499c700 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.770+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f39f400fde0 tx=0x7f39f40096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6792482bce2fe1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.770+0000 7f39e6ffd700 1 -- 172.21.15.161:0/629636712 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4016070 con 0x558a654aedd0 2023-07-14T11:21:49.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.770+0000 7f39e6ffd700 1 -- 172.21.15.161:0/629636712 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f400f8b0 con 0x558a654aedd0 2023-07-14T11:21:49.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.770+0000 7f39e6ffd700 1 -- 172.21.15.161:0/629636712 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4003bc0 con 0x558a654aedd0 2023-07-14T11:21:49.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/629636712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654aedd0 msgr2=0x558a654bf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f39f400fde0 tx=0x7f39f40096f0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/629636712 shutdown_connections 2023-07-14T11:21:49.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a654b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654ae480 0x558a654bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/629636712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/629636712 >> 172.21.15.161:0/629636712 conn(0x558a654a3d50 msgr2=0x558a654b6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/629636712 shutdown_connections 2023-07-14T11:21:49.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/629636712 wait complete. 2023-07-14T11:21:49.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.771+0000 7f3a19c7bb80 1 -- start start 2023-07-14T11:21:49.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 1 Processor -- start 2023-07-14T11:21:49.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a655172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a654b5720 con 0x558a654ae480 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a654b5890 con 0x558a654afc20 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a65517800 con 0x558a654aedd0 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a19c7bb80 10 client.0 _ll_register_callbacks cb 0x7f39f4009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f39fffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f39fffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48492/0 (socket says 172.21.15.161:48492) 2023-07-14T11:21:49.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f39fffff700 1 -- 172.21.15.161:0/921094806 learned_addr learned my addr 172.21.15.161:0/921094806 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:49.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.772+0000 7f3a0499c700 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a655172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39ff7fe700 1 -- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 msgr2=0x558a655172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a655172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39ff7fe700 1 -- 172.21.15.161:0/921094806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 msgr2=0x558a654b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:21:49.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39ff7fe700 1 -- 172.21.15.161:0/921094806 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a65216c10 con 0x558a654aedd0 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39fffff700 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39ff7fe700 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39f4005a80 tx=0x7f39f400f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05321ea42663377 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39df7fe700 1 -- 172.21.15.161:0/921094806 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4016040 con 0x558a654aedd0 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39f800be90 con 0x558a654aedd0 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.773+0000 7f39df7fe700 1 -- 172.21.15.161:0/921094806 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f39f4005590 con 0x558a654aedd0 2023-07-14T11:21:49.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f39df7fe700 1 -- 172.21.15.161:0/921094806 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f4005750 con 0x558a654aedd0 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 10 client.25399 fetch_fsmap learned FSMap version 22 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558a65254200 con 0x558a654aedd0 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f39df7fe700 1 -- 172.21.15.161:0/921094806 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f800be90 con 0x558a654aedd0 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f39df7fe700 1 -- 172.21.15.161:0/921094806 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39f4022040 con 0x558a654aedd0 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 10 client.25399 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 10 client.25399 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 20 client.25399 populate_metadata read hostname 'smithi161' 2023-07-14T11:21:49.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a6551c010 0x558a65518a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:49.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 4 client.25399 mds_command: new command op to 24575 tid=0[{"prefix": "status"}] 2023-07-14T11:21:49.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558a653bb000 con 0x558a6551c010 2023-07-14T11:21:49.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.774+0000 7f39fffff700 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a6551c010 0x558a65518a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:49.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f39df7fe700 10 client.25399 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:49.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f39fffff700 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a6551c010 0x558a65518a00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8faf96b162e0c2bd server_cookie=ae5888c1b3954c75 in_seq=0 out_seq=0 2023-07-14T11:21:49.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f39df7fe700 1 -- 172.21.15.161:0/921094806 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558a653bb000 con 0x558a6551c010 2023-07-14T11:21:49.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f39df7fe700 10 client.25399 handle_command_reply: tid=0 2023-07-14T11:21:49.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 client.25399 shutdown 2023-07-14T11:21:49.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 msgr2=0x558a654bf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39f4005a80 tx=0x7f39f400f520 comp rx=0 tx=0).stop 2023-07-14T11:21:49.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 shutdown_connections 2023-07-14T11:21:49.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558a6551c010 0x558a65518a00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a654ae480 0x558a654b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f39df7fe700 0 client.25399 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:21:49.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a654afc20 0x558a655172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 --2- 172.21.15.161:0/921094806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558a654aedd0 0x558a654bf5b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 >> 172.21.15.161:0/921094806 conn(0x558a654a3d50 msgr2=0x558a654b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.775+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 shutdown_connections 2023-07-14T11:21:49.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a19c7bb80 1 -- 172.21.15.161:0/921094806 wait complete. 2023-07-14T11:21:49.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a19c7bb80 20 client.25399 trim_cache size 0 max 16384 2023-07-14T11:21:49.787 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:21:49.788 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "2bf7fc3c-2237-11ee-9b32-001a4aab830c", 2023-07-14T11:21:49.788 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2023-07-14T11:21:49.788 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 24575, 2023-07-14T11:21:49.788 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2023-07-14T11:21:49.788 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2023-07-14T11:21:49.788 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2023-07-14T11:21:49.789 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 127.68092955100001, 2023-07-14T11:21:49.789 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 22, 2023-07-14T11:21:49.789 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 92, 2023-07-14T11:21:49.789 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T11:21:49.789 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 134.09797394200001 2023-07-14T11:21:49.789 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:21:49.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f39e80a1150 msgr2=0x7f39e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f39e80a1150 0x7f39e80a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f39f802b000 tx=0x7f39f802c000 comp rx=0 tx=0).stop 2023-07-14T11:21:49.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 msgr2=0x7f3a0019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:49.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 0x7f3a0019bef0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39f000e970 tx=0x7f39f000ec80 comp rx=0 tx=0).stop 2023-07-14T11:21:49.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 shutdown_connections 2023-07-14T11:21:49.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0010c0c0 0x7f3a001a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f39e80a1150 0x7f39e80a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a00072f70 0x7f3a0019bef0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 --2- 172.21.15.161:0/578018868 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3a00074550 0x7f3a0019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:49.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 >> 172.21.15.161:0/578018868 conn(0x7f3a000fb130 msgr2=0x7f3a0010e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:49.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 shutdown_connections 2023-07-14T11:21:49.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:49.776+0000 7f3a063ff700 1 -- 172.21.15.161:0/578018868 wait complete. 2023-07-14T11:21:49.798 INFO:teuthology.run_tasks:Running task kclient... 2023-07-14T11:21:49.812 INFO:tasks.kclient:Mounting kernel clients... 2023-07-14T11:21:49.812 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-14T11:21:49.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:49.813 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-07-14T11:21:49.856 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-07-14T11:21:49.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:49.858 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-14T11:21:49.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:49.968 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2023-07-14T11:21:50.083 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-07-14T11:21:50.084 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:21:50.084 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:21:50.085 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph fs ls 2023-07-14T11:21:50.816 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:21:51.401 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:51 smithi161 ceph-mon[95526]: pgmap v340: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:51.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:51 smithi175 ceph-mon[129081]: pgmap v340: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:51.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:51 smithi182 ceph-mon[129366]: pgmap v340: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:52.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 -- 172.21.15.161:0/3567583915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca78104ea0 msgr2=0x7fca7810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:52.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 --2- 172.21.15.161:0/3567583915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca78104ea0 0x7fca7810ca20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fca7000b0d0 tx=0x7fca7000b3e0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 -- 172.21.15.161:0/3567583915 shutdown_connections 2023-07-14T11:21:52.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 --2- 172.21.15.161:0/3567583915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca78104ea0 0x7fca7810ca20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 --2- 172.21.15.161:0/3567583915 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca780ffe00 0x7fca781001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 --2- 172.21.15.161:0/3567583915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca781007a0 0x7fca78104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 -- 172.21.15.161:0/3567583915 >> 172.21.15.161:0/3567583915 conn(0x7fca780fb660 msgr2=0x7fca780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 -- 172.21.15.161:0/3567583915 shutdown_connections 2023-07-14T11:21:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.283+0000 7fca7ce73700 1 -- 172.21.15.161:0/3567583915 wait complete. 2023-07-14T11:21:52.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.284+0000 7fca7ce73700 1 Processor -- start 2023-07-14T11:21:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.284+0000 7fca7ce73700 1 -- start start 2023-07-14T11:21:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7ce73700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca780ffe00 0x7fca78197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7ce73700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca781007a0 0x7fca781984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7ce73700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca78104ea0 0x7fca7819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7ce73700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7819cf40 con 0x7fca780ffe00 2023-07-14T11:21:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7ce73700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7819d0b0 con 0x7fca781007a0 2023-07-14T11:21:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7ce73700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7819d270 con 0x7fca78104ea0 2023-07-14T11:21:52.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca780ffe00 0x7fca78197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:52.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca780ffe00 0x7fca78197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48510/0 (socket says 172.21.15.161:48510) 2023-07-14T11:21:52.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 -- 172.21.15.161:0/1135958186 learned_addr learned my addr 172.21.15.161:0/1135958186 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:52.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca75d9b700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca781007a0 0x7fca781984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:52.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca76d9d700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca78104ea0 0x7fca7819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:52.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 -- 172.21.15.161:0/1135958186 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca78104ea0 msgr2=0x7fca7819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:52.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca78104ea0 0x7fca7819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 -- 172.21.15.161:0/1135958186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca781007a0 msgr2=0x7fca781984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:52.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca781007a0 0x7fca781984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca7659c700 1 -- 172.21.15.161:0/1135958186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca70009d00 con 0x7fca780ffe00 2023-07-14T11:21:52.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca75d9b700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca781007a0 0x7fca781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:52.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.285+0000 7fca76d9d700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca78104ea0 0x7fca7819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:52.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.286+0000 7fca7659c700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca780ffe00 0x7fca78197f80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fca6800e9a0 tx=0x7fca6800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:52.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.286+0000 7fca677fe700 1 -- 172.21.15.161:0/1135958186 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68009790 con 0x7fca780ffe00 2023-07-14T11:21:52.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.286+0000 7fca677fe700 1 -- 172.21.15.161:0/1135958186 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca6800cbd0 con 0x7fca780ffe00 2023-07-14T11:21:52.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.286+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7819d550 con 0x7fca780ffe00 2023-07-14T11:21:52.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.286+0000 7fca677fe700 1 -- 172.21.15.161:0/1135958186 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68018680 con 0x7fca780ffe00 2023-07-14T11:21:52.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.286+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca781a4db0 con 0x7fca780ffe00 2023-07-14T11:21:52.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.287+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7804e210 con 0x7fca780ffe00 2023-07-14T11:21:52.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.289+0000 7fca677fe700 1 -- 172.21.15.161:0/1135958186 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fca6800cd40 con 0x7fca780ffe00 2023-07-14T11:21:52.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.290+0000 7fca677fe700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fca600a11d0 0x7fca600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:52.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.291+0000 7fca677fe700 1 -- 172.21.15.161:0/1135958186 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fca68014070 con 0x7fca780ffe00 2023-07-14T11:21:52.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.291+0000 7fca75d9b700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fca600a11d0 0x7fca600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:52.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.291+0000 7fca75d9b700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fca600a11d0 0x7fca600a3680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca6c005e10 tx=0x7fca6c005d60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:52.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.294+0000 7fca677fe700 1 -- 172.21.15.161:0/1135958186 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fca68051790 con 0x7fca780ffe00 2023-07-14T11:21:52.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.534+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca781a5070 con 0x7fca780ffe00 2023-07-14T11:21:52.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.535+0000 7fca677fe700 1 -- 172.21.15.161:0/1135958186 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+88 (secure 0 0 0) 0x7fca68096360 con 0x7fca780ffe00 2023-07-14T11:21:52.539 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:21:52.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fca600a11d0 msgr2=0x7fca600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:52.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fca600a11d0 0x7fca600a3680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca6c005e10 tx=0x7fca6c005d60 comp rx=0 tx=0).stop 2023-07-14T11:21:52.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca780ffe00 msgr2=0x7fca78197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:52.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca780ffe00 0x7fca78197f80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fca6800e9a0 tx=0x7fca6800ed60 comp rx=0 tx=0).stop 2023-07-14T11:21:52.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 shutdown_connections 2023-07-14T11:21:52.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca780ffe00 0x7fca78197f80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fca600a11d0 0x7fca600a3680 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca78104ea0 0x7fca7819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 --2- 172.21.15.161:0/1135958186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca781007a0 0x7fca781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:52.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.537+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 >> 172.21.15.161:0/1135958186 conn(0x7fca780fb660 msgr2=0x7fca780fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:52.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.538+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 shutdown_connections 2023-07-14T11:21:52.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:52.538+0000 7fca7ce73700 1 -- 172.21.15.161:0/1135958186 wait complete. 2023-07-14T11:21:52.997 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:21:52.997 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:21:52.997 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-07-14T11:21:52.998 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:21:52.998 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:21:52.998 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:21:52.998 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:21:52.998 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:21:52.998 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:21:52.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:52.999 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-07-14T11:21:53.046 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:21:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:21:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:21:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:21:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-07-14T11:21:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:21:53.047 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:21:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:21:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-07-14T11:21:53.048 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:21:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:21:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-07-14T11:21:53.048 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:21:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:21:53.048 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:21:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:21:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-14T11:21:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 41252sec preferred_lft 41252sec 2023-07-14T11:21:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::7941:741e:f751:64dd/64 scope link noprefixroute 2023-07-14T11:21:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:21:53.050 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T11:21:53.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.051 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:21:53.051 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:21:53.051 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add name ceph-brx type bridge 2023-07-14T11:21:53.051 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr flush dev ceph-brx 2023-07-14T11:21:53.052 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set ceph-brx up 2023-07-14T11:21:53.052 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T11:21:53.052 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:21:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:53 smithi161 ceph-mon[95526]: pgmap v341: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:53.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:53 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1135958186' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:21:53.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.301 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:21:53.370 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-07-14T11:21:53.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.373 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-07-14T11:21:53.421 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-07-14T11:21:53.421 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:21:53.421 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:21:53.422 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-14T11:21:53.422 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:21:53.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.422 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:21:53.423 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:21:53.423 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:21:53.423 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:21:53.423 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:21:53.424 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:21:53.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.599 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-07-14T11:21:53.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.644 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2023-07-14T11:21:53.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:53 smithi175 ceph-mon[129081]: pgmap v341: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:53.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:53 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1135958186' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:21:53.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:53 smithi182 ceph-mon[129366]: pgmap v341: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:53.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:53 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1135958186' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:21:53.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.727 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:21:53.728 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:21:53.728 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:21:53.728 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-14T11:21:53.728 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:21:53.912 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-14T11:21:53.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:53.913 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:21:53.913 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:21:53.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-14T11:21:53.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T11:21:53.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-14T11:21:53.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-14T11:21:53.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-14T11:21:53.914 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:21:54.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:54.167 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-07-14T11:21:54.168 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-07-14T11:21:54.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-07-14T11:21:54.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T11:21:54.169 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-07-14T11:21:54.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:21:54.345 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:21:54.390 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:21:54.390 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:21:54.390 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:21:54.474 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-14T11:21:54.475 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-14T11:21:54.475 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/fbc93191-b860-463a-be29-85439d88d9de 2023-07-14T11:21:54.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:54.476 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/fbc93191-b860-463a-be29-85439d88d9de /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-14T11:21:54.620 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-14T11:21:54.620 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-07-14T11:21:54.620 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2023-07-14T11:21:55.113 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-14T11:21:55.113 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-07-14T11:21:55.113 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2023-07-14T11:21:55.113 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.175:3300,172.21.15.182:3300" 2023-07-14T11:21:55.114 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:3300,172.21.15.175:3300,172.21.15.182:3300:/volumes/_nogroup/sv_1/fbc93191-b860-463a-be29-85439d88d9de 2023-07-14T11:21:55.114 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=2bf7fc3c-2237-11ee-9b32-001a4aab830c" will pass to kernel 2023-07-14T11:21:55.119 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-14T11:21:55.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:55.119 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:21:55.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:55.192 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T11:21:55.280 INFO:teuthology.orchestra.run.smithi161.stdout:client15363 2023-07-14T11:21:55.281 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:21:55.281 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:21:55.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:55.327 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T11:21:55.452 INFO:teuthology.orchestra.run.smithi161.stdout:client15363 2023-07-14T11:21:55.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:55.453 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T11:21:55.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:55 smithi161 ceph-mon[95526]: pgmap v342: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T11:21:55.540 INFO:teuthology.orchestra.run.smithi161.stdout:client15363 2023-07-14T11:21:55.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:21:55.541 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c.client15363/status') 2023-07-14T11:21:55.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:55 smithi175 ceph-mon[129081]: pgmap v342: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T11:21:55.689 INFO:teuthology.orchestra.run.smithi161.stdout:instance: client.15363 (3)192.168.0.1:0/1646529752 2023-07-14T11:21:55.690 INFO:teuthology.orchestra.run.smithi161.stdout:blocklisted: false 2023-07-14T11:21:55.690 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records in 2023-07-14T11:21:55.690 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records out 2023-07-14T11:21:55.690 INFO:teuthology.orchestra.run.smithi161.stderr:70 bytes copied, 7.1057e-05 s, 985 kB/s 2023-07-14T11:21:55.693 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-14T11:21:55.704 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-14T11:21:55.734 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:55.735 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:55 smithi182 ceph-mon[129366]: pgmap v342: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T11:21:55.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 -- 172.21.15.161:0/654873986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 msgr2=0x7f6a780fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:55.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 --2- 172.21.15.161:0/654873986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780fff10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009990 tx=0x7f6a74009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:55.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 -- 172.21.15.161:0/654873986 shutdown_connections 2023-07-14T11:21:55.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 --2- 172.21.15.161:0/654873986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780fff10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:55.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 --2- 172.21.15.161:0/654873986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a78101680 0x7f6a78107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 --2- 172.21.15.161:0/654873986 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a78100cf0 0x7f6a78101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 -- 172.21.15.161:0/654873986 >> 172.21.15.161:0/654873986 conn(0x7f6a780fb0e0 msgr2=0x7f6a780fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.933+0000 7f6a807ff700 1 -- 172.21.15.161:0/654873986 shutdown_connections 2023-07-14T11:21:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.934+0000 7f6a807ff700 1 -- 172.21.15.161:0/654873986 wait complete. 2023-07-14T11:21:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.935+0000 7f6a807ff700 1 Processor -- start 2023-07-14T11:21:55.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.935+0000 7f6a807ff700 1 -- start start 2023-07-14T11:21:55.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a807ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:55.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a807ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a78100cf0 0x7f6a78072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a807ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a78101680 0x7f6a78072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a807ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780735e0 con 0x7f6a780ffb00 2023-07-14T11:21:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a807ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78073750 con 0x7f6a78101680 2023-07-14T11:21:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a807ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819f870 con 0x7f6a78100cf0 2023-07-14T11:21:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48516/0 (socket says 172.21.15.161:48516) 2023-07-14T11:21:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 -- 172.21.15.161:0/1385008681 learned_addr learned my addr 172.21.15.161:0/1385008681 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7dd9a700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a78100cf0 0x7f6a78072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7ed9c700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a78101680 0x7f6a78072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 -- 172.21.15.161:0/1385008681 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a78100cf0 msgr2=0x7f6a78072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a78100cf0 0x7f6a78072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 -- 172.21.15.161:0/1385008681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a78101680 msgr2=0x7f6a78072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a78101680 0x7f6a78072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.936+0000 7f6a7e59b700 1 -- 172.21.15.161:0/1385008681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740095f0 con 0x7f6a780ffb00 2023-07-14T11:21:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.937+0000 7f6a7e59b700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780742c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6a74005740 tx=0x7f6a74005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.937+0000 7f6a6f7fe700 1 -- 172.21.15.161:0/1385008681 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7401d070 con 0x7f6a780ffb00 2023-07-14T11:21:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.937+0000 7f6a6f7fe700 1 -- 172.21.15.161:0/1385008681 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a74021410 con 0x7f6a780ffb00 2023-07-14T11:21:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.937+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7819faf0 con 0x7f6a780ffb00 2023-07-14T11:21:55.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.937+0000 7f6a6f7fe700 1 -- 172.21.15.161:0/1385008681 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74017650 con 0x7f6a780ffb00 2023-07-14T11:21:55.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.937+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7819ff10 con 0x7f6a780ffb00 2023-07-14T11:21:55.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.938+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a78107050 con 0x7f6a780ffb00 2023-07-14T11:21:55.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.940+0000 7f6a6f7fe700 1 -- 172.21.15.161:0/1385008681 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6a74021580 con 0x7f6a780ffb00 2023-07-14T11:21:55.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.941+0000 7f6a6f7fe700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6a640a11a0 0x7f6a640a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:55.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.942+0000 7f6a6f7fe700 1 -- 172.21.15.161:0/1385008681 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6a740cdde0 con 0x7f6a780ffb00 2023-07-14T11:21:55.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.942+0000 7f6a7dd9a700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6a640a11a0 0x7f6a640a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.942+0000 7f6a7dd9a700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6a640a11a0 0x7f6a640a3650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009c60 tx=0x7f6a68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:55.945+0000 7f6a6f7fe700 1 -- 172.21.15.161:0/1385008681 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6a74097330 con 0x7f6a780ffb00 2023-07-14T11:21:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.144+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a7802c850 con 0x7f6a780ffb00 2023-07-14T11:21:56.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.146+0000 7f6a6f7fe700 1 -- 172.21.15.161:0/1385008681 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7f6a74026090 con 0x7f6a780ffb00 2023-07-14T11:21:56.150 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:56.150 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:56.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6a640a11a0 msgr2=0x7f6a640a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6a640a11a0 0x7f6a640a3650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009c60 tx=0x7f6a68009260 comp rx=0 tx=0).stop 2023-07-14T11:21:56.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 msgr2=0x7f6a780742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780742c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6a74005740 tx=0x7f6a74005770 comp rx=0 tx=0).stop 2023-07-14T11:21:56.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 shutdown_connections 2023-07-14T11:21:56.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a780ffb00 0x7f6a780742c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f6a640a11a0 0x7f6a640a3650 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a78101680 0x7f6a78072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 --2- 172.21.15.161:0/1385008681 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a78100cf0 0x7f6a78072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 >> 172.21.15.161:0/1385008681 conn(0x7f6a780fb0e0 msgr2=0x7f6a78104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:56.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 shutdown_connections 2023-07-14T11:21:56.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.149+0000 7f6a807ff700 1 -- 172.21.15.161:0/1385008681 wait complete. 2023-07-14T11:21:56.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:21:56.172 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:21:56.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/3089407065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 msgr2=0x7f3b5410a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/3089407065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 0x7f3b5410a140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009f90 tx=0x7f3b4c00b330 comp rx=0 tx=0).stop 2023-07-14T11:21:56.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/3089407065 shutdown_connections 2023-07-14T11:21:56.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/3089407065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 0x7f3b5410a140 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/3089407065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 0x7f3b54102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/3089407065 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3b54102ef0 0x7f3b54103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/3089407065 >> 172.21.15.161:0/3089407065 conn(0x7f3b540fd360 msgr2=0x7f3b540ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:56.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/3089407065 shutdown_connections 2023-07-14T11:21:56.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.389+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/3089407065 wait complete. 2023-07-14T11:21:56.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 Processor -- start 2023-07-14T11:21:56.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 -- start start 2023-07-14T11:21:56.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 0x7f3b54197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3b54102ef0 0x7f3b54198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 0x7f3b5419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5419f6d0 con 0x7f3b54103880 2023-07-14T11:21:56.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5419f810 con 0x7f3b54101d00 2023-07-14T11:21:56.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.390+0000 7f3b5c5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5419f9a0 con 0x7f3b54102ef0 2023-07-14T11:21:56.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5a39b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 0x7f3b54197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5ab9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 0x7f3b5419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5a39b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 0x7f3b54197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35488/0 (socket says 172.21.15.161:35488) 2023-07-14T11:21:56.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5a39b700 1 -- 172.21.15.161:0/2640817167 learned_addr learned my addr 172.21.15.161:0/2640817167 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:56.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b59b9a700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3b54102ef0 0x7f3b54198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5ab9c700 1 -- 172.21.15.161:0/2640817167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3b54102ef0 msgr2=0x7f3b54198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5ab9c700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3b54102ef0 0x7f3b54198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5ab9c700 1 -- 172.21.15.161:0/2640817167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 msgr2=0x7f3b54197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5ab9c700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 0x7f3b54197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5ab9c700 1 -- 172.21.15.161:0/2640817167 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500095f0 con 0x7f3b54103880 2023-07-14T11:21:56.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5a39b700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 0x7f3b54197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:56.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.391+0000 7f3b5ab9c700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 0x7f3b5419f100 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c007930 tx=0x7f3b4c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:56.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.392+0000 7f3b4b7fe700 1 -- 172.21.15.161:0/2640817167 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00e040 con 0x7f3b54103880 2023-07-14T11:21:56.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.392+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b44009520 con 0x7f3b54103880 2023-07-14T11:21:56.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.392+0000 7f3b4b7fe700 1 -- 172.21.15.161:0/2640817167 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b4c008ef0 con 0x7f3b54103880 2023-07-14T11:21:56.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.392+0000 7f3b4b7fe700 1 -- 172.21.15.161:0/2640817167 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0052a0 con 0x7f3b54103880 2023-07-14T11:21:56.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.392+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c009c40 con 0x7f3b54103880 2023-07-14T11:21:56.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.393+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b54109200 con 0x7f3b54103880 2023-07-14T11:21:56.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.395+0000 7f3b4b7fe700 1 -- 172.21.15.161:0/2640817167 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3b4c01d470 con 0x7f3b54103880 2023-07-14T11:21:56.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.396+0000 7f3b4b7fe700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b400a1150 0x7f3b400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.396+0000 7f3b4b7fe700 1 -- 172.21.15.161:0/2640817167 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3b4c0ce1b0 con 0x7f3b54103880 2023-07-14T11:21:56.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.396+0000 7f3b5a39b700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b400a1150 0x7f3b400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.397+0000 7f3b5a39b700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b400a1150 0x7f3b400a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005950 tx=0x7f3b50009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:56.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.400+0000 7f3b4b7fe700 1 -- 172.21.15.161:0/2640817167 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3b4c0975e0 con 0x7f3b54103880 2023-07-14T11:21:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:56 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1385008681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.600+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b54061120 con 0x7f3b54103880 2023-07-14T11:21:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.600+0000 7f3b4b7fe700 1 -- 172.21.15.161:0/2640817167 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b4c09ac00 con 0x7f3b54103880 2023-07-14T11:21:56.603 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:21:56.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b400a1150 msgr2=0x7f3b400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b400a1150 0x7f3b400a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005950 tx=0x7f3b50009310 comp rx=0 tx=0).stop 2023-07-14T11:21:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 msgr2=0x7f3b5419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 0x7f3b5419f100 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c007930 tx=0x7f3b4c004c80 comp rx=0 tx=0).stop 2023-07-14T11:21:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 shutdown_connections 2023-07-14T11:21:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54103880 0x7f3b5419f100 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3b400a1150 0x7f3b400a3600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b54101d00 0x7f3b54197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 --2- 172.21.15.161:0/2640817167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3b54102ef0 0x7f3b54198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 >> 172.21.15.161:0/2640817167 conn(0x7f3b540fd360 msgr2=0x7f3b54106340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 shutdown_connections 2023-07-14T11:21:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.603+0000 7f3b5c5ff700 1 -- 172.21.15.161:0/2640817167 wait complete. 2023-07-14T11:21:56.624 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:21:56.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:56 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1385008681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:56.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:56 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1385008681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:56.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 -- 172.21.15.161:0/2438857139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 msgr2=0x7f636c102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 --2- 172.21.15.161:0/2438857139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c102a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f635c009990 tx=0x7f635c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 -- 172.21.15.161:0/2438857139 shutdown_connections 2023-07-14T11:21:56.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 --2- 172.21.15.161:0/2438857139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c102a00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 --2- 172.21.15.161:0/2438857139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f636c105860 0x7f636c107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 --2- 172.21.15.161:0/2438857139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c102f40 0x7f636c105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 -- 172.21.15.161:0/2438857139 >> 172.21.15.161:0/2438857139 conn(0x7f636c0fa000 msgr2=0x7f636c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:56.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 -- 172.21.15.161:0/2438857139 shutdown_connections 2023-07-14T11:21:56.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.824+0000 7f63721ff700 1 -- 172.21.15.161:0/2438857139 wait complete. 2023-07-14T11:21:56.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 Processor -- start 2023-07-14T11:21:56.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 -- start start 2023-07-14T11:21:56.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f636c102f40 0x7f636c0fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c105860 0x7f636c0feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0ff2b0 con 0x7f636c100620 2023-07-14T11:21:56.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0ff420 con 0x7f636c105860 2023-07-14T11:21:56.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.825+0000 7f63721ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19f8d0 con 0x7f636c102f40 2023-07-14T11:21:56.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c0fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48552/0 (socket says 172.21.15.161:48552) 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 -- 172.21.15.161:0/4219978884 learned_addr learned my addr 172.21.15.161:0/4219978884 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636bfff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c105860 0x7f636c0feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f6363fff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f636c102f40 0x7f636c0fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 -- 172.21.15.161:0/4219978884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f636c102f40 msgr2=0x7f636c0fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f636c102f40 0x7f636c0fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 -- 172.21.15.161:0/4219978884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c105860 msgr2=0x7f636c0feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:56.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c105860 0x7f636c0feb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:56.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 -- 172.21.15.161:0/4219978884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354009520 con 0x7f636c100620 2023-07-14T11:21:56.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f636b7fe700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c0fff90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f635c000c00 tx=0x7f635c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:56.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.826+0000 7f63697fa700 1 -- 172.21.15.161:0/4219978884 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c01d070 con 0x7f636c100620 2023-07-14T11:21:56.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.827+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c0095f0 con 0x7f636c100620 2023-07-14T11:21:56.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.827+0000 7f63697fa700 1 -- 172.21.15.161:0/4219978884 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635c00fb90 con 0x7f636c100620 2023-07-14T11:21:56.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.827+0000 7f63697fa700 1 -- 172.21.15.161:0/4219978884 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c0173f0 con 0x7f636c100620 2023-07-14T11:21:56.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.827+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c19fec0 con 0x7f636c100620 2023-07-14T11:21:56.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.828+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c0fbb70 con 0x7f636c100620 2023-07-14T11:21:56.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.830+0000 7f63697fa700 1 -- 172.21.15.161:0/4219978884 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f635c021410 con 0x7f636c100620 2023-07-14T11:21:56.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.831+0000 7f63697fa700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63580a1150 0x7f63580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:56.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.831+0000 7f63697fa700 1 -- 172.21.15.161:0/4219978884 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f635c0ce6c0 con 0x7f636c100620 2023-07-14T11:21:56.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.831+0000 7f6363fff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63580a1150 0x7f63580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:56.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.832+0000 7f6363fff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63580a1150 0x7f63580a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6354009df0 tx=0x7f6354009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:56.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:56.834+0000 7f63697fa700 1 -- 172.21.15.161:0/4219978884 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f635c097b50 con 0x7f636c100620 2023-07-14T11:21:57.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.036+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f636c02caa0 con 0x7f636c100620 2023-07-14T11:21:57.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.036+0000 7f63697fa700 1 -- 172.21.15.161:0/4219978884 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f635c09b170 con 0x7f636c100620 2023-07-14T11:21:57.039 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:57.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63580a1150 msgr2=0x7f63580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63580a1150 0x7f63580a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6354009df0 tx=0x7f6354009260 comp rx=0 tx=0).stop 2023-07-14T11:21:57.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 msgr2=0x7f636c0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c0fff90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f635c000c00 tx=0x7f635c0057a0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 shutdown_connections 2023-07-14T11:21:57.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f636c100620 0x7f636c0fff90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f63580a1150 0x7f63580a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c105860 0x7f636c0feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 --2- 172.21.15.161:0/4219978884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f636c102f40 0x7f636c0fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 >> 172.21.15.161:0/4219978884 conn(0x7f636c0fa000 msgr2=0x7f636c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:57.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 shutdown_connections 2023-07-14T11:21:57.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.039+0000 7f63721ff700 1 -- 172.21.15.161:0/4219978884 wait complete. 2023-07-14T11:21:57.060 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:21:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/52737132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 msgr2=0x7fa3f8101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/52737132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8101130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/52737132 shutdown_connections 2023-07-14T11:21:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/52737132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f8101670 0x7fa3f8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/52737132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3f80ffaf0 0x7fa3f80fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/52737132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8101130 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/52737132 >> 172.21.15.161:0/52737132 conn(0x7fa3f80fb130 msgr2=0x7fa3f80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/52737132 shutdown_connections 2023-07-14T11:21:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.265+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/52737132 wait complete. 2023-07-14T11:21:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.266+0000 7fa3fe3ff700 1 Processor -- start 2023-07-14T11:21:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.266+0000 7fa3fe3ff700 1 -- start start 2023-07-14T11:21:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.266+0000 7fa3fe3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f80ffaf0 0x7fa3f8195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3fe3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3fe3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3f8101670 0x7fa3f819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3fe3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f819d3c0 con 0x7fa3f8100ce0 2023-07-14T11:21:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3fe3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f819d500 con 0x7fa3f80ffaf0 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3fe3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f819d690 con 0x7fa3f8101670 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48564/0 (socket says 172.21.15.161:48564) 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 -- 172.21.15.161:0/1752462839 learned_addr learned my addr 172.21.15.161:0/1752462839 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3f7fff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f80ffaf0 0x7fa3f8195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3fc99c700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3f8101670 0x7fa3f819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 -- 172.21.15.161:0/1752462839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3f8101670 msgr2=0x7fa3f819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3f8101670 0x7fa3f819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 -- 172.21.15.161:0/1752462839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f80ffaf0 msgr2=0x7fa3f8195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f80ffaf0 0x7fa3f8195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.267+0000 7fa3effff700 1 -- 172.21.15.161:0/1752462839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00095f0 con 0x7fa3f8100ce0 2023-07-14T11:21:57.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.268+0000 7fa3effff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8195d80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b620 tx=0x7fa3e0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:57.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.268+0000 7fa3f5ffb700 1 -- 172.21.15.161:0/1752462839 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001d070 con 0x7fa3f8100ce0 2023-07-14T11:21:57.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.268+0000 7fa3f5ffb700 1 -- 172.21.15.161:0/1752462839 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e0022470 con 0x7fa3f8100ce0 2023-07-14T11:21:57.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.268+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f819d910 con 0x7fa3f8100ce0 2023-07-14T11:21:57.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.268+0000 7fa3f5ffb700 1 -- 172.21.15.161:0/1752462839 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0017650 con 0x7fa3f8100ce0 2023-07-14T11:21:57.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.268+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f819dd30 con 0x7fa3f8100ce0 2023-07-14T11:21:57.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.269+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f810b310 con 0x7fa3f8100ce0 2023-07-14T11:21:57.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.271+0000 7fa3f5ffb700 1 -- 172.21.15.161:0/1752462839 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa3e00225e0 con 0x7fa3f8100ce0 2023-07-14T11:21:57.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.273+0000 7fa3f5ffb700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3e40a1150 0x7fa3e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.273+0000 7fa3f5ffb700 1 -- 172.21.15.161:0/1752462839 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa3e00ce250 con 0x7fa3f8100ce0 2023-07-14T11:21:57.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.274+0000 7fa3f7fff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3e40a1150 0x7fa3e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.274+0000 7fa3f7fff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3e40a1150 0x7fa3e40a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8100b40 tx=0x7fa3e800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:57.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.277+0000 7fa3f5ffb700 1 -- 172.21.15.161:0/1752462839 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa3e0097680 con 0x7fa3f8100ce0 2023-07-14T11:21:57.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.476+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3f81069f0 con 0x7fa3f8100ce0 2023-07-14T11:21:57.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.476+0000 7fa3f5ffb700 1 -- 172.21.15.161:0/1752462839 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa3e009aca0 con 0x7fa3f8100ce0 2023-07-14T11:21:57.479 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:21:57.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3e40a1150 msgr2=0x7fa3e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3e40a1150 0x7fa3e40a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8100b40 tx=0x7fa3e800b410 comp rx=0 tx=0).stop 2023-07-14T11:21:57.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 msgr2=0x7fa3f8195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8195d80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b620 tx=0x7fa3e0005740 comp rx=0 tx=0).stop 2023-07-14T11:21:57.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 shutdown_connections 2023-07-14T11:21:57.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fa3e40a1150 0x7fa3e40a3600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa3f8101670 0x7fa3f819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f80ffaf0 0x7fa3f8195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 --2- 172.21.15.161:0/1752462839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f8100ce0 0x7fa3f8195d80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 >> 172.21.15.161:0/1752462839 conn(0x7fa3f80fb130 msgr2=0x7fa3f80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:57.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 shutdown_connections 2023-07-14T11:21:57.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.479+0000 7fa3fe3ff700 1 -- 172.21.15.161:0/1752462839 wait complete. 2023-07-14T11:21:57.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:57 smithi161 ceph-mon[95526]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:57 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2640817167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:57 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4219978884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:57.501 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:21:57.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:57 smithi175 ceph-mon[129081]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:57.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:57 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2640817167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:57.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:57 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4219978884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:57.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 -- 172.21.15.161:0/130836107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 msgr2=0x7faec0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 --2- 172.21.15.161:0/130836107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 0x7faec0101130 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009990 tx=0x7faeb0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 -- 172.21.15.161:0/130836107 shutdown_connections 2023-07-14T11:21:57.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 --2- 172.21.15.161:0/130836107 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faec0101670 0x7faec0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 --2- 172.21.15.161:0/130836107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 0x7faec00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 --2- 172.21.15.161:0/130836107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 0x7faec0101130 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 -- 172.21.15.161:0/130836107 >> 172.21.15.161:0/130836107 conn(0x7faec00fb130 msgr2=0x7faec00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:57.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 -- 172.21.15.161:0/130836107 shutdown_connections 2023-07-14T11:21:57.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.700+0000 7faec83ff700 1 -- 172.21.15.161:0/130836107 wait complete. 2023-07-14T11:21:57.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.702+0000 7faec83ff700 1 Processor -- start 2023-07-14T11:21:57.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.702+0000 7faec83ff700 1 -- start start 2023-07-14T11:21:57.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.702+0000 7faec83ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 0x7faec00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.702+0000 7faec83ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 0x7faec0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.702+0000 7faec83ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faec0101670 0x7faec0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec83ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00735e0 con 0x7faec0100ce0 2023-07-14T11:21:57.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec83ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec0073750 con 0x7faec00ffaf0 2023-07-14T11:21:57.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec83ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec019f8a0 con 0x7faec0101670 2023-07-14T11:21:57.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec599a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 0x7faec0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec619b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 0x7faec00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec619b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 0x7faec00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35552/0 (socket says 172.21.15.161:35552) 2023-07-14T11:21:57.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec619b700 1 -- 172.21.15.161:0/3752482313 learned_addr learned my addr 172.21.15.161:0/3752482313 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:57.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec699c700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faec0101670 0x7faec0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec599a700 1 -- 172.21.15.161:0/3752482313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faec0101670 msgr2=0x7faec0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec599a700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faec0101670 0x7faec0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec599a700 1 -- 172.21.15.161:0/3752482313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 msgr2=0x7faec00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec599a700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 0x7faec00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec599a700 1 -- 172.21.15.161:0/3752482313 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc009520 con 0x7faec0100ce0 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec699c700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faec0101670 0x7faec0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec619b700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 0x7faec00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.703+0000 7faec599a700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 0x7faec0072900 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faeb00098e0 tx=0x7faeb0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.704+0000 7faeb77fe700 1 -- 172.21.15.161:0/3752482313 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb001d070 con 0x7faec0100ce0 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.704+0000 7faeb77fe700 1 -- 172.21.15.161:0/3752482313 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faeb0021410 con 0x7faec0100ce0 2023-07-14T11:21:57.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.704+0000 7faeb77fe700 1 -- 172.21.15.161:0/3752482313 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0017650 con 0x7faec0100ce0 2023-07-14T11:21:57.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.704+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb00095f0 con 0x7faec0100ce0 2023-07-14T11:21:57.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.704+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec019fd60 con 0x7faec0100ce0 2023-07-14T11:21:57.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.705+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec010b310 con 0x7faec0100ce0 2023-07-14T11:21:57.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.707+0000 7faeb77fe700 1 -- 172.21.15.161:0/3752482313 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7faeb0021580 con 0x7faec0100ce0 2023-07-14T11:21:57.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.708+0000 7faeb77fe700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faeac0a1150 0x7faeac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:57.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.708+0000 7faeb77fe700 1 -- 172.21.15.161:0/3752482313 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faeb00ce0c0 con 0x7faec0100ce0 2023-07-14T11:21:57.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.708+0000 7faec619b700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faeac0a1150 0x7faeac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:57.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.709+0000 7faec619b700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faeac0a1150 0x7faeac0a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faebc009fd0 tx=0x7faebc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:57.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:57 smithi182 ceph-mon[129366]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:21:57.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:57 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2640817167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:57.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:57 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4219978884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.714+0000 7faeb77fe700 1 -- 172.21.15.161:0/3752482313 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faeb00974f0 con 0x7faec0100ce0 2023-07-14T11:21:57.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.914+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faec01069f0 con 0x7faec0100ce0 2023-07-14T11:21:57.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.915+0000 7faeb77fe700 1 -- 172.21.15.161:0/3752482313 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7faeb009ab10 con 0x7faec0100ce0 2023-07-14T11:21:57.917 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:57.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faeac0a1150 msgr2=0x7faeac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faeac0a1150 0x7faeac0a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faebc009fd0 tx=0x7faebc009260 comp rx=0 tx=0).stop 2023-07-14T11:21:57.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 msgr2=0x7faec0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:57.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 0x7faec0072900 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faeb00098e0 tx=0x7faeb0005740 comp rx=0 tx=0).stop 2023-07-14T11:21:57.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 shutdown_connections 2023-07-14T11:21:57.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7faeac0a1150 0x7faeac0a3600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faec0101670 0x7faec0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ffaf0 0x7faec00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 --2- 172.21.15.161:0/3752482313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec0100ce0 0x7faec0072900 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:57.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 >> 172.21.15.161:0/3752482313 conn(0x7faec00fb130 msgr2=0x7faec00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:57.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 shutdown_connections 2023-07-14T11:21:57.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:57.917+0000 7faec83ff700 1 -- 172.21.15.161:0/3752482313 wait complete. 2023-07-14T11:21:57.939 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:21:58.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 -- 172.21.15.161:0/2149010672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 msgr2=0x7f1530101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 --2- 172.21.15.161:0/2149010672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530101140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1520009990 tx=0x7f1520009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 -- 172.21.15.161:0/2149010672 shutdown_connections 2023-07-14T11:21:58.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 --2- 172.21.15.161:0/2149010672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1530101680 0x7f1530107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 --2- 172.21.15.161:0/2149010672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15300ffb00 0x7f15300fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 --2- 172.21.15.161:0/2149010672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530101140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 -- 172.21.15.161:0/2149010672 >> 172.21.15.161:0/2149010672 conn(0x7f15300fb0e0 msgr2=0x7f15300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:58.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 -- 172.21.15.161:0/2149010672 shutdown_connections 2023-07-14T11:21:58.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.140+0000 7f15375ff700 1 -- 172.21.15.161:0/2149010672 wait complete. 2023-07-14T11:21:58.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.141+0000 7f15375ff700 1 Processor -- start 2023-07-14T11:21:58.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.141+0000 7f15375ff700 1 -- start start 2023-07-14T11:21:58.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f15375ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15300ffb00 0x7f1530197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f15375ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f15375ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1530101680 0x7f153019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f15375ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153019f600 con 0x7f1530100cf0 2023-07-14T11:21:58.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f15375ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153019f740 con 0x7f1530101680 2023-07-14T11:21:58.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f15375ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153019f8d0 con 0x7f15300ffb00 2023-07-14T11:21:58.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39566/0 (socket says 172.21.15.161:39566) 2023-07-14T11:21:58.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 -- 172.21.15.161:0/1896324788 learned_addr learned my addr 172.21.15.161:0/1896324788 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:58.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f153539b700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15300ffb00 0x7f1530197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1535b9c700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1530101680 0x7f153019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 -- 172.21.15.161:0/1896324788 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15300ffb00 msgr2=0x7f1530197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15300ffb00 0x7f1530197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 -- 172.21.15.161:0/1896324788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1530101680 msgr2=0x7f153019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1530101680 0x7f153019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.142+0000 7f1534b9a700 1 -- 172.21.15.161:0/1896324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200095f0 con 0x7f1530100cf0 2023-07-14T11:21:58.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.143+0000 7f1534b9a700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530197fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f152000b620 tx=0x7f1520005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:58.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.143+0000 7f15267fc700 1 -- 172.21.15.161:0/1896324788 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152001d070 con 0x7f1530100cf0 2023-07-14T11:21:58.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.143+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153019fb50 con 0x7f1530100cf0 2023-07-14T11:21:58.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.143+0000 7f15267fc700 1 -- 172.21.15.161:0/1896324788 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1520022470 con 0x7f1530100cf0 2023-07-14T11:21:58.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.143+0000 7f15267fc700 1 -- 172.21.15.161:0/1896324788 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520017650 con 0x7f1530100cf0 2023-07-14T11:21:58.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.143+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153019ff80 con 0x7f1530100cf0 2023-07-14T11:21:58.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.144+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153010b320 con 0x7f1530100cf0 2023-07-14T11:21:58.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.146+0000 7f15267fc700 1 -- 172.21.15.161:0/1896324788 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f15200225e0 con 0x7f1530100cf0 2023-07-14T11:21:58.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.148+0000 7f15267fc700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f151c0a11a0 0x7f151c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.149+0000 7f15267fc700 1 -- 172.21.15.161:0/1896324788 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f15200ce800 con 0x7f1530100cf0 2023-07-14T11:21:58.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.149+0000 7f153539b700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f151c0a11a0 0x7f151c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.149+0000 7f153539b700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f151c0a11a0 0x7f151c0a3650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1530100b50 tx=0x7f152c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:58.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.151+0000 7f15267fc700 1 -- 172.21.15.161:0/1896324788 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1520097c30 con 0x7f1530100cf0 2023-07-14T11:21:58.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.354+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15301a0590 con 0x7f1530100cf0 2023-07-14T11:21:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.355+0000 7f15267fc700 1 -- 172.21.15.161:0/1896324788 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7f152002e0d0 con 0x7f1530100cf0 2023-07-14T11:21:58.358 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:21:58.358 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:21:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f151c0a11a0 msgr2=0x7f151c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f151c0a11a0 0x7f151c0a3650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1530100b50 tx=0x7f152c009260 comp rx=0 tx=0).stop 2023-07-14T11:21:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 msgr2=0x7f1530197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530197fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f152000b620 tx=0x7f1520005740 comp rx=0 tx=0).stop 2023-07-14T11:21:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 shutdown_connections 2023-07-14T11:21:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f151c0a11a0 0x7f151c0a3650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1530101680 0x7f153019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f15300ffb00 0x7f1530197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 --2- 172.21.15.161:0/1896324788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1530100cf0 0x7f1530197fc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 >> 172.21.15.161:0/1896324788 conn(0x7f15300fb0e0 msgr2=0x7f1530106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:58.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 shutdown_connections 2023-07-14T11:21:58.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.358+0000 7f15375ff700 1 -- 172.21.15.161:0/1896324788 wait complete. 2023-07-14T11:21:58.379 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:21:58.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:21:58.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:58 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1752462839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:58 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3752482313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:58.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.581+0000 7f780edff700 1 -- 172.21.15.161:0/3498008826 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7808101670 msgr2=0x7f7808107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.581+0000 7f780edff700 1 --2- 172.21.15.161:0/3498008826 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7808101670 0x7f7808107f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f780000b0a0 tx=0x7f780000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.581+0000 7f780edff700 1 -- 172.21.15.161:0/3498008826 shutdown_connections 2023-07-14T11:21:58.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.581+0000 7f780edff700 1 --2- 172.21.15.161:0/3498008826 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7808101670 0x7f7808107f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.581+0000 7f780edff700 1 --2- 172.21.15.161:0/3498008826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78080ffaf0 0x7f78080fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.581+0000 7f780edff700 1 --2- 172.21.15.161:0/3498008826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808100ce0 0x7f7808101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.581+0000 7f780edff700 1 -- 172.21.15.161:0/3498008826 >> 172.21.15.161:0/3498008826 conn(0x7f78080fb130 msgr2=0x7f78080fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:58.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.582+0000 7f780edff700 1 -- 172.21.15.161:0/3498008826 shutdown_connections 2023-07-14T11:21:58.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.582+0000 7f780edff700 1 -- 172.21.15.161:0/3498008826 wait complete. 2023-07-14T11:21:58.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 Processor -- start 2023-07-14T11:21:58.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 -- start start 2023-07-14T11:21:58.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78080ffaf0 0x7f7808197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7808100ce0 0x7f7808197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808101670 0x7f780819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819f4b0 con 0x7f7808101670 2023-07-14T11:21:58.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819f5f0 con 0x7f7808100ce0 2023-07-14T11:21:58.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780edff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819f780 con 0x7f78080ffaf0 2023-07-14T11:21:58.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780d39c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808101670 0x7f780819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780cb9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78080ffaf0 0x7f7808197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780d39c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808101670 0x7f780819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39588/0 (socket says 172.21.15.161:39588) 2023-07-14T11:21:58.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.583+0000 7f780d39c700 1 -- 172.21.15.161:0/2304472480 learned_addr learned my addr 172.21.15.161:0/2304472480 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:58.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f7807fff700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7808100ce0 0x7f7808197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780d39c700 1 -- 172.21.15.161:0/2304472480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78080ffaf0 msgr2=0x7f7808197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780d39c700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78080ffaf0 0x7f7808197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780d39c700 1 -- 172.21.15.161:0/2304472480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7808100ce0 msgr2=0x7f7808197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780d39c700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7808100ce0 0x7f7808197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780d39c700 1 -- 172.21.15.161:0/2304472480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7800009d00 con 0x7f7808101670 2023-07-14T11:21:58.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780cb9b700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78080ffaf0 0x7f7808197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:58.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780d39c700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808101670 0x7f780819eee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7800009850 tx=0x7f780000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:58.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f7805ffb700 1 -- 172.21.15.161:0/2304472480 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780000e040 con 0x7f7808101670 2023-07-14T11:21:58.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780819fa00 con 0x7f7808101670 2023-07-14T11:21:58.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f7805ffb700 1 -- 172.21.15.161:0/2304472480 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7800003e10 con 0x7f7808101670 2023-07-14T11:21:58.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f7805ffb700 1 -- 172.21.15.161:0/2304472480 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780001b6c0 con 0x7f7808101670 2023-07-14T11:21:58.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.584+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780819fdd0 con 0x7f7808101670 2023-07-14T11:21:58.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.585+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780810b260 con 0x7f7808101670 2023-07-14T11:21:58.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.587+0000 7f7805ffb700 1 -- 172.21.15.161:0/2304472480 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f780001d470 con 0x7f7808101670 2023-07-14T11:21:58.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.589+0000 7f7805ffb700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77f80a1150 0x7f77f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:58.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.589+0000 7f7805ffb700 1 -- 172.21.15.161:0/2304472480 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f78000ce110 con 0x7f7808101670 2023-07-14T11:21:58.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.589+0000 7f780cb9b700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77f80a1150 0x7f77f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:58.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.590+0000 7f780cb9b700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77f80a1150 0x7f77f80a3600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7808100b40 tx=0x7f77fc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:58.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.592+0000 7f7805ffb700 1 -- 172.21.15.161:0/2304472480 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78000975a0 con 0x7f7808101670 2023-07-14T11:21:58.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:58 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1752462839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:58.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:58 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3752482313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:58.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:58 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1752462839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:21:58.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:58 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3752482313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:21:58.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.789+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78081a0410 con 0x7f7808101670 2023-07-14T11:21:58.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.789+0000 7f7805ffb700 1 -- 172.21.15.161:0/2304472480 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f780009abc0 con 0x7f7808101670 2023-07-14T11:21:58.792 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:21:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77f80a1150 msgr2=0x7f77f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77f80a1150 0x7f77f80a3600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7808100b40 tx=0x7f77fc009260 comp rx=0 tx=0).stop 2023-07-14T11:21:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808101670 msgr2=0x7f780819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808101670 0x7f780819eee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7800009850 tx=0x7f780000bce0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 shutdown_connections 2023-07-14T11:21:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7808101670 0x7f780819eee0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77f80a1150 0x7f77f80a3600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78080ffaf0 0x7f7808197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 --2- 172.21.15.161:0/2304472480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7808100ce0 0x7f7808197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 >> 172.21.15.161:0/2304472480 conn(0x7f78080fb130 msgr2=0x7f78080fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 shutdown_connections 2023-07-14T11:21:58.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:58.792+0000 7f780edff700 1 -- 172.21.15.161:0/2304472480 wait complete. 2023-07-14T11:21:58.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:21:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 -- 172.21.15.161:0/3118990967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc680ffaa0 msgr2=0x7fcc680ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 --2- 172.21.15.161:0/3118990967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc680ffaa0 0x7fcc680ffeb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009990 tx=0x7fcc64009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 -- 172.21.15.161:0/3118990967 shutdown_connections 2023-07-14T11:21:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 --2- 172.21.15.161:0/3118990967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc680ffaa0 0x7fcc680ffeb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 --2- 172.21.15.161:0/3118990967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc681016e0 0x7fcc68107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 --2- 172.21.15.161:0/3118990967 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc68100d30 0x7fcc681011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 -- 172.21.15.161:0/3118990967 >> 172.21.15.161:0/3118990967 conn(0x7fcc680fb0c0 msgr2=0x7fcc680fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.014+0000 7fcc705ff700 1 -- 172.21.15.161:0/3118990967 shutdown_connections 2023-07-14T11:21:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.015+0000 7fcc705ff700 1 -- 172.21.15.161:0/3118990967 wait complete. 2023-07-14T11:21:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.015+0000 7fcc705ff700 1 Processor -- start 2023-07-14T11:21:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc705ff700 1 -- start start 2023-07-14T11:21:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc705ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc680ffaa0 0x7fcc681979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc705ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68100d30 0x7fcc68197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc705ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc681016e0 0x7fcc6819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc705ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819f540 con 0x7fcc68100d30 2023-07-14T11:21:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc705ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819f680 con 0x7fcc681016e0 2023-07-14T11:21:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc705ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819f810 con 0x7fcc680ffaa0 2023-07-14T11:21:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc6db9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68100d30 0x7fcc68197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc6db9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68100d30 0x7fcc68197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39612/0 (socket says 172.21.15.161:39612) 2023-07-14T11:21:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc6db9a700 1 -- 172.21.15.161:0/867150774 learned_addr learned my addr 172.21.15.161:0/867150774 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc6e39b700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc680ffaa0 0x7fcc681979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.016+0000 7fcc6eb9c700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc681016e0 0x7fcc6819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc6db9a700 1 -- 172.21.15.161:0/867150774 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc680ffaa0 msgr2=0x7fcc681979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc6db9a700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc680ffaa0 0x7fcc681979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc6db9a700 1 -- 172.21.15.161:0/867150774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc681016e0 msgr2=0x7fcc6819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc6db9a700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc681016e0 0x7fcc6819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc6db9a700 1 -- 172.21.15.161:0/867150774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640095f0 con 0x7fcc68100d30 2023-07-14T11:21:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc6db9a700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68100d30 0x7fcc68197f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800d950 tx=0x7fcc5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc5f7fe700 1 -- 172.21.15.161:0/867150774 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58009940 con 0x7fcc68100d30 2023-07-14T11:21:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc5f7fe700 1 -- 172.21.15.161:0/867150774 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc58010460 con 0x7fcc68100d30 2023-07-14T11:21:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6819faf0 con 0x7fcc68100d30 2023-07-14T11:21:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc5f7fe700 1 -- 172.21.15.161:0/867150774 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800f620 con 0x7fcc68100d30 2023-07-14T11:21:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.017+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc681a0050 con 0x7fcc68100d30 2023-07-14T11:21:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.018+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc68192130 con 0x7fcc68100d30 2023-07-14T11:21:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.020+0000 7fcc5f7fe700 1 -- 172.21.15.161:0/867150774 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcc580108d0 con 0x7fcc68100d30 2023-07-14T11:21:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.021+0000 7fcc5f7fe700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcc540a11d0 0x7fcc540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.022+0000 7fcc5f7fe700 1 -- 172.21.15.161:0/867150774 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcc580cca30 con 0x7fcc68100d30 2023-07-14T11:21:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.022+0000 7fcc6e39b700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcc540a11d0 0x7fcc540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.022+0000 7fcc6e39b700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcc540a11d0 0x7fcc540a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400b5c0 tx=0x7fcc64005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:59.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.025+0000 7fcc5f7fe700 1 -- 172.21.15.161:0/867150774 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcc58095e30 con 0x7fcc68100d30 2023-07-14T11:21:59.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.224+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc680611c0 con 0x7fcc68100d30 2023-07-14T11:21:59.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.225+0000 7fcc5f7fe700 1 -- 172.21.15.161:0/867150774 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc58016020 con 0x7fcc68100d30 2023-07-14T11:21:59.227 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:21:59.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcc540a11d0 msgr2=0x7fcc540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcc540a11d0 0x7fcc540a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400b5c0 tx=0x7fcc64005960 comp rx=0 tx=0).stop 2023-07-14T11:21:59.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68100d30 msgr2=0x7fcc68197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68100d30 0x7fcc68197f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800d950 tx=0x7fcc5800dc60 comp rx=0 tx=0).stop 2023-07-14T11:21:59.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 shutdown_connections 2023-07-14T11:21:59.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcc540a11d0 0x7fcc540a3680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc681016e0 0x7fcc6819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc680ffaa0 0x7fcc681979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 --2- 172.21.15.161:0/867150774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68100d30 0x7fcc68197f00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 >> 172.21.15.161:0/867150774 conn(0x7fcc680fb0c0 msgr2=0x7fcc68106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:59.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 shutdown_connections 2023-07-14T11:21:59.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.227+0000 7fcc705ff700 1 -- 172.21.15.161:0/867150774 wait complete. 2023-07-14T11:21:59.249 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:21:59.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 -- 172.21.15.161:0/327672412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1580ffaf0 msgr2=0x7fe1580fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/327672412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1580ffaf0 0x7fe1580fff00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe148009990 tx=0x7fe148009ca0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 -- 172.21.15.161:0/327672412 shutdown_connections 2023-07-14T11:21:59.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/327672412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1580ffaf0 0x7fe1580fff00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/327672412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe158101670 0x7fe158107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/327672412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe158100ce0 0x7fe158101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 -- 172.21.15.161:0/327672412 >> 172.21.15.161:0/327672412 conn(0x7fe1580fb130 msgr2=0x7fe1580fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:59.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 -- 172.21.15.161:0/327672412 shutdown_connections 2023-07-14T11:21:59.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.448+0000 7fe15e7ff700 1 -- 172.21.15.161:0/327672412 wait complete. 2023-07-14T11:21:59.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.449+0000 7fe15e7ff700 1 Processor -- start 2023-07-14T11:21:59.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.449+0000 7fe15e7ff700 1 -- start start 2023-07-14T11:21:59.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe15e7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580ffaf0 0x7fe158197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe15e7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe158100ce0 0x7fe158197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe15e7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe158101670 0x7fe15819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe15e7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15819f5f0 con 0x7fe158100ce0 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe15e7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15819f730 con 0x7fe1580ffaf0 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe15e7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15819f8c0 con 0x7fe158101670 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe1577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe158100ce0 0x7fe158197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe157fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580ffaf0 0x7fe158197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe157fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580ffaf0 0x7fe158197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54370/0 (socket says 172.21.15.161:54370) 2023-07-14T11:21:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe157fff700 1 -- 172.21.15.161:0/4005506183 learned_addr learned my addr 172.21.15.161:0/4005506183 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe15cd9c700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe158101670 0x7fe15819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe1577fe700 1 -- 172.21.15.161:0/4005506183 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe158101670 msgr2=0x7fe15819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe1577fe700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe158101670 0x7fe15819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe1577fe700 1 -- 172.21.15.161:0/4005506183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580ffaf0 msgr2=0x7fe158197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe1577fe700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580ffaf0 0x7fe158197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe1577fe700 1 -- 172.21.15.161:0/4005506183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480095f0 con 0x7fe158100ce0 2023-07-14T11:21:59.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.450+0000 7fe157fff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580ffaf0 0x7fe158197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:21:59.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.451+0000 7fe1577fe700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe158100ce0 0x7fe158197fb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00d950 tx=0x7fe14c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:59.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.451+0000 7fe1557fa700 1 -- 172.21.15.161:0/4005506183 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00f890 con 0x7fe158100ce0 2023-07-14T11:21:59.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.451+0000 7fe1557fa700 1 -- 172.21.15.161:0/4005506183 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe14c010460 con 0x7fe158100ce0 2023-07-14T11:21:59.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.451+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15819fba0 con 0x7fe158100ce0 2023-07-14T11:21:59.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.451+0000 7fe1557fa700 1 -- 172.21.15.161:0/4005506183 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00e610 con 0x7fe158100ce0 2023-07-14T11:21:59.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.451+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1581a00b0 con 0x7fe158100ce0 2023-07-14T11:21:59.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.452+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1581921d0 con 0x7fe158100ce0 2023-07-14T11:21:59.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.454+0000 7fe1557fa700 1 -- 172.21.15.161:0/4005506183 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe14c0108d0 con 0x7fe158100ce0 2023-07-14T11:21:59.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.455+0000 7fe1557fa700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe1400a1150 0x7fe1400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.456+0000 7fe1557fa700 1 -- 172.21.15.161:0/4005506183 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe14c0cc900 con 0x7fe158100ce0 2023-07-14T11:21:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.456+0000 7fe157fff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe1400a1150 0x7fe1400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.456+0000 7fe157fff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe1400a1150 0x7fe1400a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe14800b5c0 tx=0x7fe148005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:59.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.459+0000 7fe1557fa700 1 -- 172.21.15.161:0/4005506183 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe14c095ca0 con 0x7fe158100ce0 2023-07-14T11:21:59.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:59 smithi161 ceph-mon[95526]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-14T11:21:59.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:59 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1896324788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:59.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:59 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2304472480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:59.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:21:59 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/867150774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.658+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1580611c0 con 0x7fe158100ce0 2023-07-14T11:21:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.659+0000 7fe1557fa700 1 -- 172.21.15.161:0/4005506183 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe14c0992c0 con 0x7fe158100ce0 2023-07-14T11:21:59.662 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:21:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe1400a1150 msgr2=0x7fe1400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe1400a1150 0x7fe1400a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe14800b5c0 tx=0x7fe148005fd0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe158100ce0 msgr2=0x7fe158197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe158100ce0 0x7fe158197fb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00d950 tx=0x7fe14c00dc60 comp rx=0 tx=0).stop 2023-07-14T11:21:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 shutdown_connections 2023-07-14T11:21:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe1400a1150 0x7fe1400a3600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe158101670 0x7fe15819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580ffaf0 0x7fe158197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 --2- 172.21.15.161:0/4005506183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe158100ce0 0x7fe158197fb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.661+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 >> 172.21.15.161:0/4005506183 conn(0x7fe1580fb130 msgr2=0x7fe1581065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.662+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 shutdown_connections 2023-07-14T11:21:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.662+0000 7fe15e7ff700 1 -- 172.21.15.161:0/4005506183 wait complete. 2023-07-14T11:21:59.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:59 smithi175 ceph-mon[129081]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-14T11:21:59.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:59 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1896324788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:59.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:59 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2304472480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:59.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:21:59 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/867150774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:59.682 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:21:59.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:59 smithi182 ceph-mon[129366]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-14T11:21:59.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:59 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1896324788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:21:59.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:59 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2304472480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:21:59.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:21:59 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/867150774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:21:59.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 -- 172.21.15.161:0/2346284293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 msgr2=0x7f8798107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 --2- 172.21.15.161:0/2346284293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f8798107f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f879000b0a0 tx=0x7f879000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 -- 172.21.15.161:0/2346284293 shutdown_connections 2023-07-14T11:21:59.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 --2- 172.21.15.161:0/2346284293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f8798107f20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 --2- 172.21.15.161:0/2346284293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87980ffae0 0x7f87980ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 --2- 172.21.15.161:0/2346284293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8798100cd0 0x7f8798101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 -- 172.21.15.161:0/2346284293 >> 172.21.15.161:0/2346284293 conn(0x7f87980fb0e0 msgr2=0x7f87980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:21:59.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 -- 172.21.15.161:0/2346284293 shutdown_connections 2023-07-14T11:21:59.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.883+0000 7f879edff700 1 -- 172.21.15.161:0/2346284293 wait complete. 2023-07-14T11:21:59.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.884+0000 7f879edff700 1 Processor -- start 2023-07-14T11:21:59.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879edff700 1 -- start start 2023-07-14T11:21:59.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879edff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87980ffae0 0x7f8798195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879edff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8798100cd0 0x7f8798195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879edff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f879819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879edff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879819d4b0 con 0x7f8798101660 2023-07-14T11:21:59.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879edff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879819d5f0 con 0x7f8798100cd0 2023-07-14T11:21:59.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879edff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879819d780 con 0x7f87980ffae0 2023-07-14T11:21:59.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879d39c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f879819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879d39c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f879819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39650/0 (socket says 172.21.15.161:39650) 2023-07-14T11:21:59.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879d39c700 1 -- 172.21.15.161:0/4079446136 learned_addr learned my addr 172.21.15.161:0/4079446136 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:21:59.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f879cb9b700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87980ffae0 0x7f8798195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.885+0000 7f8797fff700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8798100cd0 0x7f8798195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f879d39c700 1 -- 172.21.15.161:0/4079446136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87980ffae0 msgr2=0x7f8798195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f879d39c700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87980ffae0 0x7f8798195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f879d39c700 1 -- 172.21.15.161:0/4079446136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8798100cd0 msgr2=0x7f8798195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:21:59.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f879d39c700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8798100cd0 0x7f8798195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:21:59.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f879d39c700 1 -- 172.21.15.161:0/4079446136 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790009d00 con 0x7f8798101660 2023-07-14T11:21:59.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f879d39c700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f879819cee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f879000d010 tx=0x7f879000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:59.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f8795ffb700 1 -- 172.21.15.161:0/4079446136 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879000e040 con 0x7f8798101660 2023-07-14T11:21:59.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f8795ffb700 1 -- 172.21.15.161:0/4079446136 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8790003e10 con 0x7f8798101660 2023-07-14T11:21:59.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f8795ffb700 1 -- 172.21.15.161:0/4079446136 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87900052a0 con 0x7f8798101660 2023-07-14T11:21:59.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.886+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879819da00 con 0x7f8798101660 2023-07-14T11:21:59.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.887+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879819ddd0 con 0x7f8798101660 2023-07-14T11:21:59.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.887+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8798106ff0 con 0x7f8798101660 2023-07-14T11:21:59.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.889+0000 7f8795ffb700 1 -- 172.21.15.161:0/4079446136 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f879001d480 con 0x7f8798101660 2023-07-14T11:21:59.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.891+0000 7f8795ffb700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f87880a1150 0x7f87880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:21:59.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.891+0000 7f8795ffb700 1 -- 172.21.15.161:0/4079446136 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f87900cdfb0 con 0x7f8798101660 2023-07-14T11:21:59.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.891+0000 7f879cb9b700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f87880a1150 0x7f87880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:21:59.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.891+0000 7f879cb9b700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f87880a1150 0x7f87880a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8798100b30 tx=0x7f878c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:21:59.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:21:59.894+0000 7f8795ffb700 1 -- 172.21.15.161:0/4079446136 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8790097440 con 0x7f8798101660 2023-07-14T11:22:00.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.091+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f87980611c0 con 0x7f8798101660 2023-07-14T11:22:00.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.091+0000 7f8795ffb700 1 -- 172.21.15.161:0/4079446136 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f879009aa60 con 0x7f8798101660 2023-07-14T11:22:00.094 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:22:00.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f87880a1150 msgr2=0x7f87880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f87880a1150 0x7f87880a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8798100b30 tx=0x7f878c009260 comp rx=0 tx=0).stop 2023-07-14T11:22:00.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 msgr2=0x7f879819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f879819cee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f879000d010 tx=0x7f879000bc80 comp rx=0 tx=0).stop 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 shutdown_connections 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798101660 0x7f879819cee0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f87880a1150 0x7f87880a3600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87980ffae0 0x7f8798195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.093+0000 7f879edff700 1 --2- 172.21.15.161:0/4079446136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8798100cd0 0x7f8798195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.094+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 >> 172.21.15.161:0/4079446136 conn(0x7f87980fb0e0 msgr2=0x7f8798104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.094+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 shutdown_connections 2023-07-14T11:22:00.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.094+0000 7f879edff700 1 -- 172.21.15.161:0/4079446136 wait complete. 2023-07-14T11:22:00.114 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-14T11:22:00.115 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:22:00.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 -- 172.21.15.161:0/2791844740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c101680 msgr2=0x7ff41c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 --2- 172.21.15.161:0/2791844740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c101680 0x7ff41c107f40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b0a0 tx=0x7ff41400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 -- 172.21.15.161:0/2791844740 shutdown_connections 2023-07-14T11:22:00.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 --2- 172.21.15.161:0/2791844740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c101680 0x7ff41c107f40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 --2- 172.21.15.161:0/2791844740 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff41c0ffb00 0x7ff41c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 --2- 172.21.15.161:0/2791844740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41c100cf0 0x7ff41c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 -- 172.21.15.161:0/2791844740 >> 172.21.15.161:0/2791844740 conn(0x7ff41c0fb0e0 msgr2=0x7ff41c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:00.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 -- 172.21.15.161:0/2791844740 shutdown_connections 2023-07-14T11:22:00.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.313+0000 7ff420bff700 1 -- 172.21.15.161:0/2791844740 wait complete. 2023-07-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 Processor -- start 2023-07-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 -- start start 2023-07-14T11:22:00.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41c0ffb00 0x7ff41c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c100cf0 0x7ff41c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff41c101680 0x7ff41c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0735e0 con 0x7ff41c100cf0 2023-07-14T11:22:00.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c073750 con 0x7ff41c0ffb00 2023-07-14T11:22:00.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff420bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c19f9a0 con 0x7ff41c101680 2023-07-14T11:22:00.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff419d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c100cf0 0x7ff41c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff419d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c100cf0 0x7ff41c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39664/0 (socket says 172.21.15.161:39664) 2023-07-14T11:22:00.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.314+0000 7ff419d9b700 1 -- 172.21.15.161:0/2189129385 learned_addr learned my addr 172.21.15.161:0/2189129385 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:00.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff41a59c700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41c0ffb00 0x7ff41c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff41ad9d700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff41c101680 0x7ff41c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff419d9b700 1 -- 172.21.15.161:0/2189129385 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff41c101680 msgr2=0x7ff41c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff419d9b700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff41c101680 0x7ff41c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff419d9b700 1 -- 172.21.15.161:0/2189129385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41c0ffb00 msgr2=0x7ff41c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff419d9b700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41c0ffb00 0x7ff41c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff419d9b700 1 -- 172.21.15.161:0/2189129385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414009d00 con 0x7ff41c100cf0 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff419d9b700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c100cf0 0x7ff41c072900 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff41000d950 tx=0x7ff41000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff40b7fe700 1 -- 172.21.15.161:0/2189129385 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410009940 con 0x7ff41c100cf0 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff40b7fe700 1 -- 172.21.15.161:0/2189129385 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff410010460 con 0x7ff41c100cf0 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c19fc80 con 0x7ff41c100cf0 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.315+0000 7ff40b7fe700 1 -- 172.21.15.161:0/2189129385 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41000f620 con 0x7ff41c100cf0 2023-07-14T11:22:00.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.316+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c1a01b0 con 0x7ff41c100cf0 2023-07-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.317+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c106f90 con 0x7ff41c100cf0 2023-07-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.319+0000 7ff40b7fe700 1 -- 172.21.15.161:0/2189129385 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff4100108d0 con 0x7ff41c100cf0 2023-07-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.320+0000 7ff40b7fe700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff4040a1220 0x7ff4040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.320+0000 7ff40b7fe700 1 -- 172.21.15.161:0/2189129385 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff4100cca90 con 0x7ff41c100cf0 2023-07-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.320+0000 7ff41a59c700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff4040a1220 0x7ff4040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.321+0000 7ff41a59c700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff4040a1220 0x7ff4040a36d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff40c005950 tx=0x7ff40c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:00.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.324+0000 7ff40b7fe700 1 -- 172.21.15.161:0/2189129385 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff410095e30 con 0x7ff41c100cf0 2023-07-14T11:22:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:00 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4005506183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:22:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:00 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4079446136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:22:00.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.568+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff41c02c850 con 0x7ff41c100cf0 2023-07-14T11:22:00.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.568+0000 7ff40b7fe700 1 -- 172.21.15.161:0/2189129385 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7ff410016090 con 0x7ff41c100cf0 2023-07-14T11:22:00.571 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:00.572 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:22:00.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff4040a1220 msgr2=0x7ff4040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff4040a1220 0x7ff4040a36d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff40c005950 tx=0x7ff40c0058e0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c100cf0 msgr2=0x7ff41c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c100cf0 0x7ff41c072900 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff41000d950 tx=0x7ff41000dc60 comp rx=0 tx=0).stop 2023-07-14T11:22:00.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 shutdown_connections 2023-07-14T11:22:00.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7ff4040a1220 0x7ff4040a36d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff41c101680 0x7ff41c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41c0ffb00 0x7ff41c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 --2- 172.21.15.161:0/2189129385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff41c100cf0 0x7ff41c072900 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 >> 172.21.15.161:0/2189129385 conn(0x7ff41c0fb0e0 msgr2=0x7ff41c1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:00.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 shutdown_connections 2023-07-14T11:22:00.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.571+0000 7ff420bff700 1 -- 172.21.15.161:0/2189129385 wait complete. 2023-07-14T11:22:00.576 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:22:00.592 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-14T11:22:00.593 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-14T11:22:00.593 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:22:00.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:00 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4005506183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:22:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:00 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4079446136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:22:00.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:00 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4005506183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:22:00.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:00 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4079446136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:22:00.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 -- 172.21.15.161:0/1827045354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d60100cf0 msgr2=0x7f7d60101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 --2- 172.21.15.161:0/1827045354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d60100cf0 0x7f7d60101140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009990 tx=0x7f7d54009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 -- 172.21.15.161:0/1827045354 shutdown_connections 2023-07-14T11:22:00.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 --2- 172.21.15.161:0/1827045354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d60101680 0x7f7d60107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 --2- 172.21.15.161:0/1827045354 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d600ffb00 0x7f7d600fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 --2- 172.21.15.161:0/1827045354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d60100cf0 0x7f7d60101140 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 -- 172.21.15.161:0/1827045354 >> 172.21.15.161:0/1827045354 conn(0x7f7d600fb0e0 msgr2=0x7f7d600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:00.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 -- 172.21.15.161:0/1827045354 shutdown_connections 2023-07-14T11:22:00.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.791+0000 7f7d65dff700 1 -- 172.21.15.161:0/1827045354 wait complete. 2023-07-14T11:22:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.792+0000 7f7d65dff700 1 Processor -- start 2023-07-14T11:22:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d65dff700 1 -- start start 2023-07-14T11:22:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d65dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600ffb00 0x7f7d600742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d65dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d60100cf0 0x7f7d60072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d65dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d60101680 0x7f7d60072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d65dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600735e0 con 0x7f7d600ffb00 2023-07-14T11:22:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d65dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60073750 con 0x7f7d60100cf0 2023-07-14T11:22:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d65dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6019f860 con 0x7f7d60101680 2023-07-14T11:22:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.793+0000 7f7d5f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600ffb00 0x7f7d600742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600ffb00 0x7f7d600742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39672/0 (socket says 172.21.15.161:39672) 2023-07-14T11:22:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 -- 172.21.15.161:0/134110350 learned_addr learned my addr 172.21.15.161:0/134110350 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5effd700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d60100cf0 0x7f7d60072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5ffff700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d60101680 0x7f7d60072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 -- 172.21.15.161:0/134110350 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d60101680 msgr2=0x7f7d60072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d60101680 0x7f7d60072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 -- 172.21.15.161:0/134110350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d60100cf0 msgr2=0x7f7d60072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d60100cf0 0x7f7d60072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 -- 172.21.15.161:0/134110350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540095f0 con 0x7f7d600ffb00 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5f7fe700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600ffb00 0x7f7d600742c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e970 tx=0x7f7d5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5cff9700 1 -- 172.21.15.161:0/134110350 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50009790 con 0x7f7d600ffb00 2023-07-14T11:22:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5cff9700 1 -- 172.21.15.161:0/134110350 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d5000c7e0 con 0x7f7d600ffb00 2023-07-14T11:22:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d5cff9700 1 -- 172.21.15.161:0/134110350 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50003c70 con 0x7f7d600ffb00 2023-07-14T11:22:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.794+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6019fb40 con 0x7f7d600ffb00 2023-07-14T11:22:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.795+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d601a0000 con 0x7f7d600ffb00 2023-07-14T11:22:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.796+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d60106ff0 con 0x7f7d600ffb00 2023-07-14T11:22:00.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.798+0000 7f7d5cff9700 1 -- 172.21.15.161:0/134110350 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7d5000cc50 con 0x7f7d600ffb00 2023-07-14T11:22:00.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.799+0000 7f7d5cff9700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d480a1220 0x7f7d480a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:00.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.799+0000 7f7d5cff9700 1 -- 172.21.15.161:0/134110350 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7d50014070 con 0x7f7d600ffb00 2023-07-14T11:22:00.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.799+0000 7f7d5effd700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d480a1220 0x7f7d480a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:00.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.800+0000 7f7d5effd700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d480a1220 0x7f7d480a36d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400b2e0 tx=0x7f7d54005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:00.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:00.803+0000 7f7d5cff9700 1 -- 172.21.15.161:0/134110350 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7d50096170 con 0x7f7d600ffb00 2023-07-14T11:22:01.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.002+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d60060950 con 0x7f7d480a1220 2023-07-14T11:22:01.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.015+0000 7f7d5cff9700 1 -- 172.21.15.161:0/134110350 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426827 (secure 0 0 0) 0x7f7d60060950 con 0x7f7d480a1220 2023-07-14T11:22:01.023 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:01.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d480a1220 msgr2=0x7f7d480a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d480a1220 0x7f7d480a36d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400b2e0 tx=0x7f7d54005fb0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600ffb00 msgr2=0x7f7d600742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600ffb00 0x7f7d600742c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e970 tx=0x7f7d5000ec80 comp rx=0 tx=0).stop 2023-07-14T11:22:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 shutdown_connections 2023-07-14T11:22:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600ffb00 0x7f7d600742c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f7d480a1220 0x7f7d480a36d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7d60101680 0x7f7d60072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 --2- 172.21.15.161:0/134110350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d60100cf0 0x7f7d60072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 >> 172.21.15.161:0/134110350 conn(0x7f7d600fb0e0 msgr2=0x7f7d60104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 shutdown_connections 2023-07-14T11:22:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.021+0000 7f7d65dff700 1 -- 172.21.15.161:0/134110350 wait complete. 2023-07-14T11:22:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:22:01.042 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2023-07-14T11:22:00.132430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657042,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":795,"num_read_kb":519,"num_write":343,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":779,"ondisk_log_size":779,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335648,"kb_used_data":12224,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606176,"statfs":{"total":1151940427776,"available":1151596724224,"internally_reserved":0,"allocated":12517376,"data_stored":8056329,"data_compressed":390111,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":41,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132890+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:21:44.132890+0000","last_peered":"2023-07-14T11:21:44.132890+0000","last_clean":"2023-07-14T11:21:44.132890+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:21:44.132890+0000","last_undegraded":"2023-07-14T11:21:44.132890+0000","last_fullsized":"2023-07-14T11:21:44.132890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:45:53.898814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131698+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:21:44.131698+0000","last_peered":"2023-07-14T11:21:44.131698+0000","last_clean":"2023-07-14T11:21:44.131698+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:21:44.131698+0000","last_undegraded":"2023-07-14T11:21:44.131698+0000","last_fullsized":"2023-07-14T11:21:44.131698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:27:43.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120443+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:21:44.120443+0000","last_peered":"2023-07-14T11:21:44.120443+0000","last_clean":"2023-07-14T11:21:44.120443+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:21:44.120443+0000","last_undegraded":"2023-07-14T11:21:44.120443+0000","last_fullsized":"2023-07-14T11:21:44.120443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:30:20.721165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133183+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:21:44.133183+0000","last_peered":"2023-07-14T11:21:44.133183+0000","last_clean":"2023-07-14T11:21:44.133183+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:21:44.133183+0000","last_undegraded":"2023-07-14T11:21:44.133183+0000","last_fullsized":"2023-07-14T11:21:44.133183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:05:47.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.124390+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:21:44.124390+0000","last_peered":"2023-07-14T11:21:44.124390+0000","last_clean":"2023-07-14T11:21:44.124390+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:21:44.124390+0000","last_undegraded":"2023-07-14T11:21:44.124390+0000","last_fullsized":"2023-07-14T11:21:44.124390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:03.043850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133900+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:21:44.133900+0000","last_peered":"2023-07-14T11:21:44.133900+0000","last_clean":"2023-07-14T11:21:44.133900+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:21:44.133900+0000","last_undegraded":"2023-07-14T11:21:44.133900+0000","last_fullsized":"2023-07-14T11:21:44.133900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:02.922415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128281+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:21:44.128281+0000","last_peered":"2023-07-14T11:21:44.128281+0000","last_clean":"2023-07-14T11:21:44.128281+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:21:44.128281+0000","last_undegraded":"2023-07-14T11:21:44.128281+0000","last_fullsized":"2023-07-14T11:21:44.128281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:20:08.848005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132015+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:21:44.132015+0000","last_peered":"2023-07-14T11:21:44.132015+0000","last_clean":"2023-07-14T11:21:44.132015+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:21:44.132015+0000","last_undegraded":"2023-07-14T11:21:44.132015+0000","last_fullsized":"2023-07-14T11:21:44.132015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:54.048999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129256+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:21:44.129256+0000","last_peered":"2023-07-14T11:21:44.129256+0000","last_clean":"2023-07-14T11:21:44.129256+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:21:44.129256+0000","last_undegraded":"2023-07-14T11:21:44.129256+0000","last_fullsized":"2023-07-14T11:21:44.129256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:25.896648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131299+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:21:44.131299+0000","last_peered":"2023-07-14T11:21:44.131299+0000","last_clean":"2023-07-14T11:21:44.131299+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:21:44.131299+0000","last_undegraded":"2023-07-14T11:21:44.131299+0000","last_fullsized":"2023-07-14T11:21:44.131299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:00.377250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.126348+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:21:44.126348+0000","last_peered":"2023-07-14T11:21:44.126348+0000","last_clean":"2023-07-14T11:21:44.126348+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:21:44.126348+0000","last_undegraded":"2023-07-14T11:21:44.126348+0000","last_fullsized":"2023-07-14T11:21:44.126348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:51.642943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133671+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:21:44.133671+0000","last_peered":"2023-07-14T11:21:44.133671+0000","last_clean":"2023-07-14T11:21:44.133671+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:21:44.133671+0000","last_undegraded":"2023-07-14T11:21:44.133671+0000","last_fullsized":"2023-07-14T11:21:44.133671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:40:35.925610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130954+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:21:44.130954+0000","last_peered":"2023-07-14T11:21:44.130954+0000","last_clean":"2023-07-14T11:21:44.130954+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:21:44.130954+0000","last_undegraded":"2023-07-14T11:21:44.130954+0000","last_fullsized":"2023-07-14T11:21:44.130954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:46:47.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129701+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:21:44.129701+0000","last_peered":"2023-07-14T11:21:44.129701+0000","last_clean":"2023-07-14T11:21:44.129701+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:21:44.129701+0000","last_undegraded":"2023-07-14T11:21:44.129701+0000","last_fullsized":"2023-07-14T11:21:44.129701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:40:58.143556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134018+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:21:44.134018+0000","last_peered":"2023-07-14T11:21:44.134018+0000","last_clean":"2023-07-14T11:21:44.134018+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:21:44.134018+0000","last_undegraded":"2023-07-14T11:21:44.134018+0000","last_fullsized":"2023-07-14T11:21:44.134018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:24.775441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.127674+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:21:44.127674+0000","last_peered":"2023-07-14T11:21:44.127674+0000","last_clean":"2023-07-14T11:21:44.127674+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:21:44.127674+0000","last_undegraded":"2023-07-14T11:21:44.127674+0000","last_fullsized":"2023-07-14T11:21:44.127674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:49.835385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132947+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:21:44.132947+0000","last_peered":"2023-07-14T11:21:44.132947+0000","last_clean":"2023-07-14T11:21:44.132947+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:21:44.132947+0000","last_undegraded":"2023-07-14T11:21:44.132947+0000","last_fullsized":"2023-07-14T11:21:44.132947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:58:04.718574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.114499+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:21:44.114499+0000","last_peered":"2023-07-14T11:21:44.114499+0000","last_clean":"2023-07-14T11:21:44.114499+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:21:44.114499+0000","last_undegraded":"2023-07-14T11:21:44.114499+0000","last_fullsized":"2023-07-14T11:21:44.114499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:45.295320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129091+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:21:44.129091+0000","last_peered":"2023-07-14T11:21:44.129091+0000","last_clean":"2023-07-14T11:21:44.129091+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:21:44.129091+0000","last_undegraded":"2023-07-14T11:21:44.129091+0000","last_fullsized":"2023-07-14T11:21:44.129091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:38:25.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761984+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:21:16.761984+0000","last_peered":"2023-07-14T11:21:16.761984+0000","last_clean":"2023-07-14T11:21:16.761984+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:21:16.761984+0000","last_undegraded":"2023-07-14T11:21:16.761984+0000","last_fullsized":"2023-07-14T11:21:16.761984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:32.149825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.757056+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:21:16.757056+0000","last_peered":"2023-07-14T11:21:16.757056+0000","last_clean":"2023-07-14T11:21:16.757056+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:21:16.757056+0000","last_undegraded":"2023-07-14T11:21:16.757056+0000","last_fullsized":"2023-07-14T11:21:16.757056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:27:34.767964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128034+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:21:44.128034+0000","last_peered":"2023-07-14T11:21:44.128034+0000","last_clean":"2023-07-14T11:21:44.128034+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:21:44.128034+0000","last_undegraded":"2023-07-14T11:21:44.128034+0000","last_fullsized":"2023-07-14T11:21:44.128034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:23:39.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782856+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:21:16.782856+0000","last_peered":"2023-07-14T11:21:16.782856+0000","last_clean":"2023-07-14T11:21:16.782856+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:21:16.782856+0000","last_undegraded":"2023-07-14T11:21:16.782856+0000","last_fullsized":"2023-07-14T11:21:16.782856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:47.727483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761802+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:21:16.761802+0000","last_peered":"2023-07-14T11:21:16.761802+0000","last_clean":"2023-07-14T11:21:16.761802+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:21:16.761802+0000","last_undegraded":"2023-07-14T11:21:16.761802+0000","last_fullsized":"2023-07-14T11:21:16.761802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:29:59.179624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130426+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:21:44.130426+0000","last_peered":"2023-07-14T11:21:44.130426+0000","last_clean":"2023-07-14T11:21:44.130426+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:21:44.130426+0000","last_undegraded":"2023-07-14T11:21:44.130426+0000","last_fullsized":"2023-07-14T11:21:44.130426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:18.435015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782915+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:21:16.782915+0000","last_peered":"2023-07-14T11:21:16.782915+0000","last_clean":"2023-07-14T11:21:16.782915+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:21:16.782915+0000","last_undegraded":"2023-07-14T11:21:16.782915+0000","last_fullsized":"2023-07-14T11:21:16.782915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:27.267522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760533+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:21:16.760533+0000","last_peered":"2023-07-14T11:21:16.760533+0000","last_clean":"2023-07-14T11:21:16.760533+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:21:16.760533+0000","last_undegraded":"2023-07-14T11:21:16.760533+0000","last_fullsized":"2023-07-14T11:21:16.760533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:28.556454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130958+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:21:44.130958+0000","last_peered":"2023-07-14T11:21:44.130958+0000","last_clean":"2023-07-14T11:21:44.130958+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:21:44.130958+0000","last_undegraded":"2023-07-14T11:21:44.130958+0000","last_fullsized":"2023-07-14T11:21:44.130958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:23.714644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782878+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:21:16.782878+0000","last_peered":"2023-07-14T11:21:16.782878+0000","last_clean":"2023-07-14T11:21:16.782878+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:21:16.782878+0000","last_undegraded":"2023-07-14T11:21:16.782878+0000","last_fullsized":"2023-07-14T11:21:16.782878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:14.385740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759437+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:21:16.759437+0000","last_peered":"2023-07-14T11:21:16.759437+0000","last_clean":"2023-07-14T11:21:16.759437+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:21:16.759437+0000","last_undegraded":"2023-07-14T11:21:16.759437+0000","last_fullsized":"2023-07-14T11:21:16.759437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:24:07.117370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120539+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:21:44.120539+0000","last_peered":"2023-07-14T11:21:44.120539+0000","last_clean":"2023-07-14T11:21:44.120539+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:21:44.120539+0000","last_undegraded":"2023-07-14T11:21:44.120539+0000","last_fullsized":"2023-07-14T11:21:44.120539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:01:41.642530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761645+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:21:16.761645+0000","last_peered":"2023-07-14T11:21:16.761645+0000","last_clean":"2023-07-14T11:21:16.761645+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:21:16.761645+0000","last_undegraded":"2023-07-14T11:21:16.761645+0000","last_fullsized":"2023-07-14T11:21:16.761645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:22:33.821715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":143,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:56.841418+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:21:56.841418+0000","last_peered":"2023-07-14T11:21:56.841418+0000","last_clean":"2023-07-14T11:21:56.841418+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:21:56.841418+0000","last_undegraded":"2023-07-14T11:21:56.841418+0000","last_fullsized":"2023-07-14T11:21:56.841418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:55:16.898539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.117612+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:21:44.117612+0000","last_peered":"2023-07-14T11:21:44.117612+0000","last_clean":"2023-07-14T11:21:44.117612+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:21:44.117612+0000","last_undegraded":"2023-07-14T11:21:44.117612+0000","last_fullsized":"2023-07-14T11:21:44.117612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:49:45.959442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.762138+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:21:16.762138+0000","last_peered":"2023-07-14T11:21:16.762138+0000","last_clean":"2023-07-14T11:21:16.762138+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:21:16.762138+0000","last_undegraded":"2023-07-14T11:21:16.762138+0000","last_fullsized":"2023-07-14T11:21:16.762138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:29:21.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.754843+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:21:16.754843+0000","last_peered":"2023-07-14T11:21:16.754843+0000","last_clean":"2023-07-14T11:21:16.754843+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:21:16.754843+0000","last_undegraded":"2023-07-14T11:21:16.754843+0000","last_fullsized":"2023-07-14T11:21:16.754843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:42.195752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.126818+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:21:44.126818+0000","last_peered":"2023-07-14T11:21:44.126818+0000","last_clean":"2023-07-14T11:21:44.126818+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:21:44.126818+0000","last_undegraded":"2023-07-14T11:21:44.126818+0000","last_fullsized":"2023-07-14T11:21:44.126818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:15:54.989039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.758730+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:21:16.758730+0000","last_peered":"2023-07-14T11:21:16.758730+0000","last_clean":"2023-07-14T11:21:16.758730+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:21:16.758730+0000","last_undegraded":"2023-07-14T11:21:16.758730+0000","last_fullsized":"2023-07-14T11:21:16.758730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:13:50.965885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780547+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:21:16.780547+0000","last_peered":"2023-07-14T11:21:16.780547+0000","last_clean":"2023-07-14T11:21:16.780547+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:21:16.780547+0000","last_undegraded":"2023-07-14T11:21:16.780547+0000","last_fullsized":"2023-07-14T11:21:16.780547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:20:20.452801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134597+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:21:44.134597+0000","last_peered":"2023-07-14T11:21:44.134597+0000","last_clean":"2023-07-14T11:21:44.134597+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:21:44.134597+0000","last_undegraded":"2023-07-14T11:21:44.134597+0000","last_fullsized":"2023-07-14T11:21:44.134597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:26:38.989878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761717+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:21:16.761717+0000","last_peered":"2023-07-14T11:21:16.761717+0000","last_clean":"2023-07-14T11:21:16.761717+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:21:16.761717+0000","last_undegraded":"2023-07-14T11:21:16.761717+0000","last_fullsized":"2023-07-14T11:21:16.761717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:06:57.319545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.779919+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:21:16.779919+0000","last_peered":"2023-07-14T11:21:16.779919+0000","last_clean":"2023-07-14T11:21:16.779919+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:21:16.779919+0000","last_undegraded":"2023-07-14T11:21:16.779919+0000","last_fullsized":"2023-07-14T11:21:16.779919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:45.971471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132864+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:21:44.132864+0000","last_peered":"2023-07-14T11:21:44.132864+0000","last_clean":"2023-07-14T11:21:44.132864+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:21:44.132864+0000","last_undegraded":"2023-07-14T11:21:44.132864+0000","last_fullsized":"2023-07-14T11:21:44.132864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:22.629848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760884+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:21:16.760884+0000","last_peered":"2023-07-14T11:21:16.760884+0000","last_clean":"2023-07-14T11:21:16.760884+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:21:16.760884+0000","last_undegraded":"2023-07-14T11:21:16.760884+0000","last_fullsized":"2023-07-14T11:21:16.760884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:07.800244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:55.839185+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:21:55.839185+0000","last_peered":"2023-07-14T11:21:55.839185+0000","last_clean":"2023-07-14T11:21:55.839185+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:21:55.839185+0000","last_undegraded":"2023-07-14T11:21:55.839185+0000","last_fullsized":"2023-07-14T11:21:55.839185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:15:34.152726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129759+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:21:44.129759+0000","last_peered":"2023-07-14T11:21:44.129759+0000","last_clean":"2023-07-14T11:21:44.129759+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:21:44.129759+0000","last_undegraded":"2023-07-14T11:21:44.129759+0000","last_fullsized":"2023-07-14T11:21:44.129759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:56.754661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756218+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:21:16.756218+0000","last_peered":"2023-07-14T11:21:16.756218+0000","last_clean":"2023-07-14T11:21:16.756218+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:21:16.756218+0000","last_undegraded":"2023-07-14T11:21:16.756218+0000","last_fullsized":"2023-07-14T11:21:16.756218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:28.728921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827514+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:21:17.827514+0000","last_peered":"2023-07-14T11:21:17.827514+0000","last_clean":"2023-07-14T11:21:17.827514+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:21:17.827514+0000","last_undegraded":"2023-07-14T11:21:17.827514+0000","last_fullsized":"2023-07-14T11:21:17.827514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:16:06.201581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135307+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:21:44.135307+0000","last_peered":"2023-07-14T11:21:44.135307+0000","last_clean":"2023-07-14T11:21:44.135307+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:21:44.135307+0000","last_undegraded":"2023-07-14T11:21:44.135307+0000","last_fullsized":"2023-07-14T11:21:44.135307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:08.534228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783067+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:21:16.783067+0000","last_peered":"2023-07-14T11:21:16.783067+0000","last_clean":"2023-07-14T11:21:16.783067+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:21:16.783067+0000","last_undegraded":"2023-07-14T11:21:16.783067+0000","last_fullsized":"2023-07-14T11:21:16.783067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:37.851033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781109+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:21:16.781109+0000","last_peered":"2023-07-14T11:21:16.781109+0000","last_clean":"2023-07-14T11:21:16.781109+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:21:16.781109+0000","last_undegraded":"2023-07-14T11:21:16.781109+0000","last_fullsized":"2023-07-14T11:21:16.781109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:17:24.738586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133580+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:21:44.133580+0000","last_peered":"2023-07-14T11:21:44.133580+0000","last_clean":"2023-07-14T11:21:44.133580+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:21:44.133580+0000","last_undegraded":"2023-07-14T11:21:44.133580+0000","last_fullsized":"2023-07-14T11:21:44.133580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:08:23.851839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756241+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:21:16.756241+0000","last_peered":"2023-07-14T11:21:16.756241+0000","last_clean":"2023-07-14T11:21:16.756241+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:21:16.756241+0000","last_undegraded":"2023-07-14T11:21:16.756241+0000","last_fullsized":"2023-07-14T11:21:16.756241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:29:43.064170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":284,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:56.841699+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:21:56.841699+0000","last_peered":"2023-07-14T11:21:56.841699+0000","last_clean":"2023-07-14T11:21:56.841699+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:21:56.841699+0000","last_undegraded":"2023-07-14T11:21:56.841699+0000","last_fullsized":"2023-07-14T11:21:56.841699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:27:38.535607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":219,"num_read_kb":171,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.122999+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:21:44.122999+0000","last_peered":"2023-07-14T11:21:44.122999+0000","last_clean":"2023-07-14T11:21:44.122999+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:21:44.122999+0000","last_undegraded":"2023-07-14T11:21:44.122999+0000","last_fullsized":"2023-07-14T11:21:44.122999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:09:57.788775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781384+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:21:16.781384+0000","last_peered":"2023-07-14T11:21:16.781384+0000","last_clean":"2023-07-14T11:21:16.781384+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:21:16.781384+0000","last_undegraded":"2023-07-14T11:21:16.781384+0000","last_fullsized":"2023-07-14T11:21:16.781384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:12.644525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781070+0000","last_change":"2023-07-14T11:21:16.780317+0000","last_active":"2023-07-14T11:21:16.781070+0000","last_peered":"2023-07-14T11:21:16.781070+0000","last_clean":"2023-07-14T11:21:16.781070+0000","last_became_active":"2023-07-14T11:21:16.763144+0000","last_became_peered":"2023-07-14T11:21:16.763144+0000","last_unstale":"2023-07-14T11:21:16.781070+0000","last_undegraded":"2023-07-14T11:21:16.781070+0000","last_fullsized":"2023-07-14T11:21:16.781070+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:59.524343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132499+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:21:44.132499+0000","last_peered":"2023-07-14T11:21:44.132499+0000","last_clean":"2023-07-14T11:21:44.132499+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:21:44.132499+0000","last_undegraded":"2023-07-14T11:21:44.132499+0000","last_fullsized":"2023-07-14T11:21:44.132499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:16.990224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.758571+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:21:16.758571+0000","last_peered":"2023-07-14T11:21:16.758571+0000","last_clean":"2023-07-14T11:21:16.758571+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:21:16.758571+0000","last_undegraded":"2023-07-14T11:21:16.758571+0000","last_fullsized":"2023-07-14T11:21:16.758571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:30:05.342050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780126+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:21:16.780126+0000","last_peered":"2023-07-14T11:21:16.780126+0000","last_clean":"2023-07-14T11:21:16.780126+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:21:16.780126+0000","last_undegraded":"2023-07-14T11:21:16.780126+0000","last_fullsized":"2023-07-14T11:21:16.780126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:59:52.431744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130683+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:21:44.130683+0000","last_peered":"2023-07-14T11:21:44.130683+0000","last_clean":"2023-07-14T11:21:44.130683+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:21:44.130683+0000","last_undegraded":"2023-07-14T11:21:44.130683+0000","last_fullsized":"2023-07-14T11:21:44.130683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:10:00.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756683+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:21:16.756683+0000","last_peered":"2023-07-14T11:21:16.756683+0000","last_clean":"2023-07-14T11:21:16.756683+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:21:16.756683+0000","last_undegraded":"2023-07-14T11:21:16.756683+0000","last_fullsized":"2023-07-14T11:21:16.756683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:40.564962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:55.839289+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:21:55.839289+0000","last_peered":"2023-07-14T11:21:55.839289+0000","last_clean":"2023-07-14T11:21:55.839289+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:21:55.839289+0000","last_undegraded":"2023-07-14T11:21:55.839289+0000","last_fullsized":"2023-07-14T11:21:55.839289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:26.857871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.123757+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:21:44.123757+0000","last_peered":"2023-07-14T11:21:44.123757+0000","last_clean":"2023-07-14T11:21:44.123757+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:21:44.123757+0000","last_undegraded":"2023-07-14T11:21:44.123757+0000","last_fullsized":"2023-07-14T11:21:44.123757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:37.070346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780756+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:21:16.780756+0000","last_peered":"2023-07-14T11:21:16.780756+0000","last_clean":"2023-07-14T11:21:16.780756+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:21:16.780756+0000","last_undegraded":"2023-07-14T11:21:16.780756+0000","last_fullsized":"2023-07-14T11:21:16.780756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:33:44.207912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.774327+0000","last_change":"2023-07-14T11:21:16.773940+0000","last_active":"2023-07-14T11:21:16.774327+0000","last_peered":"2023-07-14T11:21:16.774327+0000","last_clean":"2023-07-14T11:21:16.774327+0000","last_became_active":"2023-07-14T11:21:16.762658+0000","last_became_peered":"2023-07-14T11:21:16.762658+0000","last_unstale":"2023-07-14T11:21:16.774327+0000","last_undegraded":"2023-07-14T11:21:16.774327+0000","last_fullsized":"2023-07-14T11:21:16.774327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:05.595583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120231+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:21:44.120231+0000","last_peered":"2023-07-14T11:21:44.120231+0000","last_clean":"2023-07-14T11:21:44.120231+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:21:44.120231+0000","last_undegraded":"2023-07-14T11:21:44.120231+0000","last_fullsized":"2023-07-14T11:21:44.120231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:59:04.043553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780006+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:21:16.780006+0000","last_peered":"2023-07-14T11:21:16.780006+0000","last_clean":"2023-07-14T11:21:16.780006+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:21:16.780006+0000","last_undegraded":"2023-07-14T11:21:16.780006+0000","last_fullsized":"2023-07-14T11:21:16.780006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:19.174537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760420+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:21:16.760420+0000","last_peered":"2023-07-14T11:21:16.760420+0000","last_clean":"2023-07-14T11:21:16.760420+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:21:16.760420+0000","last_undegraded":"2023-07-14T11:21:16.760420+0000","last_fullsized":"2023-07-14T11:21:16.760420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:13.662336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129915+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:21:44.129915+0000","last_peered":"2023-07-14T11:21:44.129915+0000","last_clean":"2023-07-14T11:21:44.129915+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:21:44.129915+0000","last_undegraded":"2023-07-14T11:21:44.129915+0000","last_fullsized":"2023-07-14T11:21:44.129915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:12:29.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759722+0000","last_change":"2023-07-14T11:20:15.731601+0000","last_active":"2023-07-14T11:21:16.759722+0000","last_peered":"2023-07-14T11:21:16.759722+0000","last_clean":"2023-07-14T11:21:16.759722+0000","last_became_active":"2023-07-14T11:20:15.731121+0000","last_became_peered":"2023-07-14T11:20:15.731121+0000","last_unstale":"2023-07-14T11:21:16.759722+0000","last_undegraded":"2023-07-14T11:21:16.759722+0000","last_fullsized":"2023-07-14T11:21:16.759722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:15:52.609093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756735+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:21:16.756735+0000","last_peered":"2023-07-14T11:21:16.756735+0000","last_clean":"2023-07-14T11:21:16.756735+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:21:16.756735+0000","last_undegraded":"2023-07-14T11:21:16.756735+0000","last_fullsized":"2023-07-14T11:21:16.756735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:57.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133807+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:21:44.133807+0000","last_peered":"2023-07-14T11:21:44.133807+0000","last_clean":"2023-07-14T11:21:44.133807+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:21:44.133807+0000","last_undegraded":"2023-07-14T11:21:44.133807+0000","last_fullsized":"2023-07-14T11:21:44.133807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:15:22.547786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781033+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:21:16.781033+0000","last_peered":"2023-07-14T11:21:16.781033+0000","last_clean":"2023-07-14T11:21:16.781033+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:21:16.781033+0000","last_undegraded":"2023-07-14T11:21:16.781033+0000","last_fullsized":"2023-07-14T11:21:16.781033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:43:54.535268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827424+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:21:17.827424+0000","last_peered":"2023-07-14T11:21:17.827424+0000","last_clean":"2023-07-14T11:21:17.827424+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:21:17.827424+0000","last_undegraded":"2023-07-14T11:21:17.827424+0000","last_fullsized":"2023-07-14T11:21:17.827424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:34:07.328659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128632+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:21:44.128632+0000","last_peered":"2023-07-14T11:21:44.128632+0000","last_clean":"2023-07-14T11:21:44.128632+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:21:44.128632+0000","last_undegraded":"2023-07-14T11:21:44.128632+0000","last_fullsized":"2023-07-14T11:21:44.128632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:58:43.563311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780882+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:21:16.780882+0000","last_peered":"2023-07-14T11:21:16.780882+0000","last_clean":"2023-07-14T11:21:16.780882+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:21:16.780882+0000","last_undegraded":"2023-07-14T11:21:16.780882+0000","last_fullsized":"2023-07-14T11:21:16.780882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:56:02.843006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.121291+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:21:44.121291+0000","last_peered":"2023-07-14T11:21:44.121291+0000","last_clean":"2023-07-14T11:21:44.121291+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:21:44.121291+0000","last_undegraded":"2023-07-14T11:21:44.121291+0000","last_fullsized":"2023-07-14T11:21:44.121291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:22:39.851277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:57.841537+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:21:57.841537+0000","last_peered":"2023-07-14T11:21:57.841537+0000","last_clean":"2023-07-14T11:21:57.841537+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:21:57.841537+0000","last_undegraded":"2023-07-14T11:21:57.841537+0000","last_fullsized":"2023-07-14T11:21:57.841537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:39:23.337780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.828305+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:21:17.828305+0000","last_peered":"2023-07-14T11:21:17.828305+0000","last_clean":"2023-07-14T11:21:17.828305+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:21:17.828305+0000","last_undegraded":"2023-07-14T11:21:17.828305+0000","last_fullsized":"2023-07-14T11:21:17.828305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:47:27.405242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135278+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:21:44.135278+0000","last_peered":"2023-07-14T11:21:44.135278+0000","last_clean":"2023-07-14T11:21:44.135278+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:21:44.135278+0000","last_undegraded":"2023-07-14T11:21:44.135278+0000","last_fullsized":"2023-07-14T11:21:44.135278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:51.335219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783295+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:21:16.783295+0000","last_peered":"2023-07-14T11:21:16.783295+0000","last_clean":"2023-07-14T11:21:16.783295+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:21:16.783295+0000","last_undegraded":"2023-07-14T11:21:16.783295+0000","last_fullsized":"2023-07-14T11:21:16.783295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:42:44.347182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827912+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:21:17.827912+0000","last_peered":"2023-07-14T11:21:17.827912+0000","last_clean":"2023-07-14T11:21:17.827912+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:21:17.827912+0000","last_undegraded":"2023-07-14T11:21:17.827912+0000","last_fullsized":"2023-07-14T11:21:17.827912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:53.938118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781156+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:21:16.781156+0000","last_peered":"2023-07-14T11:21:16.781156+0000","last_clean":"2023-07-14T11:21:16.781156+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:21:16.781156+0000","last_undegraded":"2023-07-14T11:21:16.781156+0000","last_fullsized":"2023-07-14T11:21:16.781156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:33:14.547044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129505+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:21:44.129505+0000","last_peered":"2023-07-14T11:21:44.129505+0000","last_clean":"2023-07-14T11:21:44.129505+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:21:44.129505+0000","last_undegraded":"2023-07-14T11:21:44.129505+0000","last_fullsized":"2023-07-14T11:21:44.129505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:53:19.310433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827364+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:21:17.827364+0000","last_peered":"2023-07-14T11:21:17.827364+0000","last_clean":"2023-07-14T11:21:17.827364+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:21:17.827364+0000","last_undegraded":"2023-07-14T11:21:17.827364+0000","last_fullsized":"2023-07-14T11:21:17.827364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:25:55.640663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759127+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:21:16.759127+0000","last_peered":"2023-07-14T11:21:16.759127+0000","last_clean":"2023-07-14T11:21:16.759127+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:21:16.759127+0000","last_undegraded":"2023-07-14T11:21:16.759127+0000","last_fullsized":"2023-07-14T11:21:16.759127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:28.280537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760321+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:21:16.760321+0000","last_peered":"2023-07-14T11:21:16.760321+0000","last_clean":"2023-07-14T11:21:16.760321+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:21:16.760321+0000","last_undegraded":"2023-07-14T11:21:16.760321+0000","last_fullsized":"2023-07-14T11:21:16.760321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:46.760759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133293+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:21:44.133293+0000","last_peered":"2023-07-14T11:21:44.133293+0000","last_clean":"2023-07-14T11:21:44.133293+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:21:44.133293+0000","last_undegraded":"2023-07-14T11:21:44.133293+0000","last_fullsized":"2023-07-14T11:21:44.133293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:20:33.855972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761100+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:21:16.761100+0000","last_peered":"2023-07-14T11:21:16.761100+0000","last_clean":"2023-07-14T11:21:16.761100+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:21:16.761100+0000","last_undegraded":"2023-07-14T11:21:16.761100+0000","last_fullsized":"2023-07-14T11:21:16.761100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:01.581528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:51.950589+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:21:51.950589+0000","last_peered":"2023-07-14T11:21:51.950589+0000","last_clean":"2023-07-14T11:21:51.950589+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:21:51.950589+0000","last_undegraded":"2023-07-14T11:21:51.950589+0000","last_fullsized":"2023-07-14T11:21:51.950589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.789040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131085+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:21:44.131085+0000","last_peered":"2023-07-14T11:21:44.131085+0000","last_clean":"2023-07-14T11:21:44.131085+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:21:44.131085+0000","last_undegraded":"2023-07-14T11:21:44.131085+0000","last_fullsized":"2023-07-14T11:21:44.131085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:10.033823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761906+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:21:16.761906+0000","last_peered":"2023-07-14T11:21:16.761906+0000","last_clean":"2023-07-14T11:21:16.761906+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:21:16.761906+0000","last_undegraded":"2023-07-14T11:21:16.761906+0000","last_fullsized":"2023-07-14T11:21:16.761906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:19:06.706597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783437+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:21:16.783437+0000","last_peered":"2023-07-14T11:21:16.783437+0000","last_clean":"2023-07-14T11:21:16.783437+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:21:16.783437+0000","last_undegraded":"2023-07-14T11:21:16.783437+0000","last_fullsized":"2023-07-14T11:21:16.783437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:04:22.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135985+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:21:44.135985+0000","last_peered":"2023-07-14T11:21:44.135985+0000","last_clean":"2023-07-14T11:21:44.135985+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:21:44.135985+0000","last_undegraded":"2023-07-14T11:21:44.135985+0000","last_fullsized":"2023-07-14T11:21:44.135985+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:50:49.239460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783236+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:21:16.783236+0000","last_peered":"2023-07-14T11:21:16.783236+0000","last_clean":"2023-07-14T11:21:16.783236+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:21:16.783236+0000","last_undegraded":"2023-07-14T11:21:16.783236+0000","last_fullsized":"2023-07-14T11:21:16.783236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:00:23.953512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:18.022769+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:21:18.022769+0000","last_peered":"2023-07-14T11:21:18.022769+0000","last_clean":"2023-07-14T11:21:18.022769+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:21:18.022769+0000","last_undegraded":"2023-07-14T11:21:18.022769+0000","last_fullsized":"2023-07-14T11:21:18.022769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:12:28.851941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.108769+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:21:44.108769+0000","last_peered":"2023-07-14T11:21:44.108769+0000","last_clean":"2023-07-14T11:21:44.108769+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:21:44.108769+0000","last_undegraded":"2023-07-14T11:21:44.108769+0000","last_fullsized":"2023-07-14T11:21:44.108769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:23:32.637906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756893+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:21:16.756893+0000","last_peered":"2023-07-14T11:21:16.756893+0000","last_clean":"2023-07-14T11:21:16.756893+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:21:16.756893+0000","last_undegraded":"2023-07-14T11:21:16.756893+0000","last_fullsized":"2023-07-14T11:21:16.756893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:52:08.577965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761060+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:21:16.761060+0000","last_peered":"2023-07-14T11:21:16.761060+0000","last_clean":"2023-07-14T11:21:16.761060+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:21:16.761060+0000","last_undegraded":"2023-07-14T11:21:16.761060+0000","last_fullsized":"2023-07-14T11:21:16.761060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:11:05.150069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132051+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:21:44.132051+0000","last_peered":"2023-07-14T11:21:44.132051+0000","last_clean":"2023-07-14T11:21:44.132051+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:21:44.132051+0000","last_undegraded":"2023-07-14T11:21:44.132051+0000","last_fullsized":"2023-07-14T11:21:44.132051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:20:37.760123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783130+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:21:16.783130+0000","last_peered":"2023-07-14T11:21:16.783130+0000","last_clean":"2023-07-14T11:21:16.783130+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:21:16.783130+0000","last_undegraded":"2023-07-14T11:21:16.783130+0000","last_fullsized":"2023-07-14T11:21:16.783130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:05:52.217578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761847+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:21:16.761847+0000","last_peered":"2023-07-14T11:21:16.761847+0000","last_clean":"2023-07-14T11:21:16.761847+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:21:16.761847+0000","last_undegraded":"2023-07-14T11:21:16.761847+0000","last_fullsized":"2023-07-14T11:21:16.761847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:24.188762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132768+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:21:44.132768+0000","last_peered":"2023-07-14T11:21:44.132768+0000","last_clean":"2023-07-14T11:21:44.132768+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:21:44.132768+0000","last_undegraded":"2023-07-14T11:21:44.132768+0000","last_fullsized":"2023-07-14T11:21:44.132768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:44.830001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783284+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:21:16.783284+0000","last_peered":"2023-07-14T11:21:16.783284+0000","last_clean":"2023-07-14T11:21:16.783284+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:21:16.783284+0000","last_undegraded":"2023-07-14T11:21:16.783284+0000","last_fullsized":"2023-07-14T11:21:16.783284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:07:04.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759626+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:21:16.759626+0000","last_peered":"2023-07-14T11:21:16.759626+0000","last_clean":"2023-07-14T11:21:16.759626+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:21:16.759626+0000","last_undegraded":"2023-07-14T11:21:16.759626+0000","last_fullsized":"2023-07-14T11:21:16.759626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:43:00.377914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134559+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:21:44.134559+0000","last_peered":"2023-07-14T11:21:44.134559+0000","last_clean":"2023-07-14T11:21:44.134559+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:21:44.134559+0000","last_undegraded":"2023-07-14T11:21:44.134559+0000","last_fullsized":"2023-07-14T11:21:44.134559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:03.031833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:18.023102+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:21:18.023102+0000","last_peered":"2023-07-14T11:21:18.023102+0000","last_clean":"2023-07-14T11:21:18.023102+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:21:18.023102+0000","last_undegraded":"2023-07-14T11:21:18.023102+0000","last_fullsized":"2023-07-14T11:21:18.023102+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:04:51.829719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.762268+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:21:16.762268+0000","last_peered":"2023-07-14T11:21:16.762268+0000","last_clean":"2023-07-14T11:21:16.762268+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:21:16.762268+0000","last_undegraded":"2023-07-14T11:21:16.762268+0000","last_fullsized":"2023-07-14T11:21:16.762268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:41:38.935430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131585+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:21:44.131585+0000","last_peered":"2023-07-14T11:21:44.131585+0000","last_clean":"2023-07-14T11:21:44.131585+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:21:44.131585+0000","last_undegraded":"2023-07-14T11:21:44.131585+0000","last_fullsized":"2023-07-14T11:21:44.131585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:00.381695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760768+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:21:16.760768+0000","last_peered":"2023-07-14T11:21:16.760768+0000","last_clean":"2023-07-14T11:21:16.760768+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:21:16.760768+0000","last_undegraded":"2023-07-14T11:21:16.760768+0000","last_fullsized":"2023-07-14T11:21:16.760768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:22:56.514930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781299+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:21:16.781299+0000","last_peered":"2023-07-14T11:21:16.781299+0000","last_clean":"2023-07-14T11:21:16.781299+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:21:16.781299+0000","last_undegraded":"2023-07-14T11:21:16.781299+0000","last_fullsized":"2023-07-14T11:21:16.781299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:39.416802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128900+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:21:44.128900+0000","last_peered":"2023-07-14T11:21:44.128900+0000","last_clean":"2023-07-14T11:21:44.128900+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:21:44.128900+0000","last_undegraded":"2023-07-14T11:21:44.128900+0000","last_fullsized":"2023-07-14T11:21:44.128900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:58.657251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.779781+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:21:16.779781+0000","last_peered":"2023-07-14T11:21:16.779781+0000","last_clean":"2023-07-14T11:21:16.779781+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:21:16.779781+0000","last_undegraded":"2023-07-14T11:21:16.779781+0000","last_fullsized":"2023-07-14T11:21:16.779781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:32:23.647113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780749+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:21:16.780749+0000","last_peered":"2023-07-14T11:21:16.780749+0000","last_clean":"2023-07-14T11:21:16.780749+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:21:16.780749+0000","last_undegraded":"2023-07-14T11:21:16.780749+0000","last_fullsized":"2023-07-14T11:21:16.780749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:31.036473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131916+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:21:44.131916+0000","last_peered":"2023-07-14T11:21:44.131916+0000","last_clean":"2023-07-14T11:21:44.131916+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:21:44.131916+0000","last_undegraded":"2023-07-14T11:21:44.131916+0000","last_fullsized":"2023-07-14T11:21:44.131916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:54.620604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.123597+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:21:44.123597+0000","last_peered":"2023-07-14T11:21:44.123597+0000","last_clean":"2023-07-14T11:21:44.123597+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:21:44.123597+0000","last_undegraded":"2023-07-14T11:21:44.123597+0000","last_fullsized":"2023-07-14T11:21:44.123597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:13:16.901921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.127676+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:21:44.127676+0000","last_peered":"2023-07-14T11:21:44.127676+0000","last_clean":"2023-07-14T11:21:44.127676+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:21:44.127676+0000","last_undegraded":"2023-07-14T11:21:44.127676+0000","last_fullsized":"2023-07-14T11:21:44.127676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:06:24.239682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130965+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:21:44.130965+0000","last_peered":"2023-07-14T11:21:44.130965+0000","last_clean":"2023-07-14T11:21:44.130965+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:21:44.130965+0000","last_undegraded":"2023-07-14T11:21:44.130965+0000","last_fullsized":"2023-07-14T11:21:44.130965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:38:17.082600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135133+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:21:44.135133+0000","last_peered":"2023-07-14T11:21:44.135133+0000","last_clean":"2023-07-14T11:21:44.135133+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:21:44.135133+0000","last_undegraded":"2023-07-14T11:21:44.135133+0000","last_fullsized":"2023-07-14T11:21:44.135133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:19.948875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132889+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:21:44.132889+0000","last_peered":"2023-07-14T11:21:44.132889+0000","last_clean":"2023-07-14T11:21:44.132889+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:21:44.132889+0000","last_undegraded":"2023-07-14T11:21:44.132889+0000","last_fullsized":"2023-07-14T11:21:44.132889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:24.946793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120491+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:21:44.120491+0000","last_peered":"2023-07-14T11:21:44.120491+0000","last_clean":"2023-07-14T11:21:44.120491+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:21:44.120491+0000","last_undegraded":"2023-07-14T11:21:44.120491+0000","last_fullsized":"2023-07-14T11:21:44.120491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:33:36.799060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.122053+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:21:44.122053+0000","last_peered":"2023-07-14T11:21:44.122053+0000","last_clean":"2023-07-14T11:21:44.122053+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:21:44.122053+0000","last_undegraded":"2023-07-14T11:21:44.122053+0000","last_fullsized":"2023-07-14T11:21:44.122053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:55.929013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134984+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:21:44.134984+0000","last_peered":"2023-07-14T11:21:44.134984+0000","last_clean":"2023-07-14T11:21:44.134984+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:21:44.134984+0000","last_undegraded":"2023-07-14T11:21:44.134984+0000","last_fullsized":"2023-07-14T11:21:44.134984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:58.128316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.118051+0000","last_change":"2023-07-14T11:20:15.765762+0000","last_active":"2023-07-14T11:21:44.118051+0000","last_peered":"2023-07-14T11:21:44.118051+0000","last_clean":"2023-07-14T11:21:44.118051+0000","last_became_active":"2023-07-14T11:20:15.765349+0000","last_became_peered":"2023-07-14T11:20:15.765349+0000","last_unstale":"2023-07-14T11:21:44.118051+0000","last_undegraded":"2023-07-14T11:21:44.118051+0000","last_fullsized":"2023-07-14T11:21:44.118051+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:11:07.179820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131580+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:21:44.131580+0000","last_peered":"2023-07-14T11:21:44.131580+0000","last_clean":"2023-07-14T11:21:44.131580+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:21:44.131580+0000","last_undegraded":"2023-07-14T11:21:44.131580+0000","last_fullsized":"2023-07-14T11:21:44.131580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:48.020842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133597+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:21:44.133597+0000","last_peered":"2023-07-14T11:21:44.133597+0000","last_clean":"2023-07-14T11:21:44.133597+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:21:44.133597+0000","last_undegraded":"2023-07-14T11:21:44.133597+0000","last_fullsized":"2023-07-14T11:21:44.133597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:23.156838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134494+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:21:44.134494+0000","last_peered":"2023-07-14T11:21:44.134494+0000","last_clean":"2023-07-14T11:21:44.134494+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:21:44.134494+0000","last_undegraded":"2023-07-14T11:21:44.134494+0000","last_fullsized":"2023-07-14T11:21:44.134494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:03:17.632302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130072+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:21:44.130072+0000","last_peered":"2023-07-14T11:21:44.130072+0000","last_clean":"2023-07-14T11:21:44.130072+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:21:44.130072+0000","last_undegraded":"2023-07-14T11:21:44.130072+0000","last_fullsized":"2023-07-14T11:21:44.130072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:27:11.918619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66428,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":400,"num_write":160,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416425,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":510275,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.35999999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.52800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86699999999999999,"5min":1.097,"15min":1.097},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69099999999999995,"5min":0.75,"15min":0.75},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92000000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79400000000000004,"5min":1.042,"15min":1.042},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52500000000000002,"15min":0.50800000000000001},"min":{"1min":0.42799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.623},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89900000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.55700000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94799999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.72599999999999998},"min":{"1min":0.68799999999999994,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.02,"5min":1.159,"15min":1.159},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.92500000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73799999999999999,"15min":0.73899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":1.014,"15min":1.014},"last":0.97599999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.751,"15min":0.749},"min":{"1min":0.71899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.042,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.01,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.01}]},{"osd":10,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.96799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0640000000000001,"5min":1.456,"15min":1.456},"last":0.86599999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1150976,"data_stored":1160263,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.84899999999999998,"15min":0.89000000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.746,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.83599999999999997,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.877,"5min":6.0090000000000003,"15min":6.0090000000000003},"last":0.621}]},{"osd":1,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.52900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.76700000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67000000000000004,"15min":0.69699999999999995},"min":{"1min":0.42699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73599999999999999,"5min":2.46,"15min":2.46},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.66800000000000004},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77200000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.66500000000000004,"15min":0.69699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.496},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.75,"15min":0.745},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82899999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.42099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79200000000000004,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89900000000000002,"5min":2.431,"15min":2.431},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.59099999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.88300000000000001,"15min":0.84699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":9.5269999999999992,"5min":9.5269999999999992,"15min":9.5269999999999992},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.83699999999999997,"15min":0.81999999999999995},"min":{"1min":0.65000000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.7709999999999999,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.85499999999999998,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":2.536,"15min":2.536},"last":0.70199999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.69699999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.754},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.97299999999999998,"15min":0.97499999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":6.1559999999999997,"5min":6.1559999999999997,"15min":6.1559999999999997},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86799999999999999,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95999999999999996,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.70299999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.054},"last":0.68500000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":514461,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.38}]},{"osd":1,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":1.0289999999999999},"min":{"1min":0.51500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.355,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.439},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0620000000000001,"5min":1.073,"15min":1.073},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85099999999999998,"15min":0.877},"min":{"1min":0.53900000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1679999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88400000000000001,"15min":0.92200000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.65400000000000003}]},{"osd":8,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.077,"15min":1.1850000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3210000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.129,"5min":1.171,"15min":1.171},"last":0.628}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.503},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.4489999999999998,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.70999999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91500000000000004,"15min":0.93899999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.04,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.88600000000000001,"15min":0.90100000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0269999999999999,"5min":1.141,"15min":1.141},"last":0.80300000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":510173,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.50700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.436,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.68100000000000005,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.93000000000000005,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48499999999999999,"15min":0.44800000000000001},"min":{"1min":0.40600000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.008,"5min":2.032,"15min":2.032},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.998,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.998}]},{"osd":4,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.64400000000000002,"15min":0.59899999999999998},"min":{"1min":0.36099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.0329999999999999,"5min":1.046,"15min":1.046},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.69299999999999995,"15min":0.64100000000000001},"min":{"1min":0.68400000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.437,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":1.038,"15min":1.038},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.76000000000000001,"15min":0.72599999999999998},"min":{"1min":0.56699999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.048,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.048},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.68999999999999995},"min":{"1min":0.504,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.90100000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.71599999999999997,"15min":0.67700000000000005},"min":{"1min":0.55900000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.95799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.751,"15min":0.71699999999999997},"min":{"1min":0.63100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.02},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.80800000000000005,"15min":0.77400000000000002},"min":{"1min":0.69599999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.1299999999999999,"5min":1.319,"15min":1.319},"last":1.0800000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.70499999999999996},"min":{"1min":0.65100000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.79900000000000004,"15min":0.749},"min":{"1min":0.71199999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.093,"5min":1.218,"15min":1.218},"last":1.105}]},{"osd":11,"last update":"Fri Jul 14 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.84699999999999998,"15min":0.84499999999999997},"min":{"1min":0.64600000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.077,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.94599999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":1105376,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.32600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.376},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.34000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.378,"15min":0.378},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.621,"15min":0.58699999999999997},"min":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.2450000000000001,"5min":4.2450000000000001,"15min":4.2450000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.36299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85199999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.629}]},{"osd":6,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.75600000000000001,"15min":0.73599999999999999},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.3390000000000004,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69099999999999995,"15min":0.68600000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.51200000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.746,"15min":0.69999999999999996},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":2.762,"15min":2.762},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.51700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.075,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.752,"15min":0.748},"min":{"1min":0.56399999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.754}]},{"osd":10,"last update":"Fri Jul 14 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.75800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.501,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.052,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70599999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":505268,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.51900000000000002,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":0.378},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.55400000000000005,"15min":0.497},"min":{"1min":0.33000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.1180000000000003,"5min":5.1180000000000003,"15min":5.1180000000000003},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.65800000000000003},"min":{"1min":0.55800000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.86899999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.2360000000000002,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.71099999999999997,"15min":0.66600000000000004},"min":{"1min":0.64200000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68100000000000005,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.443,"15min":0.443},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.77400000000000002,"15min":0.753},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.76200000000000001,"15min":0.69999999999999996},"min":{"1min":0.57599999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.2730000000000001,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.73599999999999999},"min":{"1min":0.55900000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3009999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73799999999999999,"15min":0.72299999999999998},"min":{"1min":0.52800000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.745},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.78100000000000003},"min":{"1min":0.621,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Jul 14 11:21:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.81299999999999994,"15min":0.80300000000000005},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.83199999999999996,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0620000000000001,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.78300000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84699999999999998,"15min":0.85799999999999998},"min":{"1min":0.495,"5min":0.442,"15min":0.442},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.89800000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93400000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Jul 14 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.234,"5min":1.359,"15min":1.359},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.998,"15min":1.03},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3029999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":562986,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.32400000000000001},"max":{"1min":4.077,"5min":4.077,"15min":4.077},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61199999999999999,"15min":0.51600000000000001},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.625,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.32200000000000001},"max":{"1min":0.82599999999999996,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70199999999999996,"15min":0.64900000000000002},"min":{"1min":0.44700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":4.016,"5min":4.016,"15min":4.016},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51900000000000002,"15min":0.47199999999999998},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68899999999999995,"15min":0.63400000000000001},"min":{"1min":0.54500000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67700000000000005,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.495,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.1459999999999999,"5min":4.1459999999999999,"15min":4.1459999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.69199999999999995},"min":{"1min":0.59499999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.63600000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97699999999999998,"5min":1.036,"15min":1.036},"last":0.95899999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79100000000000004,"15min":0.78600000000000003},"min":{"1min":0.58699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.88300000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.439,"15min":0.439},"max":{"1min":0.98699999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.208}]},{"osd":11,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.016},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1,"5min":1.044,"15min":1.044},"last":1.0349999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":506656,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65400000000000003,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":0.504},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.33900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.59299999999999997},"min":{"1min":0.45600000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.873,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.97499999999999998,"5min":5.2060000000000004,"15min":5.2060000000000004},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.71599999999999997,"15min":0.61799999999999999},"min":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6680000000000001,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.57299999999999995},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90200000000000002,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72299999999999998,"15min":0.63900000000000001},"min":{"1min":0.51200000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.66800000000000004},"min":{"1min":0.48899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.996,"5min":5.2489999999999997,"15min":5.2489999999999997},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64700000000000002,"15min":0.63200000000000001},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.748,"15min":0.67600000000000005},"min":{"1min":0.442,"5min":0.38,"15min":0.38},"max":{"1min":6.3150000000000004,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.91900000000000004},"min":{"1min":0.60999999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.88600000000000001,"15min":0.91700000000000004},"min":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92800000000000005,"5min":5.351,"15min":5.351},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.84399999999999997,"15min":0.86299999999999999},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78300000000000003,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.036,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.748}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85499999999999998,"15min":0.89300000000000002},"min":{"1min":0.58899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.577,"15min":1.577},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.075,"5min":0.93000000000000005,"15min":0.92900000000000005},"min":{"1min":0.62,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.2720000000000002,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.64000000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.89300000000000002,"15min":0.86899999999999999},"min":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":5.7080000000000002,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.86299999999999999,"15min":0.88800000000000001},"min":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.018,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.58199999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.85399999999999998,"15min":0.81599999999999995},"min":{"1min":0.628,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.88600000000000001,"15min":0.84399999999999997},"min":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":0.81699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":563981,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72699999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":1.012,"15min":1.012},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.628,"15min":0.54400000000000004},"min":{"1min":0.57799999999999996,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.317,"5min":0.254,"15min":0.254},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.50600000000000001},"min":{"1min":0.45500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64800000000000002,"15min":0.626},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.50900000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.70799999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66400000000000003,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Jul 14 11:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.75900000000000001},"min":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65300000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.505,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90100000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.69699999999999995,"15min":0.67300000000000004},"min":{"1min":0.622,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89400000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.748,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.54300000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93899999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56399999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":505735,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.52800000000000002,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.4649999999999999,"5min":7.4649999999999999,"15min":7.4649999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.54700000000000004,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":6.4809999999999999,"5min":6.4809999999999999,"15min":6.4809999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.71699999999999997,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.7329999999999997,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.73799999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":6.335,"5min":6.335,"15min":6.335},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60399999999999998,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.69999999999999996,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.73799999999999999,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":6.242,"5min":6.242,"15min":6.242},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77700000000000002,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.133,"5min":6.8570000000000002,"15min":6.8570000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.65300000000000002},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.663,"15min":1.663},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:21:09 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.746,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.77500000000000002,"15min":0.69799999999999995},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.5140000000000002,"5min":6.5140000000000002,"15min":6.5140000000000002},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:32 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.83199999999999996,"15min":0.755},"min":{"1min":0.57199999999999995,"5min":0.505,"15min":0.505},"max":{"1min":6.415,"5min":6.415,"15min":6.415},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.63400000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":0.94799999999999995,"15min":0.89200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":13.365,"5min":13.365,"15min":13.365},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.048,"5min":0.98899999999999999,"15min":1.018},"min":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.4489999999999998,"5min":6.4489999999999998,"15min":6.4489999999999998},"last":0.68700000000000006}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":6.9649999999999999,"5min":6.9649999999999999,"15min":6.9649999999999999},"last":0.751},{"interface":"front","average":{"1min":1.069,"5min":0.86699999999999999,"15min":0.84399999999999997},"min":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.55800000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93700000000000006,"5min":1.7,"15min":1.7},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.0780000000000001,"15min":1.1200000000000001},"min":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1759999999999999,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.80800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":510143,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57399999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.432},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.40000000000000002},"min":{"1min":0.28199999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86399999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.745},{"interface":"front","average":{"1min":0.629,"5min":0.60099999999999998,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84199999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.69899999999999995,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3289999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":5,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.40100000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82699999999999996,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.622,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.745,"15min":0.72299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.7779999999999996,"5min":5.7779999999999996,"15min":5.7779999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.496,"5min":0.438,"15min":0.438},"max":{"1min":5.7309999999999999,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:21:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.47999999999999998},"min":{"1min":0.32100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77300000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68300000000000005,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91800000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.628,"15min":0.59099999999999997},"min":{"1min":0.54800000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.73099999999999998,"15min":0.69899999999999995},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.262,"5min":0.85899999999999999,"15min":0.80500000000000005},"min":{"1min":0.48199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.2640000000000002,"5min":6.2640000000000002,"15min":6.2640000000000002},"last":0.84499999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.78600000000000003},"min":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79600000000000004,"15min":0.79200000000000004},"min":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97499999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.82399999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1073152,"data_stored":1101012,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58799999999999997,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77300000000000002,"5min":2.843,"15min":2.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67300000000000004,"5min":6.6680000000000001,"15min":6.6680000000000001},"last":0.379}]},{"osd":1,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.58199999999999996,"15min":0.503},"min":{"1min":0.49099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.51700000000000002},"min":{"1min":0.46200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.504}]},{"osd":2,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.36899999999999999},"min":{"1min":0.34899999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.438,"15min":0.39500000000000002},"min":{"1min":0.36699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.622,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.59299999999999997},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.747,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61499999999999999,"15min":0.55400000000000005},"min":{"1min":0.52100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94799999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.55000000000000004},"min":{"1min":0.55000000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.68200000000000005,"15min":0.60999999999999999},"min":{"1min":0.54400000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.65100000000000002,"15min":0.60499999999999998},"min":{"1min":0.49399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:21:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64000000000000001,"15min":0.60199999999999998},"min":{"1min":0.59699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.625},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.52600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.038},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82899999999999996,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67100000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.751,"15min":0.72599999999999998},"min":{"1min":0.61899999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.77400000000000002,"15min":0.75},"min":{"1min":0.63,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73099999999999998,"15min":0.71399999999999997},"min":{"1min":0.60299999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.748,"15min":0.72399999999999998},"min":{"1min":0.67800000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.996,"5min":1.073,"15min":1.073},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:22:01.104 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:22:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 -- 172.21.15.161:0/1419453236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 msgr2=0x7fcf1c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 --2- 172.21.15.161:0/1419453236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c101140 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009990 tx=0x7fcf08009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 -- 172.21.15.161:0/1419453236 shutdown_connections 2023-07-14T11:22:01.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 --2- 172.21.15.161:0/1419453236 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcf1c101680 0x7fcf1c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 --2- 172.21.15.161:0/1419453236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf1c0ffb00 0x7fcf1c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 --2- 172.21.15.161:0/1419453236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c101140 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 -- 172.21.15.161:0/1419453236 >> 172.21.15.161:0/1419453236 conn(0x7fcf1c0fb0e0 msgr2=0x7fcf1c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:01.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 -- 172.21.15.161:0/1419453236 shutdown_connections 2023-07-14T11:22:01.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.303+0000 7fcf22dff700 1 -- 172.21.15.161:0/1419453236 wait complete. 2023-07-14T11:22:01.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.304+0000 7fcf22dff700 1 Processor -- start 2023-07-14T11:22:01.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.304+0000 7fcf22dff700 1 -- start start 2023-07-14T11:22:01.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.304+0000 7fcf22dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcf1c0ffb00 0x7fcf1c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.304+0000 7fcf22dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.304+0000 7fcf22dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf1c101680 0x7fcf1c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.304+0000 7fcf22dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c19f5a0 con 0x7fcf1c100cf0 2023-07-14T11:22:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf22dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c19f6e0 con 0x7fcf1c101680 2023-07-14T11:22:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf22dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c19f870 con 0x7fcf1c0ffb00 2023-07-14T11:22:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39684/0 (socket says 172.21.15.161:39684) 2023-07-14T11:22:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 -- 172.21.15.161:0/991593216 learned_addr learned my addr 172.21.15.161:0/991593216 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf20b9b700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcf1c0ffb00 0x7fcf1c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf2139c700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf1c101680 0x7fcf1c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 -- 172.21.15.161:0/991593216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcf1c0ffb00 msgr2=0x7fcf1c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcf1c0ffb00 0x7fcf1c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 -- 172.21.15.161:0/991593216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf1c101680 msgr2=0x7fcf1c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf1c101680 0x7fcf1c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 -- 172.21.15.161:0/991593216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf080095f0 con 0x7fcf1c100cf0 2023-07-14T11:22:01.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.305+0000 7fcf1bfff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c197f60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800b620 tx=0x7fcf08005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:01.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.306+0000 7fcf19ffb700 1 -- 172.21.15.161:0/991593216 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0801d070 con 0x7fcf1c100cf0 2023-07-14T11:22:01.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.306+0000 7fcf19ffb700 1 -- 172.21.15.161:0/991593216 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf08022470 con 0x7fcf1c100cf0 2023-07-14T11:22:01.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.306+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c19faf0 con 0x7fcf1c100cf0 2023-07-14T11:22:01.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.306+0000 7fcf19ffb700 1 -- 172.21.15.161:0/991593216 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08017650 con 0x7fcf1c100cf0 2023-07-14T11:22:01.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.306+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c19ff10 con 0x7fcf1c100cf0 2023-07-14T11:22:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.307+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c10b390 con 0x7fcf1c100cf0 2023-07-14T11:22:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.309+0000 7fcf19ffb700 1 -- 172.21.15.161:0/991593216 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcf080225e0 con 0x7fcf1c100cf0 2023-07-14T11:22:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.310+0000 7fcf19ffb700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcf0c0a11a0 0x7fcf0c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.310+0000 7fcf19ffb700 1 -- 172.21.15.161:0/991593216 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcf080ce110 con 0x7fcf1c100cf0 2023-07-14T11:22:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.310+0000 7fcf20b9b700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcf0c0a11a0 0x7fcf0c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.311+0000 7fcf20b9b700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcf0c0a11a0 0x7fcf0c0a3650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c100b50 tx=0x7fcf10009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:01.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.314+0000 7fcf19ffb700 1 -- 172.21.15.161:0/991593216 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcf080975a0 con 0x7fcf1c100cf0 2023-07-14T11:22:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:01 smithi161 ceph-mon[95526]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:01 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2189129385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:01.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.509+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf1c1075f0 con 0x7fcf0c0a11a0 2023-07-14T11:22:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.521+0000 7fcf19ffb700 1 -- 172.21.15.161:0/991593216 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426827 (secure 0 0 0) 0x7fcf1c1075f0 con 0x7fcf0c0a11a0 2023-07-14T11:22:01.528 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:01.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcf0c0a11a0 msgr2=0x7fcf0c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcf0c0a11a0 0x7fcf0c0a3650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c100b50 tx=0x7fcf10009260 comp rx=0 tx=0).stop 2023-07-14T11:22:01.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 msgr2=0x7fcf1c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c197f60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800b620 tx=0x7fcf08005740 comp rx=0 tx=0).stop 2023-07-14T11:22:01.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 shutdown_connections 2023-07-14T11:22:01.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fcf0c0a11a0 0x7fcf0c0a3650 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf1c101680 0x7fcf1c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcf1c0ffb00 0x7fcf1c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 --2- 172.21.15.161:0/991593216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1c100cf0 0x7fcf1c197f60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.526+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 >> 172.21.15.161:0/991593216 conn(0x7fcf1c0fb0e0 msgr2=0x7fcf1c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:01.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.527+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 shutdown_connections 2023-07-14T11:22:01.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.527+0000 7fcf22dff700 1 -- 172.21.15.161:0/991593216 wait complete. 2023-07-14T11:22:01.532 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:22:01.548 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2023-07-14T11:22:00.132430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657042,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":795,"num_read_kb":519,"num_write":343,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":779,"ondisk_log_size":779,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335648,"kb_used_data":12224,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606176,"statfs":{"total":1151940427776,"available":1151596724224,"internally_reserved":0,"allocated":12517376,"data_stored":8056329,"data_compressed":390111,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":41,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132890+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:21:44.132890+0000","last_peered":"2023-07-14T11:21:44.132890+0000","last_clean":"2023-07-14T11:21:44.132890+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:21:44.132890+0000","last_undegraded":"2023-07-14T11:21:44.132890+0000","last_fullsized":"2023-07-14T11:21:44.132890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:45:53.898814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131698+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:21:44.131698+0000","last_peered":"2023-07-14T11:21:44.131698+0000","last_clean":"2023-07-14T11:21:44.131698+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:21:44.131698+0000","last_undegraded":"2023-07-14T11:21:44.131698+0000","last_fullsized":"2023-07-14T11:21:44.131698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:27:43.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120443+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:21:44.120443+0000","last_peered":"2023-07-14T11:21:44.120443+0000","last_clean":"2023-07-14T11:21:44.120443+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:21:44.120443+0000","last_undegraded":"2023-07-14T11:21:44.120443+0000","last_fullsized":"2023-07-14T11:21:44.120443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:30:20.721165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133183+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:21:44.133183+0000","last_peered":"2023-07-14T11:21:44.133183+0000","last_clean":"2023-07-14T11:21:44.133183+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:21:44.133183+0000","last_undegraded":"2023-07-14T11:21:44.133183+0000","last_fullsized":"2023-07-14T11:21:44.133183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:05:47.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.124390+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:21:44.124390+0000","last_peered":"2023-07-14T11:21:44.124390+0000","last_clean":"2023-07-14T11:21:44.124390+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:21:44.124390+0000","last_undegraded":"2023-07-14T11:21:44.124390+0000","last_fullsized":"2023-07-14T11:21:44.124390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:03.043850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133900+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:21:44.133900+0000","last_peered":"2023-07-14T11:21:44.133900+0000","last_clean":"2023-07-14T11:21:44.133900+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:21:44.133900+0000","last_undegraded":"2023-07-14T11:21:44.133900+0000","last_fullsized":"2023-07-14T11:21:44.133900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:02.922415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128281+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:21:44.128281+0000","last_peered":"2023-07-14T11:21:44.128281+0000","last_clean":"2023-07-14T11:21:44.128281+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:21:44.128281+0000","last_undegraded":"2023-07-14T11:21:44.128281+0000","last_fullsized":"2023-07-14T11:21:44.128281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:20:08.848005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132015+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:21:44.132015+0000","last_peered":"2023-07-14T11:21:44.132015+0000","last_clean":"2023-07-14T11:21:44.132015+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:21:44.132015+0000","last_undegraded":"2023-07-14T11:21:44.132015+0000","last_fullsized":"2023-07-14T11:21:44.132015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:54.048999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129256+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:21:44.129256+0000","last_peered":"2023-07-14T11:21:44.129256+0000","last_clean":"2023-07-14T11:21:44.129256+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:21:44.129256+0000","last_undegraded":"2023-07-14T11:21:44.129256+0000","last_fullsized":"2023-07-14T11:21:44.129256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:25.896648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131299+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:21:44.131299+0000","last_peered":"2023-07-14T11:21:44.131299+0000","last_clean":"2023-07-14T11:21:44.131299+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:21:44.131299+0000","last_undegraded":"2023-07-14T11:21:44.131299+0000","last_fullsized":"2023-07-14T11:21:44.131299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:00.377250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.126348+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:21:44.126348+0000","last_peered":"2023-07-14T11:21:44.126348+0000","last_clean":"2023-07-14T11:21:44.126348+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:21:44.126348+0000","last_undegraded":"2023-07-14T11:21:44.126348+0000","last_fullsized":"2023-07-14T11:21:44.126348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:51.642943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133671+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:21:44.133671+0000","last_peered":"2023-07-14T11:21:44.133671+0000","last_clean":"2023-07-14T11:21:44.133671+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:21:44.133671+0000","last_undegraded":"2023-07-14T11:21:44.133671+0000","last_fullsized":"2023-07-14T11:21:44.133671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:40:35.925610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130954+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:21:44.130954+0000","last_peered":"2023-07-14T11:21:44.130954+0000","last_clean":"2023-07-14T11:21:44.130954+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:21:44.130954+0000","last_undegraded":"2023-07-14T11:21:44.130954+0000","last_fullsized":"2023-07-14T11:21:44.130954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:46:47.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129701+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:21:44.129701+0000","last_peered":"2023-07-14T11:21:44.129701+0000","last_clean":"2023-07-14T11:21:44.129701+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:21:44.129701+0000","last_undegraded":"2023-07-14T11:21:44.129701+0000","last_fullsized":"2023-07-14T11:21:44.129701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:40:58.143556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134018+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:21:44.134018+0000","last_peered":"2023-07-14T11:21:44.134018+0000","last_clean":"2023-07-14T11:21:44.134018+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:21:44.134018+0000","last_undegraded":"2023-07-14T11:21:44.134018+0000","last_fullsized":"2023-07-14T11:21:44.134018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:24.775441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.127674+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:21:44.127674+0000","last_peered":"2023-07-14T11:21:44.127674+0000","last_clean":"2023-07-14T11:21:44.127674+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:21:44.127674+0000","last_undegraded":"2023-07-14T11:21:44.127674+0000","last_fullsized":"2023-07-14T11:21:44.127674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:49.835385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132947+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:21:44.132947+0000","last_peered":"2023-07-14T11:21:44.132947+0000","last_clean":"2023-07-14T11:21:44.132947+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:21:44.132947+0000","last_undegraded":"2023-07-14T11:21:44.132947+0000","last_fullsized":"2023-07-14T11:21:44.132947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:58:04.718574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.114499+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:21:44.114499+0000","last_peered":"2023-07-14T11:21:44.114499+0000","last_clean":"2023-07-14T11:21:44.114499+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:21:44.114499+0000","last_undegraded":"2023-07-14T11:21:44.114499+0000","last_fullsized":"2023-07-14T11:21:44.114499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:45.295320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129091+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:21:44.129091+0000","last_peered":"2023-07-14T11:21:44.129091+0000","last_clean":"2023-07-14T11:21:44.129091+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:21:44.129091+0000","last_undegraded":"2023-07-14T11:21:44.129091+0000","last_fullsized":"2023-07-14T11:21:44.129091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:38:25.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761984+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:21:16.761984+0000","last_peered":"2023-07-14T11:21:16.761984+0000","last_clean":"2023-07-14T11:21:16.761984+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:21:16.761984+0000","last_undegraded":"2023-07-14T11:21:16.761984+0000","last_fullsized":"2023-07-14T11:21:16.761984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:32.149825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.757056+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:21:16.757056+0000","last_peered":"2023-07-14T11:21:16.757056+0000","last_clean":"2023-07-14T11:21:16.757056+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:21:16.757056+0000","last_undegraded":"2023-07-14T11:21:16.757056+0000","last_fullsized":"2023-07-14T11:21:16.757056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:27:34.767964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128034+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:21:44.128034+0000","last_peered":"2023-07-14T11:21:44.128034+0000","last_clean":"2023-07-14T11:21:44.128034+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:21:44.128034+0000","last_undegraded":"2023-07-14T11:21:44.128034+0000","last_fullsized":"2023-07-14T11:21:44.128034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:23:39.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782856+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:21:16.782856+0000","last_peered":"2023-07-14T11:21:16.782856+0000","last_clean":"2023-07-14T11:21:16.782856+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:21:16.782856+0000","last_undegraded":"2023-07-14T11:21:16.782856+0000","last_fullsized":"2023-07-14T11:21:16.782856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:47.727483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761802+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:21:16.761802+0000","last_peered":"2023-07-14T11:21:16.761802+0000","last_clean":"2023-07-14T11:21:16.761802+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:21:16.761802+0000","last_undegraded":"2023-07-14T11:21:16.761802+0000","last_fullsized":"2023-07-14T11:21:16.761802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:29:59.179624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130426+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:21:44.130426+0000","last_peered":"2023-07-14T11:21:44.130426+0000","last_clean":"2023-07-14T11:21:44.130426+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:21:44.130426+0000","last_undegraded":"2023-07-14T11:21:44.130426+0000","last_fullsized":"2023-07-14T11:21:44.130426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:18.435015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782915+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:21:16.782915+0000","last_peered":"2023-07-14T11:21:16.782915+0000","last_clean":"2023-07-14T11:21:16.782915+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:21:16.782915+0000","last_undegraded":"2023-07-14T11:21:16.782915+0000","last_fullsized":"2023-07-14T11:21:16.782915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:27.267522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760533+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:21:16.760533+0000","last_peered":"2023-07-14T11:21:16.760533+0000","last_clean":"2023-07-14T11:21:16.760533+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:21:16.760533+0000","last_undegraded":"2023-07-14T11:21:16.760533+0000","last_fullsized":"2023-07-14T11:21:16.760533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:28.556454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130958+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:21:44.130958+0000","last_peered":"2023-07-14T11:21:44.130958+0000","last_clean":"2023-07-14T11:21:44.130958+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:21:44.130958+0000","last_undegraded":"2023-07-14T11:21:44.130958+0000","last_fullsized":"2023-07-14T11:21:44.130958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:23.714644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782878+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:21:16.782878+0000","last_peered":"2023-07-14T11:21:16.782878+0000","last_clean":"2023-07-14T11:21:16.782878+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:21:16.782878+0000","last_undegraded":"2023-07-14T11:21:16.782878+0000","last_fullsized":"2023-07-14T11:21:16.782878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:14.385740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759437+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:21:16.759437+0000","last_peered":"2023-07-14T11:21:16.759437+0000","last_clean":"2023-07-14T11:21:16.759437+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:21:16.759437+0000","last_undegraded":"2023-07-14T11:21:16.759437+0000","last_fullsized":"2023-07-14T11:21:16.759437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:24:07.117370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120539+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:21:44.120539+0000","last_peered":"2023-07-14T11:21:44.120539+0000","last_clean":"2023-07-14T11:21:44.120539+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:21:44.120539+0000","last_undegraded":"2023-07-14T11:21:44.120539+0000","last_fullsized":"2023-07-14T11:21:44.120539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:01:41.642530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761645+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:21:16.761645+0000","last_peered":"2023-07-14T11:21:16.761645+0000","last_clean":"2023-07-14T11:21:16.761645+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:21:16.761645+0000","last_undegraded":"2023-07-14T11:21:16.761645+0000","last_fullsized":"2023-07-14T11:21:16.761645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:22:33.821715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":143,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:56.841418+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:21:56.841418+0000","last_peered":"2023-07-14T11:21:56.841418+0000","last_clean":"2023-07-14T11:21:56.841418+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:21:56.841418+0000","last_undegraded":"2023-07-14T11:21:56.841418+0000","last_fullsized":"2023-07-14T11:21:56.841418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:55:16.898539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.117612+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:21:44.117612+0000","last_peered":"2023-07-14T11:21:44.117612+0000","last_clean":"2023-07-14T11:21:44.117612+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:21:44.117612+0000","last_undegraded":"2023-07-14T11:21:44.117612+0000","last_fullsized":"2023-07-14T11:21:44.117612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:49:45.959442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.762138+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:21:16.762138+0000","last_peered":"2023-07-14T11:21:16.762138+0000","last_clean":"2023-07-14T11:21:16.762138+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:21:16.762138+0000","last_undegraded":"2023-07-14T11:21:16.762138+0000","last_fullsized":"2023-07-14T11:21:16.762138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:29:21.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.754843+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:21:16.754843+0000","last_peered":"2023-07-14T11:21:16.754843+0000","last_clean":"2023-07-14T11:21:16.754843+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:21:16.754843+0000","last_undegraded":"2023-07-14T11:21:16.754843+0000","last_fullsized":"2023-07-14T11:21:16.754843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:42.195752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.126818+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:21:44.126818+0000","last_peered":"2023-07-14T11:21:44.126818+0000","last_clean":"2023-07-14T11:21:44.126818+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:21:44.126818+0000","last_undegraded":"2023-07-14T11:21:44.126818+0000","last_fullsized":"2023-07-14T11:21:44.126818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:15:54.989039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.758730+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:21:16.758730+0000","last_peered":"2023-07-14T11:21:16.758730+0000","last_clean":"2023-07-14T11:21:16.758730+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:21:16.758730+0000","last_undegraded":"2023-07-14T11:21:16.758730+0000","last_fullsized":"2023-07-14T11:21:16.758730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:13:50.965885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780547+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:21:16.780547+0000","last_peered":"2023-07-14T11:21:16.780547+0000","last_clean":"2023-07-14T11:21:16.780547+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:21:16.780547+0000","last_undegraded":"2023-07-14T11:21:16.780547+0000","last_fullsized":"2023-07-14T11:21:16.780547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:20:20.452801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134597+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:21:44.134597+0000","last_peered":"2023-07-14T11:21:44.134597+0000","last_clean":"2023-07-14T11:21:44.134597+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:21:44.134597+0000","last_undegraded":"2023-07-14T11:21:44.134597+0000","last_fullsized":"2023-07-14T11:21:44.134597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:26:38.989878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761717+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:21:16.761717+0000","last_peered":"2023-07-14T11:21:16.761717+0000","last_clean":"2023-07-14T11:21:16.761717+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:21:16.761717+0000","last_undegraded":"2023-07-14T11:21:16.761717+0000","last_fullsized":"2023-07-14T11:21:16.761717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:06:57.319545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.779919+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:21:16.779919+0000","last_peered":"2023-07-14T11:21:16.779919+0000","last_clean":"2023-07-14T11:21:16.779919+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:21:16.779919+0000","last_undegraded":"2023-07-14T11:21:16.779919+0000","last_fullsized":"2023-07-14T11:21:16.779919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:45.971471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132864+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:21:44.132864+0000","last_peered":"2023-07-14T11:21:44.132864+0000","last_clean":"2023-07-14T11:21:44.132864+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:21:44.132864+0000","last_undegraded":"2023-07-14T11:21:44.132864+0000","last_fullsized":"2023-07-14T11:21:44.132864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:22.629848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760884+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:21:16.760884+0000","last_peered":"2023-07-14T11:21:16.760884+0000","last_clean":"2023-07-14T11:21:16.760884+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:21:16.760884+0000","last_undegraded":"2023-07-14T11:21:16.760884+0000","last_fullsized":"2023-07-14T11:21:16.760884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:07.800244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:55.839185+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:21:55.839185+0000","last_peered":"2023-07-14T11:21:55.839185+0000","last_clean":"2023-07-14T11:21:55.839185+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:21:55.839185+0000","last_undegraded":"2023-07-14T11:21:55.839185+0000","last_fullsized":"2023-07-14T11:21:55.839185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:15:34.152726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129759+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:21:44.129759+0000","last_peered":"2023-07-14T11:21:44.129759+0000","last_clean":"2023-07-14T11:21:44.129759+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:21:44.129759+0000","last_undegraded":"2023-07-14T11:21:44.129759+0000","last_fullsized":"2023-07-14T11:21:44.129759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:56.754661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756218+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:21:16.756218+0000","last_peered":"2023-07-14T11:21:16.756218+0000","last_clean":"2023-07-14T11:21:16.756218+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:21:16.756218+0000","last_undegraded":"2023-07-14T11:21:16.756218+0000","last_fullsized":"2023-07-14T11:21:16.756218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:28.728921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827514+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:21:17.827514+0000","last_peered":"2023-07-14T11:21:17.827514+0000","last_clean":"2023-07-14T11:21:17.827514+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:21:17.827514+0000","last_undegraded":"2023-07-14T11:21:17.827514+0000","last_fullsized":"2023-07-14T11:21:17.827514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:16:06.201581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135307+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:21:44.135307+0000","last_peered":"2023-07-14T11:21:44.135307+0000","last_clean":"2023-07-14T11:21:44.135307+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:21:44.135307+0000","last_undegraded":"2023-07-14T11:21:44.135307+0000","last_fullsized":"2023-07-14T11:21:44.135307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:08.534228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783067+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:21:16.783067+0000","last_peered":"2023-07-14T11:21:16.783067+0000","last_clean":"2023-07-14T11:21:16.783067+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:21:16.783067+0000","last_undegraded":"2023-07-14T11:21:16.783067+0000","last_fullsized":"2023-07-14T11:21:16.783067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:37.851033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781109+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:21:16.781109+0000","last_peered":"2023-07-14T11:21:16.781109+0000","last_clean":"2023-07-14T11:21:16.781109+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:21:16.781109+0000","last_undegraded":"2023-07-14T11:21:16.781109+0000","last_fullsized":"2023-07-14T11:21:16.781109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:17:24.738586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133580+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:21:44.133580+0000","last_peered":"2023-07-14T11:21:44.133580+0000","last_clean":"2023-07-14T11:21:44.133580+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:21:44.133580+0000","last_undegraded":"2023-07-14T11:21:44.133580+0000","last_fullsized":"2023-07-14T11:21:44.133580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:08:23.851839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756241+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:21:16.756241+0000","last_peered":"2023-07-14T11:21:16.756241+0000","last_clean":"2023-07-14T11:21:16.756241+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:21:16.756241+0000","last_undegraded":"2023-07-14T11:21:16.756241+0000","last_fullsized":"2023-07-14T11:21:16.756241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:29:43.064170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":284,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:56.841699+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:21:56.841699+0000","last_peered":"2023-07-14T11:21:56.841699+0000","last_clean":"2023-07-14T11:21:56.841699+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:21:56.841699+0000","last_undegraded":"2023-07-14T11:21:56.841699+0000","last_fullsized":"2023-07-14T11:21:56.841699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:27:38.535607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":219,"num_read_kb":171,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.122999+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:21:44.122999+0000","last_peered":"2023-07-14T11:21:44.122999+0000","last_clean":"2023-07-14T11:21:44.122999+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:21:44.122999+0000","last_undegraded":"2023-07-14T11:21:44.122999+0000","last_fullsized":"2023-07-14T11:21:44.122999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:09:57.788775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781384+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:21:16.781384+0000","last_peered":"2023-07-14T11:21:16.781384+0000","last_clean":"2023-07-14T11:21:16.781384+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:21:16.781384+0000","last_undegraded":"2023-07-14T11:21:16.781384+0000","last_fullsized":"2023-07-14T11:21:16.781384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:12.644525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781070+0000","last_change":"2023-07-14T11:21:16.780317+0000","last_active":"2023-07-14T11:21:16.781070+0000","last_peered":"2023-07-14T11:21:16.781070+0000","last_clean":"2023-07-14T11:21:16.781070+0000","last_became_active":"2023-07-14T11:21:16.763144+0000","last_became_peered":"2023-07-14T11:21:16.763144+0000","last_unstale":"2023-07-14T11:21:16.781070+0000","last_undegraded":"2023-07-14T11:21:16.781070+0000","last_fullsized":"2023-07-14T11:21:16.781070+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:59.524343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132499+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:21:44.132499+0000","last_peered":"2023-07-14T11:21:44.132499+0000","last_clean":"2023-07-14T11:21:44.132499+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:21:44.132499+0000","last_undegraded":"2023-07-14T11:21:44.132499+0000","last_fullsized":"2023-07-14T11:21:44.132499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:16.990224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.758571+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:21:16.758571+0000","last_peered":"2023-07-14T11:21:16.758571+0000","last_clean":"2023-07-14T11:21:16.758571+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:21:16.758571+0000","last_undegraded":"2023-07-14T11:21:16.758571+0000","last_fullsized":"2023-07-14T11:21:16.758571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:30:05.342050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780126+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:21:16.780126+0000","last_peered":"2023-07-14T11:21:16.780126+0000","last_clean":"2023-07-14T11:21:16.780126+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:21:16.780126+0000","last_undegraded":"2023-07-14T11:21:16.780126+0000","last_fullsized":"2023-07-14T11:21:16.780126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:59:52.431744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130683+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:21:44.130683+0000","last_peered":"2023-07-14T11:21:44.130683+0000","last_clean":"2023-07-14T11:21:44.130683+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:21:44.130683+0000","last_undegraded":"2023-07-14T11:21:44.130683+0000","last_fullsized":"2023-07-14T11:21:44.130683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:10:00.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756683+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:21:16.756683+0000","last_peered":"2023-07-14T11:21:16.756683+0000","last_clean":"2023-07-14T11:21:16.756683+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:21:16.756683+0000","last_undegraded":"2023-07-14T11:21:16.756683+0000","last_fullsized":"2023-07-14T11:21:16.756683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:40.564962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:55.839289+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:21:55.839289+0000","last_peered":"2023-07-14T11:21:55.839289+0000","last_clean":"2023-07-14T11:21:55.839289+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:21:55.839289+0000","last_undegraded":"2023-07-14T11:21:55.839289+0000","last_fullsized":"2023-07-14T11:21:55.839289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:26.857871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.123757+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:21:44.123757+0000","last_peered":"2023-07-14T11:21:44.123757+0000","last_clean":"2023-07-14T11:21:44.123757+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:21:44.123757+0000","last_undegraded":"2023-07-14T11:21:44.123757+0000","last_fullsized":"2023-07-14T11:21:44.123757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:37.070346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780756+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:21:16.780756+0000","last_peered":"2023-07-14T11:21:16.780756+0000","last_clean":"2023-07-14T11:21:16.780756+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:21:16.780756+0000","last_undegraded":"2023-07-14T11:21:16.780756+0000","last_fullsized":"2023-07-14T11:21:16.780756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:33:44.207912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.774327+0000","last_change":"2023-07-14T11:21:16.773940+0000","last_active":"2023-07-14T11:21:16.774327+0000","last_peered":"2023-07-14T11:21:16.774327+0000","last_clean":"2023-07-14T11:21:16.774327+0000","last_became_active":"2023-07-14T11:21:16.762658+0000","last_became_peered":"2023-07-14T11:21:16.762658+0000","last_unstale":"2023-07-14T11:21:16.774327+0000","last_undegraded":"2023-07-14T11:21:16.774327+0000","last_fullsized":"2023-07-14T11:21:16.774327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:05.595583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120231+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:21:44.120231+0000","last_peered":"2023-07-14T11:21:44.120231+0000","last_clean":"2023-07-14T11:21:44.120231+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:21:44.120231+0000","last_undegraded":"2023-07-14T11:21:44.120231+0000","last_fullsized":"2023-07-14T11:21:44.120231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:59:04.043553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780006+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:21:16.780006+0000","last_peered":"2023-07-14T11:21:16.780006+0000","last_clean":"2023-07-14T11:21:16.780006+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:21:16.780006+0000","last_undegraded":"2023-07-14T11:21:16.780006+0000","last_fullsized":"2023-07-14T11:21:16.780006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:19.174537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760420+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:21:16.760420+0000","last_peered":"2023-07-14T11:21:16.760420+0000","last_clean":"2023-07-14T11:21:16.760420+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:21:16.760420+0000","last_undegraded":"2023-07-14T11:21:16.760420+0000","last_fullsized":"2023-07-14T11:21:16.760420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:13.662336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129915+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:21:44.129915+0000","last_peered":"2023-07-14T11:21:44.129915+0000","last_clean":"2023-07-14T11:21:44.129915+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:21:44.129915+0000","last_undegraded":"2023-07-14T11:21:44.129915+0000","last_fullsized":"2023-07-14T11:21:44.129915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:12:29.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759722+0000","last_change":"2023-07-14T11:20:15.731601+0000","last_active":"2023-07-14T11:21:16.759722+0000","last_peered":"2023-07-14T11:21:16.759722+0000","last_clean":"2023-07-14T11:21:16.759722+0000","last_became_active":"2023-07-14T11:20:15.731121+0000","last_became_peered":"2023-07-14T11:20:15.731121+0000","last_unstale":"2023-07-14T11:21:16.759722+0000","last_undegraded":"2023-07-14T11:21:16.759722+0000","last_fullsized":"2023-07-14T11:21:16.759722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:15:52.609093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756735+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:21:16.756735+0000","last_peered":"2023-07-14T11:21:16.756735+0000","last_clean":"2023-07-14T11:21:16.756735+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:21:16.756735+0000","last_undegraded":"2023-07-14T11:21:16.756735+0000","last_fullsized":"2023-07-14T11:21:16.756735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:57.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133807+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:21:44.133807+0000","last_peered":"2023-07-14T11:21:44.133807+0000","last_clean":"2023-07-14T11:21:44.133807+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:21:44.133807+0000","last_undegraded":"2023-07-14T11:21:44.133807+0000","last_fullsized":"2023-07-14T11:21:44.133807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:15:22.547786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781033+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:21:16.781033+0000","last_peered":"2023-07-14T11:21:16.781033+0000","last_clean":"2023-07-14T11:21:16.781033+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:21:16.781033+0000","last_undegraded":"2023-07-14T11:21:16.781033+0000","last_fullsized":"2023-07-14T11:21:16.781033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:43:54.535268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827424+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:21:17.827424+0000","last_peered":"2023-07-14T11:21:17.827424+0000","last_clean":"2023-07-14T11:21:17.827424+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:21:17.827424+0000","last_undegraded":"2023-07-14T11:21:17.827424+0000","last_fullsized":"2023-07-14T11:21:17.827424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:34:07.328659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128632+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:21:44.128632+0000","last_peered":"2023-07-14T11:21:44.128632+0000","last_clean":"2023-07-14T11:21:44.128632+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:21:44.128632+0000","last_undegraded":"2023-07-14T11:21:44.128632+0000","last_fullsized":"2023-07-14T11:21:44.128632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:58:43.563311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780882+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:21:16.780882+0000","last_peered":"2023-07-14T11:21:16.780882+0000","last_clean":"2023-07-14T11:21:16.780882+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:21:16.780882+0000","last_undegraded":"2023-07-14T11:21:16.780882+0000","last_fullsized":"2023-07-14T11:21:16.780882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:56:02.843006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.121291+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:21:44.121291+0000","last_peered":"2023-07-14T11:21:44.121291+0000","last_clean":"2023-07-14T11:21:44.121291+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:21:44.121291+0000","last_undegraded":"2023-07-14T11:21:44.121291+0000","last_fullsized":"2023-07-14T11:21:44.121291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:22:39.851277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:57.841537+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:21:57.841537+0000","last_peered":"2023-07-14T11:21:57.841537+0000","last_clean":"2023-07-14T11:21:57.841537+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:21:57.841537+0000","last_undegraded":"2023-07-14T11:21:57.841537+0000","last_fullsized":"2023-07-14T11:21:57.841537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:39:23.337780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.828305+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:21:17.828305+0000","last_peered":"2023-07-14T11:21:17.828305+0000","last_clean":"2023-07-14T11:21:17.828305+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:21:17.828305+0000","last_undegraded":"2023-07-14T11:21:17.828305+0000","last_fullsized":"2023-07-14T11:21:17.828305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:47:27.405242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135278+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:21:44.135278+0000","last_peered":"2023-07-14T11:21:44.135278+0000","last_clean":"2023-07-14T11:21:44.135278+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:21:44.135278+0000","last_undegraded":"2023-07-14T11:21:44.135278+0000","last_fullsized":"2023-07-14T11:21:44.135278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:51.335219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783295+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:21:16.783295+0000","last_peered":"2023-07-14T11:21:16.783295+0000","last_clean":"2023-07-14T11:21:16.783295+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:21:16.783295+0000","last_undegraded":"2023-07-14T11:21:16.783295+0000","last_fullsized":"2023-07-14T11:21:16.783295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:42:44.347182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827912+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:21:17.827912+0000","last_peered":"2023-07-14T11:21:17.827912+0000","last_clean":"2023-07-14T11:21:17.827912+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:21:17.827912+0000","last_undegraded":"2023-07-14T11:21:17.827912+0000","last_fullsized":"2023-07-14T11:21:17.827912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:53.938118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781156+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:21:16.781156+0000","last_peered":"2023-07-14T11:21:16.781156+0000","last_clean":"2023-07-14T11:21:16.781156+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:21:16.781156+0000","last_undegraded":"2023-07-14T11:21:16.781156+0000","last_fullsized":"2023-07-14T11:21:16.781156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:33:14.547044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129505+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:21:44.129505+0000","last_peered":"2023-07-14T11:21:44.129505+0000","last_clean":"2023-07-14T11:21:44.129505+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:21:44.129505+0000","last_undegraded":"2023-07-14T11:21:44.129505+0000","last_fullsized":"2023-07-14T11:21:44.129505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:53:19.310433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827364+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:21:17.827364+0000","last_peered":"2023-07-14T11:21:17.827364+0000","last_clean":"2023-07-14T11:21:17.827364+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:21:17.827364+0000","last_undegraded":"2023-07-14T11:21:17.827364+0000","last_fullsized":"2023-07-14T11:21:17.827364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:25:55.640663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759127+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:21:16.759127+0000","last_peered":"2023-07-14T11:21:16.759127+0000","last_clean":"2023-07-14T11:21:16.759127+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:21:16.759127+0000","last_undegraded":"2023-07-14T11:21:16.759127+0000","last_fullsized":"2023-07-14T11:21:16.759127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:28.280537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760321+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:21:16.760321+0000","last_peered":"2023-07-14T11:21:16.760321+0000","last_clean":"2023-07-14T11:21:16.760321+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:21:16.760321+0000","last_undegraded":"2023-07-14T11:21:16.760321+0000","last_fullsized":"2023-07-14T11:21:16.760321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:46.760759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133293+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:21:44.133293+0000","last_peered":"2023-07-14T11:21:44.133293+0000","last_clean":"2023-07-14T11:21:44.133293+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:21:44.133293+0000","last_undegraded":"2023-07-14T11:21:44.133293+0000","last_fullsized":"2023-07-14T11:21:44.133293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:20:33.855972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761100+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:21:16.761100+0000","last_peered":"2023-07-14T11:21:16.761100+0000","last_clean":"2023-07-14T11:21:16.761100+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:21:16.761100+0000","last_undegraded":"2023-07-14T11:21:16.761100+0000","last_fullsized":"2023-07-14T11:21:16.761100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:01.581528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:51.950589+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:21:51.950589+0000","last_peered":"2023-07-14T11:21:51.950589+0000","last_clean":"2023-07-14T11:21:51.950589+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:21:51.950589+0000","last_undegraded":"2023-07-14T11:21:51.950589+0000","last_fullsized":"2023-07-14T11:21:51.950589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.789040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131085+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:21:44.131085+0000","last_peered":"2023-07-14T11:21:44.131085+0000","last_clean":"2023-07-14T11:21:44.131085+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:21:44.131085+0000","last_undegraded":"2023-07-14T11:21:44.131085+0000","last_fullsized":"2023-07-14T11:21:44.131085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:10.033823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761906+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:21:16.761906+0000","last_peered":"2023-07-14T11:21:16.761906+0000","last_clean":"2023-07-14T11:21:16.761906+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:21:16.761906+0000","last_undegraded":"2023-07-14T11:21:16.761906+0000","last_fullsized":"2023-07-14T11:21:16.761906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:19:06.706597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783437+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:21:16.783437+0000","last_peered":"2023-07-14T11:21:16.783437+0000","last_clean":"2023-07-14T11:21:16.783437+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:21:16.783437+0000","last_undegraded":"2023-07-14T11:21:16.783437+0000","last_fullsized":"2023-07-14T11:21:16.783437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:04:22.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135985+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:21:44.135985+0000","last_peered":"2023-07-14T11:21:44.135985+0000","last_clean":"2023-07-14T11:21:44.135985+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:21:44.135985+0000","last_undegraded":"2023-07-14T11:21:44.135985+0000","last_fullsized":"2023-07-14T11:21:44.135985+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:50:49.239460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783236+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:21:16.783236+0000","last_peered":"2023-07-14T11:21:16.783236+0000","last_clean":"2023-07-14T11:21:16.783236+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:21:16.783236+0000","last_undegraded":"2023-07-14T11:21:16.783236+0000","last_fullsized":"2023-07-14T11:21:16.783236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:00:23.953512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:18.022769+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:21:18.022769+0000","last_peered":"2023-07-14T11:21:18.022769+0000","last_clean":"2023-07-14T11:21:18.022769+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:21:18.022769+0000","last_undegraded":"2023-07-14T11:21:18.022769+0000","last_fullsized":"2023-07-14T11:21:18.022769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:12:28.851941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.108769+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:21:44.108769+0000","last_peered":"2023-07-14T11:21:44.108769+0000","last_clean":"2023-07-14T11:21:44.108769+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:21:44.108769+0000","last_undegraded":"2023-07-14T11:21:44.108769+0000","last_fullsized":"2023-07-14T11:21:44.108769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:23:32.637906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756893+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:21:16.756893+0000","last_peered":"2023-07-14T11:21:16.756893+0000","last_clean":"2023-07-14T11:21:16.756893+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:21:16.756893+0000","last_undegraded":"2023-07-14T11:21:16.756893+0000","last_fullsized":"2023-07-14T11:21:16.756893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:52:08.577965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761060+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:21:16.761060+0000","last_peered":"2023-07-14T11:21:16.761060+0000","last_clean":"2023-07-14T11:21:16.761060+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:21:16.761060+0000","last_undegraded":"2023-07-14T11:21:16.761060+0000","last_fullsized":"2023-07-14T11:21:16.761060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:11:05.150069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132051+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:21:44.132051+0000","last_peered":"2023-07-14T11:21:44.132051+0000","last_clean":"2023-07-14T11:21:44.132051+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:21:44.132051+0000","last_undegraded":"2023-07-14T11:21:44.132051+0000","last_fullsized":"2023-07-14T11:21:44.132051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:20:37.760123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783130+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:21:16.783130+0000","last_peered":"2023-07-14T11:21:16.783130+0000","last_clean":"2023-07-14T11:21:16.783130+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:21:16.783130+0000","last_undegraded":"2023-07-14T11:21:16.783130+0000","last_fullsized":"2023-07-14T11:21:16.783130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:05:52.217578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761847+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:21:16.761847+0000","last_peered":"2023-07-14T11:21:16.761847+0000","last_clean":"2023-07-14T11:21:16.761847+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:21:16.761847+0000","last_undegraded":"2023-07-14T11:21:16.761847+0000","last_fullsized":"2023-07-14T11:21:16.761847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:24.188762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132768+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:21:44.132768+0000","last_peered":"2023-07-14T11:21:44.132768+0000","last_clean":"2023-07-14T11:21:44.132768+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:21:44.132768+0000","last_undegraded":"2023-07-14T11:21:44.132768+0000","last_fullsized":"2023-07-14T11:21:44.132768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:44.830001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783284+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:21:16.783284+0000","last_peered":"2023-07-14T11:21:16.783284+0000","last_clean":"2023-07-14T11:21:16.783284+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:21:16.783284+0000","last_undegraded":"2023-07-14T11:21:16.783284+0000","last_fullsized":"2023-07-14T11:21:16.783284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:07:04.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759626+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:21:16.759626+0000","last_peered":"2023-07-14T11:21:16.759626+0000","last_clean":"2023-07-14T11:21:16.759626+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:21:16.759626+0000","last_undegraded":"2023-07-14T11:21:16.759626+0000","last_fullsized":"2023-07-14T11:21:16.759626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:43:00.377914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134559+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:21:44.134559+0000","last_peered":"2023-07-14T11:21:44.134559+0000","last_clean":"2023-07-14T11:21:44.134559+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:21:44.134559+0000","last_undegraded":"2023-07-14T11:21:44.134559+0000","last_fullsized":"2023-07-14T11:21:44.134559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:03.031833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:18.023102+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:21:18.023102+0000","last_peered":"2023-07-14T11:21:18.023102+0000","last_clean":"2023-07-14T11:21:18.023102+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:21:18.023102+0000","last_undegraded":"2023-07-14T11:21:18.023102+0000","last_fullsized":"2023-07-14T11:21:18.023102+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:04:51.829719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.762268+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:21:16.762268+0000","last_peered":"2023-07-14T11:21:16.762268+0000","last_clean":"2023-07-14T11:21:16.762268+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:21:16.762268+0000","last_undegraded":"2023-07-14T11:21:16.762268+0000","last_fullsized":"2023-07-14T11:21:16.762268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:41:38.935430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131585+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:21:44.131585+0000","last_peered":"2023-07-14T11:21:44.131585+0000","last_clean":"2023-07-14T11:21:44.131585+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:21:44.131585+0000","last_undegraded":"2023-07-14T11:21:44.131585+0000","last_fullsized":"2023-07-14T11:21:44.131585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:00.381695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760768+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:21:16.760768+0000","last_peered":"2023-07-14T11:21:16.760768+0000","last_clean":"2023-07-14T11:21:16.760768+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:21:16.760768+0000","last_undegraded":"2023-07-14T11:21:16.760768+0000","last_fullsized":"2023-07-14T11:21:16.760768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:22:56.514930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781299+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:21:16.781299+0000","last_peered":"2023-07-14T11:21:16.781299+0000","last_clean":"2023-07-14T11:21:16.781299+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:21:16.781299+0000","last_undegraded":"2023-07-14T11:21:16.781299+0000","last_fullsized":"2023-07-14T11:21:16.781299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:39.416802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128900+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:21:44.128900+0000","last_peered":"2023-07-14T11:21:44.128900+0000","last_clean":"2023-07-14T11:21:44.128900+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:21:44.128900+0000","last_undegraded":"2023-07-14T11:21:44.128900+0000","last_fullsized":"2023-07-14T11:21:44.128900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:58.657251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.779781+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:21:16.779781+0000","last_peered":"2023-07-14T11:21:16.779781+0000","last_clean":"2023-07-14T11:21:16.779781+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:21:16.779781+0000","last_undegraded":"2023-07-14T11:21:16.779781+0000","last_fullsized":"2023-07-14T11:21:16.779781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:32:23.647113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780749+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:21:16.780749+0000","last_peered":"2023-07-14T11:21:16.780749+0000","last_clean":"2023-07-14T11:21:16.780749+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:21:16.780749+0000","last_undegraded":"2023-07-14T11:21:16.780749+0000","last_fullsized":"2023-07-14T11:21:16.780749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:31.036473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131916+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:21:44.131916+0000","last_peered":"2023-07-14T11:21:44.131916+0000","last_clean":"2023-07-14T11:21:44.131916+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:21:44.131916+0000","last_undegraded":"2023-07-14T11:21:44.131916+0000","last_fullsized":"2023-07-14T11:21:44.131916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:54.620604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.123597+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:21:44.123597+0000","last_peered":"2023-07-14T11:21:44.123597+0000","last_clean":"2023-07-14T11:21:44.123597+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:21:44.123597+0000","last_undegraded":"2023-07-14T11:21:44.123597+0000","last_fullsized":"2023-07-14T11:21:44.123597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:13:16.901921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.127676+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:21:44.127676+0000","last_peered":"2023-07-14T11:21:44.127676+0000","last_clean":"2023-07-14T11:21:44.127676+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:21:44.127676+0000","last_undegraded":"2023-07-14T11:21:44.127676+0000","last_fullsized":"2023-07-14T11:21:44.127676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:06:24.239682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130965+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:21:44.130965+0000","last_peered":"2023-07-14T11:21:44.130965+0000","last_clean":"2023-07-14T11:21:44.130965+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:21:44.130965+0000","last_undegraded":"2023-07-14T11:21:44.130965+0000","last_fullsized":"2023-07-14T11:21:44.130965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:38:17.082600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135133+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:21:44.135133+0000","last_peered":"2023-07-14T11:21:44.135133+0000","last_clean":"2023-07-14T11:21:44.135133+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:21:44.135133+0000","last_undegraded":"2023-07-14T11:21:44.135133+0000","last_fullsized":"2023-07-14T11:21:44.135133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:19.948875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132889+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:21:44.132889+0000","last_peered":"2023-07-14T11:21:44.132889+0000","last_clean":"2023-07-14T11:21:44.132889+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:21:44.132889+0000","last_undegraded":"2023-07-14T11:21:44.132889+0000","last_fullsized":"2023-07-14T11:21:44.132889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:24.946793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120491+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:21:44.120491+0000","last_peered":"2023-07-14T11:21:44.120491+0000","last_clean":"2023-07-14T11:21:44.120491+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:21:44.120491+0000","last_undegraded":"2023-07-14T11:21:44.120491+0000","last_fullsized":"2023-07-14T11:21:44.120491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:33:36.799060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.122053+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:21:44.122053+0000","last_peered":"2023-07-14T11:21:44.122053+0000","last_clean":"2023-07-14T11:21:44.122053+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:21:44.122053+0000","last_undegraded":"2023-07-14T11:21:44.122053+0000","last_fullsized":"2023-07-14T11:21:44.122053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:55.929013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134984+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:21:44.134984+0000","last_peered":"2023-07-14T11:21:44.134984+0000","last_clean":"2023-07-14T11:21:44.134984+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:21:44.134984+0000","last_undegraded":"2023-07-14T11:21:44.134984+0000","last_fullsized":"2023-07-14T11:21:44.134984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:58.128316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.118051+0000","last_change":"2023-07-14T11:20:15.765762+0000","last_active":"2023-07-14T11:21:44.118051+0000","last_peered":"2023-07-14T11:21:44.118051+0000","last_clean":"2023-07-14T11:21:44.118051+0000","last_became_active":"2023-07-14T11:20:15.765349+0000","last_became_peered":"2023-07-14T11:20:15.765349+0000","last_unstale":"2023-07-14T11:21:44.118051+0000","last_undegraded":"2023-07-14T11:21:44.118051+0000","last_fullsized":"2023-07-14T11:21:44.118051+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:11:07.179820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131580+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:21:44.131580+0000","last_peered":"2023-07-14T11:21:44.131580+0000","last_clean":"2023-07-14T11:21:44.131580+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:21:44.131580+0000","last_undegraded":"2023-07-14T11:21:44.131580+0000","last_fullsized":"2023-07-14T11:21:44.131580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:48.020842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133597+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:21:44.133597+0000","last_peered":"2023-07-14T11:21:44.133597+0000","last_clean":"2023-07-14T11:21:44.133597+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:21:44.133597+0000","last_undegraded":"2023-07-14T11:21:44.133597+0000","last_fullsized":"2023-07-14T11:21:44.133597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:23.156838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134494+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:21:44.134494+0000","last_peered":"2023-07-14T11:21:44.134494+0000","last_clean":"2023-07-14T11:21:44.134494+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:21:44.134494+0000","last_undegraded":"2023-07-14T11:21:44.134494+0000","last_fullsized":"2023-07-14T11:21:44.134494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:03:17.632302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130072+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:21:44.130072+0000","last_peered":"2023-07-14T11:21:44.130072+0000","last_clean":"2023-07-14T11:21:44.130072+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:21:44.130072+0000","last_undegraded":"2023-07-14T11:21:44.130072+0000","last_fullsized":"2023-07-14T11:21:44.130072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:27:11.918619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66428,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":400,"num_write":160,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416425,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":510275,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.35999999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.52800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86699999999999999,"5min":1.097,"15min":1.097},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69099999999999995,"5min":0.75,"15min":0.75},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92000000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79400000000000004,"5min":1.042,"15min":1.042},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52500000000000002,"15min":0.50800000000000001},"min":{"1min":0.42799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.623},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89900000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.55700000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94799999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.72599999999999998},"min":{"1min":0.68799999999999994,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.02,"5min":1.159,"15min":1.159},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.92500000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73799999999999999,"15min":0.73899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":1.014,"15min":1.014},"last":0.97599999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.751,"15min":0.749},"min":{"1min":0.71899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.042,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.01,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.01}]},{"osd":10,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.96799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0640000000000001,"5min":1.456,"15min":1.456},"last":0.86599999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1150976,"data_stored":1160263,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.84899999999999998,"15min":0.89000000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.746,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.83599999999999997,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.877,"5min":6.0090000000000003,"15min":6.0090000000000003},"last":0.621}]},{"osd":1,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.52900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.76700000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67000000000000004,"15min":0.69699999999999995},"min":{"1min":0.42699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73599999999999999,"5min":2.46,"15min":2.46},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.66800000000000004},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77200000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.66500000000000004,"15min":0.69699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.496},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.75,"15min":0.745},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82899999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.42099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79200000000000004,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89900000000000002,"5min":2.431,"15min":2.431},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.59099999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.88300000000000001,"15min":0.84699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":9.5269999999999992,"5min":9.5269999999999992,"15min":9.5269999999999992},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.83699999999999997,"15min":0.81999999999999995},"min":{"1min":0.65000000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.7709999999999999,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.85499999999999998,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":2.536,"15min":2.536},"last":0.70199999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.69699999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.754},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.97299999999999998,"15min":0.97499999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":6.1559999999999997,"5min":6.1559999999999997,"15min":6.1559999999999997},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86799999999999999,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95999999999999996,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.70299999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.054},"last":0.68500000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":514461,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.38}]},{"osd":1,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":1.0289999999999999},"min":{"1min":0.51500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.355,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.439},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0620000000000001,"5min":1.073,"15min":1.073},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85099999999999998,"15min":0.877},"min":{"1min":0.53900000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1679999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88400000000000001,"15min":0.92200000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.65400000000000003}]},{"osd":8,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.077,"15min":1.1850000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3210000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.129,"5min":1.171,"15min":1.171},"last":0.628}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.503},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.4489999999999998,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.70999999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91500000000000004,"15min":0.93899999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.04,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.88600000000000001,"15min":0.90100000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0269999999999999,"5min":1.141,"15min":1.141},"last":0.80300000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":510173,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.50700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.436,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.68100000000000005,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.93000000000000005,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48499999999999999,"15min":0.44800000000000001},"min":{"1min":0.40600000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.008,"5min":2.032,"15min":2.032},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.998,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.998}]},{"osd":4,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.64400000000000002,"15min":0.59899999999999998},"min":{"1min":0.36099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.0329999999999999,"5min":1.046,"15min":1.046},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.69299999999999995,"15min":0.64100000000000001},"min":{"1min":0.68400000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.437,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":1.038,"15min":1.038},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.76000000000000001,"15min":0.72599999999999998},"min":{"1min":0.56699999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.048,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.048},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.68999999999999995},"min":{"1min":0.504,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.90100000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.71599999999999997,"15min":0.67700000000000005},"min":{"1min":0.55900000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.95799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.751,"15min":0.71699999999999997},"min":{"1min":0.63100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.02},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.80800000000000005,"15min":0.77400000000000002},"min":{"1min":0.69599999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.1299999999999999,"5min":1.319,"15min":1.319},"last":1.0800000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.70499999999999996},"min":{"1min":0.65100000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.79900000000000004,"15min":0.749},"min":{"1min":0.71199999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.093,"5min":1.218,"15min":1.218},"last":1.105}]},{"osd":11,"last update":"Fri Jul 14 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.84699999999999998,"15min":0.84499999999999997},"min":{"1min":0.64600000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.077,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.94599999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":1105376,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.32600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.376},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.34000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.378,"15min":0.378},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.621,"15min":0.58699999999999997},"min":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.2450000000000001,"5min":4.2450000000000001,"15min":4.2450000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.36299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85199999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.629}]},{"osd":6,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.75600000000000001,"15min":0.73599999999999999},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.3390000000000004,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69099999999999995,"15min":0.68600000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.51200000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.746,"15min":0.69999999999999996},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":2.762,"15min":2.762},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.51700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.075,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.752,"15min":0.748},"min":{"1min":0.56399999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.754}]},{"osd":10,"last update":"Fri Jul 14 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.75800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.501,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.052,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70599999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":505268,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.51900000000000002,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":0.378},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.55400000000000005,"15min":0.497},"min":{"1min":0.33000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.1180000000000003,"5min":5.1180000000000003,"15min":5.1180000000000003},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.65800000000000003},"min":{"1min":0.55800000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.86899999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.2360000000000002,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.71099999999999997,"15min":0.66600000000000004},"min":{"1min":0.64200000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68100000000000005,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.443,"15min":0.443},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.77400000000000002,"15min":0.753},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.76200000000000001,"15min":0.69999999999999996},"min":{"1min":0.57599999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.2730000000000001,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.73599999999999999},"min":{"1min":0.55900000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3009999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73799999999999999,"15min":0.72299999999999998},"min":{"1min":0.52800000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.745},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.78100000000000003},"min":{"1min":0.621,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Jul 14 11:21:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.81299999999999994,"15min":0.80300000000000005},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.83199999999999996,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0620000000000001,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.78300000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84699999999999998,"15min":0.85799999999999998},"min":{"1min":0.495,"5min":0.442,"15min":0.442},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.89800000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93400000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Jul 14 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.234,"5min":1.359,"15min":1.359},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.998,"15min":1.03},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3029999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":562986,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.32400000000000001},"max":{"1min":4.077,"5min":4.077,"15min":4.077},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61199999999999999,"15min":0.51600000000000001},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.625,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.32200000000000001},"max":{"1min":0.82599999999999996,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70199999999999996,"15min":0.64900000000000002},"min":{"1min":0.44700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":4.016,"5min":4.016,"15min":4.016},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51900000000000002,"15min":0.47199999999999998},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68899999999999995,"15min":0.63400000000000001},"min":{"1min":0.54500000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67700000000000005,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.495,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.1459999999999999,"5min":4.1459999999999999,"15min":4.1459999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.69199999999999995},"min":{"1min":0.59499999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.63600000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97699999999999998,"5min":1.036,"15min":1.036},"last":0.95899999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79100000000000004,"15min":0.78600000000000003},"min":{"1min":0.58699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.88300000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.439,"15min":0.439},"max":{"1min":0.98699999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.208}]},{"osd":11,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.016},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1,"5min":1.044,"15min":1.044},"last":1.0349999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":506656,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65400000000000003,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":0.504},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.33900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.59299999999999997},"min":{"1min":0.45600000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.873,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.97499999999999998,"5min":5.2060000000000004,"15min":5.2060000000000004},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.71599999999999997,"15min":0.61799999999999999},"min":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6680000000000001,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.57299999999999995},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90200000000000002,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72299999999999998,"15min":0.63900000000000001},"min":{"1min":0.51200000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.66800000000000004},"min":{"1min":0.48899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.996,"5min":5.2489999999999997,"15min":5.2489999999999997},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64700000000000002,"15min":0.63200000000000001},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.748,"15min":0.67600000000000005},"min":{"1min":0.442,"5min":0.38,"15min":0.38},"max":{"1min":6.3150000000000004,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.91900000000000004},"min":{"1min":0.60999999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.88600000000000001,"15min":0.91700000000000004},"min":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92800000000000005,"5min":5.351,"15min":5.351},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.84399999999999997,"15min":0.86299999999999999},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78300000000000003,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.036,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.748}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85499999999999998,"15min":0.89300000000000002},"min":{"1min":0.58899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.577,"15min":1.577},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.075,"5min":0.93000000000000005,"15min":0.92900000000000005},"min":{"1min":0.62,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.2720000000000002,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.64000000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.89300000000000002,"15min":0.86899999999999999},"min":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":5.7080000000000002,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.86299999999999999,"15min":0.88800000000000001},"min":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.018,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.58199999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.85399999999999998,"15min":0.81599999999999995},"min":{"1min":0.628,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.88600000000000001,"15min":0.84399999999999997},"min":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":0.81699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":563981,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72699999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":1.012,"15min":1.012},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.628,"15min":0.54400000000000004},"min":{"1min":0.57799999999999996,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.317,"5min":0.254,"15min":0.254},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.50600000000000001},"min":{"1min":0.45500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64800000000000002,"15min":0.626},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.50900000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.70799999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66400000000000003,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Jul 14 11:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.75900000000000001},"min":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65300000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.505,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90100000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.69699999999999995,"15min":0.67300000000000004},"min":{"1min":0.622,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89400000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.748,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.54300000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93899999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56399999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":505735,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.52800000000000002,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.4649999999999999,"5min":7.4649999999999999,"15min":7.4649999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.54700000000000004,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":6.4809999999999999,"5min":6.4809999999999999,"15min":6.4809999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.71699999999999997,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.7329999999999997,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.73799999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":6.335,"5min":6.335,"15min":6.335},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60399999999999998,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.69999999999999996,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.73799999999999999,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":6.242,"5min":6.242,"15min":6.242},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77700000000000002,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.133,"5min":6.8570000000000002,"15min":6.8570000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.65300000000000002},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.663,"15min":1.663},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:21:09 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.746,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.77500000000000002,"15min":0.69799999999999995},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.5140000000000002,"5min":6.5140000000000002,"15min":6.5140000000000002},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:32 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.83199999999999996,"15min":0.755},"min":{"1min":0.57199999999999995,"5min":0.505,"15min":0.505},"max":{"1min":6.415,"5min":6.415,"15min":6.415},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.63400000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":0.94799999999999995,"15min":0.89200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":13.365,"5min":13.365,"15min":13.365},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.048,"5min":0.98899999999999999,"15min":1.018},"min":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.4489999999999998,"5min":6.4489999999999998,"15min":6.4489999999999998},"last":0.68700000000000006}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":6.9649999999999999,"5min":6.9649999999999999,"15min":6.9649999999999999},"last":0.751},{"interface":"front","average":{"1min":1.069,"5min":0.86699999999999999,"15min":0.84399999999999997},"min":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.55800000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93700000000000006,"5min":1.7,"15min":1.7},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.0780000000000001,"15min":1.1200000000000001},"min":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1759999999999999,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.80800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":510143,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57399999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.432},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.40000000000000002},"min":{"1min":0.28199999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86399999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.745},{"interface":"front","average":{"1min":0.629,"5min":0.60099999999999998,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84199999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.69899999999999995,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3289999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":5,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.40100000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82699999999999996,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.622,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.745,"15min":0.72299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.7779999999999996,"5min":5.7779999999999996,"15min":5.7779999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.496,"5min":0.438,"15min":0.438},"max":{"1min":5.7309999999999999,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:21:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.47999999999999998},"min":{"1min":0.32100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77300000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68300000000000005,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91800000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.628,"15min":0.59099999999999997},"min":{"1min":0.54800000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.73099999999999998,"15min":0.69899999999999995},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.262,"5min":0.85899999999999999,"15min":0.80500000000000005},"min":{"1min":0.48199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.2640000000000002,"5min":6.2640000000000002,"15min":6.2640000000000002},"last":0.84499999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.78600000000000003},"min":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79600000000000004,"15min":0.79200000000000004},"min":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97499999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.82399999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1073152,"data_stored":1101012,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58799999999999997,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77300000000000002,"5min":2.843,"15min":2.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67300000000000004,"5min":6.6680000000000001,"15min":6.6680000000000001},"last":0.379}]},{"osd":1,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.58199999999999996,"15min":0.503},"min":{"1min":0.49099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.51700000000000002},"min":{"1min":0.46200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.504}]},{"osd":2,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.36899999999999999},"min":{"1min":0.34899999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.438,"15min":0.39500000000000002},"min":{"1min":0.36699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.622,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.59299999999999997},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.747,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61499999999999999,"15min":0.55400000000000005},"min":{"1min":0.52100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94799999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.55000000000000004},"min":{"1min":0.55000000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.68200000000000005,"15min":0.60999999999999999},"min":{"1min":0.54400000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.65100000000000002,"15min":0.60499999999999998},"min":{"1min":0.49399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:21:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64000000000000001,"15min":0.60199999999999998},"min":{"1min":0.59699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.625},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.52600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.038},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82899999999999996,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67100000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.751,"15min":0.72599999999999998},"min":{"1min":0.61899999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.77400000000000002,"15min":0.75},"min":{"1min":0.63,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73099999999999998,"15min":0.71399999999999997},"min":{"1min":0.60299999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.748,"15min":0.72399999999999998},"min":{"1min":0.67800000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.996,"5min":1.073,"15min":1.073},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:22:01.558 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-14T11:22:01.558 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:22:01.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:01 smithi175 ceph-mon[129081]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:01.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:01 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2189129385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:01.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:01 smithi182 ceph-mon[129366]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:01.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:01 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2189129385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:01.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 -- 172.21.15.161:0/940737551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd214101680 msgr2=0x7fd214107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 --2- 172.21.15.161:0/940737551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd214101680 0x7fd214107f40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b0a0 tx=0x7fd20c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 -- 172.21.15.161:0/940737551 shutdown_connections 2023-07-14T11:22:01.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 --2- 172.21.15.161:0/940737551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd214101680 0x7fd214107f40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 --2- 172.21.15.161:0/940737551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2140ffb00 0x7fd2140fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 --2- 172.21.15.161:0/940737551 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd214100cf0 0x7fd214101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 -- 172.21.15.161:0/940737551 >> 172.21.15.161:0/940737551 conn(0x7fd2140fb0e0 msgr2=0x7fd2140fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:01.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.784+0000 7fd218dff700 1 -- 172.21.15.161:0/940737551 shutdown_connections 2023-07-14T11:22:01.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.785+0000 7fd218dff700 1 -- 172.21.15.161:0/940737551 wait complete. 2023-07-14T11:22:01.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 Processor -- start 2023-07-14T11:22:01.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 -- start start 2023-07-14T11:22:01.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2140ffb00 0x7fd214195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd214100cf0 0x7fd214195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd214101680 0x7fd21419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21419d4c0 con 0x7fd2140ffb00 2023-07-14T11:22:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21419d600 con 0x7fd214101680 2023-07-14T11:22:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd218dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21419d790 con 0x7fd214100cf0 2023-07-14T11:22:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd21259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2140ffb00 0x7fd214195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd21259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2140ffb00 0x7fd214195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39698/0 (socket says 172.21.15.161:39698) 2023-07-14T11:22:01.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd21259c700 1 -- 172.21.15.161:0/1076860645 learned_addr learned my addr 172.21.15.161:0/1076860645 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:01.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.786+0000 7fd211d9b700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd214100cf0 0x7fd214195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd212d9d700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd214101680 0x7fd21419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd21259c700 1 -- 172.21.15.161:0/1076860645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd214100cf0 msgr2=0x7fd214195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd21259c700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd214100cf0 0x7fd214195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd21259c700 1 -- 172.21.15.161:0/1076860645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd214101680 msgr2=0x7fd21419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:01.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd21259c700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd214101680 0x7fd21419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd21259c700 1 -- 172.21.15.161:0/1076860645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c009d00 con 0x7fd2140ffb00 2023-07-14T11:22:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd21259c700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2140ffb00 0x7fd214195940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e970 tx=0x7fd20400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd2037fe700 1 -- 172.21.15.161:0/1076860645 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204009790 con 0x7fd2140ffb00 2023-07-14T11:22:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd2037fe700 1 -- 172.21.15.161:0/1076860645 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd20400c7e0 con 0x7fd2140ffb00 2023-07-14T11:22:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21419da70 con 0x7fd2140ffb00 2023-07-14T11:22:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd2037fe700 1 -- 172.21.15.161:0/1076860645 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204003c70 con 0x7fd2140ffb00 2023-07-14T11:22:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.787+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21419dfd0 con 0x7fd2140ffb00 2023-07-14T11:22:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.788+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21410b330 con 0x7fd2140ffb00 2023-07-14T11:22:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.791+0000 7fd2037fe700 1 -- 172.21.15.161:0/1076860645 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd20400cc50 con 0x7fd2140ffb00 2023-07-14T11:22:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.792+0000 7fd2037fe700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1fc0a1220 0x7fd1fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:01.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.792+0000 7fd2037fe700 1 -- 172.21.15.161:0/1076860645 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd204014070 con 0x7fd2140ffb00 2023-07-14T11:22:01.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.792+0000 7fd211d9b700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1fc0a1220 0x7fd1fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:01.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.793+0000 7fd211d9b700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1fc0a1220 0x7fd1fc0a36d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd208009c60 tx=0x7fd208009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:01.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.795+0000 7fd2037fe700 1 -- 172.21.15.161:0/1076860645 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2040961a0 con 0x7fd2140ffb00 2023-07-14T11:22:01.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:01.989+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd214060950 con 0x7fd1fc0a1220 2023-07-14T11:22:02.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.002+0000 7fd2037fe700 1 -- 172.21.15.161:0/1076860645 <== mgr.14148 v2:172.21.15.161:6800/2004305703 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426827 (secure 0 0 0) 0x7fd214060950 con 0x7fd1fc0a1220 2023-07-14T11:22:02.009 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:02.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1fc0a1220 msgr2=0x7fd1fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1fc0a1220 0x7fd1fc0a36d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd208009c60 tx=0x7fd208009260 comp rx=0 tx=0).stop 2023-07-14T11:22:02.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2140ffb00 msgr2=0x7fd214195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2140ffb00 0x7fd214195940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e970 tx=0x7fd20400ec80 comp rx=0 tx=0).stop 2023-07-14T11:22:02.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 shutdown_connections 2023-07-14T11:22:02.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2140ffb00 0x7fd214195940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd1fc0a1220 0x7fd1fc0a36d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd214101680 0x7fd21419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 --2- 172.21.15.161:0/1076860645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd214100cf0 0x7fd214195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 >> 172.21.15.161:0/1076860645 conn(0x7fd2140fb0e0 msgr2=0x7fd2140fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.007+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 shutdown_connections 2023-07-14T11:22:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.008+0000 7fd218dff700 1 -- 172.21.15.161:0/1076860645 wait complete. 2023-07-14T11:22:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-07-14T11:22:02.029 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2023-07-14T11:22:00.132430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657042,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":795,"num_read_kb":519,"num_write":343,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":779,"ondisk_log_size":779,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335648,"kb_used_data":12224,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606176,"statfs":{"total":1151940427776,"available":1151596724224,"internally_reserved":0,"allocated":12517376,"data_stored":8056329,"data_compressed":390111,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":41,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132890+0000","last_change":"2023-07-14T11:19:42.082032+0000","last_active":"2023-07-14T11:21:44.132890+0000","last_peered":"2023-07-14T11:21:44.132890+0000","last_clean":"2023-07-14T11:21:44.132890+0000","last_became_active":"2023-07-14T11:19:42.081419+0000","last_became_peered":"2023-07-14T11:19:42.081419+0000","last_unstale":"2023-07-14T11:21:44.132890+0000","last_undegraded":"2023-07-14T11:21:44.132890+0000","last_fullsized":"2023-07-14T11:21:44.132890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:45:53.898814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131698+0000","last_change":"2023-07-14T11:19:42.086118+0000","last_active":"2023-07-14T11:21:44.131698+0000","last_peered":"2023-07-14T11:21:44.131698+0000","last_clean":"2023-07-14T11:21:44.131698+0000","last_became_active":"2023-07-14T11:19:42.084212+0000","last_became_peered":"2023-07-14T11:19:42.084212+0000","last_unstale":"2023-07-14T11:21:44.131698+0000","last_undegraded":"2023-07-14T11:21:44.131698+0000","last_fullsized":"2023-07-14T11:21:44.131698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:27:43.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120443+0000","last_change":"2023-07-14T11:19:42.090978+0000","last_active":"2023-07-14T11:21:44.120443+0000","last_peered":"2023-07-14T11:21:44.120443+0000","last_clean":"2023-07-14T11:21:44.120443+0000","last_became_active":"2023-07-14T11:19:42.090623+0000","last_became_peered":"2023-07-14T11:19:42.090623+0000","last_unstale":"2023-07-14T11:21:44.120443+0000","last_undegraded":"2023-07-14T11:21:44.120443+0000","last_fullsized":"2023-07-14T11:21:44.120443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:30:20.721165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133183+0000","last_change":"2023-07-14T11:19:42.077586+0000","last_active":"2023-07-14T11:21:44.133183+0000","last_peered":"2023-07-14T11:21:44.133183+0000","last_clean":"2023-07-14T11:21:44.133183+0000","last_became_active":"2023-07-14T11:19:42.077137+0000","last_became_peered":"2023-07-14T11:19:42.077137+0000","last_unstale":"2023-07-14T11:21:44.133183+0000","last_undegraded":"2023-07-14T11:21:44.133183+0000","last_fullsized":"2023-07-14T11:21:44.133183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:05:47.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.124390+0000","last_change":"2023-07-14T11:19:42.079031+0000","last_active":"2023-07-14T11:21:44.124390+0000","last_peered":"2023-07-14T11:21:44.124390+0000","last_clean":"2023-07-14T11:21:44.124390+0000","last_became_active":"2023-07-14T11:19:42.078673+0000","last_became_peered":"2023-07-14T11:19:42.078673+0000","last_unstale":"2023-07-14T11:21:44.124390+0000","last_undegraded":"2023-07-14T11:21:44.124390+0000","last_fullsized":"2023-07-14T11:21:44.124390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:03.043850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133900+0000","last_change":"2023-07-14T11:20:15.765846+0000","last_active":"2023-07-14T11:21:44.133900+0000","last_peered":"2023-07-14T11:21:44.133900+0000","last_clean":"2023-07-14T11:21:44.133900+0000","last_became_active":"2023-07-14T11:20:15.765317+0000","last_became_peered":"2023-07-14T11:20:15.765317+0000","last_unstale":"2023-07-14T11:21:44.133900+0000","last_undegraded":"2023-07-14T11:21:44.133900+0000","last_fullsized":"2023-07-14T11:21:44.133900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:02.922415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128281+0000","last_change":"2023-07-14T11:19:42.071682+0000","last_active":"2023-07-14T11:21:44.128281+0000","last_peered":"2023-07-14T11:21:44.128281+0000","last_clean":"2023-07-14T11:21:44.128281+0000","last_became_active":"2023-07-14T11:19:42.071387+0000","last_became_peered":"2023-07-14T11:19:42.071387+0000","last_unstale":"2023-07-14T11:21:44.128281+0000","last_undegraded":"2023-07-14T11:21:44.128281+0000","last_fullsized":"2023-07-14T11:21:44.128281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:20:08.848005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132015+0000","last_change":"2023-07-14T11:19:42.083565+0000","last_active":"2023-07-14T11:21:44.132015+0000","last_peered":"2023-07-14T11:21:44.132015+0000","last_clean":"2023-07-14T11:21:44.132015+0000","last_became_active":"2023-07-14T11:19:42.083256+0000","last_became_peered":"2023-07-14T11:19:42.083256+0000","last_unstale":"2023-07-14T11:21:44.132015+0000","last_undegraded":"2023-07-14T11:21:44.132015+0000","last_fullsized":"2023-07-14T11:21:44.132015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:54.048999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129256+0000","last_change":"2023-07-14T11:19:42.071424+0000","last_active":"2023-07-14T11:21:44.129256+0000","last_peered":"2023-07-14T11:21:44.129256+0000","last_clean":"2023-07-14T11:21:44.129256+0000","last_became_active":"2023-07-14T11:19:42.071145+0000","last_became_peered":"2023-07-14T11:19:42.071145+0000","last_unstale":"2023-07-14T11:21:44.129256+0000","last_undegraded":"2023-07-14T11:21:44.129256+0000","last_fullsized":"2023-07-14T11:21:44.129256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:25.896648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131299+0000","last_change":"2023-07-14T11:19:42.083978+0000","last_active":"2023-07-14T11:21:44.131299+0000","last_peered":"2023-07-14T11:21:44.131299+0000","last_clean":"2023-07-14T11:21:44.131299+0000","last_became_active":"2023-07-14T11:19:42.083673+0000","last_became_peered":"2023-07-14T11:19:42.083673+0000","last_unstale":"2023-07-14T11:21:44.131299+0000","last_undegraded":"2023-07-14T11:21:44.131299+0000","last_fullsized":"2023-07-14T11:21:44.131299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:00.377250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.126348+0000","last_change":"2023-07-14T11:19:42.074075+0000","last_active":"2023-07-14T11:21:44.126348+0000","last_peered":"2023-07-14T11:21:44.126348+0000","last_clean":"2023-07-14T11:21:44.126348+0000","last_became_active":"2023-07-14T11:19:42.073789+0000","last_became_peered":"2023-07-14T11:19:42.073789+0000","last_unstale":"2023-07-14T11:21:44.126348+0000","last_undegraded":"2023-07-14T11:21:44.126348+0000","last_fullsized":"2023-07-14T11:21:44.126348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:51.642943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133671+0000","last_change":"2023-07-14T11:19:42.078220+0000","last_active":"2023-07-14T11:21:44.133671+0000","last_peered":"2023-07-14T11:21:44.133671+0000","last_clean":"2023-07-14T11:21:44.133671+0000","last_became_active":"2023-07-14T11:19:42.077914+0000","last_became_peered":"2023-07-14T11:19:42.077914+0000","last_unstale":"2023-07-14T11:21:44.133671+0000","last_undegraded":"2023-07-14T11:21:44.133671+0000","last_fullsized":"2023-07-14T11:21:44.133671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:40:35.925610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130954+0000","last_change":"2023-07-14T11:19:42.081211+0000","last_active":"2023-07-14T11:21:44.130954+0000","last_peered":"2023-07-14T11:21:44.130954+0000","last_clean":"2023-07-14T11:21:44.130954+0000","last_became_active":"2023-07-14T11:19:42.080846+0000","last_became_peered":"2023-07-14T11:19:42.080846+0000","last_unstale":"2023-07-14T11:21:44.130954+0000","last_undegraded":"2023-07-14T11:21:44.130954+0000","last_fullsized":"2023-07-14T11:21:44.130954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:46:47.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129701+0000","last_change":"2023-07-14T11:19:42.088531+0000","last_active":"2023-07-14T11:21:44.129701+0000","last_peered":"2023-07-14T11:21:44.129701+0000","last_clean":"2023-07-14T11:21:44.129701+0000","last_became_active":"2023-07-14T11:19:42.088241+0000","last_became_peered":"2023-07-14T11:19:42.088241+0000","last_unstale":"2023-07-14T11:21:44.129701+0000","last_undegraded":"2023-07-14T11:21:44.129701+0000","last_fullsized":"2023-07-14T11:21:44.129701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:40:58.143556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134018+0000","last_change":"2023-07-14T11:19:42.081757+0000","last_active":"2023-07-14T11:21:44.134018+0000","last_peered":"2023-07-14T11:21:44.134018+0000","last_clean":"2023-07-14T11:21:44.134018+0000","last_became_active":"2023-07-14T11:19:42.081403+0000","last_became_peered":"2023-07-14T11:19:42.081403+0000","last_unstale":"2023-07-14T11:21:44.134018+0000","last_undegraded":"2023-07-14T11:21:44.134018+0000","last_fullsized":"2023-07-14T11:21:44.134018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:24.775441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.127674+0000","last_change":"2023-07-14T11:19:42.082821+0000","last_active":"2023-07-14T11:21:44.127674+0000","last_peered":"2023-07-14T11:21:44.127674+0000","last_clean":"2023-07-14T11:21:44.127674+0000","last_became_active":"2023-07-14T11:19:42.082477+0000","last_became_peered":"2023-07-14T11:19:42.082477+0000","last_unstale":"2023-07-14T11:21:44.127674+0000","last_undegraded":"2023-07-14T11:21:44.127674+0000","last_fullsized":"2023-07-14T11:21:44.127674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:49.835385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132947+0000","last_change":"2023-07-14T11:19:42.081116+0000","last_active":"2023-07-14T11:21:44.132947+0000","last_peered":"2023-07-14T11:21:44.132947+0000","last_clean":"2023-07-14T11:21:44.132947+0000","last_became_active":"2023-07-14T11:19:42.080801+0000","last_became_peered":"2023-07-14T11:19:42.080801+0000","last_unstale":"2023-07-14T11:21:44.132947+0000","last_undegraded":"2023-07-14T11:21:44.132947+0000","last_fullsized":"2023-07-14T11:21:44.132947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:58:04.718574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.114499+0000","last_change":"2023-07-14T11:19:42.083154+0000","last_active":"2023-07-14T11:21:44.114499+0000","last_peered":"2023-07-14T11:21:44.114499+0000","last_clean":"2023-07-14T11:21:44.114499+0000","last_became_active":"2023-07-14T11:19:42.082701+0000","last_became_peered":"2023-07-14T11:19:42.082701+0000","last_unstale":"2023-07-14T11:21:44.114499+0000","last_undegraded":"2023-07-14T11:21:44.114499+0000","last_fullsized":"2023-07-14T11:21:44.114499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:45.295320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129091+0000","last_change":"2023-07-14T11:19:42.073649+0000","last_active":"2023-07-14T11:21:44.129091+0000","last_peered":"2023-07-14T11:21:44.129091+0000","last_clean":"2023-07-14T11:21:44.129091+0000","last_became_active":"2023-07-14T11:19:42.073328+0000","last_became_peered":"2023-07-14T11:19:42.073328+0000","last_unstale":"2023-07-14T11:21:44.129091+0000","last_undegraded":"2023-07-14T11:21:44.129091+0000","last_fullsized":"2023-07-14T11:21:44.129091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:38:25.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761984+0000","last_change":"2023-07-14T11:19:46.099811+0000","last_active":"2023-07-14T11:21:16.761984+0000","last_peered":"2023-07-14T11:21:16.761984+0000","last_clean":"2023-07-14T11:21:16.761984+0000","last_became_active":"2023-07-14T11:19:46.099388+0000","last_became_peered":"2023-07-14T11:19:46.099388+0000","last_unstale":"2023-07-14T11:21:16.761984+0000","last_undegraded":"2023-07-14T11:21:16.761984+0000","last_fullsized":"2023-07-14T11:21:16.761984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:17:32.149825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.757056+0000","last_change":"2023-07-14T11:19:41.105263+0000","last_active":"2023-07-14T11:21:16.757056+0000","last_peered":"2023-07-14T11:21:16.757056+0000","last_clean":"2023-07-14T11:21:16.757056+0000","last_became_active":"2023-07-14T11:19:41.104703+0000","last_became_peered":"2023-07-14T11:19:41.104703+0000","last_unstale":"2023-07-14T11:21:16.757056+0000","last_undegraded":"2023-07-14T11:21:16.757056+0000","last_fullsized":"2023-07-14T11:21:16.757056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:27:34.767964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128034+0000","last_change":"2023-07-14T11:19:42.082729+0000","last_active":"2023-07-14T11:21:44.128034+0000","last_peered":"2023-07-14T11:21:44.128034+0000","last_clean":"2023-07-14T11:21:44.128034+0000","last_became_active":"2023-07-14T11:19:42.082438+0000","last_became_peered":"2023-07-14T11:19:42.082438+0000","last_unstale":"2023-07-14T11:21:44.128034+0000","last_undegraded":"2023-07-14T11:21:44.128034+0000","last_fullsized":"2023-07-14T11:21:44.128034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:23:39.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782856+0000","last_change":"2023-07-14T11:19:46.098803+0000","last_active":"2023-07-14T11:21:16.782856+0000","last_peered":"2023-07-14T11:21:16.782856+0000","last_clean":"2023-07-14T11:21:16.782856+0000","last_became_active":"2023-07-14T11:19:46.098377+0000","last_became_peered":"2023-07-14T11:19:46.098377+0000","last_unstale":"2023-07-14T11:21:16.782856+0000","last_undegraded":"2023-07-14T11:21:16.782856+0000","last_fullsized":"2023-07-14T11:21:16.782856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:47.727483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761802+0000","last_change":"2023-07-14T11:19:41.108832+0000","last_active":"2023-07-14T11:21:16.761802+0000","last_peered":"2023-07-14T11:21:16.761802+0000","last_clean":"2023-07-14T11:21:16.761802+0000","last_became_active":"2023-07-14T11:19:41.108493+0000","last_became_peered":"2023-07-14T11:19:41.108493+0000","last_unstale":"2023-07-14T11:21:16.761802+0000","last_undegraded":"2023-07-14T11:21:16.761802+0000","last_fullsized":"2023-07-14T11:21:16.761802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:29:59.179624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130426+0000","last_change":"2023-07-14T11:19:42.085457+0000","last_active":"2023-07-14T11:21:44.130426+0000","last_peered":"2023-07-14T11:21:44.130426+0000","last_clean":"2023-07-14T11:21:44.130426+0000","last_became_active":"2023-07-14T11:19:42.082525+0000","last_became_peered":"2023-07-14T11:19:42.082525+0000","last_unstale":"2023-07-14T11:21:44.130426+0000","last_undegraded":"2023-07-14T11:21:44.130426+0000","last_fullsized":"2023-07-14T11:21:44.130426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:18.435015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782915+0000","last_change":"2023-07-14T11:19:46.102928+0000","last_active":"2023-07-14T11:21:16.782915+0000","last_peered":"2023-07-14T11:21:16.782915+0000","last_clean":"2023-07-14T11:21:16.782915+0000","last_became_active":"2023-07-14T11:19:46.102552+0000","last_became_peered":"2023-07-14T11:19:46.102552+0000","last_unstale":"2023-07-14T11:21:16.782915+0000","last_undegraded":"2023-07-14T11:21:16.782915+0000","last_fullsized":"2023-07-14T11:21:16.782915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:27.267522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760533+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:21:16.760533+0000","last_peered":"2023-07-14T11:21:16.760533+0000","last_clean":"2023-07-14T11:21:16.760533+0000","last_became_active":"2023-07-14T11:19:41.111317+0000","last_became_peered":"2023-07-14T11:19:41.111317+0000","last_unstale":"2023-07-14T11:21:16.760533+0000","last_undegraded":"2023-07-14T11:21:16.760533+0000","last_fullsized":"2023-07-14T11:21:16.760533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:28.556454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130958+0000","last_change":"2023-07-14T11:19:42.078805+0000","last_active":"2023-07-14T11:21:44.130958+0000","last_peered":"2023-07-14T11:21:44.130958+0000","last_clean":"2023-07-14T11:21:44.130958+0000","last_became_active":"2023-07-14T11:19:42.078517+0000","last_became_peered":"2023-07-14T11:19:42.078517+0000","last_unstale":"2023-07-14T11:21:44.130958+0000","last_undegraded":"2023-07-14T11:21:44.130958+0000","last_fullsized":"2023-07-14T11:21:44.130958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:23.714644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.782878+0000","last_change":"2023-07-14T11:19:46.093916+0000","last_active":"2023-07-14T11:21:16.782878+0000","last_peered":"2023-07-14T11:21:16.782878+0000","last_clean":"2023-07-14T11:21:16.782878+0000","last_became_active":"2023-07-14T11:19:46.093431+0000","last_became_peered":"2023-07-14T11:19:46.093431+0000","last_unstale":"2023-07-14T11:21:16.782878+0000","last_undegraded":"2023-07-14T11:21:16.782878+0000","last_fullsized":"2023-07-14T11:21:16.782878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:14.385740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759437+0000","last_change":"2023-07-14T11:19:41.106970+0000","last_active":"2023-07-14T11:21:16.759437+0000","last_peered":"2023-07-14T11:21:16.759437+0000","last_clean":"2023-07-14T11:21:16.759437+0000","last_became_active":"2023-07-14T11:19:41.106655+0000","last_became_peered":"2023-07-14T11:19:41.106655+0000","last_unstale":"2023-07-14T11:21:16.759437+0000","last_undegraded":"2023-07-14T11:21:16.759437+0000","last_fullsized":"2023-07-14T11:21:16.759437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:24:07.117370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120539+0000","last_change":"2023-07-14T11:19:42.075980+0000","last_active":"2023-07-14T11:21:44.120539+0000","last_peered":"2023-07-14T11:21:44.120539+0000","last_clean":"2023-07-14T11:21:44.120539+0000","last_became_active":"2023-07-14T11:19:42.075692+0000","last_became_peered":"2023-07-14T11:19:42.075692+0000","last_unstale":"2023-07-14T11:21:44.120539+0000","last_undegraded":"2023-07-14T11:21:44.120539+0000","last_fullsized":"2023-07-14T11:21:44.120539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:01:41.642530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761645+0000","last_change":"2023-07-14T11:19:46.100963+0000","last_active":"2023-07-14T11:21:16.761645+0000","last_peered":"2023-07-14T11:21:16.761645+0000","last_clean":"2023-07-14T11:21:16.761645+0000","last_became_active":"2023-07-14T11:19:46.100523+0000","last_became_peered":"2023-07-14T11:19:46.100523+0000","last_unstale":"2023-07-14T11:21:16.761645+0000","last_undegraded":"2023-07-14T11:21:16.761645+0000","last_fullsized":"2023-07-14T11:21:16.761645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:22:33.821715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":143,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:56.841418+0000","last_change":"2023-07-14T11:19:41.114077+0000","last_active":"2023-07-14T11:21:56.841418+0000","last_peered":"2023-07-14T11:21:56.841418+0000","last_clean":"2023-07-14T11:21:56.841418+0000","last_became_active":"2023-07-14T11:19:41.113762+0000","last_became_peered":"2023-07-14T11:19:41.113762+0000","last_unstale":"2023-07-14T11:21:56.841418+0000","last_undegraded":"2023-07-14T11:21:56.841418+0000","last_fullsized":"2023-07-14T11:21:56.841418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:55:16.898539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.117612+0000","last_change":"2023-07-14T11:19:42.079783+0000","last_active":"2023-07-14T11:21:44.117612+0000","last_peered":"2023-07-14T11:21:44.117612+0000","last_clean":"2023-07-14T11:21:44.117612+0000","last_became_active":"2023-07-14T11:19:42.079442+0000","last_became_peered":"2023-07-14T11:19:42.079442+0000","last_unstale":"2023-07-14T11:21:44.117612+0000","last_undegraded":"2023-07-14T11:21:44.117612+0000","last_fullsized":"2023-07-14T11:21:44.117612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:49:45.959442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.762138+0000","last_change":"2023-07-14T11:20:15.766231+0000","last_active":"2023-07-14T11:21:16.762138+0000","last_peered":"2023-07-14T11:21:16.762138+0000","last_clean":"2023-07-14T11:21:16.762138+0000","last_became_active":"2023-07-14T11:20:15.765718+0000","last_became_peered":"2023-07-14T11:20:15.765718+0000","last_unstale":"2023-07-14T11:21:16.762138+0000","last_undegraded":"2023-07-14T11:21:16.762138+0000","last_fullsized":"2023-07-14T11:21:16.762138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:29:21.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.754843+0000","last_change":"2023-07-14T11:19:41.113142+0000","last_active":"2023-07-14T11:21:16.754843+0000","last_peered":"2023-07-14T11:21:16.754843+0000","last_clean":"2023-07-14T11:21:16.754843+0000","last_became_active":"2023-07-14T11:19:41.112872+0000","last_became_peered":"2023-07-14T11:19:41.112872+0000","last_unstale":"2023-07-14T11:21:16.754843+0000","last_undegraded":"2023-07-14T11:21:16.754843+0000","last_fullsized":"2023-07-14T11:21:16.754843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:42.195752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.126818+0000","last_change":"2023-07-14T11:19:42.078686+0000","last_active":"2023-07-14T11:21:44.126818+0000","last_peered":"2023-07-14T11:21:44.126818+0000","last_clean":"2023-07-14T11:21:44.126818+0000","last_became_active":"2023-07-14T11:19:42.078408+0000","last_became_peered":"2023-07-14T11:19:42.078408+0000","last_unstale":"2023-07-14T11:21:44.126818+0000","last_undegraded":"2023-07-14T11:21:44.126818+0000","last_fullsized":"2023-07-14T11:21:44.126818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:15:54.989039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.758730+0000","last_change":"2023-07-14T11:19:46.103364+0000","last_active":"2023-07-14T11:21:16.758730+0000","last_peered":"2023-07-14T11:21:16.758730+0000","last_clean":"2023-07-14T11:21:16.758730+0000","last_became_active":"2023-07-14T11:19:46.103009+0000","last_became_peered":"2023-07-14T11:19:46.103009+0000","last_unstale":"2023-07-14T11:21:16.758730+0000","last_undegraded":"2023-07-14T11:21:16.758730+0000","last_fullsized":"2023-07-14T11:21:16.758730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:13:50.965885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780547+0000","last_change":"2023-07-14T11:19:41.103212+0000","last_active":"2023-07-14T11:21:16.780547+0000","last_peered":"2023-07-14T11:21:16.780547+0000","last_clean":"2023-07-14T11:21:16.780547+0000","last_became_active":"2023-07-14T11:19:41.102069+0000","last_became_peered":"2023-07-14T11:19:41.102069+0000","last_unstale":"2023-07-14T11:21:16.780547+0000","last_undegraded":"2023-07-14T11:21:16.780547+0000","last_fullsized":"2023-07-14T11:21:16.780547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:20:20.452801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134597+0000","last_change":"2023-07-14T11:19:42.078888+0000","last_active":"2023-07-14T11:21:44.134597+0000","last_peered":"2023-07-14T11:21:44.134597+0000","last_clean":"2023-07-14T11:21:44.134597+0000","last_became_active":"2023-07-14T11:19:42.078468+0000","last_became_peered":"2023-07-14T11:19:42.078468+0000","last_unstale":"2023-07-14T11:21:44.134597+0000","last_undegraded":"2023-07-14T11:21:44.134597+0000","last_fullsized":"2023-07-14T11:21:44.134597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:26:38.989878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761717+0000","last_change":"2023-07-14T11:19:46.091368+0000","last_active":"2023-07-14T11:21:16.761717+0000","last_peered":"2023-07-14T11:21:16.761717+0000","last_clean":"2023-07-14T11:21:16.761717+0000","last_became_active":"2023-07-14T11:19:46.091019+0000","last_became_peered":"2023-07-14T11:19:46.091019+0000","last_unstale":"2023-07-14T11:21:16.761717+0000","last_undegraded":"2023-07-14T11:21:16.761717+0000","last_fullsized":"2023-07-14T11:21:16.761717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:06:57.319545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.779919+0000","last_change":"2023-07-14T11:19:41.112524+0000","last_active":"2023-07-14T11:21:16.779919+0000","last_peered":"2023-07-14T11:21:16.779919+0000","last_clean":"2023-07-14T11:21:16.779919+0000","last_became_active":"2023-07-14T11:19:41.112200+0000","last_became_peered":"2023-07-14T11:19:41.112200+0000","last_unstale":"2023-07-14T11:21:16.779919+0000","last_undegraded":"2023-07-14T11:21:16.779919+0000","last_fullsized":"2023-07-14T11:21:16.779919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:45.971471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132864+0000","last_change":"2023-07-14T11:19:42.090908+0000","last_active":"2023-07-14T11:21:44.132864+0000","last_peered":"2023-07-14T11:21:44.132864+0000","last_clean":"2023-07-14T11:21:44.132864+0000","last_became_active":"2023-07-14T11:19:42.090593+0000","last_became_peered":"2023-07-14T11:19:42.090593+0000","last_unstale":"2023-07-14T11:21:44.132864+0000","last_undegraded":"2023-07-14T11:21:44.132864+0000","last_fullsized":"2023-07-14T11:21:44.132864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:22.629848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760884+0000","last_change":"2023-07-14T11:19:46.101263+0000","last_active":"2023-07-14T11:21:16.760884+0000","last_peered":"2023-07-14T11:21:16.760884+0000","last_clean":"2023-07-14T11:21:16.760884+0000","last_became_active":"2023-07-14T11:19:46.100818+0000","last_became_peered":"2023-07-14T11:19:46.100818+0000","last_unstale":"2023-07-14T11:21:16.760884+0000","last_undegraded":"2023-07-14T11:21:16.760884+0000","last_fullsized":"2023-07-14T11:21:16.760884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:07.800244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:55.839185+0000","last_change":"2023-07-14T11:19:41.112489+0000","last_active":"2023-07-14T11:21:55.839185+0000","last_peered":"2023-07-14T11:21:55.839185+0000","last_clean":"2023-07-14T11:21:55.839185+0000","last_became_active":"2023-07-14T11:19:41.112043+0000","last_became_peered":"2023-07-14T11:19:41.112043+0000","last_unstale":"2023-07-14T11:21:55.839185+0000","last_undegraded":"2023-07-14T11:21:55.839185+0000","last_fullsized":"2023-07-14T11:21:55.839185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:15:34.152726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129759+0000","last_change":"2023-07-14T11:19:42.079437+0000","last_active":"2023-07-14T11:21:44.129759+0000","last_peered":"2023-07-14T11:21:44.129759+0000","last_clean":"2023-07-14T11:21:44.129759+0000","last_became_active":"2023-07-14T11:19:42.079154+0000","last_became_peered":"2023-07-14T11:19:42.079154+0000","last_unstale":"2023-07-14T11:21:44.129759+0000","last_undegraded":"2023-07-14T11:21:44.129759+0000","last_fullsized":"2023-07-14T11:21:44.129759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:56.754661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756218+0000","last_change":"2023-07-14T11:19:46.096268+0000","last_active":"2023-07-14T11:21:16.756218+0000","last_peered":"2023-07-14T11:21:16.756218+0000","last_clean":"2023-07-14T11:21:16.756218+0000","last_became_active":"2023-07-14T11:19:46.095822+0000","last_became_peered":"2023-07-14T11:19:46.095822+0000","last_unstale":"2023-07-14T11:21:16.756218+0000","last_undegraded":"2023-07-14T11:21:16.756218+0000","last_fullsized":"2023-07-14T11:21:16.756218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:28.728921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827514+0000","last_change":"2023-07-14T11:19:41.111332+0000","last_active":"2023-07-14T11:21:17.827514+0000","last_peered":"2023-07-14T11:21:17.827514+0000","last_clean":"2023-07-14T11:21:17.827514+0000","last_became_active":"2023-07-14T11:19:41.111030+0000","last_became_peered":"2023-07-14T11:19:41.111030+0000","last_unstale":"2023-07-14T11:21:17.827514+0000","last_undegraded":"2023-07-14T11:21:17.827514+0000","last_fullsized":"2023-07-14T11:21:17.827514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:16:06.201581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135307+0000","last_change":"2023-07-14T11:19:42.091786+0000","last_active":"2023-07-14T11:21:44.135307+0000","last_peered":"2023-07-14T11:21:44.135307+0000","last_clean":"2023-07-14T11:21:44.135307+0000","last_became_active":"2023-07-14T11:19:42.091401+0000","last_became_peered":"2023-07-14T11:19:42.091401+0000","last_unstale":"2023-07-14T11:21:44.135307+0000","last_undegraded":"2023-07-14T11:21:44.135307+0000","last_fullsized":"2023-07-14T11:21:44.135307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:08.534228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783067+0000","last_change":"2023-07-14T11:19:46.098915+0000","last_active":"2023-07-14T11:21:16.783067+0000","last_peered":"2023-07-14T11:21:16.783067+0000","last_clean":"2023-07-14T11:21:16.783067+0000","last_became_active":"2023-07-14T11:19:46.098516+0000","last_became_peered":"2023-07-14T11:19:46.098516+0000","last_unstale":"2023-07-14T11:21:16.783067+0000","last_undegraded":"2023-07-14T11:21:16.783067+0000","last_fullsized":"2023-07-14T11:21:16.783067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:37.851033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781109+0000","last_change":"2023-07-14T11:19:41.109710+0000","last_active":"2023-07-14T11:21:16.781109+0000","last_peered":"2023-07-14T11:21:16.781109+0000","last_clean":"2023-07-14T11:21:16.781109+0000","last_became_active":"2023-07-14T11:19:41.109374+0000","last_became_peered":"2023-07-14T11:19:41.109374+0000","last_unstale":"2023-07-14T11:21:16.781109+0000","last_undegraded":"2023-07-14T11:21:16.781109+0000","last_fullsized":"2023-07-14T11:21:16.781109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:17:24.738586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133580+0000","last_change":"2023-07-14T11:19:42.077297+0000","last_active":"2023-07-14T11:21:44.133580+0000","last_peered":"2023-07-14T11:21:44.133580+0000","last_clean":"2023-07-14T11:21:44.133580+0000","last_became_active":"2023-07-14T11:19:42.076938+0000","last_became_peered":"2023-07-14T11:19:42.076938+0000","last_unstale":"2023-07-14T11:21:44.133580+0000","last_undegraded":"2023-07-14T11:21:44.133580+0000","last_fullsized":"2023-07-14T11:21:44.133580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:08:23.851839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756241+0000","last_change":"2023-07-14T11:19:46.093473+0000","last_active":"2023-07-14T11:21:16.756241+0000","last_peered":"2023-07-14T11:21:16.756241+0000","last_clean":"2023-07-14T11:21:16.756241+0000","last_became_active":"2023-07-14T11:19:46.093038+0000","last_became_peered":"2023-07-14T11:19:46.093038+0000","last_unstale":"2023-07-14T11:21:16.756241+0000","last_undegraded":"2023-07-14T11:21:16.756241+0000","last_fullsized":"2023-07-14T11:21:16.756241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:29:43.064170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":284,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:56.841699+0000","last_change":"2023-07-14T11:19:41.110107+0000","last_active":"2023-07-14T11:21:56.841699+0000","last_peered":"2023-07-14T11:21:56.841699+0000","last_clean":"2023-07-14T11:21:56.841699+0000","last_became_active":"2023-07-14T11:19:41.109866+0000","last_became_peered":"2023-07-14T11:19:41.109866+0000","last_unstale":"2023-07-14T11:21:56.841699+0000","last_undegraded":"2023-07-14T11:21:56.841699+0000","last_fullsized":"2023-07-14T11:21:56.841699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:27:38.535607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":219,"num_read_kb":171,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.122999+0000","last_change":"2023-07-14T11:19:42.074076+0000","last_active":"2023-07-14T11:21:44.122999+0000","last_peered":"2023-07-14T11:21:44.122999+0000","last_clean":"2023-07-14T11:21:44.122999+0000","last_became_active":"2023-07-14T11:19:42.073793+0000","last_became_peered":"2023-07-14T11:19:42.073793+0000","last_unstale":"2023-07-14T11:21:44.122999+0000","last_undegraded":"2023-07-14T11:21:44.122999+0000","last_fullsized":"2023-07-14T11:21:44.122999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:09:57.788775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781384+0000","last_change":"2023-07-14T11:19:46.098894+0000","last_active":"2023-07-14T11:21:16.781384+0000","last_peered":"2023-07-14T11:21:16.781384+0000","last_clean":"2023-07-14T11:21:16.781384+0000","last_became_active":"2023-07-14T11:19:46.098595+0000","last_became_peered":"2023-07-14T11:19:46.098595+0000","last_unstale":"2023-07-14T11:21:16.781384+0000","last_undegraded":"2023-07-14T11:21:16.781384+0000","last_fullsized":"2023-07-14T11:21:16.781384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:12.644525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781070+0000","last_change":"2023-07-14T11:21:16.780317+0000","last_active":"2023-07-14T11:21:16.781070+0000","last_peered":"2023-07-14T11:21:16.781070+0000","last_clean":"2023-07-14T11:21:16.781070+0000","last_became_active":"2023-07-14T11:21:16.763144+0000","last_became_peered":"2023-07-14T11:21:16.763144+0000","last_unstale":"2023-07-14T11:21:16.781070+0000","last_undegraded":"2023-07-14T11:21:16.781070+0000","last_fullsized":"2023-07-14T11:21:16.781070+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:41:59.524343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132499+0000","last_change":"2023-07-14T11:19:42.092407+0000","last_active":"2023-07-14T11:21:44.132499+0000","last_peered":"2023-07-14T11:21:44.132499+0000","last_clean":"2023-07-14T11:21:44.132499+0000","last_became_active":"2023-07-14T11:19:42.092107+0000","last_became_peered":"2023-07-14T11:19:42.092107+0000","last_unstale":"2023-07-14T11:21:44.132499+0000","last_undegraded":"2023-07-14T11:21:44.132499+0000","last_fullsized":"2023-07-14T11:21:44.132499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:48:16.990224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.758571+0000","last_change":"2023-07-14T11:20:15.732150+0000","last_active":"2023-07-14T11:21:16.758571+0000","last_peered":"2023-07-14T11:21:16.758571+0000","last_clean":"2023-07-14T11:21:16.758571+0000","last_became_active":"2023-07-14T11:20:15.731490+0000","last_became_peered":"2023-07-14T11:20:15.731490+0000","last_unstale":"2023-07-14T11:21:16.758571+0000","last_undegraded":"2023-07-14T11:21:16.758571+0000","last_fullsized":"2023-07-14T11:21:16.758571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:30:05.342050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780126+0000","last_change":"2023-07-14T11:19:41.113867+0000","last_active":"2023-07-14T11:21:16.780126+0000","last_peered":"2023-07-14T11:21:16.780126+0000","last_clean":"2023-07-14T11:21:16.780126+0000","last_became_active":"2023-07-14T11:19:41.113565+0000","last_became_peered":"2023-07-14T11:19:41.113565+0000","last_unstale":"2023-07-14T11:21:16.780126+0000","last_undegraded":"2023-07-14T11:21:16.780126+0000","last_fullsized":"2023-07-14T11:21:16.780126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:59:52.431744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130683+0000","last_change":"2023-07-14T11:20:15.765761+0000","last_active":"2023-07-14T11:21:44.130683+0000","last_peered":"2023-07-14T11:21:44.130683+0000","last_clean":"2023-07-14T11:21:44.130683+0000","last_became_active":"2023-07-14T11:20:15.765313+0000","last_became_peered":"2023-07-14T11:20:15.765313+0000","last_unstale":"2023-07-14T11:21:44.130683+0000","last_undegraded":"2023-07-14T11:21:44.130683+0000","last_fullsized":"2023-07-14T11:21:44.130683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:10:00.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756683+0000","last_change":"2023-07-14T11:19:46.097037+0000","last_active":"2023-07-14T11:21:16.756683+0000","last_peered":"2023-07-14T11:21:16.756683+0000","last_clean":"2023-07-14T11:21:16.756683+0000","last_became_active":"2023-07-14T11:19:46.096591+0000","last_became_peered":"2023-07-14T11:19:46.096591+0000","last_unstale":"2023-07-14T11:21:16.756683+0000","last_undegraded":"2023-07-14T11:21:16.756683+0000","last_fullsized":"2023-07-14T11:21:16.756683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:40.564962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:55.839289+0000","last_change":"2023-07-14T11:19:41.110197+0000","last_active":"2023-07-14T11:21:55.839289+0000","last_peered":"2023-07-14T11:21:55.839289+0000","last_clean":"2023-07-14T11:21:55.839289+0000","last_became_active":"2023-07-14T11:19:41.109892+0000","last_became_peered":"2023-07-14T11:19:41.109892+0000","last_unstale":"2023-07-14T11:21:55.839289+0000","last_undegraded":"2023-07-14T11:21:55.839289+0000","last_fullsized":"2023-07-14T11:21:55.839289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:26.857871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.123757+0000","last_change":"2023-07-14T11:19:42.078923+0000","last_active":"2023-07-14T11:21:44.123757+0000","last_peered":"2023-07-14T11:21:44.123757+0000","last_clean":"2023-07-14T11:21:44.123757+0000","last_became_active":"2023-07-14T11:19:42.078626+0000","last_became_peered":"2023-07-14T11:19:42.078626+0000","last_unstale":"2023-07-14T11:21:44.123757+0000","last_undegraded":"2023-07-14T11:21:44.123757+0000","last_fullsized":"2023-07-14T11:21:44.123757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:37.070346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780756+0000","last_change":"2023-07-14T11:19:46.096106+0000","last_active":"2023-07-14T11:21:16.780756+0000","last_peered":"2023-07-14T11:21:16.780756+0000","last_clean":"2023-07-14T11:21:16.780756+0000","last_became_active":"2023-07-14T11:19:46.095564+0000","last_became_peered":"2023-07-14T11:19:46.095564+0000","last_unstale":"2023-07-14T11:21:16.780756+0000","last_undegraded":"2023-07-14T11:21:16.780756+0000","last_fullsized":"2023-07-14T11:21:16.780756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:33:44.207912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.774327+0000","last_change":"2023-07-14T11:21:16.773940+0000","last_active":"2023-07-14T11:21:16.774327+0000","last_peered":"2023-07-14T11:21:16.774327+0000","last_clean":"2023-07-14T11:21:16.774327+0000","last_became_active":"2023-07-14T11:21:16.762658+0000","last_became_peered":"2023-07-14T11:21:16.762658+0000","last_unstale":"2023-07-14T11:21:16.774327+0000","last_undegraded":"2023-07-14T11:21:16.774327+0000","last_fullsized":"2023-07-14T11:21:16.774327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:05.595583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120231+0000","last_change":"2023-07-14T11:19:42.073383+0000","last_active":"2023-07-14T11:21:44.120231+0000","last_peered":"2023-07-14T11:21:44.120231+0000","last_clean":"2023-07-14T11:21:44.120231+0000","last_became_active":"2023-07-14T11:19:42.073095+0000","last_became_peered":"2023-07-14T11:19:42.073095+0000","last_unstale":"2023-07-14T11:21:44.120231+0000","last_undegraded":"2023-07-14T11:21:44.120231+0000","last_fullsized":"2023-07-14T11:21:44.120231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:59:04.043553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780006+0000","last_change":"2023-07-14T11:19:46.100998+0000","last_active":"2023-07-14T11:21:16.780006+0000","last_peered":"2023-07-14T11:21:16.780006+0000","last_clean":"2023-07-14T11:21:16.780006+0000","last_became_active":"2023-07-14T11:19:46.100577+0000","last_became_peered":"2023-07-14T11:19:46.100577+0000","last_unstale":"2023-07-14T11:21:16.780006+0000","last_undegraded":"2023-07-14T11:21:16.780006+0000","last_fullsized":"2023-07-14T11:21:16.780006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:19.174537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760420+0000","last_change":"2023-07-14T11:19:41.106923+0000","last_active":"2023-07-14T11:21:16.760420+0000","last_peered":"2023-07-14T11:21:16.760420+0000","last_clean":"2023-07-14T11:21:16.760420+0000","last_became_active":"2023-07-14T11:19:41.106629+0000","last_became_peered":"2023-07-14T11:19:41.106629+0000","last_unstale":"2023-07-14T11:21:16.760420+0000","last_undegraded":"2023-07-14T11:21:16.760420+0000","last_fullsized":"2023-07-14T11:21:16.760420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:13.662336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129915+0000","last_change":"2023-07-14T11:19:42.079789+0000","last_active":"2023-07-14T11:21:44.129915+0000","last_peered":"2023-07-14T11:21:44.129915+0000","last_clean":"2023-07-14T11:21:44.129915+0000","last_became_active":"2023-07-14T11:19:42.079436+0000","last_became_peered":"2023-07-14T11:19:42.079436+0000","last_unstale":"2023-07-14T11:21:44.129915+0000","last_undegraded":"2023-07-14T11:21:44.129915+0000","last_fullsized":"2023-07-14T11:21:44.129915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:12:29.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759722+0000","last_change":"2023-07-14T11:20:15.731601+0000","last_active":"2023-07-14T11:21:16.759722+0000","last_peered":"2023-07-14T11:21:16.759722+0000","last_clean":"2023-07-14T11:21:16.759722+0000","last_became_active":"2023-07-14T11:20:15.731121+0000","last_became_peered":"2023-07-14T11:20:15.731121+0000","last_unstale":"2023-07-14T11:21:16.759722+0000","last_undegraded":"2023-07-14T11:21:16.759722+0000","last_fullsized":"2023-07-14T11:21:16.759722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:15:52.609093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756735+0000","last_change":"2023-07-14T11:19:41.110688+0000","last_active":"2023-07-14T11:21:16.756735+0000","last_peered":"2023-07-14T11:21:16.756735+0000","last_clean":"2023-07-14T11:21:16.756735+0000","last_became_active":"2023-07-14T11:19:41.110138+0000","last_became_peered":"2023-07-14T11:19:41.110138+0000","last_unstale":"2023-07-14T11:21:16.756735+0000","last_undegraded":"2023-07-14T11:21:16.756735+0000","last_fullsized":"2023-07-14T11:21:16.756735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:57.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133807+0000","last_change":"2023-07-14T11:19:42.088052+0000","last_active":"2023-07-14T11:21:44.133807+0000","last_peered":"2023-07-14T11:21:44.133807+0000","last_clean":"2023-07-14T11:21:44.133807+0000","last_became_active":"2023-07-14T11:19:42.087800+0000","last_became_peered":"2023-07-14T11:19:42.087800+0000","last_unstale":"2023-07-14T11:21:44.133807+0000","last_undegraded":"2023-07-14T11:21:44.133807+0000","last_fullsized":"2023-07-14T11:21:44.133807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:15:22.547786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781033+0000","last_change":"2023-07-14T11:19:46.093515+0000","last_active":"2023-07-14T11:21:16.781033+0000","last_peered":"2023-07-14T11:21:16.781033+0000","last_clean":"2023-07-14T11:21:16.781033+0000","last_became_active":"2023-07-14T11:19:46.093166+0000","last_became_peered":"2023-07-14T11:19:46.093166+0000","last_unstale":"2023-07-14T11:21:16.781033+0000","last_undegraded":"2023-07-14T11:21:16.781033+0000","last_fullsized":"2023-07-14T11:21:16.781033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:43:54.535268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827424+0000","last_change":"2023-07-14T11:19:41.111616+0000","last_active":"2023-07-14T11:21:17.827424+0000","last_peered":"2023-07-14T11:21:17.827424+0000","last_clean":"2023-07-14T11:21:17.827424+0000","last_became_active":"2023-07-14T11:19:41.110669+0000","last_became_peered":"2023-07-14T11:19:41.110669+0000","last_unstale":"2023-07-14T11:21:17.827424+0000","last_undegraded":"2023-07-14T11:21:17.827424+0000","last_fullsized":"2023-07-14T11:21:17.827424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:34:07.328659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128632+0000","last_change":"2023-07-14T11:20:15.765912+0000","last_active":"2023-07-14T11:21:44.128632+0000","last_peered":"2023-07-14T11:21:44.128632+0000","last_clean":"2023-07-14T11:21:44.128632+0000","last_became_active":"2023-07-14T11:20:15.765489+0000","last_became_peered":"2023-07-14T11:20:15.765489+0000","last_unstale":"2023-07-14T11:21:44.128632+0000","last_undegraded":"2023-07-14T11:21:44.128632+0000","last_fullsized":"2023-07-14T11:21:44.128632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:58:43.563311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780882+0000","last_change":"2023-07-14T11:19:46.272661+0000","last_active":"2023-07-14T11:21:16.780882+0000","last_peered":"2023-07-14T11:21:16.780882+0000","last_clean":"2023-07-14T11:21:16.780882+0000","last_became_active":"2023-07-14T11:19:46.272253+0000","last_became_peered":"2023-07-14T11:19:46.272253+0000","last_unstale":"2023-07-14T11:21:16.780882+0000","last_undegraded":"2023-07-14T11:21:16.780882+0000","last_fullsized":"2023-07-14T11:21:16.780882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:56:02.843006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.121291+0000","last_change":"2023-07-14T11:19:42.078681+0000","last_active":"2023-07-14T11:21:44.121291+0000","last_peered":"2023-07-14T11:21:44.121291+0000","last_clean":"2023-07-14T11:21:44.121291+0000","last_became_active":"2023-07-14T11:19:42.078231+0000","last_became_peered":"2023-07-14T11:19:42.078231+0000","last_unstale":"2023-07-14T11:21:44.121291+0000","last_undegraded":"2023-07-14T11:21:44.121291+0000","last_fullsized":"2023-07-14T11:21:44.121291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:22:39.851277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:57.841537+0000","last_change":"2023-07-14T11:19:41.109401+0000","last_active":"2023-07-14T11:21:57.841537+0000","last_peered":"2023-07-14T11:21:57.841537+0000","last_clean":"2023-07-14T11:21:57.841537+0000","last_became_active":"2023-07-14T11:19:41.109044+0000","last_became_peered":"2023-07-14T11:19:41.109044+0000","last_unstale":"2023-07-14T11:21:57.841537+0000","last_undegraded":"2023-07-14T11:21:57.841537+0000","last_fullsized":"2023-07-14T11:21:57.841537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:39:23.337780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.828305+0000","last_change":"2023-07-14T11:19:46.093809+0000","last_active":"2023-07-14T11:21:17.828305+0000","last_peered":"2023-07-14T11:21:17.828305+0000","last_clean":"2023-07-14T11:21:17.828305+0000","last_became_active":"2023-07-14T11:19:46.093407+0000","last_became_peered":"2023-07-14T11:19:46.093407+0000","last_unstale":"2023-07-14T11:21:17.828305+0000","last_undegraded":"2023-07-14T11:21:17.828305+0000","last_fullsized":"2023-07-14T11:21:17.828305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:47:27.405242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135278+0000","last_change":"2023-07-14T11:19:42.083713+0000","last_active":"2023-07-14T11:21:44.135278+0000","last_peered":"2023-07-14T11:21:44.135278+0000","last_clean":"2023-07-14T11:21:44.135278+0000","last_became_active":"2023-07-14T11:19:42.083349+0000","last_became_peered":"2023-07-14T11:19:42.083349+0000","last_unstale":"2023-07-14T11:21:44.135278+0000","last_undegraded":"2023-07-14T11:21:44.135278+0000","last_fullsized":"2023-07-14T11:21:44.135278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:51.335219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783295+0000","last_change":"2023-07-14T11:19:41.106889+0000","last_active":"2023-07-14T11:21:16.783295+0000","last_peered":"2023-07-14T11:21:16.783295+0000","last_clean":"2023-07-14T11:21:16.783295+0000","last_became_active":"2023-07-14T11:19:41.106586+0000","last_became_peered":"2023-07-14T11:19:41.106586+0000","last_unstale":"2023-07-14T11:21:16.783295+0000","last_undegraded":"2023-07-14T11:21:16.783295+0000","last_fullsized":"2023-07-14T11:21:16.783295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:42:44.347182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827912+0000","last_change":"2023-07-14T11:18:36.215705+0000","last_active":"2023-07-14T11:21:17.827912+0000","last_peered":"2023-07-14T11:21:17.827912+0000","last_clean":"2023-07-14T11:21:17.827912+0000","last_became_active":"2023-07-14T11:18:36.215340+0000","last_became_peered":"2023-07-14T11:18:36.215340+0000","last_unstale":"2023-07-14T11:21:17.827912+0000","last_undegraded":"2023-07-14T11:21:17.827912+0000","last_fullsized":"2023-07-14T11:21:17.827912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:15:32.636472+0000","last_clean_scrub_stamp":"2023-07-14T11:15:32.636472+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:44:53.938118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781156+0000","last_change":"2023-07-14T11:19:46.273594+0000","last_active":"2023-07-14T11:21:16.781156+0000","last_peered":"2023-07-14T11:21:16.781156+0000","last_clean":"2023-07-14T11:21:16.781156+0000","last_became_active":"2023-07-14T11:19:46.273121+0000","last_became_peered":"2023-07-14T11:19:46.273121+0000","last_unstale":"2023-07-14T11:21:16.781156+0000","last_undegraded":"2023-07-14T11:21:16.781156+0000","last_fullsized":"2023-07-14T11:21:16.781156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:33:14.547044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.129505+0000","last_change":"2023-07-14T11:19:42.079969+0000","last_active":"2023-07-14T11:21:44.129505+0000","last_peered":"2023-07-14T11:21:44.129505+0000","last_clean":"2023-07-14T11:21:44.129505+0000","last_became_active":"2023-07-14T11:19:42.079652+0000","last_became_peered":"2023-07-14T11:19:42.079652+0000","last_unstale":"2023-07-14T11:21:44.129505+0000","last_undegraded":"2023-07-14T11:21:44.129505+0000","last_fullsized":"2023-07-14T11:21:44.129505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:53:19.310433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:17.827364+0000","last_change":"2023-07-14T11:19:41.109808+0000","last_active":"2023-07-14T11:21:17.827364+0000","last_peered":"2023-07-14T11:21:17.827364+0000","last_clean":"2023-07-14T11:21:17.827364+0000","last_became_active":"2023-07-14T11:19:41.109495+0000","last_became_peered":"2023-07-14T11:19:41.109495+0000","last_unstale":"2023-07-14T11:21:17.827364+0000","last_undegraded":"2023-07-14T11:21:17.827364+0000","last_fullsized":"2023-07-14T11:21:17.827364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:25:55.640663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759127+0000","last_change":"2023-07-14T11:19:46.101133+0000","last_active":"2023-07-14T11:21:16.759127+0000","last_peered":"2023-07-14T11:21:16.759127+0000","last_clean":"2023-07-14T11:21:16.759127+0000","last_became_active":"2023-07-14T11:19:46.100627+0000","last_became_peered":"2023-07-14T11:19:46.100627+0000","last_unstale":"2023-07-14T11:21:16.759127+0000","last_undegraded":"2023-07-14T11:21:16.759127+0000","last_fullsized":"2023-07-14T11:21:16.759127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:35:28.280537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760321+0000","last_change":"2023-07-14T11:19:41.105151+0000","last_active":"2023-07-14T11:21:16.760321+0000","last_peered":"2023-07-14T11:21:16.760321+0000","last_clean":"2023-07-14T11:21:16.760321+0000","last_became_active":"2023-07-14T11:19:41.104787+0000","last_became_peered":"2023-07-14T11:19:41.104787+0000","last_unstale":"2023-07-14T11:21:16.760321+0000","last_undegraded":"2023-07-14T11:21:16.760321+0000","last_fullsized":"2023-07-14T11:21:16.760321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:46.760759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133293+0000","last_change":"2023-07-14T11:19:42.089191+0000","last_active":"2023-07-14T11:21:44.133293+0000","last_peered":"2023-07-14T11:21:44.133293+0000","last_clean":"2023-07-14T11:21:44.133293+0000","last_became_active":"2023-07-14T11:19:42.088866+0000","last_became_peered":"2023-07-14T11:19:42.088866+0000","last_unstale":"2023-07-14T11:21:44.133293+0000","last_undegraded":"2023-07-14T11:21:44.133293+0000","last_fullsized":"2023-07-14T11:21:44.133293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:20:33.855972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761100+0000","last_change":"2023-07-14T11:19:46.273422+0000","last_active":"2023-07-14T11:21:16.761100+0000","last_peered":"2023-07-14T11:21:16.761100+0000","last_clean":"2023-07-14T11:21:16.761100+0000","last_became_active":"2023-07-14T11:19:46.273018+0000","last_became_peered":"2023-07-14T11:19:46.273018+0000","last_unstale":"2023-07-14T11:21:16.761100+0000","last_undegraded":"2023-07-14T11:21:16.761100+0000","last_fullsized":"2023-07-14T11:21:16.761100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:01.581528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:51.950589+0000","last_change":"2023-07-14T11:19:41.109698+0000","last_active":"2023-07-14T11:21:51.950589+0000","last_peered":"2023-07-14T11:21:51.950589+0000","last_clean":"2023-07-14T11:21:51.950589+0000","last_became_active":"2023-07-14T11:19:41.109348+0000","last_became_peered":"2023-07-14T11:19:41.109348+0000","last_unstale":"2023-07-14T11:21:51.950589+0000","last_undegraded":"2023-07-14T11:21:51.950589+0000","last_fullsized":"2023-07-14T11:21:51.950589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:31:42.789040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131085+0000","last_change":"2023-07-14T11:19:42.079905+0000","last_active":"2023-07-14T11:21:44.131085+0000","last_peered":"2023-07-14T11:21:44.131085+0000","last_clean":"2023-07-14T11:21:44.131085+0000","last_became_active":"2023-07-14T11:19:42.079424+0000","last_became_peered":"2023-07-14T11:19:42.079424+0000","last_unstale":"2023-07-14T11:21:44.131085+0000","last_undegraded":"2023-07-14T11:21:44.131085+0000","last_fullsized":"2023-07-14T11:21:44.131085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:30:10.033823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761906+0000","last_change":"2023-07-14T11:19:46.093934+0000","last_active":"2023-07-14T11:21:16.761906+0000","last_peered":"2023-07-14T11:21:16.761906+0000","last_clean":"2023-07-14T11:21:16.761906+0000","last_became_active":"2023-07-14T11:19:46.093319+0000","last_became_peered":"2023-07-14T11:19:46.093319+0000","last_unstale":"2023-07-14T11:21:16.761906+0000","last_undegraded":"2023-07-14T11:21:16.761906+0000","last_fullsized":"2023-07-14T11:21:16.761906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:19:06.706597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783437+0000","last_change":"2023-07-14T11:19:41.105338+0000","last_active":"2023-07-14T11:21:16.783437+0000","last_peered":"2023-07-14T11:21:16.783437+0000","last_clean":"2023-07-14T11:21:16.783437+0000","last_became_active":"2023-07-14T11:19:41.105088+0000","last_became_peered":"2023-07-14T11:19:41.105088+0000","last_unstale":"2023-07-14T11:21:16.783437+0000","last_undegraded":"2023-07-14T11:21:16.783437+0000","last_fullsized":"2023-07-14T11:21:16.783437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:04:22.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135985+0000","last_change":"2023-07-14T11:20:15.729830+0000","last_active":"2023-07-14T11:21:44.135985+0000","last_peered":"2023-07-14T11:21:44.135985+0000","last_clean":"2023-07-14T11:21:44.135985+0000","last_became_active":"2023-07-14T11:20:15.729494+0000","last_became_peered":"2023-07-14T11:20:15.729494+0000","last_unstale":"2023-07-14T11:21:44.135985+0000","last_undegraded":"2023-07-14T11:21:44.135985+0000","last_fullsized":"2023-07-14T11:21:44.135985+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:50:49.239460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783236+0000","last_change":"2023-07-14T11:19:46.272734+0000","last_active":"2023-07-14T11:21:16.783236+0000","last_peered":"2023-07-14T11:21:16.783236+0000","last_clean":"2023-07-14T11:21:16.783236+0000","last_became_active":"2023-07-14T11:19:46.272367+0000","last_became_peered":"2023-07-14T11:19:46.272367+0000","last_unstale":"2023-07-14T11:21:16.783236+0000","last_undegraded":"2023-07-14T11:21:16.783236+0000","last_fullsized":"2023-07-14T11:21:16.783236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:00:23.953512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:18.022769+0000","last_change":"2023-07-14T11:19:41.110146+0000","last_active":"2023-07-14T11:21:18.022769+0000","last_peered":"2023-07-14T11:21:18.022769+0000","last_clean":"2023-07-14T11:21:18.022769+0000","last_became_active":"2023-07-14T11:19:41.109844+0000","last_became_peered":"2023-07-14T11:19:41.109844+0000","last_unstale":"2023-07-14T11:21:18.022769+0000","last_undegraded":"2023-07-14T11:21:18.022769+0000","last_fullsized":"2023-07-14T11:21:18.022769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:12:28.851941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.108769+0000","last_change":"2023-07-14T11:19:42.079824+0000","last_active":"2023-07-14T11:21:44.108769+0000","last_peered":"2023-07-14T11:21:44.108769+0000","last_clean":"2023-07-14T11:21:44.108769+0000","last_became_active":"2023-07-14T11:19:42.079379+0000","last_became_peered":"2023-07-14T11:19:42.079379+0000","last_unstale":"2023-07-14T11:21:44.108769+0000","last_undegraded":"2023-07-14T11:21:44.108769+0000","last_fullsized":"2023-07-14T11:21:44.108769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:23:32.637906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.756893+0000","last_change":"2023-07-14T11:19:46.096087+0000","last_active":"2023-07-14T11:21:16.756893+0000","last_peered":"2023-07-14T11:21:16.756893+0000","last_clean":"2023-07-14T11:21:16.756893+0000","last_became_active":"2023-07-14T11:19:46.095725+0000","last_became_peered":"2023-07-14T11:19:46.095725+0000","last_unstale":"2023-07-14T11:21:16.756893+0000","last_undegraded":"2023-07-14T11:21:16.756893+0000","last_fullsized":"2023-07-14T11:21:16.756893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:52:08.577965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761060+0000","last_change":"2023-07-14T11:19:41.110440+0000","last_active":"2023-07-14T11:21:16.761060+0000","last_peered":"2023-07-14T11:21:16.761060+0000","last_clean":"2023-07-14T11:21:16.761060+0000","last_became_active":"2023-07-14T11:19:41.110119+0000","last_became_peered":"2023-07-14T11:19:41.110119+0000","last_unstale":"2023-07-14T11:21:16.761060+0000","last_undegraded":"2023-07-14T11:21:16.761060+0000","last_fullsized":"2023-07-14T11:21:16.761060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:11:05.150069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132051+0000","last_change":"2023-07-14T11:19:42.081152+0000","last_active":"2023-07-14T11:21:44.132051+0000","last_peered":"2023-07-14T11:21:44.132051+0000","last_clean":"2023-07-14T11:21:44.132051+0000","last_became_active":"2023-07-14T11:19:42.080867+0000","last_became_peered":"2023-07-14T11:19:42.080867+0000","last_unstale":"2023-07-14T11:21:44.132051+0000","last_undegraded":"2023-07-14T11:21:44.132051+0000","last_fullsized":"2023-07-14T11:21:44.132051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:20:37.760123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783130+0000","last_change":"2023-07-14T11:19:46.102894+0000","last_active":"2023-07-14T11:21:16.783130+0000","last_peered":"2023-07-14T11:21:16.783130+0000","last_clean":"2023-07-14T11:21:16.783130+0000","last_became_active":"2023-07-14T11:19:46.102555+0000","last_became_peered":"2023-07-14T11:19:46.102555+0000","last_unstale":"2023-07-14T11:21:16.783130+0000","last_undegraded":"2023-07-14T11:21:16.783130+0000","last_fullsized":"2023-07-14T11:21:16.783130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:05:52.217578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.761847+0000","last_change":"2023-07-14T11:19:41.113703+0000","last_active":"2023-07-14T11:21:16.761847+0000","last_peered":"2023-07-14T11:21:16.761847+0000","last_clean":"2023-07-14T11:21:16.761847+0000","last_became_active":"2023-07-14T11:19:41.113421+0000","last_became_peered":"2023-07-14T11:19:41.113421+0000","last_unstale":"2023-07-14T11:21:16.761847+0000","last_undegraded":"2023-07-14T11:21:16.761847+0000","last_fullsized":"2023-07-14T11:21:16.761847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:49:24.188762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132768+0000","last_change":"2023-07-14T11:19:42.084782+0000","last_active":"2023-07-14T11:21:44.132768+0000","last_peered":"2023-07-14T11:21:44.132768+0000","last_clean":"2023-07-14T11:21:44.132768+0000","last_became_active":"2023-07-14T11:19:42.084462+0000","last_became_peered":"2023-07-14T11:19:42.084462+0000","last_unstale":"2023-07-14T11:21:44.132768+0000","last_undegraded":"2023-07-14T11:21:44.132768+0000","last_fullsized":"2023-07-14T11:21:44.132768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:44.830001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.783284+0000","last_change":"2023-07-14T11:19:46.273783+0000","last_active":"2023-07-14T11:21:16.783284+0000","last_peered":"2023-07-14T11:21:16.783284+0000","last_clean":"2023-07-14T11:21:16.783284+0000","last_became_active":"2023-07-14T11:19:46.273385+0000","last_became_peered":"2023-07-14T11:19:46.273385+0000","last_unstale":"2023-07-14T11:21:16.783284+0000","last_undegraded":"2023-07-14T11:21:16.783284+0000","last_fullsized":"2023-07-14T11:21:16.783284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:07:04.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.759626+0000","last_change":"2023-07-14T11:19:41.112213+0000","last_active":"2023-07-14T11:21:16.759626+0000","last_peered":"2023-07-14T11:21:16.759626+0000","last_clean":"2023-07-14T11:21:16.759626+0000","last_became_active":"2023-07-14T11:19:41.111906+0000","last_became_peered":"2023-07-14T11:19:41.111906+0000","last_unstale":"2023-07-14T11:21:16.759626+0000","last_undegraded":"2023-07-14T11:21:16.759626+0000","last_fullsized":"2023-07-14T11:21:16.759626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:43:00.377914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134559+0000","last_change":"2023-07-14T11:19:42.078219+0000","last_active":"2023-07-14T11:21:44.134559+0000","last_peered":"2023-07-14T11:21:44.134559+0000","last_clean":"2023-07-14T11:21:44.134559+0000","last_became_active":"2023-07-14T11:19:42.077942+0000","last_became_peered":"2023-07-14T11:19:42.077942+0000","last_unstale":"2023-07-14T11:21:44.134559+0000","last_undegraded":"2023-07-14T11:21:44.134559+0000","last_fullsized":"2023-07-14T11:21:44.134559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:03.031833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:18.023102+0000","last_change":"2023-07-14T11:19:46.091759+0000","last_active":"2023-07-14T11:21:18.023102+0000","last_peered":"2023-07-14T11:21:18.023102+0000","last_clean":"2023-07-14T11:21:18.023102+0000","last_became_active":"2023-07-14T11:19:46.091296+0000","last_became_peered":"2023-07-14T11:19:46.091296+0000","last_unstale":"2023-07-14T11:21:18.023102+0000","last_undegraded":"2023-07-14T11:21:18.023102+0000","last_fullsized":"2023-07-14T11:21:18.023102+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:04:51.829719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.762268+0000","last_change":"2023-07-14T11:19:41.109154+0000","last_active":"2023-07-14T11:21:16.762268+0000","last_peered":"2023-07-14T11:21:16.762268+0000","last_clean":"2023-07-14T11:21:16.762268+0000","last_became_active":"2023-07-14T11:19:41.108799+0000","last_became_peered":"2023-07-14T11:19:41.108799+0000","last_unstale":"2023-07-14T11:21:16.762268+0000","last_undegraded":"2023-07-14T11:21:16.762268+0000","last_fullsized":"2023-07-14T11:21:16.762268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:41:38.935430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131585+0000","last_change":"2023-07-14T11:19:42.076470+0000","last_active":"2023-07-14T11:21:44.131585+0000","last_peered":"2023-07-14T11:21:44.131585+0000","last_clean":"2023-07-14T11:21:44.131585+0000","last_became_active":"2023-07-14T11:19:42.076188+0000","last_became_peered":"2023-07-14T11:19:42.076188+0000","last_unstale":"2023-07-14T11:21:44.131585+0000","last_undegraded":"2023-07-14T11:21:44.131585+0000","last_fullsized":"2023-07-14T11:21:44.131585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:00.381695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.760768+0000","last_change":"2023-07-14T11:20:15.729884+0000","last_active":"2023-07-14T11:21:16.760768+0000","last_peered":"2023-07-14T11:21:16.760768+0000","last_clean":"2023-07-14T11:21:16.760768+0000","last_became_active":"2023-07-14T11:20:15.729501+0000","last_became_peered":"2023-07-14T11:20:15.729501+0000","last_unstale":"2023-07-14T11:21:16.760768+0000","last_undegraded":"2023-07-14T11:21:16.760768+0000","last_fullsized":"2023-07-14T11:21:16.760768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:22:56.514930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.781299+0000","last_change":"2023-07-14T11:19:41.115621+0000","last_active":"2023-07-14T11:21:16.781299+0000","last_peered":"2023-07-14T11:21:16.781299+0000","last_clean":"2023-07-14T11:21:16.781299+0000","last_became_active":"2023-07-14T11:19:41.115330+0000","last_became_peered":"2023-07-14T11:19:41.115330+0000","last_unstale":"2023-07-14T11:21:16.781299+0000","last_undegraded":"2023-07-14T11:21:16.781299+0000","last_fullsized":"2023-07-14T11:21:16.781299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:39.416802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.128900+0000","last_change":"2023-07-14T11:20:15.766177+0000","last_active":"2023-07-14T11:21:44.128900+0000","last_peered":"2023-07-14T11:21:44.128900+0000","last_clean":"2023-07-14T11:21:44.128900+0000","last_became_active":"2023-07-14T11:20:15.765650+0000","last_became_peered":"2023-07-14T11:20:15.765650+0000","last_unstale":"2023-07-14T11:21:44.128900+0000","last_undegraded":"2023-07-14T11:21:44.128900+0000","last_fullsized":"2023-07-14T11:21:44.128900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:58.657251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.779781+0000","last_change":"2023-07-14T11:19:46.098158+0000","last_active":"2023-07-14T11:21:16.779781+0000","last_peered":"2023-07-14T11:21:16.779781+0000","last_clean":"2023-07-14T11:21:16.779781+0000","last_became_active":"2023-07-14T11:19:46.097781+0000","last_became_peered":"2023-07-14T11:19:46.097781+0000","last_unstale":"2023-07-14T11:21:16.779781+0000","last_undegraded":"2023-07-14T11:21:16.779781+0000","last_fullsized":"2023-07-14T11:21:16.779781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:45.066838+0000","last_clean_scrub_stamp":"2023-07-14T11:19:45.066838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:32:23.647113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:16.780749+0000","last_change":"2023-07-14T11:19:41.110513+0000","last_active":"2023-07-14T11:21:16.780749+0000","last_peered":"2023-07-14T11:21:16.780749+0000","last_clean":"2023-07-14T11:21:16.780749+0000","last_became_active":"2023-07-14T11:19:41.110194+0000","last_became_peered":"2023-07-14T11:19:41.110194+0000","last_unstale":"2023-07-14T11:21:16.780749+0000","last_undegraded":"2023-07-14T11:21:16.780749+0000","last_fullsized":"2023-07-14T11:21:16.780749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:40.084035+0000","last_clean_scrub_stamp":"2023-07-14T11:19:40.084035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:36:31.036473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131916+0000","last_change":"2023-07-14T11:19:42.073779+0000","last_active":"2023-07-14T11:21:44.131916+0000","last_peered":"2023-07-14T11:21:44.131916+0000","last_clean":"2023-07-14T11:21:44.131916+0000","last_became_active":"2023-07-14T11:19:42.073475+0000","last_became_peered":"2023-07-14T11:19:42.073475+0000","last_unstale":"2023-07-14T11:21:44.131916+0000","last_undegraded":"2023-07-14T11:21:44.131916+0000","last_fullsized":"2023-07-14T11:21:44.131916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:54.620604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.123597+0000","last_change":"2023-07-14T11:19:42.081349+0000","last_active":"2023-07-14T11:21:44.123597+0000","last_peered":"2023-07-14T11:21:44.123597+0000","last_clean":"2023-07-14T11:21:44.123597+0000","last_became_active":"2023-07-14T11:19:42.079710+0000","last_became_peered":"2023-07-14T11:19:42.079710+0000","last_unstale":"2023-07-14T11:21:44.123597+0000","last_undegraded":"2023-07-14T11:21:44.123597+0000","last_fullsized":"2023-07-14T11:21:44.123597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:13:16.901921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.127676+0000","last_change":"2023-07-14T11:19:42.072310+0000","last_active":"2023-07-14T11:21:44.127676+0000","last_peered":"2023-07-14T11:21:44.127676+0000","last_clean":"2023-07-14T11:21:44.127676+0000","last_became_active":"2023-07-14T11:19:42.071538+0000","last_became_peered":"2023-07-14T11:19:42.071538+0000","last_unstale":"2023-07-14T11:21:44.127676+0000","last_undegraded":"2023-07-14T11:21:44.127676+0000","last_fullsized":"2023-07-14T11:21:44.127676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:06:24.239682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130965+0000","last_change":"2023-07-14T11:19:42.084678+0000","last_active":"2023-07-14T11:21:44.130965+0000","last_peered":"2023-07-14T11:21:44.130965+0000","last_clean":"2023-07-14T11:21:44.130965+0000","last_became_active":"2023-07-14T11:19:42.084308+0000","last_became_peered":"2023-07-14T11:19:42.084308+0000","last_unstale":"2023-07-14T11:21:44.130965+0000","last_undegraded":"2023-07-14T11:21:44.130965+0000","last_fullsized":"2023-07-14T11:21:44.130965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:38:17.082600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.135133+0000","last_change":"2023-07-14T11:19:42.086797+0000","last_active":"2023-07-14T11:21:44.135133+0000","last_peered":"2023-07-14T11:21:44.135133+0000","last_clean":"2023-07-14T11:21:44.135133+0000","last_became_active":"2023-07-14T11:19:42.082469+0000","last_became_peered":"2023-07-14T11:19:42.082469+0000","last_unstale":"2023-07-14T11:21:44.135133+0000","last_undegraded":"2023-07-14T11:21:44.135133+0000","last_fullsized":"2023-07-14T11:21:44.135133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:07:19.948875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.132889+0000","last_change":"2023-07-14T11:19:42.089089+0000","last_active":"2023-07-14T11:21:44.132889+0000","last_peered":"2023-07-14T11:21:44.132889+0000","last_clean":"2023-07-14T11:21:44.132889+0000","last_became_active":"2023-07-14T11:19:42.088352+0000","last_became_peered":"2023-07-14T11:19:42.088352+0000","last_unstale":"2023-07-14T11:21:44.132889+0000","last_undegraded":"2023-07-14T11:21:44.132889+0000","last_fullsized":"2023-07-14T11:21:44.132889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:24.946793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.120491+0000","last_change":"2023-07-14T11:19:42.079134+0000","last_active":"2023-07-14T11:21:44.120491+0000","last_peered":"2023-07-14T11:21:44.120491+0000","last_clean":"2023-07-14T11:21:44.120491+0000","last_became_active":"2023-07-14T11:19:42.078855+0000","last_became_peered":"2023-07-14T11:19:42.078855+0000","last_unstale":"2023-07-14T11:21:44.120491+0000","last_undegraded":"2023-07-14T11:21:44.120491+0000","last_fullsized":"2023-07-14T11:21:44.120491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:33:36.799060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.122053+0000","last_change":"2023-07-14T11:19:42.073564+0000","last_active":"2023-07-14T11:21:44.122053+0000","last_peered":"2023-07-14T11:21:44.122053+0000","last_clean":"2023-07-14T11:21:44.122053+0000","last_became_active":"2023-07-14T11:19:42.073258+0000","last_became_peered":"2023-07-14T11:19:42.073258+0000","last_unstale":"2023-07-14T11:21:44.122053+0000","last_undegraded":"2023-07-14T11:21:44.122053+0000","last_fullsized":"2023-07-14T11:21:44.122053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:55.929013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134984+0000","last_change":"2023-07-14T11:19:42.079710+0000","last_active":"2023-07-14T11:21:44.134984+0000","last_peered":"2023-07-14T11:21:44.134984+0000","last_clean":"2023-07-14T11:21:44.134984+0000","last_became_active":"2023-07-14T11:19:42.078925+0000","last_became_peered":"2023-07-14T11:19:42.078925+0000","last_unstale":"2023-07-14T11:21:44.134984+0000","last_undegraded":"2023-07-14T11:21:44.134984+0000","last_fullsized":"2023-07-14T11:21:44.134984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:58.128316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.118051+0000","last_change":"2023-07-14T11:20:15.765762+0000","last_active":"2023-07-14T11:21:44.118051+0000","last_peered":"2023-07-14T11:21:44.118051+0000","last_clean":"2023-07-14T11:21:44.118051+0000","last_became_active":"2023-07-14T11:20:15.765349+0000","last_became_peered":"2023-07-14T11:20:15.765349+0000","last_unstale":"2023-07-14T11:21:44.118051+0000","last_undegraded":"2023-07-14T11:21:44.118051+0000","last_fullsized":"2023-07-14T11:21:44.118051+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:11:07.179820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.131580+0000","last_change":"2023-07-14T11:19:42.083489+0000","last_active":"2023-07-14T11:21:44.131580+0000","last_peered":"2023-07-14T11:21:44.131580+0000","last_clean":"2023-07-14T11:21:44.131580+0000","last_became_active":"2023-07-14T11:19:42.076966+0000","last_became_peered":"2023-07-14T11:19:42.076966+0000","last_unstale":"2023-07-14T11:21:44.131580+0000","last_undegraded":"2023-07-14T11:21:44.131580+0000","last_fullsized":"2023-07-14T11:21:44.131580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:48.020842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.133597+0000","last_change":"2023-07-14T11:19:42.075298+0000","last_active":"2023-07-14T11:21:44.133597+0000","last_peered":"2023-07-14T11:21:44.133597+0000","last_clean":"2023-07-14T11:21:44.133597+0000","last_became_active":"2023-07-14T11:19:42.073685+0000","last_became_peered":"2023-07-14T11:19:42.073685+0000","last_unstale":"2023-07-14T11:21:44.133597+0000","last_undegraded":"2023-07-14T11:21:44.133597+0000","last_fullsized":"2023-07-14T11:21:44.133597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:23.156838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.134494+0000","last_change":"2023-07-14T11:19:42.089613+0000","last_active":"2023-07-14T11:21:44.134494+0000","last_peered":"2023-07-14T11:21:44.134494+0000","last_clean":"2023-07-14T11:21:44.134494+0000","last_became_active":"2023-07-14T11:19:42.089130+0000","last_became_peered":"2023-07-14T11:19:42.089130+0000","last_unstale":"2023-07-14T11:21:44.134494+0000","last_undegraded":"2023-07-14T11:21:44.134494+0000","last_fullsized":"2023-07-14T11:21:44.134494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:03:17.632302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T11:21:44.130072+0000","last_change":"2023-07-14T11:19:42.075237+0000","last_active":"2023-07-14T11:21:44.130072+0000","last_peered":"2023-07-14T11:21:44.130072+0000","last_clean":"2023-07-14T11:21:44.130072+0000","last_became_active":"2023-07-14T11:19:42.074996+0000","last_became_peered":"2023-07-14T11:19:42.074996+0000","last_unstale":"2023-07-14T11:21:44.130072+0000","last_undegraded":"2023-07-14T11:21:44.130072+0000","last_fullsized":"2023-07-14T11:21:44.130072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:19:41.086153+0000","last_clean_scrub_stamp":"2023-07-14T11:19:41.086153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:27:11.918619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66428,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":400,"num_write":160,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416425,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":510275,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.35999999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.52800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86699999999999999,"5min":1.097,"15min":1.097},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69099999999999995,"5min":0.75,"15min":0.75},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92000000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79400000000000004,"5min":1.042,"15min":1.042},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52500000000000002,"15min":0.50800000000000001},"min":{"1min":0.42799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.623},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89900000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.55700000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94799999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.72599999999999998},"min":{"1min":0.68799999999999994,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.02,"5min":1.159,"15min":1.159},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.92500000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73799999999999999,"15min":0.73899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":1.014,"15min":1.014},"last":0.97599999999999998}]},{"osd":9,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.751,"15min":0.749},"min":{"1min":0.71899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.042,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.01,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.01}]},{"osd":10,"last update":"Fri Jul 14 11:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.96799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0640000000000001,"5min":1.456,"15min":1.456},"last":0.86599999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1150976,"data_stored":1160263,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.84899999999999998,"15min":0.89000000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.746,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.83599999999999997,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.877,"5min":6.0090000000000003,"15min":6.0090000000000003},"last":0.621}]},{"osd":1,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.52900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.76700000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67000000000000004,"15min":0.69699999999999995},"min":{"1min":0.42699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73599999999999999,"5min":2.46,"15min":2.46},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.66800000000000004},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77200000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.66500000000000004,"15min":0.69699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.496},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.75,"15min":0.745},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82899999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.42099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79200000000000004,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89900000000000002,"5min":2.431,"15min":2.431},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.59099999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.88300000000000001,"15min":0.84699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":9.5269999999999992,"5min":9.5269999999999992,"15min":9.5269999999999992},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.83699999999999997,"15min":0.81999999999999995},"min":{"1min":0.65000000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.7709999999999999,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.85499999999999998,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":2.536,"15min":2.536},"last":0.70199999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.69699999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.754},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.97299999999999998,"15min":0.97499999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":6.1559999999999997,"5min":6.1559999999999997,"15min":6.1559999999999997},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86799999999999999,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95999999999999996,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.70299999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.054},"last":0.68500000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":514461,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.38}]},{"osd":1,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":1.0289999999999999},"min":{"1min":0.51500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.355,"5min":6.1639999999999997,"15min":6.1639999999999997},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.439},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0620000000000001,"5min":1.073,"15min":1.073},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85099999999999998,"15min":0.877},"min":{"1min":0.53900000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1679999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88400000000000001,"15min":0.92200000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.65400000000000003}]},{"osd":8,"last update":"Fri Jul 14 11:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.077,"15min":1.1850000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3210000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.129,"5min":1.171,"15min":1.171},"last":0.628}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.503},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.4489999999999998,"5min":2.8900000000000001,"15min":2.8900000000000001},"last":0.70999999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91500000000000004,"15min":0.93899999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.04,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.88600000000000001,"15min":0.90100000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0269999999999999,"5min":1.141,"15min":1.141},"last":0.80300000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":510173,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.50700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.436,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.68100000000000005,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.93000000000000005,"5min":5.1680000000000001,"15min":5.1680000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48499999999999999,"15min":0.44800000000000001},"min":{"1min":0.40600000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.008,"5min":2.032,"15min":2.032},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.998,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.998}]},{"osd":4,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.64400000000000002,"15min":0.59899999999999998},"min":{"1min":0.36099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.0329999999999999,"5min":1.046,"15min":1.046},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.69299999999999995,"15min":0.64100000000000001},"min":{"1min":0.68400000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.437,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":1.038,"15min":1.038},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.76000000000000001,"15min":0.72599999999999998},"min":{"1min":0.56699999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.048,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.048},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.68999999999999995},"min":{"1min":0.504,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.90100000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.71599999999999997,"15min":0.67700000000000005},"min":{"1min":0.55900000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.95799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.751,"15min":0.71699999999999997},"min":{"1min":0.63100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.02},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.80800000000000005,"15min":0.77400000000000002},"min":{"1min":0.69599999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.1299999999999999,"5min":1.319,"15min":1.319},"last":1.0800000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.70499999999999996},"min":{"1min":0.65100000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.79900000000000004,"15min":0.749},"min":{"1min":0.71199999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.093,"5min":1.218,"15min":1.218},"last":1.105}]},{"osd":11,"last update":"Fri Jul 14 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.84699999999999998,"15min":0.84499999999999997},"min":{"1min":0.64600000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.077,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.94599999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":1105376,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.32600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.376},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.34000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.378,"15min":0.378},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.621,"15min":0.58699999999999997},"min":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.2450000000000001,"5min":4.2450000000000001,"15min":4.2450000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.36299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85199999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.629}]},{"osd":6,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.75600000000000001,"15min":0.73599999999999999},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.3390000000000004,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69099999999999995,"15min":0.68600000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.51200000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.746,"15min":0.69999999999999996},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":2.762,"15min":2.762},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.51700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.075,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.752,"15min":0.748},"min":{"1min":0.56399999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.754}]},{"osd":10,"last update":"Fri Jul 14 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.75800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.501,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.052,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70599999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":505268,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.51900000000000002,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":0.378},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.55400000000000005,"15min":0.497},"min":{"1min":0.33000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.1180000000000003,"5min":5.1180000000000003,"15min":5.1180000000000003},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.65800000000000003},"min":{"1min":0.55800000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.86899999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.2360000000000002,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.71099999999999997,"15min":0.66600000000000004},"min":{"1min":0.64200000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68100000000000005,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.443,"15min":0.443},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.77400000000000002,"15min":0.753},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.76200000000000001,"15min":0.69999999999999996},"min":{"1min":0.57599999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.2730000000000001,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.73599999999999999},"min":{"1min":0.55900000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3009999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73799999999999999,"15min":0.72299999999999998},"min":{"1min":0.52800000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.745},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.78100000000000003},"min":{"1min":0.621,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Jul 14 11:21:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.81299999999999994,"15min":0.80300000000000005},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.83199999999999996,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0620000000000001,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.78300000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84699999999999998,"15min":0.85799999999999998},"min":{"1min":0.495,"5min":0.442,"15min":0.442},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.89800000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93400000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Jul 14 11:21:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.234,"5min":1.359,"15min":1.359},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.998,"15min":1.03},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3029999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":562986,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.32400000000000001},"max":{"1min":4.077,"5min":4.077,"15min":4.077},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61199999999999999,"15min":0.51600000000000001},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.625,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.32200000000000001},"max":{"1min":0.82599999999999996,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70199999999999996,"15min":0.64900000000000002},"min":{"1min":0.44700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":4.016,"5min":4.016,"15min":4.016},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51900000000000002,"15min":0.47199999999999998},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68899999999999995,"15min":0.63400000000000001},"min":{"1min":0.54500000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67700000000000005,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Jul 14 11:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.495,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.1459999999999999,"5min":4.1459999999999999,"15min":4.1459999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.69199999999999995},"min":{"1min":0.59499999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.63600000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97699999999999998,"5min":1.036,"15min":1.036},"last":0.95899999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79100000000000004,"15min":0.78600000000000003},"min":{"1min":0.58699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.88300000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.439,"15min":0.439},"max":{"1min":0.98699999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.208}]},{"osd":11,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.016},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1,"5min":1.044,"15min":1.044},"last":1.0349999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":506656,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65400000000000003,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":0.504},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.33900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.59299999999999997},"min":{"1min":0.45600000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.873,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.97499999999999998,"5min":5.2060000000000004,"15min":5.2060000000000004},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.71599999999999997,"15min":0.61799999999999999},"min":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6680000000000001,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.57299999999999995},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90200000000000002,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72299999999999998,"15min":0.63900000000000001},"min":{"1min":0.51200000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.66800000000000004},"min":{"1min":0.48899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.996,"5min":5.2489999999999997,"15min":5.2489999999999997},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64700000000000002,"15min":0.63200000000000001},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.748,"15min":0.67600000000000005},"min":{"1min":0.442,"5min":0.38,"15min":0.38},"max":{"1min":6.3150000000000004,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.91900000000000004},"min":{"1min":0.60999999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.88600000000000001,"15min":0.91700000000000004},"min":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92800000000000005,"5min":5.351,"15min":5.351},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jul 14 11:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.84399999999999997,"15min":0.86299999999999999},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78300000000000003,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.036,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.748}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85499999999999998,"15min":0.89300000000000002},"min":{"1min":0.58899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.577,"15min":1.577},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.075,"5min":0.93000000000000005,"15min":0.92900000000000005},"min":{"1min":0.62,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.2720000000000002,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.64000000000000001}]},{"osd":10,"last update":"Fri Jul 14 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.89300000000000002,"15min":0.86899999999999999},"min":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":5.7080000000000002,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.86299999999999999,"15min":0.88800000000000001},"min":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.018,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.58199999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.85399999999999998,"15min":0.81599999999999995},"min":{"1min":0.628,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.88600000000000001,"15min":0.84399999999999997},"min":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":0.81699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":563981,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72699999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":1.012,"15min":1.012},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.29399999999999998},"max":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.628,"15min":0.54400000000000004},"min":{"1min":0.57799999999999996,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.317,"5min":0.254,"15min":0.254},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.50600000000000001},"min":{"1min":0.45500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64800000000000002,"15min":0.626},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jul 14 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.50900000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.70799999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66400000000000003,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.95899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Jul 14 11:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.75900000000000001},"min":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65300000000000002}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.505,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90100000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.69699999999999995,"15min":0.67300000000000004},"min":{"1min":0.622,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89400000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri Jul 14 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.748,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.54300000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93899999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56399999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":505735,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.52800000000000002,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.4649999999999999,"5min":7.4649999999999999,"15min":7.4649999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.54700000000000004,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":6.4809999999999999,"5min":6.4809999999999999,"15min":6.4809999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.71699999999999997,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.7329999999999997,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.73799999999999999,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.376,"15min":0.376},"max":{"1min":6.335,"5min":6.335,"15min":6.335},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60399999999999998,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.69999999999999996,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.73799999999999999,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":6.242,"5min":6.242,"15min":6.242},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77700000000000002,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.133,"5min":6.8570000000000002,"15min":6.8570000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.65300000000000002},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.663,"15min":1.663},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:21:09 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.746,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.77500000000000002,"15min":0.69799999999999995},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.5140000000000002,"5min":6.5140000000000002,"15min":6.5140000000000002},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:21:32 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.83199999999999996,"15min":0.755},"min":{"1min":0.57199999999999995,"5min":0.505,"15min":0.505},"max":{"1min":6.415,"5min":6.415,"15min":6.415},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.63400000000000001}]},{"osd":8,"last update":"Fri Jul 14 11:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":0.94799999999999995,"15min":0.89200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":13.365,"5min":13.365,"15min":13.365},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.048,"5min":0.98899999999999999,"15min":1.018},"min":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.4489999999999998,"5min":6.4489999999999998,"15min":6.4489999999999998},"last":0.68700000000000006}]},{"osd":10,"last update":"Fri Jul 14 11:21:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":6.9649999999999999,"5min":6.9649999999999999,"15min":6.9649999999999999},"last":0.751},{"interface":"front","average":{"1min":1.069,"5min":0.86699999999999999,"15min":0.84399999999999997},"min":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.55800000000000005}]},{"osd":11,"last update":"Fri Jul 14 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93700000000000006,"5min":1.7,"15min":1.7},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.0780000000000001,"15min":1.1200000000000001},"min":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1759999999999999,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.80800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":510143,"data_compressed":23585,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57399999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.432},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.40000000000000002},"min":{"1min":0.28199999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86399999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.745},{"interface":"front","average":{"1min":0.629,"5min":0.60099999999999998,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84199999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Jul 14 11:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.69899999999999995,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3289999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":5,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.40100000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82699999999999996,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.622,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.745,"15min":0.72299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.7779999999999996,"5min":5.7779999999999996,"15min":5.7779999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.496,"5min":0.438,"15min":0.438},"max":{"1min":5.7309999999999999,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:21:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.47999999999999998},"min":{"1min":0.32100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77300000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68300000000000005,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91800000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.628,"15min":0.59099999999999997},"min":{"1min":0.54800000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90700000000000003,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.73099999999999998,"15min":0.69899999999999995},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.262,"5min":0.85899999999999999,"15min":0.80500000000000005},"min":{"1min":0.48199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.2640000000000002,"5min":6.2640000000000002,"15min":6.2640000000000002},"last":0.84499999999999997}]},{"osd":10,"last update":"Fri Jul 14 11:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.78600000000000003},"min":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79600000000000004,"15min":0.79200000000000004},"min":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97499999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.82399999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1073152,"data_stored":1101012,"data_compressed":59282,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58799999999999997,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77300000000000002,"5min":2.843,"15min":2.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67300000000000004,"5min":6.6680000000000001,"15min":6.6680000000000001},"last":0.379}]},{"osd":1,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.58199999999999996,"15min":0.503},"min":{"1min":0.49099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.51700000000000002},"min":{"1min":0.46200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.504}]},{"osd":2,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.36899999999999999},"min":{"1min":0.34899999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.438,"15min":0.39500000000000002},"min":{"1min":0.36699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.622,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.59299999999999997},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.747,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61499999999999999,"15min":0.55400000000000005},"min":{"1min":0.52100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94799999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.55000000000000004},"min":{"1min":0.55000000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.68200000000000005,"15min":0.60999999999999999},"min":{"1min":0.54400000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.65100000000000002,"15min":0.60499999999999998},"min":{"1min":0.49399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Jul 14 11:21:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64000000000000001,"15min":0.60199999999999998},"min":{"1min":0.59699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.625},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.52600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.038},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82899999999999996,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67100000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.751,"15min":0.72599999999999998},"min":{"1min":0.61899999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.77400000000000002,"15min":0.75},"min":{"1min":0.63,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73099999999999998,"15min":0.71399999999999997},"min":{"1min":0.60299999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.748,"15min":0.72399999999999998},"min":{"1min":0.67800000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.996,"5min":1.073,"15min":1.073},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35697,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:22:02.039 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:22:02.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 -- 172.21.15.161:0/1017493680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f8102ed0 msgr2=0x7fd0f81052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1017493680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f8102ed0 0x7fd0f81052b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009990 tx=0x7fd0e8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 -- 172.21.15.161:0/1017493680 shutdown_connections 2023-07-14T11:22:02.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1017493680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0f81057f0 0x7fd0f8107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1017493680 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0f81005b0 0x7fd0f8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1017493680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f8102ed0 0x7fd0f81052b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 -- 172.21.15.161:0/1017493680 >> 172.21.15.161:0/1017493680 conn(0x7fd0f80f9fd0 msgr2=0x7fd0f80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:02.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 -- 172.21.15.161:0/1017493680 shutdown_connections 2023-07-14T11:22:02.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.269+0000 7fd0fffff700 1 -- 172.21.15.161:0/1017493680 wait complete. 2023-07-14T11:22:02.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.270+0000 7fd0fffff700 1 Processor -- start 2023-07-14T11:22:02.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.271+0000 7fd0fffff700 1 -- start start 2023-07-14T11:22:02.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.271+0000 7fd0fffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0f81005b0 0x7fd0f80fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.271+0000 7fd0fffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0f8102ed0 0x7fd0f80fe560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.271+0000 7fd0fffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f81057f0 0x7fd0f80feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.271+0000 7fd0fffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80ff240 con 0x7fd0f81057f0 2023-07-14T11:22:02.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.271+0000 7fd0fffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80ff3b0 con 0x7fd0f8102ed0 2023-07-14T11:22:02.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.271+0000 7fd0fffff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f819f800 con 0x7fd0f81005b0 2023-07-14T11:22:02.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f81057f0 0x7fd0f80feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0f81005b0 0x7fd0f80fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f81057f0 0x7fd0f80feaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39716/0 (socket says 172.21.15.161:39716) 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 -- 172.21.15.161:0/1376524293 learned_addr learned my addr 172.21.15.161:0/1376524293 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fd59a700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0f8102ed0 0x7fd0f80fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 -- 172.21.15.161:0/1376524293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0f81005b0 msgr2=0x7fd0f80fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0f81005b0 0x7fd0f80fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 -- 172.21.15.161:0/1376524293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0f8102ed0 msgr2=0x7fd0f80fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0f8102ed0 0x7fd0f80fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 -- 172.21.15.161:0/1376524293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80095f0 con 0x7fd0f81057f0 2023-07-14T11:22:02.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fe59c700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f81057f0 0x7fd0f80feaa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0007ee0 tx=0x7fd0f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:02.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0eeffd700 1 -- 172.21.15.161:0/1376524293 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0017070 con 0x7fd0f81057f0 2023-07-14T11:22:02.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f819fae0 con 0x7fd0f81057f0 2023-07-14T11:22:02.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0eeffd700 1 -- 172.21.15.161:0/1376524293 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f000f040 con 0x7fd0f81057f0 2023-07-14T11:22:02.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.272+0000 7fd0eeffd700 1 -- 172.21.15.161:0/1376524293 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0005550 con 0x7fd0f81057f0 2023-07-14T11:22:02.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.273+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f819fff0 con 0x7fd0f81057f0 2023-07-14T11:22:02.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.274+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f80fbb40 con 0x7fd0f81057f0 2023-07-14T11:22:02.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.275+0000 7fd0eeffd700 1 -- 172.21.15.161:0/1376524293 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd0f000db20 con 0x7fd0f81057f0 2023-07-14T11:22:02.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.277+0000 7fd0eeffd700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd0e40a11a0 0x7fd0e40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.277+0000 7fd0eeffd700 1 -- 172.21.15.161:0/1376524293 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd0f00cc3a0 con 0x7fd0f81057f0 2023-07-14T11:22:02.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.278+0000 7fd0fdd9b700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd0e40a11a0 0x7fd0e40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.278+0000 7fd0fdd9b700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd0e40a11a0 0x7fd0e40a3650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009c60 tx=0x7fd0f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:02.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.281+0000 7fd0eeffd700 1 -- 172.21.15.161:0/1376524293 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd0f0095740 con 0x7fd0f81057f0 2023-07-14T11:22:02.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.482+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0f802c850 con 0x7fd0f81057f0 2023-07-14T11:22:02.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.484+0000 7fd0eeffd700 1 -- 172.21.15.161:0/1376524293 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7fd0f0098d60 con 0x7fd0f81057f0 2023-07-14T11:22:02.486 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:02.487 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:22:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd0e40a11a0 msgr2=0x7fd0e40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd0e40a11a0 0x7fd0e40a3650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009c60 tx=0x7fd0f4009260 comp rx=0 tx=0).stop 2023-07-14T11:22:02.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f81057f0 msgr2=0x7fd0f80feaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f81057f0 0x7fd0f80feaa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0007ee0 tx=0x7fd0f000d3b0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 shutdown_connections 2023-07-14T11:22:02.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0f81057f0 0x7fd0f80feaa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fd0e40a11a0 0x7fd0e40a3650 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0f81005b0 0x7fd0f80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 --2- 172.21.15.161:0/1376524293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0f8102ed0 0x7fd0f80fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 >> 172.21.15.161:0/1376524293 conn(0x7fd0f80f9fd0 msgr2=0x7fd0f8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:02.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 shutdown_connections 2023-07-14T11:22:02.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.486+0000 7fd0fffff700 1 -- 172.21.15.161:0/1376524293 wait complete. 2023-07-14T11:22:02.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:02 smithi161 ceph-mon[95526]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:02 smithi161 ceph-mon[95526]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:02.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:22:02.509 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:22:02.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:02 smithi175 ceph-mon[129081]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:02.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:02 smithi175 ceph-mon[129081]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:02.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.709+0000 7f121e3ff700 1 -- 172.21.15.161:0/1069400769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218074550 msgr2=0x7f12180729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.709+0000 7f121e3ff700 1 --2- 172.21.15.161:0/1069400769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218074550 0x7f12180729a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1208009990 tx=0x7f1208009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.709+0000 7f121e3ff700 1 -- 172.21.15.161:0/1069400769 shutdown_connections 2023-07-14T11:22:02.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.709+0000 7f121e3ff700 1 --2- 172.21.15.161:0/1069400769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218074550 0x7f12180729a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.709+0000 7f121e3ff700 1 --2- 172.21.15.161:0/1069400769 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f12181079b0 0x7f1218109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.709+0000 7f121e3ff700 1 --2- 172.21.15.161:0/1069400769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 0x7f12180733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.709+0000 7f121e3ff700 1 -- 172.21.15.161:0/1069400769 >> 172.21.15.161:0/1069400769 conn(0x7f12180fb130 msgr2=0x7f12180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:02.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.710+0000 7f121e3ff700 1 -- 172.21.15.161:0/1069400769 shutdown_connections 2023-07-14T11:22:02.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.710+0000 7f121e3ff700 1 -- 172.21.15.161:0/1069400769 wait complete. 2023-07-14T11:22:02.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 Processor -- start 2023-07-14T11:22:02.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 -- start start 2023-07-14T11:22:02.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 0x7f1218197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1218074550 0x7f1218197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12181079b0 0x7f121819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121819f620 con 0x7f12181079b0 2023-07-14T11:22:02.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121819f760 con 0x7f1218072f70 2023-07-14T11:22:02.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f121e3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121819f8f0 con 0x7f1218074550 2023-07-14T11:22:02.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f1217fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 0x7f1218197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f1217fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 0x7f1218197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54522/0 (socket says 172.21.15.161:54522) 2023-07-14T11:22:02.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f1217fff700 1 -- 172.21.15.161:0/2848903458 learned_addr learned my addr 172.21.15.161:0/2848903458 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:02.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.711+0000 7f12177fe700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1218074550 0x7f1218197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.712+0000 7f121c99c700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12181079b0 0x7f121819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.712+0000 7f1217fff700 1 -- 172.21.15.161:0/2848903458 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1218074550 msgr2=0x7f1218197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.712+0000 7f1217fff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1218074550 0x7f1218197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.712+0000 7f1217fff700 1 -- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12181079b0 msgr2=0x7f121819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.712+0000 7f1217fff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12181079b0 0x7f121819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.712+0000 7f1217fff700 1 -- 172.21.15.161:0/2848903458 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080095f0 con 0x7f1218072f70 2023-07-14T11:22:02.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.712+0000 7f1217fff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 0x7f1218197aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1208005720 tx=0x7f1208009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:02.718 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:02 smithi182 ceph-mon[129366]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:02.718 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:02 smithi182 ceph-mon[129366]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:02.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.714+0000 7f12157fa700 1 -- 172.21.15.161:0/2848903458 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120801d070 con 0x7f1218072f70 2023-07-14T11:22:02.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.714+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121819fb70 con 0x7f1218072f70 2023-07-14T11:22:02.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.714+0000 7f12157fa700 1 -- 172.21.15.161:0/2848903458 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1208021410 con 0x7f1218072f70 2023-07-14T11:22:02.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.714+0000 7f12157fa700 1 -- 172.21.15.161:0/2848903458 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208017650 con 0x7f1218072f70 2023-07-14T11:22:02.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.714+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121819ff90 con 0x7f1218072f70 2023-07-14T11:22:02.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.715+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1218106060 con 0x7f1218072f70 2023-07-14T11:22:02.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.717+0000 7f12157fa700 1 -- 172.21.15.161:0/2848903458 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1208021580 con 0x7f1218072f70 2023-07-14T11:22:02.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.718+0000 7f12157fa700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f12000a1150 0x7f12000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:02.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.718+0000 7f12157fa700 1 -- 172.21.15.161:0/2848903458 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f12080cddf0 con 0x7f1218072f70 2023-07-14T11:22:02.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.718+0000 7f12177fe700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f12000a1150 0x7f12000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:02.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.719+0000 7f12177fe700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f12000a1150 0x7f12000a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1218074270 tx=0x7f120c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:02.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.720+0000 7f12157fa700 1 -- 172.21.15.161:0/2848903458 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1208097280 con 0x7f1218072f70 2023-07-14T11:22:02.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.922+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f121802c9e0 con 0x7f1218072f70 2023-07-14T11:22:02.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.922+0000 7f12157fa700 1 -- 172.21.15.161:0/2848903458 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f120809a8a0 con 0x7f1218072f70 2023-07-14T11:22:02.925 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:22:02.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f12000a1150 msgr2=0x7f12000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f12000a1150 0x7f12000a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1218074270 tx=0x7f120c009260 comp rx=0 tx=0).stop 2023-07-14T11:22:02.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 msgr2=0x7f1218197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:02.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 0x7f1218197aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1208005720 tx=0x7f1208009580 comp rx=0 tx=0).stop 2023-07-14T11:22:02.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 shutdown_connections 2023-07-14T11:22:02.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12181079b0 0x7f121819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f12000a1150 0x7f12000a3600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1218072f70 0x7f1218197aa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 --2- 172.21.15.161:0/2848903458 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1218074550 0x7f1218197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:02.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 >> 172.21.15.161:0/2848903458 conn(0x7f12180fb130 msgr2=0x7f12180fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:02.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 shutdown_connections 2023-07-14T11:22:02.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:02.925+0000 7f121e3ff700 1 -- 172.21.15.161:0/2848903458 wait complete. 2023-07-14T11:22:02.946 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:22:03.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 -- 172.21.15.161:0/2638404855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff0101670 msgr2=0x7fbff0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 --2- 172.21.15.161:0/2638404855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff0101670 0x7fbff0107f30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800b0a0 tx=0x7fbfe800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 -- 172.21.15.161:0/2638404855 shutdown_connections 2023-07-14T11:22:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 --2- 172.21.15.161:0/2638404855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff0101670 0x7fbff0107f30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 --2- 172.21.15.161:0/2638404855 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbff00ffaf0 0x7fbff00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 --2- 172.21.15.161:0/2638404855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff0100ce0 0x7fbff0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 -- 172.21.15.161:0/2638404855 >> 172.21.15.161:0/2638404855 conn(0x7fbff00fb130 msgr2=0x7fbff00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 -- 172.21.15.161:0/2638404855 shutdown_connections 2023-07-14T11:22:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.146+0000 7fbff4dff700 1 -- 172.21.15.161:0/2638404855 wait complete. 2023-07-14T11:22:03.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.147+0000 7fbff4dff700 1 Processor -- start 2023-07-14T11:22:03.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.147+0000 7fbff4dff700 1 -- start start 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbff4dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff00ffaf0 0x7fbff0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbff4dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff0100ce0 0x7fbff0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbff4dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbff0101670 0x7fbff019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbff4dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff019f6b0 con 0x7fbff00ffaf0 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbff4dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff019f7f0 con 0x7fbff0100ce0 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbff4dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff019f980 con 0x7fbff0101670 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff00ffaf0 0x7fbff0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff00ffaf0 0x7fbff0197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39756/0 (socket says 172.21.15.161:39756) 2023-07-14T11:22:03.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 -- 172.21.15.161:0/3104257492 learned_addr learned my addr 172.21.15.161:0/3104257492 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:03.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfeed9d700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbff0101670 0x7fbff019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfe7fff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff0100ce0 0x7fbff0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 -- 172.21.15.161:0/3104257492 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbff0101670 msgr2=0x7fbff019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbff0101670 0x7fbff019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 -- 172.21.15.161:0/3104257492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff0100ce0 msgr2=0x7fbff0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff0100ce0 0x7fbff0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 -- 172.21.15.161:0/3104257492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8009d00 con 0x7fbff00ffaf0 2023-07-14T11:22:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfe7fff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff0100ce0 0x7fbff0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfeed9d700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbff0101670 0x7fbff019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.148+0000 7fbfee59c700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff00ffaf0 0x7fbff0197b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000d980 tx=0x7fbfe000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.149+0000 7fbfe77fe700 1 -- 172.21.15.161:0/3104257492 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000de60 con 0x7fbff00ffaf0 2023-07-14T11:22:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.149+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff019fc60 con 0x7fbff00ffaf0 2023-07-14T11:22:03.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.149+0000 7fbfe77fe700 1 -- 172.21.15.161:0/3104257492 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfe00044e0 con 0x7fbff00ffaf0 2023-07-14T11:22:03.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.149+0000 7fbfe77fe700 1 -- 172.21.15.161:0/3104257492 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0003be0 con 0x7fbff00ffaf0 2023-07-14T11:22:03.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.149+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff01a0170 con 0x7fbff00ffaf0 2023-07-14T11:22:03.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.150+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff010b320 con 0x7fbff00ffaf0 2023-07-14T11:22:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.152+0000 7fbfe77fe700 1 -- 172.21.15.161:0/3104257492 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbfe0009bb0 con 0x7fbff00ffaf0 2023-07-14T11:22:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.153+0000 7fbfe77fe700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfdc0a1150 0x7fbfdc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.153+0000 7fbfe77fe700 1 -- 172.21.15.161:0/3104257492 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbfe00cce00 con 0x7fbff00ffaf0 2023-07-14T11:22:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.154+0000 7fbfe7fff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfdc0a1150 0x7fbfdc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.154+0000 7fbfe7fff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfdc0a1150 0x7fbfdc0a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009cb0 tx=0x7fbfd8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:03.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.156+0000 7fbfe77fe700 1 -- 172.21.15.161:0/3104257492 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbfe00961a0 con 0x7fbff00ffaf0 2023-07-14T11:22:03.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.354+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbff01a0830 con 0x7fbff00ffaf0 2023-07-14T11:22:03.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.355+0000 7fbfe77fe700 1 -- 172.21.15.161:0/3104257492 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbfe0009940 con 0x7fbff00ffaf0 2023-07-14T11:22:03.358 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:22:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfdc0a1150 msgr2=0x7fbfdc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfdc0a1150 0x7fbfdc0a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009cb0 tx=0x7fbfd8009260 comp rx=0 tx=0).stop 2023-07-14T11:22:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff00ffaf0 msgr2=0x7fbff0197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff00ffaf0 0x7fbff0197b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000d980 tx=0x7fbfe000dd40 comp rx=0 tx=0).stop 2023-07-14T11:22:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 shutdown_connections 2023-07-14T11:22:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbff00ffaf0 0x7fbff0197b30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fbfdc0a1150 0x7fbfdc0a3600 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbff0101670 0x7fbff019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 --2- 172.21.15.161:0/3104257492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff0100ce0 0x7fbff0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 >> 172.21.15.161:0/3104257492 conn(0x7fbff00fb130 msgr2=0x7fbff00fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:03.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 shutdown_connections 2023-07-14T11:22:03.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.357+0000 7fbff4dff700 1 -- 172.21.15.161:0/3104257492 wait complete. 2023-07-14T11:22:03.378 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:22:03.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 -- 172.21.15.161:0/581981056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 msgr2=0x7fe3480fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 --2- 172.21.15.161:0/581981056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe3480fff00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009990 tx=0x7fe33c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 -- 172.21.15.161:0/581981056 shutdown_connections 2023-07-14T11:22:03.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 --2- 172.21.15.161:0/581981056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe3480fff00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 --2- 172.21.15.161:0/581981056 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe348101670 0x7fe348107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 --2- 172.21.15.161:0/581981056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348100ce0 0x7fe348101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 -- 172.21.15.161:0/581981056 >> 172.21.15.161:0/581981056 conn(0x7fe3480fb130 msgr2=0x7fe3480fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:03.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.579+0000 7fe34edff700 1 -- 172.21.15.161:0/581981056 shutdown_connections 2023-07-14T11:22:03.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.580+0000 7fe34edff700 1 -- 172.21.15.161:0/581981056 wait complete. 2023-07-14T11:22:03.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 Processor -- start 2023-07-14T11:22:03.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 -- start start 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe348195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348100ce0 0x7fe348195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe348101670 0x7fe34819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34819d3f0 con 0x7fe3480ffaf0 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34819d530 con 0x7fe348100ce0 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34edff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34819d6c0 con 0x7fe348101670 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34cb9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe348195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34cb9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe348195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39774/0 (socket says 172.21.15.161:39774) 2023-07-14T11:22:03.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.581+0000 7fe34cb9b700 1 -- 172.21.15.161:0/3761985839 learned_addr learned my addr 172.21.15.161:0/3761985839 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:03.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe347fff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348100ce0 0x7fe348195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34d39c700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe348101670 0x7fe34819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34cb9b700 1 -- 172.21.15.161:0/3761985839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe348101670 msgr2=0x7fe34819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34cb9b700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe348101670 0x7fe34819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34cb9b700 1 -- 172.21.15.161:0/3761985839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348100ce0 msgr2=0x7fe348195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34cb9b700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348100ce0 0x7fe348195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34cb9b700 1 -- 172.21.15.161:0/3761985839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c0095f0 con 0x7fe3480ffaf0 2023-07-14T11:22:03.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34cb9b700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe348195870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe33c000c00 tx=0x7fe33c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:03.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe345ffb700 1 -- 172.21.15.161:0/3761985839 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c01d070 con 0x7fe3480ffaf0 2023-07-14T11:22:03.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34819d940 con 0x7fe3480ffaf0 2023-07-14T11:22:03.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe345ffb700 1 -- 172.21.15.161:0/3761985839 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33c003f60 con 0x7fe3480ffaf0 2023-07-14T11:22:03.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.582+0000 7fe345ffb700 1 -- 172.21.15.161:0/3761985839 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c022b50 con 0x7fe3480ffaf0 2023-07-14T11:22:03.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.583+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34819dd70 con 0x7fe3480ffaf0 2023-07-14T11:22:03.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.584+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34810b310 con 0x7fe3480ffaf0 2023-07-14T11:22:03.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.586+0000 7fe345ffb700 1 -- 172.21.15.161:0/3761985839 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe33c0040d0 con 0x7fe3480ffaf0 2023-07-14T11:22:03.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.587+0000 7fe345ffb700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3380a1150 0x7fe3380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:03.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.587+0000 7fe345ffb700 1 -- 172.21.15.161:0/3761985839 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe33c0ce2e0 con 0x7fe3480ffaf0 2023-07-14T11:22:03.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.587+0000 7fe347fff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3380a1150 0x7fe3380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:03.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.588+0000 7fe347fff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3380a1150 0x7fe3380a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe334007c60 tx=0x7fe3340073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:03.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.591+0000 7fe345ffb700 1 -- 172.21.15.161:0/3761985839 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe33c097770 con 0x7fe3480ffaf0 2023-07-14T11:22:03.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:03 smithi175 ceph-mon[129081]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:03.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:03 smithi175 ceph-mon[129081]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:03 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1376524293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:22:03.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:03 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2848903458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:22:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:03 smithi182 ceph-mon[129366]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:03 smithi182 ceph-mon[129366]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:03 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1376524293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:22:03.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:03 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2848903458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:22:03.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:03 smithi161 ceph-mon[95526]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:22:03.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:03 smithi161 ceph-mon[95526]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:03.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:03 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1376524293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:22:03.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:03 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2848903458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:22:03.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.788+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe348106a20 con 0x7fe3480ffaf0 2023-07-14T11:22:03.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.789+0000 7fe345ffb700 1 -- 172.21.15.161:0/3761985839 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe33c09ad90 con 0x7fe3480ffaf0 2023-07-14T11:22:03.791 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:22:03.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3380a1150 msgr2=0x7fe3380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3380a1150 0x7fe3380a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe334007c60 tx=0x7fe3340073d0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 msgr2=0x7fe348195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe348195870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe33c000c00 tx=0x7fe33c005740 comp rx=0 tx=0).stop 2023-07-14T11:22:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 shutdown_connections 2023-07-14T11:22:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3480ffaf0 0x7fe348195870 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7fe3380a1150 0x7fe3380a3600 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe348101670 0x7fe34819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 --2- 172.21.15.161:0/3761985839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348100ce0 0x7fe348195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 >> 172.21.15.161:0/3761985839 conn(0x7fe3480fb130 msgr2=0x7fe3480fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 shutdown_connections 2023-07-14T11:22:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:03.791+0000 7fe34edff700 1 -- 172.21.15.161:0/3761985839 wait complete. 2023-07-14T11:22:03.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:22:04.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/2844817720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f94101670 msgr2=0x7f5f94107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/2844817720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f94101670 0x7f5f94107f30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00b0a0 tx=0x7f5f8c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/2844817720 shutdown_connections 2023-07-14T11:22:04.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/2844817720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f94101670 0x7f5f94107f30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/2844817720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f940ffaf0 0x7f5f940fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/2844817720 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5f94100ce0 0x7f5f94101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/2844817720 >> 172.21.15.161:0/2844817720 conn(0x7f5f940fb130 msgr2=0x7f5f940fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:04.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/2844817720 shutdown_connections 2023-07-14T11:22:04.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.013+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/2844817720 wait complete. 2023-07-14T11:22:04.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 Processor -- start 2023-07-14T11:22:04.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 -- start start 2023-07-14T11:22:04.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f940ffaf0 0x7f5f94195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f94100ce0 0x7f5f94195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5f94101670 0x7f5f9419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9419d4e0 con 0x7f5f940ffaf0 2023-07-14T11:22:04.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9419d620 con 0x7f5f94100ce0 2023-07-14T11:22:04.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.015+0000 7f5f9c3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9419d7b0 con 0x7f5f94101670 2023-07-14T11:22:04.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f94100ce0 0x7f5f94195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9a19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f940ffaf0 0x7f5f94195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f94100ce0 0x7f5f94195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54590/0 (socket says 172.21.15.161:54590) 2023-07-14T11:22:04.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 -- 172.21.15.161:0/704944390 learned_addr learned my addr 172.21.15.161:0/704944390 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:04.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9a99c700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5f94101670 0x7f5f9419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 -- 172.21.15.161:0/704944390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5f94101670 msgr2=0x7f5f9419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5f94101670 0x7f5f9419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 -- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f940ffaf0 msgr2=0x7f5f94195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f940ffaf0 0x7f5f94195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 -- 172.21.15.161:0/704944390 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90009520 con 0x7f5f94100ce0 2023-07-14T11:22:04.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9a19b700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f940ffaf0 0x7f5f94195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:04.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.016+0000 7f5f9999a700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f94100ce0 0x7f5f94195ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400eb30 tx=0x7f5f8400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:04.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.017+0000 7f5f8b7fe700 1 -- 172.21.15.161:0/704944390 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84009880 con 0x7f5f94100ce0 2023-07-14T11:22:04.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.017+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f84009520 con 0x7f5f94100ce0 2023-07-14T11:22:04.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.017+0000 7f5f8b7fe700 1 -- 172.21.15.161:0/704944390 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f840044e0 con 0x7f5f94100ce0 2023-07-14T11:22:04.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.017+0000 7f5f8b7fe700 1 -- 172.21.15.161:0/704944390 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84003e90 con 0x7f5f94100ce0 2023-07-14T11:22:04.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.017+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c009d00 con 0x7f5f94100ce0 2023-07-14T11:22:04.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.018+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f94106980 con 0x7f5f94100ce0 2023-07-14T11:22:04.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.020+0000 7f5f8b7fe700 1 -- 172.21.15.161:0/704944390 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5f84004000 con 0x7f5f94100ce0 2023-07-14T11:22:04.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.021+0000 7f5f8b7fe700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5f800a1150 0x7f5f800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.022+0000 7f5f8b7fe700 1 -- 172.21.15.161:0/704944390 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5f84014070 con 0x7f5f94100ce0 2023-07-14T11:22:04.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.022+0000 7f5f9a19b700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5f800a1150 0x7f5f800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.022+0000 7f5f9a19b700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5f800a1150 0x7f5f800a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009fd0 tx=0x7f5f90009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:04.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.023+0000 7f5f8b7fe700 1 -- 172.21.15.161:0/704944390 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5f84096620 con 0x7f5f94100ce0 2023-07-14T11:22:04.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.225+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5f9402c9e0 con 0x7f5f94100ce0 2023-07-14T11:22:04.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.226+0000 7f5f8b7fe700 1 -- 172.21.15.161:0/704944390 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5f8400f3c0 con 0x7f5f94100ce0 2023-07-14T11:22:04.228 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:22:04.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5f800a1150 msgr2=0x7f5f800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5f800a1150 0x7f5f800a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009fd0 tx=0x7f5f90009260 comp rx=0 tx=0).stop 2023-07-14T11:22:04.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f94100ce0 msgr2=0x7f5f94195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f94100ce0 0x7f5f94195ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400eb30 tx=0x7f5f8400ee40 comp rx=0 tx=0).stop 2023-07-14T11:22:04.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 shutdown_connections 2023-07-14T11:22:04.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f940ffaf0 0x7f5f94195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5f800a1150 0x7f5f800a3600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5f94101670 0x7f5f9419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 --2- 172.21.15.161:0/704944390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f94100ce0 0x7f5f94195ea0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 >> 172.21.15.161:0/704944390 conn(0x7f5f940fb130 msgr2=0x7f5f940fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:04.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 shutdown_connections 2023-07-14T11:22:04.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.228+0000 7f5f9c3ff700 1 -- 172.21.15.161:0/704944390 wait complete. 2023-07-14T11:22:04.249 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:22:04.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 -- 172.21.15.161:0/777057770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 msgr2=0x7f17741052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 --2- 172.21.15.161:0/777057770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f17741052b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f175c009990 tx=0x7f175c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 -- 172.21.15.161:0/777057770 shutdown_connections 2023-07-14T11:22:04.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 --2- 172.21.15.161:0/777057770 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f17741057f0 0x7f1774107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 --2- 172.21.15.161:0/777057770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17741005b0 0x7f1774102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 --2- 172.21.15.161:0/777057770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f17741052b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 -- 172.21.15.161:0/777057770 >> 172.21.15.161:0/777057770 conn(0x7f17740f9fd0 msgr2=0x7f17740fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:04.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 -- 172.21.15.161:0/777057770 shutdown_connections 2023-07-14T11:22:04.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.447+0000 7f1778bff700 1 -- 172.21.15.161:0/777057770 wait complete. 2023-07-14T11:22:04.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.448+0000 7f1778bff700 1 Processor -- start 2023-07-14T11:22:04.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.448+0000 7f1778bff700 1 -- start start 2023-07-14T11:22:04.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f1778bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17741005b0 0x7f1774195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f1778bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f1774195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f1778bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f17741057f0 0x7f177419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f1778bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177419d380 con 0x7f1774102ed0 2023-07-14T11:22:04.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f1778bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177419d4c0 con 0x7f17741005b0 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f1778bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177419d650 con 0x7f17741057f0 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f176bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f1774195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f176bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f1774195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39796/0 (socket says 172.21.15.161:39796) 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f176bfff700 1 -- 172.21.15.161:0/2858110361 learned_addr learned my addr 172.21.15.161:0/2858110361 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f177259c700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17741005b0 0x7f1774195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.449+0000 7f1772d9d700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f17741057f0 0x7f177419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176bfff700 1 -- 172.21.15.161:0/2858110361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f17741057f0 msgr2=0x7f177419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176bfff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f17741057f0 0x7f177419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176bfff700 1 -- 172.21.15.161:0/2858110361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17741005b0 msgr2=0x7f1774195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176bfff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17741005b0 0x7f1774195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176bfff700 1 -- 172.21.15.161:0/2858110361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0095f0 con 0x7f1774102ed0 2023-07-14T11:22:04.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176bfff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f1774195d40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b680 tx=0x7f175c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:04.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176b7fe700 1 -- 172.21.15.161:0/2858110361 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c01d070 con 0x7f1774102ed0 2023-07-14T11:22:04.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176b7fe700 1 -- 172.21.15.161:0/2858110361 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c022470 con 0x7f1774102ed0 2023-07-14T11:22:04.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177419d8d0 con 0x7f1774102ed0 2023-07-14T11:22:04.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f176b7fe700 1 -- 172.21.15.161:0/2858110361 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c017650 con 0x7f1774102ed0 2023-07-14T11:22:04.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.450+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177419dcf0 con 0x7f1774102ed0 2023-07-14T11:22:04.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.451+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17740fbb40 con 0x7f1774102ed0 2023-07-14T11:22:04.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.453+0000 7f176b7fe700 1 -- 172.21.15.161:0/2858110361 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f175c0225e0 con 0x7f1774102ed0 2023-07-14T11:22:04.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.454+0000 7f176b7fe700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17600a11a0 0x7f17600a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.455+0000 7f176b7fe700 1 -- 172.21.15.161:0/2858110361 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f175c0ce170 con 0x7f1774102ed0 2023-07-14T11:22:04.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.455+0000 7f177259c700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17600a11a0 0x7f17600a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.455+0000 7f177259c700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17600a11a0 0x7f17600a3650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1764009c60 tx=0x7f1764009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.458+0000 7f176b7fe700 1 -- 172.21.15.161:0/2858110361 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f175c0975a0 con 0x7f1774102ed0 2023-07-14T11:22:04.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:04 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3104257492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:04 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3761985839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:04 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/704944390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.697+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f177402cce0 con 0x7f1774102ed0 2023-07-14T11:22:04.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.698+0000 7f176b7fe700 1 -- 172.21.15.161:0/2858110361 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f175c011110 con 0x7f1774102ed0 2023-07-14T11:22:04.701 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:04.701 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:22:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17600a11a0 msgr2=0x7f17600a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17600a11a0 0x7f17600a3650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1764009c60 tx=0x7f1764009260 comp rx=0 tx=0).stop 2023-07-14T11:22:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 msgr2=0x7f1774195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f1774195d40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b680 tx=0x7f175c005740 comp rx=0 tx=0).stop 2023-07-14T11:22:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 shutdown_connections 2023-07-14T11:22:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f17600a11a0 0x7f17600a3650 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f17741057f0 0x7f177419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17741005b0 0x7f1774195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 --2- 172.21.15.161:0/2858110361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1774102ed0 0x7f1774195d40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 >> 172.21.15.161:0/2858110361 conn(0x7f17740f9fd0 msgr2=0x7f1774101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 shutdown_connections 2023-07-14T11:22:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.700+0000 7f1778bff700 1 -- 172.21.15.161:0/2858110361 wait complete. 2023-07-14T11:22:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:22:04.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:04 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3104257492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:04 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3761985839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:04 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/704944390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.719 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:04 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3104257492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.720 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:04 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3761985839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.720 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:04 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/704944390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:22:04.721 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:22:04.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 -- 172.21.15.161:0/3201511123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 msgr2=0x7f3da00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 --2- 172.21.15.161:0/3201511123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 0x7f3da00729a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009990 tx=0x7f3d90009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 -- 172.21.15.161:0/3201511123 shutdown_connections 2023-07-14T11:22:04.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 --2- 172.21.15.161:0/3201511123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 0x7f3da00729a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 --2- 172.21.15.161:0/3201511123 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3da010c0d0 0x7f3da010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 --2- 172.21.15.161:0/3201511123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 0x7f3da00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 -- 172.21.15.161:0/3201511123 >> 172.21.15.161:0/3201511123 conn(0x7f3da00fb0e0 msgr2=0x7f3da00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:04.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 -- 172.21.15.161:0/3201511123 shutdown_connections 2023-07-14T11:22:04.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.920+0000 7f3da57ff700 1 -- 172.21.15.161:0/3201511123 wait complete. 2023-07-14T11:22:04.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 Processor -- start 2023-07-14T11:22:04.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 -- start start 2023-07-14T11:22:04.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 0x7f3da019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 0x7f3da019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3da010c0d0 0x7f3da01a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01a3a40 con 0x7f3da0074550 2023-07-14T11:22:04.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01a3b80 con 0x7f3da0072f70 2023-07-14T11:22:04.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.921+0000 7f3da57ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01a3d10 con 0x7f3da010c0d0 2023-07-14T11:22:04.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 0x7f3da019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 0x7f3da019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 0x7f3da019bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54614/0 (socket says 172.21.15.161:54614) 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 -- 172.21.15.161:0/2778283593 learned_addr learned my addr 172.21.15.161:0/2778283593 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9f7fe700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3da010c0d0 0x7f3da01a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 -- 172.21.15.161:0/2778283593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3da010c0d0 msgr2=0x7f3da01a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3da010c0d0 0x7f3da01a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 -- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 msgr2=0x7f3da019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 0x7f3da019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:04.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 -- 172.21.15.161:0/2778283593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900095f0 con 0x7f3da0072f70 2023-07-14T11:22:04.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9e7fc700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 0x7f3da019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:04.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.922+0000 7f3d9effd700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 0x7f3da019bec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d90004710 tx=0x7f3d90009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:04.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.923+0000 7f3d87fff700 1 -- 172.21.15.161:0/2778283593 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9001d070 con 0x7f3da0072f70 2023-07-14T11:22:04.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.923+0000 7f3d87fff700 1 -- 172.21.15.161:0/2778283593 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d90021410 con 0x7f3da0072f70 2023-07-14T11:22:04.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.923+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da01a3f90 con 0x7f3da0072f70 2023-07-14T11:22:04.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.923+0000 7f3d87fff700 1 -- 172.21.15.161:0/2778283593 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90017650 con 0x7f3da0072f70 2023-07-14T11:22:04.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.923+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da01a4300 con 0x7f3da0072f70 2023-07-14T11:22:04.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.924+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da0101c20 con 0x7f3da0072f70 2023-07-14T11:22:04.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.926+0000 7f3d87fff700 1 -- 172.21.15.161:0/2778283593 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3d90021580 con 0x7f3da0072f70 2023-07-14T11:22:04.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.927+0000 7f3d87fff700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3d880a11a0 0x7f3d880a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:04.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.928+0000 7f3d87fff700 1 -- 172.21.15.161:0/2778283593 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3d900ce700 con 0x7f3da0072f70 2023-07-14T11:22:04.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.928+0000 7f3d9e7fc700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3d880a11a0 0x7f3d880a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:04.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.928+0000 7f3d9e7fc700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3d880a11a0 0x7f3d880a3650 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3da0074270 tx=0x7f3d94009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:04.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:04.930+0000 7f3d87fff700 1 -- 172.21.15.161:0/2778283593 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3d90097b30 con 0x7f3da0072f70 2023-07-14T11:22:05.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.134+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3da0111930 con 0x7f3da0072f70 2023-07-14T11:22:05.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.137+0000 7f3d87fff700 1 -- 172.21.15.161:0/2778283593 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24613 (secure 0 0 0) 0x7f3d9009b150 con 0x7f3da0072f70 2023-07-14T11:22:05.140 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:05.140 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":92,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:21:16.743617+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:22:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3d880a11a0 msgr2=0x7f3d880a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3d880a11a0 0x7f3d880a3650 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3da0074270 tx=0x7f3d94009260 comp rx=0 tx=0).stop 2023-07-14T11:22:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 msgr2=0x7f3da019bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 0x7f3da019bec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d90004710 tx=0x7f3d90009580 comp rx=0 tx=0).stop 2023-07-14T11:22:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 shutdown_connections 2023-07-14T11:22:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f3d880a11a0 0x7f3d880a3650 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3da010c0d0 0x7f3da01a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0072f70 0x7f3da019bec0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 --2- 172.21.15.161:0/2778283593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da0074550 0x7f3da019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 >> 172.21.15.161:0/2778283593 conn(0x7f3da00fb0e0 msgr2=0x7f3da00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 shutdown_connections 2023-07-14T11:22:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.139+0000 7f3da57ff700 1 -- 172.21.15.161:0/2778283593 wait complete. 2023-07-14T11:22:05.163 DEBUG:tasks.fwd_scrub:Yielding 2023-07-14T11:22:05.163 INFO:teuthology.run_tasks:Running task exec... 2023-07-14T11:22:05.173 INFO:teuthology.task.exec:Executing custom commands... 2023-07-14T11:22:05.173 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T11:22:05.174 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-07-14T11:22:05.174 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-14T11:22:05.174 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:22:05.175 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:22:05.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/2058164670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 msgr2=0x7f2c040729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/2058164670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 0x7f2c040729a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c000098f0 tx=0x7f2c00009c00 comp rx=0 tx=0).stop 2023-07-14T11:22:05.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/2058164670 shutdown_connections 2023-07-14T11:22:05.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/2058164670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c0410bf10 0x7f2c0410e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/2058164670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 0x7f2c040729a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/2058164670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 0x7f2c040733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/2058164670 >> 172.21.15.161:0/2058164670 conn(0x7f2c040faf80 msgr2=0x7f2c040fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/2058164670 shutdown_connections 2023-07-14T11:22:05.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.373+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/2058164670 wait complete. 2023-07-14T11:22:05.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 Processor -- start 2023-07-14T11:22:05.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 -- start start 2023-07-14T11:22:05.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 0x7f2c0419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 0x7f2c0419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c0410bf10 0x7f2c041a3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041a3820 con 0x7f2c04072f70 2023-07-14T11:22:05.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041a3990 con 0x7f2c0410bf10 2023-07-14T11:22:05.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.375+0000 7f2c0ad9d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041a3b50 con 0x7f2c04074550 2023-07-14T11:22:05.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 0x7f2c0419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 0x7f2c0419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39844/0 (socket says 172.21.15.161:39844) 2023-07-14T11:22:05.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 -- 172.21.15.161:0/203095251 learned_addr learned my addr 172.21.15.161:0/203095251 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c0959a700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 0x7f2c0419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c0a59c700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c0410bf10 0x7f2c041a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 -- 172.21.15.161:0/203095251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 msgr2=0x7f2c0419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 0x7f2c0419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 -- 172.21.15.161:0/203095251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c0410bf10 msgr2=0x7f2c041a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c0410bf10 0x7f2c041a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 -- 172.21.15.161:0/203095251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf40095f0 con 0x7f2c04072f70 2023-07-14T11:22:05.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c0a59c700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c0410bf10 0x7f2c041a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:05.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.376+0000 7f2c09d9b700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 0x7f2c0419bcb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009fd0 tx=0x7f2c0000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.377+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0001c070 con 0x7f2c04072f70 2023-07-14T11:22:05.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.377+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0000f7d0 con 0x7f2c04072f70 2023-07-14T11:22:05.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.377+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c00009520 con 0x7f2c04072f70 2023-07-14T11:22:05.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.377+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00003b00 con 0x7f2c04072f70 2023-07-14T11:22:05.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.377+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c041a40c0 con 0x7f2c04072f70 2023-07-14T11:22:05.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.378+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c04101a60 con 0x7f2c04072f70 2023-07-14T11:22:05.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.380+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2c00020030 con 0x7f2c04072f70 2023-07-14T11:22:05.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.382+0000 7f2bfaffd700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bf00a11d0 0x7f2bf00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.382+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2c000cdc30 con 0x7f2c04072f70 2023-07-14T11:22:05.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.382+0000 7f2c0959a700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bf00a11d0 0x7f2bf00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.383+0000 7f2c0959a700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bf00a11d0 0x7f2bf00a3680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2c0419d1b0 tx=0x7f2bf4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.386+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c000970c0 con 0x7f2c04072f70 2023-07-14T11:22:05.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 -- 172.21.15.161:0/2864325449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c0ffb00 msgr2=0x7f1f3c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 --2- 172.21.15.161:0/2864325449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c0ffb00 0x7f1f3c0fff10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009990 tx=0x7f1f30009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 -- 172.21.15.161:0/2864325449 shutdown_connections 2023-07-14T11:22:05.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 --2- 172.21.15.161:0/2864325449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c0ffb00 0x7f1f3c0fff10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 --2- 172.21.15.161:0/2864325449 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f3c101680 0x7f1f3c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 --2- 172.21.15.161:0/2864325449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c100cf0 0x7f1f3c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 -- 172.21.15.161:0/2864325449 >> 172.21.15.161:0/2864325449 conn(0x7f1f3c0fb120 msgr2=0x7f1f3c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 -- 172.21.15.161:0/2864325449 shutdown_connections 2023-07-14T11:22:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.387+0000 7f1f41fff700 1 -- 172.21.15.161:0/2864325449 wait complete. 2023-07-14T11:22:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.389+0000 7f1f41fff700 1 Processor -- start 2023-07-14T11:22:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.389+0000 7f1f41fff700 1 -- start start 2023-07-14T11:22:05.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f41fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0ffb00 0x7f1f3c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f41fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f3c100cf0 0x7f1f3c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f41fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c101680 0x7f1f3c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f41fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0735e0 con 0x7f1f3c101680 2023-07-14T11:22:05.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f41fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c073750 con 0x7f1f3c0ffb00 2023-07-14T11:22:05.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f41fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c19f870 con 0x7f1f3c100cf0 2023-07-14T11:22:05.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c101680 0x7f1f3c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c101680 0x7f1f3c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39852/0 (socket says 172.21.15.161:39852) 2023-07-14T11:22:05.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0ffb00 0x7f1f3c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 -- 172.21.15.161:0/921267014 learned_addr learned my addr 172.21.15.161:0/921267014 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3affd700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f3c100cf0 0x7f1f3c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 -- 172.21.15.161:0/921267014 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f3c100cf0 msgr2=0x7f1f3c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f3c100cf0 0x7f1f3c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 -- 172.21.15.161:0/921267014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0ffb00 msgr2=0x7f1f3c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0ffb00 0x7f1f3c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3bfff700 1 -- 172.21.15.161:0/921267014 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300095f0 con 0x7f1f3c101680 2023-07-14T11:22:05.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.390+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0ffb00 0x7f1f3c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:05.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.391+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c101680 0x7f1f3c072e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400ef30 tx=0x7f1f3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.391+0000 7f1f38ff9700 1 -- 172.21.15.161:0/921267014 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34018070 con 0x7f1f3c101680 2023-07-14T11:22:05.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.391+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c19faf0 con 0x7f1f3c101680 2023-07-14T11:22:05.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.391+0000 7f1f38ff9700 1 -- 172.21.15.161:0/921267014 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3400f040 con 0x7f1f3c101680 2023-07-14T11:22:05.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.391+0000 7f1f38ff9700 1 -- 172.21.15.161:0/921267014 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400cd00 con 0x7f1f3c101680 2023-07-14T11:22:05.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.391+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c1a0050 con 0x7f1f3c101680 2023-07-14T11:22:05.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.396+0000 7f1f38ff9700 1 -- 172.21.15.161:0/921267014 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1f34003a00 con 0x7f1f3c101680 2023-07-14T11:22:05.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 Processor -- start 2023-07-14T11:22:05.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 -- start start 2023-07-14T11:22:05.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5dab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5dac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5cb0d30 con 0x5645e5dabea0 2023-07-14T11:22:05.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5d03ac0 con 0x5645e5dab550 2023-07-14T11:22:05.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f55928b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5cfcf50 con 0x5645e5daccf0 2023-07-14T11:22:05.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.397+0000 7f1f3b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5dac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58068/0 (socket says 172.21.15.161:58068) 2023-07-14T11:22:05.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3affd700 1 -- 172.21.15.161:0/2265620768 learned_addr learned my addr 172.21.15.161:0/2265620768 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3affd700 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5dab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3b7fe700 1 -- 172.21.15.161:0/2265620768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 msgr2=0x5645e5dad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3b7fe700 1 -- 172.21.15.161:0/2265620768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 msgr2=0x5645e5dab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5dab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3b7fe700 1 -- 172.21.15.161:0/2265620768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645e5b2ae60 con 0x5645e5dabea0 2023-07-14T11:22:05.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3affd700 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:05.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f38ff9700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f240a11d0 0x7f1f240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.398+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5dac760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000b620 tx=0x7f1f30005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da25118e06b62d4c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.399+0000 7f1f38ff9700 1 -- 172.21.15.161:0/921267014 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1f340cc720 con 0x7f1f3c101680 2023-07-14T11:22:05.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.399+0000 7f1f3affd700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f240a11d0 0x7f1f240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.399+0000 7f1f21ffb700 1 -- 172.21.15.161:0/2265620768 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3001d070 con 0x5645e5dabea0 2023-07-14T11:22:05.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.399+0000 7f1f21ffb700 1 -- 172.21.15.161:0/2265620768 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3001e040 con 0x5645e5dabea0 2023-07-14T11:22:05.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.399+0000 7f1f21ffb700 1 -- 172.21.15.161:0/2265620768 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30004c50 con 0x5645e5dabea0 2023-07-14T11:22:05.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 -- 172.21.15.161:0/2265620768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 msgr2=0x5645e5dac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f3affd700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f240a11d0 0x7f1f240a3680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0043e0 tx=0x7f1f2c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5dac760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000b620 tx=0x7f1f30005ee0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 -- 172.21.15.161:0/2265620768 shutdown_connections 2023-07-14T11:22:05.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5dab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2265620768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5dac760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 -- 172.21.15.161:0/2265620768 >> 172.21.15.161:0/2265620768 conn(0x5645e5da0e20 msgr2=0x5645e5da6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 -- 172.21.15.161:0/2265620768 shutdown_connections 2023-07-14T11:22:05.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 -- 172.21.15.161:0/2265620768 wait complete. 2023-07-14T11:22:05.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.400+0000 7f1f55928b80 1 -- start start 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 1 Processor -- start 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5db5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5db56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dbc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5c9cb40 con 0x5645e5dabea0 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5c9c840 con 0x5645e5dab550 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5b809a0 con 0x5645e5daccf0 2023-07-14T11:22:05.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f55928b80 10 client.0 _ll_register_callbacks cb 0x5645e5c9a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:05.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f3affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5db56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f3affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5db56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39876/0 (socket says 172.21.15.161:39876) 2023-07-14T11:22:05.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f3affd700 1 -- 172.21.15.161:0/139028273 learned_addr learned my addr 172.21.15.161:0/139028273 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.401+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dbc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5db5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f3bfff700 1 -- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 msgr2=0x5645e5db5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5db5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f3bfff700 1 -- 172.21.15.161:0/139028273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 msgr2=0x5645e5db56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5db56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f3bfff700 1 -- 172.21.15.161:0/139028273 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c019040 con 0x5645e5daccf0 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dbc6e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1f340c9e70 tx=0x7f1f34099170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe576f09431bcc46 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f1affd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f1affd700 1 -- 172.21.15.161:0/139028273 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400ce60 con 0x5645e5daccf0 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645e5c56570 con 0x5645e5daccf0 2023-07-14T11:22:05.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.402+0000 7f1f1affd700 1 -- 172.21.15.161:0/139028273 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1f34099810 con 0x5645e5daccf0 2023-07-14T11:22:05.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f1affd700 1 -- 172.21.15.161:0/139028273 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3409d840 con 0x5645e5daccf0 2023-07-14T11:22:05.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 10 client.25549 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:05.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f1f3c108870 con 0x5645e5daccf0 2023-07-14T11:22:05.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f1affd700 1 -- 172.21.15.161:0/139028273 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645e5c56570 con 0x5645e5daccf0 2023-07-14T11:22:05.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f1affd700 1 -- 172.21.15.161:0/139028273 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f340a1740 con 0x5645e5daccf0 2023-07-14T11:22:05.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 10 client.25549 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:05.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:05.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 20 client.25549 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:05.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5dc0ec0 0x5645e5dc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 4 client.25549 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:05.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.403+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645e5cf5f30 con 0x5645e5dc0ec0 2023-07-14T11:22:05.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.404+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5dc0ec0 0x5645e5dc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.404+0000 7f1f1affd700 10 client.25549 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.404+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5dc0ec0 0x5645e5dc32d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab22b34439c65a63 server_cookie=ad995b07a827a7 in_seq=0 out_seq=0 2023-07-14T11:22:05.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f1affd700 1 -- 172.21.15.161:0/139028273 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5645e5cf5f30 con 0x5645e5dc0ec0 2023-07-14T11:22:05.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f1affd700 10 client.25549 handle_command_reply: tid=0 2023-07-14T11:22:05.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 client.25549 shutdown 2023-07-14T11:22:05.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 msgr2=0x5645e5dbc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dbc6e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1f340c9e70 tx=0x7f1f34099170 comp rx=0 tx=0).stop 2023-07-14T11:22:05.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 shutdown_connections 2023-07-14T11:22:05.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5dc0ec0 0x5645e5dc32d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5daccf0 0x5645e5dbc6e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dab550 0x5645e5db5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f1affd700 0 client.25549 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 --2- 172.21.15.161:0/139028273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5dabea0 0x5645e5db56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 >> 172.21.15.161:0/139028273 conn(0x5645e5da0e20 msgr2=0x5645e5da5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 shutdown_connections 2023-07-14T11:22:05.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.405+0000 7f1f55928b80 1 -- 172.21.15.161:0/139028273 wait complete. 2023-07-14T11:22:05.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.406+0000 7f1f55928b80 20 client.25549 trim_cache size 0 max 16384 2023-07-14T11:22:05.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 Processor -- start 2023-07-14T11:22:05.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 -- start start 2023-07-14T11:22:05.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5dbc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5db5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5da2f40 con 0x5645e5daccf0 2023-07-14T11:22:05.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5db4610 con 0x5645e5dabea0 2023-07-14T11:22:05.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.423+0000 7f1f55928b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5dac2a0 con 0x5645e5dab550 2023-07-14T11:22:05.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5db5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5dbc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58076/0 (socket says 172.21.15.161:58076) 2023-07-14T11:22:05.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3b7fe700 1 -- 172.21.15.161:0/3179890948 learned_addr learned my addr 172.21.15.161:0/3179890948 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3affd700 1 -- 172.21.15.161:0/3179890948 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 msgr2=0x5645e5db30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3affd700 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3affd700 1 -- 172.21.15.161:0/3179890948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 msgr2=0x5645e5db5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3affd700 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5db5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3affd700 1 -- 172.21.15.161:0/3179890948 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645e5da2660 con 0x5645e5dabea0 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5db5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.424+0000 7f1f3affd700 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5dbc6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c003680 tx=0x7f1f2c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69430ea3bde2a0a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f227fc700 1 -- 172.21.15.161:0/3179890948 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c027050 con 0x5645e5dabea0 2023-07-14T11:22:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f227fc700 1 -- 172.21.15.161:0/3179890948 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f2c028040 con 0x5645e5dabea0 2023-07-14T11:22:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f227fc700 1 -- 172.21.15.161:0/3179890948 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c016070 con 0x5645e5dabea0 2023-07-14T11:22:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 -- 172.21.15.161:0/3179890948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 msgr2=0x5645e5dbc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5dbc6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c003680 tx=0x7f1f2c016000 comp rx=0 tx=0).stop 2023-07-14T11:22:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 -- 172.21.15.161:0/3179890948 shutdown_connections 2023-07-14T11:22:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5db5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 --2- 172.21.15.161:0/3179890948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5dbc6a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 -- 172.21.15.161:0/3179890948 >> 172.21.15.161:0/3179890948 conn(0x5645e5da0e20 msgr2=0x5645e5dbcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 -- 172.21.15.161:0/3179890948 shutdown_connections 2023-07-14T11:22:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.425+0000 7f1f55928b80 1 -- 172.21.15.161:0/3179890948 wait complete. 2023-07-14T11:22:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.426+0000 7f1f55928b80 1 -- start start 2023-07-14T11:22:05.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.426+0000 7f1f55928b80 1 Processor -- start 2023-07-14T11:22:05.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.426+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.426+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5db98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.426+0000 7f1f55928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5e14d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.426+0000 7f1f55928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5db9df0 con 0x5645e5daccf0 2023-07-14T11:22:05.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.426+0000 7f1f55928b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5db9f30 con 0x5645e5dabea0 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f55928b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645e5e152b0 con 0x5645e5dab550 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f55928b80 10 client.0 _ll_register_callbacks cb 0x7f1f34095c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5e14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5db98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5e14d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39884/0 (socket says 172.21.15.161:39884) 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5db98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54688/0 (socket says 172.21.15.161:54688) 2023-07-14T11:22:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 -- 172.21.15.161:0/2391540988 learned_addr learned my addr 172.21.15.161:0/2391540988 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 -- 172.21.15.161:0/2391540988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 msgr2=0x5645e5db9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 -- 172.21.15.161:0/2391540988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 msgr2=0x5645e5db98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5db98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 -- 172.21.15.161:0/2391540988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c108870 con 0x5645e5daccf0 2023-07-14T11:22:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f3bfff700 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5e14d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1f340997a0 tx=0x7f1f34009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9af37a1d621cd14 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.427+0000 7f1f1affd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f1affd700 1 -- 172.21.15.161:0/2391540988 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340c9030 con 0x5645e5daccf0 2023-07-14T11:22:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f1affd700 1 -- 172.21.15.161:0/2391540988 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1f3400ce60 con 0x5645e5daccf0 2023-07-14T11:22:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f3001e040 con 0x5645e5daccf0 2023-07-14T11:22:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f1affd700 1 -- 172.21.15.161:0/2391540988 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f34098fb0 con 0x5645e5daccf0 2023-07-14T11:22:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 10 client.15513 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f1f2c011d60 con 0x5645e5daccf0 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f1affd700 1 -- 172.21.15.161:0/2391540988 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3001e040 con 0x5645e5daccf0 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f1affd700 1 -- 172.21.15.161:0/2391540988 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f3401baf0 con 0x5645e5daccf0 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 20 client.15513 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5e169a0 0x5645e5e18db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 4 client.15513 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.428+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5645e5cb80d0 con 0x5645e5e169a0 2023-07-14T11:22:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.429+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5e169a0 0x5645e5e18db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.429+0000 7f1f3b7fe700 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5e169a0 0x5645e5e18db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a97975c8246d5a3 server_cookie=a71c24aab4e89163 in_seq=0 out_seq=0 2023-07-14T11:22:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.429+0000 7f1f1affd700 10 client.15513 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.430+0000 7f1f1affd700 1 -- 172.21.15.161:0/2391540988 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5645e5cb80d0 con 0x5645e5e169a0 2023-07-14T11:22:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.430+0000 7f1f1affd700 10 client.15513 handle_command_reply: tid=0 2023-07-14T11:22:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.430+0000 7f1f55928b80 1 client.15513 shutdown 2023-07-14T11:22:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 msgr2=0x5645e5e14d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5e14d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1f340997a0 tx=0x7f1f34009ed0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 shutdown_connections 2023-07-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645e5e169a0 0x5645e5e18db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645e5daccf0 0x5645e5e14d70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5645e5dab550 0x5645e5db9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f1affd700 0 client.15513 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 --2- 172.21.15.161:0/2391540988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645e5dabea0 0x5645e5db98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 >> 172.21.15.161:0/2391540988 conn(0x5645e5da0e20 msgr2=0x5645e5db30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 shutdown_connections 2023-07-14T11:22:05.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 1 -- 172.21.15.161:0/2391540988 wait complete. 2023-07-14T11:22:05.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.431+0000 7f1f55928b80 20 client.15513 trim_cache size 0 max 16384 2023-07-14T11:22:05.441 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:22:05.441 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:22:05.442 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "87db24b0-2ae8-4419-b5e6-5d9a88d9d080", 2023-07-14T11:22:05.442 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:22:05.442 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f240a11d0 msgr2=0x7f1f240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f240a11d0 0x7f1f240a3680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0043e0 tx=0x7f1f2c009260 comp rx=0 tx=0).stop 2023-07-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c101680 msgr2=0x7f1f3c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c101680 0x7f1f3c072e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400ef30 tx=0x7f1f3400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 shutdown_connections 2023-07-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3c101680 0x7f1f3c072e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f1f240a11d0 0x7f1f240a3680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0ffb00 0x7f1f3c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 --2- 172.21.15.161:0/921267014 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f3c100cf0 0x7f1f3c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 >> 172.21.15.161:0/921267014 conn(0x7f1f3c0fb120 msgr2=0x7f1f3c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 shutdown_connections 2023-07-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.432+0000 7f1f41fff700 1 -- 172.21.15.161:0/921267014 wait complete. 2023-07-14T11:22:05.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:05 smithi161 ceph-mon[95526]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2023-07-14T11:22:05.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:05 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2858110361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:05.450 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:05 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2778283593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:22:05.452 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 -- 172.21.15.161:0/749321255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 msgr2=0x7f277c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 --2- 172.21.15.161:0/749321255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c0fff00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098f0 tx=0x7f276c009c00 comp rx=0 tx=0).stop 2023-07-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 -- 172.21.15.161:0/749321255 shutdown_connections 2023-07-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 --2- 172.21.15.161:0/749321255 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f277c101670 0x7f277c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 --2- 172.21.15.161:0/749321255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c0fff00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 --2- 172.21.15.161:0/749321255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f277c100ce0 0x7f277c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 -- 172.21.15.161:0/749321255 >> 172.21.15.161:0/749321255 conn(0x7f277c0fb130 msgr2=0x7f277c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 -- 172.21.15.161:0/749321255 shutdown_connections 2023-07-14T11:22:05.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.654+0000 7f27817ff700 1 -- 172.21.15.161:0/749321255 wait complete. 2023-07-14T11:22:05.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.655+0000 7f27817ff700 1 Processor -- start 2023-07-14T11:22:05.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.655+0000 7f27817ff700 1 -- start start 2023-07-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f27817ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f27817ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f277c100ce0 0x7f277c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f27817ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f277c101670 0x7f277c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f27817ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c19f680 con 0x7f277c101670 2023-07-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f27817ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c19f7c0 con 0x7f277c0ffaf0 2023-07-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f27817ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c19f950 con 0x7f277c100ce0 2023-07-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54706/0 (socket says 172.21.15.161:54706) 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 -- 172.21.15.161:0/3342035028 learned_addr learned my addr 172.21.15.161:0/3342035028 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f277c100ce0 0x7f277c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277b7fe700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f277c101670 0x7f277c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 -- 172.21.15.161:0/3342035028 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f277c100ce0 msgr2=0x7f277c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f277c100ce0 0x7f277c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 -- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f277c101670 msgr2=0x7f277c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f277c101670 0x7f277c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.656+0000 7f277affd700 1 -- 172.21.15.161:0/3342035028 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c009520 con 0x7f277c0ffaf0 2023-07-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.657+0000 7f277affd700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c197b00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f276c000c00 tx=0x7f276c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.657+0000 7f2763fff700 1 -- 172.21.15.161:0/3342035028 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c01c070 con 0x7f277c0ffaf0 2023-07-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.657+0000 7f2763fff700 1 -- 172.21.15.161:0/3342035028 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c00f7b0 con 0x7f277c0ffaf0 2023-07-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.657+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c19fbd0 con 0x7f277c0ffaf0 2023-07-14T11:22:05.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.657+0000 7f2763fff700 1 -- 172.21.15.161:0/3342035028 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0039b0 con 0x7f277c0ffaf0 2023-07-14T11:22:05.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.657+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c19fff0 con 0x7f277c0ffaf0 2023-07-14T11:22:05.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.660+0000 7f2763fff700 1 -- 172.21.15.161:0/3342035028 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f276c020030 con 0x7f277c0ffaf0 2023-07-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.661+0000 7f279510fb80 1 Processor -- start 2023-07-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.661+0000 7f279510fb80 1 -- start start 2023-07-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f279510fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc74d0 0x564e98fc78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f279510fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fc86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f279510fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fc90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f279510fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98ecccb0 con 0x564e98fc7e20 2023-07-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f279510fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98f1fa40 con 0x564e98fc74d0 2023-07-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f279510fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98f18ed0 con 0x564e98fc8c70 2023-07-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f277affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fc86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc74d0 0x564e98fc78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc74d0 0x564e98fc78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54716/0 (socket says 172.21.15.161:54716) 2023-07-14T11:22:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f277b7fe700 1 -- 172.21.15.161:0/3805608991 learned_addr learned my addr 172.21.15.161:0/3805608991 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fc90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.662+0000 7f2763fff700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27640a1150 0x7f27640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f277a7fc700 1 -- 172.21.15.161:0/3805608991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc74d0 msgr2=0x564e98fc78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc74d0 0x564e98fc78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f277a7fc700 1 -- 172.21.15.161:0/3805608991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 msgr2=0x564e98fc86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fc86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f277a7fc700 1 -- 172.21.15.161:0/3805608991 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e98d46de0 con 0x564e98fc8c70 2023-07-14T11:22:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f2763fff700 1 -- 172.21.15.161:0/3342035028 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f276c0cddb0 con 0x7f277c0ffaf0 2023-07-14T11:22:05.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2c040611c0 con 0x7f2c04072f70 2023-07-14T11:22:05.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f277b7fe700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27640a1150 0x7f27640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fc90e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2770009cc0 tx=0x7f2770011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca646d95651e8dcf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f27617fa700 1 -- 172.21.15.161:0/3805608991 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770011cf0 con 0x564e98fc8c70 2023-07-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f27617fa700 1 -- 172.21.15.161:0/3805608991 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2770011e50 con 0x564e98fc8c70 2023-07-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.663+0000 7f27617fa700 1 -- 172.21.15.161:0/3805608991 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770005240 con 0x564e98fc8c70 2023-07-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 -- 172.21.15.161:0/3805608991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 msgr2=0x564e98fc90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f277b7fe700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27640a1150 0x7f27640a3600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f27740075c0 tx=0x7f2774009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fc90e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2770009cc0 tx=0x7f2770011440 comp rx=0 tx=0).stop 2023-07-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 -- 172.21.15.161:0/3805608991 shutdown_connections 2023-07-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fc90e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc74d0 0x564e98fc78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 --2- 172.21.15.161:0/3805608991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fc86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 -- 172.21.15.161:0/3805608991 >> 172.21.15.161:0/3805608991 conn(0x564e98fbcda0 msgr2=0x564e98fc21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 -- 172.21.15.161:0/3805608991 shutdown_connections 2023-07-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.664+0000 7f279510fb80 1 -- 172.21.15.161:0/3805608991 wait complete. 2023-07-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 -- start start 2023-07-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 Processor -- start 2023-07-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc74d0 0x564e98fd10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fd8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98eb8ac0 con 0x564e98fc74d0 2023-07-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98eb87c0 con 0x564e98fc7e20 2023-07-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98d9c920 con 0x564e98fc8c70 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.665+0000 7f279510fb80 10 client.0 _ll_register_callbacks cb 0x564e98eb6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc74d0 0x564e98fd10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc74d0 0x564e98fd10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39922/0 (socket says 172.21.15.161:39922) 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 -- 172.21.15.161:0/3705711622 learned_addr learned my addr 172.21.15.161:0/3705711622 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277b7fe700 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fd8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 -- 172.21.15.161:0/3705711622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 msgr2=0x564e98fd8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fd8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 -- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 msgr2=0x564e98fd1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 -- 172.21.15.161:0/3705711622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2770009920 con 0x564e98fc74d0 2023-07-14T11:22:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f277affd700 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc74d0 0x564e98fd10c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fd0 tx=0x7f276c0d1510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4135d6c768f3a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.666+0000 7f275eff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f275eff5700 1 -- 172.21.15.161:0/3705711622 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0a0070 con 0x564e98fc74d0 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f275eff5700 1 -- 172.21.15.161:0/3705711622 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f276c09aea0 con 0x564e98fc74d0 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e98e724f0 con 0x564e98fc74d0 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f275eff5700 1 -- 172.21.15.161:0/3705711622 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f276c09ac70 con 0x564e98fc74d0 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 10 client.15525 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x564e98d2fc60 con 0x564e98fc74d0 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f275eff5700 1 -- 172.21.15.161:0/3705711622 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e98e724f0 con 0x564e98fc74d0 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f275eff5700 1 -- 172.21.15.161:0/3705711622 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f276c09b100 con 0x564e98fc74d0 2023-07-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 10 client.15525 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 20 client.15525 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e98fdc950 0x564e98fded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 4 client.15525 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.667+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e98f11eb0 con 0x564e98fdc950 2023-07-14T11:22:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.668+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e98fdc950 0x564e98fded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.668+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e98fdc950 0x564e98fded30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb0051ed385749 server_cookie=b682bd4e28acac98 in_seq=0 out_seq=0 2023-07-14T11:22:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.668+0000 7f275eff5700 10 client.15525 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:05 smithi175 ceph-mon[129081]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2023-07-14T11:22:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:05 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2858110361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:05 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2778283593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:22:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.669+0000 7f275eff5700 1 -- 172.21.15.161:0/3705711622 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e98f11eb0 con 0x564e98fdc950 2023-07-14T11:22:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.669+0000 7f275eff5700 10 client.15525 handle_command_reply: tid=0 2023-07-14T11:22:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.669+0000 7f279510fb80 1 client.15525 shutdown 2023-07-14T11:22:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.669+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc74d0 msgr2=0x564e98fd10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.669+0000 7f279510fb80 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc74d0 0x564e98fd10c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fd0 tx=0x7f276c0d1510 comp rx=0 tx=0).stop 2023-07-14T11:22:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.669+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 shutdown_connections 2023-07-14T11:22:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.669+0000 7f279510fb80 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e98fdc950 0x564e98fded30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f279510fb80 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc74d0 0x564e98fd10c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f279510fb80 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc8c70 0x564e98fd8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f279510fb80 1 --2- 172.21.15.161:0/3705711622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f275eff5700 0 client.15525 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 >> 172.21.15.161:0/3705711622 conn(0x564e98fbcda0 msgr2=0x564e98fc18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 shutdown_connections 2023-07-14T11:22:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f279510fb80 1 -- 172.21.15.161:0/3705711622 wait complete. 2023-07-14T11:22:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.670+0000 7f279510fb80 20 client.15525 trim_cache size 0 max 16384 2023-07-14T11:22:05.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 Processor -- start 2023-07-14T11:22:05.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 -- start start 2023-07-14T11:22:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fcfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fd8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc8c70 0x564e98fcefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98fcee50 con 0x564e98fc7e20 2023-07-14T11:22:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98fdc720 con 0x564e98fc8c70 2023-07-14T11:22:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f279510fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98fcfb80 con 0x564e98fc74d0 2023-07-14T11:22:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f277affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fcfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc8c70 0x564e98fcefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f277affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fcfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58136/0 (socket says 172.21.15.161:58136) 2023-07-14T11:22:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.687+0000 7f277affd700 1 -- 172.21.15.161:0/2234171897 learned_addr learned my addr 172.21.15.161:0/2234171897 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f277a7fc700 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fd8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f277affd700 1 -- 172.21.15.161:0/2234171897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc8c70 msgr2=0x564e98fcefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f277affd700 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc8c70 0x564e98fcefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f277affd700 1 -- 172.21.15.161:0/2234171897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 msgr2=0x564e98fd8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f277affd700 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fd8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f277affd700 1 -- 172.21.15.161:0/2234171897 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e98d6d250 con 0x564e98fc74d0 2023-07-14T11:22:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f277affd700 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fcfe50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f276c0973a0 tx=0x7f276c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4b13640ea08706 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f2761ffb700 1 -- 172.21.15.161:0/2234171897 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0a0070 con 0x564e98fc74d0 2023-07-14T11:22:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f2761ffb700 1 -- 172.21.15.161:0/2234171897 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c0b9250 con 0x564e98fc74d0 2023-07-14T11:22:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.688+0000 7f2761ffb700 1 -- 172.21.15.161:0/2234171897 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c021c50 con 0x564e98fc74d0 2023-07-14T11:22:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 -- 172.21.15.161:0/2234171897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 msgr2=0x564e98fcfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fcfe50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f276c0973a0 tx=0x7f276c009c00 comp rx=0 tx=0).stop 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 -- 172.21.15.161:0/2234171897 shutdown_connections 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc8c70 0x564e98fcefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fcfe50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 --2- 172.21.15.161:0/2234171897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc7e20 0x564e98fd8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 -- 172.21.15.161:0/2234171897 >> 172.21.15.161:0/2234171897 conn(0x564e98fbcda0 msgr2=0x564e98fe3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 -- 172.21.15.161:0/2234171897 shutdown_connections 2023-07-14T11:22:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 -- 172.21.15.161:0/2234171897 wait complete. 2023-07-14T11:22:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.689+0000 7f279510fb80 1 -- start start 2023-07-14T11:22:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 1 Processor -- start 2023-07-14T11:22:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fd7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc8c70 0x564e98fd49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98fd4fd0 con 0x564e98fc8c70 2023-07-14T11:22:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98fd5140 con 0x564e98fc7e20 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98fd5300 con 0x564e98fc74d0 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f279510fb80 10 client.0 _ll_register_callbacks cb 0x7f276c00b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc8c70 0x564e98fd49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f277affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fd7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc8c70 0x564e98fd49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39942/0 (socket says 172.21.15.161:39942) 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f277b7fe700 1 -- 172.21.15.161:0/3351283415 learned_addr learned my addr 172.21.15.161:0/3351283415 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.690+0000 7f277a7fc700 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f277b7fe700 1 -- 172.21.15.161:0/3351283415 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 msgr2=0x564e98fd7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f277b7fe700 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fd7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f277b7fe700 1 -- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 msgr2=0x564e98fd8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:05.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f277b7fe700 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f277b7fe700 1 -- 172.21.15.161:0/3351283415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2770009920 con 0x564e98fc8c70 2023-07-14T11:22:05.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f277b7fe700 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc8c70 0x564e98fd49b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27740246d0 tx=0x7f2774004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482da91cbabdc85f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:05.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f275eff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:05.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f275eff5700 1 -- 172.21.15.161:0/3351283415 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277400e040 con 0x564e98fc8c70 2023-07-14T11:22:05.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f275eff5700 1 -- 172.21.15.161:0/3351283415 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f277401c070 con 0x564e98fc8c70 2023-07-14T11:22:05.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2770011e50 con 0x564e98fc8c70 2023-07-14T11:22:05.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f275eff5700 1 -- 172.21.15.161:0/3351283415 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2774024e60 con 0x564e98fc8c70 2023-07-14T11:22:05.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.691+0000 7f279510fb80 10 client.15531 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:05.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x564e98d452b0 con 0x564e98fc8c70 2023-07-14T11:22:05.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f275eff5700 1 -- 172.21.15.161:0/3351283415 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2770011e50 con 0x564e98fc8c70 2023-07-14T11:22:05.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f275eff5700 1 -- 172.21.15.161:0/3351283415 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2774025670 con 0x564e98fc8c70 2023-07-14T11:22:05.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f279510fb80 10 client.15531 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:05.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f279510fb80 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:05.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f279510fb80 20 client.15531 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:05.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f279510fb80 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e99031e50 0x564e99034260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:05.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f279510fb80 4 client.15531 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:22:05.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e98ed4050 con 0x564e99031e50 2023-07-14T11:22:05.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.692+0000 7f277affd700 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e99031e50 0x564e99034260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:05.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f275eff5700 10 client.15531 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f277affd700 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e99031e50 0x564e99034260 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce84c1fd21a7ee75 server_cookie=dbe75c46751c31ef in_seq=0 out_seq=0 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f275eff5700 1 -- 172.21.15.161:0/3351283415 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564e98ed4050 con 0x564e99031e50 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f275eff5700 10 client.15531 handle_command_reply: tid=0 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 client.15531 shutdown 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc8c70 msgr2=0x564e98fd49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc8c70 0x564e98fd49b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27740246d0 tx=0x7f2774004ad0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 shutdown_connections 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564e99031e50 0x564e99034260 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e98fc8c70 0x564e98fd49b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e98fc74d0 0x564e98fd7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f275eff5700 0 client.15531 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:05.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 --2- 172.21.15.161:0/3351283415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e98fc7e20 0x564e98fd8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 >> 172.21.15.161:0/3351283415 conn(0x564e98fbcda0 msgr2=0x564e98fcefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 shutdown_connections 2023-07-14T11:22:05.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.693+0000 7f279510fb80 1 -- 172.21.15.161:0/3351283415 wait complete. 2023-07-14T11:22:05.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f279510fb80 20 client.15531 trim_cache size 0 max 16384 2023-07-14T11:22:05.705 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:22:05.705 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:22:05.705 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:22:05.705 INFO:teuthology.orchestra.run.smithi161.stdout: "87db24b0-2ae8-4419-b5e6-5d9a88d9d080": { 2023-07-14T11:22:05.705 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:22:05.705 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "87db24b0-2ae8-4419-b5e6-5d9a88d9d080", 2023-07-14T11:22:05.706 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:22:05.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:22:05.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:22:05.706 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:22:05.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27640a1150 msgr2=0x7f27640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27640a1150 0x7f27640a3600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f27740075c0 tx=0x7f2774009040 comp rx=0 tx=0).stop 2023-07-14T11:22:05.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 msgr2=0x7f277c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:05.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c197b00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f276c000c00 tx=0x7f276c00b730 comp rx=0 tx=0).stop 2023-07-14T11:22:05.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 shutdown_connections 2023-07-14T11:22:05.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f277c101670 0x7f277c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f27640a1150 0x7f27640a3600 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c0ffaf0 0x7f277c197b00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 --2- 172.21.15.161:0/3342035028 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f277c100ce0 0x7f277c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:05.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 >> 172.21.15.161:0/3342035028 conn(0x7f277c0fb130 msgr2=0x7f277c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:05.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 shutdown_connections 2023-07-14T11:22:05.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:05.694+0000 7f27817ff700 1 -- 172.21.15.161:0/3342035028 wait complete. 2023-07-14T11:22:05.714 INFO:tasks.cephfs.filesystem:scrub status for tag:87db24b0-2ae8-4419-b5e6-5d9a88d9d080 - {'path': '/', 'tag': '87db24b0-2ae8-4419-b5e6-5d9a88d9d080', 'options': 'recursive,force'} 2023-07-14T11:22:05.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:05 smithi182 ceph-mon[129366]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2023-07-14T11:22:05.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:05 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2858110361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:05.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:05 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2778283593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:22:06.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.275+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f2c00097490 con 0x7f2c04072f70 2023-07-14T11:22:06.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.276+0000 7f2bfaffd700 1 -- 172.21.15.161:0/203095251 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2c000636b0 con 0x7f2c04072f70 2023-07-14T11:22:06.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.278+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bf00a11d0 msgr2=0x7f2bf00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:06.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bf00a11d0 0x7f2bf00a3680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2c0419d1b0 tx=0x7f2bf4009310 comp rx=0 tx=0).stop 2023-07-14T11:22:06.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 msgr2=0x7f2c0419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:06.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 0x7f2c0419bcb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009fd0 tx=0x7f2c0000b790 comp rx=0 tx=0).stop 2023-07-14T11:22:06.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 shutdown_connections 2023-07-14T11:22:06.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04072f70 0x7f2c0419bcb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f2bf00a11d0 0x7f2bf00a3680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c0410bf10 0x7f2c041a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 --2- 172.21.15.161:0/203095251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c04074550 0x7f2c0419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 >> 172.21.15.161:0/203095251 conn(0x7f2c040faf80 msgr2=0x7f2c040fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:06.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 shutdown_connections 2023-07-14T11:22:06.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.279+0000 7f2c0ad9d700 1 -- 172.21.15.161:0/203095251 wait complete. 2023-07-14T11:22:06.302 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-07-14T11:22:06.528 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: ignoring --setuser ceph since I am not root 2023-07-14T11:22:06.528 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: ignoring --setgroup ceph since I am not root 2023-07-14T11:22:06.528 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-mgr[131476]: -- 172.21.15.182:0/3840434773 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f8024b1e0 con 0x561f80220800 2023-07-14T11:22:06.528 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:06.442+0000 7fa889f0e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:22:06.529 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:06.525+0000 7fa889f0e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:22:06.529 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:22:06.529 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:06.529 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:22:06.529 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:06.529 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/203095251' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-14T11:22:06.530 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-mon[129366]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:06.531 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: ignoring --setuser ceph since I am not root 2023-07-14T11:22:06.531 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: ignoring --setgroup ceph since I am not root 2023-07-14T11:22:06.531 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-mgr[130571]: -- 172.21.15.175:0/454890665 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562d7d7d91e0 con 0x562d7d7aec00 2023-07-14T11:22:06.532 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:06.441+0000 7fc0e34f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:22:06.532 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:22:06.532 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:06.532 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:22:06.532 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:06.533 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/203095251' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-14T11:22:06.533 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-mon[129081]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:06.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.542+0000 7f25d35ff700 1 -- 172.21.15.161:0/1579885600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc10c0d0 msgr2=0x7f25cc10e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:06.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.542+0000 7f25d35ff700 1 --2- 172.21.15.161:0/1579885600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc10c0d0 0x7f25cc10e4b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25c400b0a0 tx=0x7f25c400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.543+0000 7f25d35ff700 1 -- 172.21.15.161:0/1579885600 shutdown_connections 2023-07-14T11:22:06.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.543+0000 7f25d35ff700 1 --2- 172.21.15.161:0/1579885600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25cc074550 0x7f25cc0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.543+0000 7f25d35ff700 1 --2- 172.21.15.161:0/1579885600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc10c0d0 0x7f25cc10e4b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.543+0000 7f25d35ff700 1 --2- 172.21.15.161:0/1579885600 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25cc072f70 0x7f25cc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.543+0000 7f25d35ff700 1 -- 172.21.15.161:0/1579885600 >> 172.21.15.161:0/1579885600 conn(0x7f25cc0fb0e0 msgr2=0x7f25cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:06.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.543+0000 7f25d35ff700 1 -- 172.21.15.161:0/1579885600 shutdown_connections 2023-07-14T11:22:06.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.543+0000 7f25d35ff700 1 -- 172.21.15.161:0/1579885600 wait complete. 2023-07-14T11:22:06.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 Processor -- start 2023-07-14T11:22:06.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 -- start start 2023-07-14T11:22:06.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc072f70 0x7f25cc19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:06.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25cc074550 0x7f25cc19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:06.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25cc10c0d0 0x7f25cc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:06.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc19daa0 con 0x7f25cc074550 2023-07-14T11:22:06.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc19dc10 con 0x7f25cc072f70 2023-07-14T11:22:06.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.544+0000 7f25d35ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc19ddd0 con 0x7f25cc10c0d0 2023-07-14T11:22:06.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc072f70 0x7f25cc19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:06.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc072f70 0x7f25cc19c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54784/0 (socket says 172.21.15.161:54784) 2023-07-14T11:22:06.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 -- 172.21.15.161:0/2236103380 learned_addr learned my addr 172.21.15.161:0/2236103380 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:06.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d0b9a700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25cc074550 0x7f25cc19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:06.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d1b9c700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25cc10c0d0 0x7f25cc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:06.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 -- 172.21.15.161:0/2236103380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25cc10c0d0 msgr2=0x7f25cc19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:06.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25cc10c0d0 0x7f25cc19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 -- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25cc074550 msgr2=0x7f25cc19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:06.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25cc074550 0x7f25cc19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 -- 172.21.15.161:0/2236103380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c4009d00 con 0x7f25cc072f70 2023-07-14T11:22:06.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d0b9a700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25cc074550 0x7f25cc19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:06.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.545+0000 7f25d139b700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc072f70 0x7f25cc19c080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25c800e970 tx=0x7f25c800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:06.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.546+0000 7f25c27fc700 1 -- 172.21.15.161:0/2236103380 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8009790 con 0x7f25cc072f70 2023-07-14T11:22:06.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.546+0000 7f25c27fc700 1 -- 172.21.15.161:0/2236103380 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25c800eec0 con 0x7f25cc072f70 2023-07-14T11:22:06.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.546+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc19e0b0 con 0x7f25cc072f70 2023-07-14T11:22:06.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.546+0000 7f25c27fc700 1 -- 172.21.15.161:0/2236103380 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8005590 con 0x7f25cc072f70 2023-07-14T11:22:06.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.546+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc1a4870 con 0x7f25cc072f70 2023-07-14T11:22:06.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.547+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc101c20 con 0x7f25cc072f70 2023-07-14T11:22:06.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.550+0000 7f25c27fc700 1 -- 172.21.15.161:0/2236103380 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f25c800cbd0 con 0x7f25cc072f70 2023-07-14T11:22:06.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.551+0000 7f25c27fc700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 0x7f25b80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:06.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.552+0000 7f25c27fc700 1 -- 172.21.15.161:0/2236103380 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f25c8014070 con 0x7f25cc072f70 2023-07-14T11:22:06.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.552+0000 7f25d0b9a700 1 -- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 msgr2=0x7f25b80a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:06.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.552+0000 7f25d0b9a700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 0x7f25b80a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:22:06.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.555+0000 7f25c27fc700 1 -- 172.21.15.161:0/2236103380 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f25c8096300 con 0x7f25cc072f70 2023-07-14T11:22:06.594 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:22:06.594 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:06.595 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:22:06.595 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:06.595 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/203095251' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-14T11:22:06.595 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-mon[95526]: from='mgr.14148 172.21.15.161:0/1697250790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:06.595 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: ignoring --setuser ceph since I am not root 2023-07-14T11:22:06.595 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: ignoring --setgroup ceph since I am not root 2023-07-14T11:22:06.596 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:06.591+0000 7f338a615200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:22:06.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.753+0000 7f25d0b9a700 1 -- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 msgr2=0x7f25b80a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:06.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.753+0000 7f25d0b9a700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 0x7f25b80a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:22:06.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.804+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f25cc04e2f0 con 0x7f25cc072f70 2023-07-14T11:22:06.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.809+0000 7f25c27fc700 1 -- 172.21.15.161:0/2236103380 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f25c8004ef0 con 0x7f25cc072f70 2023-07-14T11:22:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 msgr2=0x7f25b80a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:22:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 0x7f25b80a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc072f70 msgr2=0x7f25cc19c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc072f70 0x7f25cc19c080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25c800e970 tx=0x7f25c800ec80 comp rx=0 tx=0).stop 2023-07-14T11:22:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 shutdown_connections 2023-07-14T11:22:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f25b80a11a0 0x7f25b80a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25cc10c0d0 0x7f25cc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25cc072f70 0x7f25cc19c080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 --2- 172.21.15.161:0/2236103380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25cc074550 0x7f25cc19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 >> 172.21.15.161:0/2236103380 conn(0x7f25cc0fb0e0 msgr2=0x7f25cc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 shutdown_connections 2023-07-14T11:22:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:06.812+0000 7f25d35ff700 1 -- 172.21.15.161:0/2236103380 wait complete. 2023-07-14T11:22:06.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-07-14T11:22:06.898 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:06.622+0000 7fa889f0e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:22:06.898 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:06.716+0000 7fa889f0e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:22:06.898 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:06.895+0000 7fa889f0e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:22:06.905 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:06.702+0000 7f338a615200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:22:06.905 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:06.826+0000 7f338a615200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:22:06.906 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:06.528+0000 7fc0e34f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:22:06.906 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:06.625+0000 7fc0e34f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:22:06.906 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:06.721+0000 7fc0e34f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:22:06.906 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:06.902+0000 7fc0e34f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:22:07.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 -- 172.21.15.161:0/2225713195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac100cf0 msgr2=0x7f77ac101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 --2- 172.21.15.161:0/2225713195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac100cf0 0x7f77ac101140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009990 tx=0x7f77a0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 -- 172.21.15.161:0/2225713195 shutdown_connections 2023-07-14T11:22:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 --2- 172.21.15.161:0/2225713195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77ac0ffb00 0x7f77ac0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 --2- 172.21.15.161:0/2225713195 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f77ac101680 0x7f77ac107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 --2- 172.21.15.161:0/2225713195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac100cf0 0x7f77ac101140 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 -- 172.21.15.161:0/2225713195 >> 172.21.15.161:0/2225713195 conn(0x7f77ac0fb0e0 msgr2=0x7f77ac0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 -- 172.21.15.161:0/2225713195 shutdown_connections 2023-07-14T11:22:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.054+0000 7f77b1dff700 1 -- 172.21.15.161:0/2225713195 wait complete. 2023-07-14T11:22:07.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.055+0000 7f77b1dff700 1 Processor -- start 2023-07-14T11:22:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.055+0000 7f77b1dff700 1 -- start start 2023-07-14T11:22:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77b1dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac0ffb00 0x7f77ac1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77b1dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77ac100cf0 0x7f77ac195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77b1dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f77ac101680 0x7f77ac196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77b1dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac197300 con 0x7f77ac100cf0 2023-07-14T11:22:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77b1dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac197470 con 0x7f77ac0ffb00 2023-07-14T11:22:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77b1dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac197630 con 0x7f77ac101680 2023-07-14T11:22:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77ab7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac0ffb00 0x7f77ac1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77ab7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac0ffb00 0x7f77ac1958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:54804/0 (socket says 172.21.15.161:54804) 2023-07-14T11:22:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77ab7fe700 1 -- 172.21.15.161:0/387522299 learned_addr learned my addr 172.21.15.161:0/387522299 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77aaffd700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77ac100cf0 0x7f77ac195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.056+0000 7f77abfff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f77ac101680 0x7f77ac196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77ab7fe700 1 -- 172.21.15.161:0/387522299 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f77ac101680 msgr2=0x7f77ac196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77ab7fe700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f77ac101680 0x7f77ac196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77ab7fe700 1 -- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77ac100cf0 msgr2=0x7f77ac195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77ab7fe700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77ac100cf0 0x7f77ac195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77ab7fe700 1 -- 172.21.15.161:0/387522299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c009520 con 0x7f77ac0ffb00 2023-07-14T11:22:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77abfff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f77ac101680 0x7f77ac196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77ab7fe700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac0ffb00 0x7f77ac1958e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f779c00ea50 tx=0x7f779c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77a8ff9700 1 -- 172.21.15.161:0/387522299 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00cbd0 con 0x7f77ac0ffb00 2023-07-14T11:22:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77a8ff9700 1 -- 172.21.15.161:0/387522299 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f779c00cd30 con 0x7f77ac0ffb00 2023-07-14T11:22:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77a8ff9700 1 -- 172.21.15.161:0/387522299 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c0186c0 con 0x7f77ac0ffb00 2023-07-14T11:22:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a00095f0 con 0x7f77ac0ffb00 2023-07-14T11:22:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.057+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ac19def0 con 0x7f77ac0ffb00 2023-07-14T11:22:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.059+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ac107040 con 0x7f77ac0ffb00 2023-07-14T11:22:07.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.060+0000 7f77a8ff9700 1 -- 172.21.15.161:0/387522299 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f779c018820 con 0x7f77ac0ffb00 2023-07-14T11:22:07.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.062+0000 7f77a8ff9700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 0x7f77940a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.062+0000 7f77aaffd700 1 -- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 msgr2=0x7f77940a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:07.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.062+0000 7f77a8ff9700 1 -- 172.21.15.161:0/387522299 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f779c014070 con 0x7f77ac0ffb00 2023-07-14T11:22:07.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.062+0000 7f77aaffd700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 0x7f77940a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:22:07.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.065+0000 7f77a8ff9700 1 -- 172.21.15.161:0/387522299 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f779c096620 con 0x7f77ac0ffb00 2023-07-14T11:22:07.227 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:06.981+0000 7f338a615200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:22:07.227 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:07.224+0000 7f338a615200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:22:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.263+0000 7f77aaffd700 1 -- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 msgr2=0x7f77940a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.263+0000 7f77aaffd700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 0x7f77940a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:22:07.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.283+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f77ac02cb00 con 0x7f77ac0ffb00 2023-07-14T11:22:07.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.291+0000 7f77a8ff9700 1 -- 172.21.15.161:0/387522299 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f779c099c40 con 0x7f77ac0ffb00 2023-07-14T11:22:07.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.293+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 msgr2=0x7f77940a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:22:07.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.293+0000 7f77b1dff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 0x7f77940a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.293+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac0ffb00 msgr2=0x7f77ac1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:07.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.293+0000 7f77b1dff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac0ffb00 0x7f77ac1958e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f779c00ea50 tx=0x7f779c00ed60 comp rx=0 tx=0).stop 2023-07-14T11:22:07.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 shutdown_connections 2023-07-14T11:22:07.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f77940a11a0 0x7f77940a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f77ac101680 0x7f77ac196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ac0ffb00 0x7f77ac1958e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 --2- 172.21.15.161:0/387522299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77ac100cf0 0x7f77ac195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 >> 172.21.15.161:0/387522299 conn(0x7f77ac0fb0e0 msgr2=0x7f77ac104180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:07.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 shutdown_connections 2023-07-14T11:22:07.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.294+0000 7f77b1dff700 1 -- 172.21.15.161:0/387522299 wait complete. 2023-07-14T11:22:07.317 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-07-14T11:22:07.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:07 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/203095251' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-14T11:22:07.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:07 smithi161 ceph-mon[95526]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-07-14T11:22:07.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:07 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' 2023-07-14T11:22:07.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3694771361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c540ffb00 msgr2=0x7f5c540fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:07.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/3694771361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c540ffb00 0x7f5c540fff10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009990 tx=0x7f5c50009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3694771361 shutdown_connections 2023-07-14T11:22:07.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/3694771361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c54101680 0x7f5c54107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/3694771361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c540ffb00 0x7f5c540fff10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/3694771361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c54100cf0 0x7f5c54101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3694771361 >> 172.21.15.161:0/3694771361 conn(0x7f5c540fb0e0 msgr2=0x7f5c540fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:07.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3694771361 shutdown_connections 2023-07-14T11:22:07.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.534+0000 7f5c5bfff700 1 -- 172.21.15.161:0/3694771361 wait complete. 2023-07-14T11:22:07.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.535+0000 7f5c5bfff700 1 Processor -- start 2023-07-14T11:22:07.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5bfff700 1 -- start start 2023-07-14T11:22:07.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c540ffb00 0x7f5c54072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c54100cf0 0x7f5c54072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c54101680 0x7f5c54073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5bfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f8d0 con 0x7f5c540ffb00 2023-07-14T11:22:07.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419fa10 con 0x7f5c54100cf0 2023-07-14T11:22:07.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5bfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419fb50 con 0x7f5c54101680 2023-07-14T11:22:07.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c59d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c540ffb00 0x7f5c54072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:07.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5959a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c54100cf0 0x7f5c54072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:07.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c59d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c540ffb00 0x7f5c54072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40010/0 (socket says 172.21.15.161:40010) 2023-07-14T11:22:07.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c59d9b700 1 -- 172.21.15.161:0/19608639 learned_addr learned my addr 172.21.15.161:0/19608639 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.536+0000 7f5c5a59c700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c54101680 0x7f5c54073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5959a700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c54101680 msgr2=0x7f5c54073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5959a700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c54101680 0x7f5c54073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5959a700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c540ffb00 msgr2=0x7f5c54072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5959a700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c540ffb00 0x7f5c54072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5959a700 1 -- 172.21.15.161:0/19608639 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009520 con 0x7f5c54100cf0 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5a59c700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5c54101680 0x7f5c54073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:07.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c540ffb00 0x7f5c54072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:07.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5959a700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c54100cf0 0x7f5c54072e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9f0 tx=0x7f5c4400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:07.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400cb70 con 0x7f5c54100cf0 2023-07-14T11:22:07.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c440044e0 con 0x7f5c54100cf0 2023-07-14T11:22:07.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c500095f0 con 0x7f5c54100cf0 2023-07-14T11:22:07.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44003d50 con 0x7f5c54100cf0 2023-07-14T11:22:07.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.537+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c541a01b0 con 0x7f5c54100cf0 2023-07-14T11:22:07.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.539+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5404e2f0 con 0x7f5c54100cf0 2023-07-14T11:22:07.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.540+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f5c44003ef0 con 0x7f5c54100cf0 2023-07-14T11:22:07.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.542+0000 7f5c4affd700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:07.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.542+0000 7f5c59d9b700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 msgr2=0x7f5c400a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:07.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.542+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5c44014070 con 0x7f5c54100cf0 2023-07-14T11:22:07.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.542+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:22:07.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.545+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c440963e0 con 0x7f5c54100cf0 2023-07-14T11:22:07.567 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:07 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/203095251' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-14T11:22:07.567 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:07 smithi182 ceph-mon[129366]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-07-14T11:22:07.567 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:07 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' 2023-07-14T11:22:07.568 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:07.223+0000 7fa889f0e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:22:07.568 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:07.318+0000 7fa889f0e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:22:07.568 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:07.466+0000 7fa889f0e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:22:07.568 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:07.565+0000 7fa889f0e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:22:07.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:07 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/203095251' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-14T11:22:07.587 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:07 smithi175 ceph-mon[129081]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-07-14T11:22:07.587 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:07 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' 2023-07-14T11:22:07.587 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:07.235+0000 7fc0e34f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:22:07.587 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:07.332+0000 7fc0e34f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:22:07.587 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:07.470+0000 7fc0e34f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:22:07.588 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:07.583+0000 7fc0e34f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:22:07.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.743+0000 7f5c59d9b700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 msgr2=0x7f5c400a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:07.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.743+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:22:07.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:07.805+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 --> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c541a03a0 con 0x7f5c400a1220 2023-07-14T11:22:07.996 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:07.683+0000 7f338a615200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:22:07.996 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:07.820+0000 7f338a615200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:22:08.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:08.144+0000 7f5c59d9b700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 msgr2=0x7f5c400a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:08.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:08.144+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T11:22:08.291 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:08.015+0000 7fa889f0e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:22:08.292 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:07.993+0000 7f338a615200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:22:08.292 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:08.121+0000 7f338a615200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:22:08.292 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:08.031+0000 7fc0e34f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:22:08.601 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:08 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' 2023-07-14T11:22:08.638 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:08 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' 2023-07-14T11:22:08.701 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:08 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' 2023-07-14T11:22:08.919 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:08.635+0000 7fc0e34f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:22:08.919 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:08.812+0000 7fc0e34f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:22:08.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:08.945+0000 7f5c59d9b700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 msgr2=0x7f5c400a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:08.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:08.945+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T11:22:08.964 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:08.598+0000 7fa889f0e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:22:08.964 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:08.775+0000 7fa889f0e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:22:08.996 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:08.698+0000 7f338a615200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:22:09.746 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:09.486+0000 7f338a615200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:22:09.746 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:09.699+0000 7f338a615200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:22:09.801 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:09.537+0000 7fa889f0e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:22:09.802 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:09.628+0000 7fa889f0e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:22:09.841 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:09.576+0000 7fc0e34f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:22:09.841 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:09.666+0000 7fc0e34f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:22:10.119 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:09.837+0000 7fc0e34f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:22:10.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:10.115+0000 7fc0e34f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:22:10.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:10.344+0000 7fc0e34f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:22:10.454 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:09.799+0000 7fa889f0e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:22:10.455 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:10.075+0000 7fa889f0e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:22:10.455 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:10.305+0000 7fa889f0e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:22:10.455 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:10.452+0000 7fa889f0e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:22:10.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:10.546+0000 7f5c59d9b700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 msgr2=0x7f5c400a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:10.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:10.546+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T11:22:10.756 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:10.491+0000 7fc0e34f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:22:10.986 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:10.642+0000 7f338a615200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:22:10.987 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:10.761+0000 7f338a615200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:22:11.169 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:10.752+0000 7fc0e34f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:22:11.214 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:10.713+0000 7fa889f0e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:22:11.625 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:10.984+0000 7f338a615200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:22:11.626 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:11 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:11.345+0000 7f338a615200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:22:11.714 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:11.417+0000 7fa889f0e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:22:11.714 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:11.547+0000 7fa889f0e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:22:11.774 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:11.452+0000 7fc0e34f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:22:11.775 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:11.582+0000 7fc0e34f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:22:11.996 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:11 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:11.623+0000 7f338a615200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:22:11.996 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:11 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:11.814+0000 7f338a615200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:22:12.067 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:11.736+0000 7fa889f0e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:22:12.068 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:11.850+0000 7fa889f0e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:22:12.103 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:11.771+0000 7fc0e34f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:22:12.103 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:11.885+0000 7fc0e34f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:22:12.419 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:12.099+0000 7fc0e34f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:22:12.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:12 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:12.157+0000 7f338a615200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:22:13.214 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:12.064+0000 7fa889f0e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:22:13.214 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:22:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z[131390]: 2023-07-14T11:22:12.922+0000 7fa889f0e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:22:13.420 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:22:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y[130547]: 2023-07-14T11:22:12.959+0000 7fc0e34f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:22:13.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:13.087+0000 7f338a615200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:22:13.496 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:13.262+0000 7f338a615200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:22:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:13.729+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 20) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f5c440967b0 con 0x7f5c54100cf0 2023-07-14T11:22:13.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:13.749+0000 7f5c59d9b700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 msgr2=0x7f5c400a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.161:6800/2004305703 2023-07-14T11:22:13.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:13.749+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-14T11:22:13.943 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:13.509+0000 7f338a615200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:22:13.943 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:13.658+0000 7f338a615200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:22:13.943 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:13.940+0000 7f338a615200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:22:13.944 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: Standby manager daemon z restarted 2023-07-14T11:22:13.944 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: Standby manager daemon z started 2023-07-14T11:22:13.944 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:22:13.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:22:13.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:22:13.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:22:13.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: Standby manager daemon y restarted 2023-07-14T11:22:13.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: Standby manager daemon y started 2023-07-14T11:22:13.945 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:22:13.946 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:22:13.946 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:22:13.946 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:13 smithi161 ceph-mon[95526]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:22:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: Standby manager daemon z restarted 2023-07-14T11:22:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: Standby manager daemon z started 2023-07-14T11:22:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:22:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:22:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:22:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:22:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: Standby manager daemon y restarted 2023-07-14T11:22:14.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: Standby manager daemon y started 2023-07-14T11:22:14.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:22:14.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:22:14.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:22:14.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:13 smithi175 ceph-mon[129081]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:22:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: Standby manager daemon z restarted 2023-07-14T11:22:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: Standby manager daemon z started 2023-07-14T11:22:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:22:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:22:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:22:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.182:0/3720380216' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:22:14.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: Standby manager daemon y restarted 2023-07-14T11:22:14.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: Standby manager daemon y started 2023-07-14T11:22:14.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:22:14.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:22:14.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:22:14.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:13 smithi182 ceph-mon[129366]: from='mgr.? 172.21.15.175:0/3581935592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:22:14.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:14 smithi161 ceph-mon[95526]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-07-14T11:22:15.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:15.032+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 8 ==== mgrmap(e 21) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f5c440ca7e0 con 0x7f5c54100cf0 2023-07-14T11:22:15.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:15.032+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 msgr2=0x7f5c400a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:22:15.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:15.032+0000 7f5c4affd700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/2004305703,v1:172.21.15.161:6801/2004305703] conn(0x7f5c400a1220 0x7f5c400a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:14 smithi175 ceph-mon[129081]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-07-14T11:22:15.173 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:14 smithi182 ceph-mon[129366]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-07-14T11:22:15.299 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.012+0000 7f338a615200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:22:15.299 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.229+0000 7f32e9333700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.300 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.229+0000 7f32e9333700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.300 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.229+0000 7f32e9333700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.300 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.229+0000 7f32e9333700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.300 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.229+0000 7f32e9333700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.300 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.230+0000 7f32e632d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.300 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.230+0000 7f32e632d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.300 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.230+0000 7f32e632d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.301 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.230+0000 7f32e632d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.301 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:22:15.230+0000 7f32e632d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:22:15.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: Active manager daemon x restarted 2023-07-14T11:22:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: Activating manager daemon x 2023-07-14T11:22:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T11:22:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: mgrmap e21: x(active, starting, since 0.0146949s), standbys: y, z 2023-07-14T11:22:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:22:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:22:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:22:15.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:22:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:22:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:22:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:22:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:22:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:22:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:22:15.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:22:15.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:22:15.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:22:15.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:22:15.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:22:15.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:22:15.999 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:22:16.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:22:16.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:22:16.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:22:16.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:22:16.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:22:16.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:22:16.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:22:16.001 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:22:16.001 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:22:16.001 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:22:16.001 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:22:16.001 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:22:16.001 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:22:16.001 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:22:16.002 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:22:16.002 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:22:16.002 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: Manager daemon x is now available 2023-07-14T11:22:16.002 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:16.002 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:16.002 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:22:16.002 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:22:16.003 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:15 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:22:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.035+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mon.2 v2:172.21.15.175:3300/0 9 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f5c440041a0 con 0x7f5c54100cf0 2023-07-14T11:22:16.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.036+0000 7f5c4affd700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5c40027f50 0x7f5c40028320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.036+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 --> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c541a03a0 con 0x7f5c40027f50 2023-07-14T11:22:16.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.037+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5c40027f50 0x7f5c40028320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.042+0000 7f5c59d9b700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5c40027f50 0x7f5c40028320 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b2e0 tx=0x7f5c50005fb0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:16.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.107+0000 7f5c4affd700 1 -- 172.21.15.161:0/19608639 <== mgr.15543 v2:172.21.15.161:6800/4046273551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5c541a03a0 con 0x7f5c40027f50 2023-07-14T11:22:16.110 INFO:teuthology.orchestra.run.smithi161.stdout:Schedule set for path / 2023-07-14T11:22:16.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.109+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5c40027f50 msgr2=0x7f5c40028320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.109+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5c40027f50 0x7f5c40028320 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b2e0 tx=0x7f5c50005fb0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.109+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c54100cf0 msgr2=0x7f5c54072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.109+0000 7f5c5bfff700 1 --2- 172.21.15.161:0/19608639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c54100cf0 0x7f5c54072e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9f0 tx=0x7f5c4400ed00 comp rx=0 tx=0).stop 2023-07-14T11:22:16.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.109+0000 7f5c5a59c700 1 -- 172.21.15.161:0/19608639 reap_dead start 2023-07-14T11:22:16.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.109+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 shutdown_connections 2023-07-14T11:22:16.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.109+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 >> 172.21.15.161:0/19608639 conn(0x7f5c540fb0e0 msgr2=0x7f5c540fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:16.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.110+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 shutdown_connections 2023-07-14T11:22:16.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.110+0000 7f5c5bfff700 1 -- 172.21.15.161:0/19608639 wait complete. 2023-07-14T11:22:16.133 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-07-14T11:22:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: Active manager daemon x restarted 2023-07-14T11:22:16.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: Activating manager daemon x 2023-07-14T11:22:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T11:22:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: mgrmap e21: x(active, starting, since 0.0146949s), standbys: y, z 2023-07-14T11:22:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:22:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:22:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:22:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:22:16.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:22:16.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:22:16.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:22:16.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:22:16.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:22:16.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:22:16.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:22:16.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:22:16.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:22:16.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:22:16.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:22:16.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:22:16.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:22:16.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:22:16.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:22:16.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: Manager daemon x is now available 2023-07-14T11:22:16.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:16.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:16.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:22:16.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:22:16.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:15 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:22:16.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: Active manager daemon x restarted 2023-07-14T11:22:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: Activating manager daemon x 2023-07-14T11:22:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T11:22:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: mgrmap e21: x(active, starting, since 0.0146949s), standbys: y, z 2023-07-14T11:22:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:22:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:22:16.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:22:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:22:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:22:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:22:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:22:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:22:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:22:16.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:22:16.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:22:16.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:22:16.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:22:16.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:22:16.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:22:16.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:22:16.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:22:16.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:22:16.219 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:22:16.220 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: Manager daemon x is now available 2023-07-14T11:22:16.220 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:16.220 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:16.220 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:22:16.220 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:22:16.220 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:15 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:22:16.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 -- 172.21.15.161:0/487290893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800ffb00 msgr2=0x7fd0800fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 --2- 172.21.15.161:0/487290893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800ffb00 0x7fd0800fff10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd070009990 tx=0x7fd070009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 -- 172.21.15.161:0/487290893 shutdown_connections 2023-07-14T11:22:16.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 --2- 172.21.15.161:0/487290893 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd080101680 0x7fd080107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 --2- 172.21.15.161:0/487290893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800ffb00 0x7fd0800fff10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 --2- 172.21.15.161:0/487290893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080100cf0 0x7fd080101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 -- 172.21.15.161:0/487290893 >> 172.21.15.161:0/487290893 conn(0x7fd0800fb0e0 msgr2=0x7fd0800fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:16.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 -- 172.21.15.161:0/487290893 shutdown_connections 2023-07-14T11:22:16.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.340+0000 7fd0859ff700 1 -- 172.21.15.161:0/487290893 wait complete. 2023-07-14T11:22:16.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.342+0000 7fd0859ff700 1 Processor -- start 2023-07-14T11:22:16.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.342+0000 7fd0859ff700 1 -- start start 2023-07-14T11:22:16.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd0859ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0800ffb00 0x7fd08006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd0859ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080100cf0 0x7fd08006f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd0859ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd080101680 0x7fd080070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd0859ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800708a0 con 0x7fd080100cf0 2023-07-14T11:22:16.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd0859ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080070a10 con 0x7fd080101680 2023-07-14T11:22:16.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd0859ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080070bd0 con 0x7fd0800ffb00 2023-07-14T11:22:16.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0800ffb00 0x7fd08006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0800ffb00 0x7fd08006ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:40804/0 (socket says 172.21.15.161:40804) 2023-07-14T11:22:16.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 -- 172.21.15.161:0/3118930401 learned_addr learned my addr 172.21.15.161:0/3118930401 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:16.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07f7fe700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd080101680 0x7fd080070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07e7fc700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080100cf0 0x7fd08006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 -- 172.21.15.161:0/3118930401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd080101680 msgr2=0x7fd080070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd080101680 0x7fd080070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 -- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080100cf0 msgr2=0x7fd08006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080100cf0 0x7fd08006f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.343+0000 7fd07effd700 1 -- 172.21.15.161:0/3118930401 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0700095f0 con 0x7fd0800ffb00 2023-07-14T11:22:16.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.344+0000 7fd07effd700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0800ffb00 0x7fd08006ee30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd070004730 tx=0x7fd070004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:16.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.344+0000 7fd0849fd700 1 -- 172.21.15.161:0/3118930401 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07001d070 con 0x7fd0800ffb00 2023-07-14T11:22:16.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.344+0000 7fd0849fd700 1 -- 172.21.15.161:0/3118930401 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd070021410 con 0x7fd0800ffb00 2023-07-14T11:22:16.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.344+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd080070e50 con 0x7fd0800ffb00 2023-07-14T11:22:16.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.344+0000 7fd0849fd700 1 -- 172.21.15.161:0/3118930401 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070017650 con 0x7fd0800ffb00 2023-07-14T11:22:16.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.344+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd080073670 con 0x7fd0800ffb00 2023-07-14T11:22:16.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.345+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08004e2f0 con 0x7fd0800ffb00 2023-07-14T11:22:16.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.347+0000 7fd0849fd700 1 -- 172.21.15.161:0/3118930401 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fd070021580 con 0x7fd0800ffb00 2023-07-14T11:22:16.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.350+0000 7fd0849fd700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0680a0f50 0x7fd0680a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.350+0000 7fd0849fd700 1 -- 172.21.15.161:0/3118930401 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fd0700ce4a0 con 0x7fd0800ffb00 2023-07-14T11:22:16.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.350+0000 7fd07e7fc700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0680a0f50 0x7fd0680a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.351+0000 7fd07e7fc700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0680a0f50 0x7fd0680a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd08006fe80 tx=0x7fd074009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:16.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.351+0000 7fd0849fd700 1 -- 172.21.15.161:0/3118930401 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd070097b20 con 0x7fd0800ffb00 2023-07-14T11:22:16.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.592+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 --> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fd080073fe0 con 0x7fd0680a0f50 2023-07-14T11:22:16.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.645+0000 7fd0849fd700 1 -- 172.21.15.161:0/3118930401 <== mgr.15543 v2:172.21.15.161:6800/4046273551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd080073fe0 con 0x7fd0680a0f50 2023-07-14T11:22:16.648 INFO:teuthology.orchestra.run.smithi161.stdout:Retention added to path / 2023-07-14T11:22:16.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0680a0f50 msgr2=0x7fd0680a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0680a0f50 0x7fd0680a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd08006fe80 tx=0x7fd074009260 comp rx=0 tx=0).stop 2023-07-14T11:22:16.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0800ffb00 msgr2=0x7fd08006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0800ffb00 0x7fd08006ee30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd070004730 tx=0x7fd070004760 comp rx=0 tx=0).stop 2023-07-14T11:22:16.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 shutdown_connections 2023-07-14T11:22:16.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0680a0f50 0x7fd0680a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd080101680 0x7fd080070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0800ffb00 0x7fd08006ee30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 --2- 172.21.15.161:0/3118930401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080100cf0 0x7fd08006f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 >> 172.21.15.161:0/3118930401 conn(0x7fd0800fb0e0 msgr2=0x7fd0800fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:16.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 shutdown_connections 2023-07-14T11:22:16.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.648+0000 7fd0859ff700 1 -- 172.21.15.161:0/3118930401 wait complete. 2023-07-14T11:22:16.670 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-14T11:22:16.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 -- 172.21.15.161:0/442600115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 msgr2=0x7f54f80fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 --2- 172.21.15.161:0/442600115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f80fee10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009990 tx=0x7f54e8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 -- 172.21.15.161:0/442600115 shutdown_connections 2023-07-14T11:22:16.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 --2- 172.21.15.161:0/442600115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f8100640 0x7f54f8106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 --2- 172.21.15.161:0/442600115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f80fee10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 --2- 172.21.15.161:0/442600115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54f80ffc90 0x7f54f8100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 -- 172.21.15.161:0/442600115 >> 172.21.15.161:0/442600115 conn(0x7f54f80fa000 msgr2=0x7f54f80fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:16.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 -- 172.21.15.161:0/442600115 shutdown_connections 2023-07-14T11:22:16.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.876+0000 7f54fdfff700 1 -- 172.21.15.161:0/442600115 wait complete. 2023-07-14T11:22:16.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.878+0000 7f54fdfff700 1 Processor -- start 2023-07-14T11:22:16.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54fdfff700 1 -- start start 2023-07-14T11:22:16.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54fdfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f8195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54fdfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54f80ffc90 0x7f54f8195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54fdfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f8100640 0x7f54f8196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54fdfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f8197390 con 0x7f54f80ffc90 2023-07-14T11:22:16.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54fdfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f8197500 con 0x7f54f8100640 2023-07-14T11:22:16.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54fdfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f81976c0 con 0x7f54f80fea00 2023-07-14T11:22:16.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54f77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f8195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54f77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f8195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:40816/0 (socket says 172.21.15.161:40816) 2023-07-14T11:22:16.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54f77fe700 1 -- 172.21.15.161:0/1450119637 learned_addr learned my addr 172.21.15.161:0/1450119637 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:16.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.879+0000 7f54f6ffd700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54f80ffc90 0x7f54f8195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f7fff700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f8100640 0x7f54f8196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f77fe700 1 -- 172.21.15.161:0/1450119637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f8100640 msgr2=0x7f54f8196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f77fe700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f8100640 0x7f54f8196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f77fe700 1 -- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54f80ffc90 msgr2=0x7f54f8195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f77fe700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54f80ffc90 0x7f54f8195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f77fe700 1 -- 172.21.15.161:0/1450119637 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e80095f0 con 0x7f54f80fea00 2023-07-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f77fe700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f8195970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54e8004730 tx=0x7f54e8004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e801d070 con 0x7f54f80fea00 2023-07-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f8197940 con 0x7f54f80fea00 2023-07-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.880+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f819e160 con 0x7f54f80fea00 2023-07-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.881+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54e8004c90 con 0x7f54f80fea00 2023-07-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.881+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8005460 con 0x7f54f80fea00 2023-07-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.882+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e4005320 con 0x7f54f80fea00 2023-07-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.883+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f54e80055c0 con 0x7f54f80fea00 2023-07-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.885+0000 7f54f4ff9700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54e00a0fd0 0x7f54e00a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.885+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f54e80cdc20 con 0x7f54f80fea00 2023-07-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.885+0000 7f54f6ffd700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54e00a0fd0 0x7f54e00a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.886+0000 7f54f6ffd700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54e00a0fd0 0x7f54e00a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f54ec007c60 tx=0x7f54ec0073d0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:16.888+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f54e805a790 con 0x7f54f80fea00 2023-07-14T11:22:17.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.042+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f54e800fc70 con 0x7f54f80fea00 2023-07-14T11:22:17.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.148+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 --> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f54e4000bf0 con 0x7f54e00a0fd0 2023-07-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.157+0000 7f54f4ff9700 1 -- 172.21.15.161:0/1450119637 <== mgr.15543 v2:172.21.15.161:6800/4046273551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f54e4000bf0 con 0x7f54e00a0fd0 2023-07-14T11:22:17.160 INFO:teuthology.orchestra.run.smithi161.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-14T00:00:00", "created": "2023-07-14T11:22:16", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-07-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54e00a0fd0 msgr2=0x7f54e00a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54e00a0fd0 0x7f54e00a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f54ec007c60 tx=0x7f54ec0073d0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 msgr2=0x7f54f8195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f8195970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54e8004730 tx=0x7f54e8004760 comp rx=0 tx=0).stop 2023-07-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 shutdown_connections 2023-07-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54e00a0fd0 0x7f54e00a3480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f8100640 0x7f54f8196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f54f80fea00 0x7f54f8195970 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 --2- 172.21.15.161:0/1450119637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54f80ffc90 0x7f54f8195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 >> 172.21.15.161:0/1450119637 conn(0x7f54f80fa000 msgr2=0x7f54f8067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 shutdown_connections 2023-07-14T11:22:17.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.159+0000 7f54fdfff700 1 -- 172.21.15.161:0/1450119637 wait complete. 2023-07-14T11:22:17.182 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-14T11:22:17.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.387+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/3939655704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2410c0d0 msgr2=0x7f9d2410e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:17.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.387+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/3939655704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2410c0d0 0x7f9d2410e4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009f90 tx=0x7f9d1c00b330 comp rx=0 tx=0).stop 2023-07-14T11:22:17.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.387+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/3939655704 shutdown_connections 2023-07-14T11:22:17.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.387+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/3939655704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d2410c0d0 0x7f9d2410e4b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.387+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/3939655704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9d24074550 0x7f9d240729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.387+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/3939655704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d24072f70 0x7f9d240733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.387+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/3939655704 >> 172.21.15.161:0/3939655704 conn(0x7f9d240fb0e0 msgr2=0x7f9d240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:17.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.388+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/3939655704 shutdown_connections 2023-07-14T11:22:17.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.388+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/3939655704 wait complete. 2023-07-14T11:22:17.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 Processor -- start 2023-07-14T11:22:17.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 -- start start 2023-07-14T11:22:17.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d24072f70 0x7f9d2419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:17.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d24074550 0x7f9d2419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:17.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:17.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2419dab0 con 0x7f9d24074550 2023-07-14T11:22:17.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2419dc20 con 0x7f9d24072f70 2023-07-14T11:22:17.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2b3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2419dde0 con 0x7f9d2410c0d0 2023-07-14T11:22:17.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2919b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d24072f70 0x7f9d2419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:17.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2899a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d24074550 0x7f9d2419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:17.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2919b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d24072f70 0x7f9d2419c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44572/0 (socket says 172.21.15.161:44572) 2023-07-14T11:22:17.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.389+0000 7f9d2919b700 1 -- 172.21.15.161:0/1006601399 learned_addr learned my addr 172.21.15.161:0/1006601399 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2999c700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2919b700 1 -- 172.21.15.161:0/1006601399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9d2410c0d0 msgr2=0x7f9d2419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2919b700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2919b700 1 -- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d24074550 msgr2=0x7f9d2419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2919b700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d24074550 0x7f9d2419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2919b700 1 -- 172.21.15.161:0/1006601399 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009c40 con 0x7f9d24072f70 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2919b700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d24072f70 0x7f9d2419c090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000e9f0 tx=0x7f9d2000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:17.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d1a7fc700 1 -- 172.21.15.161:0/1006601399 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20009690 con 0x7f9d24072f70 2023-07-14T11:22:17.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d1a7fc700 1 -- 172.21.15.161:0/1006601399 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d2000ce80 con 0x7f9d24072f70 2023-07-14T11:22:17.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2419e0c0 con 0x7f9d24072f70 2023-07-14T11:22:17.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.390+0000 7f9d1a7fc700 1 -- 172.21.15.161:0/1006601399 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20018420 con 0x7f9d24072f70 2023-07-14T11:22:17.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.391+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d241a4880 con 0x7f9d24072f70 2023-07-14T11:22:17.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.392+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24101c20 con 0x7f9d24072f70 2023-07-14T11:22:17.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.394+0000 7f9d1a7fc700 1 -- 172.21.15.161:0/1006601399 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f9d2000c9c0 con 0x7f9d24072f70 2023-07-14T11:22:17.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.395+0000 7f9d1a7fc700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9d100a1350 0x7f9d100a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:17.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.395+0000 7f9d1a7fc700 1 -- 172.21.15.161:0/1006601399 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f9d20014070 con 0x7f9d24072f70 2023-07-14T11:22:17.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.395+0000 7f9d2899a700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9d100a1350 0x7f9d100a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:17.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.396+0000 7f9d2899a700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9d100a1350 0x7f9d100a3800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d24074270 tx=0x7f9d14009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:17.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.399+0000 7f9d1a7fc700 1 -- 172.21.15.161:0/1006601399 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9d20051790 con 0x7f9d24072f70 2023-07-14T11:22:17.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:17 smithi175 ceph-mon[129081]: [14/Jul/2023:11:22:15] ENGINE Bus STARTING 2023-07-14T11:22:17.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:17 smithi175 ceph-mon[129081]: [14/Jul/2023:11:22:15] ENGINE Serving on http://172.21.15.161:8765 2023-07-14T11:22:17.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:17 smithi175 ceph-mon[129081]: [14/Jul/2023:11:22:15] ENGINE Serving on https://172.21.15.161:7150 2023-07-14T11:22:17.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:17 smithi175 ceph-mon[129081]: [14/Jul/2023:11:22:15] ENGINE Bus STARTED 2023-07-14T11:22:17.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:17 smithi175 ceph-mon[129081]: mgrmap e22: x(active, since 1.01961s), standbys: y, z 2023-07-14T11:22:17.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:17 smithi175 ceph-mon[129081]: from='client.25592 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:17.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:17 smithi175 ceph-mon[129081]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:17.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:17 smithi182 ceph-mon[129366]: [14/Jul/2023:11:22:15] ENGINE Bus STARTING 2023-07-14T11:22:17.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:17 smithi182 ceph-mon[129366]: [14/Jul/2023:11:22:15] ENGINE Serving on http://172.21.15.161:8765 2023-07-14T11:22:17.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:17 smithi182 ceph-mon[129366]: [14/Jul/2023:11:22:15] ENGINE Serving on https://172.21.15.161:7150 2023-07-14T11:22:17.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:17 smithi182 ceph-mon[129366]: [14/Jul/2023:11:22:15] ENGINE Bus STARTED 2023-07-14T11:22:17.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:17 smithi182 ceph-mon[129366]: mgrmap e22: x(active, since 1.01961s), standbys: y, z 2023-07-14T11:22:17.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:17 smithi182 ceph-mon[129366]: from='client.25592 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:17.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:17 smithi182 ceph-mon[129366]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:17 smithi161 ceph-mon[95526]: [14/Jul/2023:11:22:15] ENGINE Bus STARTING 2023-07-14T11:22:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:17 smithi161 ceph-mon[95526]: [14/Jul/2023:11:22:15] ENGINE Serving on http://172.21.15.161:8765 2023-07-14T11:22:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:17 smithi161 ceph-mon[95526]: [14/Jul/2023:11:22:15] ENGINE Serving on https://172.21.15.161:7150 2023-07-14T11:22:17.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:17 smithi161 ceph-mon[95526]: [14/Jul/2023:11:22:15] ENGINE Bus STARTED 2023-07-14T11:22:17.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:17 smithi161 ceph-mon[95526]: mgrmap e22: x(active, since 1.01961s), standbys: y, z 2023-07-14T11:22:17.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:17 smithi161 ceph-mon[95526]: from='client.25592 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:17.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:17 smithi161 ceph-mon[95526]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:17.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.640+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 --> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9d24060950 con 0x7f9d100a1350 2023-07-14T11:22:17.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.648+0000 7f9d1a7fc700 1 -- 172.21.15.161:0/1006601399 <== mgr.15543 v2:172.21.15.161:6800/4046273551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f9d24060950 con 0x7f9d100a1350 2023-07-14T11:22:17.651 INFO:teuthology.orchestra.run.smithi161.stdout:/ 1M 6M3h 2023-07-14T11:22:17.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9d100a1350 msgr2=0x7f9d100a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:17.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9d100a1350 0x7f9d100a3800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d24074270 tx=0x7f9d14009260 comp rx=0 tx=0).stop 2023-07-14T11:22:17.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d24072f70 msgr2=0x7f9d2419c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:17.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d24072f70 0x7f9d2419c090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000e9f0 tx=0x7f9d2000ed00 comp rx=0 tx=0).stop 2023-07-14T11:22:17.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 shutdown_connections 2023-07-14T11:22:17.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9d100a1350 0x7f9d100a3800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d24072f70 0x7f9d2419c090 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 --2- 172.21.15.161:0/1006601399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d24074550 0x7f9d2419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:17.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.650+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 >> 172.21.15.161:0/1006601399 conn(0x7f9d240fb0e0 msgr2=0x7f9d240fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:17.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.651+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 shutdown_connections 2023-07-14T11:22:17.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:17.651+0000 7f9d2b3ff700 1 -- 172.21.15.161:0/1006601399 wait complete. 2023-07-14T11:22:17.673 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-07-14T11:22:17.747 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-07-14T11:22:17.759 INFO:teuthology.run_tasks:Running task workunit... 2023-07-14T11:22:17.800 INFO:tasks.workunit:Pulling workunits from ref 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:22:17.800 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-14T11:22:17.800 INFO:tasks.workunit:timeout=3h 2023-07-14T11:22:17.801 INFO:tasks.workunit:cleanup=True 2023-07-14T11:22:17.801 DEBUG:teuthology.orchestra.run.smithi161:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:22:17.849 INFO:teuthology.orchestra.run.smithi161.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-14T11:22:17.850 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-14T11:22:17.850 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-07-14T11:22:17.850 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:22:17.850 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-14T11:22:17.850 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 11:20:14.135102112 +0000 2023-07-14T11:22:17.850 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 11:20:14.135102112 +0000 2023-07-14T11:22:17.851 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 11:21:55.183705422 +0000 2023-07-14T11:22:17.851 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2023-07-14 11:20:14.135102112 +0000 2023-07-14T11:22:17.851 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-14T11:22:17.851 DEBUG:teuthology.orchestra.run.smithi161:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-14T11:22:17.933 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:22:17.982 INFO:tasks.workunit.client.0.smithi161.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-14T11:22:18.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:18 smithi161 ceph-mon[95526]: from='client.25642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:18.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:18 smithi161 ceph-mon[95526]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:18.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:18 smithi161 ceph-mon[95526]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-14T11:22:18.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:18 smithi161 ceph-mon[95526]: from='client.25648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:18.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:18 smithi175 ceph-mon[129081]: from='client.25642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:18.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:18 smithi175 ceph-mon[129081]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:18.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:18 smithi175 ceph-mon[129081]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-14T11:22:18.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:18 smithi175 ceph-mon[129081]: from='client.25648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:18 smithi182 ceph-mon[129366]: from='client.25642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:18 smithi182 ceph-mon[129366]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:18 smithi182 ceph-mon[129366]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-14T11:22:18.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:18 smithi182 ceph-mon[129366]: from='client.25648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:19.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='client.25652 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:19.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='client.25652 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:19.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='client.25652 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:22:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:19.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:19.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:20 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:22:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:20 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:22:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:20 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:22:20.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:20 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:22:20.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:20 smithi175 ceph-mon[129081]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:20 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:22:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:20 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:22:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:20 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:22:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:20 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:22:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:20 smithi182 ceph-mon[129366]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:20 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:22:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:20 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:22:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:20 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:22:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:20 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:22:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:20 smithi161 ceph-mon[95526]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:22:21.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:21.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:21.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:22:21.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:22:21.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:22:21.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:22:21.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T11:22:21.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: Updating smithi175:/etc/ceph/ceph.conf 2023-07-14T11:22:21.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:21 smithi175 ceph-mon[129081]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:22:21.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:22:21.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:22:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:22:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:22:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T11:22:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: Updating smithi175:/etc/ceph/ceph.conf 2023-07-14T11:22:21.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:21 smithi182 ceph-mon[129366]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:22:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:22:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:22:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:22:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:22:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:22:21.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T11:22:21.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: Updating smithi175:/etc/ceph/ceph.conf 2023-07-14T11:22:21.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:21 smithi161 ceph-mon[95526]: Updating smithi182:/etc/ceph/ceph.conf 2023-07-14T11:22:22.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: Updating smithi182:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2023-07-14T11:22:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:22 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: Updating smithi182:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2023-07-14T11:22:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:22 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: Updating smithi182:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-14T11:22:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2023-07-14T11:22:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: Updating smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: Updating smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:22 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:23.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:22:23.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:22:23.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:23.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:22:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:22:23.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:22:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: Updating smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:22:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:23.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:22:23.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:22:23.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:22:23.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:22:24.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:24 smithi175 ceph-mon[129081]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-14T11:22:24.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:24 smithi182 ceph-mon[129366]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-14T11:22:24.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:24 smithi161 ceph-mon[95526]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-14T11:22:26.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:26 smithi175 ceph-mon[129081]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-14T11:22:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:26 smithi182 ceph-mon[129366]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-14T11:22:26.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:26 smithi161 ceph-mon[95526]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-14T11:22:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:28 smithi175 ceph-mon[129081]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-14T11:22:28.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:28 smithi182 ceph-mon[129366]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-14T11:22:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:28 smithi161 ceph-mon[95526]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-14T11:22:30.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:30 smithi175 ceph-mon[129081]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T11:22:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:30 smithi182 ceph-mon[129366]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T11:22:30.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:30 smithi161 ceph-mon[95526]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T11:22:32.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:32 smithi175 ceph-mon[129081]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-14T11:22:32.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:32 smithi182 ceph-mon[129366]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-14T11:22:32.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:32 smithi161 ceph-mon[95526]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-14T11:22:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:33 smithi161 ceph-mon[95526]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-14T11:22:34.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:33 smithi175 ceph-mon[129081]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-14T11:22:34.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:33 smithi182 ceph-mon[129366]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-14T11:22:35.715 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:22:35.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 -- 172.21.15.161:0/2707545994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa118073160 msgr2=0x7fa11810be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/2707545994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa118073160 0x7fa11810be70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa100009990 tx=0x7fa100009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 -- 172.21.15.161:0/2707545994 shutdown_connections 2023-07-14T11:22:35.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/2707545994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa11810c3b0 0x7fa11810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/2707545994 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 0x7fa118072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/2707545994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa118073160 0x7fa11810be70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 -- 172.21.15.161:0/2707545994 >> 172.21.15.161:0/2707545994 conn(0x7fa1180fb690 msgr2=0x7fa1180fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:35.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 -- 172.21.15.161:0/2707545994 shutdown_connections 2023-07-14T11:22:35.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.924+0000 7fa11e5ff700 1 -- 172.21.15.161:0/2707545994 wait complete. 2023-07-14T11:22:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.925+0000 7fa11e5ff700 1 Processor -- start 2023-07-14T11:22:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.925+0000 7fa11e5ff700 1 -- start start 2023-07-14T11:22:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.925+0000 7fa11e5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 0x7fa118197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa11e5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa118073160 0x7fa118198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa11e5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11810c3b0 0x7fa11819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa11e5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11819ce20 con 0x7fa11810c3b0 2023-07-14T11:22:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa11e5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11819cf90 con 0x7fa118073160 2023-07-14T11:22:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa11e5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11819d150 con 0x7fa118072850 2023-07-14T11:22:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 0x7fa118197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 0x7fa118197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45492/0 (socket says 172.21.15.161:45492) 2023-07-14T11:22:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 -- 172.21.15.161:0/3956916771 learned_addr learned my addr 172.21.15.161:0/3956916771 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa11cb9c700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11810c3b0 0x7fa11819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa10ffff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa118073160 0x7fa118198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 -- 172.21.15.161:0/3956916771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa118073160 msgr2=0x7fa118198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa118073160 0x7fa118198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 -- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11810c3b0 msgr2=0x7fa11819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11810c3b0 0x7fa11819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 -- 172.21.15.161:0/3956916771 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa110009c40 con 0x7fa118072850 2023-07-14T11:22:35.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.926+0000 7fa117fff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 0x7fa118197e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa10800e9a0 tx=0x7fa10800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:35.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.927+0000 7fa115ffb700 1 -- 172.21.15.161:0/3956916771 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108009790 con 0x7fa118072850 2023-07-14T11:22:35.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.927+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1000095f0 con 0x7fa118072850 2023-07-14T11:22:35.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.927+0000 7fa115ffb700 1 -- 172.21.15.161:0/3956916771 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10800cbd0 con 0x7fa118072850 2023-07-14T11:22:35.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.927+0000 7fa115ffb700 1 -- 172.21.15.161:0/3956916771 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800eee0 con 0x7fa118072850 2023-07-14T11:22:35.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.927+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1181a4a90 con 0x7fa118072850 2023-07-14T11:22:35.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.930+0000 7fa115ffb700 1 -- 172.21.15.161:0/3956916771 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7fa10800cd40 con 0x7fa118072850 2023-07-14T11:22:35.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.931+0000 7fa131ea4b80 1 Processor -- start 2023-07-14T11:22:35.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.931+0000 7fa131ea4b80 1 -- start start 2023-07-14T11:22:35.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.931+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eed06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eedb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eedbad0 0x555c8eeddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8edd6cb0 con 0x555c8eed0be0 2023-07-14T11:22:35.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8ee29a40 con 0x555c8eed02d0 2023-07-14T11:22:35.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8ee22ed0 con 0x555c8eedbad0 2023-07-14T11:22:35.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eedb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eedb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39156/0 (socket says 172.21.15.161:39156) 2023-07-14T11:22:35.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 -- 172.21.15.161:0/1385702633 learned_addr learned my addr 172.21.15.161:0/1385702633 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:35.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa11cb9c700 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eed06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa115ffb700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1040a1350 0x7fa1040a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa10ffff700 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eedbad0 0x555c8eeddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa115ffb700 1 -- 172.21.15.161:0/3956916771 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa108014070 con 0x7fa118072850 2023-07-14T11:22:35.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa11cb9c700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1040a1350 0x7fa1040a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 -- 172.21.15.161:0/1385702633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eedbad0 msgr2=0x555c8eeddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eedbad0 0x555c8eeddeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 -- 172.21.15.161:0/1385702633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 msgr2=0x555c8eed06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eed06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.932+0000 7fa117fff700 1 -- 172.21.15.161:0/1385702633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c8ec50de0 con 0x555c8eed0be0 2023-07-14T11:22:35.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.933+0000 7fa117fff700 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eedb590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa108099db0 tx=0x7fa108099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568eaa3c065b48ba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:35.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.933+0000 7fa10e7fc700 1 -- 172.21.15.161:0/1385702633 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108099a10 con 0x555c8eed0be0 2023-07-14T11:22:35.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.933+0000 7fa11cb9c700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1040a1350 0x7fa1040a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa110009fd0 tx=0x7fa110004b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:35.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.933+0000 7fa10e7fc700 1 -- 172.21.15.161:0/1385702633 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10809a0b0 con 0x555c8eed0be0 2023-07-14T11:22:35.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.933+0000 7fa10e7fc700 1 -- 172.21.15.161:0/1385702633 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10809a570 con 0x555c8eed0be0 2023-07-14T11:22:35.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1385702633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 msgr2=0x555c8eedb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eedb590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa108099db0 tx=0x7fa108099f00 comp rx=0 tx=0).stop 2023-07-14T11:22:35.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1385702633 shutdown_connections 2023-07-14T11:22:35.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eedbad0 0x555c8eeddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eed06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1385702633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eedb590 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1385702633 >> 172.21.15.161:0/1385702633 conn(0x555c8eec6da0 msgr2=0x555c8eecc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:35.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1385702633 shutdown_connections 2023-07-14T11:22:35.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1385702633 wait complete. 2023-07-14T11:22:35.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.934+0000 7fa131ea4b80 1 -- start start 2023-07-14T11:22:35.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 1 Processor -- start 2023-07-14T11:22:35.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eedb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eedbad0 0x555c8eed8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8edc2ac0 con 0x555c8eedbad0 2023-07-14T11:22:35.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8edc27c0 con 0x555c8eed02d0 2023-07-14T11:22:35.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8eca6920 con 0x555c8eed0be0 2023-07-14T11:22:35.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa131ea4b80 10 client.0 _ll_register_callbacks cb 0x555c8edc0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:35.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa11cb9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eedbad0 0x555c8eed8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa10ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa11cb9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eedbad0 0x555c8eed8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39162/0 (socket says 172.21.15.161:39162) 2023-07-14T11:22:35.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa11cb9c700 1 -- 172.21.15.161:0/1447453348 learned_addr learned my addr 172.21.15.161:0/1447453348 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:35.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.935+0000 7fa117fff700 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eedb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa10ffff700 1 -- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 msgr2=0x555c8eedb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa10ffff700 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eedb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa10ffff700 1 -- 172.21.15.161:0/1447453348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eedbad0 msgr2=0x555c8eed8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa10ffff700 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eedbad0 0x555c8eed8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa10ffff700 1 -- 172.21.15.161:0/1447453348 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1181a4a90 con 0x555c8eed0be0 2023-07-14T11:22:35.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa11cb9c700 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eedbad0 0x555c8eed8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:35.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa10ffff700 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed8840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa118199450 tx=0x7fa10000bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f91cd511c6b335 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:35.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa0f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:22:35.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa0f77fe700 1 -- 172.21.15.161:0/1447453348 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100004500 con 0x555c8eed0be0 2023-07-14T11:22:35.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.936+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c8ed7c4f0 con 0x555c8eed0be0 2023-07-14T11:22:35.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa0f77fe700 1 -- 172.21.15.161:0/1447453348 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa100004660 con 0x555c8eed0be0 2023-07-14T11:22:35.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa0f77fe700 1 -- 172.21.15.161:0/1447453348 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa100004860 con 0x555c8eed0be0 2023-07-14T11:22:35.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 10 client.25666 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:35.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555c8ec39c60 con 0x555c8eed0be0 2023-07-14T11:22:35.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa0f77fe700 1 -- 172.21.15.161:0/1447453348 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c8ed7c4f0 con 0x555c8eed0be0 2023-07-14T11:22:35.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa0f77fe700 1 -- 172.21.15.161:0/1447453348 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa10002c070 con 0x555c8eed0be0 2023-07-14T11:22:35.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 10 client.25666 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:35.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 10 client.25666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:35.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 20 client.25666 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:35.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8eeeb340 0x555c8eeed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 4 client.25666 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:35.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.937+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c8ee1beb0 con 0x555c8eeeb340 2023-07-14T11:22:35.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.938+0000 7fa117fff700 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8eeeb340 0x555c8eeed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.938+0000 7fa0f77fe700 10 client.25666 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:35.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.938+0000 7fa117fff700 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8eeeb340 0x555c8eeed750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d5a44898d663bb server_cookie=9fa2e4b48020e8f7 in_seq=0 out_seq=0 2023-07-14T11:22:35.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa0f77fe700 1 -- 172.21.15.161:0/1447453348 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555c8ee1beb0 con 0x555c8eeeb340 2023-07-14T11:22:35.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa0f77fe700 10 client.25666 handle_command_reply: tid=0 2023-07-14T11:22:35.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 client.25666 shutdown 2023-07-14T11:22:35.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 msgr2=0x555c8eed8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed8840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa118199450 tx=0x7fa10000bf80 comp rx=0 tx=0).stop 2023-07-14T11:22:35.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 shutdown_connections 2023-07-14T11:22:35.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8eeeb340 0x555c8eeed750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eedbad0 0x555c8eed8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eed02d0 0x555c8eedb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa0f77fe700 0 client.25666 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:35.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/1447453348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed8840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 >> 172.21.15.161:0/1447453348 conn(0x555c8eec6da0 msgr2=0x555c8eecc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:35.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 shutdown_connections 2023-07-14T11:22:35.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.939+0000 7fa131ea4b80 1 -- 172.21.15.161:0/1447453348 wait complete. 2023-07-14T11:22:35.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.940+0000 7fa131ea4b80 20 client.25666 trim_cache size 0 max 16384 2023-07-14T11:22:35.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.956+0000 7fa131ea4b80 1 Processor -- start 2023-07-14T11:22:35.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa131ea4b80 1 -- start start 2023-07-14T11:22:35.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed02d0 0x555c8eed3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8eed5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8eec3fa0 con 0x555c8eed02d0 2023-07-14T11:22:35.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8eec6970 con 0x555c8eedbad0 2023-07-14T11:22:35.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8eed3660 con 0x555c8eed0be0 2023-07-14T11:22:35.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa117fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed02d0 0x555c8eed3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa117fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed02d0 0x555c8eed3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39168/0 (socket says 172.21.15.161:39168) 2023-07-14T11:22:35.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa117fff700 1 -- 172.21.15.161:0/2241413996 learned_addr learned my addr 172.21.15.161:0/2241413996 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:35.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa11cb9c700 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8eed5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.957+0000 7fa10ffff700 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa117fff700 1 -- 172.21.15.161:0/2241413996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 msgr2=0x555c8eed40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa117fff700 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa117fff700 1 -- 172.21.15.161:0/2241413996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 msgr2=0x555c8eed5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa117fff700 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8eed5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa117fff700 1 -- 172.21.15.161:0/2241413996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c8ec77250 con 0x555c8eed02d0 2023-07-14T11:22:35.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa117fff700 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed02d0 0x555c8eed3b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1080965b0 tx=0x7fa1080ca7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a3fc27b771cef6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:35.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa10effd700 1 -- 172.21.15.161:0/2241413996 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10809e0e0 con 0x555c8eed02d0 2023-07-14T11:22:35.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa10effd700 1 -- 172.21.15.161:0/2241413996 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1080995a0 con 0x555c8eed02d0 2023-07-14T11:22:35.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.958+0000 7fa10effd700 1 -- 172.21.15.161:0/2241413996 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1080cd2b0 con 0x555c8eed02d0 2023-07-14T11:22:35.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 -- 172.21.15.161:0/2241413996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed02d0 msgr2=0x555c8eed3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed02d0 0x555c8eed3b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1080965b0 tx=0x7fa1080ca7a0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 -- 172.21.15.161:0/2241413996 shutdown_connections 2023-07-14T11:22:35.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed02d0 0x555c8eed3b70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8eed5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/2241413996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed0be0 0x555c8eed40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 -- 172.21.15.161:0/2241413996 >> 172.21.15.161:0/2241413996 conn(0x555c8eec6da0 msgr2=0x555c8eedb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:35.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 -- 172.21.15.161:0/2241413996 shutdown_connections 2023-07-14T11:22:35.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 -- 172.21.15.161:0/2241413996 wait complete. 2023-07-14T11:22:35.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 -- start start 2023-07-14T11:22:35.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.959+0000 7fa131ea4b80 1 Processor -- start 2023-07-14T11:22:35.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed02d0 0x555c8eee0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eee0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa131ea4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8ef46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8eee13c0 con 0x555c8eed0be0 2023-07-14T11:22:35.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8ef46f80 con 0x555c8eedbad0 2023-07-14T11:22:35.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa131ea4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8ef47140 con 0x555c8eed02d0 2023-07-14T11:22:35.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa131ea4b80 10 client.0 _ll_register_callbacks cb 0x7fa10000bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:35.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eee0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa117fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed02d0 0x555c8eee0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa11cb9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8ef46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eee0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39180/0 (socket says 172.21.15.161:39180) 2023-07-14T11:22:35.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 -- 172.21.15.161:0/89254598 learned_addr learned my addr 172.21.15.161:0/89254598 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:35.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 -- 172.21.15.161:0/89254598 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed02d0 msgr2=0x555c8eee0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed02d0 0x555c8eee0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 -- 172.21.15.161:0/89254598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 msgr2=0x555c8ef46a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:35.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8ef46a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.960+0000 7fa10ffff700 1 -- 172.21.15.161:0/89254598 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c8ec50de0 con 0x555c8eed0be0 2023-07-14T11:22:35.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa10ffff700 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eee0bc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa100009ce0 tx=0x7fa100009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6284aac55dbdd93e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:35.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa0f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:35.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa0f77fe700 1 -- 172.21.15.161:0/89254598 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000f8b0 con 0x555c8eed0be0 2023-07-14T11:22:35.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa10809a0b0 con 0x555c8eed0be0 2023-07-14T11:22:35.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa0f77fe700 1 -- 172.21.15.161:0/89254598 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa100000e40 con 0x555c8eed0be0 2023-07-14T11:22:35.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa0f77fe700 1 -- 172.21.15.161:0/89254598 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1000058e0 con 0x555c8eed0be0 2023-07-14T11:22:35.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa131ea4b80 10 client.15633 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:35.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555c8ec4f2b0 con 0x555c8eed0be0 2023-07-14T11:22:35.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.961+0000 7fa0f77fe700 1 -- 172.21.15.161:0/89254598 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10809a0b0 con 0x555c8eed0be0 2023-07-14T11:22:35.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa0f77fe700 1 -- 172.21.15.161:0/89254598 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1000099c0 con 0x555c8eed0be0 2023-07-14T11:22:35.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa131ea4b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:35.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa131ea4b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:35.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa131ea4b80 20 client.15633 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:35.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8ef48520 0x555c8ef4a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:35.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa131ea4b80 4 client.15633 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:22:35.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555c8edde050 con 0x555c8ef48520 2023-07-14T11:22:35.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa117fff700 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8ef48520 0x555c8ef4a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:35.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa117fff700 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8ef48520 0x555c8ef4a930 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f92383cb30c3d24 server_cookie=3e7c382920a63be4 in_seq=0 out_seq=0 2023-07-14T11:22:35.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.962+0000 7fa0f77fe700 10 client.15633 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:35.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa0f77fe700 1 -- 172.21.15.161:0/89254598 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555c8edde050 con 0x555c8ef48520 2023-07-14T11:22:35.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa0f77fe700 10 client.15633 handle_command_reply: tid=0 2023-07-14T11:22:35.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 client.15633 shutdown 2023-07-14T11:22:35.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 msgr2=0x555c8eee0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eee0bc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa100009ce0 tx=0x7fa100009510 comp rx=0 tx=0).stop 2023-07-14T11:22:35.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 shutdown_connections 2023-07-14T11:22:35.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555c8ef48520 0x555c8ef4a930 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8eedbad0 0x555c8ef46a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555c8eed02d0 0x555c8eee0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 --2- 172.21.15.161:0/89254598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8eed0be0 0x555c8eee0bc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 >> 172.21.15.161:0/89254598 conn(0x555c8eec6da0 msgr2=0x555c8eed3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:35.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa0f77fe700 0 client.15633 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:35.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 shutdown_connections 2023-07-14T11:22:35.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 1 -- 172.21.15.161:0/89254598 wait complete. 2023-07-14T11:22:35.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.963+0000 7fa131ea4b80 20 client.15633 trim_cache size 0 max 16384 2023-07-14T11:22:35.981 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:22:35.982 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:22:35.982 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:22:35.982 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:22:35.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1040a1350 msgr2=0x7fa1040a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1040a1350 0x7fa1040a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa110009fd0 tx=0x7fa110004b40 comp rx=0 tx=0).stop 2023-07-14T11:22:35.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 msgr2=0x7fa118197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:35.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 0x7fa118197e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa10800e9a0 tx=0x7fa10800ed60 comp rx=0 tx=0).stop 2023-07-14T11:22:35.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 shutdown_connections 2023-07-14T11:22:35.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1040a1350 0x7fa1040a3800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11810c3b0 0x7fa11819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa118072850 0x7fa118197e40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 --2- 172.21.15.161:0/3956916771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa118073160 0x7fa118198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:35.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 >> 172.21.15.161:0/3956916771 conn(0x7fa1180fb690 msgr2=0x7fa118103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:35.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 shutdown_connections 2023-07-14T11:22:35.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:35.964+0000 7fa11e5ff700 1 -- 172.21.15.161:0/3956916771 wait complete. 2023-07-14T11:22:35.985 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:22:35.986 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:22:36.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 -- 172.21.15.161:0/398003228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 msgr2=0x7f68ac105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 --2- 172.21.15.161:0/398003228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac105ed0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009990 tx=0x7f68a8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 -- 172.21.15.161:0/398003228 shutdown_connections 2023-07-14T11:22:36.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 --2- 172.21.15.161:0/398003228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac105ed0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 --2- 172.21.15.161:0/398003228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68ac1004a0 0x7f68ac10c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 --2- 172.21.15.161:0/398003228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68ac0ffaf0 0x7f68ac0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 -- 172.21.15.161:0/398003228 >> 172.21.15.161:0/398003228 conn(0x7f68ac0fb630 msgr2=0x7f68ac0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 -- 172.21.15.161:0/398003228 shutdown_connections 2023-07-14T11:22:36.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.184+0000 7f68b3fff700 1 -- 172.21.15.161:0/398003228 wait complete. 2023-07-14T11:22:36.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.185+0000 7f68b3fff700 1 Processor -- start 2023-07-14T11:22:36.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.185+0000 7f68b3fff700 1 -- start start 2023-07-14T11:22:36.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.185+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68ac0ffaf0 0x7f68ac197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68ac1004a0 0x7f68ac198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b3fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19cdc0 con 0x7f68ac105b00 2023-07-14T11:22:36.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b3fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19cf30 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b3fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19d0f0 con 0x7f68ac1004a0 2023-07-14T11:22:36.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b1d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68ac0ffaf0 0x7f68ac197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39202/0 (socket says 172.21.15.161:39202) 2023-07-14T11:22:36.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b259c700 1 -- 172.21.15.161:0/3756225411 learned_addr learned my addr 172.21.15.161:0/3756225411 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b159a700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68ac1004a0 0x7f68ac198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b1d9b700 1 -- 172.21.15.161:0/3756225411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68ac1004a0 msgr2=0x7f68ac198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b1d9b700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68ac1004a0 0x7f68ac198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b1d9b700 1 -- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 msgr2=0x7f68ac19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b1d9b700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b1d9b700 1 -- 172.21.15.161:0/3756225411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a80095f0 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.186+0000 7f68b259c700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:36.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.187+0000 7f68b1d9b700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68ac0ffaf0 0x7f68ac197de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68a80098e0 tx=0x7f68a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.187+0000 7f68a2ffd700 1 -- 172.21.15.161:0/3756225411 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a801d070 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.187+0000 7f68a2ffd700 1 -- 172.21.15.161:0/3756225411 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a8005030 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.187+0000 7f68a2ffd700 1 -- 172.21.15.161:0/3756225411 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8004ab0 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.187+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac19d370 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.187+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac1a4a30 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.188+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac067300 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.190+0000 7f68a2ffd700 1 -- 172.21.15.161:0/3756225411 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f68a8005410 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.191+0000 7f68a2ffd700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f68980a1390 0x7f68980a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.191+0000 7f68a2ffd700 1 -- 172.21.15.161:0/3756225411 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f68a80ce6b0 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.192+0000 7f68b159a700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f68980a1390 0x7f68980a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.192+0000 7f68b159a700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f68980a1390 0x7f68980a3840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68ac1993f0 tx=0x7f689c009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.193+0000 7f68a2ffd700 1 -- 172.21.15.161:0/3756225411 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f68a8097650 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:36 smithi175 ceph-mon[129081]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:22:36.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.436+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68ac199130 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.438+0000 7f68a2ffd700 1 -- 172.21.15.161:0/3756225411 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f68a80051a0 con 0x7f68ac0ffaf0 2023-07-14T11:22:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:22:36.442 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:22:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f68980a1390 msgr2=0x7f68980a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f68980a1390 0x7f68980a3840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68ac1993f0 tx=0x7f689c009260 comp rx=0 tx=0).stop 2023-07-14T11:22:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68ac0ffaf0 msgr2=0x7f68ac197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68ac0ffaf0 0x7f68ac197de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68a80098e0 tx=0x7f68a8004730 comp rx=0 tx=0).stop 2023-07-14T11:22:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 shutdown_connections 2023-07-14T11:22:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f68980a1390 0x7f68980a3840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ac105b00 0x7f68ac19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68ac0ffaf0 0x7f68ac197de0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 --2- 172.21.15.161:0/3756225411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f68ac1004a0 0x7f68ac198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 >> 172.21.15.161:0/3756225411 conn(0x7f68ac0fb630 msgr2=0x7f68ac10ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 shutdown_connections 2023-07-14T11:22:36.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.440+0000 7f68b3fff700 1 -- 172.21.15.161:0/3756225411 wait complete. 2023-07-14T11:22:36.446 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:22:36.461 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:36 smithi161 ceph-mon[95526]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:22:36.463 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:22:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:36 smithi182 ceph-mon[129366]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:22:36.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 -- 172.21.15.161:0/4085387744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8073160 msgr2=0x7f78f810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 --2- 172.21.15.161:0/4085387744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8073160 0x7f78f810be40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009990 tx=0x7f78ec009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 -- 172.21.15.161:0/4085387744 shutdown_connections 2023-07-14T11:22:36.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 --2- 172.21.15.161:0/4085387744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78f810c380 0x7f78f810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 --2- 172.21.15.161:0/4085387744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78f8072850 0x7f78f8072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 --2- 172.21.15.161:0/4085387744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8073160 0x7f78f810be40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 -- 172.21.15.161:0/4085387744 >> 172.21.15.161:0/4085387744 conn(0x7f78f80fb5e0 msgr2=0x7f78f80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 -- 172.21.15.161:0/4085387744 shutdown_connections 2023-07-14T11:22:36.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.663+0000 7f78fcbff700 1 -- 172.21.15.161:0/4085387744 wait complete. 2023-07-14T11:22:36.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.664+0000 7f78fcbff700 1 Processor -- start 2023-07-14T11:22:36.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.664+0000 7f78fcbff700 1 -- start start 2023-07-14T11:22:36.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78fcbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8072850 0x7f78f8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78fcbff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78f8073160 0x7f78f81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78fcbff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78f810c380 0x7f78f819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78fcbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f819ce50 con 0x7f78f8072850 2023-07-14T11:22:36.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78fcbff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f819cfc0 con 0x7f78f810c380 2023-07-14T11:22:36.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78fcbff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f819d180 con 0x7f78f8073160 2023-07-14T11:22:36.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8072850 0x7f78f8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8072850 0x7f78f8197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39220/0 (socket says 172.21.15.161:39220) 2023-07-14T11:22:36.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 -- 172.21.15.161:0/1297377791 learned_addr learned my addr 172.21.15.161:0/1297377791 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78f8073160 0x7f78f81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f6d9d700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78f810c380 0x7f78f819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 -- 172.21.15.161:0/1297377791 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78f8073160 msgr2=0x7f78f81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78f8073160 0x7f78f81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 -- 172.21.15.161:0/1297377791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78f810c380 msgr2=0x7f78f819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78f810c380 0x7f78f819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f659c700 1 -- 172.21.15.161:0/1297377791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec0095f0 con 0x7f78f8072850 2023-07-14T11:22:36.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.665+0000 7f78f6d9d700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78f810c380 0x7f78f819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:36.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.666+0000 7f78f659c700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8072850 0x7f78f8197e70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f78e800e9a0 tx=0x7f78e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.666+0000 7f78e77fe700 1 -- 172.21.15.161:0/1297377791 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8009790 con 0x7f78f8072850 2023-07-14T11:22:36.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.666+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f819d460 con 0x7f78f8072850 2023-07-14T11:22:36.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.666+0000 7f78e77fe700 1 -- 172.21.15.161:0/1297377791 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e800cbd0 con 0x7f78f8072850 2023-07-14T11:22:36.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.666+0000 7f78e77fe700 1 -- 172.21.15.161:0/1297377791 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8018680 con 0x7f78f8072850 2023-07-14T11:22:36.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.666+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f81a4ca0 con 0x7f78f8072850 2023-07-14T11:22:36.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.669+0000 7f78e77fe700 1 -- 172.21.15.161:0/1297377791 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f78e800cd40 con 0x7f78f8072850 2023-07-14T11:22:36.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.670+0000 7f7910466b80 1 Processor -- start 2023-07-14T11:22:36.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.670+0000 7f7910466b80 1 -- start start 2023-07-14T11:22:36.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.670+0000 7f78e77fe700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78e00a1350 0x7f78e00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f78e77fe700 1 -- 172.21.15.161:0/1297377791 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f78e8014070 con 0x7f78f8072850 2023-07-14T11:22:36.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f7910466b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a96c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f7910466b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96cbe0 0x55fc7a977590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f7910466b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a979eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f7910466b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a872cb0 con 0x55fc7a96c2d0 2023-07-14T11:22:36.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f7910466b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a8c5a40 con 0x55fc7a96cbe0 2023-07-14T11:22:36.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f7910466b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a8beed0 con 0x55fc7a977ad0 2023-07-14T11:22:36.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f78f6d9d700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78e00a1350 0x7f78e00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f78f659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a96c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f78f659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a96c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39224/0 (socket says 172.21.15.161:39224) 2023-07-14T11:22:36.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f78f659c700 1 -- 172.21.15.161:0/2556120886 learned_addr learned my addr 172.21.15.161:0/2556120886 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f78f6d9d700 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a979eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.671+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96cbe0 0x55fc7a977590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78f659c700 1 -- 172.21.15.161:0/2556120886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 msgr2=0x55fc7a979eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78f659c700 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a979eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78f659c700 1 -- 172.21.15.161:0/2556120886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96cbe0 msgr2=0x55fc7a977590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78f659c700 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96cbe0 0x55fc7a977590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78f659c700 1 -- 172.21.15.161:0/2556120886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc7a6ecde0 con 0x55fc7a96c2d0 2023-07-14T11:22:36.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78f6d9d700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78e00a1350 0x7f78e00a3800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009790 tx=0x7f78f0007a80 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78f659c700 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a96c6a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f78e8099d70 tx=0x7f78e8099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e426a1e9b819f30 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78e4ff9700 1 -- 172.21.15.161:0/2556120886 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e809a070 con 0x55fc7a96c2d0 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78e4ff9700 1 -- 172.21.15.161:0/2556120886 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e809e0a0 con 0x55fc7a96c2d0 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.672+0000 7f78e4ff9700 1 -- 172.21.15.161:0/2556120886 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8099780 con 0x55fc7a96c2d0 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 -- 172.21.15.161:0/2556120886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 msgr2=0x55fc7a96c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a96c6a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f78e8099d70 tx=0x7f78e8099f70 comp rx=0 tx=0).stop 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 -- 172.21.15.161:0/2556120886 shutdown_connections 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a96c6a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a979eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 --2- 172.21.15.161:0/2556120886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96cbe0 0x55fc7a977590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 -- 172.21.15.161:0/2556120886 >> 172.21.15.161:0/2556120886 conn(0x55fc7a962da0 msgr2=0x55fc7a9686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 -- 172.21.15.161:0/2556120886 shutdown_connections 2023-07-14T11:22:36.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 -- 172.21.15.161:0/2556120886 wait complete. 2023-07-14T11:22:36.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.673+0000 7f7910466b80 1 -- start start 2023-07-14T11:22:36.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.674+0000 7f7910466b80 1 Processor -- start 2023-07-14T11:22:36.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f7910466b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f7910466b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96cbe0 0x55fc7a974800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f7910466b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f7910466b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a85eac0 con 0x55fc7a96c2d0 2023-07-14T11:22:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f7910466b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a85e7c0 con 0x55fc7a977ad0 2023-07-14T11:22:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f7910466b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a742920 con 0x55fc7a96cbe0 2023-07-14T11:22:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f7910466b80 10 client.0 _ll_register_callbacks cb 0x55fc7a85c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39228/0 (socket says 172.21.15.161:39228) 2023-07-14T11:22:36.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f659c700 1 -- 172.21.15.161:0/1220243016 learned_addr learned my addr 172.21.15.161:0/1220243016 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96cbe0 0x55fc7a974800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f6d9d700 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f5d9b700 1 -- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 msgr2=0x55fc7a974f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f5d9b700 1 -- 172.21.15.161:0/1220243016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 msgr2=0x55fc7a977580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.675+0000 7f78f5d9b700 1 -- 172.21.15.161:0/1220243016 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f81a4ca0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96cbe0 0x55fc7a974800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00b2e0 tx=0x7f78ec00b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e274709d0e4787 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f78ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:22:36.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f78ddffb700 1 -- 172.21.15.161:0/1220243016 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec01d070 con 0x55fc7a96cbe0 2023-07-14T11:22:36.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc7a8184f0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f78ddffb700 1 -- 172.21.15.161:0/1220243016 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f78ec004500 con 0x55fc7a96cbe0 2023-07-14T11:22:36.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f78ddffb700 1 -- 172.21.15.161:0/1220243016 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78ec0046c0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f7910466b80 10 client.25690 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:36.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55fc7a6d5c60 con 0x55fc7a96cbe0 2023-07-14T11:22:36.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.676+0000 7f78ddffb700 1 -- 172.21.15.161:0/1220243016 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc7a8184f0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f78ddffb700 1 -- 172.21.15.161:0/1220243016 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f78ec00f950 con 0x55fc7a96cbe0 2023-07-14T11:22:36.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f7910466b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:36.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f7910466b80 10 client.25690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:22:36.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f7910466b80 20 client.25690 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:36.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f7910466b80 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a987440 0x55fc7a989850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f7910466b80 4 client.25690 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:36.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc7a8b7eb0 con 0x55fc7a987440 2023-07-14T11:22:36.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f78f659c700 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a987440 0x55fc7a989850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f78ddffb700 10 client.25690 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:22:36.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.677+0000 7f78f659c700 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a987440 0x55fc7a989850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e284e7937cff0be server_cookie=40806b8adc283e4b in_seq=0 out_seq=0 2023-07-14T11:22:36.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f78ddffb700 1 -- 172.21.15.161:0/1220243016 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc7a8b7eb0 con 0x55fc7a987440 2023-07-14T11:22:36.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f78ddffb700 10 client.25690 handle_command_reply: tid=0 2023-07-14T11:22:36.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 client.25690 shutdown 2023-07-14T11:22:36.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96cbe0 msgr2=0x55fc7a974800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96cbe0 0x55fc7a974800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00b2e0 tx=0x7f78ec00b650 comp rx=0 tx=0).stop 2023-07-14T11:22:36.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 shutdown_connections 2023-07-14T11:22:36.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a987440 0x55fc7a989850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 --2- 172.21.15.161:0/1220243016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96cbe0 0x55fc7a974800 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f78ddffb700 0 client.25690 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:22:36.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.678+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 >> 172.21.15.161:0/1220243016 conn(0x55fc7a962da0 msgr2=0x55fc7a968230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.679+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 shutdown_connections 2023-07-14T11:22:36.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.679+0000 7f7910466b80 1 -- 172.21.15.161:0/1220243016 wait complete. 2023-07-14T11:22:36.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.679+0000 7f7910466b80 20 client.25690 trim_cache size 0 max 16384 2023-07-14T11:22:36.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.692+0000 7f7910466b80 1 Processor -- start 2023-07-14T11:22:36.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.692+0000 7f7910466b80 1 -- start start 2023-07-14T11:22:36.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f7910466b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f7910466b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a973fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f7910466b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f7910466b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a96f6a0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f7910466b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a96d540 con 0x55fc7a977ad0 2023-07-14T11:22:36.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f7910466b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a969c70 con 0x55fc7a96c2d0 2023-07-14T11:22:36.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a973fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a973fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39244/0 (socket says 172.21.15.161:39244) 2023-07-14T11:22:36.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f5d9b700 1 -- 172.21.15.161:0/878771744 learned_addr learned my addr 172.21.15.161:0/878771744 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f6d9d700 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f659c700 1 -- 172.21.15.161:0/878771744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 msgr2=0x55fc7a974500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f659c700 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f659c700 1 -- 172.21.15.161:0/878771744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 msgr2=0x55fc7a973fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f659c700 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a973fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.693+0000 7f78f659c700 1 -- 172.21.15.161:0/878771744 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc7a713250 con 0x55fc7a96c2d0 2023-07-14T11:22:36.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f78f659c700 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78e80a1f90 tx=0x7f78e8010b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be510eb7fdf93998 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f78e57fa700 1 -- 172.21.15.161:0/878771744 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80ca6e0 con 0x55fc7a96c2d0 2023-07-14T11:22:36.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f78e57fa700 1 -- 172.21.15.161:0/878771744 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e8098410 con 0x55fc7a96c2d0 2023-07-14T11:22:36.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f78e57fa700 1 -- 172.21.15.161:0/878771744 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8099830 con 0x55fc7a96c2d0 2023-07-14T11:22:36.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 -- 172.21.15.161:0/878771744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96c2d0 msgr2=0x55fc7a977120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78e80a1f90 tx=0x7f78e8010b20 comp rx=0 tx=0).stop 2023-07-14T11:22:36.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 -- 172.21.15.161:0/878771744 shutdown_connections 2023-07-14T11:22:36.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a977ad0 0x55fc7a974500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a96c2d0 0x55fc7a977120 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 --2- 172.21.15.161:0/878771744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a973fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 -- 172.21.15.161:0/878771744 >> 172.21.15.161:0/878771744 conn(0x55fc7a962da0 msgr2=0x55fc7a983820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.694+0000 7f7910466b80 1 -- 172.21.15.161:0/878771744 shutdown_connections 2023-07-14T11:22:36.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.695+0000 7f7910466b80 1 -- 172.21.15.161:0/878771744 wait complete. 2023-07-14T11:22:36.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.695+0000 7f7910466b80 1 -- start start 2023-07-14T11:22:36.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.695+0000 7f7910466b80 1 Processor -- start 2023-07-14T11:22:36.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f7910466b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96c2d0 0x55fc7a97e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f7910466b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a97c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f7910466b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a97cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f7910466b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a97d170 con 0x55fc7a96cbe0 2023-07-14T11:22:36.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f7910466b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a97d2b0 con 0x55fc7a96c2d0 2023-07-14T11:22:36.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f7910466b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7a97d470 con 0x55fc7a977ad0 2023-07-14T11:22:36.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f7910466b80 10 client.0 _ll_register_callbacks cb 0x55fc7a85c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:36.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f78f5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a97c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f78f6d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a97cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f78f659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96c2d0 0x55fc7a97e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f78f5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a97c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39250/0 (socket says 172.21.15.161:39250) 2023-07-14T11:22:36.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.696+0000 7f78f5d9b700 1 -- 172.21.15.161:0/3797722360 learned_addr learned my addr 172.21.15.161:0/3797722360 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78f5d9b700 1 -- 172.21.15.161:0/3797722360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 msgr2=0x55fc7a97cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a97cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78f5d9b700 1 -- 172.21.15.161:0/3797722360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96c2d0 msgr2=0x55fc7a97e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96c2d0 0x55fc7a97e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78f5d9b700 1 -- 172.21.15.161:0/3797722360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc7a6ecde0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78f5d9b700 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a97c6b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009990 tx=0x7f78ec005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fa4465795a1618 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:36.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78ddffb700 1 -- 172.21.15.161:0/3797722360 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec01d050 con 0x55fc7a96cbe0 2023-07-14T11:22:36.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78e809e0a0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.697+0000 7f78ddffb700 1 -- 172.21.15.161:0/3797722360 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f78ec00f460 con 0x55fc7a96cbe0 2023-07-14T11:22:36.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f78ddffb700 1 -- 172.21.15.161:0/3797722360 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78ec00f620 con 0x55fc7a96cbe0 2023-07-14T11:22:36.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 10 client.15657 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:36.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55fc7a6eb2b0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f78ddffb700 1 -- 172.21.15.161:0/3797722360 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e809e0a0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f78ddffb700 1 -- 172.21.15.161:0/3797722360 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f78ec023be0 con 0x55fc7a96cbe0 2023-07-14T11:22:36.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:36.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 10 client.15657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:22:36.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 20 client.15657 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:36.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a9dc3e0 0x55fc7a9de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 4 client.15657 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:22:36.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc7a87a050 con 0x55fc7a9dc3e0 2023-07-14T11:22:36.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.698+0000 7f78f659c700 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a9dc3e0 0x55fc7a9de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f78f659c700 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a9dc3e0 0x55fc7a9de7f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5503c4e980de14b server_cookie=3749ef7450e89130 in_seq=0 out_seq=0 2023-07-14T11:22:36.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f78ddffb700 10 client.15657 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:22:36.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f78ddffb700 1 -- 172.21.15.161:0/3797722360 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc7a87a050 con 0x55fc7a9dc3e0 2023-07-14T11:22:36.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f78ddffb700 10 client.15657 handle_command_reply: tid=0 2023-07-14T11:22:36.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 client.15657 shutdown 2023-07-14T11:22:36.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 msgr2=0x55fc7a97c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a97c6b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009990 tx=0x7f78ec005f10 comp rx=0 tx=0).stop 2023-07-14T11:22:36.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 shutdown_connections 2023-07-14T11:22:36.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55fc7a9dc3e0 0x55fc7a9de7f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fc7a977ad0 0x55fc7a97cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc7a96c2d0 0x55fc7a97e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 --2- 172.21.15.161:0/3797722360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc7a96cbe0 0x55fc7a97c6b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 >> 172.21.15.161:0/3797722360 conn(0x55fc7a962da0 msgr2=0x55fc7a96e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.699+0000 7f78ddffb700 0 client.15657 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:22:36.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 shutdown_connections 2023-07-14T11:22:36.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f7910466b80 1 -- 172.21.15.161:0/3797722360 wait complete. 2023-07-14T11:22:36.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f7910466b80 20 client.15657 trim_cache size 0 max 16384 2023-07-14T11:22:36.714 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:22:36.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78e00a1350 msgr2=0x7f78e00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78e00a1350 0x7f78e00a3800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009790 tx=0x7f78f0007a80 comp rx=0 tx=0).stop 2023-07-14T11:22:36.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8072850 msgr2=0x7f78f8197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8072850 0x7f78f8197e70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f78e800e9a0 tx=0x7f78e800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 shutdown_connections 2023-07-14T11:22:36.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78e00a1350 0x7f78e00a3800 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78f8072850 0x7f78f8197e70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78f810c380 0x7f78f819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 --2- 172.21.15.161:0/1297377791 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78f8073160 0x7f78f81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 >> 172.21.15.161:0/1297377791 conn(0x7f78f80fb5e0 msgr2=0x7f78f810e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.700+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 shutdown_connections 2023-07-14T11:22:36.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.701+0000 7f78fcbff700 1 -- 172.21.15.161:0/1297377791 wait complete. 2023-07-14T11:22:36.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:22:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 -- 172.21.15.161:0/3066718837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c0ffb00 msgr2=0x7fa12c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 --2- 172.21.15.161:0/3066718837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c0ffb00 0x7fa12c0fff70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa120009990 tx=0x7fa120009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 -- 172.21.15.161:0/3066718837 shutdown_connections 2023-07-14T11:22:36.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 --2- 172.21.15.161:0/3066718837 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa12c1004b0 0x7fa12c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 --2- 172.21.15.161:0/3066718837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c105b10 0x7fa12c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 --2- 172.21.15.161:0/3066718837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c0ffb00 0x7fa12c0fff70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 -- 172.21.15.161:0/3066718837 >> 172.21.15.161:0/3066718837 conn(0x7fa12c0fb600 msgr2=0x7fa12c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.931+0000 7fa130dff700 1 -- 172.21.15.161:0/3066718837 shutdown_connections 2023-07-14T11:22:36.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.932+0000 7fa130dff700 1 -- 172.21.15.161:0/3066718837 wait complete. 2023-07-14T11:22:36.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.932+0000 7fa130dff700 1 Processor -- start 2023-07-14T11:22:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa130dff700 1 -- start start 2023-07-14T11:22:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa130dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0ffb00 0x7fa12c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa130dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c1004b0 0x7fa12c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa130dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa12c105b10 0x7fa12c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa130dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c19ce70 con 0x7fa12c1004b0 2023-07-14T11:22:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa130dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c19cfe0 con 0x7fa12c0ffb00 2023-07-14T11:22:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa130dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c19d1a0 con 0x7fa12c105b10 2023-07-14T11:22:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c1004b0 0x7fa12c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c1004b0 0x7fa12c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39254/0 (socket says 172.21.15.161:39254) 2023-07-14T11:22:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa129d9b700 1 -- 172.21.15.161:0/2683515239 learned_addr learned my addr 172.21.15.161:0/2683515239 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa12a59c700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0ffb00 0x7fa12c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.933+0000 7fa12ad9d700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa12c105b10 0x7fa12c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa129d9b700 1 -- 172.21.15.161:0/2683515239 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa12c105b10 msgr2=0x7fa12c19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa129d9b700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa12c105b10 0x7fa12c19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa129d9b700 1 -- 172.21.15.161:0/2683515239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0ffb00 msgr2=0x7fa12c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa129d9b700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0ffb00 0x7fa12c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa129d9b700 1 -- 172.21.15.161:0/2683515239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200095f0 con 0x7fa12c1004b0 2023-07-14T11:22:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa129d9b700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c1004b0 0x7fa12c1983d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa120000c00 tx=0x7fa120005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa11b7fe700 1 -- 172.21.15.161:0/2683515239 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12001d070 con 0x7fa12c1004b0 2023-07-14T11:22:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c19d420 con 0x7fa12c1004b0 2023-07-14T11:22:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa11b7fe700 1 -- 172.21.15.161:0/2683515239 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa120022470 con 0x7fa12c1004b0 2023-07-14T11:22:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.934+0000 7fa11b7fe700 1 -- 172.21.15.161:0/2683515239 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120017650 con 0x7fa12c1004b0 2023-07-14T11:22:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.935+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c1a4cc0 con 0x7fa12c1004b0 2023-07-14T11:22:36.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.938+0000 7fa11b7fe700 1 -- 172.21.15.161:0/2683515239 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7fa1200225e0 con 0x7fa12c1004b0 2023-07-14T11:22:36.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.939+0000 7fa11b7fe700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1140a13d0 0x7fa1140a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.939+0000 7fa11b7fe700 1 -- 172.21.15.161:0/2683515239 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa1200ce4c0 con 0x7fa12c1004b0 2023-07-14T11:22:36.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.939+0000 7fa12a59c700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1140a13d0 0x7fa1140a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 Processor -- start 2023-07-14T11:22:36.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 -- start start 2023-07-14T11:22:36.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625a2d0 0x55959625a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x559596265590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559596265ad0 0x559596267eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596160cb0 con 0x55959625a2d0 2023-07-14T11:22:36.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595961b3a40 con 0x55959625abe0 2023-07-14T11:22:36.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa144786b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595961aced0 con 0x559596265ad0 2023-07-14T11:22:36.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa12a59c700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1140a13d0 0x7fa1140a3880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009be0 tx=0x7fa11c009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa12a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625a2d0 0x55959625a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x559596265590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa12a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625a2d0 0x55959625a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39262/0 (socket says 172.21.15.161:39262) 2023-07-14T11:22:36.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.940+0000 7fa12a59c700 1 -- 172.21.15.161:0/303272720 learned_addr learned my addr 172.21.15.161:0/303272720 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa12ad9d700 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559596265ad0 0x559596267eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa129d9b700 1 -- 172.21.15.161:0/303272720 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559596265ad0 msgr2=0x559596267eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa129d9b700 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559596265ad0 0x559596267eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa129d9b700 1 -- 172.21.15.161:0/303272720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625a2d0 msgr2=0x55959625a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa129d9b700 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625a2d0 0x55959625a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa129d9b700 1 -- 172.21.15.161:0/303272720 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559595fdade0 con 0x55959625abe0 2023-07-14T11:22:36.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa12a59c700 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625a2d0 0x55959625a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:36.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa129d9b700 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x559596265590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1200cb820 tx=0x7fa12009acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b812ef76e605e767 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa118ff9700 1 -- 172.21.15.161:0/303272720 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12009b180 con 0x55959625abe0 2023-07-14T11:22:36.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa118ff9700 1 -- 172.21.15.161:0/303272720 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12009f1b0 con 0x55959625abe0 2023-07-14T11:22:36.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.941+0000 7fa118ff9700 1 -- 172.21.15.161:0/303272720 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12009b3e0 con 0x55959625abe0 2023-07-14T11:22:36.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 -- 172.21.15.161:0/303272720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 msgr2=0x559596265590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x559596265590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1200cb820 tx=0x7fa12009acc0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 -- 172.21.15.161:0/303272720 shutdown_connections 2023-07-14T11:22:36.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625a2d0 0x55959625a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559596265ad0 0x559596267eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 --2- 172.21.15.161:0/303272720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x559596265590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 -- 172.21.15.161:0/303272720 >> 172.21.15.161:0/303272720 conn(0x559596250da0 msgr2=0x5595962566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 -- 172.21.15.161:0/303272720 shutdown_connections 2023-07-14T11:22:36.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.942+0000 7fa144786b80 1 -- 172.21.15.161:0/303272720 wait complete. 2023-07-14T11:22:36.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.943+0000 7fa144786b80 1 -- start start 2023-07-14T11:22:36.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.943+0000 7fa144786b80 1 Processor -- start 2023-07-14T11:22:36.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa144786b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x55959626ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa144786b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa144786b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x5595962649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa144786b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959614cac0 con 0x55959625abe0 2023-07-14T11:22:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa144786b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959614c7c0 con 0x559596265ad0 2023-07-14T11:22:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa144786b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596030920 con 0x55959625a2d0 2023-07-14T11:22:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa144786b80 10 client.0 _ll_register_callbacks cb 0x55959614a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39278/0 (socket says 172.21.15.161:39278) 2023-07-14T11:22:36.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 -- 172.21.15.161:0/1187469937 learned_addr learned my addr 172.21.15.161:0/1187469937 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa12a59c700 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x55959626ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa12ad9d700 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x5595962649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 -- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 msgr2=0x55959626ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x55959626ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 -- 172.21.15.161:0/1187469937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 msgr2=0x5595962649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x5595962649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.944+0000 7fa129d9b700 1 -- 172.21.15.161:0/1187469937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c017070 con 0x55959625abe0 2023-07-14T11:22:36.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa129d9b700 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa1200cd040 tx=0x7fa1200cb760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e9d5f73218fe6b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa111ffb700 1 -- 172.21.15.161:0/1187469937 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200cbce0 con 0x55959625abe0 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa111ffb700 1 -- 172.21.15.161:0/1187469937 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa1200a3540 con 0x55959625abe0 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595961064f0 con 0x55959625abe0 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa111ffb700 1 -- 172.21.15.161:0/1187469937 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1200baea0 con 0x55959625abe0 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa144786b80 10 client.15669 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7fa12c1a4cc0 con 0x55959625abe0 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.945+0000 7fa111ffb700 1 -- 172.21.15.161:0/1187469937 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1200be020 con 0x55959625abe0 2023-07-14T11:22:36.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa111ffb700 1 -- 172.21.15.161:0/1187469937 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1200a30b0 con 0x55959625abe0 2023-07-14T11:22:36.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa144786b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:36.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa144786b80 10 client.15669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:22:36.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa144786b80 20 client.15669 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:36.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa144786b80 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x559596274c00 0x559596277010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa144786b80 4 client.15669 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:36.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595961a5eb0 con 0x559596274c00 2023-07-14T11:22:36.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.946+0000 7fa12a59c700 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x559596274c00 0x559596277010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.947+0000 7fa111ffb700 10 client.15669 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:22:36.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.947+0000 7fa12a59c700 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x559596274c00 0x559596277010 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0b288d64fa4c87a server_cookie=5c33f57c3da510aa in_seq=0 out_seq=0 2023-07-14T11:22:36.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.947+0000 7fa111ffb700 1 -- 172.21.15.161:0/1187469937 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5595961a5eb0 con 0x559596274c00 2023-07-14T11:22:36.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.947+0000 7fa111ffb700 10 client.15669 handle_command_reply: tid=0 2023-07-14T11:22:36.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.947+0000 7fa144786b80 1 client.15669 shutdown 2023-07-14T11:22:36.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 msgr2=0x55959626b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa1200cd040 tx=0x7fa1200cb760 comp rx=0 tx=0).stop 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 shutdown_connections 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x559596274c00 0x559596277010 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x5595962649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x55959626ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 --2- 172.21.15.161:0/1187469937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626b0a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa111ffb700 0 client.15669 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 >> 172.21.15.161:0/1187469937 conn(0x559596250da0 msgr2=0x559596256230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 shutdown_connections 2023-07-14T11:22:36.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 1 -- 172.21.15.161:0/1187469937 wait complete. 2023-07-14T11:22:36.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.948+0000 7fa144786b80 20 client.15669 trim_cache size 0 max 16384 2023-07-14T11:22:36.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 Processor -- start 2023-07-14T11:22:36.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 -- start start 2023-07-14T11:22:36.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596277010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x55959627b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559596265ad0 0x55959625f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596271950 con 0x559596265ad0 2023-07-14T11:22:36.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959627beb0 con 0x55959625abe0 2023-07-14T11:22:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.962+0000 7fa144786b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959625f700 con 0x55959625a2d0 2023-07-14T11:22:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596277010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559596265ad0 0x55959625f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559596265ad0 0x55959625f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39292/0 (socket says 172.21.15.161:39292) 2023-07-14T11:22:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12ad9d700 1 -- 172.21.15.161:0/1691675070 learned_addr learned my addr 172.21.15.161:0/1691675070 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x55959627b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596277010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45624/0 (socket says 172.21.15.161:45624) 2023-07-14T11:22:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 -- 172.21.15.161:0/1691675070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 msgr2=0x55959627b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x55959627b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 -- 172.21.15.161:0/1691675070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559596265ad0 msgr2=0x55959625f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559596265ad0 0x55959625f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 -- 172.21.15.161:0/1691675070 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559595fc3c60 con 0x55959625a2d0 2023-07-14T11:22:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12ad9d700 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559596265ad0 0x55959625f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa12a59c700 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596277010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5595962778c0 tx=0x7fa11c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66fa7f8bf3af3458 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.963+0000 7fa1197fa700 1 -- 172.21.15.161:0/1691675070 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c024040 con 0x55959625a2d0 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa1197fa700 1 -- 172.21.15.161:0/1691675070 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa11c020190 con 0x55959625a2d0 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa1197fa700 1 -- 172.21.15.161:0/1691675070 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c029e90 con 0x55959625a2d0 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 -- 172.21.15.161:0/1691675070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 msgr2=0x559596277010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596277010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5595962778c0 tx=0x7fa11c014000 comp rx=0 tx=0).stop 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 -- 172.21.15.161:0/1691675070 shutdown_connections 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559596265ad0 0x55959625f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596277010 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 --2- 172.21.15.161:0/1691675070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55959625abe0 0x55959627b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 -- 172.21.15.161:0/1691675070 >> 172.21.15.161:0/1691675070 conn(0x559596250da0 msgr2=0x559596252310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 -- 172.21.15.161:0/1691675070 shutdown_connections 2023-07-14T11:22:36.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.964+0000 7fa144786b80 1 -- 172.21.15.161:0/1691675070 wait complete. 2023-07-14T11:22:36.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.965+0000 7fa144786b80 1 -- start start 2023-07-14T11:22:36.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.965+0000 7fa144786b80 1 Processor -- start 2023-07-14T11:22:36.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.965+0000 7fa144786b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596270f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa144786b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa144786b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x55959626f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa144786b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959626fca0 con 0x55959625abe0 2023-07-14T11:22:36.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa144786b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959626fe10 con 0x559596265ad0 2023-07-14T11:22:36.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa144786b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959626ffd0 con 0x55959625a2d0 2023-07-14T11:22:36.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa144786b80 10 client.0 _ll_register_callbacks cb 0x7fa11c0119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:36.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa12a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596270f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39302/0 (socket says 172.21.15.161:39302) 2023-07-14T11:22:36.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 -- 172.21.15.161:0/10730646 learned_addr learned my addr 172.21.15.161:0/10730646 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:36.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa12a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596270f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45630/0 (socket says 172.21.15.161:45630) 2023-07-14T11:22:36.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa12ad9d700 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x55959626f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 -- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 msgr2=0x559596270f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596270f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 -- 172.21.15.161:0/10730646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 msgr2=0x55959626f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:36.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x55959626f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa129d9b700 1 -- 172.21.15.161:0/10730646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1240116b0 con 0x55959625abe0 2023-07-14T11:22:36.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.966+0000 7fa12a59c700 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596270f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:36.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa129d9b700 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626f220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa12000b2e0 tx=0x7fa120099730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb393e2ae996efb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:36.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:36.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa111ffb700 1 -- 172.21.15.161:0/10730646 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200baea0 con 0x55959625abe0 2023-07-14T11:22:36.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa12009f1b0 con 0x55959625abe0 2023-07-14T11:22:36.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa111ffb700 1 -- 172.21.15.161:0/10730646 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa1200b3e40 con 0x55959625abe0 2023-07-14T11:22:36.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa111ffb700 1 -- 172.21.15.161:0/10730646 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1200b3ce0 con 0x55959625abe0 2023-07-14T11:22:36.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa144786b80 10 client.15675 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:36.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7fa11c011b70 con 0x55959625abe0 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.967+0000 7fa111ffb700 1 -- 172.21.15.161:0/10730646 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12009f1b0 con 0x55959625abe0 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa111ffb700 1 -- 172.21.15.161:0/10730646 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa12009a670 con 0x55959625abe0 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa144786b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa144786b80 10 client.15675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa144786b80 20 client.15675 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa144786b80 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5595962c9a00 0x5595962cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa144786b80 4 client.15675 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:22:36.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559596168050 con 0x5595962c9a00 2023-07-14T11:22:36.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.968+0000 7fa12a59c700 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5595962c9a00 0x5595962cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:36.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa111ffb700 10 client.15675 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:22:36.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa12a59c700 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5595962c9a00 0x5595962cbe10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e364ff9a6a183ea server_cookie=e4d23e23ba95dd9c in_seq=0 out_seq=0 2023-07-14T11:22:36.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa111ffb700 1 -- 172.21.15.161:0/10730646 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559596168050 con 0x5595962c9a00 2023-07-14T11:22:36.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa111ffb700 10 client.15675 handle_command_reply: tid=0 2023-07-14T11:22:36.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 client.15675 shutdown 2023-07-14T11:22:36.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 msgr2=0x55959626f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626f220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa12000b2e0 tx=0x7fa120099730 comp rx=0 tx=0).stop 2023-07-14T11:22:36.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 shutdown_connections 2023-07-14T11:22:36.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5595962c9a00 0x5595962cbe10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559596265ad0 0x55959626f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55959625a2d0 0x559596270f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 --2- 172.21.15.161:0/10730646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55959625abe0 0x55959626f220 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 >> 172.21.15.161:0/10730646 conn(0x559596250da0 msgr2=0x55959626a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa111ffb700 0 client.15675 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:22:36.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 shutdown_connections 2023-07-14T11:22:36.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 1 -- 172.21.15.161:0/10730646 wait complete. 2023-07-14T11:22:36.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.969+0000 7fa144786b80 20 client.15675 trim_cache size 0 max 16384 2023-07-14T11:22:36.985 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:22:36.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1140a13d0 msgr2=0x7fa1140a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1140a13d0 0x7fa1140a3880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009be0 tx=0x7fa11c009260 comp rx=0 tx=0).stop 2023-07-14T11:22:36.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c1004b0 msgr2=0x7fa12c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:36.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c1004b0 0x7fa12c1983d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa120000c00 tx=0x7fa120005740 comp rx=0 tx=0).stop 2023-07-14T11:22:36.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 shutdown_connections 2023-07-14T11:22:36.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1140a13d0 0x7fa1140a3880 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa12c105b10 0x7fa12c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa12c0ffb00 0x7fa12c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 --2- 172.21.15.161:0/2683515239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c1004b0 0x7fa12c1983d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:36.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 >> 172.21.15.161:0/2683515239 conn(0x7fa12c0fb600 msgr2=0x7fa12c072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:36.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 shutdown_connections 2023-07-14T11:22:36.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:36.970+0000 7fa130dff700 1 -- 172.21.15.161:0/2683515239 wait complete. 2023-07-14T11:22:36.999 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:22:37.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 -- 172.21.15.161:0/3446108064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 msgr2=0x7f2b8c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3446108064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c105eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009990 tx=0x7f2b7c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 -- 172.21.15.161:0/3446108064 shutdown_connections 2023-07-14T11:22:37.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3446108064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c105eb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3446108064 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b8c100480 0x7f2b8c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3446108064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8c0ffad0 0x7f2b8c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 -- 172.21.15.161:0/3446108064 >> 172.21.15.161:0/3446108064 conn(0x7f2b8c0fb5f0 msgr2=0x7f2b8c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:37.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 -- 172.21.15.161:0/3446108064 shutdown_connections 2023-07-14T11:22:37.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.213+0000 7f2b90bff700 1 -- 172.21.15.161:0/3446108064 wait complete. 2023-07-14T11:22:37.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.214+0000 7f2b90bff700 1 Processor -- start 2023-07-14T11:22:37.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.214+0000 7f2b90bff700 1 -- start start 2023-07-14T11:22:37.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b90bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b8c0ffad0 0x7f2b8c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b90bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8c100480 0x7f2b8c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b90bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c1a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b90bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c073580 con 0x7f2b8c105ae0 2023-07-14T11:22:37.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b90bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1a4300 con 0x7f2b8c100480 2023-07-14T11:22:37.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b90bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1a4440 con 0x7f2b8c0ffad0 2023-07-14T11:22:37.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c1a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c1a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39318/0 (socket says 172.21.15.161:39318) 2023-07-14T11:22:37.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/3615379686 learned_addr learned my addr 172.21.15.161:0/3615379686 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:37.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b8c0ffad0 0x7f2b8c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8c100480 0x7f2b8c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/3615379686 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b8c0ffad0 msgr2=0x7f2b8c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b8c0ffad0 0x7f2b8c072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/3615379686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8c100480 msgr2=0x7f2b8c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8c100480 0x7f2b8c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.215+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/3615379686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0095f0 con 0x7f2b8c105ae0 2023-07-14T11:22:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8c100480 0x7f2b8c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:22:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b8c0ffad0 0x7f2b8c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c1a3dc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400ef30 tx=0x7f2b8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b7b7fe700 1 -- 172.21.15.161:0/3615379686 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84018070 con 0x7f2b8c105ae0 2023-07-14T11:22:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c1a4640 con 0x7f2b8c105ae0 2023-07-14T11:22:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b7b7fe700 1 -- 172.21.15.161:0/3615379686 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b8400f040 con 0x7f2b8c105ae0 2023-07-14T11:22:37.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b7b7fe700 1 -- 172.21.15.161:0/3615379686 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8400cdb0 con 0x7f2b8c105ae0 2023-07-14T11:22:37.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.216+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c1a4b70 con 0x7f2b8c105ae0 2023-07-14T11:22:37.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.220+0000 7f2b7b7fe700 1 -- 172.21.15.161:0/3615379686 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f2b84003a00 con 0x7f2b8c105ae0 2023-07-14T11:22:37.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 Processor -- start 2023-07-14T11:22:37.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 -- start start 2023-07-14T11:22:37.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2b7b7fe700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b740a13d0 0x7f2b740a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4ae46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4af1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec49eacb0 con 0x562ec4aefad0 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4a3da40 con 0x562ec4ae4be0 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4a36ed0 con 0x562ec4ae42d0 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.221+0000 7f2b7b7fe700 1 -- 172.21.15.161:0/3615379686 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f2b840ccd50 con 0x7f2b8c105ae0 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b740a13d0 0x7f2b740a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4af1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4ae46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4af1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39334/0 (socket says 172.21.15.161:39334) 2023-07-14T11:22:37.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/2977776433 learned_addr learned my addr 172.21.15.161:0/2977776433 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:37.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/2977776433 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 msgr2=0x562ec4ae46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4ae46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/2977776433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 msgr2=0x562ec4aef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/2977776433 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec4864de0 con 0x562ec4aefad0 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.222+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4ae46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.223+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b740a13d0 0x7f2b740a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c074000 tx=0x7f2b80009b80 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.223+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4af1eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b840a1840 tx=0x7f2b84099800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c230b9e911624aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.223+0000 7f2b78ff9700 1 -- 172.21.15.161:0/2977776433 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84004ad0 con 0x562ec4aefad0 2023-07-14T11:22:37.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.223+0000 7f2b78ff9700 1 -- 172.21.15.161:0/2977776433 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b840999b0 con 0x562ec4aefad0 2023-07-14T11:22:37.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.223+0000 7f2b78ff9700 1 -- 172.21.15.161:0/2977776433 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840ca5a0 con 0x562ec4aefad0 2023-07-14T11:22:37.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2977776433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 msgr2=0x562ec4af1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4af1eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b840a1840 tx=0x7f2b84099800 comp rx=0 tx=0).stop 2023-07-14T11:22:37.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2977776433 shutdown_connections 2023-07-14T11:22:37.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4af1eb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4ae46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2977776433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2977776433 >> 172.21.15.161:0/2977776433 conn(0x562ec4adada0 msgr2=0x562ec4ae06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:37.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2977776433 shutdown_connections 2023-07-14T11:22:37.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2977776433 wait complete. 2023-07-14T11:22:37.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 -- start start 2023-07-14T11:22:37.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.224+0000 7f2ba457ab80 1 Processor -- start 2023-07-14T11:22:37.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae42d0 0x562ec4aef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4aefad0 0x562ec4aecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec49d6ac0 con 0x562ec4ae42d0 2023-07-14T11:22:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec49d67c0 con 0x562ec4ae4be0 2023-07-14T11:22:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec48ba920 con 0x562ec4aefad0 2023-07-14T11:22:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2ba457ab80 10 client.0 _ll_register_callbacks cb 0x562ec49d4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2b89d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2b8ad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4aefad0 0x562ec4aecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2b89d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57508/0 (socket says 172.21.15.161:57508) 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2b89d9b700 1 -- 172.21.15.161:0/2386075001 learned_addr learned my addr 172.21.15.161:0/2386075001 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.225+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae42d0 0x562ec4aef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b89d9b700 1 -- 172.21.15.161:0/2386075001 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4aefad0 msgr2=0x562ec4aecec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4aefad0 0x562ec4aecec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b89d9b700 1 -- 172.21.15.161:0/2386075001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae42d0 msgr2=0x562ec4aef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae42d0 0x562ec4aef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b89d9b700 1 -- 172.21.15.161:0/2386075001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c10d210 con 0x562ec4ae4be0 2023-07-14T11:22:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aec7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b80012390 tx=0x7f2b80009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8438892126b9e86 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:22:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b71ffb700 1 -- 172.21.15.161:0/2386075001 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b800123c0 con 0x562ec4ae4be0 2023-07-14T11:22:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ec49904f0 con 0x562ec4ae4be0 2023-07-14T11:22:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.226+0000 7f2b71ffb700 1 -- 172.21.15.161:0/2386075001 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f2b800098d0 con 0x562ec4ae4be0 2023-07-14T11:22:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2b71ffb700 1 -- 172.21.15.161:0/2386075001 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b8002e5e0 con 0x562ec4ae4be0 2023-07-14T11:22:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 10 client.25724 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f2b8c1a4b70 con 0x562ec4ae4be0 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2b71ffb700 1 -- 172.21.15.161:0/2386075001 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ec49904f0 con 0x562ec4ae4be0 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2b71ffb700 1 -- 172.21.15.161:0/2386075001 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b800115d0 con 0x562ec4ae4be0 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 10 client.25724 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 20 client.25724 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4aff370 0x562ec4b01780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 4 client.25724 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:37.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.227+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ec4a2feb0 con 0x562ec4aff370 2023-07-14T11:22:37.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.228+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4aff370 0x562ec4b01780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.228+0000 7f2b71ffb700 10 client.25724 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:37.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.228+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4aff370 0x562ec4b01780 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e4f9758bdd474 server_cookie=b2a07e1a42eec794 in_seq=0 out_seq=0 2023-07-14T11:22:37.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2b71ffb700 1 -- 172.21.15.161:0/2386075001 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562ec4a2feb0 con 0x562ec4aff370 2023-07-14T11:22:37.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2b71ffb700 10 client.25724 handle_command_reply: tid=0 2023-07-14T11:22:37.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 client.25724 shutdown 2023-07-14T11:22:37.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 msgr2=0x562ec4aec7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aec7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b80012390 tx=0x7f2b80009260 comp rx=0 tx=0).stop 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 shutdown_connections 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4aff370 0x562ec4b01780 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae42d0 0x562ec4aef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2b71ffb700 0 client.25724 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4aefad0 0x562ec4aecec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2386075001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae4be0 0x562ec4aec7d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 >> 172.21.15.161:0/2386075001 conn(0x562ec4adada0 msgr2=0x562ec4ae0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:37.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 shutdown_connections 2023-07-14T11:22:37.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.229+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2386075001 wait complete. 2023-07-14T11:22:37.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.230+0000 7f2ba457ab80 20 client.25724 trim_cache size 0 max 16384 2023-07-14T11:22:37.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.244+0000 7f2ba457ab80 1 Processor -- start 2023-07-14T11:22:37.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2ba457ab80 1 -- start start 2023-07-14T11:22:37.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae42d0 0x562ec4aef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae4be0 0x562ec4aebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4aec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4ae4920 con 0x562ec4aefad0 2023-07-14T11:22:37.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4ae50d0 con 0x562ec4ae42d0 2023-07-14T11:22:37.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4aead30 con 0x562ec4ae4be0 2023-07-14T11:22:37.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2b8ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4aec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2b8ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4aec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39360/0 (socket says 172.21.15.161:39360) 2023-07-14T11:22:37.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2b8ad9d700 1 -- 172.21.15.161:0/2863980228 learned_addr learned my addr 172.21.15.161:0/2863980228 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:37.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae42d0 0x562ec4aef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.245+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae4be0 0x562ec4aebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b8a59c700 1 -- 172.21.15.161:0/2863980228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae4be0 msgr2=0x562ec4aebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae4be0 0x562ec4aebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b8a59c700 1 -- 172.21.15.161:0/2863980228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 msgr2=0x562ec4aec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4aec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b8a59c700 1 -- 172.21.15.161:0/2863980228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec484dc60 con 0x562ec4ae42d0 2023-07-14T11:22:37.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4aec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:37.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae42d0 0x562ec4aef0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00b2e0 tx=0x7f2b7c005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9043cc92654a795f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:37.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b797fa700 1 -- 172.21.15.161:0/2863980228 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c01d040 con 0x562ec4ae42d0 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b797fa700 1 -- 172.21.15.161:0/2863980228 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b7c01e040 con 0x562ec4ae42d0 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.246+0000 7f2b797fa700 1 -- 172.21.15.161:0/2863980228 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0056e0 con 0x562ec4ae42d0 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2863980228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae42d0 msgr2=0x562ec4aef0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae42d0 0x562ec4aef0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00b2e0 tx=0x7f2b7c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2863980228 shutdown_connections 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4aefad0 0x562ec4aec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4ae42d0 0x562ec4aef0f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/2863980228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae4be0 0x562ec4aebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2863980228 >> 172.21.15.161:0/2863980228 conn(0x562ec4adada0 msgr2=0x562ec4ae7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:37.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2863980228 shutdown_connections 2023-07-14T11:22:37.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.247+0000 7f2ba457ab80 1 -- 172.21.15.161:0/2863980228 wait complete. 2023-07-14T11:22:37.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 -- start start 2023-07-14T11:22:37.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 Processor -- start 2023-07-14T11:22:37.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4af6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae4be0 0x562ec4af4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4aefad0 0x562ec4af4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4af5550 con 0x562ec4ae4be0 2023-07-14T11:22:37.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4af56c0 con 0x562ec4aefad0 2023-07-14T11:22:37.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4b53240 con 0x562ec4ae42d0 2023-07-14T11:22:37.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.248+0000 7f2ba457ab80 10 client.0 _ll_register_callbacks cb 0x7f2b80012040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:37.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae4be0 0x562ec4af4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b8a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4af6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae4be0 0x562ec4af4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39362/0 (socket says 172.21.15.161:39362) 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 -- 172.21.15.161:0/1441667532 learned_addr learned my addr 172.21.15.161:0/1441667532 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4aefad0 0x562ec4af4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 -- 172.21.15.161:0/1441667532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 msgr2=0x562ec4af6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4af6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 -- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4aefad0 msgr2=0x562ec4af4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4aefad0 0x562ec4af4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 -- 172.21.15.161:0/1441667532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ec4864de0 con 0x562ec4ae4be0 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4af6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b8ad9d700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4aefad0 0x562ec4af4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b89d9b700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae4be0 0x562ec4af4900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b80011900 tx=0x7f2b800099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fdb8535cadcf0c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.249+0000 7f2b71ffb700 1 -- 172.21.15.161:0/1441667532 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80012e80 con 0x562ec4ae4be0 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b840999b0 con 0x562ec4ae4be0 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2b71ffb700 1 -- 172.21.15.161:0/1441667532 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f2b80011650 con 0x562ec4ae4be0 2023-07-14T11:22:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2b71ffb700 1 -- 172.21.15.161:0/1441667532 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b80011b50 con 0x562ec4ae4be0 2023-07-14T11:22:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 10 client.15693 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f2b7c00be60 con 0x562ec4ae4be0 2023-07-14T11:22:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2b71ffb700 1 -- 172.21.15.161:0/1441667532 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b840999b0 con 0x562ec4ae4be0 2023-07-14T11:22:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2b71ffb700 1 -- 172.21.15.161:0/1441667532 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b80037c10 con 0x562ec4ae4be0 2023-07-14T11:22:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 10 client.15693 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 20 client.15693 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4b549a0 0x562ec4b56db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 4 client.15693 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.250+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562ec49f2050 con 0x562ec4b549a0 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4b549a0 0x562ec4b56db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2b71ffb700 10 client.15693 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2b8a59c700 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4b549a0 0x562ec4b56db0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a1fb32ba608cb server_cookie=dd8d97167f41a3c0 in_seq=0 out_seq=0 2023-07-14T11:22:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2b71ffb700 1 -- 172.21.15.161:0/1441667532 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562ec49f2050 con 0x562ec4b549a0 2023-07-14T11:22:37.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2b71ffb700 10 client.15693 handle_command_reply: tid=0 2023-07-14T11:22:37.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2ba457ab80 1 client.15693 shutdown 2023-07-14T11:22:37.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae4be0 msgr2=0x562ec4af4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.251+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae4be0 0x562ec4af4900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b80011900 tx=0x7f2b800099f0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 shutdown_connections 2023-07-14T11:22:37.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562ec4b549a0 0x562ec4b56db0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ec4aefad0 0x562ec4af4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562ec4ae42d0 0x562ec4af6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2b71ffb700 0 client.15693 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 --2- 172.21.15.161:0/1441667532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ec4ae4be0 0x562ec4af4900 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 >> 172.21.15.161:0/1441667532 conn(0x562ec4adada0 msgr2=0x562ec4ae63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 shutdown_connections 2023-07-14T11:22:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 1 -- 172.21.15.161:0/1441667532 wait complete. 2023-07-14T11:22:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2ba457ab80 20 client.15693 trim_cache size 0 max 16384 2023-07-14T11:22:37.264 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:22:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b740a13d0 msgr2=0x7f2b740a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b740a13d0 0x7f2b740a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c074000 tx=0x7f2b80009b80 comp rx=0 tx=0).stop 2023-07-14T11:22:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 msgr2=0x7f2b8c1a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.252+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c1a3dc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400ef30 tx=0x7f2b8400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 shutdown_connections 2023-07-14T11:22:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b740a13d0 0x7f2b740a3880 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b8c105ae0 0x7f2b8c1a3dc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b8c0ffad0 0x7f2b8c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 --2- 172.21.15.161:0/3615379686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b8c100480 0x7f2b8c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:37.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 >> 172.21.15.161:0/3615379686 conn(0x7f2b8c0fb5f0 msgr2=0x7f2b8c0fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:37.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 shutdown_connections 2023-07-14T11:22:37.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:37.253+0000 7f2b90bff700 1 -- 172.21.15.161:0/3615379686 wait complete. 2023-07-14T11:22:37.273 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3756225411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:37 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3756225411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:37.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:37 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3756225411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:22:38.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:22:38.275 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:22:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:38 smithi175 ceph-mon[129081]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T11:22:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:38 smithi182 ceph-mon[129366]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T11:22:38.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 -- 172.21.15.161:0/198367330 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e80728b0 msgr2=0x7f26e8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/198367330 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e80728b0 0x7f26e8072c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009990 tx=0x7f26d8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 -- 172.21.15.161:0/198367330 shutdown_connections 2023-07-14T11:22:38.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/198367330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e810c310 0x7f26e810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/198367330 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e80728b0 0x7f26e8072c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/198367330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e80731c0 0x7f26e810bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 -- 172.21.15.161:0/198367330 >> 172.21.15.161:0/198367330 conn(0x7f26e80fb5f0 msgr2=0x7f26e80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 -- 172.21.15.161:0/198367330 shutdown_connections 2023-07-14T11:22:38.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.491+0000 7f26ee9ff700 1 -- 172.21.15.161:0/198367330 wait complete. 2023-07-14T11:22:38.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:38 smithi161 ceph-mon[95526]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T11:22:38.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 Processor -- start 2023-07-14T11:22:38.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 -- start start 2023-07-14T11:22:38.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e80728b0 0x7f26e8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e80731c0 0x7f26e8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e810c310 0x7f26e819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e819ce20 con 0x7f26e80728b0 2023-07-14T11:22:38.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e819cf90 con 0x7f26e80731c0 2023-07-14T11:22:38.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ee9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e819d150 con 0x7f26e810c310 2023-07-14T11:22:38.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26e7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e80728b0 0x7f26e8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26ecf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e810c310 0x7f26e819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.493+0000 7f26e7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e80728b0 0x7f26e8197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60784/0 (socket says 172.21.15.161:60784) 2023-07-14T11:22:38.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e7fff700 1 -- 172.21.15.161:0/4162224457 learned_addr learned my addr 172.21.15.161:0/4162224457 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e77fe700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e80731c0 0x7f26e8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e7fff700 1 -- 172.21.15.161:0/4162224457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e810c310 msgr2=0x7f26e819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e7fff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e810c310 0x7f26e819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e7fff700 1 -- 172.21.15.161:0/4162224457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e80731c0 msgr2=0x7f26e8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e7fff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e80731c0 0x7f26e8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e7fff700 1 -- 172.21.15.161:0/4162224457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0009c40 con 0x7f26e80728b0 2023-07-14T11:22:38.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e810c310 0x7f26e819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e7fff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e80728b0 0x7f26e8197e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005910 tx=0x7f26d8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.494+0000 7f26e57fa700 1 -- 172.21.15.161:0/4162224457 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d801d070 con 0x7f26e80728b0 2023-07-14T11:22:38.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.495+0000 7f26e57fa700 1 -- 172.21.15.161:0/4162224457 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d8022470 con 0x7f26e80728b0 2023-07-14T11:22:38.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.495+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d80095f0 con 0x7f26e80728b0 2023-07-14T11:22:38.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.495+0000 7f26e57fa700 1 -- 172.21.15.161:0/4162224457 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8017650 con 0x7f26e80728b0 2023-07-14T11:22:38.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.495+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e81a4a90 con 0x7f26e80728b0 2023-07-14T11:22:38.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.499+0000 7f26e57fa700 1 -- 172.21.15.161:0/4162224457 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f26d80225e0 con 0x7f26e80728b0 2023-07-14T11:22:38.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.500+0000 7f2702318b80 1 Processor -- start 2023-07-14T11:22:38.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.500+0000 7f2702318b80 1 -- start start 2023-07-14T11:22:38.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f2702318b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b23720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f2702318b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b2e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f2702318b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b2eb50 0x55a873b30f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f2702318b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873a29d30 con 0x55a873b23c60 2023-07-14T11:22:38.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f2702318b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873a7cac0 con 0x55a873b2eb50 2023-07-14T11:22:38.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f2702318b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873a75f50 con 0x55a873b23350 2023-07-14T11:22:38.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26e57fa700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26d00a13d0 0x7f26d00a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26e7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b2e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26e7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b2e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60796/0 (socket says 172.21.15.161:60796) 2023-07-14T11:22:38.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26e7fff700 1 -- 172.21.15.161:0/4164362884 learned_addr learned my addr 172.21.15.161:0/4164362884 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b23720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26e77fe700 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b2eb50 0x55a873b30f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26e57fa700 1 -- 172.21.15.161:0/4162224457 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f26d80ce360 con 0x7f26e80728b0 2023-07-14T11:22:38.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.501+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26d00a13d0 0x7f26d00a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26e77fe700 1 -- 172.21.15.161:0/4164362884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 msgr2=0x55a873b23720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26e77fe700 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b23720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26e77fe700 1 -- 172.21.15.161:0/4164362884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 msgr2=0x55a873b2e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26e77fe700 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b2e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26e77fe700 1 -- 172.21.15.161:0/4164362884 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8738a3e60 con 0x55a873b2eb50 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b23720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26e77fe700 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b2eb50 0x55a873b30f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009c80 tx=0x7f26dc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d9c1a6810bdb45 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26ce7fc700 1 -- 172.21.15.161:0/4164362884 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc011cb0 con 0x55a873b2eb50 2023-07-14T11:22:38.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26ce7fc700 1 -- 172.21.15.161:0/4164362884 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26dc011e10 con 0x55a873b2eb50 2023-07-14T11:22:38.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.502+0000 7f26ce7fc700 1 -- 172.21.15.161:0/4164362884 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc005280 con 0x55a873b2eb50 2023-07-14T11:22:38.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26d00a13d0 0x7f26d00a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009550 tx=0x7f26e000b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 -- 172.21.15.161:0/4164362884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b2eb50 msgr2=0x55a873b30f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b2eb50 0x55a873b30f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009c80 tx=0x7f26dc009f90 comp rx=0 tx=0).stop 2023-07-14T11:22:38.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 -- 172.21.15.161:0/4164362884 shutdown_connections 2023-07-14T11:22:38.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b2eb50 0x55a873b30f30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b23720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 --2- 172.21.15.161:0/4164362884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b2e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 -- 172.21.15.161:0/4164362884 >> 172.21.15.161:0/4164362884 conn(0x55a873b19e20 msgr2=0x55a873b1f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.503+0000 7f2702318b80 1 -- 172.21.15.161:0/4164362884 shutdown_connections 2023-07-14T11:22:38.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.504+0000 7f2702318b80 1 -- 172.21.15.161:0/4164362884 wait complete. 2023-07-14T11:22:38.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.504+0000 7f2702318b80 1 -- start start 2023-07-14T11:22:38.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 1 Processor -- start 2023-07-14T11:22:38.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b2e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23c60 0x55a873b2b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b2bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873a15b40 con 0x55a873b2eb50 2023-07-14T11:22:38.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873a15840 con 0x55a873b23c60 2023-07-14T11:22:38.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8738f99a0 con 0x55a873b23350 2023-07-14T11:22:38.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.505+0000 7f2702318b80 10 client.0 _ll_register_callbacks cb 0x55a873a137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:38.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b2e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26ecf9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b2bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b2e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57262/0 (socket says 172.21.15.161:57262) 2023-07-14T11:22:38.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 -- 172.21.15.161:0/789797141 learned_addr learned my addr 172.21.15.161:0/789797141 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e77fe700 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23c60 0x55a873b2b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 -- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23c60 msgr2=0x55a873b2b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23c60 0x55a873b2b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 -- 172.21.15.161:0/789797141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 msgr2=0x55a873b2bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b2bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 -- 172.21.15.161:0/789797141 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e0008770 con 0x55a873b23350 2023-07-14T11:22:38.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26e7fff700 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b2e490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26d80a2eb0 tx=0x7f26d806a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8250dec30ff8ee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:22:38.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26c77fe700 1 -- 172.21.15.161:0/789797141 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d809ab60 con 0x55a873b23350 2023-07-14T11:22:38.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f26c77fe700 1 -- 172.21.15.161:0/789797141 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f26d809b020 con 0x55a873b23350 2023-07-14T11:22:38.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.506+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8739cf570 con 0x55a873b23350 2023-07-14T11:22:38.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f26c77fe700 1 -- 172.21.15.161:0/789797141 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d809a980 con 0x55a873b23350 2023-07-14T11:22:38.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 10 client.25744 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:38.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f26dc0098e0 con 0x55a873b23350 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f26c77fe700 1 -- 172.21.15.161:0/789797141 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8739cf570 con 0x55a873b23350 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f26c77fe700 1 -- 172.21.15.161:0/789797141 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26d809b280 con 0x55a873b23350 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 10 client.25744 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 20 client.25744 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b3e000 0x55a873b403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 4 client.25744 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:38.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.507+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a873a6ef30 con 0x55a873b3e000 2023-07-14T11:22:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.508+0000 7f26e77fe700 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b3e000 0x55a873b403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.508+0000 7f26e77fe700 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b3e000 0x55a873b403e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a4238f55a4babf server_cookie=2a9c2d403c4d8381 in_seq=0 out_seq=0 2023-07-14T11:22:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.508+0000 7f26c77fe700 10 client.25744 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f26c77fe700 1 -- 172.21.15.161:0/789797141 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a873a6ef30 con 0x55a873b3e000 2023-07-14T11:22:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f26c77fe700 10 client.25744 handle_command_reply: tid=0 2023-07-14T11:22:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 client.25744 shutdown 2023-07-14T11:22:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 msgr2=0x55a873b2e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b2e490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26d80a2eb0 tx=0x7f26d806a4a0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 shutdown_connections 2023-07-14T11:22:38.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b3e000 0x55a873b403e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b2bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23350 0x55a873b2e490 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f26c77fe700 0 client.25744 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 --2- 172.21.15.161:0/789797141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23c60 0x55a873b2b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.509+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 >> 172.21.15.161:0/789797141 conn(0x55a873b19e20 msgr2=0x55a873b1f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.510+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 shutdown_connections 2023-07-14T11:22:38.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.510+0000 7f2702318b80 1 -- 172.21.15.161:0/789797141 wait complete. 2023-07-14T11:22:38.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.510+0000 7f2702318b80 20 client.25744 trim_cache size 0 max 16384 2023-07-14T11:22:38.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.535+0000 7f2702318b80 1 Processor -- start 2023-07-14T11:22:38.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f2702318b80 1 -- start start 2023-07-14T11:22:38.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f2702318b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b35530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f2702318b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23c60 0x55a873b28a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f2702318b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b28fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f2702318b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873b2ccf0 con 0x55a873b2eb50 2023-07-14T11:22:38.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f2702318b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873b199f0 con 0x55a873b23350 2023-07-14T11:22:38.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f2702318b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873b24240 con 0x55a873b23c60 2023-07-14T11:22:38.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f26ecf9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b28fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f26ecf9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b28fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60812/0 (socket says 172.21.15.161:60812) 2023-07-14T11:22:38.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f26ecf9c700 1 -- 172.21.15.161:0/797042647 learned_addr learned my addr 172.21.15.161:0/797042647 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f26e7fff700 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b35530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.536+0000 7f26e77fe700 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23c60 0x55a873b28a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ecf9c700 1 -- 172.21.15.161:0/797042647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23c60 msgr2=0x55a873b28a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23c60 0x55a873b28a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ecf9c700 1 -- 172.21.15.161:0/797042647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 msgr2=0x55a873b35530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b35530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ecf9c700 1 -- 172.21.15.161:0/797042647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87388cce0 con 0x55a873b2eb50 2023-07-14T11:22:38.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26e7fff700 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b35530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b28fd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f26e0008ec0 tx=0x7f26e0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f640e7f065833ae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ceffd700 1 -- 172.21.15.161:0/797042647 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e002a070 con 0x55a873b2eb50 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ceffd700 1 -- 172.21.15.161:0/797042647 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e0022040 con 0x55a873b2eb50 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.537+0000 7f26ceffd700 1 -- 172.21.15.161:0/797042647 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0021050 con 0x55a873b2eb50 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 -- 172.21.15.161:0/797042647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 msgr2=0x55a873b28fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b28fd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f26e0008ec0 tx=0x7f26e0010000 comp rx=0 tx=0).stop 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 -- 172.21.15.161:0/797042647 shutdown_connections 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b2eb50 0x55a873b28fd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b35530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 --2- 172.21.15.161:0/797042647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b23c60 0x55a873b28a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 -- 172.21.15.161:0/797042647 >> 172.21.15.161:0/797042647 conn(0x55a873b19e20 msgr2=0x55a873b1f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 -- 172.21.15.161:0/797042647 shutdown_connections 2023-07-14T11:22:38.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.538+0000 7f2702318b80 1 -- 172.21.15.161:0/797042647 wait complete. 2023-07-14T11:22:38.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 -- start start 2023-07-14T11:22:38.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 Processor -- start 2023-07-14T11:22:38.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b8c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b2eb50 0x55a873b867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873b86d80 con 0x55a873b23c60 2023-07-14T11:22:38.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873b86ef0 con 0x55a873b23350 2023-07-14T11:22:38.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873b870b0 con 0x55a873b2eb50 2023-07-14T11:22:38.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.539+0000 7f2702318b80 10 client.0 _ll_register_callbacks cb 0x7f26d8005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:38.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b2eb50 0x55a873b867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26e77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b8c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b2eb50 0x55a873b867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57276/0 (socket says 172.21.15.161:57276) 2023-07-14T11:22:38.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 -- 172.21.15.161:0/1097670430 learned_addr learned my addr 172.21.15.161:0/1097670430 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26e7fff700 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 -- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 msgr2=0x55a873b8c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b8c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 -- 172.21.15.161:0/1097670430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 msgr2=0x55a873b8c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b8c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 -- 172.21.15.161:0/1097670430 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc0098e0 con 0x55a873b2eb50 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26e7fff700 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b8c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26ecf9c700 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b2eb50 0x55a873b867a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f26e0004960 tx=0x7f26e000e910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d690998e046c31b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.540+0000 7f26c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f26c77fe700 1 -- 172.21.15.161:0/1097670430 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e002a030 con 0x55a873b2eb50 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26dc011e10 con 0x55a873b2eb50 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f26c77fe700 1 -- 172.21.15.161:0/1097670430 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f26e0008050 con 0x55a873b2eb50 2023-07-14T11:22:38.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f26c77fe700 1 -- 172.21.15.161:0/1097670430 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e0021020 con 0x55a873b2eb50 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 10 client.25750 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f26e0003e30 con 0x55a873b2eb50 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f26c77fe700 1 -- 172.21.15.161:0/1097670430 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e003c020 con 0x55a873b2eb50 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f26c77fe700 1 -- 172.21.15.161:0/1097670430 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26e0036060 con 0x55a873b2eb50 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 10 client.25750 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 10 client.25750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 20 client.25750 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b94280 0x55a873b96690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 4 client.25750 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:22:38.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.541+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a873a310d0 con 0x55a873b94280 2023-07-14T11:22:38.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.542+0000 7f26e7fff700 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b94280 0x55a873b96690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.542+0000 7f26c77fe700 10 client.25750 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.542+0000 7f26e7fff700 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b94280 0x55a873b96690 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a275a7d4aae6f4c2 server_cookie=e6734ed59eafc8bc in_seq=0 out_seq=0 2023-07-14T11:22:38.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f26c77fe700 1 -- 172.21.15.161:0/1097670430 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a873a310d0 con 0x55a873b94280 2023-07-14T11:22:38.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f26c77fe700 10 client.25750 handle_command_reply: tid=0 2023-07-14T11:22:38.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 client.25750 shutdown 2023-07-14T11:22:38.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b2eb50 msgr2=0x55a873b867a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b2eb50 0x55a873b867a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f26e0004960 tx=0x7f26e000e910 comp rx=0 tx=0).stop 2023-07-14T11:22:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 shutdown_connections 2023-07-14T11:22:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a873b94280 0x55a873b96690 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a873b2eb50 0x55a873b867a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f26c77fe700 0 client.25750 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a873b23350 0x55a873b8c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 --2- 172.21.15.161:0/1097670430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a873b23c60 0x55a873b8c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 >> 172.21.15.161:0/1097670430 conn(0x55a873b19e20 msgr2=0x55a873b25300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 shutdown_connections 2023-07-14T11:22:38.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.544+0000 7f2702318b80 1 -- 172.21.15.161:0/1097670430 wait complete. 2023-07-14T11:22:38.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f2702318b80 20 client.25750 trim_cache size 0 max 16384 2023-07-14T11:22:38.555 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:22:38.556 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:22:38.556 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "1371b9d9-2c16-46ca-8131-6177721f6b78", 2023-07-14T11:22:38.556 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:22:38.556 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:22:38.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26d00a13d0 msgr2=0x7f26d00a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26d00a13d0 0x7f26d00a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009550 tx=0x7f26e000b040 comp rx=0 tx=0).stop 2023-07-14T11:22:38.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e80728b0 msgr2=0x7f26e8197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e80728b0 0x7f26e8197e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005910 tx=0x7f26d8004710 comp rx=0 tx=0).stop 2023-07-14T11:22:38.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 shutdown_connections 2023-07-14T11:22:38.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26d00a13d0 0x7f26d00a3880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e80728b0 0x7f26e8197e40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26e810c310 0x7f26e819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 --2- 172.21.15.161:0/4162224457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e80731c0 0x7f26e8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 >> 172.21.15.161:0/4162224457 conn(0x7f26e80fb5f0 msgr2=0x7f26e810e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.545+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 shutdown_connections 2023-07-14T11:22:38.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.546+0000 7f26ee9ff700 1 -- 172.21.15.161:0/4162224457 wait complete. 2023-07-14T11:22:38.568 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:22:38.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 -- 172.21.15.161:0/2071218249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61300728b0 msgr2=0x7f6130072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 --2- 172.21.15.161:0/2071218249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61300728b0 0x7f6130072c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6124009990 tx=0x7f6124009ca0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 -- 172.21.15.161:0/2071218249 shutdown_connections 2023-07-14T11:22:38.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 --2- 172.21.15.161:0/2071218249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61300728b0 0x7f6130072c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 --2- 172.21.15.161:0/2071218249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613010c310 0x7f613010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 --2- 172.21.15.161:0/2071218249 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f61300731c0 0x7f613010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 -- 172.21.15.161:0/2071218249 >> 172.21.15.161:0/2071218249 conn(0x7f61300fb5f0 msgr2=0x7f61300fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.774+0000 7f61371ff700 1 -- 172.21.15.161:0/2071218249 shutdown_connections 2023-07-14T11:22:38.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.775+0000 7f61371ff700 1 -- 172.21.15.161:0/2071218249 wait complete. 2023-07-14T11:22:38.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.776+0000 7f61371ff700 1 Processor -- start 2023-07-14T11:22:38.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.776+0000 7f61371ff700 1 -- start start 2023-07-14T11:22:38.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f61371ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61300728b0 0x7f6130197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f61371ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f61300731c0 0x7f6130198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f61371ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613010c310 0x7f613019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f61371ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613019cdc0 con 0x7f613010c310 2023-07-14T11:22:38.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f61371ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613019cf30 con 0x7f61300728b0 2023-07-14T11:22:38.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f61371ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613019d0f0 con 0x7f61300731c0 2023-07-14T11:22:38.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61300728b0 0x7f6130197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61300728b0 0x7f6130197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49548/0 (socket says 172.21.15.161:49548) 2023-07-14T11:22:38.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f613579c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613010c310 0x7f613019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 -- 172.21.15.161:0/2373101709 learned_addr learned my addr 172.21.15.161:0/2373101709 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f612ffff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f61300731c0 0x7f6130198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 -- 172.21.15.161:0/2373101709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f61300731c0 msgr2=0x7f6130198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f61300731c0 0x7f6130198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 -- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613010c310 msgr2=0x7f613019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613010c310 0x7f613019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f6134f9b700 1 -- 172.21.15.161:0/2373101709 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128009c40 con 0x7f61300728b0 2023-07-14T11:22:38.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.777+0000 7f613579c700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613010c310 0x7f613019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.778+0000 7f6134f9b700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61300728b0 0x7f6130197de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6124000c00 tx=0x7f6124005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.778+0000 7f612dffb700 1 -- 172.21.15.161:0/2373101709 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612401d070 con 0x7f61300728b0 2023-07-14T11:22:38.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.778+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61240095f0 con 0x7f61300728b0 2023-07-14T11:22:38.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.778+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61301a4a30 con 0x7f61300728b0 2023-07-14T11:22:38.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.778+0000 7f612dffb700 1 -- 172.21.15.161:0/2373101709 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6124022470 con 0x7f61300728b0 2023-07-14T11:22:38.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.778+0000 7f612dffb700 1 -- 172.21.15.161:0/2373101709 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124017650 con 0x7f61300728b0 2023-07-14T11:22:38.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.782+0000 7f612dffb700 1 -- 172.21.15.161:0/2373101709 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f61240228a0 con 0x7f61300728b0 2023-07-14T11:22:38.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.783+0000 7f612dffb700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f61200a1350 0x7f61200a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.783+0000 7f612dffb700 1 -- 172.21.15.161:0/2373101709 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f61240ce3c0 con 0x7f61300728b0 2023-07-14T11:22:38.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.783+0000 7f612ffff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f61200a1350 0x7f61200a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.784+0000 7f612ffff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f61200a1350 0x7f61200a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61301993f0 tx=0x7f611c009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.785+0000 7f614aa9ab80 1 Processor -- start 2023-07-14T11:22:38.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.785+0000 7f614aa9ab80 1 -- start start 2023-07-14T11:22:38.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.785+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef83f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef8404590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8406eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef82ffcb0 con 0x555ef8404ad0 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef8352a40 con 0x555ef83f92d0 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef834bed0 con 0x555ef83f9be0 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef8404590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef83f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef8404590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57318/0 (socket says 172.21.15.161:57318) 2023-07-14T11:22:38.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef83f96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49562/0 (socket says 172.21.15.161:49562) 2023-07-14T11:22:38.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 -- 172.21.15.161:0/4270041947 learned_addr learned my addr 172.21.15.161:0/4270041947 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f613579c700 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8406eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 -- 172.21.15.161:0/4270041947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 msgr2=0x555ef83f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef83f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 -- 172.21.15.161:0/4270041947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 msgr2=0x555ef8406eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8406eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f612ffff700 1 -- 172.21.15.161:0/4270041947 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ef8179de0 con 0x555ef83f9be0 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.786+0000 7f6134f9b700 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef83f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.787+0000 7f612ffff700 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef8404590 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f611c011d50 tx=0x7f611c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf809b1b4a8f4dc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.787+0000 7f611affd700 1 -- 172.21.15.161:0/4270041947 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c00eb20 con 0x555ef83f9be0 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.787+0000 7f611affd700 1 -- 172.21.15.161:0/4270041947 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f611c021070 con 0x555ef83f9be0 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.787+0000 7f611affd700 1 -- 172.21.15.161:0/4270041947 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c01d410 con 0x555ef83f9be0 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 -- 172.21.15.161:0/4270041947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 msgr2=0x555ef8404590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef8404590 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f611c011d50 tx=0x7f611c014000 comp rx=0 tx=0).stop 2023-07-14T11:22:38.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 -- 172.21.15.161:0/4270041947 shutdown_connections 2023-07-14T11:22:38.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8406eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef83f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/4270041947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef8404590 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 -- 172.21.15.161:0/4270041947 >> 172.21.15.161:0/4270041947 conn(0x555ef83efda0 msgr2=0x555ef83f56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 -- 172.21.15.161:0/4270041947 shutdown_connections 2023-07-14T11:22:38.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 -- 172.21.15.161:0/4270041947 wait complete. 2023-07-14T11:22:38.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.788+0000 7f614aa9ab80 1 -- start start 2023-07-14T11:22:38.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 1 Processor -- start 2023-07-14T11:22:38.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef84044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef84017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8401e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef82ebac0 con 0x555ef8404ad0 2023-07-14T11:22:38.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef82eb7c0 con 0x555ef83f92d0 2023-07-14T11:22:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef81cf920 con 0x555ef83f9be0 2023-07-14T11:22:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f614aa9ab80 10 client.0 _ll_register_callbacks cb 0x555ef82e9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f613579c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8401e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f612ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef84017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef84044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f612ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef84017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57322/0 (socket says 172.21.15.161:57322) 2023-07-14T11:22:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f613579c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8401e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60862/0 (socket says 172.21.15.161:60862) 2023-07-14T11:22:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.789+0000 7f612ffff700 1 -- 172.21.15.161:0/127897353 learned_addr learned my addr 172.21.15.161:0/127897353 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f6134f9b700 1 -- 172.21.15.161:0/127897353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 msgr2=0x555ef84017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f6134f9b700 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef84017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f6134f9b700 1 -- 172.21.15.161:0/127897353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 msgr2=0x555ef8401e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f6134f9b700 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8401e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f6134f9b700 1 -- 172.21.15.161:0/127897353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61301a4a30 con 0x555ef83f92d0 2023-07-14T11:22:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f612ffff700 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef84017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f6134f9b700 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef84044e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f61240cb720 tx=0x7f61240cbcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c7e32d7d0bc600 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f610ffff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:22:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f610ffff700 1 -- 172.21.15.161:0/127897353 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612409b080 con 0x555ef83f92d0 2023-07-14T11:22:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ef82a54f0 con 0x555ef83f92d0 2023-07-14T11:22:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.790+0000 7f610ffff700 1 -- 172.21.15.161:0/127897353 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f612409f0b0 con 0x555ef83f92d0 2023-07-14T11:22:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f610ffff700 1 -- 172.21.15.161:0/127897353 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61240a3440 con 0x555ef83f92d0 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 10 client.25760 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f611c017070 con 0x555ef83f92d0 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f610ffff700 1 -- 172.21.15.161:0/127897353 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ef82a54f0 con 0x555ef83f92d0 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f610ffff700 1 -- 172.21.15.161:0/127897353 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6124099fe0 con 0x555ef83f92d0 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 10 client.25760 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 20 client.25760 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8414070 0x555ef8416450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 4 client.25760 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:22:38.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.791+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ef8344eb0 con 0x555ef8414070 2023-07-14T11:22:38.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.792+0000 7f612ffff700 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8414070 0x555ef8416450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.792+0000 7f610ffff700 10 client.25760 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.792+0000 7f612ffff700 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8414070 0x555ef8416450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec51d0282d47480 server_cookie=3d6726d388b035eb in_seq=0 out_seq=0 2023-07-14T11:22:38.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.793+0000 7f610ffff700 1 -- 172.21.15.161:0/127897353 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555ef8344eb0 con 0x555ef8414070 2023-07-14T11:22:38.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.793+0000 7f610ffff700 10 client.25760 handle_command_reply: tid=0 2023-07-14T11:22:38.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.793+0000 7f614aa9ab80 1 client.25760 shutdown 2023-07-14T11:22:38.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.793+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 msgr2=0x555ef84044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.793+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef84044e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f61240cb720 tx=0x7f61240cbcb0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.793+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 shutdown_connections 2023-07-14T11:22:38.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.793+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8414070 0x555ef8416450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8401e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f92d0 0x555ef84044e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f610ffff700 0 client.25760 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/127897353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f9be0 0x555ef84017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 >> 172.21.15.161:0/127897353 conn(0x555ef83efda0 msgr2=0x555ef83f5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 shutdown_connections 2023-07-14T11:22:38.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f614aa9ab80 1 -- 172.21.15.161:0/127897353 wait complete. 2023-07-14T11:22:38.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.794+0000 7f614aa9ab80 20 client.25760 trim_cache size 0 max 16384 2023-07-14T11:22:38.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.811+0000 7f614aa9ab80 1 Processor -- start 2023-07-14T11:22:38.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.811+0000 7f614aa9ab80 1 -- start start 2023-07-14T11:22:38.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef8404080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef83f9be0 0x555ef8400f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef8404ad0 0x555ef84014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef8402780 con 0x555ef83f9be0 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef83fa540 con 0x555ef8404ad0 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef83ef9a0 con 0x555ef83f92d0 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef8404080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f612ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef83f9be0 0x555ef8400f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef8404080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57330/0 (socket says 172.21.15.161:57330) 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f6134f9b700 1 -- 172.21.15.161:0/1714954057 learned_addr learned my addr 172.21.15.161:0/1714954057 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.812+0000 7f613579c700 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef8404ad0 0x555ef84014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f6134f9b700 1 -- 172.21.15.161:0/1714954057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef8404ad0 msgr2=0x555ef84014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f6134f9b700 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef8404ad0 0x555ef84014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f6134f9b700 1 -- 172.21.15.161:0/1714954057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef83f9be0 msgr2=0x555ef8400f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f6134f9b700 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef83f9be0 0x555ef8400f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f6134f9b700 1 -- 172.21.15.161:0/1714954057 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ef8162c60 con 0x555ef83f92d0 2023-07-14T11:22:38.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f6134f9b700 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef8404080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f612409ad80 tx=0x7f61240cb660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398607d736c24919 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f611b7fe700 1 -- 172.21.15.161:0/1714954057 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612409a3b0 con 0x555ef83f92d0 2023-07-14T11:22:38.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f611b7fe700 1 -- 172.21.15.161:0/1714954057 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f612409f0b0 con 0x555ef83f92d0 2023-07-14T11:22:38.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.813+0000 7f611b7fe700 1 -- 172.21.15.161:0/1714954057 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240a2fb0 con 0x555ef83f92d0 2023-07-14T11:22:38.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1714954057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 msgr2=0x555ef8404080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef8404080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f612409ad80 tx=0x7f61240cb660 comp rx=0 tx=0).stop 2023-07-14T11:22:38.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1714954057 shutdown_connections 2023-07-14T11:22:38.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef8404ad0 0x555ef84014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef8404080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1714954057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef83f9be0 0x555ef8400f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1714954057 >> 172.21.15.161:0/1714954057 conn(0x555ef83efda0 msgr2=0x555ef83fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1714954057 shutdown_connections 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1714954057 wait complete. 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.814+0000 7f614aa9ab80 1 -- start start 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 1 Processor -- start 2023-07-14T11:22:38.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef840afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f9be0 0x555ef840b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8409530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef8409b10 con 0x555ef8404ad0 2023-07-14T11:22:38.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef8409c80 con 0x555ef83f9be0 2023-07-14T11:22:38.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef8409e40 con 0x555ef83f92d0 2023-07-14T11:22:38.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f614aa9ab80 10 client.0 _ll_register_callbacks cb 0x7f6124023f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:22:38.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f613579c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8409530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.815+0000 7f613579c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8409530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60884/0 (socket says 172.21.15.161:60884) 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f613579c700 1 -- 172.21.15.161:0/1073585323 learned_addr learned my addr 172.21.15.161:0/1073585323 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f6134f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef840afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f612ffff700 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f9be0 0x555ef840b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f6134f9b700 1 -- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f9be0 msgr2=0x555ef840b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f6134f9b700 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f9be0 0x555ef840b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f6134f9b700 1 -- 172.21.15.161:0/1073585323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 msgr2=0x555ef8409530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:22:38.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f6134f9b700 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8409530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f6134f9b700 1 -- 172.21.15.161:0/1073585323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c017070 con 0x555ef83f92d0 2023-07-14T11:22:38.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f613579c700 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8409530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:22:38.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f6134f9b700 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef840afd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6124022bf0 tx=0x7f61240973c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd0fe46e48806928 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:22:38.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f610ffff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:22:38.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.816+0000 7f610ffff700 1 -- 172.21.15.161:0/1073585323 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240a3440 con 0x555ef83f92d0 2023-07-14T11:22:38.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f611c021070 con 0x555ef83f92d0 2023-07-14T11:22:38.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f610ffff700 1 -- 172.21.15.161:0/1073585323 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f61240b92a0 con 0x555ef83f92d0 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f610ffff700 1 -- 172.21.15.161:0/1073585323 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f612400fa50 con 0x555ef83f92d0 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 10 client.25768 fetch_fsmap learned FSMap version 22 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555ef81a0250 con 0x555ef83f92d0 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f610ffff700 1 -- 172.21.15.161:0/1073585323 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f611c021070 con 0x555ef83f92d0 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f610ffff700 1 -- 172.21.15.161:0/1073585323 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f612409a570 con 0x555ef83f92d0 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 10 client.25768 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 10 client.25768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:22:38.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 20 client.25768 populate_metadata read hostname 'smithi161' 2023-07-14T11:22:38.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8468c00 0x555ef846afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:22:38.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 4 client.25768 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:22:38.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555ef8307050 con 0x555ef8468c00 2023-07-14T11:22:38.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.817+0000 7f612ffff700 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8468c00 0x555ef846afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:22:38.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f610ffff700 10 client.25768 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f612ffff700 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8468c00 0x555ef846afe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc6cbcb75c3d15c server_cookie=fad3a99cd0fe31ce in_seq=0 out_seq=0 2023-07-14T11:22:38.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f610ffff700 1 -- 172.21.15.161:0/1073585323 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555ef8307050 con 0x555ef8468c00 2023-07-14T11:22:38.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f610ffff700 10 client.25768 handle_command_reply: tid=0 2023-07-14T11:22:38.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 client.25768 shutdown 2023-07-14T11:22:38.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 msgr2=0x555ef840afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef840afd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6124022bf0 tx=0x7f61240973c0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 shutdown_connections 2023-07-14T11:22:38.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x555ef8468c00 0x555ef846afe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef8404ad0 0x555ef8409530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x555ef83f92d0 0x555ef840afd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 --2- 172.21.15.161:0/1073585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ef83f9be0 0x555ef840b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f610ffff700 0 client.25768 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:22:38.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.818+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 >> 172.21.15.161:0/1073585323 conn(0x555ef83efda0 msgr2=0x555ef83fb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 shutdown_connections 2023-07-14T11:22:38.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f614aa9ab80 1 -- 172.21.15.161:0/1073585323 wait complete. 2023-07-14T11:22:38.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f614aa9ab80 20 client.25768 trim_cache size 0 max 16384 2023-07-14T11:22:38.831 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:22:38.831 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:22:38.831 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:22:38.831 INFO:teuthology.orchestra.run.smithi161.stdout: "1371b9d9-2c16-46ca-8131-6177721f6b78": { 2023-07-14T11:22:38.831 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:22:38.831 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "1371b9d9-2c16-46ca-8131-6177721f6b78", 2023-07-14T11:22:38.832 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:22:38.832 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:22:38.832 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:22:38.832 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:22:38.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f61200a1350 msgr2=0x7f61200a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f61200a1350 0x7f61200a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61301993f0 tx=0x7f611c009260 comp rx=0 tx=0).stop 2023-07-14T11:22:38.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61300728b0 msgr2=0x7f6130197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:22:38.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61300728b0 0x7f6130197de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6124000c00 tx=0x7f6124005740 comp rx=0 tx=0).stop 2023-07-14T11:22:38.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 shutdown_connections 2023-07-14T11:22:38.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f61200a1350 0x7f61200a3800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613010c310 0x7f613019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61300728b0 0x7f6130197de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 --2- 172.21.15.161:0/2373101709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f61300731c0 0x7f6130198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:22:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.819+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 >> 172.21.15.161:0/2373101709 conn(0x7f61300fb5f0 msgr2=0x7f613010e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:22:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.820+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 shutdown_connections 2023-07-14T11:22:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:22:38.820+0000 7f61371ff700 1 -- 172.21.15.161:0/2373101709 wait complete. 2023-07-14T11:22:38.842 INFO:tasks.cephfs.filesystem:scrub status for tag:1371b9d9-2c16-46ca-8131-6177721f6b78 - {'path': '/', 'tag': '1371b9d9-2c16-46ca-8131-6177721f6b78', 'options': 'recursive,force'} 2023-07-14T11:22:39.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:39 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:22:39.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:39 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:39.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:39 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:22:39.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:39 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:39.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:39 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:22:39.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:39 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:39.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:39 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:22:39.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:39 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:39.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:39 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:22:39.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:39 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:39.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:39 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:22:39.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:39 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:22:40.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:40 smithi175 ceph-mon[129081]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:22:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:40 smithi182 ceph-mon[129366]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:22:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:40 smithi161 ceph-mon[95526]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T11:22:41.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:41 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:22:41.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:41 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:22:41.495 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:41 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:22:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:42 smithi175 ceph-mon[129081]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2023-07-14T11:22:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:42 smithi182 ceph-mon[129366]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2023-07-14T11:22:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:42 smithi161 ceph-mon[95526]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2023-07-14T11:22:43.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:43 smithi161 ceph-mon[95526]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:43 smithi175 ceph-mon[129081]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:43 smithi182 ceph-mon[129366]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:46.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:46 smithi175 ceph-mon[129081]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:46 smithi182 ceph-mon[129366]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:46 smithi161 ceph-mon[95526]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:48.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:48 smithi175 ceph-mon[129081]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-14T11:22:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:48 smithi182 ceph-mon[129366]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-14T11:22:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:48 smithi161 ceph-mon[95526]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-14T11:22:50.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:50 smithi175 ceph-mon[129081]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:50 smithi182 ceph-mon[129366]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:50 smithi161 ceph-mon[95526]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:22:52.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:52 smithi175 ceph-mon[129081]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2023-07-14T11:22:52.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:52 smithi182 ceph-mon[129366]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2023-07-14T11:22:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:52 smithi161 ceph-mon[95526]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2023-07-14T11:22:53.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:53 smithi161 ceph-mon[95526]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:53 smithi175 ceph-mon[129081]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:54.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:53 smithi182 ceph-mon[129366]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:56 smithi175 ceph-mon[129081]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:56 smithi182 ceph-mon[129366]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:56 smithi161 ceph-mon[95526]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:22:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:22:58 smithi175 ceph-mon[129081]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-07-14T11:22:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:22:58 smithi182 ceph-mon[129366]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-07-14T11:22:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:22:58 smithi161 ceph-mon[95526]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-07-14T11:23:00.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:00 smithi175 ceph-mon[129081]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:23:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:00 smithi182 ceph-mon[129366]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:23:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:00 smithi161 ceph-mon[95526]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-07-14T11:23:00.996 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:23:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:23:00.660+0000 7f32c76f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:23:00.997 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:23:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:23:00.660+0000 7f32c76f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:23:00.997 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:23:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:23:00.660+0000 7f32c76f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:23:00.997 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:23:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:23:00.660+0000 7f32c76f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:23:00.998 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:23:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:23:00.660+0000 7f32c76f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T11:23:02.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:02 smithi175 ceph-mon[129081]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:02.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:02 smithi175 ceph-mon[129081]: mgrmap e24: x(active, since 46s), standbys: y, z 2023-07-14T11:23:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:02 smithi182 ceph-mon[129366]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:02 smithi182 ceph-mon[129366]: mgrmap e24: x(active, since 46s), standbys: y, z 2023-07-14T11:23:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:02 smithi161 ceph-mon[95526]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:02 smithi161 ceph-mon[95526]: mgrmap e24: x(active, since 46s), standbys: y, z 2023-07-14T11:23:03.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:03 smithi161 ceph-mon[95526]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-07-14T11:23:04.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:03 smithi175 ceph-mon[129081]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-07-14T11:23:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:03 smithi182 ceph-mon[129366]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-07-14T11:23:06.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:06 smithi175 ceph-mon[129081]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-07-14T11:23:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:06 smithi182 ceph-mon[129366]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-07-14T11:23:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:06 smithi161 ceph-mon[95526]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-07-14T11:23:08.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:08 smithi175 ceph-mon[129081]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:08.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:08 smithi182 ceph-mon[129366]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:08 smithi161 ceph-mon[95526]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:08.843 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:23:09.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.181+0000 7fc4607ff700 1 -- 172.21.15.161:0/2149714061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc458105ae0 msgr2=0x7fc458105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.181+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2149714061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc458105ae0 0x7fc458105eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc4540098f0 tx=0x7fc454009c00 comp rx=0 tx=0).stop 2023-07-14T11:23:09.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.181+0000 7fc4607ff700 1 -- 172.21.15.161:0/2149714061 shutdown_connections 2023-07-14T11:23:09.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.181+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2149714061 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc458100480 0x7fc45810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.181+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2149714061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc458105ae0 0x7fc458105eb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.181+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2149714061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4580ffad0 0x7fc4580fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.181+0000 7fc4607ff700 1 -- 172.21.15.161:0/2149714061 >> 172.21.15.161:0/2149714061 conn(0x7fc4580fb5f0 msgr2=0x7fc4580fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:09.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.182+0000 7fc4607ff700 1 -- 172.21.15.161:0/2149714061 shutdown_connections 2023-07-14T11:23:09.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.182+0000 7fc4607ff700 1 -- 172.21.15.161:0/2149714061 wait complete. 2023-07-14T11:23:09.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.183+0000 7fc4607ff700 1 Processor -- start 2023-07-14T11:23:09.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.183+0000 7fc4607ff700 1 -- start start 2023-07-14T11:23:09.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc4607ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4580ffad0 0x7fc458197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc4607ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc458100480 0x7fc458198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc4607ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc458105ae0 0x7fc45819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc4607ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45819cdf0 con 0x7fc458100480 2023-07-14T11:23:09.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc4607ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45819cf60 con 0x7fc4580ffad0 2023-07-14T11:23:09.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc4607ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45819d120 con 0x7fc458105ae0 2023-07-14T11:23:09.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc45dd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc458100480 0x7fc458198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc45ed9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc458105ae0 0x7fc45819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc45e59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4580ffad0 0x7fc458197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc45ed9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc458105ae0 0x7fc45819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33818/0 (socket says 172.21.15.161:33818) 2023-07-14T11:23:09.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.184+0000 7fc45ed9c700 1 -- 172.21.15.161:0/2909880041 learned_addr learned my addr 172.21.15.161:0/2909880041 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:09.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.185+0000 7fc45e59b700 1 -- 172.21.15.161:0/2909880041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc458105ae0 msgr2=0x7fc45819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.185+0000 7fc45e59b700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc458105ae0 0x7fc45819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.185+0000 7fc45e59b700 1 -- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc458100480 msgr2=0x7fc458198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.185+0000 7fc45e59b700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc458100480 0x7fc458198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.185+0000 7fc45e59b700 1 -- 172.21.15.161:0/2909880041 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480095f0 con 0x7fc4580ffad0 2023-07-14T11:23:09.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.185+0000 7fc45e59b700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4580ffad0 0x7fc458197e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc454009810 tx=0x7fc45400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:09.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.186+0000 7fc44f7fe700 1 -- 172.21.15.161:0/2909880041 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45401c070 con 0x7fc4580ffad0 2023-07-14T11:23:09.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.186+0000 7fc44f7fe700 1 -- 172.21.15.161:0/2909880041 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc454005040 con 0x7fc4580ffad0 2023-07-14T11:23:09.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.186+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc454009520 con 0x7fc4580ffad0 2023-07-14T11:23:09.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.186+0000 7fc44f7fe700 1 -- 172.21.15.161:0/2909880041 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540038e0 con 0x7fc4580ffad0 2023-07-14T11:23:09.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.186+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4581a4a60 con 0x7fc4580ffad0 2023-07-14T11:23:09.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.191+0000 7fc44f7fe700 1 -- 172.21.15.161:0/2909880041 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7fc454020030 con 0x7fc4580ffad0 2023-07-14T11:23:09.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.192+0000 7fc4741cbb80 1 Processor -- start 2023-07-14T11:23:09.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.193+0000 7fc44f7fe700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4440a1460 0x7fc4440a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.194+0000 7fc44f7fe700 1 -- 172.21.15.161:0/2909880041 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc4540ce0f0 con 0x7fc4580ffad0 2023-07-14T11:23:09.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.194+0000 7fc4741cbb80 1 -- start start 2023-07-14T11:23:09.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.194+0000 7fc45ed9c700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4440a1460 0x7fc4440a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed23da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed242e0 0x557b9ed266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ec16cb0 con 0x557b9ed10be0 2023-07-14T11:23:09.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ec69a40 con 0x557b9ed102d0 2023-07-14T11:23:09.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ec62ed0 con 0x557b9ed242e0 2023-07-14T11:23:09.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc45ed9c700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4440a1460 0x7fc4440a3910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc450009c10 tx=0x7fc45000b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:09.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.195+0000 7fc45dd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed23da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45dd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed23da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39452/0 (socket says 172.21.15.161:39452) 2023-07-14T11:23:09.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45dd9a700 1 -- 172.21.15.161:0/3412617615 learned_addr learned my addr 172.21.15.161:0/3412617615 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:09.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45e59b700 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45e59b700 1 -- 172.21.15.161:0/3412617615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed242e0 msgr2=0x557b9ed266c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45e59b700 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed242e0 0x557b9ed266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45e59b700 1 -- 172.21.15.161:0/3412617615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 msgr2=0x557b9ed23da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45e59b700 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed23da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45e59b700 1 -- 172.21.15.161:0/3412617615 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9ea90de0 con 0x557b9ed102d0 2023-07-14T11:23:09.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.196+0000 7fc45dd9a700 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed23da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:09.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.197+0000 7fc45e59b700 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed106a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc4540cb330 tx=0x7fc45409ab50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf056e769f1bf698 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:09.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.197+0000 7fc44cff9700 1 -- 172.21.15.161:0/3412617615 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45409ac70 con 0x557b9ed102d0 2023-07-14T11:23:09.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.197+0000 7fc44cff9700 1 -- 172.21.15.161:0/3412617615 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc45409eca0 con 0x557b9ed102d0 2023-07-14T11:23:09.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.197+0000 7fc44cff9700 1 -- 172.21.15.161:0/3412617615 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454005400 con 0x557b9ed102d0 2023-07-14T11:23:09.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 -- 172.21.15.161:0/3412617615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 msgr2=0x557b9ed106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed106a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc4540cb330 tx=0x7fc45409ab50 comp rx=0 tx=0).stop 2023-07-14T11:23:09.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 -- 172.21.15.161:0/3412617615 shutdown_connections 2023-07-14T11:23:09.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed242e0 0x557b9ed266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed106a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/3412617615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed23da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 -- 172.21.15.161:0/3412617615 >> 172.21.15.161:0/3412617615 conn(0x557b9ed06da0 msgr2=0x557b9ed0c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:09.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 -- 172.21.15.161:0/3412617615 shutdown_connections 2023-07-14T11:23:09.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.198+0000 7fc4741cbb80 1 -- 172.21.15.161:0/3412617615 wait complete. 2023-07-14T11:23:09.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.199+0000 7fc4741cbb80 1 -- start start 2023-07-14T11:23:09.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.199+0000 7fc4741cbb80 1 Processor -- start 2023-07-14T11:23:09.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.200+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed29400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.200+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed29940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.200+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.200+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ec02ac0 con 0x557b9ed10be0 2023-07-14T11:23:09.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.200+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ec027c0 con 0x557b9ed242e0 2023-07-14T11:23:09.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.200+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9eae6920 con 0x557b9ed102d0 2023-07-14T11:23:09.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.200+0000 7fc4741cbb80 10 client.0 _ll_register_callbacks cb 0x557b9ec00760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:09.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45e59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed29400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45e59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed29400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33828/0 (socket says 172.21.15.161:33828) 2023-07-14T11:23:09.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45e59b700 1 -- 172.21.15.161:0/206474405 learned_addr learned my addr 172.21.15.161:0/206474405 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:09.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45dd9a700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed29940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45ed9c700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45e59b700 1 -- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 msgr2=0x557b9ed23270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45e59b700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed23270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.201+0000 7fc45e59b700 1 -- 172.21.15.161:0/206474405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 msgr2=0x557b9ed29940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc45e59b700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed29940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc45e59b700 1 -- 172.21.15.161:0/206474405 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc448009bc0 con 0x557b9ed102d0 2023-07-14T11:23:09.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc45dd9a700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed29940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:09.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc45e59b700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed29400 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc454005bc0 tx=0x7fc4540a3030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a28cc1bc03d5e845 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:09.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc441ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:09.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc441ffb700 1 -- 172.21.15.161:0/206474405 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540b3210 con 0x557b9ed102d0 2023-07-14T11:23:09.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc441ffb700 1 -- 172.21.15.161:0/206474405 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc4540b3370 con 0x557b9ed102d0 2023-07-14T11:23:09.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b9ebbc4f0 con 0x557b9ed102d0 2023-07-14T11:23:09.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.202+0000 7fc441ffb700 1 -- 172.21.15.161:0/206474405 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc454005400 con 0x557b9ed102d0 2023-07-14T11:23:09.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 10 client.25783 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:09.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7fc4581a4a60 con 0x557b9ed102d0 2023-07-14T11:23:09.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc441ffb700 1 -- 172.21.15.161:0/206474405 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b9ebbc4f0 con 0x557b9ed102d0 2023-07-14T11:23:09.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc441ffb700 1 -- 172.21.15.161:0/206474405 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc45409a1f0 con 0x557b9ed102d0 2023-07-14T11:23:09.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 10 client.25783 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:09.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:09.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 20 client.25783 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:09.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed33360 0x557b9ed35770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 4 client.25783 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:09.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.203+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b9ec5beb0 con 0x557b9ed33360 2023-07-14T11:23:09.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.204+0000 7fc45dd9a700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed33360 0x557b9ed35770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.204+0000 7fc45dd9a700 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed33360 0x557b9ed35770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915f2dee3a067174 server_cookie=da8283817308b88f in_seq=0 out_seq=0 2023-07-14T11:23:09.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.204+0000 7fc441ffb700 10 client.25783 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:09.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.205+0000 7fc441ffb700 1 -- 172.21.15.161:0/206474405 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557b9ec5beb0 con 0x557b9ed33360 2023-07-14T11:23:09.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.205+0000 7fc441ffb700 10 client.25783 handle_command_reply: tid=0 2023-07-14T11:23:09.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.205+0000 7fc4741cbb80 1 client.25783 shutdown 2023-07-14T11:23:09.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 msgr2=0x557b9ed29400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed29400 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc454005bc0 tx=0x7fc4540a3030 comp rx=0 tx=0).stop 2023-07-14T11:23:09.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 shutdown_connections 2023-07-14T11:23:09.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed33360 0x557b9ed35770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed23270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed29400 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/206474405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed29940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 >> 172.21.15.161:0/206474405 conn(0x557b9ed06da0 msgr2=0x557b9ed0c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:09.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc441ffb700 0 client.25783 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:09.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 shutdown_connections 2023-07-14T11:23:09.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 1 -- 172.21.15.161:0/206474405 wait complete. 2023-07-14T11:23:09.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.206+0000 7fc4741cbb80 20 client.25783 trim_cache size 0 max 16384 2023-07-14T11:23:09.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.239+0000 7fc4741cbb80 1 Processor -- start 2023-07-14T11:23:09.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.240+0000 7fc4741cbb80 1 -- start start 2023-07-14T11:23:09.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.240+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.240+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed10be0 0x557b9ed28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.240+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed242e0 0x557b9ed294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.240+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ed0dad0 con 0x557b9ed242e0 2023-07-14T11:23:09.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.240+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ed11540 con 0x557b9ed102d0 2023-07-14T11:23:09.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.240+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ed11d60 con 0x557b9ed10be0 2023-07-14T11:23:09.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45dd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed10be0 0x557b9ed28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45ed9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed242e0 0x557b9ed294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45ed9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed242e0 0x557b9ed294d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39458/0 (socket says 172.21.15.161:39458) 2023-07-14T11:23:09.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45ed9c700 1 -- 172.21.15.161:0/1726237843 learned_addr learned my addr 172.21.15.161:0/1726237843 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:09.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45e59b700 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45e59b700 1 -- 172.21.15.161:0/1726237843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed10be0 msgr2=0x557b9ed28f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45e59b700 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed10be0 0x557b9ed28f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45e59b700 1 -- 172.21.15.161:0/1726237843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed242e0 msgr2=0x557b9ed294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45e59b700 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed242e0 0x557b9ed294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.241+0000 7fc45e59b700 1 -- 172.21.15.161:0/1726237843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9ed23ef0 con 0x557b9ed102d0 2023-07-14T11:23:09.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.242+0000 7fc45e59b700 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed23220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc45409ac00 tx=0x7fc4540b90d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39916b2f8de30894 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:09.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.242+0000 7fc44d7fa700 1 -- 172.21.15.161:0/1726237843 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540cb270 con 0x557b9ed102d0 2023-07-14T11:23:09.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.242+0000 7fc44d7fa700 1 -- 172.21.15.161:0/1726237843 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4540a30a0 con 0x557b9ed102d0 2023-07-14T11:23:09.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.242+0000 7fc44d7fa700 1 -- 172.21.15.161:0/1726237843 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454005400 con 0x557b9ed102d0 2023-07-14T11:23:09.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 -- 172.21.15.161:0/1726237843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 msgr2=0x557b9ed23220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed23220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc45409ac00 tx=0x7fc4540b90d0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 -- 172.21.15.161:0/1726237843 shutdown_connections 2023-07-14T11:23:09.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed242e0 0x557b9ed294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed102d0 0x557b9ed23220 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/1726237843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed10be0 0x557b9ed28f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 -- 172.21.15.161:0/1726237843 >> 172.21.15.161:0/1726237843 conn(0x557b9ed06da0 msgr2=0x557b9ed22b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:09.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 -- 172.21.15.161:0/1726237843 shutdown_connections 2023-07-14T11:23:09.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.243+0000 7fc4741cbb80 1 -- 172.21.15.161:0/1726237843 wait complete. 2023-07-14T11:23:09.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.244+0000 7fc4741cbb80 1 -- start start 2023-07-14T11:23:09.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.244+0000 7fc4741cbb80 1 Processor -- start 2023-07-14T11:23:09.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed7fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed80210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc4741cbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed28d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ed29320 con 0x557b9ed10be0 2023-07-14T11:23:09.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ed29490 con 0x557b9ed242e0 2023-07-14T11:23:09.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc4741cbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9ed29650 con 0x557b9ed102d0 2023-07-14T11:23:09.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc4741cbb80 10 client.0 _ll_register_callbacks cb 0x7fc448009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:09.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc45ed9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed28d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc45ed9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed28d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45816/0 (socket says 172.21.15.161:45816) 2023-07-14T11:23:09.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc45ed9c700 1 -- 172.21.15.161:0/2745468121 learned_addr learned my addr 172.21.15.161:0/2745468121 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:09.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.245+0000 7fc45dd9a700 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed80210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc45e59b700 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed7fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc45ed9c700 1 -- 172.21.15.161:0/2745468121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 msgr2=0x557b9ed7fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc45ed9c700 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed7fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc45ed9c700 1 -- 172.21.15.161:0/2745468121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 msgr2=0x557b9ed80210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:09.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc45ed9c700 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed80210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc45ed9c700 1 -- 172.21.15.161:0/2745468121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4581a4a60 con 0x557b9ed242e0 2023-07-14T11:23:09.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc45ed9c700 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed28d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc450015ea0 tx=0x7fc450009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dfc34cb99edc282 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:09.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc441ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:23:09.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.246+0000 7fc441ffb700 1 -- 172.21.15.161:0/2745468121 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000e050 con 0x557b9ed242e0 2023-07-14T11:23:09.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.247+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b9ec1e050 con 0x557b9ed242e0 2023-07-14T11:23:09.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.247+0000 7fc441ffb700 1 -- 172.21.15.161:0/2745468121 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc450016de0 con 0x557b9ed242e0 2023-07-14T11:23:09.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.247+0000 7fc441ffb700 1 -- 172.21.15.161:0/2745468121 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc450024430 con 0x557b9ed242e0 2023-07-14T11:23:09.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.247+0000 7fc4741cbb80 10 client.25787 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.247+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557b9ea79c60 con 0x557b9ed242e0 2023-07-14T11:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.247+0000 7fc441ffb700 1 -- 172.21.15.161:0/2745468121 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b9ec1e050 con 0x557b9ed242e0 2023-07-14T11:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc441ffb700 1 -- 172.21.15.161:0/2745468121 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc4500236d0 con 0x557b9ed242e0 2023-07-14T11:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc4741cbb80 10 client.25787 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc4741cbb80 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc4741cbb80 20 client.25787 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed87b90 0x557b9ed89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc4741cbb80 4 client.25787 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:23:09.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b9eab1f70 con 0x557b9ed87b90 2023-07-14T11:23:09.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.248+0000 7fc45e59b700 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed87b90 0x557b9ed89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.249+0000 7fc45e59b700 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed87b90 0x557b9ed89fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b739e21d958e2b4 server_cookie=5d74a4e5341a814d in_seq=0 out_seq=0 2023-07-14T11:23:09.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.249+0000 7fc441ffb700 10 client.25787 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:09.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.249+0000 7fc441ffb700 1 -- 172.21.15.161:0/2745468121 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557b9eab1f70 con 0x557b9ed87b90 2023-07-14T11:23:09.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.249+0000 7fc441ffb700 10 client.25787 handle_command_reply: tid=0 2023-07-14T11:23:09.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.249+0000 7fc4741cbb80 1 client.25787 shutdown 2023-07-14T11:23:09.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.249+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 msgr2=0x557b9ed28d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.249+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed28d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc450015ea0 tx=0x7fc450009920 comp rx=0 tx=0).stop 2023-07-14T11:23:09.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 shutdown_connections 2023-07-14T11:23:09.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557b9ed87b90 0x557b9ed89fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b9ed242e0 0x557b9ed28d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557b9ed102d0 0x557b9ed7fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 --2- 172.21.15.161:0/2745468121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b9ed10be0 0x557b9ed80210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 >> 172.21.15.161:0/2745468121 conn(0x557b9ed06da0 msgr2=0x557b9ed12a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:09.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc441ffb700 0 client.25787 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:09.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 shutdown_connections 2023-07-14T11:23:09.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 1 -- 172.21.15.161:0/2745468121 wait complete. 2023-07-14T11:23:09.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.250+0000 7fc4741cbb80 20 client.25787 trim_cache size 0 max 16384 2023-07-14T11:23:09.261 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:23:09.261 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:23:09.261 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:23:09.262 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:23:09.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4440a1460 msgr2=0x7fc4440a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4440a1460 0x7fc4440a3910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc450009c10 tx=0x7fc45000b040 comp rx=0 tx=0).stop 2023-07-14T11:23:09.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4580ffad0 msgr2=0x7fc458197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4580ffad0 0x7fc458197e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc454009810 tx=0x7fc45400fa00 comp rx=0 tx=0).stop 2023-07-14T11:23:09.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 shutdown_connections 2023-07-14T11:23:09.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4440a1460 0x7fc4440a3910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc458105ae0 0x7fc45819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4580ffad0 0x7fc458197e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 --2- 172.21.15.161:0/2909880041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc458100480 0x7fc458198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 >> 172.21.15.161:0/2909880041 conn(0x7fc4580fb5f0 msgr2=0x7fc45810ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:09.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 shutdown_connections 2023-07-14T11:23:09.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.251+0000 7fc4607ff700 1 -- 172.21.15.161:0/2909880041 wait complete. 2023-07-14T11:23:09.282 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:23:09.283 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:23:09.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 -- 172.21.15.161:0/614041859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 msgr2=0x7f5e2010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 --2- 172.21.15.161:0/614041859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 0x7f5e2010e610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009f90 tx=0x7f5e1800b330 comp rx=0 tx=0).stop 2023-07-14T11:23:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 -- 172.21.15.161:0/614041859 shutdown_connections 2023-07-14T11:23:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 --2- 172.21.15.161:0/614041859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e200728b0 0x7f5e20072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 --2- 172.21.15.161:0/614041859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 0x7f5e2010e610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 --2- 172.21.15.161:0/614041859 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5e200731c0 0x7f5e2010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 -- 172.21.15.161:0/614041859 >> 172.21.15.161:0/614041859 conn(0x7f5e200fb510 msgr2=0x7f5e200fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 -- 172.21.15.161:0/614041859 shutdown_connections 2023-07-14T11:23:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.614+0000 7f5e2719d700 1 -- 172.21.15.161:0/614041859 wait complete. 2023-07-14T11:23:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 Processor -- start 2023-07-14T11:23:09.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 -- start start 2023-07-14T11:23:09.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e200728b0 0x7f5e2019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5e200731c0 0x7f5e2019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 0x7f5e20197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20198210 con 0x7f5e200728b0 2023-07-14T11:23:09.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20198380 con 0x7f5e2010c230 2023-07-14T11:23:09.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.616+0000 7f5e2719d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20198540 con 0x7f5e200731c0 2023-07-14T11:23:09.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.617+0000 7f5e2619b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e200728b0 0x7f5e2019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.617+0000 7f5e2619b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e200728b0 0x7f5e2019da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39490/0 (socket says 172.21.15.161:39490) 2023-07-14T11:23:09.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.617+0000 7f5e2619b700 1 -- 172.21.15.161:0/2431934300 learned_addr learned my addr 172.21.15.161:0/2431934300 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:09.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.617+0000 7f5e2699c700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 0x7f5e20197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.617+0000 7f5e2599a700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5e200731c0 0x7f5e2019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.617+0000 7f5e2699c700 1 -- 172.21.15.161:0/2431934300 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5e200731c0 msgr2=0x7f5e2019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.617+0000 7f5e2699c700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5e200731c0 0x7f5e2019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.618+0000 7f5e2699c700 1 -- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e200728b0 msgr2=0x7f5e2019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.618+0000 7f5e2699c700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e200728b0 0x7f5e2019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.618+0000 7f5e2699c700 1 -- 172.21.15.161:0/2431934300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c0095f0 con 0x7f5e2010c230 2023-07-14T11:23:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.618+0000 7f5e2699c700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 0x7f5e20197a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800b6c0 tx=0x7f5e180094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.618+0000 7f5e177fe700 1 -- 172.21.15.161:0/2431934300 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800e040 con 0x7f5e2010c230 2023-07-14T11:23:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.618+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e18009c40 con 0x7f5e2010c230 2023-07-14T11:23:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.619+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e200ffa60 con 0x7f5e2010c230 2023-07-14T11:23:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.619+0000 7f5e177fe700 1 -- 172.21.15.161:0/2431934300 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e18009ac0 con 0x7f5e2010c230 2023-07-14T11:23:09.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.619+0000 7f5e177fe700 1 -- 172.21.15.161:0/2431934300 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1801de30 con 0x7f5e2010c230 2023-07-14T11:23:09.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.620+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e04005320 con 0x7f5e2010c230 2023-07-14T11:23:09.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.624+0000 7f5e177fe700 1 -- 172.21.15.161:0/2431934300 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f5e1801d470 con 0x7f5e2010c230 2023-07-14T11:23:09.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.627+0000 7f5e177fe700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5e0c0a1550 0x7f5e0c0a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:09.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.627+0000 7f5e177fe700 1 -- 172.21.15.161:0/2431934300 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f5e180ce770 con 0x7f5e2010c230 2023-07-14T11:23:09.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.627+0000 7f5e2619b700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5e0c0a1550 0x7f5e0c0a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:09.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.627+0000 7f5e177fe700 1 -- 172.21.15.161:0/2431934300 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5e180ceb90 con 0x7f5e2010c230 2023-07-14T11:23:09.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:09.628+0000 7f5e2619b700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5e0c0a1550 0x7f5e0c0a3a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c005d10 tx=0x7f5e1c009310 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.116+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e04005190 con 0x7f5e2010c230 2023-07-14T11:23:10.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.121+0000 7f5e177fe700 1 -- 172.21.15.161:0/2431934300 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f5e18097630 con 0x7f5e2010c230 2023-07-14T11:23:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:23:10.125 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:20:12.668182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:23:10.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5e0c0a1550 msgr2=0x7f5e0c0a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5e0c0a1550 0x7f5e0c0a3a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c005d10 tx=0x7f5e1c009310 comp rx=0 tx=0).stop 2023-07-14T11:23:10.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 msgr2=0x7f5e20197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 0x7f5e20197a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800b6c0 tx=0x7f5e180094b0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 shutdown_connections 2023-07-14T11:23:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5e0c0a1550 0x7f5e0c0a3a00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e200728b0 0x7f5e2019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2010c230 0x7f5e20197a40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 --2- 172.21.15.161:0/2431934300 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5e200731c0 0x7f5e2019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 >> 172.21.15.161:0/2431934300 conn(0x7f5e200fb510 msgr2=0x7f5e20103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 shutdown_connections 2023-07-14T11:23:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.124+0000 7f5e2719d700 1 -- 172.21.15.161:0/2431934300 wait complete. 2023-07-14T11:23:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 22 2023-07-14T11:23:10.157 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:23:10.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:10 smithi175 ceph-mon[129081]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-07-14T11:23:10.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:10 smithi182 ceph-mon[129366]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-07-14T11:23:10.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:10 smithi161 ceph-mon[95526]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-07-14T11:23:10.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 -- 172.21.15.161:0/1934844693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 msgr2=0x7efecc105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 --2- 172.21.15.161:0/1934844693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 0x7efecc105eb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efebc00b0a0 tx=0x7efebc00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 -- 172.21.15.161:0/1934844693 shutdown_connections 2023-07-14T11:23:10.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 --2- 172.21.15.161:0/1934844693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc100480 0x7efecc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 --2- 172.21.15.161:0/1934844693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 0x7efecc105eb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 --2- 172.21.15.161:0/1934844693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc0ffad0 0x7efecc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 -- 172.21.15.161:0/1934844693 >> 172.21.15.161:0/1934844693 conn(0x7efecc0fb5f0 msgr2=0x7efecc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 -- 172.21.15.161:0/1934844693 shutdown_connections 2023-07-14T11:23:10.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.495+0000 7efed13ff700 1 -- 172.21.15.161:0/1934844693 wait complete. 2023-07-14T11:23:10.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.497+0000 7efed13ff700 1 Processor -- start 2023-07-14T11:23:10.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.497+0000 7efed13ff700 1 -- start start 2023-07-14T11:23:10.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efed13ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc0ffad0 0x7efecc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efed13ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc100480 0x7efecc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efed13ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 0x7efecc19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efed13ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc19cf30 con 0x7efecc0ffad0 2023-07-14T11:23:10.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efed13ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc19d0a0 con 0x7efecc105ae0 2023-07-14T11:23:10.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efed13ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc19d260 con 0x7efecc100480 2023-07-14T11:23:10.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efeca7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc100480 0x7efecc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efeca7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc100480 0x7efecc198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33894/0 (socket says 172.21.15.161:33894) 2023-07-14T11:23:10.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efeca7fc700 1 -- 172.21.15.161:0/2352701337 learned_addr learned my addr 172.21.15.161:0/2352701337 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efecb7fe700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 0x7efecc19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.498+0000 7efecaffd700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc0ffad0 0x7efecc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeca7fc700 1 -- 172.21.15.161:0/2352701337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 msgr2=0x7efecc19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeca7fc700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 0x7efecc19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeca7fc700 1 -- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc0ffad0 msgr2=0x7efecc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeca7fc700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc0ffad0 0x7efecc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeca7fc700 1 -- 172.21.15.161:0/2352701337 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc009d00 con 0x7efecc100480 2023-07-14T11:23:10.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeca7fc700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc100480 0x7efecc198490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9a0 tx=0x7efec000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeb3fff700 1 -- 172.21.15.161:0/2352701337 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0009790 con 0x7efecc100480 2023-07-14T11:23:10.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc19d540 con 0x7efecc100480 2023-07-14T11:23:10.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeb3fff700 1 -- 172.21.15.161:0/2352701337 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efec000cbd0 con 0x7efecc100480 2023-07-14T11:23:10.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.499+0000 7efeb3fff700 1 -- 172.21.15.161:0/2352701337 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0018680 con 0x7efecc100480 2023-07-14T11:23:10.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.500+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc1a4d80 con 0x7efecc100480 2023-07-14T11:23:10.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.504+0000 7efeb3fff700 1 -- 172.21.15.161:0/2352701337 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7efec000cd40 con 0x7efecc100480 2023-07-14T11:23:10.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.506+0000 7efee4d3bb80 1 Processor -- start 2023-07-14T11:23:10.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.507+0000 7efee4d3bb80 1 -- start start 2023-07-14T11:23:10.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.507+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bf26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.507+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.507+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 0x55de11bffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.507+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11af8cb0 con 0x55de11bf22d0 2023-07-14T11:23:10.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.507+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11b4ba40 con 0x55de11bf2be0 2023-07-14T11:23:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.507+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11b44ed0 con 0x55de11bfdad0 2023-07-14T11:23:10.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efeb3fff700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7efeb40a1460 0x7efeb40a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bf26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bf26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39522/0 (socket says 172.21.15.161:39522) 2023-07-14T11:23:10.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecb7fe700 1 -- 172.21.15.161:0/927196015 learned_addr learned my addr 172.21.15.161:0/927196015 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecaffd700 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efeb3fff700 1 -- 172.21.15.161:0/2352701337 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7efec0014070 con 0x7efecc100480 2023-07-14T11:23:10.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecaffd700 1 -- 172.21.15.161:0/927196015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 msgr2=0x55de11bffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecaffd700 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 0x55de11bffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecaffd700 1 -- 172.21.15.161:0/927196015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 msgr2=0x55de11bf26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecaffd700 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bf26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecb7fe700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7efeb40a1460 0x7efeb40a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.508+0000 7efecaffd700 1 -- 172.21.15.161:0/927196015 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de11972de0 con 0x55de11bf2be0 2023-07-14T11:23:10.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.509+0000 7efecaffd700 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfd590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efebc009fd0 tx=0x7efebc009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8217b3efbbefe2b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.509+0000 7efeb17fa700 1 -- 172.21.15.161:0/927196015 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00e050 con 0x55de11bf2be0 2023-07-14T11:23:10.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.509+0000 7efeb17fa700 1 -- 172.21.15.161:0/927196015 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efebc009460 con 0x55de11bf2be0 2023-07-14T11:23:10.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.509+0000 7efeb17fa700 1 -- 172.21.15.161:0/927196015 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc005060 con 0x55de11bf2be0 2023-07-14T11:23:10.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efee4d3bb80 1 -- 172.21.15.161:0/927196015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 msgr2=0x55de11bfd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efecb7fe700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7efeb40a1460 0x7efeb40a3910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efec4004100 tx=0x7efec40047c0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfd590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efebc009fd0 tx=0x7efebc009c80 comp rx=0 tx=0).stop 2023-07-14T11:23:10.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efee4d3bb80 1 -- 172.21.15.161:0/927196015 shutdown_connections 2023-07-14T11:23:10.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bf26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 0x55de11bffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/927196015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfd590 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.510+0000 7efee4d3bb80 1 -- 172.21.15.161:0/927196015 >> 172.21.15.161:0/927196015 conn(0x55de11be8da0 msgr2=0x55de11bee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.511+0000 7efee4d3bb80 1 -- 172.21.15.161:0/927196015 shutdown_connections 2023-07-14T11:23:10.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.511+0000 7efee4d3bb80 1 -- 172.21.15.161:0/927196015 wait complete. 2023-07-14T11:23:10.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.512+0000 7efee4d3bb80 1 -- start start 2023-07-14T11:23:10.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.512+0000 7efee4d3bb80 1 Processor -- start 2023-07-14T11:23:10.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bfd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 0x55de11bfaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11ae4ac0 con 0x55de11bf22d0 2023-07-14T11:23:10.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11ae47c0 con 0x55de11bf2be0 2023-07-14T11:23:10.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de119c8920 con 0x55de11bfdad0 2023-07-14T11:23:10.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efee4d3bb80 10 client.0 _ll_register_callbacks cb 0x55de11ae2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:10.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efecaffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bfd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efecaffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bfd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39532/0 (socket says 172.21.15.161:39532) 2023-07-14T11:23:10.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efecaffd700 1 -- 172.21.15.161:0/206099675 learned_addr learned my addr 172.21.15.161:0/206099675 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efeca7fc700 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.513+0000 7efecb7fe700 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 0x55de11bfaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.514+0000 7efecaffd700 1 -- 172.21.15.161:0/206099675 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 msgr2=0x55de11bfaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.514+0000 7efecaffd700 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 0x55de11bfaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.514+0000 7efecaffd700 1 -- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 msgr2=0x55de11bfa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.514+0000 7efecaffd700 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.514+0000 7efecaffd700 1 -- 172.21.15.161:0/206099675 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc10d210 con 0x55de11bf22d0 2023-07-14T11:23:10.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.514+0000 7efecaffd700 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bfd530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efebc00b880 tx=0x7efebc00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39efaaabe39e1872 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.514+0000 7efeaeff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:10.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.515+0000 7efeaeff5700 1 -- 172.21.15.161:0/206099675 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc01b430 con 0x55de11bf22d0 2023-07-14T11:23:10.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.515+0000 7efeaeff5700 1 -- 172.21.15.161:0/206099675 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7efebc01b590 con 0x55de11bf22d0 2023-07-14T11:23:10.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.515+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de11a9e4f0 con 0x55de11bf22d0 2023-07-14T11:23:10.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.515+0000 7efeaeff5700 1 -- 172.21.15.161:0/206099675 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efebc01b750 con 0x55de11bf22d0 2023-07-14T11:23:10.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.515+0000 7efee4d3bb80 10 client.15768 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:10.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.515+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55de1195bc60 con 0x55de11bf22d0 2023-07-14T11:23:10.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.515+0000 7efeaeff5700 1 -- 172.21.15.161:0/206099675 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de11a9e4f0 con 0x55de11bf22d0 2023-07-14T11:23:10.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.516+0000 7efeaeff5700 1 -- 172.21.15.161:0/206099675 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efebc00b0d0 con 0x55de11bf22d0 2023-07-14T11:23:10.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.517+0000 7efee4d3bb80 10 client.15768 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:10.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.517+0000 7efee4d3bb80 10 client.15768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:23:10.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.517+0000 7efee4d3bb80 20 client.15768 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:10.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.517+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c15750 0x55de11c17b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.517+0000 7efee4d3bb80 4 client.15768 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:10.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.517+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de11b3deb0 con 0x55de11c15750 2023-07-14T11:23:10.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.517+0000 7efeca7fc700 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c15750 0x55de11c17b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.518+0000 7efeaeff5700 10 client.15768 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:10.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.518+0000 7efeca7fc700 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c15750 0x55de11c17b60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=306201b1a33306bc server_cookie=46bf25c043968c8 in_seq=0 out_seq=0 2023-07-14T11:23:10.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.518+0000 7efeaeff5700 1 -- 172.21.15.161:0/206099675 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de11b3deb0 con 0x55de11c15750 2023-07-14T11:23:10.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.518+0000 7efeaeff5700 10 client.15768 handle_command_reply: tid=0 2023-07-14T11:23:10.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.518+0000 7efee4d3bb80 1 client.15768 shutdown 2023-07-14T11:23:10.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 msgr2=0x55de11bfd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bfd530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efebc00b880 tx=0x7efebc00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 shutdown_connections 2023-07-14T11:23:10.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c15750 0x55de11c17b60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bf22d0 0x55de11bfd530 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bfdad0 0x55de11bfaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/206099675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf2be0 0x55de11bfa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 >> 172.21.15.161:0/206099675 conn(0x55de11be8da0 msgr2=0x55de11bee230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efeaeff5700 0 client.15768 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:10.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 shutdown_connections 2023-07-14T11:23:10.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 1 -- 172.21.15.161:0/206099675 wait complete. 2023-07-14T11:23:10.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.519+0000 7efee4d3bb80 20 client.15768 trim_cache size 0 max 16384 2023-07-14T11:23:10.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 Processor -- start 2023-07-14T11:23:10.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 -- start start 2023-07-14T11:23:10.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c08c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11bf5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11bf49f0 con 0x55de11bfdad0 2023-07-14T11:23:10.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11bf3540 con 0x55de11bf22d0 2023-07-14T11:23:10.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.545+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11c09de0 con 0x55de11bf2be0 2023-07-14T11:23:10.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11bf5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11bf5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39538/0 (socket says 172.21.15.161:39538) 2023-07-14T11:23:10.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c044f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45914/0 (socket says 172.21.15.161:45914) 2023-07-14T11:23:10.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecb7fe700 1 -- 172.21.15.161:0/1764063308 learned_addr learned my addr 172.21.15.161:0/1764063308 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efeca7fc700 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c08c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecaffd700 1 -- 172.21.15.161:0/1764063308 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 msgr2=0x55de11c08c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecaffd700 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c08c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecaffd700 1 -- 172.21.15.161:0/1764063308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 msgr2=0x55de11bf5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecaffd700 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11bf5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.546+0000 7efecaffd700 1 -- 172.21.15.161:0/1764063308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de11999250 con 0x55de11bf22d0 2023-07-14T11:23:10.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.547+0000 7efecaffd700 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c044f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efebc005350 tx=0x7efebc007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3bc17b41483b1c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.547+0000 7efeb1ffb700 1 -- 172.21.15.161:0/1764063308 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00e040 con 0x55de11bf22d0 2023-07-14T11:23:10.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.547+0000 7efeb1ffb700 1 -- 172.21.15.161:0/1764063308 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efebc0074a0 con 0x55de11bf22d0 2023-07-14T11:23:10.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.547+0000 7efeb1ffb700 1 -- 172.21.15.161:0/1764063308 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc005aa0 con 0x55de11bf22d0 2023-07-14T11:23:10.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 -- 172.21.15.161:0/1764063308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 msgr2=0x55de11c044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c044f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efebc005350 tx=0x7efebc007a00 comp rx=0 tx=0).stop 2023-07-14T11:23:10.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 -- 172.21.15.161:0/1764063308 shutdown_connections 2023-07-14T11:23:10.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11bf5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c044f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/1764063308 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c08c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 -- 172.21.15.161:0/1764063308 >> 172.21.15.161:0/1764063308 conn(0x55de11be8da0 msgr2=0x55de11bf42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 -- 172.21.15.161:0/1764063308 shutdown_connections 2023-07-14T11:23:10.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.548+0000 7efee4d3bb80 1 -- 172.21.15.161:0/1764063308 wait complete. 2023-07-14T11:23:10.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.549+0000 7efee4d3bb80 1 -- start start 2023-07-14T11:23:10.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.549+0000 7efee4d3bb80 1 Processor -- start 2023-07-14T11:23:10.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c02570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efee4d3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11c02ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11c04b20 con 0x55de11bfdad0 2023-07-14T11:23:10.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11c02ff0 con 0x55de11bf22d0 2023-07-14T11:23:10.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efee4d3bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11c03160 con 0x55de11bf2be0 2023-07-14T11:23:10.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efee4d3bb80 10 client.0 _ll_register_callbacks cb 0x7efec4009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:10.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efecaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c04210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efecaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c04210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45928/0 (socket says 172.21.15.161:45928) 2023-07-14T11:23:10.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.550+0000 7efecaffd700 1 -- 172.21.15.161:0/3656231554 learned_addr learned my addr 172.21.15.161:0/3656231554 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efeca7fc700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c02570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efecb7fe700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11c02ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efeca7fc700 1 -- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 msgr2=0x55de11c04210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efeca7fc700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c04210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efeca7fc700 1 -- 172.21.15.161:0/3656231554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 msgr2=0x55de11c02ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efeca7fc700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11c02ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efeca7fc700 1 -- 172.21.15.161:0/3656231554 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de1195bc60 con 0x55de11bf2be0 2023-07-14T11:23:10.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.551+0000 7efecaffd700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:10.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efeca7fc700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c02570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efec00ca8b0 tx=0x7efec0010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d980d263caf952c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efeaeff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:10.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efeaeff5700 1 -- 172.21.15.161:0/3656231554 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00b6070 con 0x55de11bf2be0 2023-07-14T11:23:10.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efebc009460 con 0x55de11bf2be0 2023-07-14T11:23:10.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efeaeff5700 1 -- 172.21.15.161:0/3656231554 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7efec009e190 con 0x55de11bf2be0 2023-07-14T11:23:10.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efeaeff5700 1 -- 172.21.15.161:0/3656231554 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec00a25b0 con 0x55de11bf2be0 2023-07-14T11:23:10.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efee4d3bb80 10 client.25810 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:10.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55de119712b0 con 0x55de11bf2be0 2023-07-14T11:23:10.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.552+0000 7efeaeff5700 1 -- 172.21.15.161:0/3656231554 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efebc009460 con 0x55de11bf2be0 2023-07-14T11:23:10.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.553+0000 7efeaeff5700 1 -- 172.21.15.161:0/3656231554 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efec00a2120 con 0x55de11bf2be0 2023-07-14T11:23:10.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.554+0000 7efee4d3bb80 10 client.25810 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:10.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.554+0000 7efee4d3bb80 10 client.25810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:23:10.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.554+0000 7efee4d3bb80 20 client.25810 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:10.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.554+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c0bde0 0x55de11c0e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.554+0000 7efee4d3bb80 4 client.25810 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:23:10.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.554+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de11b00050 con 0x55de11c0bde0 2023-07-14T11:23:10.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.554+0000 7efecaffd700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c0bde0 0x55de11c0e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.555+0000 7efecaffd700 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c0bde0 0x55de11c0e1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a15cd72926f2df8f server_cookie=39cd64794fc9f636 in_seq=0 out_seq=0 2023-07-14T11:23:10.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.555+0000 7efeaeff5700 10 client.25810 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:10.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.555+0000 7efeaeff5700 1 -- 172.21.15.161:0/3656231554 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de11b00050 con 0x55de11c0bde0 2023-07-14T11:23:10.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.555+0000 7efeaeff5700 10 client.25810 handle_command_reply: tid=0 2023-07-14T11:23:10.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.555+0000 7efee4d3bb80 1 client.25810 shutdown 2023-07-14T11:23:10.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.555+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 msgr2=0x55de11c02570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.555+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c02570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efec00ca8b0 tx=0x7efec0010bd0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 shutdown_connections 2023-07-14T11:23:10.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55de11c0bde0 0x55de11c0e1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de11bfdad0 0x55de11c02ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de11bf22d0 0x55de11c04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 --2- 172.21.15.161:0/3656231554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55de11bf2be0 0x55de11c02570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 >> 172.21.15.161:0/3656231554 conn(0x55de11be8da0 msgr2=0x55de11c08c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efeaeff5700 0 client.25810 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:10.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 shutdown_connections 2023-07-14T11:23:10.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 1 -- 172.21.15.161:0/3656231554 wait complete. 2023-07-14T11:23:10.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.556+0000 7efee4d3bb80 20 client.25810 trim_cache size 0 max 16384 2023-07-14T11:23:10.574 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:23:10.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7efeb40a1460 msgr2=0x7efeb40a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7efeb40a1460 0x7efeb40a3910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efec4004100 tx=0x7efec40047c0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc100480 msgr2=0x7efecc198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc100480 0x7efecc198490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9a0 tx=0x7efec000ed60 comp rx=0 tx=0).stop 2023-07-14T11:23:10.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 shutdown_connections 2023-07-14T11:23:10.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7efeb40a1460 0x7efeb40a3910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc0ffad0 0x7efecc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efecc105ae0 0x7efecc19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 --2- 172.21.15.161:0/2352701337 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7efecc100480 0x7efecc198490 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 >> 172.21.15.161:0/2352701337 conn(0x7efecc0fb5f0 msgr2=0x7efecc10ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 shutdown_connections 2023-07-14T11:23:10.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.557+0000 7efed13ff700 1 -- 172.21.15.161:0/2352701337 wait complete. 2023-07-14T11:23:10.588 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:23:10.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.924+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3253707917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 msgr2=0x7f2b50109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.924+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3253707917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50109a80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009990 tx=0x7f2b44009ca0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.925+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3253707917 shutdown_connections 2023-07-14T11:23:10.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.925+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3253707917 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b50109fc0 0x7f2b5010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.925+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3253707917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b50100aa0 0x7f2b50100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.925+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3253707917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50109a80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.925+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3253707917 >> 172.21.15.161:0/3253707917 conn(0x7f2b500745d0 msgr2=0x7f2b500749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.925+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3253707917 shutdown_connections 2023-07-14T11:23:10.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.925+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3253707917 wait complete. 2023-07-14T11:23:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.926+0000 7f2b54f9d700 1 Processor -- start 2023-07-14T11:23:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b54f9d700 1 -- start start 2023-07-14T11:23:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b54f9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b50100aa0 0x7f2b50108650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b54f9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50108b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b54f9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b50109fc0 0x7f2b501b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b54f9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501094b0 con 0x7f2b501013b0 2023-07-14T11:23:10.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b54f9d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b50069f30 con 0x7f2b50100aa0 2023-07-14T11:23:10.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b54f9d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5006a0f0 con 0x7f2b50109fc0 2023-07-14T11:23:10.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.927+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50108b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50108b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39572/0 (socket says 172.21.15.161:39572) 2023-07-14T11:23:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 -- 172.21.15.161:0/3477281975 learned_addr learned my addr 172.21.15.161:0/3477281975 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b50100aa0 0x7f2b50108650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4ffff700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b50109fc0 0x7f2b501b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 -- 172.21.15.161:0/3477281975 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b50109fc0 msgr2=0x7f2b501b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b50109fc0 0x7f2b501b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 -- 172.21.15.161:0/3477281975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b50100aa0 msgr2=0x7f2b50108650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b50100aa0 0x7f2b50108650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 -- 172.21.15.161:0/3477281975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b440095f0 con 0x7f2b501013b0 2023-07-14T11:23:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4ffff700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b50109fc0 0x7f2b501b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.928+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50108b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005910 tx=0x7f2b44004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.929+0000 7f2b4cff9700 1 -- 172.21.15.161:0/3477281975 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4401d070 con 0x7f2b501013b0 2023-07-14T11:23:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.929+0000 7f2b4cff9700 1 -- 172.21.15.161:0/3477281975 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b44022470 con 0x7f2b501013b0 2023-07-14T11:23:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.929+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5006a370 con 0x7f2b501013b0 2023-07-14T11:23:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.929+0000 7f2b4cff9700 1 -- 172.21.15.161:0/3477281975 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44017650 con 0x7f2b501013b0 2023-07-14T11:23:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.929+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5006a870 con 0x7f2b501013b0 2023-07-14T11:23:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.935+0000 7f2b4cff9700 1 -- 172.21.15.161:0/3477281975 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f2b4400fb10 con 0x7f2b501013b0 2023-07-14T11:23:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.935+0000 7f2b69a21b80 1 Processor -- start 2023-07-14T11:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.936+0000 7f2b69a21b80 1 -- start start 2023-07-14T11:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.936+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 0x5561846298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.936+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x5561846337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.936+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x556184636110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.936+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618452ecb0 con 0x5561846294f0 2023-07-14T11:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.936+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556184581a40 con 0x556184629e00 2023-07-14T11:23:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.936+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618457aed0 con 0x556184633d30 2023-07-14T11:23:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x5561846337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x5561846337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45944/0 (socket says 172.21.15.161:45944) 2023-07-14T11:23:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/1235262876 learned_addr learned my addr 172.21.15.161:0/1235262876 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4effd700 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x556184636110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/1235262876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 msgr2=0x556184636110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x556184636110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/1235262876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 msgr2=0x5561846298c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 0x5561846298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.937+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/1235262876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561843a8de0 con 0x556184629e00 2023-07-14T11:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.938+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x5561846337f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009d20 tx=0x7f2b40011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c6981e37409eca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.938+0000 7f2b35ffb700 1 -- 172.21.15.161:0/1235262876 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40004a30 con 0x556184629e00 2023-07-14T11:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.938+0000 7f2b35ffb700 1 -- 172.21.15.161:0/1235262876 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b400054e0 con 0x556184629e00 2023-07-14T11:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.938+0000 7f2b35ffb700 1 -- 172.21.15.161:0/1235262876 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40019830 con 0x556184629e00 2023-07-14T11:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.938+0000 7f2b4cff9700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b380a1500 0x7f2b380a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.938+0000 7f2b4cff9700 1 -- 172.21.15.161:0/3477281975 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f2b440ce140 con 0x7f2b501013b0 2023-07-14T11:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b4ffff700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b380a1500 0x7f2b380a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b69a21b80 1 -- 172.21.15.161:0/1235262876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 msgr2=0x5561846337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x5561846337f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009d20 tx=0x7f2b40011720 comp rx=0 tx=0).stop 2023-07-14T11:23:10.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b69a21b80 1 -- 172.21.15.161:0/1235262876 shutdown_connections 2023-07-14T11:23:10.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 0x5561846298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x556184636110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/1235262876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x5561846337f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.939+0000 7f2b69a21b80 1 -- 172.21.15.161:0/1235262876 >> 172.21.15.161:0/1235262876 conn(0x55618461eff0 msgr2=0x556184624930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.940+0000 7f2b69a21b80 1 -- 172.21.15.161:0/1235262876 shutdown_connections 2023-07-14T11:23:10.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.940+0000 7f2b4ffff700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b380a1500 0x7f2b380a39b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b50073240 tx=0x7f2b48009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.940+0000 7f2b69a21b80 1 -- 172.21.15.161:0/1235262876 wait complete. 2023-07-14T11:23:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.940+0000 7f2b69a21b80 1 -- start start 2023-07-14T11:23:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.941+0000 7f2b69a21b80 1 Processor -- start 2023-07-14T11:23:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.941+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x5561846337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.941+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184629e00 0x5561846309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.941+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184633d30 0x5561846310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618451aac0 con 0x556184629e00 2023-07-14T11:23:10.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618451a7c0 con 0x556184633d30 2023-07-14T11:23:10.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561843fe920 con 0x5561846294f0 2023-07-14T11:23:10.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b69a21b80 10 client.0 _ll_register_callbacks cb 0x556184518760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:10.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b4f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x5561846337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b4f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x5561846337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33956/0 (socket says 172.21.15.161:33956) 2023-07-14T11:23:10.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/3945075356 learned_addr learned my addr 172.21.15.161:0/3945075356 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184629e00 0x5561846309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.942+0000 7f2b4ffff700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184633d30 0x5561846310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b4effd700 1 -- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 msgr2=0x5561846337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x5561846337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b4effd700 1 -- 172.21.15.161:0/3945075356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184633d30 msgr2=0x5561846310b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184633d30 0x5561846310b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b4effd700 1 -- 172.21.15.161:0/3945075356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b40009980 con 0x556184629e00 2023-07-14T11:23:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x5561846337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184629e00 0x5561846309c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2b440cb3e0 tx=0x7f2b44009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66daea6dc194a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b2effd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b2effd700 1 -- 172.21.15.161:0/3945075356 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44097180 con 0x556184629e00 2023-07-14T11:23:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.943+0000 7f2b2effd700 1 -- 172.21.15.161:0/3945075356 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f2b4409ae40 con 0x556184629e00 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.944+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561844d44f0 con 0x556184629e00 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.944+0000 7f2b2effd700 1 -- 172.21.15.161:0/3945075356 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b4409ac10 con 0x556184629e00 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.944+0000 7f2b69a21b80 10 client.15783 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.944+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556184391c60 con 0x556184629e00 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.944+0000 7f2b2effd700 1 -- 172.21.15.161:0/3945075356 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b440b2030 con 0x556184629e00 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.944+0000 7f2b2effd700 1 -- 172.21.15.161:0/3945075356 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b440a2d70 con 0x556184629e00 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.945+0000 7f2b69a21b80 10 client.15783 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:10.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.945+0000 7f2b69a21b80 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:23:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.945+0000 7f2b69a21b80 20 client.15783 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.945+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846432a0 0x5561846456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.945+0000 7f2b69a21b80 4 client.15783 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.945+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556184573eb0 con 0x5561846432a0 2023-07-14T11:23:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.945+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846432a0 0x5561846456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.946+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846432a0 0x5561846456b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0492b68e660095e server_cookie=51b80026c7455ab2 in_seq=0 out_seq=0 2023-07-14T11:23:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.946+0000 7f2b2effd700 10 client.15783 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b2effd700 1 -- 172.21.15.161:0/3945075356 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556184573eb0 con 0x5561846432a0 2023-07-14T11:23:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b2effd700 10 client.15783 handle_command_reply: tid=0 2023-07-14T11:23:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 client.15783 shutdown 2023-07-14T11:23:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184629e00 msgr2=0x5561846309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184629e00 0x5561846309c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2b440cb3e0 tx=0x7f2b44009310 comp rx=0 tx=0).stop 2023-07-14T11:23:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 shutdown_connections 2023-07-14T11:23:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846432a0 0x5561846456b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184633d30 0x5561846310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x5561846337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3945075356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184629e00 0x5561846309c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 >> 172.21.15.161:0/3945075356 conn(0x55618461eff0 msgr2=0x556184624520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.947+0000 7f2b2effd700 0 client.15783 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.948+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 shutdown_connections 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.948+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3945075356 wait complete. 2023-07-14T11:23:10.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.948+0000 7f2b69a21b80 20 client.15783 trim_cache size 0 max 16384 2023-07-14T11:23:10.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.973+0000 7f2b69a21b80 1 Processor -- start 2023-07-14T11:23:10.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.973+0000 7f2b69a21b80 1 -- start start 2023-07-14T11:23:10.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.974+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 0x55618463a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.974+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.974+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x55618462bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.974+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618462b1a0 con 0x5561846294f0 2023-07-14T11:23:10.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.974+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618463ad80 con 0x556184629e00 2023-07-14T11:23:10.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.974+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556184620560 con 0x556184633d30 2023-07-14T11:23:10.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.974+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45968/0 (socket says 172.21.15.161:45968) 2023-07-14T11:23:10.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 -- 172.21.15.161:0/149420512 learned_addr learned my addr 172.21.15.161:0/149420512 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4ffff700 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x55618462bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 0x55618463a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 -- 172.21.15.161:0/149420512 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 msgr2=0x55618462bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x55618462bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 -- 172.21.15.161:0/149420512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 msgr2=0x55618463a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 0x55618463a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 -- 172.21.15.161:0/149420512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561843cf250 con 0x556184629e00 2023-07-14T11:23:10.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.975+0000 7f2b4effd700 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463efe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b4409add0 tx=0x7f2b440cb410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8fc05b8efecbc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.976+0000 7f2b367fc700 1 -- 172.21.15.161:0/149420512 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b440d3ea0 con 0x556184629e00 2023-07-14T11:23:10.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.976+0000 7f2b367fc700 1 -- 172.21.15.161:0/149420512 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b440b2070 con 0x556184629e00 2023-07-14T11:23:10.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.976+0000 7f2b367fc700 1 -- 172.21.15.161:0/149420512 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b440a2fd0 con 0x556184629e00 2023-07-14T11:23:10.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 -- 172.21.15.161:0/149420512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 msgr2=0x55618463efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463efe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b4409add0 tx=0x7f2b440cb410 comp rx=0 tx=0).stop 2023-07-14T11:23:10.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 -- 172.21.15.161:0/149420512 shutdown_connections 2023-07-14T11:23:10.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561846294f0 0x55618463a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556184633d30 0x55618462bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/149420512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463efe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 -- 172.21.15.161:0/149420512 >> 172.21.15.161:0/149420512 conn(0x55618461eff0 msgr2=0x55618463fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 -- 172.21.15.161:0/149420512 shutdown_connections 2023-07-14T11:23:10.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 -- 172.21.15.161:0/149420512 wait complete. 2023-07-14T11:23:10.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.977+0000 7f2b69a21b80 1 -- start start 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.978+0000 7f2b69a21b80 1 Processor -- start 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.978+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x55618463a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.978+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.978+0000 7f2b69a21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184633d30 0x556184638650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.978+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556184638c60 con 0x556184633d30 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.978+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556184638dd0 con 0x556184629e00 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.978+0000 7f2b69a21b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556184638f90 con 0x5561846294f0 2023-07-14T11:23:10.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.979+0000 7f2b69a21b80 10 client.0 _ll_register_callbacks cb 0x556184518760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:10.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.979+0000 7f2b4ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184633d30 0x556184638650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.979+0000 7f2b4ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184633d30 0x556184638650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39608/0 (socket says 172.21.15.161:39608) 2023-07-14T11:23:10.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.979+0000 7f2b4ffff700 1 -- 172.21.15.161:0/3915012108 learned_addr learned my addr 172.21.15.161:0/3915012108 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:10.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.979+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x55618463a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.979+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/3915012108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 msgr2=0x55618463a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/3915012108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184633d30 msgr2=0x556184638650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184633d30 0x556184638650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b4f7fe700 1 -- 172.21.15.161:0/3915012108 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556184391c60 con 0x5561846294f0 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b4ffff700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184633d30 0x556184638650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b4f7fe700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x55618463a190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b40005920 tx=0x7f2b40009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3933a21274dd0ed2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:10.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b2effd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b2effd700 1 -- 172.21.15.161:0/3915012108 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40003680 con 0x5561846294f0 2023-07-14T11:23:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b2effd700 1 -- 172.21.15.161:0/3915012108 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f2b400037e0 con 0x5561846294f0 2023-07-14T11:23:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.980+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b400054e0 con 0x5561846294f0 2023-07-14T11:23:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.981+0000 7f2b2effd700 1 -- 172.21.15.161:0/3915012108 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b400039a0 con 0x5561846294f0 2023-07-14T11:23:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.981+0000 7f2b69a21b80 10 client.25828 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.981+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5561843a72b0 con 0x5561846294f0 2023-07-14T11:23:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.981+0000 7f2b2effd700 1 -- 172.21.15.161:0/3915012108 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b400054e0 con 0x5561846294f0 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.982+0000 7f2b2effd700 1 -- 172.21.15.161:0/3915012108 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b40019b10 con 0x5561846294f0 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.982+0000 7f2b69a21b80 10 client.25828 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.982+0000 7f2b69a21b80 10 client.25828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.982+0000 7f2b69a21b80 20 client.25828 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.982+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846982d0 0x55618469a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.982+0000 7f2b69a21b80 4 client.25828 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.982+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556184536050 con 0x5561846982d0 2023-07-14T11:23:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.983+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846982d0 0x55618469a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.983+0000 7f2b4effd700 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846982d0 0x55618469a6e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17c8526a6214ac55 server_cookie=a021f24528cc0864 in_seq=0 out_seq=0 2023-07-14T11:23:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.983+0000 7f2b2effd700 10 client.25828 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b2effd700 1 -- 172.21.15.161:0/3915012108 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556184536050 con 0x5561846982d0 2023-07-14T11:23:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b2effd700 10 client.25828 handle_command_reply: tid=0 2023-07-14T11:23:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 client.25828 shutdown 2023-07-14T11:23:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 msgr2=0x55618463a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x55618463a190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b40005920 tx=0x7f2b40009760 comp rx=0 tx=0).stop 2023-07-14T11:23:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 shutdown_connections 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5561846982d0 0x55618469a6e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556184633d30 0x556184638650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5561846294f0 0x55618463a190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 --2- 172.21.15.161:0/3915012108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556184629e00 0x55618463a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 >> 172.21.15.161:0/3915012108 conn(0x55618461eff0 msgr2=0x55618463efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b2effd700 0 client.25828 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 shutdown_connections 2023-07-14T11:23:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.984+0000 7f2b69a21b80 1 -- 172.21.15.161:0/3915012108 wait complete. 2023-07-14T11:23:10.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b69a21b80 20 client.25828 trim_cache size 0 max 16384 2023-07-14T11:23:10.992 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:23:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b380a1500 msgr2=0x7f2b380a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b380a1500 0x7f2b380a39b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b50073240 tx=0x7f2b48009920 comp rx=0 tx=0).stop 2023-07-14T11:23:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 msgr2=0x7f2b50108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50108b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005910 tx=0x7f2b44004710 comp rx=0 tx=0).stop 2023-07-14T11:23:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 shutdown_connections 2023-07-14T11:23:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2b380a1500 0x7f2b380a39b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2b50109fc0 0x7f2b501b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b50100aa0 0x7f2b50108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.985+0000 7f2b54f9d700 1 --2- 172.21.15.161:0/3477281975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b501013b0 0x7f2b50108b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.986+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 >> 172.21.15.161:0/3477281975 conn(0x7f2b500745d0 msgr2=0x7f2b500fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.986+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 shutdown_connections 2023-07-14T11:23:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:10.986+0000 7f2b54f9d700 1 -- 172.21.15.161:0/3477281975 wait complete. 2023-07-14T11:23:11.016 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:23:11.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 -- 172.21.15.161:0/104811820 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1040105a00 msgr2=0x7f1040105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 --2- 172.21.15.161:0/104811820 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1040105a00 0x7f1040105dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f10300098f0 tx=0x7f1030009c00 comp rx=0 tx=0).stop 2023-07-14T11:23:11.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 -- 172.21.15.161:0/104811820 shutdown_connections 2023-07-14T11:23:11.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 --2- 172.21.15.161:0/104811820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401003a0 0x7f104010c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 --2- 172.21.15.161:0/104811820 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1040105a00 0x7f1040105dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 --2- 172.21.15.161:0/104811820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10400ff9f0 0x7f10400ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 -- 172.21.15.161:0/104811820 >> 172.21.15.161:0/104811820 conn(0x7f10400fb510 msgr2=0x7f10400fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:11.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 -- 172.21.15.161:0/104811820 shutdown_connections 2023-07-14T11:23:11.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.357+0000 7f103f59e700 1 -- 172.21.15.161:0/104811820 wait complete. 2023-07-14T11:23:11.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.359+0000 7f103f59e700 1 Processor -- start 2023-07-14T11:23:11.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.359+0000 7f103f59e700 1 -- start start 2023-07-14T11:23:11.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.359+0000 7f103f59e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f10400ff9f0 0x7f10401092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.359+0000 7f103f59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401003a0 0x7f1040109810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103f59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1040105a00 0x7f10401b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103f59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104010a1a0 con 0x7f10401003a0 2023-07-14T11:23:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103f59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040069f30 con 0x7f1040105a00 2023-07-14T11:23:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103f59e700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104006a0f0 con 0x7f10400ff9f0 2023-07-14T11:23:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103e59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f10400ff9f0 0x7f10401092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103e59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f10400ff9f0 0x7f10401092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34006/0 (socket says 172.21.15.161:34006) 2023-07-14T11:23:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103e59c700 1 -- 172.21.15.161:0/551418634 learned_addr learned my addr 172.21.15.161:0/551418634 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103dd9b700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401003a0 0x7f1040109810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.360+0000 7f103ed9d700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1040105a00 0x7f10401b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103ed9d700 1 -- 172.21.15.161:0/551418634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f10400ff9f0 msgr2=0x7f10401092d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103ed9d700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f10400ff9f0 0x7f10401092d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103ed9d700 1 -- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401003a0 msgr2=0x7f1040109810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103ed9d700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401003a0 0x7f1040109810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103ed9d700 1 -- 172.21.15.161:0/551418634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038009c40 con 0x7f1040105a00 2023-07-14T11:23:11.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103e59c700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f10400ff9f0 0x7f10401092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:11.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103ed9d700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1040105a00 0x7f10401b0730 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f103800efb0 tx=0x7f103800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:11.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f102f7fe700 1 -- 172.21.15.161:0/551418634 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038003a00 con 0x7f1040105a00 2023-07-14T11:23:11.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1030009520 con 0x7f1040105a00 2023-07-14T11:23:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f102f7fe700 1 -- 172.21.15.161:0/551418634 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f103800f040 con 0x7f1040105a00 2023-07-14T11:23:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f102f7fe700 1 -- 172.21.15.161:0/551418634 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038004520 con 0x7f1040105a00 2023-07-14T11:23:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.361+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104006a750 con 0x7f1040105a00 2023-07-14T11:23:11.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.367+0000 7f102f7fe700 1 -- 172.21.15.161:0/551418634 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f10380046a0 con 0x7f1040105a00 2023-07-14T11:23:11.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.368+0000 7f10587b1b80 1 Processor -- start 2023-07-14T11:23:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.369+0000 7f10587b1b80 1 -- start start 2023-07-14T11:23:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.369+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a074f0 0x559d40a078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.369+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.369+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.369+0000 7f10587b1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4090ccb0 con 0x559d40a11d30 2023-07-14T11:23:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.369+0000 7f10587b1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4095fa40 con 0x559d40a074f0 2023-07-14T11:23:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.369+0000 7f10587b1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d40958ed0 con 0x559d40a07e00 2023-07-14T11:23:11.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a074f0 0x559d40a078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a074f0 0x559d40a078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46008/0 (socket says 172.21.15.161:46008) 2023-07-14T11:23:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 -- 172.21.15.161:0/628357742 learned_addr learned my addr 172.21.15.161:0/628357742 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103e59c700 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f102f7fe700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f10280a1580 0x7f10280a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 -- 172.21.15.161:0/628357742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 msgr2=0x559d40a117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 -- 172.21.15.161:0/628357742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 msgr2=0x559d40a14110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a14110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103dd9b700 1 -- 172.21.15.161:0/628357742 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d40786de0 con 0x559d40a074f0 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f103e59c700 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.370+0000 7f102f7fe700 1 -- 172.21.15.161:0/551418634 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f10380cce70 con 0x7f1040105a00 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.371+0000 7f103dd9b700 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a074f0 0x559d40a078c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1034009540 tx=0x7f10340058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a15c58baea0eec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.371+0000 7f103ed9d700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f10280a1580 0x7f10280a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.371+0000 7f102cff9700 1 -- 172.21.15.161:0/628357742 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034004030 con 0x559d40a074f0 2023-07-14T11:23:11.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.371+0000 7f102cff9700 1 -- 172.21.15.161:0/628357742 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1034017070 con 0x559d40a074f0 2023-07-14T11:23:11.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.371+0000 7f102cff9700 1 -- 172.21.15.161:0/628357742 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034005310 con 0x559d40a074f0 2023-07-14T11:23:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.372+0000 7f10587b1b80 1 -- 172.21.15.161:0/628357742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a074f0 msgr2=0x559d40a078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.372+0000 7f103ed9d700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f10280a1580 0x7f10280a3a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10380ca110 tx=0x7f10380b7f90 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.372+0000 7f10587b1b80 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a074f0 0x559d40a078c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1034009540 tx=0x7f10340058e0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.373+0000 7f10587b1b80 1 -- 172.21.15.161:0/628357742 shutdown_connections 2023-07-14T11:23:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.373+0000 7f10587b1b80 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a14110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.373+0000 7f10587b1b80 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a074f0 0x559d40a078c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.373+0000 7f10587b1b80 1 --2- 172.21.15.161:0/628357742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.373+0000 7f10587b1b80 1 -- 172.21.15.161:0/628357742 >> 172.21.15.161:0/628357742 conn(0x559d409fcff0 msgr2=0x559d40a02930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.373+0000 7f10587b1b80 1 -- 172.21.15.161:0/628357742 shutdown_connections 2023-07-14T11:23:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.373+0000 7f10587b1b80 1 -- 172.21.15.161:0/628357742 wait complete. 2023-07-14T11:23:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.374+0000 7f10587b1b80 1 -- start start 2023-07-14T11:23:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.374+0000 7f10587b1b80 1 Processor -- start 2023-07-14T11:23:11.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a11730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a07e00 0x559d40a0e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f10587b1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d408f8ac0 con 0x559d40a07e00 2023-07-14T11:23:11.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f10587b1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d408f87c0 con 0x559d40a11d30 2023-07-14T11:23:11.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f10587b1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d407dc920 con 0x559d40a074f0 2023-07-14T11:23:11.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f10587b1b80 10 client.0 _ll_register_callbacks cb 0x559d408f6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:11.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f103dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f103dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a0f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46016/0 (socket says 172.21.15.161:46016) 2023-07-14T11:23:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f103dd9b700 1 -- 172.21.15.161:0/1784210152 learned_addr learned my addr 172.21.15.161:0/1784210152 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.375+0000 7f103e59c700 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a07e00 0x559d40a0e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f103dd9b700 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a11730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f103e59c700 1 -- 172.21.15.161:0/1784210152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 msgr2=0x559d40a11730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f103e59c700 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a11730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f103e59c700 1 -- 172.21.15.161:0/1784210152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 msgr2=0x559d40a0f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f103e59c700 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a0f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f103e59c700 1 -- 172.21.15.161:0/1784210152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1034019040 con 0x559d40a07e00 2023-07-14T11:23:11.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f103e59c700 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a07e00 0x559d40a0e9b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1030005e90 tx=0x7f1030005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2413c0d309625e4c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:11.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.376+0000 7f1025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:11.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.377+0000 7f1025ffb700 1 -- 172.21.15.161:0/1784210152 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103001c070 con 0x559d40a07e00 2023-07-14T11:23:11.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.377+0000 7f1025ffb700 1 -- 172.21.15.161:0/1784210152 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f103000ba30 con 0x559d40a07e00 2023-07-14T11:23:11.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.378+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d408b24f0 con 0x559d40a07e00 2023-07-14T11:23:11.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.378+0000 7f1025ffb700 1 -- 172.21.15.161:0/1784210152 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f103000bbf0 con 0x559d40a07e00 2023-07-14T11:23:11.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.378+0000 7f10587b1b80 10 client.15801 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:11.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.378+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559d4076fc60 con 0x559d40a07e00 2023-07-14T11:23:11.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.378+0000 7f1025ffb700 1 -- 172.21.15.161:0/1784210152 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d408b24f0 con 0x559d40a07e00 2023-07-14T11:23:11.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f1025ffb700 1 -- 172.21.15.161:0/1784210152 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1030015020 con 0x559d40a07e00 2023-07-14T11:23:11.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f10587b1b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:11.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f10587b1b80 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:11.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f10587b1b80 20 client.15801 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:11.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f10587b1b80 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a217b0 0x559d40a23bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f10587b1b80 4 client.15801 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:11.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d40951eb0 con 0x559d40a217b0 2023-07-14T11:23:11.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.379+0000 7f103dd9b700 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a217b0 0x559d40a23bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.380+0000 7f103dd9b700 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a217b0 0x559d40a23bc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5984d3f4d914eda server_cookie=ec568dfc9bb418e in_seq=0 out_seq=0 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.380+0000 7f1025ffb700 10 client.15801 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.380+0000 7f1025ffb700 1 -- 172.21.15.161:0/1784210152 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559d40951eb0 con 0x559d40a217b0 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.380+0000 7f1025ffb700 10 client.15801 handle_command_reply: tid=0 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 client.15801 shutdown 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a07e00 msgr2=0x559d40a0e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a07e00 0x559d40a0e9b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1030005e90 tx=0x7f1030005b40 comp rx=0 tx=0).stop 2023-07-14T11:23:11.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 shutdown_connections 2023-07-14T11:23:11.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a217b0 0x559d40a23bc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a11730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 --2- 172.21.15.161:0/1784210152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a07e00 0x559d40a0e9b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 >> 172.21.15.161:0/1784210152 conn(0x559d409fcff0 msgr2=0x559d40a02520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:11.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f1025ffb700 0 client.15801 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:11.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 shutdown_connections 2023-07-14T11:23:11.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.381+0000 7f10587b1b80 1 -- 172.21.15.161:0/1784210152 wait complete. 2023-07-14T11:23:11.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.382+0000 7f10587b1b80 20 client.15801 trim_cache size 0 max 16384 2023-07-14T11:23:11.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 Processor -- start 2023-07-14T11:23:11.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 -- start start 2023-07-14T11:23:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a074f0 0x559d40a18810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a11360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d40a09220 con 0x559d40a074f0 2023-07-14T11:23:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d40a081d0 con 0x559d40a11d30 2023-07-14T11:23:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.407+0000 7f10587b1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d40a089b0 con 0x559d40a07e00 2023-07-14T11:23:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a074f0 0x559d40a18810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a074f0 0x559d40a18810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39636/0 (socket says 172.21.15.161:39636) 2023-07-14T11:23:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 -- 172.21.15.161:0/2693800409 learned_addr learned my addr 172.21.15.161:0/2693800409 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103e59c700 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a11360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 -- 172.21.15.161:0/2693800409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 msgr2=0x559d40a11360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a11360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 -- 172.21.15.161:0/2693800409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 msgr2=0x559d40a237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103dd9b700 1 -- 172.21.15.161:0/2693800409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d407ad250 con 0x559d40a074f0 2023-07-14T11:23:11.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.408+0000 7f103e59c700 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a11360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:11.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.409+0000 7f103dd9b700 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a074f0 0x559d40a18810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1034009a30 tx=0x7f1034004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a94ef6745c5d47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:11.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.409+0000 7f102d7fa700 1 -- 172.21.15.161:0/2693800409 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034005bc0 con 0x559d40a074f0 2023-07-14T11:23:11.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.409+0000 7f102d7fa700 1 -- 172.21.15.161:0/2693800409 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1034004910 con 0x559d40a074f0 2023-07-14T11:23:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.409+0000 7f102d7fa700 1 -- 172.21.15.161:0/2693800409 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034005420 con 0x559d40a074f0 2023-07-14T11:23:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 -- 172.21.15.161:0/2693800409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a074f0 msgr2=0x559d40a18810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a074f0 0x559d40a18810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1034009a30 tx=0x7f1034004e30 comp rx=0 tx=0).stop 2023-07-14T11:23:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 -- 172.21.15.161:0/2693800409 shutdown_connections 2023-07-14T11:23:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a074f0 0x559d40a18810 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a11d30 0x559d40a237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 --2- 172.21.15.161:0/2693800409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a07e00 0x559d40a11360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 -- 172.21.15.161:0/2693800409 >> 172.21.15.161:0/2693800409 conn(0x559d409fcff0 msgr2=0x559d40a1da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 -- 172.21.15.161:0/2693800409 shutdown_connections 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.410+0000 7f10587b1b80 1 -- 172.21.15.161:0/2693800409 wait complete. 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.411+0000 7f10587b1b80 1 -- start start 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.411+0000 7f10587b1b80 1 Processor -- start 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.412+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.412+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a07e00 0x559d40a16d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.412+0000 7f10587b1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.412+0000 7f10587b1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d40a17540 con 0x559d40a11d30 2023-07-14T11:23:11.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.412+0000 7f10587b1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d40a176b0 con 0x559d40a07e00 2023-07-14T11:23:11.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.412+0000 7f10587b1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d40a74c30 con 0x559d40a074f0 2023-07-14T11:23:11.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.412+0000 7f10587b1b80 10 client.0 _ll_register_callbacks cb 0x7f1030009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:11.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a07e00 0x559d40a16d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a07e00 0x559d40a16d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46034/0 (socket says 172.21.15.161:46034) 2023-07-14T11:23:11.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 -- 172.21.15.161:0/3612754757 learned_addr learned my addr 172.21.15.161:0/3612754757 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:11.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103dd9b700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103dd9b700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:11 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2431934300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:23:11.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 -- 172.21.15.161:0/3612754757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 msgr2=0x559d40a167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 -- 172.21.15.161:0/3612754757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 msgr2=0x559d40a74670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:11.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a74670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103e59c700 1 -- 172.21.15.161:0/3612754757 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d4076fc60 con 0x559d40a07e00 2023-07-14T11:23:11.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.413+0000 7f103dd9b700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:11.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f103e59c700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a07e00 0x559d40a16d30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1030011040 tx=0x7f103000ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a464c141142a392 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:11.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f1025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:23:11.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f1025ffb700 1 -- 172.21.15.161:0/3612754757 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103001c030 con 0x559d40a07e00 2023-07-14T11:23:11.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1034017070 con 0x559d40a07e00 2023-07-14T11:23:11.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f1025ffb700 1 -- 172.21.15.161:0/3612754757 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f1030000e40 con 0x559d40a07e00 2023-07-14T11:23:11.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f1025ffb700 1 -- 172.21.15.161:0/3612754757 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f103000b2c0 con 0x559d40a07e00 2023-07-14T11:23:11.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f10587b1b80 10 client.25847 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:11.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.414+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559d40786de0 con 0x559d40a07e00 2023-07-14T11:23:11.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f1025ffb700 1 -- 172.21.15.161:0/3612754757 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1034017070 con 0x559d40a07e00 2023-07-14T11:23:11.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f1025ffb700 1 -- 172.21.15.161:0/3612754757 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f103003c070 con 0x559d40a07e00 2023-07-14T11:23:11.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f10587b1b80 10 client.25847 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:11.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f10587b1b80 10 client.25847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:11.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f10587b1b80 20 client.25847 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:11.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f10587b1b80 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a76110 0x559d40a78520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:11.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f10587b1b80 4 client.25847 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:23:11.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.415+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d40914050 con 0x559d40a76110 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.416+0000 7f103dd9b700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a76110 0x559d40a78520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.416+0000 7f103dd9b700 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a76110 0x559d40a78520 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54887d4847ccea9 server_cookie=5b717d2081d7314a in_seq=0 out_seq=0 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.416+0000 7f1025ffb700 10 client.25847 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.416+0000 7f1025ffb700 1 -- 172.21.15.161:0/3612754757 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d40914050 con 0x559d40a76110 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.416+0000 7f1025ffb700 10 client.25847 handle_command_reply: tid=0 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.416+0000 7f10587b1b80 1 client.25847 shutdown 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a07e00 msgr2=0x559d40a16d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a07e00 0x559d40a16d30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1030011040 tx=0x7f103000ba40 comp rx=0 tx=0).stop 2023-07-14T11:23:11.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 shutdown_connections 2023-07-14T11:23:11.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559d40a76110 0x559d40a78520 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40a11d30 0x559d40a74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559d40a074f0 0x559d40a167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 --2- 172.21.15.161:0/3612754757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d40a07e00 0x559d40a16d30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 >> 172.21.15.161:0/3612754757 conn(0x559d409fcff0 msgr2=0x559d40a237c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:11.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f1025ffb700 0 client.25847 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:11.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 shutdown_connections 2023-07-14T11:23:11.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 1 -- 172.21.15.161:0/3612754757 wait complete. 2023-07-14T11:23:11.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.417+0000 7f10587b1b80 20 client.25847 trim_cache size 0 max 16384 2023-07-14T11:23:11.427 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:23:11.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f10280a1580 msgr2=0x7f10280a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f10280a1580 0x7f10280a3a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10380ca110 tx=0x7f10380b7f90 comp rx=0 tx=0).stop 2023-07-14T11:23:11.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1040105a00 msgr2=0x7f10401b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:11.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1040105a00 0x7f10401b0730 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f103800efb0 tx=0x7f103800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 shutdown_connections 2023-07-14T11:23:11.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f10280a1580 0x7f10280a3a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1040105a00 0x7f10401b0730 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f10400ff9f0 0x7f10401092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 --2- 172.21.15.161:0/551418634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401003a0 0x7f1040109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:11.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 >> 172.21.15.161:0/551418634 conn(0x7f10400fb510 msgr2=0x7f1040072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:11.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.418+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 shutdown_connections 2023-07-14T11:23:11.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:11.419+0000 7f103f59e700 1 -- 172.21.15.161:0/551418634 wait complete. 2023-07-14T11:23:11.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:11 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2431934300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:23:11.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:11 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2431934300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:23:12.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:12 smithi175 ceph-mon[129081]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:12.450 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:23:12.450 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:23:12.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:12 smithi182 ceph-mon[129366]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:12 smithi161 ceph-mon[95526]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-14T11:23:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 -- 172.21.15.161:0/265588449 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a40728b0 msgr2=0x7f80a4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 --2- 172.21.15.161:0/265588449 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a40728b0 0x7f80a4072c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80a00098f0 tx=0x7f80a0009c00 comp rx=0 tx=0).stop 2023-07-14T11:23:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 -- 172.21.15.161:0/265588449 shutdown_connections 2023-07-14T11:23:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 --2- 172.21.15.161:0/265588449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a410c320 0x7f80a410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 --2- 172.21.15.161:0/265588449 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a40728b0 0x7f80a4072c80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 --2- 172.21.15.161:0/265588449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 0x7f80a410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 -- 172.21.15.161:0/265588449 >> 172.21.15.161:0/265588449 conn(0x7f80a40fb5e0 msgr2=0x7f80a40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 -- 172.21.15.161:0/265588449 shutdown_connections 2023-07-14T11:23:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.792+0000 7f80aabff700 1 -- 172.21.15.161:0/265588449 wait complete. 2023-07-14T11:23:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.794+0000 7f80aabff700 1 Processor -- start 2023-07-14T11:23:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.794+0000 7f80aabff700 1 -- start start 2023-07-14T11:23:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80aabff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a40728b0 0x7f80a40ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80aabff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 0x7f80a40fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80aabff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a410c320 0x7f80a41004c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80aabff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41a8490 con 0x7f80a40728b0 2023-07-14T11:23:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80aabff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41a8600 con 0x7f80a40731c0 2023-07-14T11:23:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80aabff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41a87c0 con 0x7f80a410c320 2023-07-14T11:23:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80a899b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a40728b0 0x7f80a40ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f809bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 0x7f80a40fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f809bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 0x7f80a40fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46050/0 (socket says 172.21.15.161:46050) 2023-07-14T11:23:12.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f809bfff700 1 -- 172.21.15.161:0/1669617540 learned_addr learned my addr 172.21.15.161:0/1669617540 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:12.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.795+0000 7f80a919c700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a410c320 0x7f80a41004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.796+0000 7f809bfff700 1 -- 172.21.15.161:0/1669617540 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a410c320 msgr2=0x7f80a41004c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.796+0000 7f809bfff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a410c320 0x7f80a41004c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.796+0000 7f809bfff700 1 -- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a40728b0 msgr2=0x7f80a40ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.796+0000 7f809bfff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a40728b0 0x7f80a40ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.796+0000 7f809bfff700 1 -- 172.21.15.161:0/1669617540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80940095f0 con 0x7f80a40731c0 2023-07-14T11:23:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.796+0000 7f809bfff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 0x7f80a40fff80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f809400ec10 tx=0x7f809400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.797+0000 7f8099ffb700 1 -- 172.21.15.161:0/1669617540 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400cd40 con 0x7f80a40731c0 2023-07-14T11:23:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.797+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a0009520 con 0x7f80a40731c0 2023-07-14T11:23:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.797+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41a8e70 con 0x7f80a40731c0 2023-07-14T11:23:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.797+0000 7f8099ffb700 1 -- 172.21.15.161:0/1669617540 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80940044e0 con 0x7f80a40731c0 2023-07-14T11:23:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.797+0000 7f8099ffb700 1 -- 172.21.15.161:0/1669617540 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094010690 con 0x7f80a40731c0 2023-07-14T11:23:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.803+0000 7f80be431b80 1 Processor -- start 2023-07-14T11:23:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.803+0000 7f8099ffb700 1 -- 172.21.15.161:0/1669617540 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f8094004020 con 0x7f80a40731c0 2023-07-14T11:23:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.804+0000 7f80be431b80 1 -- start start 2023-07-14T11:23:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.804+0000 7f80be431b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc799d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.804+0000 7f80be431b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2c60 0x55cc799ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.804+0000 7f80be431b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.805+0000 7f80be431b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc798d8d30 con 0x55cc799eeb60 2023-07-14T11:23:12.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.805+0000 7f80be431b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7992bac0 con 0x55cc799d2c60 2023-07-14T11:23:12.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.805+0000 7f80be431b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc79924f50 con 0x55cc799d2350 2023-07-14T11:23:12.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f809bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f809bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39680/0 (socket says 172.21.15.161:39680) 2023-07-14T11:23:12.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f809bfff700 1 -- 172.21.15.161:0/1132448463 learned_addr learned my addr 172.21.15.161:0/1132448463 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:12.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f80a919c700 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc799d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f80a899b700 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2c60 0x55cc799ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f809bfff700 1 -- 172.21.15.161:0/1132448463 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 msgr2=0x55cc799d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:12.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f809bfff700 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc799d2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f809bfff700 1 -- 172.21.15.161:0/1132448463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2c60 msgr2=0x55cc799ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:12.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.806+0000 7f809bfff700 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2c60 0x55cc799ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.807+0000 7f809bfff700 1 -- 172.21.15.161:0/1132448463 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc79752e60 con 0x55cc799eeb60 2023-07-14T11:23:12.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.807+0000 7f8099ffb700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f808c0a14e0 0x7f808c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.807+0000 7f8099ffb700 1 -- 172.21.15.161:0/1669617540 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f8094014070 con 0x7f80a40731c0 2023-07-14T11:23:12.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.807+0000 7f80a919c700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f808c0a14e0 0x7f808c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.807+0000 7f809bfff700 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799f0f40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f80940caa00 tx=0x7f80940d3470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f748943e1eda63b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:12.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.808+0000 7f808affd700 1 -- 172.21.15.161:0/1132448463 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094099ff0 con 0x55cc799eeb60 2023-07-14T11:23:12.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.808+0000 7f808affd700 1 -- 172.21.15.161:0/1132448463 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8094099d60 con 0x55cc799eeb60 2023-07-14T11:23:12.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.808+0000 7f808affd700 1 -- 172.21.15.161:0/1132448463 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809409a220 con 0x55cc799eeb60 2023-07-14T11:23:12.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 -- 172.21.15.161:0/1132448463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 msgr2=0x55cc799f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80a919c700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f808c0a14e0 0x7f808c0a3990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f809c007e20 tx=0x7f809c009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:12.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799f0f40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f80940caa00 tx=0x7f80940d3470 comp rx=0 tx=0).stop 2023-07-14T11:23:12.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 -- 172.21.15.161:0/1132448463 shutdown_connections 2023-07-14T11:23:12.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799f0f40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc799d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 --2- 172.21.15.161:0/1132448463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2c60 0x55cc799ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 -- 172.21.15.161:0/1132448463 >> 172.21.15.161:0/1132448463 conn(0x55cc799c8e20 msgr2=0x55cc799ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:12.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 -- 172.21.15.161:0/1132448463 shutdown_connections 2023-07-14T11:23:12.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.809+0000 7f80be431b80 1 -- 172.21.15.161:0/1132448463 wait complete. 2023-07-14T11:23:12.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.810+0000 7f80be431b80 1 -- start start 2023-07-14T11:23:12.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 1 Processor -- start 2023-07-14T11:23:12.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799ee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc799eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799eeb60 0x55cc799ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc798c4b40 con 0x55cc799d2c60 2023-07-14T11:23:12.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc798c4840 con 0x55cc799d2350 2023-07-14T11:23:12.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc797a89a0 con 0x55cc799eeb60 2023-07-14T11:23:12.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.811+0000 7f80be431b80 10 client.0 _ll_register_callbacks cb 0x55cc798c27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:12.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f80a899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f809bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc799eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f809bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc799eb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39684/0 (socket says 172.21.15.161:39684) 2023-07-14T11:23:12.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f80a899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799ee560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46072/0 (socket says 172.21.15.161:46072) 2023-07-14T11:23:12.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f809bfff700 1 -- 172.21.15.161:0/2525248913 learned_addr learned my addr 172.21.15.161:0/2525248913 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:12.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f80a919c700 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799eeb60 0x55cc799ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f809bfff700 1 -- 172.21.15.161:0/2525248913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799eeb60 msgr2=0x55cc799ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:12.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f809bfff700 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799eeb60 0x55cc799ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f809bfff700 1 -- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 msgr2=0x55cc799ee560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:12.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.812+0000 7f809bfff700 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799ee560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.813+0000 7f809bfff700 1 -- 172.21.15.161:0/2525248913 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41a8e70 con 0x55cc799d2c60 2023-07-14T11:23:12.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.813+0000 7f80a899b700 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:12.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.813+0000 7f809bfff700 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc799eb850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f80940cf040 tx=0x7f8094000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10af6e3b981ea277 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:12.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.813+0000 7f8083fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:12.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.813+0000 7f8083fff700 1 -- 172.21.15.161:0/2525248913 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940cd530 con 0x55cc799d2c60 2023-07-14T11:23:12.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.813+0000 7f8083fff700 1 -- 172.21.15.161:0/2525248913 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f8094003e60 con 0x55cc799d2c60 2023-07-14T11:23:12.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.813+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc7987e570 con 0x55cc799d2c60 2023-07-14T11:23:12.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.814+0000 7f8083fff700 1 -- 172.21.15.161:0/2525248913 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8094099b80 con 0x55cc799d2c60 2023-07-14T11:23:12.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.814+0000 7f80be431b80 10 client.15816 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:12.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.814+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55cc7973bce0 con 0x55cc799d2c60 2023-07-14T11:23:12.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.814+0000 7f8083fff700 1 -- 172.21.15.161:0/2525248913 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc7987e570 con 0x55cc799d2c60 2023-07-14T11:23:12.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f8083fff700 1 -- 172.21.15.161:0/2525248913 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f809409a480 con 0x55cc799d2c60 2023-07-14T11:23:12.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f80be431b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:12.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f80be431b80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:12.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f80be431b80 20 client.15816 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:12.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f80be431b80 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc799fe170 0x55cc79a00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f80be431b80 4 client.15816 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:12.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc7991df30 con 0x55cc799fe170 2023-07-14T11:23:12.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.815+0000 7f80a899b700 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc799fe170 0x55cc79a00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.816+0000 7f80a899b700 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc799fe170 0x55cc79a00580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af79149e2e8f90da server_cookie=ac0815cfdd827bc7 in_seq=0 out_seq=0 2023-07-14T11:23:12.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.816+0000 7f8083fff700 10 client.15816 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:12.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f8083fff700 1 -- 172.21.15.161:0/2525248913 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cc7991df30 con 0x55cc799fe170 2023-07-14T11:23:12.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f8083fff700 10 client.15816 handle_command_reply: tid=0 2023-07-14T11:23:12.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 client.15816 shutdown 2023-07-14T11:23:12.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 msgr2=0x55cc799eb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc799eb850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f80940cf040 tx=0x7f8094000bc0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 shutdown_connections 2023-07-14T11:23:12.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc799fe170 0x55cc79a00580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799eeb60 0x55cc799ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 --2- 172.21.15.161:0/2525248913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc799eb850 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 >> 172.21.15.161:0/2525248913 conn(0x55cc799c8e20 msgr2=0x55cc799ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:12.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f8083fff700 0 client.15816 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:12.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.817+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 shutdown_connections 2023-07-14T11:23:12.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.818+0000 7f80be431b80 1 -- 172.21.15.161:0/2525248913 wait complete. 2023-07-14T11:23:12.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.818+0000 7f80be431b80 20 client.15816 trim_cache size 0 max 16384 2023-07-14T11:23:12.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 Processor -- start 2023-07-14T11:23:12.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 -- start start 2023-07-14T11:23:12.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799eb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2c60 0x55cc799eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc799d4220 con 0x55cc799eeb60 2023-07-14T11:23:12.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc799e9db0 con 0x55cc799d2350 2023-07-14T11:23:12.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.852+0000 7f80be431b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc799ee810 con 0x55cc799d2c60 2023-07-14T11:23:12.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799eb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46088/0 (socket says 172.21.15.161:46088) 2023-07-14T11:23:12.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 -- 172.21.15.161:0/2043433604 learned_addr learned my addr 172.21.15.161:0/2043433604 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:12.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a919c700 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 -- 172.21.15.161:0/2043433604 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2c60 msgr2=0x55cc799eb550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:23:12.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2c60 0x55cc799eb550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 -- 172.21.15.161:0/2043433604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 msgr2=0x55cc799e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:12.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.853+0000 7f80a899b700 1 -- 172.21.15.161:0/2043433604 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc797792d0 con 0x55cc799d2350 2023-07-14T11:23:12.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.854+0000 7f80a899b700 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799eb010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c010 tx=0x7f80a000f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61f4acb4854db7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:12.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.854+0000 7f808b7fe700 1 -- 172.21.15.161:0/2043433604 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000f930 con 0x55cc799d2350 2023-07-14T11:23:12.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.854+0000 7f808b7fe700 1 -- 172.21.15.161:0/2043433604 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80a001d070 con 0x55cc799d2350 2023-07-14T11:23:12.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.854+0000 7f808b7fe700 1 -- 172.21.15.161:0/2043433604 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00052a0 con 0x55cc799d2350 2023-07-14T11:23:12.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 -- 172.21.15.161:0/2043433604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 msgr2=0x55cc799eb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799eb010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c010 tx=0x7f80a000f630 comp rx=0 tx=0).stop 2023-07-14T11:23:12.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 -- 172.21.15.161:0/2043433604 shutdown_connections 2023-07-14T11:23:12.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799eeb60 0x55cc799e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799d2350 0x55cc799eb010 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 --2- 172.21.15.161:0/2043433604 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2c60 0x55cc799eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 -- 172.21.15.161:0/2043433604 >> 172.21.15.161:0/2043433604 conn(0x55cc799c8e20 msgr2=0x55cc799e6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:12.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 -- 172.21.15.161:0/2043433604 shutdown_connections 2023-07-14T11:23:12.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.855+0000 7f80be431b80 1 -- 172.21.15.161:0/2043433604 wait complete. 2023-07-14T11:23:12.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.856+0000 7f80be431b80 1 -- start start 2023-07-14T11:23:12.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.856+0000 7f80be431b80 1 Processor -- start 2023-07-14T11:23:12.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80be431b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc79a4b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80be431b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc79a4b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80be431b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799eeb60 0x55cc799f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80be431b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc799f4bb0 con 0x55cc799d2c60 2023-07-14T11:23:12.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80be431b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc799f4d20 con 0x55cc799eeb60 2023-07-14T11:23:12.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80be431b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc799f4ee0 con 0x55cc799d2350 2023-07-14T11:23:12.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80be431b80 10 client.0 _ll_register_callbacks cb 0x7f80940d32f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:12.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80a899b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc79a4b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80a899b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc79a4b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34100/0 (socket says 172.21.15.161:34100) 2023-07-14T11:23:12.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80a899b700 1 -- 172.21.15.161:0/2579011913 learned_addr learned my addr 172.21.15.161:0/2579011913 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:12.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.857+0000 7f80a919c700 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799eeb60 0x55cc799f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f80a899b700 1 -- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799eeb60 msgr2=0x55cc799f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:12.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f80a899b700 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799eeb60 0x55cc799f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f80a899b700 1 -- 172.21.15.161:0/2579011913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 msgr2=0x55cc79a4b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:12.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f80a899b700 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc79a4b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f80a899b700 1 -- 172.21.15.161:0/2579011913 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a410f3a0 con 0x55cc799d2350 2023-07-14T11:23:12.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f80a919c700 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799eeb60 0x55cc799f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:12.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f80a899b700 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc79a4b3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f80a0005e90 tx=0x7f80a000f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff3d13bc08f016e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:12.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f8083fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:12.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.858+0000 7f8083fff700 1 -- 172.21.15.161:0/2579011913 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000bb30 con 0x55cc799d2350 2023-07-14T11:23:12.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.859+0000 7f8083fff700 1 -- 172.21.15.161:0/2579011913 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f80a0032070 con 0x55cc799d2350 2023-07-14T11:23:12.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.859+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8094099d60 con 0x55cc799d2350 2023-07-14T11:23:12.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.859+0000 7f8083fff700 1 -- 172.21.15.161:0/2579011913 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80a000b770 con 0x55cc799d2350 2023-07-14T11:23:12.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.859+0000 7f80be431b80 10 client.25855 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:12.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.859+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55cc79751330 con 0x55cc799d2350 2023-07-14T11:23:12.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f8083fff700 1 -- 172.21.15.161:0/2579011913 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8094099d60 con 0x55cc799d2350 2023-07-14T11:23:12.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f8083fff700 1 -- 172.21.15.161:0/2579011913 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f80a0009920 con 0x55cc799d2350 2023-07-14T11:23:12.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f80be431b80 10 client.25855 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:12.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f80be431b80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:12.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f80be431b80 20 client.25855 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:12.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f80be431b80 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc79a53dd0 0x55cc79a561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:12.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f80be431b80 4 client.25855 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:23:12.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.860+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cc798e00d0 con 0x55cc79a53dd0 2023-07-14T11:23:12.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.861+0000 7f809bfff700 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc79a53dd0 0x55cc79a561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:12.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.861+0000 7f809bfff700 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc79a53dd0 0x55cc79a561b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4444db9afae76e server_cookie=9e0ddcecc811912d in_seq=0 out_seq=0 2023-07-14T11:23:12.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.861+0000 7f8083fff700 10 client.25855 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:12.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.862+0000 7f8083fff700 1 -- 172.21.15.161:0/2579011913 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cc798e00d0 con 0x55cc79a53dd0 2023-07-14T11:23:12.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.862+0000 7f8083fff700 10 client.25855 handle_command_reply: tid=0 2023-07-14T11:23:12.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.862+0000 7f80be431b80 1 client.25855 shutdown 2023-07-14T11:23:12.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 msgr2=0x55cc79a4b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc79a4b3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f80a0005e90 tx=0x7f80a000f9e0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 shutdown_connections 2023-07-14T11:23:12.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc79a53dd0 0x55cc79a561b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc799eeb60 0x55cc799f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc799d2350 0x55cc79a4b3f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f8083fff700 0 client.25855 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:12.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 --2- 172.21.15.161:0/2579011913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc799d2c60 0x55cc79a4b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 >> 172.21.15.161:0/2579011913 conn(0x55cc799c8e20 msgr2=0x55cc799ed0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:12.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 shutdown_connections 2023-07-14T11:23:12.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.863+0000 7f80be431b80 1 -- 172.21.15.161:0/2579011913 wait complete. 2023-07-14T11:23:12.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.864+0000 7f80be431b80 20 client.25855 trim_cache size 0 max 16384 2023-07-14T11:23:12.875 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:23:12.876 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:23:12.876 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "7cc23272-11fd-4060-afcc-426d111f550b", 2023-07-14T11:23:12.876 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:23:12.876 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:23:12.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.864+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f808c0a14e0 msgr2=0x7f808c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.864+0000 7f80aabff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f808c0a14e0 0x7f808c0a3990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f809c007e20 tx=0x7f809c009040 comp rx=0 tx=0).stop 2023-07-14T11:23:12.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.864+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 msgr2=0x7f80a40fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:12.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.864+0000 7f80aabff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 0x7f80a40fff80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f809400ec10 tx=0x7f809400ef20 comp rx=0 tx=0).stop 2023-07-14T11:23:12.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 shutdown_connections 2023-07-14T11:23:12.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f808c0a14e0 0x7f808c0a3990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a40728b0 0x7f80a40ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f80a410c320 0x7f80a41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 --2- 172.21.15.161:0/1669617540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a40731c0 0x7f80a40fff80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:12.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 >> 172.21.15.161:0/1669617540 conn(0x7f80a40fb5e0 msgr2=0x7f80a410e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:12.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 shutdown_connections 2023-07-14T11:23:12.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:12.865+0000 7f80aabff700 1 -- 172.21.15.161:0/1669617540 wait complete. 2023-07-14T11:23:12.890 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:23:13.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.139+0000 7f93d49ff700 1 -- 172.21.15.161:0/2895856046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d010c310 msgr2=0x7f93d010e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.139+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2895856046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d010c310 0x7f93d010e6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b0a0 tx=0x7f93c800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.140+0000 7f93d49ff700 1 -- 172.21.15.161:0/2895856046 shutdown_connections 2023-07-14T11:23:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.140+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2895856046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d010c310 0x7f93d010e6f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.140+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2895856046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d00728b0 0x7f93d0072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.140+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2895856046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93d00731c0 0x7f93d010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.140+0000 7f93d49ff700 1 -- 172.21.15.161:0/2895856046 >> 172.21.15.161:0/2895856046 conn(0x7f93d00fb5f0 msgr2=0x7f93d00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.140+0000 7f93d49ff700 1 -- 172.21.15.161:0/2895856046 shutdown_connections 2023-07-14T11:23:13.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.140+0000 7f93d49ff700 1 -- 172.21.15.161:0/2895856046 wait complete. 2023-07-14T11:23:13.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 Processor -- start 2023-07-14T11:23:13.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 -- start start 2023-07-14T11:23:13.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d00728b0 0x7f93d0197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93d00731c0 0x7f93d0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d010c310 0x7f93d019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d019cef0 con 0x7f93d00728b0 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d019d060 con 0x7f93d010c310 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.141+0000 7f93d49ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d019d220 con 0x7f93d00731c0 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d00728b0 0x7f93d0197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d00728b0 0x7f93d0197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39714/0 (socket says 172.21.15.161:39714) 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 -- 172.21.15.161:0/2961439452 learned_addr learned my addr 172.21.15.161:0/2961439452 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:13.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93d00731c0 0x7f93d0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ced9d700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d010c310 0x7f93d019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 -- 172.21.15.161:0/2961439452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93d00731c0 msgr2=0x7f93d0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93d00731c0 0x7f93d0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 -- 172.21.15.161:0/2961439452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d010c310 msgr2=0x7f93d019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d010c310 0x7f93d019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ce59c700 1 -- 172.21.15.161:0/2961439452 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8009d00 con 0x7f93d00728b0 2023-07-14T11:23:13.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93d00731c0 0x7f93d0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:13.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.142+0000 7f93ced9d700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d010c310 0x7f93d019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:13.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.143+0000 7f93ce59c700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d00728b0 0x7f93d0197f10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e9a0 tx=0x7f93c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:13.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.143+0000 7f93bf7fe700 1 -- 172.21.15.161:0/2961439452 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0009790 con 0x7f93d00728b0 2023-07-14T11:23:13.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.143+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d019d500 con 0x7f93d00728b0 2023-07-14T11:23:13.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.143+0000 7f93bf7fe700 1 -- 172.21.15.161:0/2961439452 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c0010430 con 0x7f93d00728b0 2023-07-14T11:23:13.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.143+0000 7f93bf7fe700 1 -- 172.21.15.161:0/2961439452 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0018680 con 0x7f93d00728b0 2023-07-14T11:23:13.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.143+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d01a4d40 con 0x7f93d00728b0 2023-07-14T11:23:13.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.146+0000 7f93bf7fe700 1 -- 172.21.15.161:0/2961439452 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f93c00105a0 con 0x7f93d00728b0 2023-07-14T11:23:13.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 Processor -- start 2023-07-14T11:23:13.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 -- start start 2023-07-14T11:23:13.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93bf7fe700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93b80a1460 0x7f93b80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bd26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bdd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93bf7fe700 1 -- 172.21.15.161:0/2961439452 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f93c0014070 con 0x7f93d00728b0 2023-07-14T11:23:13.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460ad8cb0 con 0x55b460bd22d0 2023-07-14T11:23:13.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460b2ba40 con 0x55b460bddad0 2023-07-14T11:23:13.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93e8231b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460b24ed0 con 0x55b460bd2be0 2023-07-14T11:23:13.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.148+0000 7f93ced9d700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93b80a1460 0x7f93b80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bdd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bd26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bdd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34124/0 (socket says 172.21.15.161:34124) 2023-07-14T11:23:13.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 -- 172.21.15.161:0/2933855707 learned_addr learned my addr 172.21.15.161:0/2933855707 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:13.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93ced9d700 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 -- 172.21.15.161:0/2933855707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 msgr2=0x55b460bdfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 -- 172.21.15.161:0/2933855707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 msgr2=0x55b460bd26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bd26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93cdd9b700 1 -- 172.21.15.161:0/2933855707 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b460952de0 con 0x55b460bd2be0 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93ced9d700 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93ce59c700 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bd26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.149+0000 7f93ced9d700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93b80a1460 0x7f93b80a3910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b780 tx=0x7f93c8009560 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.150+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bdd590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93c40114f0 tx=0x7f93c4011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74986b4bb2f96cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.150+0000 7f93bcff9700 1 -- 172.21.15.161:0/2933855707 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4004230 con 0x55b460bd2be0 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.150+0000 7f93bcff9700 1 -- 172.21.15.161:0/2933855707 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c4004390 con 0x55b460bd2be0 2023-07-14T11:23:13.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.150+0000 7f93bcff9700 1 -- 172.21.15.161:0/2933855707 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c400e8b0 con 0x55b460bd2be0 2023-07-14T11:23:13.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 -- 172.21.15.161:0/2933855707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 msgr2=0x55b460bdd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bdd590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93c40114f0 tx=0x7f93c4011800 comp rx=0 tx=0).stop 2023-07-14T11:23:13.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 -- 172.21.15.161:0/2933855707 shutdown_connections 2023-07-14T11:23:13.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bd26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 --2- 172.21.15.161:0/2933855707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bdd590 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 -- 172.21.15.161:0/2933855707 >> 172.21.15.161:0/2933855707 conn(0x55b460bc8da0 msgr2=0x55b460bce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:13.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 -- 172.21.15.161:0/2933855707 shutdown_connections 2023-07-14T11:23:13.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.151+0000 7f93e8231b80 1 -- 172.21.15.161:0/2933855707 wait complete. 2023-07-14T11:23:13.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.152+0000 7f93e8231b80 1 -- start start 2023-07-14T11:23:13.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.152+0000 7f93e8231b80 1 Processor -- start 2023-07-14T11:23:13.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.152+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bdd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.152+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bda6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.152+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93e8231b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460ac4ac0 con 0x55b460bd22d0 2023-07-14T11:23:13.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93e8231b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460ac47c0 con 0x55b460bddad0 2023-07-14T11:23:13.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93e8231b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4609a8920 con 0x55b460bd2be0 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93e8231b80 10 client.0 _ll_register_callbacks cb 0x55b460ac2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bdd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bdd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39736/0 (socket says 172.21.15.161:39736) 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 -- 172.21.15.161:0/3686433625 learned_addr learned my addr 172.21.15.161:0/3686433625 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ced9d700 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bda6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 -- 172.21.15.161:0/3686433625 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 msgr2=0x55b460bda6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bda6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 -- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 msgr2=0x55b460bdacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 -- 172.21.15.161:0/3686433625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d01a4d40 con 0x55b460bd22d0 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93ce59c700 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bdd460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e970 tx=0x7f93c009a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0678f43fc5888e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.153+0000 7f93b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3686433625 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c00b1070 con 0x55b460bd22d0 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3686433625 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f93c009dfd0 con 0x55b460bd22d0 2023-07-14T11:23:13.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b460a7e4f0 con 0x55b460bd22d0 2023-07-14T11:23:13.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3686433625 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93c00a23f0 con 0x55b460bd22d0 2023-07-14T11:23:13.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 10 client.15831 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:13.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f93c4009cd0 con 0x55b460bd22d0 2023-07-14T11:23:13.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3686433625 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c00c8080 con 0x55b460bd22d0 2023-07-14T11:23:13.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3686433625 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93c00a1f60 con 0x55b460bd22d0 2023-07-14T11:23:13.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 10 client.15831 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:13.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 20 client.15831 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460bed320 0x55b460bef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 4 client.15831 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b460b1deb0 con 0x55b460bed320 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460bed320 0x55b460bef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460bed320 0x55b460bef700 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83fdd8c20d4e4103 server_cookie=6c375ee51d925471 in_seq=0 out_seq=0 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.154+0000 7f93b5ffb700 10 client.15831 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:13.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3686433625 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b460b1deb0 con 0x55b460bed320 2023-07-14T11:23:13.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93b5ffb700 10 client.15831 handle_command_reply: tid=0 2023-07-14T11:23:13.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 client.15831 shutdown 2023-07-14T11:23:13.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 msgr2=0x55b460bdd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bdd460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e970 tx=0x7f93c009a4e0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 shutdown_connections 2023-07-14T11:23:13.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460bed320 0x55b460bef700 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd22d0 0x55b460bdd460 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bdacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93b5ffb700 0 client.15831 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:13.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.155+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3686433625 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460bda6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.156+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 >> 172.21.15.161:0/3686433625 conn(0x55b460bc8da0 msgr2=0x55b460bce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:13.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.156+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 shutdown_connections 2023-07-14T11:23:13.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.156+0000 7f93e8231b80 1 -- 172.21.15.161:0/3686433625 wait complete. 2023-07-14T11:23:13.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.156+0000 7f93e8231b80 20 client.15831 trim_cache size 0 max 16384 2023-07-14T11:23:13.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.173+0000 7f93e8231b80 1 Processor -- start 2023-07-14T11:23:13.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.173+0000 7f93e8231b80 1 -- start start 2023-07-14T11:23:13.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd22d0 0x55b460be44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd2be0 0x55b460bd4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bd7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93e8231b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460bd3540 con 0x55b460bd2be0 2023-07-14T11:23:13.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93e8231b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460bdbcc0 con 0x55b460bddad0 2023-07-14T11:23:13.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93e8231b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460bd8c40 con 0x55b460bd22d0 2023-07-14T11:23:13.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93cdd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd2be0 0x55b460bd4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93cdd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd2be0 0x55b460bd4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39752/0 (socket says 172.21.15.161:39752) 2023-07-14T11:23:13.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93cdd9b700 1 -- 172.21.15.161:0/1411048709 learned_addr learned my addr 172.21.15.161:0/1411048709 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:13.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93ced9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bd7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93ce59c700 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd22d0 0x55b460be44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93cdd9b700 1 -- 172.21.15.161:0/1411048709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd22d0 msgr2=0x55b460be44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd22d0 0x55b460be44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.174+0000 7f93cdd9b700 1 -- 172.21.15.161:0/1411048709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 msgr2=0x55b460bd7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.175+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bd7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.175+0000 7f93cdd9b700 1 -- 172.21.15.161:0/1411048709 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46093bc60 con 0x55b460bd2be0 2023-07-14T11:23:13.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.175+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd2be0 0x55b460bd4af0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f93c40114c0 tx=0x7f93c4011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc2418a41b29ac7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:13.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.175+0000 7f93bd7fa700 1 -- 172.21.15.161:0/1411048709 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c402a300 con 0x55b460bd2be0 2023-07-14T11:23:13.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.175+0000 7f93bd7fa700 1 -- 172.21.15.161:0/1411048709 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c402a460 con 0x55b460bd2be0 2023-07-14T11:23:13.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.175+0000 7f93bd7fa700 1 -- 172.21.15.161:0/1411048709 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c40052a0 con 0x55b460bd2be0 2023-07-14T11:23:13.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 -- 172.21.15.161:0/1411048709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd2be0 msgr2=0x55b460bd4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd2be0 0x55b460bd4af0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f93c40114c0 tx=0x7f93c4011800 comp rx=0 tx=0).stop 2023-07-14T11:23:13.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 -- 172.21.15.161:0/1411048709 shutdown_connections 2023-07-14T11:23:13.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bddad0 0x55b460bd7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd22d0 0x55b460be44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 --2- 172.21.15.161:0/1411048709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bd2be0 0x55b460bd4af0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 -- 172.21.15.161:0/1411048709 >> 172.21.15.161:0/1411048709 conn(0x55b460bc8da0 msgr2=0x55b460bd43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:13.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 -- 172.21.15.161:0/1411048709 shutdown_connections 2023-07-14T11:23:13.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 -- 172.21.15.161:0/1411048709 wait complete. 2023-07-14T11:23:13.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.176+0000 7f93e8231b80 1 -- start start 2023-07-14T11:23:13.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 1 Processor -- start 2023-07-14T11:23:13.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bd22d0 0x55b460c3aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460c3b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bddad0 0x55b460c35230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460c35810 con 0x55b460bddad0 2023-07-14T11:23:13.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460c35980 con 0x55b460bd22d0 2023-07-14T11:23:13.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b460c35b40 con 0x55b460bd2be0 2023-07-14T11:23:13.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93e8231b80 10 client.0 _ll_register_callbacks cb 0x55b460ac2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:13.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93cdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460c3b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bd22d0 0x55b460c3aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93cdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460c3b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34148/0 (socket says 172.21.15.161:34148) 2023-07-14T11:23:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.177+0000 7f93cdd9b700 1 -- 172.21.15.161:0/3380056581 learned_addr learned my addr 172.21.15.161:0/3380056581 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93ced9d700 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bddad0 0x55b460c35230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93ce59c700 1 -- 172.21.15.161:0/3380056581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 msgr2=0x55b460c3b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93ce59c700 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460c3b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93ce59c700 1 -- 172.21.15.161:0/3380056581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bddad0 msgr2=0x55b460c35230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:13.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93ce59c700 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bddad0 0x55b460c35230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93ce59c700 1 -- 172.21.15.161:0/3380056581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b460952de0 con 0x55b460bd22d0 2023-07-14T11:23:13.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460c3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:13.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93ce59c700 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bd22d0 0x55b460c3aed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93c000c2d0 tx=0x7f93c0010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20648d5d51681868 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:13.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:23:13.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3380056581 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c00b1070 con 0x55b460bd22d0 2023-07-14T11:23:13.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3380056581 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f93c009dfd0 con 0x55b460bd22d0 2023-07-14T11:23:13.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.178+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93c4004390 con 0x55b460bd22d0 2023-07-14T11:23:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3380056581 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93c00ca630 con 0x55b460bd22d0 2023-07-14T11:23:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 10 client.25880 fetch_fsmap learned FSMap version 22 2023-07-14T11:23:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b460979250 con 0x55b460bd22d0 2023-07-14T11:23:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3380056581 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c007c080 con 0x55b460bd22d0 2023-07-14T11:23:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3380056581 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93c00caa20 con 0x55b460bd22d0 2023-07-14T11:23:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 10 client.25880 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T11:23:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 20 client.25880 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460c42870 0x55b460c44c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 4 client.25880 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:23:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b460ae0050 con 0x55b460c42870 2023-07-14T11:23:13.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.179+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460c42870 0x55b460c44c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.180+0000 7f93b5ffb700 10 client.25880 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.180+0000 7f93cdd9b700 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460c42870 0x55b460c44c80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b53a977b5e7c25f server_cookie=27f44cb40fdef450 in_seq=0 out_seq=0 2023-07-14T11:23:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.180+0000 7f93b5ffb700 1 -- 172.21.15.161:0/3380056581 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b460ae0050 con 0x55b460c42870 2023-07-14T11:23:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.180+0000 7f93b5ffb700 10 client.25880 handle_command_reply: tid=0 2023-07-14T11:23:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.180+0000 7f93e8231b80 1 client.25880 shutdown 2023-07-14T11:23:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bd22d0 msgr2=0x55b460c3aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bd22d0 0x55b460c3aed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93c000c2d0 tx=0x7f93c0010c80 comp rx=0 tx=0).stop 2023-07-14T11:23:13.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 shutdown_connections 2023-07-14T11:23:13.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b460c42870 0x55b460c44c80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b460bddad0 0x55b460c35230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b460bd22d0 0x55b460c3aed0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93b5ffb700 0 client.25880 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:13.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 --2- 172.21.15.161:0/3380056581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b460bd2be0 0x55b460c3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 >> 172.21.15.161:0/3380056581 conn(0x55b460bc8da0 msgr2=0x55b460bd4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:13.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 shutdown_connections 2023-07-14T11:23:13.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 1 -- 172.21.15.161:0/3380056581 wait complete. 2023-07-14T11:23:13.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.181+0000 7f93e8231b80 20 client.25880 trim_cache size 0 max 16384 2023-07-14T11:23:13.194 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:23:13.194 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:23:13.195 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:23:13.195 INFO:teuthology.orchestra.run.smithi161.stdout: "7cc23272-11fd-4060-afcc-426d111f550b": { 2023-07-14T11:23:13.195 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:23:13.195 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "7cc23272-11fd-4060-afcc-426d111f550b", 2023-07-14T11:23:13.195 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:23:13.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:23:13.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:23:13.196 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:23:13.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93b80a1460 msgr2=0x7f93b80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93b80a1460 0x7f93b80a3910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b780 tx=0x7f93c8009560 comp rx=0 tx=0).stop 2023-07-14T11:23:13.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d00728b0 msgr2=0x7f93d0197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:13.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d00728b0 0x7f93d0197f10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e9a0 tx=0x7f93c000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 shutdown_connections 2023-07-14T11:23:13.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93b80a1460 0x7f93b80a3910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d00728b0 0x7f93d0197f10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d010c310 0x7f93d019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 --2- 172.21.15.161:0/2961439452 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93d00731c0 0x7f93d0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:13.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 >> 172.21.15.161:0/2961439452 conn(0x7f93d00fb5f0 msgr2=0x7f93d0103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:13.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 shutdown_connections 2023-07-14T11:23:13.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:13.182+0000 7f93d49ff700 1 -- 172.21.15.161:0/2961439452 wait complete. 2023-07-14T11:23:13.204 INFO:tasks.cephfs.filesystem:scrub status for tag:7cc23272-11fd-4060-afcc-426d111f550b - {'path': '/', 'tag': '7cc23272-11fd-4060-afcc-426d111f550b', 'options': 'recursive,force'} 2023-07-14T11:23:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:13 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:23:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:13 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:13.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:13 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:23:13.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:13 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:13.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:13 smithi161 ceph-mon[95526]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:23:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:13 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:23:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:13 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:13 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:23:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:13 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:13 smithi175 ceph-mon[129081]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:23:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:13 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:23:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:13 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:13 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:23:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:13 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:13 smithi182 ceph-mon[129366]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:23:16.282 INFO:tasks.workunit.client.0.smithi161.stderr:Updating files: 64% (7499/11710) Updating files: 65% (7612/11710) Updating files: 66% (7729/11710) Updating files: 67% (7846/11710) Updating files: 68% (7963/11710) Updating files: 69% (8080/11710) Updating files: 70% (8197/11710) Updating files: 71% (8315/11710) Updating files: 72% (8432/11710) Updating files: 73% (8549/11710) Updating files: 74% (8666/11710) Updating files: 75% (8783/11710) Updating files: 76% (8900/11710) Updating files: 77% (9017/11710) Updating files: 78% (9134/11710) Updating files: 79% (9251/11710) Updating files: 80% (9368/11710) Updating files: 81% (9486/11710) Updating files: 82% (9603/11710) Updating files: 83% (9720/11710) Updating files: 84% (9837/11710) Updating files: 85% (9954/11710) Updating files: 86% (10071/11710) Updating files: 87% (10188/11710) Updating files: 88% (10305/11710) Updating files: 89% (10422/11710) Updating files: 90% (10539/11710) Updating files: 91% (10657/11710) Updating files: 92% (10774/11710) Updating files: 93% (10891/11710) Updating files: 94% (11008/11710) Updating files: 95% (11125/11710) Updating files: 96% (11242/11710) Updating files: 97% (11359/11710) Updating files: 98% (11476/11710) Updating files: 99% (11593/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-07-14T11:23:16.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:16 smithi175 ceph-mon[129081]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:23:16.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:16 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:23:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:16 smithi182 ceph-mon[129366]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:23:16.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:16 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:23:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:16 smithi161 ceph-mon[95526]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T11:23:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:16 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:23:16.962 INFO:tasks.workunit.client.0.smithi161.stderr:Note: switching to '3fc0089cc1082313a98853d48b892acdc1385b8b'. 2023-07-14T11:23:16.962 INFO:tasks.workunit.client.0.smithi161.stderr: 2023-07-14T11:23:16.963 INFO:tasks.workunit.client.0.smithi161.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-14T11:23:16.963 INFO:tasks.workunit.client.0.smithi161.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-14T11:23:16.964 INFO:tasks.workunit.client.0.smithi161.stderr:state without impacting any branches by switching back to a branch. 2023-07-14T11:23:16.964 INFO:tasks.workunit.client.0.smithi161.stderr: 2023-07-14T11:23:16.964 INFO:tasks.workunit.client.0.smithi161.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-14T11:23:16.964 INFO:tasks.workunit.client.0.smithi161.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-14T11:23:16.964 INFO:tasks.workunit.client.0.smithi161.stderr: 2023-07-14T11:23:16.964 INFO:tasks.workunit.client.0.smithi161.stderr: git switch -c 2023-07-14T11:23:16.965 INFO:tasks.workunit.client.0.smithi161.stderr: 2023-07-14T11:23:16.965 INFO:tasks.workunit.client.0.smithi161.stderr:Or undo this operation with: 2023-07-14T11:23:16.965 INFO:tasks.workunit.client.0.smithi161.stderr: 2023-07-14T11:23:16.965 INFO:tasks.workunit.client.0.smithi161.stderr: git switch - 2023-07-14T11:23:16.965 INFO:tasks.workunit.client.0.smithi161.stderr: 2023-07-14T11:23:16.965 INFO:tasks.workunit.client.0.smithi161.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-14T11:23:16.965 INFO:tasks.workunit.client.0.smithi161.stderr: 2023-07-14T11:23:16.966 INFO:tasks.workunit.client.0.smithi161.stderr:HEAD is now at 3fc0089cc10 Merge remote-tracking branch 'rishabh/fs-auth-subcmd' into wip-rishabh-2023Jul13 2023-07-14T11:23:16.980 DEBUG:teuthology.orchestra.run.smithi161:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-14T11:23:17.032 INFO:tasks.workunit.client.0.smithi161.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-14T11:23:17.037 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-14T11:23:17.037 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-14T11:23:17.613 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-14T11:23:17.752 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-14T11:23:17.816 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-14T11:23:17.820 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-14T11:23:17.820 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-14T11:23:17.883 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-14T11:23:17.890 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:23:17.891 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-14T11:23:17.934 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-07-14T11:23:17.936 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-07-14T11:23:17.936 DEBUG:teuthology.orchestra.run.smithi161:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3fc0089cc1082313a98853d48b892acdc1385b8b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-07-14T11:23:18.033 INFO:tasks.workunit.client.0.smithi161.stderr:+ dbench 1 2023-07-14T11:23:18.052 INFO:tasks.workunit.client.0.smithi161.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-14T11:23:18.052 INFO:tasks.workunit.client.0.smithi161.stdout: 2023-07-14T11:23:18.052 INFO:tasks.workunit.client.0.smithi161.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-14T11:23:18.061 INFO:tasks.workunit.client.0.smithi161.stdout:0 of 1 processes prepared for launch 0 sec 2023-07-14T11:23:18.161 INFO:tasks.workunit.client.0.smithi161.stdout:1 of 1 processes prepared for launch 0 sec 2023-07-14T11:23:18.162 INFO:tasks.workunit.client.0.smithi161.stdout:releasing clients 2023-07-14T11:23:18.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:18 smithi175 ceph-mon[129081]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:23:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:18 smithi182 ceph-mon[129366]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:23:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:18 smithi161 ceph-mon[95526]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T11:23:19.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 3189 34.24 MB/sec warmup 1 sec latency 6.752 ms 2023-07-14T11:23:20.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 5988 23.61 MB/sec warmup 2 sec latency 5.988 ms 2023-07-14T11:23:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:20 smithi175 ceph-mon[129081]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:23:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:20 smithi182 ceph-mon[129366]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:23:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:20 smithi161 ceph-mon[95526]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T11:23:21.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 8611 21.43 MB/sec warmup 3 sec latency 6.029 ms 2023-07-14T11:23:22.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 11456 19.70 MB/sec warmup 4 sec latency 8.154 ms 2023-07-14T11:23:22.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:22 smithi175 ceph-mon[129081]: pgmap v36: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 417 KiB/s wr, 57 op/s 2023-07-14T11:23:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:22 smithi182 ceph-mon[129366]: pgmap v36: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 417 KiB/s wr, 57 op/s 2023-07-14T11:23:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:22 smithi161 ceph-mon[95526]: pgmap v36: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 417 KiB/s wr, 57 op/s 2023-07-14T11:23:23.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 14499 19.42 MB/sec warmup 5 sec latency 23.596 ms 2023-07-14T11:23:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:23:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:23 smithi161 ceph-mon[95526]: pgmap v37: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 417 KiB/s wr, 54 op/s 2023-07-14T11:23:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:23.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:23 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:24.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 16590 17.41 MB/sec warmup 6 sec latency 90.124 ms 2023-07-14T11:23:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:23:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:23 smithi175 ceph-mon[129081]: pgmap v37: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 417 KiB/s wr, 54 op/s 2023-07-14T11:23:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:23 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:23:24.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:23 smithi182 ceph-mon[129366]: pgmap v37: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 417 KiB/s wr, 54 op/s 2023-07-14T11:23:24.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:24.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:23 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:24.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:23:24.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:23:24.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:25.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 18879 16.94 MB/sec warmup 7 sec latency 12.899 ms 2023-07-14T11:23:25.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:23:25.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:23:25.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:25.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:23:25.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:23:25.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:23:26.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 21992 16.98 MB/sec warmup 8 sec latency 5.878 ms 2023-07-14T11:23:26.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:25 smithi175 ceph-mon[129081]: pgmap v38: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 417 KiB/s wr, 54 op/s 2023-07-14T11:23:26.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:25 smithi182 ceph-mon[129366]: pgmap v38: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 417 KiB/s wr, 54 op/s 2023-07-14T11:23:26.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:25 smithi161 ceph-mon[95526]: pgmap v38: 129 pgs: 129 active+clean; 5.6 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 417 KiB/s wr, 54 op/s 2023-07-14T11:23:27.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 25127 17.17 MB/sec warmup 9 sec latency 30.970 ms 2023-07-14T11:23:28.162 INFO:tasks.workunit.client.0.smithi161.stdout: 1 27802 16.74 MB/sec warmup 10 sec latency 17.989 ms 2023-07-14T11:23:28.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:28 smithi175 ceph-mon[129081]: pgmap v39: 129 pgs: 129 active+clean; 11 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2023-07-14T11:23:28.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:28 smithi182 ceph-mon[129366]: pgmap v39: 129 pgs: 129 active+clean; 11 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2023-07-14T11:23:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:28 smithi161 ceph-mon[95526]: pgmap v39: 129 pgs: 129 active+clean; 11 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2023-07-14T11:23:29.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 29871 16.27 MB/sec warmup 11 sec latency 77.528 ms 2023-07-14T11:23:30.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 32702 16.12 MB/sec warmup 12 sec latency 15.919 ms 2023-07-14T11:23:30.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:30 smithi175 ceph-mon[129081]: pgmap v40: 129 pgs: 129 active+clean; 11 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-07-14T11:23:30.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:30 smithi182 ceph-mon[129366]: pgmap v40: 129 pgs: 129 active+clean; 11 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-07-14T11:23:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:30 smithi161 ceph-mon[95526]: pgmap v40: 129 pgs: 129 active+clean; 11 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-07-14T11:23:31.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 35879 16.32 MB/sec warmup 13 sec latency 7.132 ms 2023-07-14T11:23:32.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 38876 16.19 MB/sec warmup 14 sec latency 6.907 ms 2023-07-14T11:23:32.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:32 smithi175 ceph-mon[129081]: pgmap v41: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-07-14T11:23:32.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:32 smithi182 ceph-mon[129366]: pgmap v41: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-07-14T11:23:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:32 smithi161 ceph-mon[95526]: pgmap v41: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-07-14T11:23:33.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 41465 15.89 MB/sec warmup 15 sec latency 8.705 ms 2023-07-14T11:23:33.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:33 smithi161 ceph-mon[95526]: pgmap v42: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2023-07-14T11:23:34.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 43424 15.68 MB/sec warmup 16 sec latency 74.102 ms 2023-07-14T11:23:34.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:33 smithi175 ceph-mon[129081]: pgmap v42: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2023-07-14T11:23:34.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:33 smithi182 ceph-mon[129366]: pgmap v42: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2023-07-14T11:23:35.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 46503 15.86 MB/sec warmup 17 sec latency 39.925 ms 2023-07-14T11:23:36.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 49319 15.71 MB/sec warmup 18 sec latency 6.974 ms 2023-07-14T11:23:36.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:36 smithi175 ceph-mon[129081]: pgmap v43: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2023-07-14T11:23:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:36 smithi182 ceph-mon[129366]: pgmap v43: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2023-07-14T11:23:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:36 smithi161 ceph-mon[95526]: pgmap v43: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2023-07-14T11:23:37.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 51465 15.47 MB/sec warmup 19 sec latency 6.921 ms 2023-07-14T11:23:38.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 54039 15.41 MB/sec warmup 20 sec latency 8.315 ms 2023-07-14T11:23:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:38 smithi175 ceph-mon[129081]: pgmap v44: 129 pgs: 129 active+clean; 24 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-07-14T11:23:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:38 smithi182 ceph-mon[129366]: pgmap v44: 129 pgs: 129 active+clean; 24 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-07-14T11:23:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:38 smithi161 ceph-mon[95526]: pgmap v44: 129 pgs: 129 active+clean; 24 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-07-14T11:23:39.163 INFO:tasks.workunit.client.0.smithi161.stdout: 1 57044 15.55 MB/sec warmup 21 sec latency 19.627 ms 2023-07-14T11:23:40.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 59754 15.43 MB/sec warmup 22 sec latency 6.960 ms 2023-07-14T11:23:40.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:40 smithi175 ceph-mon[129081]: pgmap v45: 129 pgs: 129 active+clean; 24 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-07-14T11:23:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:40 smithi182 ceph-mon[129366]: pgmap v45: 129 pgs: 129 active+clean; 24 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-07-14T11:23:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:40 smithi161 ceph-mon[95526]: pgmap v45: 129 pgs: 129 active+clean; 24 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-07-14T11:23:41.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 61852 15.27 MB/sec warmup 23 sec latency 15.408 ms 2023-07-14T11:23:42.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 64158 15.20 MB/sec warmup 24 sec latency 216.946 ms 2023-07-14T11:23:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:42 smithi175 ceph-mon[129081]: pgmap v46: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-07-14T11:23:42.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:42 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:23:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:42 smithi182 ceph-mon[129366]: pgmap v46: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-07-14T11:23:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:42 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:23:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:42 smithi161 ceph-mon[95526]: pgmap v46: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-07-14T11:23:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:42 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:23:43.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 64258 14.61 MB/sec warmup 25 sec latency 890.195 ms 2023-07-14T11:23:43.205 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:23:43.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 -- 172.21.15.161:0/141089503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 msgr2=0x7f59cc109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 --2- 172.21.15.161:0/141089503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc109aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009990 tx=0x7f59b4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 -- 172.21.15.161:0/141089503 shutdown_connections 2023-07-14T11:23:43.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 --2- 172.21.15.161:0/141089503 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f59cc10a070 0x7f59cc10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 --2- 172.21.15.161:0/141089503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59cc100ca0 0x7f59cc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 --2- 172.21.15.161:0/141089503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc109aa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 -- 172.21.15.161:0/141089503 >> 172.21.15.161:0/141089503 conn(0x7f59cc0fa4b0 msgr2=0x7f59cc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 -- 172.21.15.161:0/141089503 shutdown_connections 2023-07-14T11:23:43.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.415+0000 7f59d15ff700 1 -- 172.21.15.161:0/141089503 wait complete. 2023-07-14T11:23:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.416+0000 7f59d15ff700 1 Processor -- start 2023-07-14T11:23:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.416+0000 7f59d15ff700 1 -- start start 2023-07-14T11:23:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59d15ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f59cc100ca0 0x7f59cc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59d15ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59d15ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59cc10a070 0x7f59cc19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59d15ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc19ce80 con 0x7f59cc1015b0 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59d15ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc19cff0 con 0x7f59cc10a070 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59d15ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc19d1b0 con 0x7f59cc100ca0 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54752/0 (socket says 172.21.15.161:54752) 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 -- 172.21.15.161:0/1849931457 learned_addr learned my addr 172.21.15.161:0/1849931457 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59caffd700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f59cc100ca0 0x7f59cc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59cc10a070 0x7f59cc19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 -- 172.21.15.161:0/1849931457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f59cc100ca0 msgr2=0x7f59cc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f59cc100ca0 0x7f59cc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 -- 172.21.15.161:0/1849931457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59cc10a070 msgr2=0x7f59cc19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59cc10a070 0x7f59cc19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.417+0000 7f59c3fff700 1 -- 172.21.15.161:0/1849931457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40095f0 con 0x7f59cc1015b0 2023-07-14T11:23:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.418+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc1983e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005910 tx=0x7f59b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.418+0000 7f59c8ff9700 1 -- 172.21.15.161:0/1849931457 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b401d070 con 0x7f59cc1015b0 2023-07-14T11:23:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.418+0000 7f59c8ff9700 1 -- 172.21.15.161:0/1849931457 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b4022470 con 0x7f59cc1015b0 2023-07-14T11:23:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.418+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc19d430 con 0x7f59cc1015b0 2023-07-14T11:23:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.418+0000 7f59c8ff9700 1 -- 172.21.15.161:0/1849931457 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4017650 con 0x7f59cc1015b0 2023-07-14T11:23:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.418+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc1a4cd0 con 0x7f59cc1015b0 2023-07-14T11:23:43.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.421+0000 7f59c8ff9700 1 -- 172.21.15.161:0/1849931457 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f59b400fad0 con 0x7f59cc1015b0 2023-07-14T11:23:43.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.423+0000 7f59e4f49b80 1 Processor -- start 2023-07-14T11:23:43.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.423+0000 7f59e4f49b80 1 -- start start 2023-07-14T11:23:43.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59c8ff9700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59b80a13e0 0x7f59b80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f922d0 0x55d065f926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f92be0 0x55d065f9d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59c8ff9700 1 -- 172.21.15.161:0/1849931457 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f59b40ce7b0 con 0x7f59cc1015b0 2023-07-14T11:23:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f9dad0 0x55d065f9feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065e98cb0 con 0x55d065f9dad0 2023-07-14T11:23:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065eeba40 con 0x55d065f922d0 2023-07-14T11:23:43.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065ee4ed0 con 0x55d065f92be0 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59b80a13e0 0x7f59b80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59caffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f922d0 0x55d065f926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59caffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f922d0 0x55d065f926a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57274/0 (socket says 172.21.15.161:57274) 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59caffd700 1 -- 172.21.15.161:0/568164408 learned_addr learned my addr 172.21.15.161:0/568164408 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f9dad0 0x55d065f9feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.424+0000 7f59c3fff700 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f92be0 0x55d065f9d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59caffd700 1 -- 172.21.15.161:0/568164408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f92be0 msgr2=0x55d065f9d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59caffd700 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f92be0 0x55d065f9d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59caffd700 1 -- 172.21.15.161:0/568164408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f9dad0 msgr2=0x55d065f9feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59caffd700 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f9dad0 0x55d065f9feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59caffd700 1 -- 172.21.15.161:0/568164408 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d065d12de0 con 0x55d065f922d0 2023-07-14T11:23:43.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59caffd700 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f922d0 0x55d065f926a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f59bc007cf0 tx=0x7f59bc0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f34090ae5ad08e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59b80a13e0 0x7f59b80a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009ec0 tx=0x7f59c40092f0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59c17fa700 1 -- 172.21.15.161:0/568164408 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc010d10 con 0x55d065f922d0 2023-07-14T11:23:43.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59c17fa700 1 -- 172.21.15.161:0/568164408 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59bc015070 con 0x55d065f922d0 2023-07-14T11:23:43.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.425+0000 7f59c17fa700 1 -- 172.21.15.161:0/568164408 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc019630 con 0x55d065f922d0 2023-07-14T11:23:43.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 -- 172.21.15.161:0/568164408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f922d0 msgr2=0x55d065f926a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f922d0 0x55d065f926a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f59bc007cf0 tx=0x7f59bc0103d0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 -- 172.21.15.161:0/568164408 shutdown_connections 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f9dad0 0x55d065f9feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f922d0 0x55d065f926a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/568164408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f92be0 0x55d065f9d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 -- 172.21.15.161:0/568164408 >> 172.21.15.161:0/568164408 conn(0x55d065f88da0 msgr2=0x55d065f8e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 -- 172.21.15.161:0/568164408 shutdown_connections 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.426+0000 7f59e4f49b80 1 -- 172.21.15.161:0/568164408 wait complete. 2023-07-14T11:23:43.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 -- start start 2023-07-14T11:23:43.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 Processor -- start 2023-07-14T11:23:43.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f922d0 0x55d065f9d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f92be0 0x55d065f9a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f9dad0 0x55d065f9ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065e84ac0 con 0x55d065f92be0 2023-07-14T11:23:43.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065e847c0 con 0x55d065f9dad0 2023-07-14T11:23:43.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065d68920 con 0x55d065f922d0 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.427+0000 7f59e4f49b80 10 client.0 _ll_register_callbacks cb 0x55d065e82760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59caffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f922d0 0x55d065f9d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f92be0 0x55d065f9a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59caffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f922d0 0x55d065f9d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:48924/0 (socket says 172.21.15.161:48924) 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59caffd700 1 -- 172.21.15.161:0/1261301444 learned_addr learned my addr 172.21.15.161:0/1261301444 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f9dad0 0x55d065f9ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59c3fff700 1 -- 172.21.15.161:0/1261301444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f922d0 msgr2=0x55d065f9d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f922d0 0x55d065f9d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59c3fff700 1 -- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f9dad0 msgr2=0x55d065f9ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f9dad0 0x55d065f9ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59c3fff700 1 -- 172.21.15.161:0/1261301444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc007920 con 0x55d065f92be0 2023-07-14T11:23:43.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59caffd700 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f922d0 0x55d065f9d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:43.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f92be0 0x55d065f9a6d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59b40cb9f0 tx=0x7f59b409af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1022e59c0595854c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59b27fc700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:43.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59b27fc700 1 -- 172.21.15.161:0/1261301444 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b409b330 con 0x55d065f92be0 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.428+0000 7f59b27fc700 1 -- 172.21.15.161:0/1261301444 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f59b409f360 con 0x55d065f92be0 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d065e3e4f0 con 0x55d065f92be0 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59b27fc700 1 -- 172.21.15.161:0/1261301444 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b40a36f0 con 0x55d065f92be0 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 10 client.15849 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d065cfbc60 con 0x55d065f92be0 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59b27fc700 1 -- 172.21.15.161:0/1261301444 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d065e3e4f0 con 0x55d065f92be0 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59b27fc700 1 -- 172.21.15.161:0/1261301444 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f59b40ce460 con 0x55d065f92be0 2023-07-14T11:23:43.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 10 client.15849 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:43.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:43.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 20 client.15849 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:43.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065fad060 0x55d065faf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 4 client.15849 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:43.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.429+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d065eddeb0 con 0x55d065fad060 2023-07-14T11:23:43.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.430+0000 7f59caffd700 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065fad060 0x55d065faf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.430+0000 7f59caffd700 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065fad060 0x55d065faf470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc87c35d783a201a server_cookie=d5323027cf29cebd in_seq=0 out_seq=0 2023-07-14T11:23:43.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.430+0000 7f59b27fc700 10 client.15849 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:43.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59b27fc700 1 -- 172.21.15.161:0/1261301444 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d065eddeb0 con 0x55d065fad060 2023-07-14T11:23:43.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59b27fc700 10 client.15849 handle_command_reply: tid=0 2023-07-14T11:23:43.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 client.15849 shutdown 2023-07-14T11:23:43.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f92be0 msgr2=0x55d065f9a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f92be0 0x55d065f9a6d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59b40cb9f0 tx=0x7f59b409af40 comp rx=0 tx=0).stop 2023-07-14T11:23:43.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 shutdown_connections 2023-07-14T11:23:43.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065fad060 0x55d065faf470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f9dad0 0x55d065f9ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f922d0 0x55d065f9d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1261301444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f92be0 0x55d065f9a6d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59b27fc700 0 client.15849 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 >> 172.21.15.161:0/1261301444 conn(0x55d065f88da0 msgr2=0x55d065f8e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 shutdown_connections 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1261301444 wait complete. 2023-07-14T11:23:43.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.431+0000 7f59e4f49b80 20 client.15849 trim_cache size 0 max 16384 2023-07-14T11:23:43.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.448+0000 7f59e4f49b80 1 Processor -- start 2023-07-14T11:23:43.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59e4f49b80 1 -- start start 2023-07-14T11:23:43.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065f959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065f95f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065f9eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065f93e70 con 0x55d065f922d0 2023-07-14T11:23:43.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065eddee0 con 0x55d065f92be0 2023-07-14T11:23:43.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d065f8aec0 con 0x55d065f88da0 2023-07-14T11:23:43.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065f95f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065f95f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54790/0 (socket says 172.21.15.161:54790) 2023-07-14T11:23:43.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59c3fff700 1 -- 172.21.15.161:0/1003884215 learned_addr learned my addr 172.21.15.161:0/1003884215 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59caffd700 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065f959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.449+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065f9eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c3fff700 1 -- 172.21.15.161:0/1003884215 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 msgr2=0x55d065f959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065f959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c3fff700 1 -- 172.21.15.161:0/1003884215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 msgr2=0x55d065f9eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065f9eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c3fff700 1 -- 172.21.15.161:0/1003884215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d065d39250 con 0x55d065f922d0 2023-07-14T11:23:43.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c3fff700 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065f95f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f59b4097da0 tx=0x7f59b40998e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fe26aee9462794 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c1ffb700 1 -- 172.21.15.161:0/1003884215 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b409aa30 con 0x55d065f922d0 2023-07-14T11:23:43.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c1ffb700 1 -- 172.21.15.161:0/1003884215 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b409f360 con 0x55d065f922d0 2023-07-14T11:23:43.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59c1ffb700 1 -- 172.21.15.161:0/1003884215 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4022850 con 0x55d065f922d0 2023-07-14T11:23:43.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1003884215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 msgr2=0x55d065f95f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.450+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065f95f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f59b4097da0 tx=0x7f59b40998e0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1003884215 shutdown_connections 2023-07-14T11:23:43.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065f9eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065f959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/1003884215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065f95f30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1003884215 >> 172.21.15.161:0/1003884215 conn(0x55d065fa0020 msgr2=0x55d065fa9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1003884215 shutdown_connections 2023-07-14T11:23:43.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 -- 172.21.15.161:0/1003884215 wait complete. 2023-07-14T11:23:43.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 -- start start 2023-07-14T11:23:43.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.451+0000 7f59e4f49b80 1 Processor -- start 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065fa22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065fa2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59e4f49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065fffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d066000450 con 0x55d065f922d0 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0660005c0 con 0x55d065f92be0 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59e4f49b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d066000780 con 0x55d065f88da0 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59e4f49b80 10 client.0 _ll_register_callbacks cb 0x7f59b409b1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59caffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065fa22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065fa2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59caffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065fa22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:48932/0 (socket says 172.21.15.161:48932) 2023-07-14T11:23:43.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.452+0000 7f59caffd700 1 -- 172.21.15.161:0/3124088439 learned_addr learned my addr 172.21.15.161:0/3124088439 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:43.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065fffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59caffd700 1 -- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 msgr2=0x55d065fffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59caffd700 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065fffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59caffd700 1 -- 172.21.15.161:0/3124088439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 msgr2=0x55d065fa2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59caffd700 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065fa2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59caffd700 1 -- 172.21.15.161:0/3124088439 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d065cfbc60 con 0x55d065f88da0 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59cb7fe700 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065fffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59caffd700 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065fa22e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f59bc012db0 tx=0x7f59bc0073d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc26c602dad1e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59b27fc700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59b27fc700 1 -- 172.21.15.161:0/3124088439 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc028cf0 con 0x55d065f88da0 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59bc015070 con 0x55d065f88da0 2023-07-14T11:23:43.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.453+0000 7f59b27fc700 1 -- 172.21.15.161:0/3124088439 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f59bc010e50 con 0x55d065f88da0 2023-07-14T11:23:43.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59b27fc700 1 -- 172.21.15.161:0/3124088439 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59bc028e50 con 0x55d065f88da0 2023-07-14T11:23:43.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 10 client.25891 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:43.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d065d112b0 con 0x55d065f88da0 2023-07-14T11:23:43.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59b27fc700 1 -- 172.21.15.161:0/3124088439 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59bc015070 con 0x55d065f88da0 2023-07-14T11:23:43.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59b27fc700 1 -- 172.21.15.161:0/3124088439 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f59bc0050d0 con 0x55d065f88da0 2023-07-14T11:23:43.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 10 client.25891 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:43.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 10 client.25891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 20 client.25891 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065f9dad0 0x55d066003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 4 client.25891 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:23:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d065ea0050 con 0x55d065f9dad0 2023-07-14T11:23:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.454+0000 7f59c3fff700 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065f9dad0 0x55d066003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59c3fff700 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065f9dad0 0x55d066003df0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d065feba6f6541 server_cookie=212ba3eceb800d2f in_seq=0 out_seq=0 2023-07-14T11:23:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59b27fc700 10 client.25891 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59b27fc700 1 -- 172.21.15.161:0/3124088439 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d065ea0050 con 0x55d065f9dad0 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59b27fc700 10 client.25891 handle_command_reply: tid=0 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 client.25891 shutdown 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 msgr2=0x55d065fa22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065fa22e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f59bc012db0 tx=0x7f59bc0073d0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 shutdown_connections 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d065f9dad0 0x55d066003df0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d065f92be0 0x55d065fffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d065f88da0 0x55d065fa22e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 --2- 172.21.15.161:0/3124088439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d065f922d0 0x55d065fa2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59b27fc700 0 client.25891 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 >> 172.21.15.161:0/3124088439 conn(0x55d065fa0020 msgr2=0x55d065f959f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 shutdown_connections 2023-07-14T11:23:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.455+0000 7f59e4f49b80 1 -- 172.21.15.161:0/3124088439 wait complete. 2023-07-14T11:23:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59e4f49b80 20 client.25891 trim_cache size 0 max 16384 2023-07-14T11:23:43.467 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:23:43.467 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:23:43.467 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:23:43.467 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:23:43.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59b80a13e0 msgr2=0x7f59b80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59b80a13e0 0x7f59b80a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009ec0 tx=0x7f59c40092f0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 msgr2=0x7f59cc1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc1983e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005910 tx=0x7f59b4005740 comp rx=0 tx=0).stop 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 shutdown_connections 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59b80a13e0 0x7f59b80a3890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59cc10a070 0x7f59cc19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f59cc100ca0 0x7f59cc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 --2- 172.21.15.161:0/1849931457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc1015b0 0x7f59cc1983e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 >> 172.21.15.161:0/1849931457 conn(0x7f59cc0fa4b0 msgr2=0x7f59cc10c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 shutdown_connections 2023-07-14T11:23:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.456+0000 7f59d15ff700 1 -- 172.21.15.161:0/1849931457 wait complete. 2023-07-14T11:23:43.476 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:23:43.476 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:23:43.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 -- 172.21.15.161:0/1506947080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc101470 msgr2=0x7f27dc109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1506947080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc101470 0x7f27dc109b40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009990 tx=0x7f27cc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 -- 172.21.15.161:0/1506947080 shutdown_connections 2023-07-14T11:23:43.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1506947080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc10a080 0x7f27dc10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1506947080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27dc100b60 0x7f27dc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1506947080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc101470 0x7f27dc109b40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 -- 172.21.15.161:0/1506947080 >> 172.21.15.161:0/1506947080 conn(0x7f27dc0fa490 msgr2=0x7f27dc0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 -- 172.21.15.161:0/1506947080 shutdown_connections 2023-07-14T11:23:43.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.672+0000 7f27e39ff700 1 -- 172.21.15.161:0/1506947080 wait complete. 2023-07-14T11:23:43.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 Processor -- start 2023-07-14T11:23:43.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 -- start start 2023-07-14T11:23:43.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc100b60 0x7f27dc19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc101470 0x7f27dc19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27dc10a080 0x7f27dc195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc196080 con 0x7f27dc100b60 2023-07-14T11:23:43.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc1961f0 con 0x7f27dc101470 2023-07-14T11:23:43.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.673+0000 7f27e39ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc1963b0 con 0x7f27dc10a080 2023-07-14T11:23:43.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc100b60 0x7f27dc19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc100b60 0x7f27dc19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54822/0 (socket says 172.21.15.161:54822) 2023-07-14T11:23:43.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 -- 172.21.15.161:0/1294086476 learned_addr learned my addr 172.21.15.161:0/1294086476 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:43.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e0f9a700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc101470 0x7f27dc19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e1f9c700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27dc10a080 0x7f27dc195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 -- 172.21.15.161:0/1294086476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27dc10a080 msgr2=0x7f27dc195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27dc10a080 0x7f27dc195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 -- 172.21.15.161:0/1294086476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc101470 msgr2=0x7f27dc19bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc101470 0x7f27dc19bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 -- 172.21.15.161:0/1294086476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0095f0 con 0x7f27dc100b60 2023-07-14T11:23:43.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e0f9a700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc101470 0x7f27dc19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:43.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.674+0000 7f27e179b700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc100b60 0x7f27dc19b9a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f27d800d980 tx=0x7f27d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.675+0000 7f27d27fc700 1 -- 172.21.15.161:0/1294086476 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8004220 con 0x7f27dc100b60 2023-07-14T11:23:43.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.675+0000 7f27d27fc700 1 -- 172.21.15.161:0/1294086476 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d8007640 con 0x7f27dc100b60 2023-07-14T11:23:43.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.675+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc196690 con 0x7f27dc100b60 2023-07-14T11:23:43.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.675+0000 7f27d27fc700 1 -- 172.21.15.161:0/1294086476 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8005690 con 0x7f27dc100b60 2023-07-14T11:23:43.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.675+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc1a2990 con 0x7f27dc100b60 2023-07-14T11:23:43.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.676+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc067300 con 0x7f27dc100b60 2023-07-14T11:23:43.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.678+0000 7f27d27fc700 1 -- 172.21.15.161:0/1294086476 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f27d8004750 con 0x7f27dc100b60 2023-07-14T11:23:43.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.679+0000 7f27d27fc700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f27c80a1430 0x7f27c80a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:43.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.679+0000 7f27d27fc700 1 -- 172.21.15.161:0/1294086476 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f27d80cd130 con 0x7f27dc100b60 2023-07-14T11:23:43.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.679+0000 7f27e0f9a700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f27c80a1430 0x7f27c80a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:43.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.680+0000 7f27e0f9a700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f27c80a1430 0x7f27c80a38e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27dc10b590 tx=0x7f27cc005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:43.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.683+0000 7f27d27fc700 1 -- 172.21.15.161:0/1294086476 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f27d8095f60 con 0x7f27dc100b60 2023-07-14T11:23:43.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.972+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27dc196f20 con 0x7f27dc100b60 2023-07-14T11:23:43.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.973+0000 7f27d27fc700 1 -- 172.21.15.161:0/1294086476 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f27d8016090 con 0x7f27dc100b60 2023-07-14T11:23:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:23:43.976 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:23:41.117589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:23:43.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f27c80a1430 msgr2=0x7f27c80a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f27c80a1430 0x7f27c80a38e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27dc10b590 tx=0x7f27cc005fd0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc100b60 msgr2=0x7f27dc19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:43.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc100b60 0x7f27dc19b9a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f27d800d980 tx=0x7f27d800dc90 comp rx=0 tx=0).stop 2023-07-14T11:23:43.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 shutdown_connections 2023-07-14T11:23:43.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f27c80a1430 0x7f27c80a38e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27dc100b60 0x7f27dc19b9a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f27dc10a080 0x7f27dc195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 --2- 172.21.15.161:0/1294086476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27dc101470 0x7f27dc19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:43.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.975+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 >> 172.21.15.161:0/1294086476 conn(0x7f27dc0fa490 msgr2=0x7f27dc10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:43.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.976+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 shutdown_connections 2023-07-14T11:23:43.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:43.976+0000 7f27e39ff700 1 -- 172.21.15.161:0/1294086476 wait complete. 2023-07-14T11:23:43.980 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2023-07-14T11:23:43.994 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:43 smithi161 ceph-mon[95526]: pgmap v47: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2023-07-14T11:23:43.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:23:44.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 64258 14.04 MB/sec warmup 26 sec latency 1890.274 ms 2023-07-14T11:23:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:43 smithi175 ceph-mon[129081]: pgmap v47: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2023-07-14T11:23:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 -- 172.21.15.161:0/3912475909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 msgr2=0x7f9c0c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 --2- 172.21.15.161:0/3912475909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c0fff50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009990 tx=0x7f9c00009ca0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 -- 172.21.15.161:0/3912475909 shutdown_connections 2023-07-14T11:23:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 --2- 172.21.15.161:0/3912475909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c0c100490 0x7f9c0c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 --2- 172.21.15.161:0/3912475909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9c0c105af0 0x7f9c0c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 --2- 172.21.15.161:0/3912475909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c0fff50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 -- 172.21.15.161:0/3912475909 >> 172.21.15.161:0/3912475909 conn(0x7f9c0c0fb5e0 msgr2=0x7f9c0c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 -- 172.21.15.161:0/3912475909 shutdown_connections 2023-07-14T11:23:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.196+0000 7f9c11dff700 1 -- 172.21.15.161:0/3912475909 wait complete. 2023-07-14T11:23:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.197+0000 7f9c11dff700 1 Processor -- start 2023-07-14T11:23:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.197+0000 7f9c11dff700 1 -- start start 2023-07-14T11:23:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.197+0000 7f9c11dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.197+0000 7f9c11dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9c0c100490 0x7f9c0c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c11dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c0c105af0 0x7f9c0c195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c11dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c196070 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c11dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c1961e0 con 0x7f9c0c105af0 2023-07-14T11:23:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c11dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c1963a0 con 0x7f9c0c100490 2023-07-14T11:23:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54832/0 (socket says 172.21.15.161:54832) 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1097392590 learned_addr learned my addr 172.21.15.161:0/1097392590 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c0c105af0 0x7f9c0c195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0affd700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9c0c100490 0x7f9c0c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1097392590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9c0c100490 msgr2=0x7f9c0c19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9c0c100490 0x7f9c0c19bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1097392590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c0c105af0 msgr2=0x7f9c0c195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c0c105af0 0x7f9c0c195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.198+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1097392590 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000095f0 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.199+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c19b990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00e9a0 tx=0x7f9bfc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.199+0000 7f9c08ff9700 1 -- 172.21.15.161:0/1097392590 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc009790 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.199+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0c196680 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.199+0000 7f9c08ff9700 1 -- 172.21.15.161:0/1097392590 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bfc010430 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.199+0000 7f9c08ff9700 1 -- 172.21.15.161:0/1097392590 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc018680 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.199+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c1a2980 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.202+0000 7f9c08ff9700 1 -- 172.21.15.161:0/1097392590 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f9bfc0105a0 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.203+0000 7f9c25633b80 1 Processor -- start 2023-07-14T11:23:44.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.203+0000 7f9c25633b80 1 -- start start 2023-07-14T11:23:44.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c08ff9700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9bf40a13e0 0x7f9bf40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4881622d0 0x55a4881626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a48816d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a48816feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c25633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a488068cb0 con 0x55a488162be0 2023-07-14T11:23:44.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c25633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4880bba40 con 0x55a4881622d0 2023-07-14T11:23:44.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c25633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4880b4ed0 con 0x55a48816dad0 2023-07-14T11:23:44.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c08ff9700 1 -- 172.21.15.161:0/1097392590 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f9bfc014070 con 0x7f9c0c0ffae0 2023-07-14T11:23:44.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4881622d0 0x55a4881626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9bf40a13e0 0x7f9bf40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4881622d0 0x55a4881626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57350/0 (socket says 172.21.15.161:57350) 2023-07-14T11:23:44.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/2234068594 learned_addr learned my addr 172.21.15.161:0/2234068594 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0affd700 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a48816d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/2234068594 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 msgr2=0x55a48816feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a48816feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a48816feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/2234068594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 msgr2=0x55a48816d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a48816d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/2234068594 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a487ee2de0 con 0x55a4881622d0 2023-07-14T11:23:44.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.204+0000 7f9c0affd700 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a48816d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:44.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.205+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4881622d0 0x55a4881626a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc096480 tx=0x7f9bfc099e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff225b7254bdde70 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.205+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9bf40a13e0 0x7f9bf40a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009ec0 tx=0x7f9c04009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.205+0000 7f9bf1ffb700 1 -- 172.21.15.161:0/2234068594 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0a3290 con 0x55a4881622d0 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.205+0000 7f9bf1ffb700 1 -- 172.21.15.161:0/2234068594 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bfc099f80 con 0x55a4881622d0 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.205+0000 7f9bf1ffb700 1 -- 172.21.15.161:0/2234068594 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc09b610 con 0x55a4881622d0 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 -- 172.21.15.161:0/2234068594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4881622d0 msgr2=0x55a4881626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4881622d0 0x55a4881626a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc096480 tx=0x7f9bfc099e80 comp rx=0 tx=0).stop 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 -- 172.21.15.161:0/2234068594 shutdown_connections 2023-07-14T11:23:44.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a48816feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4881622d0 0x55a4881626a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 --2- 172.21.15.161:0/2234068594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a48816d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 -- 172.21.15.161:0/2234068594 >> 172.21.15.161:0/2234068594 conn(0x55a488158da0 msgr2=0x55a48815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 -- 172.21.15.161:0/2234068594 shutdown_connections 2023-07-14T11:23:44.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 -- 172.21.15.161:0/2234068594 wait complete. 2023-07-14T11:23:44.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 -- start start 2023-07-14T11:23:44.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.206+0000 7f9c25633b80 1 Processor -- start 2023-07-14T11:23:44.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a48816d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a488162be0 0x55a48816a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a48816af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c25633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a488054ac0 con 0x55a4881622d0 2023-07-14T11:23:44.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c25633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4880547c0 con 0x55a48816dad0 2023-07-14T11:23:44.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c25633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a487f38920 con 0x55a488162be0 2023-07-14T11:23:44.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c25633b80 10 client.0 _ll_register_callbacks cb 0x55a488052760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:44.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a48816d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a48816d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54846/0 (socket says 172.21.15.161:54846) 2023-07-14T11:23:44.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/995393753 learned_addr learned my addr 172.21.15.161:0/995393753 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a48816af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0affd700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a488162be0 0x55a48816a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/995393753 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a488162be0 msgr2=0x55a48816a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a488162be0 0x55a48816a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 msgr2=0x55a48816af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.207+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a48816af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/995393753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c1a2980 con 0x55a4881622d0 2023-07-14T11:23:44.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a48816af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:44.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a48816d520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0a1ea0 tx=0x7f9bfc00c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1168d373ac818d65 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9beaffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:44.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9beaffd700 1 -- 172.21.15.161:0/995393753 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc09ad60 con 0x55a4881622d0 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9beaffd700 1 -- 172.21.15.161:0/995393753 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f9bfc09dfb0 con 0x55a4881622d0 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a48800e4f0 con 0x55a4881622d0 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9beaffd700 1 -- 172.21.15.161:0/995393753 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bfc09ee10 con 0x55a4881622d0 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9c25633b80 10 client.15873 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a487ecbc60 con 0x55a4881622d0 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9beaffd700 1 -- 172.21.15.161:0/995393753 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bfc0c8020 con 0x55a4881622d0 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.208+0000 7f9beaffd700 1 -- 172.21.15.161:0/995393753 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9bfc0b8080 con 0x55a4881622d0 2023-07-14T11:23:44.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c25633b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:44.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c25633b80 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:23:44.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c25633b80 20 client.15873 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:44.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c25633b80 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a48817cef0 0x55a48817f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c25633b80 4 client.15873 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:44.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4880adeb0 con 0x55a48817cef0 2023-07-14T11:23:44.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c0affd700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a48817cef0 0x55a48817f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9beaffd700 10 client.15873 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.209+0000 7f9c0affd700 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a48817cef0 0x55a48817f2d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf481fa1d30c6ff0 server_cookie=addaf6e0cda33f90 in_seq=0 out_seq=0 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.210+0000 7f9beaffd700 1 -- 172.21.15.161:0/995393753 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a4880adeb0 con 0x55a48817cef0 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.210+0000 7f9beaffd700 10 client.15873 handle_command_reply: tid=0 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.210+0000 7f9c25633b80 1 client.15873 shutdown 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.210+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 msgr2=0x55a48816d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.210+0000 7f9c25633b80 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a48816d520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0a1ea0 tx=0x7f9bfc00c400 comp rx=0 tx=0).stop 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 shutdown_connections 2023-07-14T11:23:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a48817cef0 0x55a48817f2d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a48816d520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a48816af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9beaffd700 0 client.15873 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 --2- 172.21.15.161:0/995393753 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a488162be0 0x55a48816a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 >> 172.21.15.161:0/995393753 conn(0x55a488158da0 msgr2=0x55a48815e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 shutdown_connections 2023-07-14T11:23:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 1 -- 172.21.15.161:0/995393753 wait complete. 2023-07-14T11:23:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.211+0000 7f9c25633b80 20 client.15873 trim_cache size 0 max 16384 2023-07-14T11:23:44.222 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:43 smithi182 ceph-mon[129366]: pgmap v47: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2023-07-14T11:23:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 Processor -- start 2023-07-14T11:23:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 -- start start 2023-07-14T11:23:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a4881622d0 0x55a48816af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a4881744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a488179760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4881749f0 con 0x55a488162be0 2023-07-14T11:23:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a488179ca0 con 0x55a48816dad0 2023-07-14T11:23:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c25633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48815d1d0 con 0x55a4881622d0 2023-07-14T11:23:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a4881744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a4881744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54848/0 (socket says 172.21.15.161:54848) 2023-07-14T11:23:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c0affd700 1 -- 172.21.15.161:0/3381623658 learned_addr learned my addr 172.21.15.161:0/3381623658 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a4881622d0 0x55a48816af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.225+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a488179760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9c0affd700 1 -- 172.21.15.161:0/3381623658 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a4881622d0 msgr2=0x55a48816af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9c0affd700 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a4881622d0 0x55a48816af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9c0affd700 1 -- 172.21.15.161:0/3381623658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 msgr2=0x55a488179760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9c0affd700 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a488179760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9c0affd700 1 -- 172.21.15.161:0/3381623658 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a487f09250 con 0x55a488162be0 2023-07-14T11:23:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a488179760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9c0affd700 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a4881744b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009990 tx=0x7f9c00009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2eaca023d412579 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9bf27fc700 1 -- 172.21.15.161:0/3381623658 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0001d070 con 0x55a488162be0 2023-07-14T11:23:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9bf27fc700 1 -- 172.21.15.161:0/3381623658 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c0001e040 con 0x55a488162be0 2023-07-14T11:23:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.226+0000 7f9bf27fc700 1 -- 172.21.15.161:0/3381623658 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000054a0 con 0x55a488162be0 2023-07-14T11:23:44.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 -- 172.21.15.161:0/3381623658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 msgr2=0x55a4881744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a4881744b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009990 tx=0x7f9c00009310 comp rx=0 tx=0).stop 2023-07-14T11:23:44.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 -- 172.21.15.161:0/3381623658 shutdown_connections 2023-07-14T11:23:44.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a48816dad0 0x55a488179760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a4881622d0 0x55a48816af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 --2- 172.21.15.161:0/3381623658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a488162be0 0x55a4881744b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 -- 172.21.15.161:0/3381623658 >> 172.21.15.161:0/3381623658 conn(0x55a488158da0 msgr2=0x55a488183bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 -- 172.21.15.161:0/3381623658 shutdown_connections 2023-07-14T11:23:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 -- 172.21.15.161:0/3381623658 wait complete. 2023-07-14T11:23:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 -- start start 2023-07-14T11:23:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.227+0000 7f9c25633b80 1 Processor -- start 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a488174200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a488162be0 0x55a488172530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c25633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a488172a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c25633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a488173090 con 0x55a4881622d0 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c25633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4881731d0 con 0x55a488162be0 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c25633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a488173390 con 0x55a48816dad0 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c25633b80 10 client.0 _ll_register_callbacks cb 0x7f9bfc099ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a488174200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a488174200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54862/0 (socket says 172.21.15.161:54862) 2023-07-14T11:23:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1002375464 learned_addr learned my addr 172.21.15.161:0/1002375464 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0affd700 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a488162be0 0x55a488172530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0bfff700 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a488172a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1002375464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 msgr2=0x55a488172a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a488172a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.228+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a488162be0 msgr2=0x55a488172530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a488162be0 0x55a488172530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9c0b7fe700 1 -- 172.21.15.161:0/1002375464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c10d220 con 0x55a4881622d0 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9c0b7fe700 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a488174200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c2d0 tx=0x7f9bfc099b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467dd78ac5ad7c70 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9beaffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9beaffd700 1 -- 172.21.15.161:0/1002375464 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc09dfb0 con 0x55a4881622d0 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9beaffd700 1 -- 172.21.15.161:0/1002375464 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f9bfc0c9080 con 0x55a4881622d0 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bfc099f80 con 0x55a4881622d0 2023-07-14T11:23:44.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9beaffd700 1 -- 172.21.15.161:0/1002375464 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bfc09ad60 con 0x55a4881622d0 2023-07-14T11:23:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9c25633b80 10 client.15879 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a487ee12b0 con 0x55a4881622d0 2023-07-14T11:23:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.229+0000 7f9beaffd700 1 -- 172.21.15.161:0/1002375464 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bfc099f80 con 0x55a4881622d0 2023-07-14T11:23:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9beaffd700 1 -- 172.21.15.161:0/1002375464 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9bfc0b8080 con 0x55a4881622d0 2023-07-14T11:23:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c25633b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c25633b80 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:23:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c25633b80 20 client.15879 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c25633b80 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a4881d1e10 0x55a4881d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c25633b80 4 client.15879 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a488070050 con 0x55a4881d1e10 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c0affd700 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a4881d1e10 0x55a4881d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9beaffd700 10 client.15879 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.230+0000 7f9c0affd700 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a4881d1e10 0x55a4881d41f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f947ff9de6ab0c8 server_cookie=5d8dd718a0135eca in_seq=0 out_seq=0 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9beaffd700 1 -- 172.21.15.161:0/1002375464 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a488070050 con 0x55a4881d1e10 2023-07-14T11:23:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9beaffd700 10 client.15879 handle_command_reply: tid=0 2023-07-14T11:23:44.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 client.15879 shutdown 2023-07-14T11:23:44.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 msgr2=0x55a488174200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a488174200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c2d0 tx=0x7f9bfc099b70 comp rx=0 tx=0).stop 2023-07-14T11:23:44.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 shutdown_connections 2023-07-14T11:23:44.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a4881d1e10 0x55a4881d41f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4881622d0 0x55a488174200 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a48816dad0 0x55a488172a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 --2- 172.21.15.161:0/1002375464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a488162be0 0x55a488172530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 >> 172.21.15.161:0/1002375464 conn(0x55a488158da0 msgr2=0x55a488165af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9beaffd700 0 client.15879 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:23:44.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 shutdown_connections 2023-07-14T11:23:44.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 1 -- 172.21.15.161:0/1002375464 wait complete. 2023-07-14T11:23:44.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.231+0000 7f9c25633b80 20 client.15879 trim_cache size 0 max 16384 2023-07-14T11:23:44.242 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:23:44.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9bf40a13e0 msgr2=0x7f9bf40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9bf40a13e0 0x7f9bf40a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009ec0 tx=0x7f9c04009920 comp rx=0 tx=0).stop 2023-07-14T11:23:44.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 msgr2=0x7f9c0c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c19b990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00e9a0 tx=0x7f9bfc00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 shutdown_connections 2023-07-14T11:23:44.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9bf40a13e0 0x7f9bf40a3890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c0ffae0 0x7f9c0c19b990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c0c105af0 0x7f9c0c195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 --2- 172.21.15.161:0/1097392590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9c0c100490 0x7f9c0c19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 >> 172.21.15.161:0/1097392590 conn(0x7f9c0c0fb5e0 msgr2=0x7f9c0c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 shutdown_connections 2023-07-14T11:23:44.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.232+0000 7f9c11dff700 1 -- 172.21.15.161:0/1097392590 wait complete. 2023-07-14T11:23:44.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:23:44.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 -- 172.21.15.161:0/2307252060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef28105ae0 msgr2=0x7fef28105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 --2- 172.21.15.161:0/2307252060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef28105ae0 0x7fef28105eb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fef18009990 tx=0x7fef18009ca0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 -- 172.21.15.161:0/2307252060 shutdown_connections 2023-07-14T11:23:44.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 --2- 172.21.15.161:0/2307252060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef28105ae0 0x7fef28105eb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 --2- 172.21.15.161:0/2307252060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28100480 0x7fef2810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 --2- 172.21.15.161:0/2307252060 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fef280ffad0 0x7fef280fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 -- 172.21.15.161:0/2307252060 >> 172.21.15.161:0/2307252060 conn(0x7fef280fb5f0 msgr2=0x7fef280fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 -- 172.21.15.161:0/2307252060 shutdown_connections 2023-07-14T11:23:44.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.451+0000 7fef2cfff700 1 -- 172.21.15.161:0/2307252060 wait complete. 2023-07-14T11:23:44.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.453+0000 7fef2cfff700 1 Processor -- start 2023-07-14T11:23:44.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.453+0000 7fef2cfff700 1 -- start start 2023-07-14T11:23:44.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2cfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef280ffad0 0x7fef2819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2cfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fef28100480 0x7fef2819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2cfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28105ae0 0x7fef28195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2cfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef281960a0 con 0x7fef280ffad0 2023-07-14T11:23:44.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2cfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef28196210 con 0x7fef28105ae0 2023-07-14T11:23:44.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2cfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef281963d0 con 0x7fef28100480 2023-07-14T11:23:44.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef280ffad0 0x7fef2819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef280ffad0 0x7fef2819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54878/0 (socket says 172.21.15.161:54878) 2023-07-14T11:23:44.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 -- 172.21.15.161:0/4063504813 learned_addr learned my addr 172.21.15.161:0/4063504813 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef25d9b700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fef28100480 0x7fef2819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef26d9d700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28105ae0 0x7fef28195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 -- 172.21.15.161:0/4063504813 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fef28100480 msgr2=0x7fef2819bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fef28100480 0x7fef2819bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 -- 172.21.15.161:0/4063504813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28105ae0 msgr2=0x7fef28195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28105ae0 0x7fef28195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef2659c700 1 -- 172.21.15.161:0/4063504813 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180095f0 con 0x7fef280ffad0 2023-07-14T11:23:44.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.454+0000 7fef26d9d700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28105ae0 0x7fef28195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:23:44.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.455+0000 7fef2659c700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef280ffad0 0x7fef2819b9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fef18005910 tx=0x7fef18005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.455+0000 7fef177fe700 1 -- 172.21.15.161:0/4063504813 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1801d070 con 0x7fef280ffad0 2023-07-14T11:23:44.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.455+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef28196650 con 0x7fef280ffad0 2023-07-14T11:23:44.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.455+0000 7fef177fe700 1 -- 172.21.15.161:0/4063504813 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef18022470 con 0x7fef280ffad0 2023-07-14T11:23:44.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.455+0000 7fef177fe700 1 -- 172.21.15.161:0/4063504813 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18017650 con 0x7fef280ffad0 2023-07-14T11:23:44.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.455+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef281a29b0 con 0x7fef280ffad0 2023-07-14T11:23:44.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.459+0000 7fef177fe700 1 -- 172.21.15.161:0/4063504813 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7fef180225e0 con 0x7fef280ffad0 2023-07-14T11:23:44.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.460+0000 7fef40933b80 1 Processor -- start 2023-07-14T11:23:44.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.460+0000 7fef40933b80 1 -- start start 2023-07-14T11:23:44.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef177fe700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fef100a13e0 0x7fef100a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef40933b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee5056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef40933b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee5149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef40933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee5172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef40933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee40bcb0 con 0x5618ee514ee0 2023-07-14T11:23:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef40933b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee45ea40 con 0x5618ee5052d0 2023-07-14T11:23:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef40933b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee457ed0 con 0x5618ee505be0 2023-07-14T11:23:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef177fe700 1 -- 172.21.15.161:0/4063504813 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fef180ce6c0 con 0x7fef280ffad0 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee5149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef2659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee5056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef26d9d700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fef100a13e0 0x7fef100a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee5149a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49032/0 (socket says 172.21.15.161:49032) 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef25d9b700 1 -- 172.21.15.161:0/2548901128 learned_addr learned my addr 172.21.15.161:0/2548901128 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.461+0000 7fef26d9d700 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee5172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef2659c700 1 -- 172.21.15.161:0/2548901128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 msgr2=0x5618ee5149a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef2659c700 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee5149a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef2659c700 1 -- 172.21.15.161:0/2548901128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 msgr2=0x5618ee5172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef2659c700 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee5172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef2659c700 1 -- 172.21.15.161:0/2548901128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618ee285de0 con 0x5618ee5052d0 2023-07-14T11:23:44.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef25d9b700 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee5149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:44.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef26d9d700 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee5172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:44.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef2659c700 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee5056a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef180cb960 tx=0x7fef18009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c020b31ac51daab9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef26d9d700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fef100a13e0 0x7fef100a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef20009fd0 tx=0x7fef2000d040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef14ff9700 1 -- 172.21.15.161:0/2548901128 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1809ade0 con 0x5618ee5052d0 2023-07-14T11:23:44.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef14ff9700 1 -- 172.21.15.161:0/2548901128 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef1809ac00 con 0x5618ee5052d0 2023-07-14T11:23:44.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.462+0000 7fef14ff9700 1 -- 172.21.15.161:0/2548901128 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1809f2d0 con 0x5618ee5052d0 2023-07-14T11:23:44.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 -- 172.21.15.161:0/2548901128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 msgr2=0x5618ee5056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee5056a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef180cb960 tx=0x7fef18009310 comp rx=0 tx=0).stop 2023-07-14T11:23:44.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 -- 172.21.15.161:0/2548901128 shutdown_connections 2023-07-14T11:23:44.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee5172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee5056a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 --2- 172.21.15.161:0/2548901128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee5149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 -- 172.21.15.161:0/2548901128 >> 172.21.15.161:0/2548901128 conn(0x5618ee4fbda0 msgr2=0x5618ee5016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 -- 172.21.15.161:0/2548901128 shutdown_connections 2023-07-14T11:23:44.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 -- 172.21.15.161:0/2548901128 wait complete. 2023-07-14T11:23:44.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.463+0000 7fef40933b80 1 -- start start 2023-07-14T11:23:44.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 1 Processor -- start 2023-07-14T11:23:44.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee519fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee51a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee513e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee3f7ac0 con 0x5618ee505be0 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee3f77c0 con 0x5618ee5052d0 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee2db920 con 0x5618ee514ee0 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef40933b80 10 client.0 _ll_register_callbacks cb 0x5618ee3f5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee51a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee51a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54894/0 (socket says 172.21.15.161:54894) 2023-07-14T11:23:44.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef25d9b700 1 -- 172.21.15.161:0/1667491804 learned_addr learned my addr 172.21.15.161:0/1667491804 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.464+0000 7fef2659c700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee519fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef26d9d700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee513e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef25d9b700 1 -- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 msgr2=0x5618ee513e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef25d9b700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee513e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef25d9b700 1 -- 172.21.15.161:0/1667491804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 msgr2=0x5618ee519fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef25d9b700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee519fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef25d9b700 1 -- 172.21.15.161:0/1667491804 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c009980 con 0x5618ee505be0 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef2659c700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee519fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef25d9b700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee51a510 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fef1c012980 tx=0x7fef1c012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8bb35e9b2eec41 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef0dffb700 1 -- 172.21.15.161:0/1667491804 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c017070 con 0x5618ee505be0 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef0dffb700 1 -- 172.21.15.161:0/1667491804 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fef1c004020 con 0x5618ee505be0 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.465+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618ee3b14f0 con 0x5618ee505be0 2023-07-14T11:23:44.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef0dffb700 1 -- 172.21.15.161:0/1667491804 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef1c0041e0 con 0x5618ee505be0 2023-07-14T11:23:44.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 10 client.15891 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:44.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fef281a29b0 con 0x5618ee505be0 2023-07-14T11:23:44.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef0dffb700 1 -- 172.21.15.161:0/1667491804 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618ee3b14f0 con 0x5618ee505be0 2023-07-14T11:23:44.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef0dffb700 1 -- 172.21.15.161:0/1667491804 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fef1c00f5e0 con 0x5618ee505be0 2023-07-14T11:23:44.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:44.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:23:44.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 20 client.15891 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee524490 0x5618ee5268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 4 client.15891 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618ee450eb0 con 0x5618ee524490 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.466+0000 7fef2659c700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee524490 0x5618ee5268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.467+0000 7fef2659c700 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee524490 0x5618ee5268a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1959169a6a51f3d0 server_cookie=83207c3206f3ce80 in_seq=0 out_seq=0 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.467+0000 7fef0dffb700 10 client.15891 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef0dffb700 1 -- 172.21.15.161:0/1667491804 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5618ee450eb0 con 0x5618ee524490 2023-07-14T11:23:44.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef0dffb700 10 client.15891 handle_command_reply: tid=0 2023-07-14T11:23:44.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 client.15891 shutdown 2023-07-14T11:23:44.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 msgr2=0x5618ee51a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee51a510 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fef1c012980 tx=0x7fef1c012c90 comp rx=0 tx=0).stop 2023-07-14T11:23:44.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 shutdown_connections 2023-07-14T11:23:44.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee524490 0x5618ee5268a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee513e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef0dffb700 0 client.15891 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:44.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee519fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 --2- 172.21.15.161:0/1667491804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee51a510 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 >> 172.21.15.161:0/1667491804 conn(0x5618ee4fbda0 msgr2=0x5618ee501290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 shutdown_connections 2023-07-14T11:23:44.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 1 -- 172.21.15.161:0/1667491804 wait complete. 2023-07-14T11:23:44.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.468+0000 7fef40933b80 20 client.15891 trim_cache size 0 max 16384 2023-07-14T11:23:44.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.481+0000 7fef40933b80 1 Processor -- start 2023-07-14T11:23:44.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef40933b80 1 -- start start 2023-07-14T11:23:44.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef40933b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee512d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef40933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee519b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef40933b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee51a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef40933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee3b14f0 con 0x5618ee505be0 2023-07-14T11:23:44.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef40933b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee506540 con 0x5618ee5052d0 2023-07-14T11:23:44.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef40933b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee506300 con 0x5618ee514ee0 2023-07-14T11:23:44.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee519b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee519b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54910/0 (socket says 172.21.15.161:54910) 2023-07-14T11:23:44.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef25d9b700 1 -- 172.21.15.161:0/3188835768 learned_addr learned my addr 172.21.15.161:0/3188835768 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef2659c700 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee512d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef26d9d700 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee51a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef25d9b700 1 -- 172.21.15.161:0/3188835768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 msgr2=0x5618ee51a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef25d9b700 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee51a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef25d9b700 1 -- 172.21.15.161:0/3188835768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 msgr2=0x5618ee512d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.482+0000 7fef25d9b700 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee512d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.483+0000 7fef25d9b700 1 -- 172.21.15.161:0/3188835768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618ee5140e0 con 0x5618ee505be0 2023-07-14T11:23:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.483+0000 7fef25d9b700 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee519b60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000c00 tx=0x7fef1c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d4d858103e47b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.483+0000 7fef157fa700 1 -- 172.21.15.161:0/3188835768 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c017070 con 0x5618ee505be0 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.483+0000 7fef157fa700 1 -- 172.21.15.161:0/3188835768 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef1c018040 con 0x5618ee505be0 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.483+0000 7fef157fa700 1 -- 172.21.15.161:0/3188835768 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0038e0 con 0x5618ee505be0 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 -- 172.21.15.161:0/3188835768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 msgr2=0x5618ee519b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee519b60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000c00 tx=0x7fef1c009f60 comp rx=0 tx=0).stop 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 -- 172.21.15.161:0/3188835768 shutdown_connections 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee514ee0 0x5618ee51a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee512d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 --2- 172.21.15.161:0/3188835768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee505be0 0x5618ee519b60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 -- 172.21.15.161:0/3188835768 >> 172.21.15.161:0/3188835768 conn(0x5618ee4fbda0 msgr2=0x5618ee512600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 -- 172.21.15.161:0/3188835768 shutdown_connections 2023-07-14T11:23:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 -- 172.21.15.161:0/3188835768 wait complete. 2023-07-14T11:23:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 -- start start 2023-07-14T11:23:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.484+0000 7fef40933b80 1 Processor -- start 2023-07-14T11:23:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef40933b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee571680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef40933b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee571bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef40933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee56f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef40933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee56f880 con 0x5618ee514ee0 2023-07-14T11:23:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef40933b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee56f9f0 con 0x5618ee5052d0 2023-07-14T11:23:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef40933b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ee56fbb0 con 0x5618ee505be0 2023-07-14T11:23:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef40933b80 10 client.0 _ll_register_callbacks cb 0x7fef1c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee571bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef2659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee571680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee571bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49060/0 (socket says 172.21.15.161:49060) 2023-07-14T11:23:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef25d9b700 1 -- 172.21.15.161:0/704829303 learned_addr learned my addr 172.21.15.161:0/704829303 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef26d9d700 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee56f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef25d9b700 1 -- 172.21.15.161:0/704829303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 msgr2=0x5618ee571680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef25d9b700 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee571680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef25d9b700 1 -- 172.21.15.161:0/704829303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 msgr2=0x5618ee56f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.485+0000 7fef25d9b700 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee56f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef25d9b700 1 -- 172.21.15.161:0/704829303 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef281a29b0 con 0x5618ee505be0 2023-07-14T11:23:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef25d9b700 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee571bc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00a010 tx=0x7fef1c00f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6c2e9e7326466 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef0dffb700 1 -- 172.21.15.161:0/704829303 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c017030 con 0x5618ee505be0 2023-07-14T11:23:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef1809ac00 con 0x5618ee505be0 2023-07-14T11:23:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef0dffb700 1 -- 172.21.15.161:0/704829303 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fef1c00fc60 con 0x5618ee505be0 2023-07-14T11:23:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef0dffb700 1 -- 172.21.15.161:0/704829303 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef1c0054c0 con 0x5618ee505be0 2023-07-14T11:23:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef40933b80 10 client.25930 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.486+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5618ee26ec60 con 0x5618ee505be0 2023-07-14T11:23:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef0dffb700 1 -- 172.21.15.161:0/704829303 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef1809ac00 con 0x5618ee505be0 2023-07-14T11:23:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef0dffb700 1 -- 172.21.15.161:0/704829303 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fef1c01c500 con 0x5618ee505be0 2023-07-14T11:23:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef40933b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef40933b80 10 client.25930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef40933b80 20 client.25930 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef40933b80 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee579260 0x5618ee57b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef40933b80 4 client.25930 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5618ee413050 con 0x5618ee579260 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.487+0000 7fef2659c700 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee579260 0x5618ee57b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef2659c700 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee579260 0x5618ee57b640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95d0633971058056 server_cookie=375042048efc1ed6 in_seq=0 out_seq=0 2023-07-14T11:23:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef0dffb700 10 client.25930 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef0dffb700 1 -- 172.21.15.161:0/704829303 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5618ee413050 con 0x5618ee579260 2023-07-14T11:23:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef0dffb700 10 client.25930 handle_command_reply: tid=0 2023-07-14T11:23:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 client.25930 shutdown 2023-07-14T11:23:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 msgr2=0x5618ee571bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee571bc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00a010 tx=0x7fef1c00f890 comp rx=0 tx=0).stop 2023-07-14T11:23:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 shutdown_connections 2023-07-14T11:23:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5618ee579260 0x5618ee57b640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618ee514ee0 0x5618ee56f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618ee5052d0 0x5618ee571680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 --2- 172.21.15.161:0/704829303 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5618ee505be0 0x5618ee571bc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef0dffb700 0 client.25930 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:23:44.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 >> 172.21.15.161:0/704829303 conn(0x5618ee4fbda0 msgr2=0x5618ee512e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 shutdown_connections 2023-07-14T11:23:44.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 1 -- 172.21.15.161:0/704829303 wait complete. 2023-07-14T11:23:44.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.488+0000 7fef40933b80 20 client.25930 trim_cache size 0 max 16384 2023-07-14T11:23:44.500 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:23:44.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fef100a13e0 msgr2=0x7fef100a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fef100a13e0 0x7fef100a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef20009fd0 tx=0x7fef2000d040 comp rx=0 tx=0).stop 2023-07-14T11:23:44.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef280ffad0 msgr2=0x7fef2819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef280ffad0 0x7fef2819b9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fef18005910 tx=0x7fef18005720 comp rx=0 tx=0).stop 2023-07-14T11:23:44.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 shutdown_connections 2023-07-14T11:23:44.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fef100a13e0 0x7fef100a3890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef280ffad0 0x7fef2819b9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef28105ae0 0x7fef28195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 --2- 172.21.15.161:0/4063504813 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fef28100480 0x7fef2819bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 >> 172.21.15.161:0/4063504813 conn(0x7fef280fb5f0 msgr2=0x7fef281035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 shutdown_connections 2023-07-14T11:23:44.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.489+0000 7fef2cfff700 1 -- 172.21.15.161:0/4063504813 wait complete. 2023-07-14T11:23:44.509 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:23:44.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 -- 172.21.15.161:0/2758771869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec0ffad0 msgr2=0x7f16ec0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 --2- 172.21.15.161:0/2758771869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec0ffad0 0x7f16ec0fff40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009990 tx=0x7f16dc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 -- 172.21.15.161:0/2758771869 shutdown_connections 2023-07-14T11:23:44.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 --2- 172.21.15.161:0/2758771869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec105ae0 0x7f16ec105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 --2- 172.21.15.161:0/2758771869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16ec100480 0x7f16ec10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 --2- 172.21.15.161:0/2758771869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec0ffad0 0x7f16ec0fff40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 -- 172.21.15.161:0/2758771869 >> 172.21.15.161:0/2758771869 conn(0x7f16ec0fb5f0 msgr2=0x7f16ec0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 -- 172.21.15.161:0/2758771869 shutdown_connections 2023-07-14T11:23:44.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.708+0000 7f16f47ff700 1 -- 172.21.15.161:0/2758771869 wait complete. 2023-07-14T11:23:44.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.709+0000 7f16f47ff700 1 Processor -- start 2023-07-14T11:23:44.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f47ff700 1 -- start start 2023-07-14T11:23:44.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f47ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec0ffad0 0x7f16ec072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f47ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec100480 0x7f16ec072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f47ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16ec105ae0 0x7f16ec073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f47ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1a4060 con 0x7f16ec0ffad0 2023-07-14T11:23:44.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f47ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1a41d0 con 0x7f16ec100480 2023-07-14T11:23:44.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f47ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1a4390 con 0x7f16ec105ae0 2023-07-14T11:23:44.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f259b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec0ffad0 0x7f16ec072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f259b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec0ffad0 0x7f16ec072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54944/0 (socket says 172.21.15.161:54944) 2023-07-14T11:23:44.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f259b700 1 -- 172.21.15.161:0/4066252612 learned_addr learned my addr 172.21.15.161:0/4066252612 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec100480 0x7f16ec072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.710+0000 7f16f2d9c700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16ec105ae0 0x7f16ec073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f259b700 1 -- 172.21.15.161:0/4066252612 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16ec105ae0 msgr2=0x7f16ec073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f259b700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16ec105ae0 0x7f16ec073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f259b700 1 -- 172.21.15.161:0/4066252612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec100480 msgr2=0x7f16ec072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f259b700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec100480 0x7f16ec072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f259b700 1 -- 172.21.15.161:0/4066252612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0095f0 con 0x7f16ec0ffad0 2023-07-14T11:23:44.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f259b700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec0ffad0 0x7f16ec072810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9a0 tx=0x7f16e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16e37fe700 1 -- 172.21.15.161:0/4066252612 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8009790 con 0x7f16ec0ffad0 2023-07-14T11:23:44.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16e37fe700 1 -- 172.21.15.161:0/4066252612 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16e8010430 con 0x7f16ec0ffad0 2023-07-14T11:23:44.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec1a4670 con 0x7f16ec0ffad0 2023-07-14T11:23:44.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16e37fe700 1 -- 172.21.15.161:0/4066252612 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8018680 con 0x7f16ec0ffad0 2023-07-14T11:23:44.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.711+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1a4b80 con 0x7f16ec0ffad0 2023-07-14T11:23:44.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.714+0000 7f16e37fe700 1 -- 172.21.15.161:0/4066252612 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f16e80105a0 con 0x7f16ec0ffad0 2023-07-14T11:23:44.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f170810ab80 1 Processor -- start 2023-07-14T11:23:44.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f16e37fe700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16d80a13e0 0x7f16d80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f16e37fe700 1 -- 172.21.15.161:0/4066252612 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f16e8014070 con 0x7f16ec0ffad0 2023-07-14T11:23:44.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f170810ab80 1 -- start start 2023-07-14T11:23:44.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f16f2d9c700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16d80a13e0 0x7f16d80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f170810ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f170810ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.716+0000 7f170810ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f623ad0 0x557f4f625eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f170810ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f51ecb0 con 0x557f4f618be0 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f170810ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f571a40 con 0x557f4f6182d0 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f170810ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f56aed0 con 0x557f4f623ad0 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f2d9c700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16d80a13e0 0x7f16d80a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16e400f540 tx=0x7f16e400b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f1d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f259b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f259b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6186a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57430/0 (socket says 172.21.15.161:57430) 2023-07-14T11:23:44.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f259b700 1 -- 172.21.15.161:0/3651707574 learned_addr learned my addr 172.21.15.161:0/3651707574 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54962/0 (socket says 172.21.15.161:54962) 2023-07-14T11:23:44.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f2d9c700 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f623ad0 0x557f4f625eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f1d9a700 1 -- 172.21.15.161:0/3651707574 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f623ad0 msgr2=0x557f4f625eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f623ad0 0x557f4f625eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f1d9a700 1 -- 172.21.15.161:0/3651707574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 msgr2=0x557f4f6186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f1d9a700 1 -- 172.21.15.161:0/3651707574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4f398de0 con 0x557f4f618be0 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.717+0000 7f16f259b700 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.718+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00b2e0 tx=0x7f16dc005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cf8f218e404ce4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.718+0000 7f16e0ff9700 1 -- 172.21.15.161:0/3651707574 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc01d070 con 0x557f4f618be0 2023-07-14T11:23:44.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.718+0000 7f16e0ff9700 1 -- 172.21.15.161:0/3651707574 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc01e040 con 0x557f4f618be0 2023-07-14T11:23:44.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.718+0000 7f16e0ff9700 1 -- 172.21.15.161:0/3651707574 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc004760 con 0x557f4f618be0 2023-07-14T11:23:44.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.718+0000 7f170810ab80 1 -- 172.21.15.161:0/3651707574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 msgr2=0x557f4f623590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.718+0000 7f170810ab80 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00b2e0 tx=0x7f16dc005ea0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 -- 172.21.15.161:0/3651707574 shutdown_connections 2023-07-14T11:23:44.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f623ad0 0x557f4f625eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 --2- 172.21.15.161:0/3651707574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623590 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 -- 172.21.15.161:0/3651707574 >> 172.21.15.161:0/3651707574 conn(0x557f4f60eda0 msgr2=0x557f4f6146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 -- 172.21.15.161:0/3651707574 shutdown_connections 2023-07-14T11:23:44.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 -- 172.21.15.161:0/3651707574 wait complete. 2023-07-14T11:23:44.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 -- start start 2023-07-14T11:23:44.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.719+0000 7f170810ab80 1 Processor -- start 2023-07-14T11:23:44.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f170810ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f623490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f170810ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f620780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f170810ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f620e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f170810ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f50aac0 con 0x557f4f618be0 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f170810ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f50a7c0 con 0x557f4f623ad0 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f170810ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f3ee920 con 0x557f4f6182d0 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f170810ab80 10 client.0 _ll_register_callbacks cb 0x557f4f508760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f16f1d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f620780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f16f259b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f623490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f16f259b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f623490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49090/0 (socket says 172.21.15.161:49090) 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f16f259b700 1 -- 172.21.15.161:0/3704572257 learned_addr learned my addr 172.21.15.161:0/3704572257 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.720+0000 7f16f2d9c700 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f620e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16f259b700 1 -- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 msgr2=0x557f4f620e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16f259b700 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f620e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16f259b700 1 -- 172.21.15.161:0/3704572257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 msgr2=0x557f4f620780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16f259b700 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f620780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16f259b700 1 -- 172.21.15.161:0/3704572257 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1a4b80 con 0x557f4f6182d0 2023-07-14T11:23:44.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16f2d9c700 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f620e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:44.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16f259b700 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f623490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16e80969f0 tx=0x7f16e809a4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e571542050d26ebc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:44.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3704572257 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e809dfb0 con 0x557f4f6182d0 2023-07-14T11:23:44.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3704572257 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f16e80a23d0 con 0x557f4f6182d0 2023-07-14T11:23:44.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f4f4c44f0 con 0x557f4f6182d0 2023-07-14T11:23:44.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3704572257 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16e80a1f40 con 0x557f4f6182d0 2023-07-14T11:23:44.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f170810ab80 10 client.25942 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:44.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f16ec10d210 con 0x557f4f6182d0 2023-07-14T11:23:44.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.721+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3704572257 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f4f4c44f0 con 0x557f4f6182d0 2023-07-14T11:23:44.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3704572257 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f16e80b80d0 con 0x557f4f6182d0 2023-07-14T11:23:44.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f170810ab80 10 client.25942 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:44.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f170810ab80 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:44.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f170810ab80 20 client.25942 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:44.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f170810ab80 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f6333d0 0x557f4f6357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f170810ab80 4 client.25942 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f4f563eb0 con 0x557f4f6333d0 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.722+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f6333d0 0x557f4f6357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.723+0000 7f16d5ffb700 10 client.25942 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.723+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f6333d0 0x557f4f6357e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9e6a7c1449941c server_cookie=7f85a726413066d in_seq=0 out_seq=0 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.723+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3704572257 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f4f563eb0 con 0x557f4f6333d0 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.723+0000 7f16d5ffb700 10 client.25942 handle_command_reply: tid=0 2023-07-14T11:23:44.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 client.25942 shutdown 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 msgr2=0x557f4f623490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f623490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16e80969f0 tx=0x7f16e809a4c0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 shutdown_connections 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f6333d0 0x557f4f6357e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f620e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f623490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f16d5ffb700 0 client.25942 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:44.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 --2- 172.21.15.161:0/3704572257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f620780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 >> 172.21.15.161:0/3704572257 conn(0x557f4f60eda0 msgr2=0x557f4f614230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 shutdown_connections 2023-07-14T11:23:44.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 1 -- 172.21.15.161:0/3704572257 wait complete. 2023-07-14T11:23:44.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.724+0000 7f170810ab80 20 client.25942 trim_cache size 0 max 16384 2023-07-14T11:23:44.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.737+0000 7f170810ab80 1 Processor -- start 2023-07-14T11:23:44.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.737+0000 7f170810ab80 1 -- start start 2023-07-14T11:23:44.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f170810ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f6357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f170810ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f170810ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f61bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f170810ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f619540 con 0x557f4f618be0 2023-07-14T11:23:44.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f170810ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f621cf0 con 0x557f4f623ad0 2023-07-14T11:23:44.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f170810ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f60e970 con 0x557f4f6182d0 2023-07-14T11:23:44.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54966/0 (socket says 172.21.15.161:54966) 2023-07-14T11:23:44.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 -- 172.21.15.161:0/1242212940 learned_addr learned my addr 172.21.15.161:0/1242212940 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f259b700 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f6357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f2d9c700 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f61bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 -- 172.21.15.161:0/1242212940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 msgr2=0x557f4f6357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f6357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 -- 172.21.15.161:0/1242212940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 msgr2=0x557f4f61bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f61bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.738+0000 7f16f1d9a700 1 -- 172.21.15.161:0/1242212940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4f381c60 con 0x557f4f618be0 2023-07-14T11:23:44.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623030 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16dc000c00 tx=0x7f16dc005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d56937668b9bc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f16e17fa700 1 -- 172.21.15.161:0/1242212940 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc01d040 con 0x557f4f618be0 2023-07-14T11:23:44.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f16e17fa700 1 -- 172.21.15.161:0/1242212940 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc005bc0 con 0x557f4f618be0 2023-07-14T11:23:44.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f16e17fa700 1 -- 172.21.15.161:0/1242212940 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc004cb0 con 0x557f4f618be0 2023-07-14T11:23:44.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f170810ab80 1 -- 172.21.15.161:0/1242212940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 msgr2=0x557f4f623030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f170810ab80 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623030 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16dc000c00 tx=0x7f16dc005fb0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f170810ab80 1 -- 172.21.15.161:0/1242212940 shutdown_connections 2023-07-14T11:23:44.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f170810ab80 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f623ad0 0x557f4f61bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f170810ab80 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f6182d0 0x557f4f6357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f170810ab80 1 --2- 172.21.15.161:0/1242212940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f618be0 0x557f4f623030 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.739+0000 7f170810ab80 1 -- 172.21.15.161:0/1242212940 >> 172.21.15.161:0/1242212940 conn(0x557f4f60eda0 msgr2=0x557f4f62ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.740+0000 7f170810ab80 1 -- 172.21.15.161:0/1242212940 shutdown_connections 2023-07-14T11:23:44.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.740+0000 7f170810ab80 1 -- 172.21.15.161:0/1242212940 wait complete. 2023-07-14T11:23:44.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.740+0000 7f170810ab80 1 -- start start 2023-07-14T11:23:44.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.740+0000 7f170810ab80 1 Processor -- start 2023-07-14T11:23:44.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f170810ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f170810ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f618be0 0x557f4f629c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f170810ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f623ad0 0x557f4f686290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f170810ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f62a520 con 0x557f4f623ad0 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f170810ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f6867d0 con 0x557f4f6182d0 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f170810ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f686910 con 0x557f4f618be0 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f170810ab80 10 client.0 _ll_register_callbacks cb 0x7f16e8099ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f2d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f623ad0 0x557f4f686290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6296d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57468/0 (socket says 172.21.15.161:57468) 2023-07-14T11:23:44.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f2d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f623ad0 0x557f4f686290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54982/0 (socket says 172.21.15.161:54982) 2023-07-14T11:23:44.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 -- 172.21.15.161:0/3069741219 learned_addr learned my addr 172.21.15.161:0/3069741219 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:44.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f618be0 0x557f4f629c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 -- 172.21.15.161:0/3069741219 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f618be0 msgr2=0x557f4f629c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f618be0 0x557f4f629c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 -- 172.21.15.161:0/3069741219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f623ad0 msgr2=0x557f4f686290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:44.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f623ad0 0x557f4f686290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.741+0000 7f16f259b700 1 -- 172.21.15.161:0/3069741219 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f4f398de0 con 0x557f4f6182d0 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f16f259b700 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6296d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16e8096480 tx=0x7f16e800c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892cd6c418deb8e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f16d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3069741219 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80992b0 con 0x557f4f6182d0 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3069741219 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f16e809dfb0 con 0x557f4f6182d0 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f4f526050 con 0x557f4f6182d0 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3069741219 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16e80a1f40 con 0x557f4f6182d0 2023-07-14T11:23:44.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 10 client.25955 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:44.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557f4f3bf250 con 0x557f4f6182d0 2023-07-14T11:23:44.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3069741219 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f4f526050 con 0x557f4f6182d0 2023-07-14T11:23:44.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3069741219 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f16e80ca9b0 con 0x557f4f6182d0 2023-07-14T11:23:44.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 10 client.25955 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:44.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:44.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 20 client.25955 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:44.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f687cd0 0x557f4f68a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:44.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 4 client.25955 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:23:44.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.742+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f4f3b9f70 con 0x557f4f687cd0 2023-07-14T11:23:44.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.743+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f687cd0 0x557f4f68a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:44.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.743+0000 7f16f1d9a700 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f687cd0 0x557f4f68a0e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36dc76fdac9f1aff server_cookie=6c84a322bdd38707 in_seq=0 out_seq=0 2023-07-14T11:23:44.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.743+0000 7f16d5ffb700 10 client.25955 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:44.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f16d5ffb700 1 -- 172.21.15.161:0/3069741219 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f4f3b9f70 con 0x557f4f687cd0 2023-07-14T11:23:44.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f16d5ffb700 10 client.25955 handle_command_reply: tid=0 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 client.25955 shutdown 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 msgr2=0x557f4f6296d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6296d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16e8096480 tx=0x7f16e800c400 comp rx=0 tx=0).stop 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 shutdown_connections 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557f4f687cd0 0x557f4f68a0e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557f4f623ad0 0x557f4f686290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f4f6182d0 0x557f4f6296d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f16d5ffb700 0 client.25955 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 --2- 172.21.15.161:0/3069741219 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557f4f618be0 0x557f4f629c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 >> 172.21.15.161:0/3069741219 conn(0x557f4f60eda0 msgr2=0x557f4f61bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 shutdown_connections 2023-07-14T11:23:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 1 -- 172.21.15.161:0/3069741219 wait complete. 2023-07-14T11:23:44.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.744+0000 7f170810ab80 20 client.25955 trim_cache size 0 max 16384 2023-07-14T11:23:44.755 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:23:44.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16d80a13e0 msgr2=0x7f16d80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16d80a13e0 0x7f16d80a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16e400f540 tx=0x7f16e400b040 comp rx=0 tx=0).stop 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec0ffad0 msgr2=0x7f16ec072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec0ffad0 0x7f16ec072810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9a0 tx=0x7f16e800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 shutdown_connections 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16d80a13e0 0x7f16d80a3890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec0ffad0 0x7f16ec072810 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16ec105ae0 0x7f16ec073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 --2- 172.21.15.161:0/4066252612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16ec100480 0x7f16ec072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:44.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 >> 172.21.15.161:0/4066252612 conn(0x7f16ec0fb5f0 msgr2=0x7f16ec1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:44.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 shutdown_connections 2023-07-14T11:23:44.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:44.745+0000 7f16f47ff700 1 -- 172.21.15.161:0/4066252612 wait complete. 2023-07-14T11:23:44.763 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:44 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1294086476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:23:45.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 64258 13.52 MB/sec warmup 27 sec latency 2890.353 ms 2023-07-14T11:23:45.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:44 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1294086476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:23:45.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:44 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1294086476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:23:45.766 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:23:45.767 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:23:45.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.981+0000 7fd87b59e700 1 -- 172.21.15.161:0/3646109596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 msgr2=0x7fd87c100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:45.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.981+0000 7fd87b59e700 1 --2- 172.21.15.161:0/3646109596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c100fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0098f0 tx=0x7fd86c009c00 comp rx=0 tx=0).stop 2023-07-14T11:23:45.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.981+0000 7fd87b59e700 1 -- 172.21.15.161:0/3646109596 shutdown_connections 2023-07-14T11:23:45.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.981+0000 7fd87b59e700 1 --2- 172.21.15.161:0/3646109596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd87c109f50 0x7fd87c10c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:45.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.981+0000 7fd87b59e700 1 --2- 172.21.15.161:0/3646109596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c100fe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:45.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.981+0000 7fd87b59e700 1 --2- 172.21.15.161:0/3646109596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd87c101520 0x7fd87c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.981+0000 7fd87b59e700 1 -- 172.21.15.161:0/3646109596 >> 172.21.15.161:0/3646109596 conn(0x7fd87c0fa3c0 msgr2=0x7fd87c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.982+0000 7fd87b59e700 1 -- 172.21.15.161:0/3646109596 shutdown_connections 2023-07-14T11:23:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.982+0000 7fd87b59e700 1 -- 172.21.15.161:0/3646109596 wait complete. 2023-07-14T11:23:45.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 Processor -- start 2023-07-14T11:23:45.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 -- start start 2023-07-14T11:23:45.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:45.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd87c101520 0x7fd87c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:45.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd87c109f50 0x7fd87c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:45.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c198150 con 0x7fd87c109f50 2023-07-14T11:23:45.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c1982c0 con 0x7fd87c100c10 2023-07-14T11:23:45.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87b59e700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c198480 con 0x7fd87c101520 2023-07-14T11:23:45.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:45.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57494/0 (socket says 172.21.15.161:57494) 2023-07-14T11:23:45.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd879d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd87c101520 0x7fd87c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:45.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87a59c700 1 -- 172.21.15.161:0/1668952293 learned_addr learned my addr 172.21.15.161:0/1668952293 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:45.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.983+0000 7fd87ad9d700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd87c109f50 0x7fd87c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:45.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87a59c700 1 -- 172.21.15.161:0/1668952293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd87c101520 msgr2=0x7fd87c19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:45.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87a59c700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd87c101520 0x7fd87c19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:45.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87a59c700 1 -- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd87c109f50 msgr2=0x7fd87c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:45.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87a59c700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd87c109f50 0x7fd87c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:45.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87a59c700 1 -- 172.21.15.161:0/1668952293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c009520 con 0x7fd87c100c10 2023-07-14T11:23:45.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87a59c700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c19da70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009810 tx=0x7fd86c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:45.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd86b7fe700 1 -- 172.21.15.161:0/1668952293 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c01c070 con 0x7fd87c100c10 2023-07-14T11:23:45.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd86b7fe700 1 -- 172.21.15.161:0/1668952293 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd86c00bbd0 con 0x7fd87c100c10 2023-07-14T11:23:45.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87c198700 con 0x7fd87c100c10 2023-07-14T11:23:45.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd86b7fe700 1 -- 172.21.15.161:0/1668952293 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c004930 con 0x7fd87c100c10 2023-07-14T11:23:45.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.984+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87c1a4a60 con 0x7fd87c100c10 2023-07-14T11:23:45.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.988+0000 7fd86b7fe700 1 -- 172.21.15.161:0/1668952293 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7fd86c020070 con 0x7fd87c100c10 2023-07-14T11:23:45.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 Processor -- start 2023-07-14T11:23:45.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 -- start start 2023-07-14T11:23:45.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ebb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:45.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:45.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:45.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54dc0d30 con 0x55be54ec5db0 2023-07-14T11:23:45.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54e13ac0 con 0x55be54ebb570 2023-07-14T11:23:45.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.989+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54e0cf50 con 0x55be54ebbe80 2023-07-14T11:23:45.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd87ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ebb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:45.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd87ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ebb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57500/0 (socket says 172.21.15.161:57500) 2023-07-14T11:23:45.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd86b7fe700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd8640a1480 0x7fd8640a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:45.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd87ad9d700 1 -- 172.21.15.161:0/1805251043 learned_addr learned my addr 172.21.15.161:0/1805251043 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:45.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd879d9b700 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:45.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd87a59c700 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:45.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd86b7fe700 1 -- 172.21.15.161:0/1668952293 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fd86c0cdf10 con 0x7fd87c100c10 2023-07-14T11:23:45.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd87ad9d700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd8640a1480 0x7fd8640a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:45.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd879d9b700 1 -- 172.21.15.161:0/1805251043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 msgr2=0x55be54ec5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:45.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd879d9b700 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:45.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd879d9b700 1 -- 172.21.15.161:0/1805251043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 msgr2=0x55be54ebb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:45.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd879d9b700 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ebb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:45.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd879d9b700 1 -- 172.21.15.161:0/1805251043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be54c3ae60 con 0x55be54ec5db0 2023-07-14T11:23:45.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.990+0000 7fd879d9b700 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec8190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fd0 tx=0x7fd870011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3aaa9ba443e526 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:45.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.991+0000 7fd87ad9d700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd8640a1480 0x7fd8640a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8740060b0 tx=0x7fd874004b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:45.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.991+0000 7fd868ff9700 1 -- 172.21.15.161:0/1805251043 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870011cc0 con 0x55be54ec5db0 2023-07-14T11:23:45.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.991+0000 7fd868ff9700 1 -- 172.21.15.161:0/1805251043 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd870011e20 con 0x55be54ec5db0 2023-07-14T11:23:45.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.991+0000 7fd868ff9700 1 -- 172.21.15.161:0/1805251043 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870005280 con 0x55be54ec5db0 2023-07-14T11:23:46.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.991+0000 7fd8947e7b80 1 -- 172.21.15.161:0/1805251043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 msgr2=0x55be54ec8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.991+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec8190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fd0 tx=0x7fd870011440 comp rx=0 tx=0).stop 2023-07-14T11:23:46.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 -- 172.21.15.161:0/1805251043 shutdown_connections 2023-07-14T11:23:46.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec8190 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ebb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/1805251043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 -- 172.21.15.161:0/1805251043 >> 172.21.15.161:0/1805251043 conn(0x55be54eb1070 msgr2=0x55be54eb69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 -- 172.21.15.161:0/1805251043 shutdown_connections 2023-07-14T11:23:46.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 -- 172.21.15.161:0/1805251043 wait complete. 2023-07-14T11:23:46.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.992+0000 7fd8947e7b80 1 -- start start 2023-07-14T11:23:46.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 1 Processor -- start 2023-07-14T11:23:46.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ec5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54dacb40 con 0x55be54ec5db0 2023-07-14T11:23:46.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54dac840 con 0x55be54ebb570 2023-07-14T11:23:46.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54c909a0 con 0x55be54ebbe80 2023-07-14T11:23:46.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd8947e7b80 10 client.0 _ll_register_callbacks cb 0x55be54daa7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:46.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ec5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ec5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57514/0 (socket says 172.21.15.161:57514) 2023-07-14T11:23:46.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd87a59c700 1 -- 172.21.15.161:0/3028940380 learned_addr learned my addr 172.21.15.161:0/3028940380 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.993+0000 7fd87ad9d700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd879d9b700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd87a59c700 1 -- 172.21.15.161:0/3028940380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 msgr2=0x55be54ec29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd87a59c700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd87a59c700 1 -- 172.21.15.161:0/3028940380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 msgr2=0x55be54ec2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd87a59c700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd87a59c700 1 -- 172.21.15.161:0/3028940380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd870009920 con 0x55be54ebb570 2023-07-14T11:23:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd87ad9d700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:46.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd87a59c700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ec5730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd86c096ff0 tx=0x7fd86c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=842c88007e79e4c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd861ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:23:46.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd861ffb700 1 -- 172.21.15.161:0/3028940380 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0b3200 con 0x55be54ebb570 2023-07-14T11:23:46.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be54d66570 con 0x55be54ebb570 2023-07-14T11:23:46.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd861ffb700 1 -- 172.21.15.161:0/3028940380 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fd86c09aaf0 con 0x55be54ebb570 2023-07-14T11:23:46.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd861ffb700 1 -- 172.21.15.161:0/3028940380 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd86c09eb20 con 0x55be54ebb570 2023-07-14T11:23:46.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd8947e7b80 10 client.25967 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:46.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.994+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55be54c23ce0 con 0x55be54ebb570 2023-07-14T11:23:46.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd861ffb700 1 -- 172.21.15.161:0/3028940380 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be54d66570 con 0x55be54ebb570 2023-07-14T11:23:46.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd861ffb700 1 -- 172.21.15.161:0/3028940380 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd86c0cb5b0 con 0x55be54ebb570 2023-07-14T11:23:46.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd8947e7b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:46.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd8947e7b80 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:46.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd8947e7b80 20 client.25967 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:46.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54ed5340 0x55be54ed7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd8947e7b80 4 client.25967 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:46.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be54e05f30 con 0x55be54ed5340 2023-07-14T11:23:46.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.995+0000 7fd879d9b700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54ed5340 0x55be54ed7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.996+0000 7fd879d9b700 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54ed5340 0x55be54ed7750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225818e5023507b9 server_cookie=f3dfe8c912dbf164 in_seq=0 out_seq=0 2023-07-14T11:23:46.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.996+0000 7fd861ffb700 10 client.25967 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd861ffb700 1 -- 172.21.15.161:0/3028940380 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55be54e05f30 con 0x55be54ed5340 2023-07-14T11:23:46.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd861ffb700 10 client.25967 handle_command_reply: tid=0 2023-07-14T11:23:46.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 client.25967 shutdown 2023-07-14T11:23:46.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 msgr2=0x55be54ec5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ec5730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd86c096ff0 tx=0x7fd86c009260 comp rx=0 tx=0).stop 2023-07-14T11:23:46.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 shutdown_connections 2023-07-14T11:23:46.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54ed5340 0x55be54ed7750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ec5db0 0x55be54ec2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd861ffb700 0 client.25967 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ebb570 0x55be54ec5730 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3028940380 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ec29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 >> 172.21.15.161:0/3028940380 conn(0x55be54eb1070 msgr2=0x55be54eb65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 shutdown_connections 2023-07-14T11:23:46.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3028940380 wait complete. 2023-07-14T11:23:46.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:45.997+0000 7fd8947e7b80 20 client.25967 trim_cache size 0 max 16384 2023-07-14T11:23:46.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.016+0000 7fd8947e7b80 1 Processor -- start 2023-07-14T11:23:46.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.016+0000 7fd8947e7b80 1 -- start start 2023-07-14T11:23:46.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.016+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebb570 0x55be54ecc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.016+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ebabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54ec5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54ed2120 con 0x55be54ebb570 2023-07-14T11:23:46.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54eb0bf0 con 0x55be54ec5db0 2023-07-14T11:23:46.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54ec0f10 con 0x55be54ebbe80 2023-07-14T11:23:46.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebb570 0x55be54ecc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54ec5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebb570 0x55be54ecc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55026/0 (socket says 172.21.15.161:55026) 2023-07-14T11:23:46.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 -- 172.21.15.161:0/3066151039 learned_addr learned my addr 172.21.15.161:0/3066151039 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd879d9b700 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ebabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 -- 172.21.15.161:0/3066151039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 msgr2=0x55be54ebabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ebabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 -- 172.21.15.161:0/3066151039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 msgr2=0x55be54ec5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54ec5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 -- 172.21.15.161:0/3066151039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be54c612d0 con 0x55be54ebb570 2023-07-14T11:23:46.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.017+0000 7fd87a59c700 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebb570 0x55be54ecc890 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd86c097560 tx=0x7fd86c0054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba67cded007aa7b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8697fa700 1 -- 172.21.15.161:0/3066151039 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0cb0f0 con 0x55be54ebb570 2023-07-14T11:23:46.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8697fa700 1 -- 172.21.15.161:0/3066151039 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd86c0b3200 con 0x55be54ebb570 2023-07-14T11:23:46.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8697fa700 1 -- 172.21.15.161:0/3066151039 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0cb6f0 con 0x55be54ebb570 2023-07-14T11:23:46.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3066151039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebb570 msgr2=0x55be54ecc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebb570 0x55be54ecc890 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd86c097560 tx=0x7fd86c0054c0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3066151039 shutdown_connections 2023-07-14T11:23:46.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebb570 0x55be54ecc890 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54ec5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/3066151039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebbe80 0x55be54ebabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3066151039 >> 172.21.15.161:0/3066151039 conn(0x55be54eb1070 msgr2=0x55be54eba4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.018+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3066151039 shutdown_connections 2023-07-14T11:23:46.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.019+0000 7fd8947e7b80 1 -- 172.21.15.161:0/3066151039 wait complete. 2023-07-14T11:23:46.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.019+0000 7fd8947e7b80 1 -- start start 2023-07-14T11:23:46.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.019+0000 7fd8947e7b80 1 Processor -- start 2023-07-14T11:23:46.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.019+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebb570 0x55be54ecb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.019+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebbe80 0x55be54ecbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.019+0000 7fd8947e7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54f232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54f23830 con 0x55be54ebbe80 2023-07-14T11:23:46.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54f239a0 con 0x55be54ec5db0 2023-07-14T11:23:46.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd8947e7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be54f23b60 con 0x55be54ebb570 2023-07-14T11:23:46.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd8947e7b80 10 client.0 _ll_register_callbacks cb 0x55be54daa7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:46.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebb570 0x55be54ecb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54f232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebb570 0x55be54ecb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49172/0 (socket says 172.21.15.161:49172) 2023-07-14T11:23:46.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54f232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57528/0 (socket says 172.21.15.161:57528) 2023-07-14T11:23:46.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 -- 172.21.15.161:0/2432431167 learned_addr learned my addr 172.21.15.161:0/2432431167 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd879d9b700 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebbe80 0x55be54ecbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 -- 172.21.15.161:0/2432431167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 msgr2=0x55be54f232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54f232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 -- 172.21.15.161:0/2432431167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebbe80 msgr2=0x55be54ecbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebbe80 0x55be54ecbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 -- 172.21.15.161:0/2432431167 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be54c23ce0 con 0x55be54ebb570 2023-07-14T11:23:46.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd87a59c700 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebb570 0x55be54ecb8b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0b0040 tx=0x7fd86c00b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=414158e4cd61e915 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.020+0000 7fd861ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:46.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd861ffb700 1 -- 172.21.15.161:0/2432431167 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0cb690 con 0x55be54ebb570 2023-07-14T11:23:46.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd861ffb700 1 -- 172.21.15.161:0/2432431167 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fd86c09aaf0 con 0x55be54ebb570 2023-07-14T11:23:46.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd870011e20 con 0x55be54ebb570 2023-07-14T11:23:46.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd861ffb700 1 -- 172.21.15.161:0/2432431167 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd86c09eb20 con 0x55be54ebb570 2023-07-14T11:23:46.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 10 client.25966 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:46.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55be54c39330 con 0x55be54ebb570 2023-07-14T11:23:46.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd861ffb700 1 -- 172.21.15.161:0/2432431167 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd870011e20 con 0x55be54ebb570 2023-07-14T11:23:46.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd861ffb700 1 -- 172.21.15.161:0/2432431167 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd86c09b2d0 con 0x55be54ebb570 2023-07-14T11:23:46.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 10 client.25966 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:46.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:46.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 20 client.25966 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:46.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54f2ae20 0x55be54f2d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 4 client.25966 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:23:46.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55be54dc80d0 con 0x55be54f2ae20 2023-07-14T11:23:46.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.021+0000 7fd879d9b700 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54f2ae20 0x55be54f2d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.022+0000 7fd879d9b700 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54f2ae20 0x55be54f2d200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737eb08336f515e server_cookie=529403621c857ba1 in_seq=0 out_seq=0 2023-07-14T11:23:46.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.022+0000 7fd861ffb700 10 client.25966 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd861ffb700 1 -- 172.21.15.161:0/2432431167 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55be54dc80d0 con 0x55be54f2ae20 2023-07-14T11:23:46.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd861ffb700 10 client.25966 handle_command_reply: tid=0 2023-07-14T11:23:46.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 client.25966 shutdown 2023-07-14T11:23:46.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebb570 msgr2=0x55be54ecb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebb570 0x55be54ecb8b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0b0040 tx=0x7fd86c00b2e0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 shutdown_connections 2023-07-14T11:23:46.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55be54f2ae20 0x55be54f2d200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be54ec5db0 0x55be54f232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55be54ebb570 0x55be54ecb8b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd861ffb700 0 client.25966 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 --2- 172.21.15.161:0/2432431167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be54ebbe80 0x55be54ecbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 >> 172.21.15.161:0/2432431167 conn(0x55be54eb1070 msgr2=0x55be54eb5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 shutdown_connections 2023-07-14T11:23:46.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.034+0000 7fd8947e7b80 1 -- 172.21.15.161:0/2432431167 wait complete. 2023-07-14T11:23:46.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd8947e7b80 20 client.25966 trim_cache size 0 max 16384 2023-07-14T11:23:46.039 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:23:46.040 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:23:46.040 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "96150ac7-8932-4247-ae78-c76fa1cc9590", 2023-07-14T11:23:46.040 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:23:46.040 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:23:46.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd8640a1480 msgr2=0x7fd8640a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd8640a1480 0x7fd8640a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8740060b0 tx=0x7fd874004b40 comp rx=0 tx=0).stop 2023-07-14T11:23:46.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 msgr2=0x7fd87c19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c19da70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009810 tx=0x7fd86c00fa00 comp rx=0 tx=0).stop 2023-07-14T11:23:46.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 shutdown_connections 2023-07-14T11:23:46.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd8640a1480 0x7fd8640a3930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd87c109f50 0x7fd87c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87c100c10 0x7fd87c19da70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 --2- 172.21.15.161:0/1668952293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd87c101520 0x7fd87c19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 >> 172.21.15.161:0/1668952293 conn(0x7fd87c0fa3c0 msgr2=0x7fd87c10c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 shutdown_connections 2023-07-14T11:23:46.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.035+0000 7fd87b59e700 1 -- 172.21.15.161:0/1668952293 wait complete. 2023-07-14T11:23:46.056 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:45 smithi161 ceph-mon[95526]: pgmap v48: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2023-07-14T11:23:46.059 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:23:46.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 66198 13.29 MB/sec warmup 28 sec latency 2957.214 ms 2023-07-14T11:23:46.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:45 smithi175 ceph-mon[129081]: pgmap v48: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2023-07-14T11:23:46.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:45 smithi182 ceph-mon[129366]: pgmap v48: 129 pgs: 129 active+clean; 30 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2023-07-14T11:23:46.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.291+0000 7f5b123ff700 1 -- 172.21.15.161:0/866815348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c0ffad0 msgr2=0x7f5b0c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.291+0000 7f5b123ff700 1 --2- 172.21.15.161:0/866815348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c0ffad0 0x7f5b0c0fff40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009990 tx=0x7f5b00009ca0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.292+0000 7f5b123ff700 1 -- 172.21.15.161:0/866815348 shutdown_connections 2023-07-14T11:23:46.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.292+0000 7f5b123ff700 1 --2- 172.21.15.161:0/866815348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c105ae0 0x7f5b0c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.292+0000 7f5b123ff700 1 --2- 172.21.15.161:0/866815348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5b0c100480 0x7f5b0c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.292+0000 7f5b123ff700 1 --2- 172.21.15.161:0/866815348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c0ffad0 0x7f5b0c0fff40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.292+0000 7f5b123ff700 1 -- 172.21.15.161:0/866815348 >> 172.21.15.161:0/866815348 conn(0x7f5b0c0fb5f0 msgr2=0x7f5b0c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.292+0000 7f5b123ff700 1 -- 172.21.15.161:0/866815348 shutdown_connections 2023-07-14T11:23:46.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.292+0000 7f5b123ff700 1 -- 172.21.15.161:0/866815348 wait complete. 2023-07-14T11:23:46.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.293+0000 7f5b123ff700 1 Processor -- start 2023-07-14T11:23:46.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.293+0000 7f5b123ff700 1 -- start start 2023-07-14T11:23:46.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b123ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0ffad0 0x7f5b0c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b123ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c100480 0x7f5b0c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b123ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5b0c105ae0 0x7f5b0c195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b123ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c1960b0 con 0x7f5b0c0ffad0 2023-07-14T11:23:46.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b123ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c196220 con 0x7f5b0c100480 2023-07-14T11:23:46.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b123ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c1963e0 con 0x7f5b0c105ae0 2023-07-14T11:23:46.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c100480 0x7f5b0c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c100480 0x7f5b0c19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57544/0 (socket says 172.21.15.161:57544) 2023-07-14T11:23:46.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b0b7fe700 1 -- 172.21.15.161:0/1648539118 learned_addr learned my addr 172.21.15.161:0/1648539118 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b1099c700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5b0c105ae0 0x7f5b0c195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0ffad0 0x7f5b0c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b1099c700 1 -- 172.21.15.161:0/1648539118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c100480 msgr2=0x7f5b0c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b1099c700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c100480 0x7f5b0c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b1099c700 1 -- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0ffad0 msgr2=0x7f5b0c19b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b1099c700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0ffad0 0x7f5b0c19b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b1099c700 1 -- 172.21.15.161:0/1648539118 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b000095f0 con 0x7f5b0c105ae0 2023-07-14T11:23:46.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c100480 0x7f5b0c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:46.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.294+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0ffad0 0x7f5b0c19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:46.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.295+0000 7f5b1099c700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5b0c105ae0 0x7f5b0c195a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400efc0 tx=0x7f5b0400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.295+0000 7f5b097fa700 1 -- 172.21.15.161:0/1648539118 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04018070 con 0x7f5b0c105ae0 2023-07-14T11:23:46.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.295+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c196660 con 0x7f5b0c105ae0 2023-07-14T11:23:46.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.295+0000 7f5b097fa700 1 -- 172.21.15.161:0/1648539118 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0400f040 con 0x7f5b0c105ae0 2023-07-14T11:23:46.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.295+0000 7f5b097fa700 1 -- 172.21.15.161:0/1648539118 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400cd40 con 0x7f5b0c105ae0 2023-07-14T11:23:46.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.295+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c1a29c0 con 0x7f5b0c105ae0 2023-07-14T11:23:46.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.299+0000 7f5b097fa700 1 -- 172.21.15.161:0/1648539118 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f5b04003a00 con 0x7f5b0c105ae0 2023-07-14T11:23:46.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 Processor -- start 2023-07-14T11:23:46.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 -- start start 2023-07-14T11:23:46.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b097fa700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5af40a1460 0x7f5af40a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db183ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db18e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b097fa700 1 -- 172.21.15.161:0/1648539118 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f5b040ccdd0 con 0x7f5b0c105ae0 2023-07-14T11:23:46.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db085cb0 con 0x5582db18eee0 2023-07-14T11:23:46.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db0d8a40 con 0x5582db1836e0 2023-07-14T11:23:46.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db0d1ed0 con 0x5582db183ff0 2023-07-14T11:23:46.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db18e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b1099c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db18e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49194/0 (socket says 172.21.15.161:49194) 2023-07-14T11:23:46.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.301+0000 7f5b0b7fe700 1 -- 172.21.15.161:0/3143969016 learned_addr learned my addr 172.21.15.161:0/3143969016 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5af40a1460 0x7f5af40a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db183ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b1099c700 1 -- 172.21.15.161:0/3143969016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 msgr2=0x5582db18e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b1099c700 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db18e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b1099c700 1 -- 172.21.15.161:0/3143969016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 msgr2=0x5582db183ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b1099c700 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db183ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b1099c700 1 -- 172.21.15.161:0/3143969016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582daeffde0 con 0x5582db18eee0 2023-07-14T11:23:46.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db18e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:46.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5b1099c700 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1912c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b040ca070 tx=0x7f5b04095e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9027342dbb0abd0f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5af27fc700 1 -- 172.21.15.161:0/3143969016 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04004ad0 con 0x5582db18eee0 2023-07-14T11:23:46.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5af27fc700 1 -- 172.21.15.161:0/3143969016 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b040999b0 con 0x5582db18eee0 2023-07-14T11:23:46.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.302+0000 7f5af27fc700 1 -- 172.21.15.161:0/3143969016 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04099100 con 0x5582db18eee0 2023-07-14T11:23:46.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/3143969016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 msgr2=0x5582db1912c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1912c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b040ca070 tx=0x7f5b04095e10 comp rx=0 tx=0).stop 2023-07-14T11:23:46.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5af40a1460 0x7f5af40a3910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000b6f0 tx=0x7f5b0000b260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/3143969016 shutdown_connections 2023-07-14T11:23:46.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1912c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db183ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/3143969016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db18e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/3143969016 >> 172.21.15.161:0/3143969016 conn(0x5582db175da0 msgr2=0x5582db17b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/3143969016 shutdown_connections 2023-07-14T11:23:46.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.303+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/3143969016 wait complete. 2023-07-14T11:23:46.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.304+0000 7f5b25dc0b80 1 -- start start 2023-07-14T11:23:46.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.304+0000 7f5b25dc0b80 1 Processor -- start 2023-07-14T11:23:46.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.304+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db1836e0 0x5582db18e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.304+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db183ff0 0x5582db18bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db18c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db071ac0 con 0x5582db18eee0 2023-07-14T11:23:46.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db0717c0 con 0x5582db183ff0 2023-07-14T11:23:46.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582daf55920 con 0x5582db1836e0 2023-07-14T11:23:46.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b25dc0b80 10 client.0 _ll_register_callbacks cb 0x5582db06f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:46.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db183ff0 0x5582db18bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db1836e0 0x5582db18e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db1836e0 0x5582db18e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49204/0 (socket says 172.21.15.161:49204) 2023-07-14T11:23:46.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db183ff0 0x5582db18bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57568/0 (socket says 172.21.15.161:57568) 2023-07-14T11:23:46.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 -- 172.21.15.161:0/35743414 learned_addr learned my addr 172.21.15.161:0/35743414 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b1099c700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db18c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 -- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db183ff0 msgr2=0x5582db18bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db183ff0 0x5582db18bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 -- 172.21.15.161:0/35743414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 msgr2=0x5582db18c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db18c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0bfff700 1 -- 172.21.15.161:0/35743414 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c1a29c0 con 0x5582db1836e0 2023-07-14T11:23:46.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db183ff0 0x5582db18bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:23:46.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.305+0000 7f5b1099c700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db18c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:46.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db1836e0 0x5582db18e940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5afc011ad0 tx=0x7f5afc011e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f4960fbf7fa701 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5aeb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:23:46.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/35743414 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc016070 con 0x5582db1836e0 2023-07-14T11:23:46.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582db02b4f0 con 0x5582db1836e0 2023-07-14T11:23:46.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/35743414 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f5afc00fb60 con 0x5582db1836e0 2023-07-14T11:23:46.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/35743414 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5afc00fd20 con 0x5582db1836e0 2023-07-14T11:23:46.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5b25dc0b80 10 client.25978 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:46.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5582daee8c60 con 0x5582db1836e0 2023-07-14T11:23:46.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/35743414 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582db02b4f0 con 0x5582db1836e0 2023-07-14T11:23:46.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.306+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/35743414 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5afc019390 con 0x5582db1836e0 2023-07-14T11:23:46.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.307+0000 7f5b25dc0b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:46.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.307+0000 7f5b25dc0b80 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:46.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.307+0000 7f5b25dc0b80 20 client.25978 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:46.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.307+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db19e8d0 0x5582db1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.307+0000 7f5b25dc0b80 4 client.25978 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:23:46.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.307+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582db0caeb0 con 0x5582db19e8d0 2023-07-14T11:23:46.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.307+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db19e8d0 0x5582db1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.308+0000 7f5aeb7fe700 10 client.25978 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.308+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db19e8d0 0x5582db1a0ce0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650ae0edc6403d0a server_cookie=dc9f91686d8bf480 in_seq=0 out_seq=0 2023-07-14T11:23:46.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/35743414 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5582db0caeb0 con 0x5582db19e8d0 2023-07-14T11:23:46.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5aeb7fe700 10 client.25978 handle_command_reply: tid=0 2023-07-14T11:23:46.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 client.25978 shutdown 2023-07-14T11:23:46.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db1836e0 msgr2=0x5582db18e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db1836e0 0x5582db18e940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5afc011ad0 tx=0x7f5afc011e90 comp rx=0 tx=0).stop 2023-07-14T11:23:46.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 shutdown_connections 2023-07-14T11:23:46.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db19e8d0 0x5582db1a0ce0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db18c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db1836e0 0x5582db18e940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5aeb7fe700 0 client.25978 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/35743414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db183ff0 0x5582db18bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.309+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 >> 172.21.15.161:0/35743414 conn(0x5582db175da0 msgr2=0x5582db17b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.310+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 shutdown_connections 2023-07-14T11:23:46.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.310+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/35743414 wait complete. 2023-07-14T11:23:46.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.310+0000 7f5b25dc0b80 20 client.25978 trim_cache size 0 max 16384 2023-07-14T11:23:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.328+0000 7f5b25dc0b80 1 Processor -- start 2023-07-14T11:23:46.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.328+0000 7f5b25dc0b80 1 -- start start 2023-07-14T11:23:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db195900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db183ff0 0x5582db185f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db18eee0 0x5582db188ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db184950 con 0x5582db183ff0 2023-07-14T11:23:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db185170 con 0x5582db1836e0 2023-07-14T11:23:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db172fa0 con 0x5582db18eee0 2023-07-14T11:23:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db183ff0 0x5582db185f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db18eee0 0x5582db188ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db18eee0 0x5582db188ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49206/0 (socket says 172.21.15.161:49206) 2023-07-14T11:23:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 -- 172.21.15.161:0/2257805389 learned_addr learned my addr 172.21.15.161:0/2257805389 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 -- 172.21.15.161:0/2257805389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 msgr2=0x5582db195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 -- 172.21.15.161:0/2257805389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db183ff0 msgr2=0x5582db185f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db183ff0 0x5582db185f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.329+0000 7f5b1099c700 1 -- 172.21.15.161:0/2257805389 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582daf26250 con 0x5582db18eee0 2023-07-14T11:23:46.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.330+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:23:46.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.330+0000 7f5b1099c700 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db18eee0 0x5582db188ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400ef90 tx=0x7f5b04099b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35c8cbebcc6570b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.330+0000 7f5af2ffd700 1 -- 172.21.15.161:0/2257805389 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0409d9e0 con 0x5582db18eee0 2023-07-14T11:23:46.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.330+0000 7f5af2ffd700 1 -- 172.21.15.161:0/2257805389 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b04004ad0 con 0x5582db18eee0 2023-07-14T11:23:46.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.330+0000 7f5af2ffd700 1 -- 172.21.15.161:0/2257805389 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b040a1b40 con 0x5582db18eee0 2023-07-14T11:23:46.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/2257805389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db18eee0 msgr2=0x5582db188ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db18eee0 0x5582db188ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400ef90 tx=0x7f5b04099b20 comp rx=0 tx=0).stop 2023-07-14T11:23:46.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/2257805389 shutdown_connections 2023-07-14T11:23:46.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db18eee0 0x5582db188ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/2257805389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db183ff0 0x5582db185f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/2257805389 >> 172.21.15.161:0/2257805389 conn(0x5582db175da0 msgr2=0x5582db1857f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/2257805389 shutdown_connections 2023-07-14T11:23:46.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/2257805389 wait complete. 2023-07-14T11:23:46.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.331+0000 7f5b25dc0b80 1 -- start start 2023-07-14T11:23:46.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 1 Processor -- start 2023-07-14T11:23:46.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db1937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db193cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1eb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db1ebc80 con 0x5582db18eee0 2023-07-14T11:23:46.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db1ebdf0 con 0x5582db1836e0 2023-07-14T11:23:46.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582db1ebfb0 con 0x5582db183ff0 2023-07-14T11:23:46.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b25dc0b80 10 client.0 _ll_register_callbacks cb 0x7f5afc0117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:23:46.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b0bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db1937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b0b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db193cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b0bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db1937b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57588/0 (socket says 172.21.15.161:57588) 2023-07-14T11:23:46.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.332+0000 7f5b0bfff700 1 -- 172.21.15.161:0/1879897305 learned_addr learned my addr 172.21.15.161:0/1879897305 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:23:46.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b1099c700 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b0bfff700 1 -- 172.21.15.161:0/1879897305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 msgr2=0x5582db193cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db193cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b0bfff700 1 -- 172.21.15.161:0/1879897305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 msgr2=0x5582db1eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:23:46.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b0bfff700 1 -- 172.21.15.161:0/1879897305 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582daeffde0 con 0x5582db1836e0 2023-07-14T11:23:46.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b0bfff700 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db1937b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009c90 tx=0x7f5afc019930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd5043324b9a1a12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:23:46.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5aeb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:23:46.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/1879897305 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc016040 con 0x5582db1836e0 2023-07-14T11:23:46.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b040999b0 con 0x5582db1836e0 2023-07-14T11:23:46.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.333+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/1879897305 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f5afc000e40 con 0x5582db1836e0 2023-07-14T11:23:46.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/1879897305 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5afc019e60 con 0x5582db1836e0 2023-07-14T11:23:46.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 10 client.25991 fetch_fsmap learned FSMap version 23 2023-07-14T11:23:46.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5582daefe2b0 con 0x5582db1836e0 2023-07-14T11:23:46.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/1879897305 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b040999b0 con 0x5582db1836e0 2023-07-14T11:23:46.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/1879897305 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5afc003680 con 0x5582db1836e0 2023-07-14T11:23:46.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 10 client.25991 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:23:46.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 10 client.25991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:23:46.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 20 client.25991 populate_metadata read hostname 'smithi161' 2023-07-14T11:23:46.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db17d6e0 0x5582db17faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:23:46.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 4 client.25991 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:23:46.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5582db08d050 con 0x5582db17d6e0 2023-07-14T11:23:46.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.334+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db17d6e0 0x5582db17faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:23:46.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.335+0000 7f5b0b7fe700 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db17d6e0 0x5582db17faf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f640eecc690fff4 server_cookie=a4f29aee536b492e in_seq=0 out_seq=0 2023-07-14T11:23:46.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.335+0000 7f5aeb7fe700 10 client.25991 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5aeb7fe700 1 -- 172.21.15.161:0/1879897305 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5582db08d050 con 0x5582db17d6e0 2023-07-14T11:23:46.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5aeb7fe700 10 client.25991 handle_command_reply: tid=0 2023-07-14T11:23:46.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 client.25991 shutdown 2023-07-14T11:23:46.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 msgr2=0x5582db1937b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db1937b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009c90 tx=0x7f5afc019930 comp rx=0 tx=0).stop 2023-07-14T11:23:46.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 shutdown_connections 2023-07-14T11:23:46.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5582db17d6e0 0x5582db17faf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582db18eee0 0x5582db1eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582db1836e0 0x5582db1937b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 --2- 172.21.15.161:0/1879897305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582db183ff0 0x5582db193cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5aeb7fe700 0 client.25991 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:23:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 >> 172.21.15.161:0/1879897305 conn(0x5582db175da0 msgr2=0x5582db186080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 shutdown_connections 2023-07-14T11:23:46.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.336+0000 7f5b25dc0b80 1 -- 172.21.15.161:0/1879897305 wait complete. 2023-07-14T11:23:46.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b25dc0b80 20 client.25991 trim_cache size 0 max 16384 2023-07-14T11:23:46.348 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:23:46.348 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:23:46.349 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:23:46.349 INFO:teuthology.orchestra.run.smithi161.stdout: "96150ac7-8932-4247-ae78-c76fa1cc9590": { 2023-07-14T11:23:46.349 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:23:46.349 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "96150ac7-8932-4247-ae78-c76fa1cc9590", 2023-07-14T11:23:46.349 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:23:46.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:23:46.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:23:46.350 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:23:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5af40a1460 msgr2=0x7f5af40a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5af40a1460 0x7f5af40a3910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000b6f0 tx=0x7f5b0000b260 comp rx=0 tx=0).stop 2023-07-14T11:23:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5b0c105ae0 msgr2=0x7f5b0c195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:23:46.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5b0c105ae0 0x7f5b0c195a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400efc0 tx=0x7f5b0400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 shutdown_connections 2023-07-14T11:23:46.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5af40a1460 0x7f5af40a3910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0ffad0 0x7f5b0c19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5b0c105ae0 0x7f5b0c195a40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 --2- 172.21.15.161:0/1648539118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b0c100480 0x7f5b0c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:23:46.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 >> 172.21.15.161:0/1648539118 conn(0x7f5b0c0fb5f0 msgr2=0x7f5b0c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:23:46.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 shutdown_connections 2023-07-14T11:23:46.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:23:46.337+0000 7f5b123ff700 1 -- 172.21.15.161:0/1648539118 wait complete. 2023-07-14T11:23:46.359 INFO:tasks.cephfs.filesystem:scrub status for tag:96150ac7-8932-4247-ae78-c76fa1cc9590 - {'path': '/', 'tag': '96150ac7-8932-4247-ae78-c76fa1cc9590', 'options': 'recursive,force'} 2023-07-14T11:23:47.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 68027 13.25 MB/sec warmup 29 sec latency 18.719 ms 2023-07-14T11:23:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:23:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:23:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:23:47.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:23:47.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:46 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:23:47.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:23:47.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:23:47.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:23:47.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:46 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:23:47.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:23:47.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:23:47.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:23:47.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:47.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:46 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:23:48.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 71121 13.40 MB/sec warmup 30 sec latency 12.477 ms 2023-07-14T11:23:48.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:47 smithi175 ceph-mon[129081]: pgmap v49: 129 pgs: 129 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-07-14T11:23:48.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:47 smithi182 ceph-mon[129366]: pgmap v49: 129 pgs: 129 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-07-14T11:23:48.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:47 smithi161 ceph-mon[95526]: pgmap v49: 129 pgs: 129 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-07-14T11:23:49.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 72739 13.18 MB/sec warmup 31 sec latency 40.311 ms 2023-07-14T11:23:49.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:48 smithi175 ceph-mon[129081]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-14T11:23:49.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:48 smithi182 ceph-mon[129366]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-14T11:23:49.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:48 smithi161 ceph-mon[95526]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-14T11:23:50.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 74986 13.21 MB/sec warmup 32 sec latency 21.905 ms 2023-07-14T11:23:50.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:49 smithi175 ceph-mon[129081]: pgmap v50: 129 pgs: 129 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 891 KiB/s wr, 116 op/s 2023-07-14T11:23:50.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:49 smithi182 ceph-mon[129366]: pgmap v50: 129 pgs: 129 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 891 KiB/s wr, 116 op/s 2023-07-14T11:23:50.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:49 smithi161 ceph-mon[95526]: pgmap v50: 129 pgs: 129 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 891 KiB/s wr, 116 op/s 2023-07-14T11:23:51.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 77468 13.18 MB/sec warmup 33 sec latency 57.321 ms 2023-07-14T11:23:52.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 79000 13.01 MB/sec warmup 34 sec latency 15.238 ms 2023-07-14T11:23:52.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:52 smithi175 ceph-mon[129081]: pgmap v51: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.1 MiB/s wr, 213 op/s 2023-07-14T11:23:52.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:52 smithi182 ceph-mon[129366]: pgmap v51: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.1 MiB/s wr, 213 op/s 2023-07-14T11:23:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:52 smithi161 ceph-mon[95526]: pgmap v51: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.1 MiB/s wr, 213 op/s 2023-07-14T11:23:53.164 INFO:tasks.workunit.client.0.smithi161.stdout: 1 81776 13.13 MB/sec warmup 35 sec latency 11.411 ms 2023-07-14T11:23:54.165 INFO:tasks.workunit.client.0.smithi161.stdout: 1 82805 12.86 MB/sec warmup 36 sec latency 61.793 ms 2023-07-14T11:23:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:53 smithi175 ceph-mon[129081]: pgmap v52: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 2.5 MiB/s wr, 130 op/s 2023-07-14T11:23:54.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:53 smithi182 ceph-mon[129366]: pgmap v52: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 2.5 MiB/s wr, 130 op/s 2023-07-14T11:23:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:53 smithi161 ceph-mon[95526]: pgmap v52: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 2.5 MiB/s wr, 130 op/s 2023-07-14T11:23:55.165 INFO:tasks.workunit.client.0.smithi161.stdout: 1 85186 12.93 MB/sec warmup 37 sec latency 110.250 ms 2023-07-14T11:23:56.165 INFO:tasks.workunit.client.0.smithi161.stdout: 1 86284 12.68 MB/sec warmup 38 sec latency 29.411 ms 2023-07-14T11:23:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:56 smithi175 ceph-mon[129081]: pgmap v53: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 2.5 MiB/s wr, 130 op/s 2023-07-14T11:23:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:56 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:23:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:56 smithi182 ceph-mon[129366]: pgmap v53: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 2.5 MiB/s wr, 130 op/s 2023-07-14T11:23:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:56 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:23:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:56 smithi161 ceph-mon[95526]: pgmap v53: 129 pgs: 129 active+clean; 57 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 2.5 MiB/s wr, 130 op/s 2023-07-14T11:23:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:56 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:23:57.165 INFO:tasks.workunit.client.0.smithi161.stdout: 1 88963 12.79 MB/sec warmup 39 sec latency 14.071 ms 2023-07-14T11:23:57.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:57 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:23:57.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:57 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:23:57.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:57 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:23:58.165 INFO:tasks.workunit.client.0.smithi161.stdout: 1 90418 12.61 MB/sec warmup 40 sec latency 10.899 ms 2023-07-14T11:23:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:23:58 smithi175 ceph-mon[129081]: pgmap v54: 129 pgs: 129 active+clean; 61 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 251 op/s 2023-07-14T11:23:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:23:58 smithi182 ceph-mon[129366]: pgmap v54: 129 pgs: 129 active+clean; 61 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 251 op/s 2023-07-14T11:23:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:23:58 smithi161 ceph-mon[95526]: pgmap v54: 129 pgs: 129 active+clean; 61 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 251 op/s 2023-07-14T11:23:59.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 92698 12.64 MB/sec warmup 41 sec latency 19.821 ms 2023-07-14T11:24:00.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 94333 12.47 MB/sec warmup 42 sec latency 80.201 ms 2023-07-14T11:24:00.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:00 smithi175 ceph-mon[129081]: pgmap v55: 129 pgs: 129 active+clean; 61 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2023-07-14T11:24:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:00 smithi182 ceph-mon[129366]: pgmap v55: 129 pgs: 129 active+clean; 61 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2023-07-14T11:24:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:00 smithi161 ceph-mon[95526]: pgmap v55: 129 pgs: 129 active+clean; 61 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2023-07-14T11:24:01.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 95993 12.47 MB/sec warmup 43 sec latency 182.937 ms 2023-07-14T11:24:02.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 96382 12.21 MB/sec warmup 44 sec latency 80.116 ms 2023-07-14T11:24:02.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:02 smithi175 ceph-mon[129081]: pgmap v56: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 352 op/s 2023-07-14T11:24:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:02 smithi182 ceph-mon[129366]: pgmap v56: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 352 op/s 2023-07-14T11:24:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:02 smithi161 ceph-mon[95526]: pgmap v56: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 352 op/s 2023-07-14T11:24:03.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 98595 12.18 MB/sec warmup 45 sec latency 25.126 ms 2023-07-14T11:24:04.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 100233 12.11 MB/sec warmup 46 sec latency 13.859 ms 2023-07-14T11:24:04.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:03 smithi175 ceph-mon[129081]: pgmap v57: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2023-07-14T11:24:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:03 smithi182 ceph-mon[129366]: pgmap v57: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2023-07-14T11:24:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:03 smithi161 ceph-mon[95526]: pgmap v57: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2023-07-14T11:24:05.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 102569 12.12 MB/sec warmup 47 sec latency 60.665 ms 2023-07-14T11:24:06.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 103736 12.00 MB/sec warmup 48 sec latency 36.886 ms 2023-07-14T11:24:06.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:06 smithi175 ceph-mon[129081]: pgmap v58: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2023-07-14T11:24:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:06 smithi182 ceph-mon[129366]: pgmap v58: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2023-07-14T11:24:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:06 smithi161 ceph-mon[95526]: pgmap v58: 129 pgs: 129 active+clean; 63 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2023-07-14T11:24:07.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 106542 12.11 MB/sec warmup 49 sec latency 9.548 ms 2023-07-14T11:24:08.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 107634 11.93 MB/sec warmup 50 sec latency 27.714 ms 2023-07-14T11:24:08.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:08 smithi175 ceph-mon[129081]: pgmap v59: 129 pgs: 129 active+clean; 67 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 382 op/s 2023-07-14T11:24:08.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:08 smithi182 ceph-mon[129366]: pgmap v59: 129 pgs: 129 active+clean; 67 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 382 op/s 2023-07-14T11:24:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:08 smithi161 ceph-mon[95526]: pgmap v59: 129 pgs: 129 active+clean; 67 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 382 op/s 2023-07-14T11:24:09.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 110242 12.03 MB/sec warmup 51 sec latency 19.419 ms 2023-07-14T11:24:10.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 111425 11.90 MB/sec warmup 52 sec latency 106.876 ms 2023-07-14T11:24:10.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:10 smithi175 ceph-mon[129081]: pgmap v60: 129 pgs: 129 active+clean; 67 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2023-07-14T11:24:10.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:10 smithi182 ceph-mon[129366]: pgmap v60: 129 pgs: 129 active+clean; 67 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2023-07-14T11:24:10.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:10 smithi161 ceph-mon[95526]: pgmap v60: 129 pgs: 129 active+clean; 67 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2023-07-14T11:24:11.166 INFO:tasks.workunit.client.0.smithi161.stdout: 1 113806 11.94 MB/sec warmup 53 sec latency 40.260 ms 2023-07-14T11:24:12.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 115152 11.82 MB/sec warmup 54 sec latency 17.585 ms 2023-07-14T11:24:12.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:12 smithi175 ceph-mon[129081]: pgmap v61: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2023-07-14T11:24:12.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:12 smithi182 ceph-mon[129366]: pgmap v61: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2023-07-14T11:24:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:12 smithi161 ceph-mon[95526]: pgmap v61: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2023-07-14T11:24:13.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 117489 11.85 MB/sec warmup 55 sec latency 41.895 ms 2023-07-14T11:24:14.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 119508 11.77 MB/sec warmup 56 sec latency 22.608 ms 2023-07-14T11:24:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:13 smithi175 ceph-mon[129081]: pgmap v62: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-07-14T11:24:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:13 smithi182 ceph-mon[129366]: pgmap v62: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-07-14T11:24:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:13 smithi161 ceph-mon[95526]: pgmap v62: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-07-14T11:24:15.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 121190 11.77 MB/sec warmup 57 sec latency 34.786 ms 2023-07-14T11:24:16.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 123509 11.77 MB/sec warmup 58 sec latency 27.946 ms 2023-07-14T11:24:16.360 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:24:16.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:16 smithi175 ceph-mon[129081]: pgmap v63: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-07-14T11:24:16.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:16 smithi161 ceph-mon[95526]: pgmap v63: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-07-14T11:24:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:16 smithi182 ceph-mon[129366]: pgmap v63: 129 pgs: 129 active+clean; 71 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-07-14T11:24:16.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 -- 172.21.15.161:0/2015999283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8102910 msgr2=0x7f93b81068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 --2- 172.21.15.161:0/2015999283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8102910 0x7f93b81068d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009990 tx=0x7f93a8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 -- 172.21.15.161:0/2015999283 shutdown_connections 2023-07-14T11:24:16.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 --2- 172.21.15.161:0/2015999283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8101f70 0x7f93b8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 --2- 172.21.15.161:0/2015999283 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93b8106fb0 0x7f93b810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 --2- 172.21.15.161:0/2015999283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8102910 0x7f93b81068d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 -- 172.21.15.161:0/2015999283 >> 172.21.15.161:0/2015999283 conn(0x7f93b80fd7e0 msgr2=0x7f93b80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:16.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 -- 172.21.15.161:0/2015999283 shutdown_connections 2023-07-14T11:24:16.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.572+0000 7f93c01ff700 1 -- 172.21.15.161:0/2015999283 wait complete. 2023-07-14T11:24:16.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.573+0000 7f93c01ff700 1 Processor -- start 2023-07-14T11:24:16.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.573+0000 7f93c01ff700 1 -- start start 2023-07-14T11:24:16.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.573+0000 7f93c01ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8101f70 0x7f93b8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.573+0000 7f93c01ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93b8102910 0x7f93b8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93c01ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8106fb0 0x7f93b819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93c01ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819ce20 con 0x7f93b8106fb0 2023-07-14T11:24:16.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93c01ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819cf90 con 0x7f93b8101f70 2023-07-14T11:24:16.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93c01ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819d150 con 0x7f93b8102910 2023-07-14T11:24:16.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93be79c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8106fb0 0x7f93b819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bd79a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93b8102910 0x7f93b8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bdf9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8101f70 0x7f93b8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93be79c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8106fb0 0x7f93b819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49618/0 (socket says 172.21.15.161:49618) 2023-07-14T11:24:16.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93be79c700 1 -- 172.21.15.161:0/3357138539 learned_addr learned my addr 172.21.15.161:0/3357138539 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:16.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bd79a700 1 -- 172.21.15.161:0/3357138539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8101f70 msgr2=0x7f93b8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bd79a700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8101f70 0x7f93b8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bd79a700 1 -- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8106fb0 msgr2=0x7f93b819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bd79a700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8106fb0 0x7f93b819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bd79a700 1 -- 172.21.15.161:0/3357138539 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009520 con 0x7f93b8102910 2023-07-14T11:24:16.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93be79c700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8106fb0 0x7f93b819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:16.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.574+0000 7f93bd79a700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93b8102910 0x7f93b8198380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f93a80098e0 tx=0x7f93a8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.575+0000 7f93aeffd700 1 -- 172.21.15.161:0/3357138539 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a801d070 con 0x7f93b8102910 2023-07-14T11:24:16.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.575+0000 7f93aeffd700 1 -- 172.21.15.161:0/3357138539 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a8021410 con 0x7f93b8102910 2023-07-14T11:24:16.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.575+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a80095f0 con 0x7f93b8102910 2023-07-14T11:24:16.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.575+0000 7f93aeffd700 1 -- 172.21.15.161:0/3357138539 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8017650 con 0x7f93b8102910 2023-07-14T11:24:16.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.575+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b819d660 con 0x7f93b8102910 2023-07-14T11:24:16.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.578+0000 7f93aeffd700 1 -- 172.21.15.161:0/3357138539 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f93a80217f0 con 0x7f93b8102910 2023-07-14T11:24:16.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.579+0000 7f93d3a60b80 1 Processor -- start 2023-07-14T11:24:16.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.579+0000 7f93d3a60b80 1 -- start start 2023-07-14T11:24:16.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93aeffd700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93a40a12c0 0x7f93a40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93aeffd700 1 -- 172.21.15.161:0/3357138539 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f93a80ce3a0 con 0x7f93b8102910 2023-07-14T11:24:16.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf30590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf32eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be2bcb0 con 0x56296bf252d0 2023-07-14T11:24:16.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be7ea40 con 0x56296bf25be0 2023-07-14T11:24:16.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be77ed0 con 0x56296bf30ad0 2023-07-14T11:24:16.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93be79c700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93a40a12c0 0x7f93a40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93bdf9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93bdf9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf256a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49632/0 (socket says 172.21.15.161:49632) 2023-07-14T11:24:16.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93bdf9b700 1 -- 172.21.15.161:0/3174502730 learned_addr learned my addr 172.21.15.161:0/3174502730 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:16.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93bd79a700 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf30590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.580+0000 7f93be79c700 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf32eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93bd79a700 1 -- 172.21.15.161:0/3174502730 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 msgr2=0x56296bf32eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93bd79a700 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf32eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93bd79a700 1 -- 172.21.15.161:0/3174502730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 msgr2=0x56296bf256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93bd79a700 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93bd79a700 1 -- 172.21.15.161:0/3174502730 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296bca5de0 con 0x56296bf25be0 2023-07-14T11:24:16.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93bd79a700 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf30590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93a80cb760 tx=0x7f93a80975e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b98c1f5a56b5e42 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93be79c700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93a40a12c0 0x7f93a40a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009fd0 tx=0x7f93b0007c40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93a3fff700 1 -- 172.21.15.161:0/3174502730 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80ce170 con 0x56296bf25be0 2023-07-14T11:24:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93a3fff700 1 -- 172.21.15.161:0/3174502730 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a8021e90 con 0x56296bf25be0 2023-07-14T11:24:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.581+0000 7f93a3fff700 1 -- 172.21.15.161:0/3174502730 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a809b5d0 con 0x56296bf25be0 2023-07-14T11:24:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 -- 172.21.15.161:0/3174502730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 msgr2=0x56296bf30590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf30590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93a80cb760 tx=0x7f93a80975e0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 -- 172.21.15.161:0/3174502730 shutdown_connections 2023-07-14T11:24:16.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf32eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/3174502730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf30590 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 -- 172.21.15.161:0/3174502730 >> 172.21.15.161:0/3174502730 conn(0x56296bf1bda0 msgr2=0x56296bf216a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:16.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 -- 172.21.15.161:0/3174502730 shutdown_connections 2023-07-14T11:24:16.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.582+0000 7f93d3a60b80 1 -- 172.21.15.161:0/3174502730 wait complete. 2023-07-14T11:24:16.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.583+0000 7f93d3a60b80 1 -- start start 2023-07-14T11:24:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.583+0000 7f93d3a60b80 1 Processor -- start 2023-07-14T11:24:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.583+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf30ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.583+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf31920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be17ac0 con 0x56296bf252d0 2023-07-14T11:24:16.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be177c0 con 0x56296bf25be0 2023-07-14T11:24:16.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bcfb920 con 0x56296bf30ad0 2023-07-14T11:24:16.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93d3a60b80 10 client.0 _ll_register_callbacks cb 0x56296be15760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf31920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf31920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34502/0 (socket says 172.21.15.161:34502) 2023-07-14T11:24:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 -- 172.21.15.161:0/2435766824 learned_addr learned my addr 172.21.15.161:0/2435766824 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:16.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93bdf9b700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf30ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93bd79a700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 -- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 msgr2=0x56296bf313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 -- 172.21.15.161:0/2435766824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 msgr2=0x56296bf30ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf30ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93be79c700 1 -- 172.21.15.161:0/2435766824 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b819d660 con 0x56296bf30ad0 2023-07-14T11:24:16.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.584+0000 7f93bdf9b700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf30ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:16.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93be79c700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf31920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93b001a6f0 tx=0x7f93b0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d958686d9101a795 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:16.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93a17fa700 1 -- 172.21.15.161:0/2435766824 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0020040 con 0x56296bf30ad0 2023-07-14T11:24:16.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56296bdd14f0 con 0x56296bf30ad0 2023-07-14T11:24:16.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93a17fa700 1 -- 172.21.15.161:0/2435766824 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f93b0003e10 con 0x56296bf30ad0 2023-07-14T11:24:16.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93a17fa700 1 -- 172.21.15.161:0/2435766824 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b0007e80 con 0x56296bf30ad0 2023-07-14T11:24:16.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93d3a60b80 10 client.25996 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:16.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.585+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56296bc8ec60 con 0x56296bf30ad0 2023-07-14T11:24:16.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93a17fa700 1 -- 172.21.15.161:0/2435766824 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56296bdd14f0 con 0x56296bf30ad0 2023-07-14T11:24:16.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93a17fa700 1 -- 172.21.15.161:0/2435766824 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f93b0023070 con 0x56296bf30ad0 2023-07-14T11:24:16.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93d3a60b80 10 client.25996 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93d3a60b80 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93d3a60b80 20 client.25996 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf40120 0x56296bf42530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93d3a60b80 4 client.25996 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56296be70eb0 con 0x56296bf40120 2023-07-14T11:24:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.586+0000 7f93bdf9b700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf40120 0x56296bf42530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.587+0000 7f93bdf9b700 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf40120 0x56296bf42530 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1283e451d7f64a59 server_cookie=b828a08985e78e56 in_seq=0 out_seq=0 2023-07-14T11:24:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.587+0000 7f93a17fa700 10 client.25996 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.587+0000 7f93a17fa700 1 -- 172.21.15.161:0/2435766824 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56296be70eb0 con 0x56296bf40120 2023-07-14T11:24:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.587+0000 7f93a17fa700 10 client.25996 handle_command_reply: tid=0 2023-07-14T11:24:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.587+0000 7f93d3a60b80 1 client.25996 shutdown 2023-07-14T11:24:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 msgr2=0x56296bf31920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf31920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93b001a6f0 tx=0x7f93b0004b40 comp rx=0 tx=0).stop 2023-07-14T11:24:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 shutdown_connections 2023-07-14T11:24:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf40120 0x56296bf42530 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf30ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf31920 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2435766824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93a17fa700 0 client.25996 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:16.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 >> 172.21.15.161:0/2435766824 conn(0x56296bf1bda0 msgr2=0x56296bf21290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:16.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 shutdown_connections 2023-07-14T11:24:16.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2435766824 wait complete. 2023-07-14T11:24:16.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.588+0000 7f93d3a60b80 20 client.25996 trim_cache size 0 max 16384 2023-07-14T11:24:16.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.607+0000 7f93d3a60b80 1 Processor -- start 2023-07-14T11:24:16.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93d3a60b80 1 -- start start 2023-07-14T11:24:16.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf2a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf3b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf1b970 con 0x56296bf252d0 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf30160 con 0x56296bf25be0 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf224c0 con 0x56296bf30ad0 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93bdf9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf2a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93be79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf3b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93bdf9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf2a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49662/0 (socket says 172.21.15.161:49662) 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93bdf9b700 1 -- 172.21.15.161:0/4092173977 learned_addr learned my addr 172.21.15.161:0/4092173977 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.608+0000 7f93bd79a700 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93be79c700 1 -- 172.21.15.161:0/4092173977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 msgr2=0x56296bf315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93be79c700 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93be79c700 1 -- 172.21.15.161:0/4092173977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 msgr2=0x56296bf2a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93be79c700 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf2a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93be79c700 1 -- 172.21.15.161:0/4092173977 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296bccc250 con 0x56296bf30ad0 2023-07-14T11:24:16.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93be79c700 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf3b660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009d30 tx=0x7f93b0025690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a337c3c8b750e611 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93acff9700 1 -- 172.21.15.161:0/4092173977 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00257e0 con 0x56296bf30ad0 2023-07-14T11:24:16.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93acff9700 1 -- 172.21.15.161:0/4092173977 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93b000f040 con 0x56296bf30ad0 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.609+0000 7f93acff9700 1 -- 172.21.15.161:0/4092173977 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00175b0 con 0x56296bf30ad0 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 -- 172.21.15.161:0/4092173977 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 msgr2=0x56296bf3b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf3b660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009d30 tx=0x7f93b0025690 comp rx=0 tx=0).stop 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 -- 172.21.15.161:0/4092173977 shutdown_connections 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf2a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf3b660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/4092173977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 -- 172.21.15.161:0/4092173977 >> 172.21.15.161:0/4092173977 conn(0x56296bf1bda0 msgr2=0x56296bf30ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 -- 172.21.15.161:0/4092173977 shutdown_connections 2023-07-14T11:24:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.610+0000 7f93d3a60b80 1 -- 172.21.15.161:0/4092173977 wait complete. 2023-07-14T11:24:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.611+0000 7f93d3a60b80 1 -- start start 2023-07-14T11:24:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.611+0000 7f93d3a60b80 1 Processor -- start 2023-07-14T11:24:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf34470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93d3a60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf364c0 con 0x56296bf252d0 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf93fb0 con 0x56296bf25be0 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93d3a60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf94170 con 0x56296bf30ad0 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93d3a60b80 10 client.0 _ll_register_callbacks cb 0x7f93b001e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93bdf9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf34470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf34ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93bd79a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf34ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34534/0 (socket says 172.21.15.161:34534) 2023-07-14T11:24:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93bdf9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf34470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49674/0 (socket says 172.21.15.161:49674) 2023-07-14T11:24:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 -- 172.21.15.161:0/2747770487 learned_addr learned my addr 172.21.15.161:0/2747770487 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 -- 172.21.15.161:0/2747770487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 msgr2=0x56296bf349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 -- 172.21.15.161:0/2747770487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 msgr2=0x56296bf34470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf34470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.612+0000 7f93be79c700 1 -- 172.21.15.161:0/2747770487 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56296bc8ec60 con 0x56296bf30ad0 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93be79c700 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf34ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f93b0038040 tx=0x7f93b0025a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d50d2b33557963 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93a17fa700 1 -- 172.21.15.161:0/2747770487 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b003e440 con 0x56296bf30ad0 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93a17fa700 1 -- 172.21.15.161:0/2747770487 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f93b003e5a0 con 0x56296bf30ad0 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93a8021e90 con 0x56296bf30ad0 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93a17fa700 1 -- 172.21.15.161:0/2747770487 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b003e760 con 0x56296bf30ad0 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93d3a60b80 10 client.26002 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56296bca42b0 con 0x56296bf30ad0 2023-07-14T11:24:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.613+0000 7f93a17fa700 1 -- 172.21.15.161:0/2747770487 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a8021e90 con 0x56296bf30ad0 2023-07-14T11:24:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93a17fa700 1 -- 172.21.15.161:0/2747770487 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f93b002d050 con 0x56296bf30ad0 2023-07-14T11:24:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93d3a60b80 10 client.26002 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93d3a60b80 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93d3a60b80 20 client.26002 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf95510 0x56296bf978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93d3a60b80 4 client.26002 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56296be33050 con 0x56296bf95510 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93bdf9b700 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf95510 0x56296bf978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93a17fa700 10 client.26002 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93bdf9b700 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf95510 0x56296bf978f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5331b7ec7dd682 server_cookie=2994b3931ce2dcb in_seq=0 out_seq=0 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.614+0000 7f93a17fa700 1 -- 172.21.15.161:0/2747770487 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56296be33050 con 0x56296bf95510 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93a17fa700 10 client.26002 handle_command_reply: tid=0 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 client.26002 shutdown 2023-07-14T11:24:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 msgr2=0x56296bf34ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf34ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f93b0038040 tx=0x7f93b0025a90 comp rx=0 tx=0).stop 2023-07-14T11:24:16.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 shutdown_connections 2023-07-14T11:24:16.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56296bf95510 0x56296bf978f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56296bf252d0 0x56296bf34470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56296bf30ad0 0x56296bf34ef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 --2- 172.21.15.161:0/2747770487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56296bf25be0 0x56296bf349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93a17fa700 0 client.26002 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 >> 172.21.15.161:0/2747770487 conn(0x56296bf1bda0 msgr2=0x56296bf2eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 shutdown_connections 2023-07-14T11:24:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 1 -- 172.21.15.161:0/2747770487 wait complete. 2023-07-14T11:24:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.615+0000 7f93d3a60b80 20 client.26002 trim_cache size 0 max 16384 2023-07-14T11:24:16.635 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:24:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:24:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:24:16.636 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93a40a12c0 msgr2=0x7f93a40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93a40a12c0 0x7f93a40a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009fd0 tx=0x7f93b0007c40 comp rx=0 tx=0).stop 2023-07-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93b8102910 msgr2=0x7f93b8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93b8102910 0x7f93b8198380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f93a80098e0 tx=0x7f93a8005740 comp rx=0 tx=0).stop 2023-07-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 shutdown_connections 2023-07-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93a40a12c0 0x7f93a40a3770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b8106fb0 0x7f93b819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b8101f70 0x7f93b8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 --2- 172.21.15.161:0/3357138539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f93b8102910 0x7f93b8198380 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 >> 172.21.15.161:0/3357138539 conn(0x7f93b80fd7e0 msgr2=0x7f93b80fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 shutdown_connections 2023-07-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.616+0000 7f93c01ff700 1 -- 172.21.15.161:0/3357138539 wait complete. 2023-07-14T11:24:16.639 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:24:16.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:24:16.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.852+0000 7f84c93ff700 1 -- 172.21.15.161:0/1167173735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 msgr2=0x7f84c4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.852+0000 7f84c93ff700 1 --2- 172.21.15.161:0/1167173735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4105eb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009990 tx=0x7f84b4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.853+0000 7f84c93ff700 1 -- 172.21.15.161:0/1167173735 shutdown_connections 2023-07-14T11:24:16.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.853+0000 7f84c93ff700 1 --2- 172.21.15.161:0/1167173735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4100480 0x7f84c410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.853+0000 7f84c93ff700 1 --2- 172.21.15.161:0/1167173735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4105eb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.853+0000 7f84c93ff700 1 --2- 172.21.15.161:0/1167173735 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84c40ffad0 0x7f84c40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.853+0000 7f84c93ff700 1 -- 172.21.15.161:0/1167173735 >> 172.21.15.161:0/1167173735 conn(0x7f84c40fb590 msgr2=0x7f84c40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:16.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.854+0000 7f84c93ff700 1 -- 172.21.15.161:0/1167173735 shutdown_connections 2023-07-14T11:24:16.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.854+0000 7f84c93ff700 1 -- 172.21.15.161:0/1167173735 wait complete. 2023-07-14T11:24:16.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.855+0000 7f84c93ff700 1 Processor -- start 2023-07-14T11:24:16.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.856+0000 7f84c93ff700 1 -- start start 2023-07-14T11:24:16.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.856+0000 7f84c93ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c40ffad0 0x7f84c4108b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.856+0000 7f84c93ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84c4100480 0x7f84c4109050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.856+0000 7f84c93ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4109590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.856+0000 7f84c93ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c41a8430 con 0x7f84c40ffad0 2023-07-14T11:24:16.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.856+0000 7f84c93ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4112df0 con 0x7f84c4105ae0 2023-07-14T11:24:16.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.856+0000 7f84c93ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4112fb0 con 0x7f84c4100480 2023-07-14T11:24:16.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.857+0000 7f84c37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.857+0000 7f84c37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48306/0 (socket says 172.21.15.161:48306) 2023-07-14T11:24:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.857+0000 7f84c37fe700 1 -- 172.21.15.161:0/3536095164 learned_addr learned my addr 172.21.15.161:0/3536095164 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.858+0000 7f84c27fc700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84c4100480 0x7f84c4109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.858+0000 7f84c2ffd700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c40ffad0 0x7f84c4108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.858+0000 7f84c37fe700 1 -- 172.21.15.161:0/3536095164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84c4100480 msgr2=0x7f84c4109050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.858+0000 7f84c37fe700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84c4100480 0x7f84c4109050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.858+0000 7f84c37fe700 1 -- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c40ffad0 msgr2=0x7f84c4108b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.858+0000 7f84c37fe700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c40ffad0 0x7f84c4108b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.858+0000 7f84c37fe700 1 -- 172.21.15.161:0/3536095164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40095f0 con 0x7f84c4105ae0 2023-07-14T11:24:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.859+0000 7f84c37fe700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4109590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00ef30 tx=0x7f84bc00c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.859+0000 7f84abfff700 1 -- 172.21.15.161:0/3536095164 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc018070 con 0x7f84c4105ae0 2023-07-14T11:24:16.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.859+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c4113230 con 0x7f84c4105ae0 2023-07-14T11:24:16.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.860+0000 7f84abfff700 1 -- 172.21.15.161:0/3536095164 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84bc00f040 con 0x7f84c4105ae0 2023-07-14T11:24:16.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.860+0000 7f84abfff700 1 -- 172.21.15.161:0/3536095164 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc004020 con 0x7f84c4105ae0 2023-07-14T11:24:16.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.860+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c4113790 con 0x7f84c4105ae0 2023-07-14T11:24:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.862+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b0005320 con 0x7f84c4105ae0 2023-07-14T11:24:16.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.865+0000 7f84abfff700 1 -- 172.21.15.161:0/3536095164 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f84bc004180 con 0x7f84c4105ae0 2023-07-14T11:24:16.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.867+0000 7f84abfff700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84ac0a1400 0x7f84ac0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:16.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.867+0000 7f84abfff700 1 -- 172.21.15.161:0/3536095164 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f84bc0ccae0 con 0x7f84c4105ae0 2023-07-14T11:24:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.868+0000 7f84c2ffd700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84ac0a1400 0x7f84ac0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:16.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.869+0000 7f84c2ffd700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84ac0a1400 0x7f84ac0a38b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84b400b2e0 tx=0x7f84b4005fb0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:16.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:16.873+0000 7f84abfff700 1 -- 172.21.15.161:0/3536095164 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f84bc095bf0 con 0x7f84c4105ae0 2023-07-14T11:24:17.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.152+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84b0005190 con 0x7f84c4105ae0 2023-07-14T11:24:17.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.153+0000 7f84abfff700 1 -- 172.21.15.161:0/3536095164 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f84bc099210 con 0x7f84c4105ae0 2023-07-14T11:24:17.156 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:24:17.156 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:23:41.117589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:24:17.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84ac0a1400 msgr2=0x7f84ac0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84ac0a1400 0x7f84ac0a38b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84b400b2e0 tx=0x7f84b4005fb0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 msgr2=0x7f84c4109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4109590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00ef30 tx=0x7f84bc00c450 comp rx=0 tx=0).stop 2023-07-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 shutdown_connections 2023-07-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84ac0a1400 0x7f84ac0a38b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c40ffad0 0x7f84c4108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84c4105ae0 0x7f84c4109590 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 --2- 172.21.15.161:0/3536095164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f84c4100480 0x7f84c4109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 >> 172.21.15.161:0/3536095164 conn(0x7f84c40fb590 msgr2=0x7f84c40fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 shutdown_connections 2023-07-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.156+0000 7f84c93ff700 1 -- 172.21.15.161:0/3536095164 wait complete. 2023-07-14T11:24:17.161 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2023-07-14T11:24:17.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 124661 11.70 MB/sec warmup 59 sec latency 33.089 ms 2023-07-14T11:24:17.177 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:24:17.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 -- 172.21.15.161:0/1395993029 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 msgr2=0x7f26f8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/1395993029 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f8072c80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009990 tx=0x7f26e8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 -- 172.21.15.161:0/1395993029 shutdown_connections 2023-07-14T11:24:17.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/1395993029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26f8110af0 0x7f26f8112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/1395993029 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f8072c80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/1395993029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f80731c0 0x7f26f81105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 -- 172.21.15.161:0/1395993029 >> 172.21.15.161:0/1395993029 conn(0x7f26f80fb5e0 msgr2=0x7f26f80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 -- 172.21.15.161:0/1395993029 shutdown_connections 2023-07-14T11:24:17.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.398+0000 7f26fe9ff700 1 -- 172.21.15.161:0/1395993029 wait complete. 2023-07-14T11:24:17.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.399+0000 7f26fe9ff700 1 Processor -- start 2023-07-14T11:24:17.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.399+0000 7f26fe9ff700 1 -- start start 2023-07-14T11:24:17.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fe9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fe9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26f80731c0 0x7f26f819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fe9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f8110af0 0x7f26f81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fe9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81a1220 con 0x7f26f80731c0 2023-07-14T11:24:17.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fe9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81a1390 con 0x7f26f8110af0 2023-07-14T11:24:17.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fe9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81a1550 con 0x7f26f80728b0 2023-07-14T11:24:17.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26f80731c0 0x7f26f819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fcf9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f8110af0 0x7f26f81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f819c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34568/0 (socket says 172.21.15.161:34568) 2023-07-14T11:24:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 -- 172.21.15.161:0/385052108 learned_addr learned my addr 172.21.15.161:0/385052108 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 -- 172.21.15.161:0/385052108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f8110af0 msgr2=0x7f26f81a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f8110af0 0x7f26f81a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 -- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26f80731c0 msgr2=0x7f26f819c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26f80731c0 0x7f26f819c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f7fff700 1 -- 172.21.15.161:0/385052108 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec009520 con 0x7f26f80728b0 2023-07-14T11:24:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26f77fe700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26f80731c0 0x7f26f819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.400+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f8110af0 0x7f26f81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.401+0000 7f26f7fff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f819c240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f26e8005910 tx=0x7f26e8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.401+0000 7f26f57fa700 1 -- 172.21.15.161:0/385052108 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e801d070 con 0x7f26f80728b0 2023-07-14T11:24:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.401+0000 7f26f57fa700 1 -- 172.21.15.161:0/385052108 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e8022470 con 0x7f26f80728b0 2023-07-14T11:24:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.401+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f0009c40 con 0x7f26f80728b0 2023-07-14T11:24:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.401+0000 7f26f57fa700 1 -- 172.21.15.161:0/385052108 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8017650 con 0x7f26f80728b0 2023-07-14T11:24:17.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.401+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e80095f0 con 0x7f26f80728b0 2023-07-14T11:24:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.405+0000 7f26f57fa700 1 -- 172.21.15.161:0/385052108 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f26e80225e0 con 0x7f26f80728b0 2023-07-14T11:24:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.405+0000 7f27123d4b80 1 Processor -- start 2023-07-14T11:24:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f27123d4b80 1 -- start start 2023-07-14T11:24:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e16b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e176590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e178eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f27123d4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e071cb0 con 0x56069e16b2d0 2023-07-14T11:24:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f27123d4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e0c4a40 con 0x56069e16bbe0 2023-07-14T11:24:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f27123d4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e0bded0 con 0x56069e176ad0 2023-07-14T11:24:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e176590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e176590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48336/0 (socket says 172.21.15.161:48336) 2023-07-14T11:24:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f26f7fff700 1 -- 172.21.15.161:0/3495034722 learned_addr learned my addr 172.21.15.161:0/3495034722 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f26f57fa700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26e00a1340 0x7f26e00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f26f77fe700 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e178eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.406+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e16b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f57fa700 1 -- 172.21.15.161:0/385052108 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f26e80ce350 con 0x7f26f80728b0 2023-07-14T11:24:17.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f7fff700 1 -- 172.21.15.161:0/3495034722 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 msgr2=0x56069e178eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f7fff700 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e178eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f7fff700 1 -- 172.21.15.161:0/3495034722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 msgr2=0x56069e16b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f7fff700 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e16b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f7fff700 1 -- 172.21.15.161:0/3495034722 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56069deebde0 con 0x56069e16bbe0 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f77fe700 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e178eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26e00a1340 0x7f26e00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26f7fff700 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e176590 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26e8097620 tx=0x7f26e8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc3b369d5647d2b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26de7fc700 1 -- 172.21.15.161:0/3495034722 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e809aef0 con 0x56069e16bbe0 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26de7fc700 1 -- 172.21.15.161:0/3495034722 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e809ac60 con 0x56069e16bbe0 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.407+0000 7f26de7fc700 1 -- 172.21.15.161:0/3495034722 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8096f10 con 0x56069e16bbe0 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 -- 172.21.15.161:0/3495034722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 msgr2=0x56069e176590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e176590 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26e8097620 tx=0x7f26e8009310 comp rx=0 tx=0).stop 2023-07-14T11:24:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26e00a1340 0x7f26e00a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009fa0 tx=0x7f26f0009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 -- 172.21.15.161:0/3495034722 shutdown_connections 2023-07-14T11:24:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e16b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e178eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3495034722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e176590 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 -- 172.21.15.161:0/3495034722 >> 172.21.15.161:0/3495034722 conn(0x56069e161da0 msgr2=0x56069e1676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 -- 172.21.15.161:0/3495034722 shutdown_connections 2023-07-14T11:24:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.408+0000 7f27123d4b80 1 -- 172.21.15.161:0/3495034722 wait complete. 2023-07-14T11:24:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.409+0000 7f27123d4b80 1 -- start start 2023-07-14T11:24:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.409+0000 7f27123d4b80 1 Processor -- start 2023-07-14T11:24:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.409+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e17aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.409+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.409+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e175aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.409+0000 7f27123d4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e05dac0 con 0x56069e16b2d0 2023-07-14T11:24:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.409+0000 7f27123d4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e05d7c0 con 0x56069e176ad0 2023-07-14T11:24:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f27123d4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069df41920 con 0x56069e16bbe0 2023-07-14T11:24:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f27123d4b80 10 client.0 _ll_register_callbacks cb 0x56069e05b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e17aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26fcf9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e175aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34586/0 (socket says 172.21.15.161:34586) 2023-07-14T11:24:17.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 -- 172.21.15.161:0/1382088626 learned_addr learned my addr 172.21.15.161:0/1382088626 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 -- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 msgr2=0x56069e175aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e175aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 -- 172.21.15.161:0/1382088626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 msgr2=0x56069e17aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e17aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 -- 172.21.15.161:0/1382088626 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec009e20 con 0x56069e16bbe0 2023-07-14T11:24:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e175aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:17.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26f77fe700 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17afb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26ec012e20 tx=0x7f26ec00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce88beb5ad253e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.410+0000 7f26d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:17.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.411+0000 7f26d77fe700 1 -- 172.21.15.161:0/1382088626 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec017070 con 0x56069e16bbe0 2023-07-14T11:24:17.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.411+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56069e0174f0 con 0x56069e16bbe0 2023-07-14T11:24:17.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.411+0000 7f26d77fe700 1 -- 172.21.15.161:0/1382088626 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f26ec00fb80 con 0x56069e16bbe0 2023-07-14T11:24:17.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.411+0000 7f26d77fe700 1 -- 172.21.15.161:0/1382088626 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26ec00fd40 con 0x56069e16bbe0 2023-07-14T11:24:17.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.411+0000 7f27123d4b80 10 client.26020 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.411+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f26e80095f0 con 0x56069e16bbe0 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.411+0000 7f26d77fe700 1 -- 172.21.15.161:0/1382088626 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56069e0174f0 con 0x56069e16bbe0 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f26d77fe700 1 -- 172.21.15.161:0/1382088626 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f26ec003be0 con 0x56069e16bbe0 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f27123d4b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f27123d4b80 10 client.26020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f27123d4b80 20 client.26020 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f27123d4b80 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e185fc0 0x56069e1883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f27123d4b80 4 client.26020 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56069e0b6eb0 con 0x56069e185fc0 2023-07-14T11:24:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.412+0000 7f26f7fff700 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e185fc0 0x56069e1883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.413+0000 7f26f7fff700 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e185fc0 0x56069e1883d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5207b1d5a6623f3 server_cookie=6a8945220a10175a in_seq=0 out_seq=0 2023-07-14T11:24:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.413+0000 7f26d77fe700 10 client.26020 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.413+0000 7f26d77fe700 1 -- 172.21.15.161:0/1382088626 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56069e0b6eb0 con 0x56069e185fc0 2023-07-14T11:24:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.413+0000 7f26d77fe700 10 client.26020 handle_command_reply: tid=0 2023-07-14T11:24:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.413+0000 7f27123d4b80 1 client.26020 shutdown 2023-07-14T11:24:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 msgr2=0x56069e17afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17afb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26ec012e20 tx=0x7f26ec00f6a0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 shutdown_connections 2023-07-14T11:24:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e185fc0 0x56069e1883d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e17aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e175aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 --2- 172.21.15.161:0/1382088626 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17afb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f26d77fe700 0 client.26020 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:17.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 >> 172.21.15.161:0/1382088626 conn(0x56069e161da0 msgr2=0x56069e167290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 shutdown_connections 2023-07-14T11:24:17.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 1 -- 172.21.15.161:0/1382088626 wait complete. 2023-07-14T11:24:17.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.414+0000 7f27123d4b80 20 client.26020 trim_cache size 0 max 16384 2023-07-14T11:24:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 Processor -- start 2023-07-14T11:24:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 -- start start 2023-07-14T11:24:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e175240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e17a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e17ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e163ec0 con 0x56069e16b2d0 2023-07-14T11:24:17.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e16ce30 con 0x56069e16bbe0 2023-07-14T11:24:17.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.428+0000 7f27123d4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e1772d0 con 0x56069e176ad0 2023-07-14T11:24:17.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26f77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e17a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e175240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e17ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26f77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e17a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48356/0 (socket says 172.21.15.161:48356) 2023-07-14T11:24:17.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26f77fe700 1 -- 172.21.15.161:0/3710747787 learned_addr learned my addr 172.21.15.161:0/3710747787 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e17ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34598/0 (socket says 172.21.15.161:34598) 2023-07-14T11:24:17.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 -- 172.21.15.161:0/3710747787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 msgr2=0x56069e17a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/3710747787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e17a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 -- 172.21.15.161:0/3710747787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 msgr2=0x56069e175240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/3710747787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e175240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 -- 172.21.15.161:0/3710747787 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56069ded4c60 con 0x56069e176ad0 2023-07-14T11:24:17.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/3710747787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e17ac70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009fd0 tx=0x7f26f0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7d8eb4fa5399ed server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26deffd700 1 -- 172.21.15.161:0/3710747787 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0003c10 con 0x56069e176ad0 2023-07-14T11:24:17.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.429+0000 7f26deffd700 1 -- 172.21.15.161:0/3710747787 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f000f040 con 0x56069e176ad0 2023-07-14T11:24:17.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f26deffd700 1 -- 172.21.15.161:0/3710747787 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0004690 con 0x56069e176ad0 2023-07-14T11:24:17.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 -- 172.21.15.161:0/3710747787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 msgr2=0x56069e17ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3710747787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e17ac70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009fd0 tx=0x7f26f0004b40 comp rx=0 tx=0).stop 2023-07-14T11:24:17.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 -- 172.21.15.161:0/3710747787 shutdown_connections 2023-07-14T11:24:17.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3710747787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e175240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3710747787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e176ad0 0x56069e17ac70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 --2- 172.21.15.161:0/3710747787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e16bbe0 0x56069e17a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 -- 172.21.15.161:0/3710747787 >> 172.21.15.161:0/3710747787 conn(0x56069e161da0 msgr2=0x56069e16dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 -- 172.21.15.161:0/3710747787 shutdown_connections 2023-07-14T11:24:17.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.430+0000 7f27123d4b80 1 -- 172.21.15.161:0/3710747787 wait complete. 2023-07-14T11:24:17.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 -- start start 2023-07-14T11:24:17.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 Processor -- start 2023-07-14T11:24:17.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e182450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e17eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e17f220 con 0x56069e16b2d0 2023-07-14T11:24:17.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e17f390 con 0x56069e176ad0 2023-07-14T11:24:17.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069e17f550 con 0x56069e16bbe0 2023-07-14T11:24:17.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.431+0000 7f27123d4b80 10 client.0 _ll_register_callbacks cb 0x7f26ec012900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:17.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26f77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e17eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e17eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48364/0 (socket says 172.21.15.161:48364) 2023-07-14T11:24:17.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 -- 172.21.15.161:0/2042627698 learned_addr learned my addr 172.21.15.161:0/2042627698 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26f77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34608/0 (socket says 172.21.15.161:34608) 2023-07-14T11:24:17.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26f7fff700 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e182450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 -- 172.21.15.161:0/2042627698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 msgr2=0x56069e17e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 -- 172.21.15.161:0/2042627698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 msgr2=0x56069e182450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e182450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 -- 172.21.15.161:0/2042627698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f000e800 con 0x56069e176ad0 2023-07-14T11:24:17.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26fcf9c700 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e17eca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009960 tx=0x7f26f0016f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636d68c73a3fccc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.432+0000 7f26d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:17.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.433+0000 7f26d77fe700 1 -- 172.21.15.161:0/2042627698 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0004270 con 0x56069e176ad0 2023-07-14T11:24:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.433+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26e809ac60 con 0x56069e176ad0 2023-07-14T11:24:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.433+0000 7f26d77fe700 1 -- 172.21.15.161:0/2042627698 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f26f001c040 con 0x56069e176ad0 2023-07-14T11:24:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.433+0000 7f26d77fe700 1 -- 172.21.15.161:0/2042627698 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f0003fb0 con 0x56069e176ad0 2023-07-14T11:24:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.433+0000 7f27123d4b80 10 client.26033 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.433+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56069df12250 con 0x56069e176ad0 2023-07-14T11:24:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.433+0000 7f26d77fe700 1 -- 172.21.15.161:0/2042627698 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e809ac60 con 0x56069e176ad0 2023-07-14T11:24:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f26d77fe700 1 -- 172.21.15.161:0/2042627698 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f26f00239c0 con 0x56069e176ad0 2023-07-14T11:24:17.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f27123d4b80 10 client.26033 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:17.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f27123d4b80 10 client.26033 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:24:17.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f27123d4b80 20 client.26033 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:17.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f27123d4b80 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e1db3a0 0x56069e1dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f27123d4b80 4 client.26033 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:24:17.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56069e079050 con 0x56069e1db3a0 2023-07-14T11:24:17.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f26f7fff700 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e1db3a0 0x56069e1dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f26d77fe700 10 client.26033 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:17.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f26f7fff700 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e1db3a0 0x56069e1dd7b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c49f9f8484c843fa server_cookie=366775e8ab3f0b75 in_seq=0 out_seq=0 2023-07-14T11:24:17.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f26d77fe700 1 -- 172.21.15.161:0/2042627698 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56069e079050 con 0x56069e1db3a0 2023-07-14T11:24:17.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f26d77fe700 10 client.26033 handle_command_reply: tid=0 2023-07-14T11:24:17.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.434+0000 7f27123d4b80 1 client.26033 shutdown 2023-07-14T11:24:17.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 msgr2=0x56069e17eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e17eca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009960 tx=0x7f26f0016f00 comp rx=0 tx=0).stop 2023-07-14T11:24:17.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 shutdown_connections 2023-07-14T11:24:17.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x56069e1db3a0 0x56069e1dd7b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56069e16b2d0 0x56069e182450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56069e176ad0 0x56069e17eca0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f26d77fe700 0 client.26033 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:17.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 --2- 172.21.15.161:0/2042627698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56069e16bbe0 0x56069e17e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 >> 172.21.15.161:0/2042627698 conn(0x56069e161da0 msgr2=0x56069e17b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 shutdown_connections 2023-07-14T11:24:17.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 1 -- 172.21.15.161:0/2042627698 wait complete. 2023-07-14T11:24:17.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.435+0000 7f27123d4b80 20 client.26033 trim_cache size 0 max 16384 2023-07-14T11:24:17.449 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:24:17.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26e00a1340 msgr2=0x7f26e00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26e00a1340 0x7f26e00a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009fa0 tx=0x7f26f0009040 comp rx=0 tx=0).stop 2023-07-14T11:24:17.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 msgr2=0x7f26f819c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f819c240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f26e8005910 tx=0x7f26e8005720 comp rx=0 tx=0).stop 2023-07-14T11:24:17.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 shutdown_connections 2023-07-14T11:24:17.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f26e00a1340 0x7f26e00a37f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26f8110af0 0x7f26f81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f26f80728b0 0x7f26f819c240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 --2- 172.21.15.161:0/385052108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26f80731c0 0x7f26f819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 >> 172.21.15.161:0/385052108 conn(0x7f26f80fb5e0 msgr2=0x7f26f80fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 shutdown_connections 2023-07-14T11:24:17.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.436+0000 7f26fe9ff700 1 -- 172.21.15.161:0/385052108 wait complete. 2023-07-14T11:24:17.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:24:17.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 -- 172.21.15.161:0/1280459131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc100490 msgr2=0x7fb5dc10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/1280459131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc100490 0x7fb5dc10c920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009f90 tx=0x7fb5d400b330 comp rx=0 tx=0).stop 2023-07-14T11:24:17.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 -- 172.21.15.161:0/1280459131 shutdown_connections 2023-07-14T11:24:17.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/1280459131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc100490 0x7fb5dc10c920 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/1280459131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5dc105af0 0x7fb5dc105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/1280459131 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5dc0ffae0 0x7fb5dc0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 -- 172.21.15.161:0/1280459131 >> 172.21.15.161:0/1280459131 conn(0x7fb5dc0fb5e0 msgr2=0x7fb5dc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 -- 172.21.15.161:0/1280459131 shutdown_connections 2023-07-14T11:24:17.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.674+0000 7fb5e13ff700 1 -- 172.21.15.161:0/1280459131 wait complete. 2023-07-14T11:24:17.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.675+0000 7fb5e13ff700 1 Processor -- start 2023-07-14T11:24:17.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5e13ff700 1 -- start start 2023-07-14T11:24:17.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5e13ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc0ffae0 0x7fb5dc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5e13ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5dc100490 0x7fb5dc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5e13ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5dc105af0 0x7fb5dc19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5e13ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc19cf10 con 0x7fb5dc0ffae0 2023-07-14T11:24:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5e13ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc19d080 con 0x7fb5dc100490 2023-07-14T11:24:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5e13ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc19d240 con 0x7fb5dc105af0 2023-07-14T11:24:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5daffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc0ffae0 0x7fb5dc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5da7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5dc100490 0x7fb5dc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5daffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc0ffae0 0x7fb5dc197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49784/0 (socket says 172.21.15.161:49784) 2023-07-14T11:24:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5daffd700 1 -- 172.21.15.161:0/2970647562 learned_addr learned my addr 172.21.15.161:0/2970647562 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.676+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5dc105af0 0x7fb5dc19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5da7fc700 1 -- 172.21.15.161:0/2970647562 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5dc105af0 msgr2=0x7fb5dc19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5dc105af0 0x7fb5dc19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5da7fc700 1 -- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc0ffae0 msgr2=0x7fb5dc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc0ffae0 0x7fb5dc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5da7fc700 1 -- 172.21.15.161:0/2970647562 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc0095f0 con 0x7fb5dc100490 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5daffd700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc0ffae0 0x7fb5dc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5dc100490 0x7fb5dc198470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000e9f0 tx=0x7fb5d000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5c3fff700 1 -- 172.21.15.161:0/2970647562 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d000c9c0 con 0x7fb5dc100490 2023-07-14T11:24:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d4009c40 con 0x7fb5dc100490 2023-07-14T11:24:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5c3fff700 1 -- 172.21.15.161:0/2970647562 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5d0010430 con 0x7fb5dc100490 2023-07-14T11:24:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5c3fff700 1 -- 172.21.15.161:0/2970647562 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0018680 con 0x7fb5dc100490 2023-07-14T11:24:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.677+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc1a4b80 con 0x7fb5dc100490 2023-07-14T11:24:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.681+0000 7fb5c3fff700 1 -- 172.21.15.161:0/2970647562 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb5d0010860 con 0x7fb5dc100490 2023-07-14T11:24:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 Processor -- start 2023-07-14T11:24:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 -- start start 2023-07-14T11:24:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025b2d0 0x56109025b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025bbe0 0x561090266590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090268eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561090161cb0 con 0x56109025bbe0 2023-07-14T11:24:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610901b4a40 con 0x561090266ad0 2023-07-14T11:24:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610901aded0 con 0x56109025b2d0 2023-07-14T11:24:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5c3fff700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb5c40a12c0 0x7fb5c40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.682+0000 7fb5da7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090268eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5daffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025bbe0 0x561090266590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090268eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48400/0 (socket says 172.21.15.161:48400) 2023-07-14T11:24:17.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3729171066 learned_addr learned my addr 172.21.15.161:0/3729171066 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025b2d0 0x56109025b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5c3fff700 1 -- 172.21.15.161:0/2970647562 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fb5d0014070 con 0x7fb5dc100490 2023-07-14T11:24:17.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3729171066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025b2d0 msgr2=0x56109025b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025b2d0 0x56109025b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3729171066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025bbe0 msgr2=0x561090266590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025bbe0 0x561090266590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3729171066 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56108ffdbde0 con 0x561090266ad0 2023-07-14T11:24:17.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb5c40a12c0 0x7fb5c40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090268eb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0019000 tx=0x7fb5d0099d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a976fb86df2743b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5c17fa700 1 -- 172.21.15.161:0/3729171066 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00a2340 con 0x561090266ad0 2023-07-14T11:24:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5c17fa700 1 -- 172.21.15.161:0/3729171066 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5d0099ef0 con 0x561090266ad0 2023-07-14T11:24:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.683+0000 7fb5c17fa700 1 -- 172.21.15.161:0/3729171066 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0099640 con 0x561090266ad0 2023-07-14T11:24:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb5c40a12c0 0x7fb5c40a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40060b0 tx=0x7fb5d4009700 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3729171066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 msgr2=0x561090268eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090268eb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0019000 tx=0x7fb5d0099d70 comp rx=0 tx=0).stop 2023-07-14T11:24:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3729171066 shutdown_connections 2023-07-14T11:24:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090268eb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025b2d0 0x56109025b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3729171066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025bbe0 0x561090266590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3729171066 >> 172.21.15.161:0/3729171066 conn(0x561090251da0 msgr2=0x5610902576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3729171066 shutdown_connections 2023-07-14T11:24:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.684+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3729171066 wait complete. 2023-07-14T11:24:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.685+0000 7fb5f4dd5b80 1 -- start start 2023-07-14T11:24:17.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.685+0000 7fb5f4dd5b80 1 Processor -- start 2023-07-14T11:24:17.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x561090266520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56109025bbe0 0x5610902637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561090266ad0 0x561090263ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109014dac0 con 0x56109025b2d0 2023-07-14T11:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109014d7c0 con 0x56109025bbe0 2023-07-14T11:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561090031920 con 0x561090266ad0 2023-07-14T11:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5f4dd5b80 10 client.0 _ll_register_callbacks cb 0x56109014b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5da7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56109025bbe0 0x5610902637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5da7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56109025bbe0 0x5610902637a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48402/0 (socket says 172.21.15.161:48402) 2023-07-14T11:24:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5da7fc700 1 -- 172.21.15.161:0/1194190460 learned_addr learned my addr 172.21.15.161:0/1194190460 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5daffd700 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x561090266520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.686+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561090266ad0 0x561090263ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5da7fc700 1 -- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561090266ad0 msgr2=0x561090263ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561090266ad0 0x561090263ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5da7fc700 1 -- 172.21.15.161:0/1194190460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 msgr2=0x561090266520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x561090266520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5da7fc700 1 -- 172.21.15.161:0/1194190460 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc1a4b80 con 0x56109025bbe0 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56109025bbe0 0x5610902637a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000e9c0 tx=0x7fb5d009e0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1655ea346be1afb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5beff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5beff5700 1 -- 172.21.15.161:0/1194190460 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00a2340 con 0x56109025bbe0 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610901074f0 con 0x56109025bbe0 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5beff5700 1 -- 172.21.15.161:0/1194190460 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fb5d00b25e0 con 0x56109025bbe0 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5beff5700 1 -- 172.21.15.161:0/1194190460 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5d0096230 con 0x56109025bbe0 2023-07-14T11:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5f4dd5b80 10 client.26045 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56108ffc4c60 con 0x56109025bbe0 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.687+0000 7fb5beff5700 1 -- 172.21.15.161:0/1194190460 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610901074f0 con 0x56109025bbe0 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5beff5700 1 -- 172.21.15.161:0/1194190460 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb5d00a1eb0 con 0x56109025bbe0 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5f4dd5b80 10 client.26045 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5f4dd5b80 10 client.26045 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5f4dd5b80 20 client.26045 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561090275ef0 0x5610902782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5f4dd5b80 4 client.26045 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610901a6eb0 con 0x561090275ef0 2023-07-14T11:24:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.688+0000 7fb5daffd700 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561090275ef0 0x5610902782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.689+0000 7fb5daffd700 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561090275ef0 0x5610902782d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1da3b7c9589b1b4 server_cookie=303fb8968de113cb in_seq=0 out_seq=0 2023-07-14T11:24:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.689+0000 7fb5beff5700 10 client.26045 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.689+0000 7fb5beff5700 1 -- 172.21.15.161:0/1194190460 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5610901a6eb0 con 0x561090275ef0 2023-07-14T11:24:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5beff5700 10 client.26045 handle_command_reply: tid=0 2023-07-14T11:24:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 client.26045 shutdown 2023-07-14T11:24:17.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56109025bbe0 msgr2=0x5610902637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56109025bbe0 0x5610902637a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000e9c0 tx=0x7fb5d009e0a0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 shutdown_connections 2023-07-14T11:24:17.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561090275ef0 0x5610902782d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x561090266520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561090266ad0 0x561090263ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/1194190460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56109025bbe0 0x5610902637a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5beff5700 0 client.26045 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:17.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 >> 172.21.15.161:0/1194190460 conn(0x561090251da0 msgr2=0x561090257230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 shutdown_connections 2023-07-14T11:24:17.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/1194190460 wait complete. 2023-07-14T11:24:17.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.690+0000 7fb5f4dd5b80 20 client.26045 trim_cache size 0 max 16384 2023-07-14T11:24:17.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 Processor -- start 2023-07-14T11:24:17.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 -- start start 2023-07-14T11:24:17.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109025eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109025f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090262f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561090256520 con 0x56109025b2d0 2023-07-14T11:24:17.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109025c540 con 0x561090266ad0 2023-07-14T11:24:17.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561090272d80 con 0x56109025bbe0 2023-07-14T11:24:17.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5db7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090262f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5db7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090262f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48412/0 (socket says 172.21.15.161:48412) 2023-07-14T11:24:17.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5db7fe700 1 -- 172.21.15.161:0/2408540674 learned_addr learned my addr 172.21.15.161:0/2408540674 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109025f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.704+0000 7fb5daffd700 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109025eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5db7fe700 1 -- 172.21.15.161:0/2408540674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 msgr2=0x56109025f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109025f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5db7fe700 1 -- 172.21.15.161:0/2408540674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 msgr2=0x56109025eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109025eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5db7fe700 1 -- 172.21.15.161:0/2408540674 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561090002250 con 0x561090266ad0 2023-07-14T11:24:17.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090262f80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4008d70 tx=0x7fb5d4006020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a302ff5da796daf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5c1ffb700 1 -- 172.21.15.161:0/2408540674 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4025040 con 0x561090266ad0 2023-07-14T11:24:17.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5c1ffb700 1 -- 172.21.15.161:0/2408540674 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5d4003e10 con 0x561090266ad0 2023-07-14T11:24:17.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.705+0000 7fb5c1ffb700 1 -- 172.21.15.161:0/2408540674 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400e040 con 0x561090266ad0 2023-07-14T11:24:17.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/2408540674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 msgr2=0x561090262f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090262f80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4008d70 tx=0x7fb5d4006020 comp rx=0 tx=0).stop 2023-07-14T11:24:17.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/2408540674 shutdown_connections 2023-07-14T11:24:17.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109025eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x561090262f80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/2408540674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109025f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/2408540674 >> 172.21.15.161:0/2408540674 conn(0x561090251da0 msgr2=0x561090255e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/2408540674 shutdown_connections 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/2408540674 wait complete. 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.706+0000 7fb5f4dd5b80 1 -- start start 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.707+0000 7fb5f4dd5b80 1 Processor -- start 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.707+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109026d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.707+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109026bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.707+0000 7fb5f4dd5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x56109026c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.707+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109026c780 con 0x56109025b2d0 2023-07-14T11:24:17.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.707+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109026c8f0 con 0x561090266ad0 2023-07-14T11:24:17.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.707+0000 7fb5f4dd5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610902c9d60 con 0x56109025bbe0 2023-07-14T11:24:17.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5f4dd5b80 10 client.0 _ll_register_callbacks cb 0x7fb5cc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:17.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109026bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109026bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34646/0 (socket says 172.21.15.161:34646) 2023-07-14T11:24:17.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3095724954 learned_addr learned my addr 172.21.15.161:0/3095724954 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5db7fe700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x56109026c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5daffd700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109026d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3095724954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 msgr2=0x56109026c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x56109026c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3095724954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 msgr2=0x56109026d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109026d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 -- 172.21.15.161:0/3095724954 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc10d220 con 0x56109025bbe0 2023-07-14T11:24:17.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5daffd700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109026d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:17.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5da7fc700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109026bad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0018ee0 tx=0x7fb5d0099df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c02769c7b7b02a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:17.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5beff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:17.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.708+0000 7fb5beff5700 1 -- 172.21.15.161:0/3095724954 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0096230 con 0x56109025bbe0 2023-07-14T11:24:17.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5beff5700 1 -- 172.21.15.161:0/3095724954 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fb5d00a2340 con 0x56109025bbe0 2023-07-14T11:24:17.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5d0099ef0 con 0x56109025bbe0 2023-07-14T11:24:17.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5beff5700 1 -- 172.21.15.161:0/3095724954 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5d000cd30 con 0x56109025bbe0 2023-07-14T11:24:17.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 10 client.26044 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56108ffda2b0 con 0x56109025bbe0 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5beff5700 1 -- 172.21.15.161:0/3095724954 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5d0099ef0 con 0x56109025bbe0 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5beff5700 1 -- 172.21.15.161:0/3095724954 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb5d0099050 con 0x56109025bbe0 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 10 client.26044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 20 client.26044 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5610902caf80 0x5610902cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:17.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 4 client.26044 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:24:17.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.709+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561090169050 con 0x5610902caf80 2023-07-14T11:24:17.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.710+0000 7fb5daffd700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5610902caf80 0x5610902cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:17.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.710+0000 7fb5daffd700 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5610902caf80 0x5610902cd360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae1800431b151c22 server_cookie=decac917ab88ddf5 in_seq=0 out_seq=0 2023-07-14T11:24:17.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.710+0000 7fb5beff5700 10 client.26044 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:17.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.777+0000 7fb5beff5700 1 -- 172.21.15.161:0/3095724954 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561090169050 con 0x5610902caf80 2023-07-14T11:24:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.777+0000 7fb5beff5700 10 client.26044 handle_command_reply: tid=0 2023-07-14T11:24:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.777+0000 7fb5f4dd5b80 1 client.26044 shutdown 2023-07-14T11:24:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.777+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 msgr2=0x56109026bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.777+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109026bad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0018ee0 tx=0x7fb5d0099df0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.777+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 shutdown_connections 2023-07-14T11:24:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.777+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5610902caf80 0x5610902cd360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56109025b2d0 0x56109026d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561090266ad0 0x56109026c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5f4dd5b80 1 --2- 172.21.15.161:0/3095724954 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56109025bbe0 0x56109026bad0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 >> 172.21.15.161:0/3095724954 conn(0x561090251da0 msgr2=0x56109025eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5beff5700 0 client.26044 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:17.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 shutdown_connections 2023-07-14T11:24:17.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5f4dd5b80 1 -- 172.21.15.161:0/3095724954 wait complete. 2023-07-14T11:24:17.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5f4dd5b80 20 client.26044 trim_cache size 0 max 16384 2023-07-14T11:24:17.783 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:24:17.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb5c40a12c0 msgr2=0x7fb5c40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb5c40a12c0 0x7fb5c40a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40060b0 tx=0x7fb5d4009700 comp rx=0 tx=0).stop 2023-07-14T11:24:17.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5dc100490 msgr2=0x7fb5dc198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:17.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.778+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5dc100490 0x7fb5dc198470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000e9f0 tx=0x7fb5d000ed00 comp rx=0 tx=0).stop 2023-07-14T11:24:17.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 shutdown_connections 2023-07-14T11:24:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb5c40a12c0 0x7fb5c40a3770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5dc0ffae0 0x7fb5dc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb5dc105af0 0x7fb5dc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 --2- 172.21.15.161:0/2970647562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5dc100490 0x7fb5dc198470 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 >> 172.21.15.161:0/2970647562 conn(0x7fb5dc0fb5e0 msgr2=0x7fb5dc10ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 shutdown_connections 2023-07-14T11:24:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:17.779+0000 7fb5e13ff700 1 -- 172.21.15.161:0/2970647562 wait complete. 2023-07-14T11:24:17.800 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:24:18.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 -- 172.21.15.161:0/1691740232 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f76680ff9d0 msgr2=0x7f76680ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 --2- 172.21.15.161:0/1691740232 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f76680ff9d0 0x7f76680ffe40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f765c009990 tx=0x7f765c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 -- 172.21.15.161:0/1691740232 shutdown_connections 2023-07-14T11:24:18.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 --2- 172.21.15.161:0/1691740232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76681059e0 0x7f7668105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 --2- 172.21.15.161:0/1691740232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7668100380 0x7f766810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 --2- 172.21.15.161:0/1691740232 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f76680ff9d0 0x7f76680ffe40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 -- 172.21.15.161:0/1691740232 >> 172.21.15.161:0/1691740232 conn(0x7f76680fb4b0 msgr2=0x7f76680fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:18.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 -- 172.21.15.161:0/1691740232 shutdown_connections 2023-07-14T11:24:18.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.015+0000 7f766d39d700 1 -- 172.21.15.161:0/1691740232 wait complete. 2023-07-14T11:24:18.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.016+0000 7f766d39d700 1 Processor -- start 2023-07-14T11:24:18.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.016+0000 7f766d39d700 1 -- start start 2023-07-14T11:24:18.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766d39d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76680ff9d0 0x7f766819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766d39d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7668100380 0x7f766819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766d39d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76681059e0 0x7f76681956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766d39d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668195d60 con 0x7f76680ff9d0 2023-07-14T11:24:18.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766d39d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668195ed0 con 0x7f76681059e0 2023-07-14T11:24:18.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766d39d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668196090 con 0x7f7668100380 2023-07-14T11:24:18.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f7667fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76680ff9d0 0x7f766819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766cb9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76681059e0 0x7f76681956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f76677fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7668100380 0x7f766819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f7667fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76680ff9d0 0x7f766819b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52454/0 (socket says 172.21.15.161:52454) 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f766cb9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76681059e0 0x7f76681956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51844/0 (socket says 172.21.15.161:51844) 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f7667fff700 1 -- 172.21.15.161:0/1201408909 learned_addr learned my addr 172.21.15.161:0/1201408909 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f76677fe700 1 -- 172.21.15.161:0/1201408909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76681059e0 msgr2=0x7f76681956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f76677fe700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76681059e0 0x7f76681956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f76677fe700 1 -- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76680ff9d0 msgr2=0x7f766819b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.017+0000 7f76677fe700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76680ff9d0 0x7f766819b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f76677fe700 1 -- 172.21.15.161:0/1201408909 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0095f0 con 0x7f7668100380 2023-07-14T11:24:18.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f766cb9c700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76681059e0 0x7f76681956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:18.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f76677fe700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7668100380 0x7f766819bbc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f765c009990 tx=0x7f765c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:18.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f76657fa700 1 -- 172.21.15.161:0/1201408909 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c01d070 con 0x7f7668100380 2023-07-14T11:24:18.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f76657fa700 1 -- 172.21.15.161:0/1201408909 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f765c005030 con 0x7f7668100380 2023-07-14T11:24:18.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7668196310 con 0x7f7668100380 2023-07-14T11:24:18.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f76657fa700 1 -- 172.21.15.161:0/1201408909 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c004990 con 0x7f7668100380 2023-07-14T11:24:18.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.018+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76681a2670 con 0x7f7668100380 2023-07-14T11:24:18.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.021+0000 7f76657fa700 1 -- 172.21.15.161:0/1201408909 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f765c005410 con 0x7f7668100380 2023-07-14T11:24:18.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.023+0000 7f7681ee3b80 1 Processor -- start 2023-07-14T11:24:18.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.023+0000 7f76657fa700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f76500a13e0 0x7f76500a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.023+0000 7f7681ee3b80 1 -- start start 2023-07-14T11:24:18.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.023+0000 7f76657fa700 1 -- 172.21.15.161:0/1201408909 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f765c0cde90 con 0x7f7668100380 2023-07-14T11:24:18.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.023+0000 7f766cb9c700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f76500a13e0 0x7f76500a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.023+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b9502520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b93f6cb0 con 0x5573b94f1e00 2023-07-14T11:24:18.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b9449a40 con 0x5573b94f14f0 2023-07-14T11:24:18.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b9442ed0 con 0x5573b9500140 2023-07-14T11:24:18.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f76677fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7667fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f766cb9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b9502520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7667fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94f18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51860/0 (socket says 172.21.15.161:51860) 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7667fff700 1 -- 172.21.15.161:0/2891298346 learned_addr learned my addr 172.21.15.161:0/2891298346 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f766cb9c700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f76500a13e0 0x7f76500a3890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f766819c380 tx=0x7f7660009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f766cb9c700 1 -- 172.21.15.161:0/2891298346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 msgr2=0x5573b94f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f766cb9c700 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94f18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f766cb9c700 1 -- 172.21.15.161:0/2891298346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 msgr2=0x5573b94ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f766cb9c700 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94ffc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f766cb9c700 1 -- 172.21.15.161:0/2891298346 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573b9270de0 con 0x5573b9500140 2023-07-14T11:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.024+0000 7f7667fff700 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.025+0000 7f766cb9c700 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b9502520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7660004600 tx=0x7f766001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1260356e8b2751f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.025+0000 7f764e7fc700 1 -- 172.21.15.161:0/2891298346 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660025070 con 0x5573b9500140 2023-07-14T11:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.025+0000 7f764e7fc700 1 -- 172.21.15.161:0/2891298346 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f766000f040 con 0x5573b9500140 2023-07-14T11:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.025+0000 7f764e7fc700 1 -- 172.21.15.161:0/2891298346 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660019460 con 0x5573b9500140 2023-07-14T11:24:18.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.025+0000 7f7681ee3b80 1 -- 172.21.15.161:0/2891298346 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 msgr2=0x5573b9502520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.025+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b9502520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7660004600 tx=0x7f766001c000 comp rx=0 tx=0).stop 2023-07-14T11:24:18.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 -- 172.21.15.161:0/2891298346 shutdown_connections 2023-07-14T11:24:18.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b9502520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/2891298346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 -- 172.21.15.161:0/2891298346 >> 172.21.15.161:0/2891298346 conn(0x5573b94e6ff0 msgr2=0x5573b94ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 -- 172.21.15.161:0/2891298346 shutdown_connections 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 -- 172.21.15.161:0/2891298346 wait complete. 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 -- start start 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.026+0000 7f7681ee3b80 1 Processor -- start 2023-07-14T11:24:18.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b93e2ac0 con 0x5573b94f1e00 2023-07-14T11:24:18.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b93e27c0 con 0x5573b94f14f0 2023-07-14T11:24:18.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b92c6920 con 0x5573b9500140 2023-07-14T11:24:18.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7681ee3b80 10 client.0 _ll_register_callbacks cb 0x5573b93e0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:18.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7667fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7667fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51874/0 (socket says 172.21.15.161:51874) 2023-07-14T11:24:18.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7667fff700 1 -- 172.21.15.161:0/4203091132 learned_addr learned my addr 172.21.15.161:0/4203091132 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:18.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f766cb9c700 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f76677fe700 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7667fff700 1 -- 172.21.15.161:0/4203091132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 msgr2=0x5573b94fd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7667fff700 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94fd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7667fff700 1 -- 172.21.15.161:0/4203091132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 msgr2=0x5573b94fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.027+0000 7f7667fff700 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f7667fff700 1 -- 172.21.15.161:0/4203091132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7658009a50 con 0x5573b94f14f0 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f7667fff700 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94ffa90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7658012cf0 tx=0x7f7658009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272672837db70ff8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f76477fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f76477fe700 1 -- 172.21.15.161:0/4203091132 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658017070 con 0x5573b94f14f0 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573b939c4f0 con 0x5573b94f14f0 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f76477fe700 1 -- 172.21.15.161:0/4203091132 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f7658004030 con 0x5573b94f14f0 2023-07-14T11:24:18.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f76477fe700 1 -- 172.21.15.161:0/4203091132 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76580122e0 con 0x5573b94f14f0 2023-07-14T11:24:18.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f7681ee3b80 10 client.26063 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:18.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.028+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f7660004880 con 0x5573b94f14f0 2023-07-14T11:24:18.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f76477fe700 1 -- 172.21.15.161:0/4203091132 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573b939c4f0 con 0x5573b94f14f0 2023-07-14T11:24:18.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f76477fe700 1 -- 172.21.15.161:0/4203091132 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7658003710 con 0x5573b94f14f0 2023-07-14T11:24:18.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f7681ee3b80 10 client.26063 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:18.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f7681ee3b80 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:18.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f7681ee3b80 20 client.26063 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:18.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b950fbd0 0x5573b9511fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f7681ee3b80 4 client.26063 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:18.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573b943beb0 con 0x5573b950fbd0 2023-07-14T11:24:18.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.029+0000 7f76677fe700 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b950fbd0 0x5573b9511fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f76477fe700 10 client.26063 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:18.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f76677fe700 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b950fbd0 0x5573b9511fe0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5bc9cc54cf5dc0 server_cookie=ca8c98cdfed8f5cd in_seq=0 out_seq=0 2023-07-14T11:24:18.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f76477fe700 1 -- 172.21.15.161:0/4203091132 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573b943beb0 con 0x5573b950fbd0 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f76477fe700 10 client.26063 handle_command_reply: tid=0 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f7681ee3b80 1 client.26063 shutdown 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 msgr2=0x5573b94ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94ffa90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7658012cf0 tx=0x7f7658009d50 comp rx=0 tx=0).stop 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 shutdown_connections 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.030+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b950fbd0 0x5573b9511fe0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f76477fe700 0 client.26063 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:18.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f14f0 0x5573b94ffa90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/4203091132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b94fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 >> 172.21.15.161:0/4203091132 conn(0x5573b94e6ff0 msgr2=0x5573b94ec520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:18.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 shutdown_connections 2023-07-14T11:24:18.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f7681ee3b80 1 -- 172.21.15.161:0/4203091132 wait complete. 2023-07-14T11:24:18.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.031+0000 7f7681ee3b80 20 client.26063 trim_cache size 0 max 16384 2023-07-14T11:24:18.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.046+0000 7f7681ee3b80 1 Processor -- start 2023-07-14T11:24:18.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7681ee3b80 1 -- start start 2023-07-14T11:24:18.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f14f0 0x5573b9506c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f1e00 0x5573b950b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b94f2980 con 0x5573b94f14f0 2023-07-14T11:24:18.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b94f31a0 con 0x5573b94f1e00 2023-07-14T11:24:18.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b94f21d0 con 0x5573b9500140 2023-07-14T11:24:18.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f76677fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f1e00 0x5573b950b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f76677fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f1e00 0x5573b950b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51888/0 (socket says 172.21.15.161:51888) 2023-07-14T11:24:18.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f7667fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f14f0 0x5573b9506c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f76677fe700 1 -- 172.21.15.161:0/3986069242 learned_addr learned my addr 172.21.15.161:0/3986069242 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:18.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f766cb9c700 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f76677fe700 1 -- 172.21.15.161:0/3986069242 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 msgr2=0x5573b94ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f76677fe700 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f76677fe700 1 -- 172.21.15.161:0/3986069242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f14f0 msgr2=0x5573b9506c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.047+0000 7f76677fe700 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f14f0 0x5573b9506c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.048+0000 7f76677fe700 1 -- 172.21.15.161:0/3986069242 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573b9259c60 con 0x5573b94f1e00 2023-07-14T11:24:18.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.048+0000 7f76677fe700 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f1e00 0x5573b950b3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f765c005cd0 tx=0x7f765c0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab72261b7881d380 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:18.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.048+0000 7f764effd700 1 -- 172.21.15.161:0/3986069242 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c09e040 con 0x5573b94f1e00 2023-07-14T11:24:18.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.048+0000 7f764effd700 1 -- 172.21.15.161:0/3986069242 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f765c0a3020 con 0x5573b94f1e00 2023-07-14T11:24:18.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.048+0000 7f764effd700 1 -- 172.21.15.161:0/3986069242 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c09a150 con 0x5573b94f1e00 2023-07-14T11:24:18.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3986069242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f1e00 msgr2=0x5573b950b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f1e00 0x5573b950b3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f765c005cd0 tx=0x7f765c0057b0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3986069242 shutdown_connections 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f14f0 0x5573b9506c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b9500140 0x5573b94ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3986069242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b94f1e00 0x5573b950b3f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3986069242 >> 172.21.15.161:0/3986069242 conn(0x5573b94e6ff0 msgr2=0x5573b950bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3986069242 shutdown_connections 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3986069242 wait complete. 2023-07-14T11:24:18.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 -- start start 2023-07-14T11:24:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.049+0000 7f7681ee3b80 1 Processor -- start 2023-07-14T11:24:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b94f14f0 0x5573b95068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b9506de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7681ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b9500140 0x5573b9504d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b9505430 con 0x5573b94f1e00 2023-07-14T11:24:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b95055a0 con 0x5573b9500140 2023-07-14T11:24:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7681ee3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573b9505760 con 0x5573b94f14f0 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7681ee3b80 10 client.0 _ll_register_callbacks cb 0x5573b93e0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7667fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b94f14f0 0x5573b95068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7667fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b94f14f0 0x5573b95068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38014/0 (socket says 172.21.15.161:38014) 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f7667fff700 1 -- 172.21.15.161:0/3396166645 learned_addr learned my addr 172.21.15.161:0/3396166645 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f766cb9c700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b9500140 0x5573b9504d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.050+0000 7f76677fe700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b9506de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7667fff700 1 -- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b9500140 msgr2=0x5573b9504d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7667fff700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b9500140 0x5573b9504d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7667fff700 1 -- 172.21.15.161:0/3396166645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 msgr2=0x5573b9506de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7667fff700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b9506de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7667fff700 1 -- 172.21.15.161:0/3396166645 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76681a2670 con 0x5573b94f14f0 2023-07-14T11:24:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f766cb9c700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b9500140 0x5573b9504d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7667fff700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b94f14f0 0x5573b95068a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7658005950 tx=0x7f76580038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f74028ff4d8cf8a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f76477fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f76477fe700 1 -- 172.21.15.161:0/3396166645 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658017070 con 0x5573b94f14f0 2023-07-14T11:24:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f766000f040 con 0x5573b94f14f0 2023-07-14T11:24:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f76477fe700 1 -- 172.21.15.161:0/3396166645 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f7658003d90 con 0x5573b94f14f0 2023-07-14T11:24:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f76477fe700 1 -- 172.21.15.161:0/3396166645 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7658009ee0 con 0x5573b94f14f0 2023-07-14T11:24:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7681ee3b80 10 client.26062 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.051+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5573b9297250 con 0x5573b94f14f0 2023-07-14T11:24:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f76477fe700 1 -- 172.21.15.161:0/3396166645 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f766000f040 con 0x5573b94f14f0 2023-07-14T11:24:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f76477fe700 1 -- 172.21.15.161:0/3396166645 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f765801c460 con 0x5573b94f14f0 2023-07-14T11:24:18.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f7681ee3b80 10 client.26062 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:18.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f7681ee3b80 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:18.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f7681ee3b80 20 client.26062 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:18.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b9564c00 0x5573b9567010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:18.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f7681ee3b80 4 client.26062 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:24:18.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573b93fe050 con 0x5573b9564c00 2023-07-14T11:24:18.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.052+0000 7f76677fe700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b9564c00 0x5573b9567010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:18.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f76477fe700 10 client.26062 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f76677fe700 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b9564c00 0x5573b9567010 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ebc2507298d4ff server_cookie=5fbc67fe15aeb3de in_seq=0 out_seq=0 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f76477fe700 1 -- 172.21.15.161:0/3396166645 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573b93fe050 con 0x5573b9564c00 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f76477fe700 10 client.26062 handle_command_reply: tid=0 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 client.26062 shutdown 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b94f14f0 msgr2=0x5573b95068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b94f14f0 0x5573b95068a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7658005950 tx=0x7f76580038b0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 shutdown_connections 2023-07-14T11:24:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5573b9564c00 0x5573b9567010 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573b9500140 0x5573b9504d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5573b94f14f0 0x5573b95068a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 --2- 172.21.15.161:0/3396166645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573b94f1e00 0x5573b9506de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f76477fe700 0 client.26062 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 >> 172.21.15.161:0/3396166645 conn(0x5573b94e6ff0 msgr2=0x5573b950b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 shutdown_connections 2023-07-14T11:24:18.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 1 -- 172.21.15.161:0/3396166645 wait complete. 2023-07-14T11:24:18.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.053+0000 7f7681ee3b80 20 client.26062 trim_cache size 0 max 16384 2023-07-14T11:24:18.065 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:24:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f76500a13e0 msgr2=0x7f76500a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f76500a13e0 0x7f76500a3890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f766819c380 tx=0x7f7660009920 comp rx=0 tx=0).stop 2023-07-14T11:24:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7668100380 msgr2=0x7f766819bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7668100380 0x7f766819bbc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f765c009990 tx=0x7f765c005f10 comp rx=0 tx=0).stop 2023-07-14T11:24:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 shutdown_connections 2023-07-14T11:24:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f76500a13e0 0x7f76500a3890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76680ff9d0 0x7f766819b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76681059e0 0x7f76681956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 --2- 172.21.15.161:0/1201408909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7668100380 0x7f766819bbc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 >> 172.21.15.161:0/1201408909 conn(0x7f76680fb4b0 msgr2=0x7f7668106850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 shutdown_connections 2023-07-14T11:24:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:18.054+0000 7f766d39d700 1 -- 172.21.15.161:0/1201408909 wait complete. 2023-07-14T11:24:18.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 126582 11.62 MB/sec warmup 60 sec latency 70.459 ms 2023-07-14T11:24:18.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:18 smithi175 ceph-mon[129081]: pgmap v64: 129 pgs: 129 active+clean; 78 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 363 op/s 2023-07-14T11:24:18.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:18 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3536095164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:24:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:18 smithi182 ceph-mon[129366]: pgmap v64: 129 pgs: 129 active+clean; 78 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 363 op/s 2023-07-14T11:24:18.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:18 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3536095164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:24:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:18 smithi161 ceph-mon[95526]: pgmap v64: 129 pgs: 129 active+clean; 78 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 363 op/s 2023-07-14T11:24:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:18 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3536095164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:24:19.075 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:24:19.076 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:24:19.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 128408 11.63 MB/sec warmup 61 sec latency 14.194 ms 2023-07-14T11:24:19.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 -- 172.21.15.161:0/1768608887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c105af0 msgr2=0x7f1d9c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 --2- 172.21.15.161:0/1768608887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c105af0 0x7f1d9c105ec0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009990 tx=0x7f1d90009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 -- 172.21.15.161:0/1768608887 shutdown_connections 2023-07-14T11:24:19.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 --2- 172.21.15.161:0/1768608887 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c100490 0x7f1d9c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 --2- 172.21.15.161:0/1768608887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c105af0 0x7f1d9c105ec0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 --2- 172.21.15.161:0/1768608887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d9c0ffae0 0x7f1d9c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 -- 172.21.15.161:0/1768608887 >> 172.21.15.161:0/1768608887 conn(0x7f1d9c0fb5e0 msgr2=0x7f1d9c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.300+0000 7f1da2bff700 1 -- 172.21.15.161:0/1768608887 shutdown_connections 2023-07-14T11:24:19.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.301+0000 7f1da2bff700 1 -- 172.21.15.161:0/1768608887 wait complete. 2023-07-14T11:24:19.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.302+0000 7f1da2bff700 1 Processor -- start 2023-07-14T11:24:19.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.302+0000 7f1da2bff700 1 -- start start 2023-07-14T11:24:19.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da2bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d9c0ffae0 0x7f1d9c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da2bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c100490 0x7f1d9c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da2bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c105af0 0x7f1d9c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da2bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c1960c0 con 0x7f1d9c0ffae0 2023-07-14T11:24:19.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da2bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c196230 con 0x7f1d9c100490 2023-07-14T11:24:19.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da2bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c1963f0 con 0x7f1d9c105af0 2023-07-14T11:24:19.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c105af0 0x7f1d9c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c100490 0x7f1d9c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c105af0 0x7f1d9c195a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38042/0 (socket says 172.21.15.161:38042) 2023-07-14T11:24:19.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da119c700 1 -- 172.21.15.161:0/307152000 learned_addr learned my addr 172.21.15.161:0/307152000 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da099b700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d9c0ffae0 0x7f1d9c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1d9bfff700 1 -- 172.21.15.161:0/307152000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c105af0 msgr2=0x7f1d9c195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c105af0 0x7f1d9c195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1d9bfff700 1 -- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d9c0ffae0 msgr2=0x7f1d9c19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d9c0ffae0 0x7f1d9c19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1d9bfff700 1 -- 172.21.15.161:0/307152000 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900095f0 con 0x7f1d9c100490 2023-07-14T11:24:19.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.303+0000 7f1da119c700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c105af0 0x7f1d9c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:19.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.304+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c100490 0x7f1d9c19bf20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800e9f0 tx=0x7f1d8800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.304+0000 7f1d99ffb700 1 -- 172.21.15.161:0/307152000 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800c9c0 con 0x7f1d9c100490 2023-07-14T11:24:19.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.304+0000 7f1d99ffb700 1 -- 172.21.15.161:0/307152000 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d88010430 con 0x7f1d9c100490 2023-07-14T11:24:19.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.304+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c1966d0 con 0x7f1d9c100490 2023-07-14T11:24:19.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.304+0000 7f1d99ffb700 1 -- 172.21.15.161:0/307152000 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88018680 con 0x7f1d9c100490 2023-07-14T11:24:19.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.304+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c1a29d0 con 0x7f1d9c100490 2023-07-14T11:24:19.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.307+0000 7f1d99ffb700 1 -- 172.21.15.161:0/307152000 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1d88010860 con 0x7f1d9c100490 2023-07-14T11:24:19.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 Processor -- start 2023-07-14T11:24:19.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 -- start start 2023-07-14T11:24:19.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x564584566720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x564584575a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584578340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458446cd30 con 0x564584566c60 2023-07-14T11:24:19.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645844bfac0 con 0x564584566350 2023-07-14T11:24:19.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1db6512b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645844b8f50 con 0x564584575f60 2023-07-14T11:24:19.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1da119c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x564584566720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584578340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1da119c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x564584566720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51914/0 (socket says 172.21.15.161:51914) 2023-07-14T11:24:19.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.309+0000 7f1da119c700 1 -- 172.21.15.161:0/110411507 learned_addr learned my addr 172.21.15.161:0/110411507 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1da099b700 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x564584575a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d99ffb700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1d8c0a1340 0x7f1d8c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d99ffb700 1 -- 172.21.15.161:0/307152000 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f1d88014070 con 0x7f1d9c100490 2023-07-14T11:24:19.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d9bfff700 1 -- 172.21.15.161:0/110411507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 msgr2=0x564584566720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x564584566720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d9bfff700 1 -- 172.21.15.161:0/110411507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 msgr2=0x564584575a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x564584575a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d9bfff700 1 -- 172.21.15.161:0/110411507 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645842e6e60 con 0x564584575f60 2023-07-14T11:24:19.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1da119c700 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x564584566720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:19.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1da119c700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1d8c0a1340 0x7f1d8c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584578340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d880ca530 tx=0x7f1d8801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684f03fe15e3d4a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d86ffd700 1 -- 172.21.15.161:0/110411507 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88099810 con 0x564584575f60 2023-07-14T11:24:19.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d86ffd700 1 -- 172.21.15.161:0/110411507 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d880d0630 con 0x564584575f60 2023-07-14T11:24:19.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.310+0000 7f1d86ffd700 1 -- 172.21.15.161:0/110411507 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8809a370 con 0x564584575f60 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 -- 172.21.15.161:0/110411507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 msgr2=0x564584578340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1da119c700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1d8c0a1340 0x7f1d8c0a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1d940079a0 tx=0x7f1d94009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584578340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d880ca530 tx=0x7f1d8801f000 comp rx=0 tx=0).stop 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 -- 172.21.15.161:0/110411507 shutdown_connections 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584578340 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x564584566720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 --2- 172.21.15.161:0/110411507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x564584575a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 -- 172.21.15.161:0/110411507 >> 172.21.15.161:0/110411507 conn(0x56458455ce20 msgr2=0x564584562720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 -- 172.21.15.161:0/110411507 shutdown_connections 2023-07-14T11:24:19.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.311+0000 7f1db6512b80 1 -- 172.21.15.161:0/110411507 wait complete. 2023-07-14T11:24:19.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.312+0000 7f1db6512b80 1 -- start start 2023-07-14T11:24:19.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.312+0000 7f1db6512b80 1 Processor -- start 2023-07-14T11:24:19.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x5645845759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566c60 0x564584572c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584573360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1db6512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564584458b40 con 0x564584566350 2023-07-14T11:24:19.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1db6512b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564584458840 con 0x564584566c60 2023-07-14T11:24:19.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1db6512b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458433c9a0 con 0x564584575f60 2023-07-14T11:24:19.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1db6512b80 10 client.0 _ll_register_callbacks cb 0x5645844567e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:19.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566c60 0x564584572c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566c60 0x564584572c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51922/0 (socket says 172.21.15.161:51922) 2023-07-14T11:24:19.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1179132816 learned_addr learned my addr 172.21.15.161:0/1179132816 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1da099b700 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x5645845759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1da119c700 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584573360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1179132816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 msgr2=0x564584573360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584573360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1179132816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 msgr2=0x5645845759a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x5645845759a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.313+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1179132816 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d940091c0 con 0x564584566c60 2023-07-14T11:24:19.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566c60 0x564584572c20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d880963b0 tx=0x7f1d880107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43fd6c3fd5a71531 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:19.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1179132816 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880b1070 con 0x564584566c60 2023-07-14T11:24:19.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1179132816 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f1d880a2300 con 0x564584566c60 2023-07-14T11:24:19.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564584412570 con 0x564584566c60 2023-07-14T11:24:19.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1179132816 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d88099eb0 con 0x564584566c60 2023-07-14T11:24:19.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 10 client.26081 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:19.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f1d9c1a29d0 con 0x564584566c60 2023-07-14T11:24:19.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1179132816 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d880c8080 con 0x564584566c60 2023-07-14T11:24:19.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1179132816 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1d880b4d20 con 0x564584566c60 2023-07-14T11:24:19.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 10 client.26081 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:19.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:19.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 20 client.26081 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:19.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564584585510 0x564584587920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 4 client.26081 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:19.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.314+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645844b1f30 con 0x564584585510 2023-07-14T11:24:19.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.315+0000 7f1da099b700 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564584585510 0x564584587920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.315+0000 7f1da099b700 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564584585510 0x564584587920 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d6480af4d027f6 server_cookie=4d46ad0f00fe0933 in_seq=0 out_seq=0 2023-07-14T11:24:19.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.315+0000 7f1d7bfff700 10 client.26081 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.316+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1179132816 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5645844b1f30 con 0x564584585510 2023-07-14T11:24:19.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.316+0000 7f1d7bfff700 10 client.26081 handle_command_reply: tid=0 2023-07-14T11:24:19.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.316+0000 7f1db6512b80 1 client.26081 shutdown 2023-07-14T11:24:19.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566c60 msgr2=0x564584572c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566c60 0x564584572c20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d880963b0 tx=0x7f1d880107d0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 shutdown_connections 2023-07-14T11:24:19.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564584585510 0x564584587920 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x5645845759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x564584573360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1d7bfff700 0 client.26081 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1179132816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566c60 0x564584572c20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 >> 172.21.15.161:0/1179132816 conn(0x56458455ce20 msgr2=0x5645845622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 shutdown_connections 2023-07-14T11:24:19.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 1 -- 172.21.15.161:0/1179132816 wait complete. 2023-07-14T11:24:19.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.317+0000 7f1db6512b80 20 client.26081 trim_cache size 0 max 16384 2023-07-14T11:24:19.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 Processor -- start 2023-07-14T11:24:19.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 -- start start 2023-07-14T11:24:19.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x56458457c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584566c60 0x564584575540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584575f60 0x56458456df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564584581830 con 0x564584566350 2023-07-14T11:24:19.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564584571180 con 0x564584575f60 2023-07-14T11:24:19.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1db6512b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564584587f80 con 0x564584566c60 2023-07-14T11:24:19.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584566c60 0x564584575540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584566c60 0x564584575540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38072/0 (socket says 172.21.15.161:38072) 2023-07-14T11:24:19.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1741147314 learned_addr learned my addr 172.21.15.161:0/1741147314 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1da119c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584575f60 0x56458456df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.336+0000 7f1da099b700 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x56458457c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1741147314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584575f60 msgr2=0x56458456df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584575f60 0x56458456df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1741147314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 msgr2=0x56458457c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x56458457c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d9bfff700 1 -- 172.21.15.161:0/1741147314 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645842cfce0 con 0x564584566c60 2023-07-14T11:24:19.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584566c60 0x564584575540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d88099e40 tx=0x7f1d88005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee986de9f8524a84 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d877fe700 1 -- 172.21.15.161:0/1741147314 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880a2120 con 0x564584566c60 2023-07-14T11:24:19.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d877fe700 1 -- 172.21.15.161:0/1741147314 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564584412570 con 0x564584566c60 2023-07-14T11:24:19.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.337+0000 7f1d877fe700 1 -- 172.21.15.161:0/1741147314 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880adbb0 con 0x564584566c60 2023-07-14T11:24:19.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 -- 172.21.15.161:0/1741147314 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584566c60 msgr2=0x564584575540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584566c60 0x564584575540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d88099e40 tx=0x7f1d88005bf0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 -- 172.21.15.161:0/1741147314 shutdown_connections 2023-07-14T11:24:19.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566350 0x56458457c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584575f60 0x56458456df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1741147314 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584566c60 0x564584575540 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 -- 172.21.15.161:0/1741147314 >> 172.21.15.161:0/1741147314 conn(0x56458455ce20 msgr2=0x5645845810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 -- 172.21.15.161:0/1741147314 shutdown_connections 2023-07-14T11:24:19.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.338+0000 7f1db6512b80 1 -- 172.21.15.161:0/1741147314 wait complete. 2023-07-14T11:24:19.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.339+0000 7f1db6512b80 1 -- start start 2023-07-14T11:24:19.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.339+0000 7f1db6512b80 1 Processor -- start 2023-07-14T11:24:19.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x5645845d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x5645845d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1db6512b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x56458457be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1db6512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458457c440 con 0x564584566c60 2023-07-14T11:24:19.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1db6512b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458457c5b0 con 0x564584566350 2023-07-14T11:24:19.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1db6512b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458457c770 con 0x564584575f60 2023-07-14T11:24:19.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1db6512b80 10 client.0 _ll_register_callbacks cb 0x7f1d88005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:19.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x5645845d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x5645845d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51946/0 (socket says 172.21.15.161:51946) 2023-07-14T11:24:19.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 -- 172.21.15.161:0/1396626203 learned_addr learned my addr 172.21.15.161:0/1396626203 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x5645845d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da119c700 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x56458457be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 -- 172.21.15.161:0/1396626203 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 msgr2=0x56458457be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x56458457be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 -- 172.21.15.161:0/1396626203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 msgr2=0x5645845d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x5645845d4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 -- 172.21.15.161:0/1396626203 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c1a29d0 con 0x564584566350 2023-07-14T11:24:19.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1da099b700 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x5645845d3d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000b6f0 tx=0x7f1d90005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c56fbf0f412ee73 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.340+0000 7f1d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:19.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1396626203 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001d070 con 0x564584566350 2023-07-14T11:24:19.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1396626203 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f1d90028d40 con 0x564584566350 2023-07-14T11:24:19.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d880d0630 con 0x564584566350 2023-07-14T11:24:19.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1396626203 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d900280b0 con 0x564584566350 2023-07-14T11:24:19.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1db6512b80 10 client.26087 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:19.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56458430d2d0 con 0x564584566350 2023-07-14T11:24:19.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1396626203 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d880d0630 con 0x564584566350 2023-07-14T11:24:19.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.341+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1396626203 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1d90004500 con 0x564584566350 2023-07-14T11:24:19.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1db6512b80 10 client.26087 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:19.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1db6512b80 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:19.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1db6512b80 20 client.26087 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:19.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645845dbcb0 0x5645845de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1db6512b80 4 client.26087 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:24:19.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5645844740d0 con 0x5645845dbcb0 2023-07-14T11:24:19.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645845dbcb0 0x5645845de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1d7bfff700 10 client.26087 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.342+0000 7f1d9bfff700 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645845dbcb0 0x5645845de0c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8ef51d1cc16470 server_cookie=ebb049af56af691 in_seq=0 out_seq=0 2023-07-14T11:24:19.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1d7bfff700 1 -- 172.21.15.161:0/1396626203 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5645844740d0 con 0x5645845dbcb0 2023-07-14T11:24:19.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1d7bfff700 10 client.26087 handle_command_reply: tid=0 2023-07-14T11:24:19.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 client.26087 shutdown 2023-07-14T11:24:19.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 msgr2=0x5645845d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x5645845d3d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000b6f0 tx=0x7f1d90005fb0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 shutdown_connections 2023-07-14T11:24:19.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5645845dbcb0 0x5645845de0c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564584575f60 0x56458457be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564584566350 0x5645845d3d30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1d7bfff700 0 client.26087 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 --2- 172.21.15.161:0/1396626203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564584566c60 0x5645845d4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 >> 172.21.15.161:0/1396626203 conn(0x56458455ce20 msgr2=0x56458456df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 shutdown_connections 2023-07-14T11:24:19.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.345+0000 7f1db6512b80 1 -- 172.21.15.161:0/1396626203 wait complete. 2023-07-14T11:24:19.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1db6512b80 20 client.26087 trim_cache size 0 max 16384 2023-07-14T11:24:19.359 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:24:19.360 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:24:19.360 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "44e63868-f865-42ec-ae93-2e278538d82e", 2023-07-14T11:24:19.360 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:24:19.361 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:24:19.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1d8c0a1340 msgr2=0x7f1d8c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1d8c0a1340 0x7f1d8c0a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1d940079a0 tx=0x7f1d94009040 comp rx=0 tx=0).stop 2023-07-14T11:24:19.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c100490 msgr2=0x7f1d9c19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c100490 0x7f1d9c19bf20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800e9f0 tx=0x7f1d8800ed00 comp rx=0 tx=0).stop 2023-07-14T11:24:19.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 shutdown_connections 2023-07-14T11:24:19.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1d8c0a1340 0x7f1d8c0a37f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d9c0ffae0 0x7f1d9c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1d9c105af0 0x7f1d9c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 --2- 172.21.15.161:0/307152000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9c100490 0x7f1d9c19bf20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 >> 172.21.15.161:0/307152000 conn(0x7f1d9c0fb5e0 msgr2=0x7f1d9c0fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.346+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 shutdown_connections 2023-07-14T11:24:19.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.347+0000 7f1da2bff700 1 -- 172.21.15.161:0/307152000 wait complete. 2023-07-14T11:24:19.372 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:24:19.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 -- 172.21.15.161:0/2975751274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb84105af0 msgr2=0x7fcb84105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 --2- 172.21.15.161:0/2975751274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb84105af0 0x7fcb84105ec0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009990 tx=0x7fcb74009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 -- 172.21.15.161:0/2975751274 shutdown_connections 2023-07-14T11:24:19.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 --2- 172.21.15.161:0/2975751274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb84100490 0x7fcb8410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 --2- 172.21.15.161:0/2975751274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb84105af0 0x7fcb84105ec0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 --2- 172.21.15.161:0/2975751274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb840ffae0 0x7fcb840fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 -- 172.21.15.161:0/2975751274 >> 172.21.15.161:0/2975751274 conn(0x7fcb840fb5e0 msgr2=0x7fcb840fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 -- 172.21.15.161:0/2975751274 shutdown_connections 2023-07-14T11:24:19.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.587+0000 7fcb889ff700 1 -- 172.21.15.161:0/2975751274 wait complete. 2023-07-14T11:24:19.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.588+0000 7fcb889ff700 1 Processor -- start 2023-07-14T11:24:19.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb889ff700 1 -- start start 2023-07-14T11:24:19.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb889ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb840ffae0 0x7fcb84197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb889ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb84100490 0x7fcb841983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb889ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb84105af0 0x7fcb8419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb889ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8419ce50 con 0x7fcb84105af0 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb889ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8419cfc0 con 0x7fcb84100490 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb889ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8419d180 con 0x7fcb840ffae0 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb8259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb840ffae0 0x7fcb84197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb8259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb840ffae0 0x7fcb84197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38080/0 (socket says 172.21.15.161:38080) 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb8259c700 1 -- 172.21.15.161:0/4143417025 learned_addr learned my addr 172.21.15.161:0/4143417025 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb84100490 0x7fcb841983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb82d9d700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb84105af0 0x7fcb8419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb8259c700 1 -- 172.21.15.161:0/4143417025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb84100490 msgr2=0x7fcb841983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb8259c700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb84100490 0x7fcb841983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb8259c700 1 -- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb84105af0 msgr2=0x7fcb8419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.589+0000 7fcb8259c700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb84105af0 0x7fcb8419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb8259c700 1 -- 172.21.15.161:0/4143417025 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c009c40 con 0x7fcb840ffae0 2023-07-14T11:24:19.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb82d9d700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb84105af0 0x7fcb8419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:19.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb8259c700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb840ffae0 0x7fcb84197e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005910 tx=0x7fcb74004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb737fe700 1 -- 172.21.15.161:0/4143417025 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7401d070 con 0x7fcb840ffae0 2023-07-14T11:24:19.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb737fe700 1 -- 172.21.15.161:0/4143417025 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb74022470 con 0x7fcb840ffae0 2023-07-14T11:24:19.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb740095f0 con 0x7fcb840ffae0 2023-07-14T11:24:19.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb737fe700 1 -- 172.21.15.161:0/4143417025 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74017650 con 0x7fcb840ffae0 2023-07-14T11:24:19.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.590+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb841a4ac0 con 0x7fcb840ffae0 2023-07-14T11:24:19.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.594+0000 7fcb737fe700 1 -- 172.21.15.161:0/4143417025 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcb740225e0 con 0x7fcb840ffae0 2023-07-14T11:24:19.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.595+0000 7fcb737fe700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcb6c0a1340 0x7fcb6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.595+0000 7fcb9c3f5b80 1 Processor -- start 2023-07-14T11:24:19.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.595+0000 7fcb737fe700 1 -- 172.21.15.161:0/4143417025 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fcb740ce2c0 con 0x7fcb840ffae0 2023-07-14T11:24:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.595+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcb6c0a1340 0x7fcb6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.595+0000 7fcb9c3f5b80 1 -- start start 2023-07-14T11:24:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607472062d0 0x5607472066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560747206be0 0x560747211590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747213eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074710ccb0 con 0x560747206be0 2023-07-14T11:24:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074715fa40 con 0x5607472062d0 2023-07-14T11:24:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560747158ed0 con 0x560747211ad0 2023-07-14T11:24:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb8259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607472062d0 0x5607472066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb82d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747213eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcb6c0a1340 0x7fcb6c0a37f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb84199480 tx=0x7fcb78009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb81d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560747206be0 0x560747211590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb82d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747213eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38096/0 (socket says 172.21.15.161:38096) 2023-07-14T11:24:19.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb8259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607472062d0 0x5607472066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51986/0 (socket says 172.21.15.161:51986) 2023-07-14T11:24:19.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.596+0000 7fcb82d9d700 1 -- 172.21.15.161:0/2437168946 learned_addr learned my addr 172.21.15.161:0/2437168946 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb8259c700 1 -- 172.21.15.161:0/2437168946 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 msgr2=0x560747213eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb8259c700 1 --2- 172.21.15.161:0/2437168946 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747213eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb8259c700 1 -- 172.21.15.161:0/2437168946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560747206be0 msgr2=0x560747211590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb8259c700 1 --2- 172.21.15.161:0/2437168946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560747206be0 0x560747211590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb8259c700 1 -- 172.21.15.161:0/2437168946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560746f86de0 con 0x5607472062d0 2023-07-14T11:24:19.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb8259c700 1 --2- 172.21.15.161:0/2437168946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607472062d0 0x5607472066a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb740cb680 tx=0x7fcb7409abd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b870a13ef64bd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb70ff9700 1 -- 172.21.15.161:0/2437168946 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7409f0c0 con 0x5607472062d0 2023-07-14T11:24:19.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb70ff9700 1 -- 172.21.15.161:0/2437168946 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb740a3450 con 0x5607472062d0 2023-07-14T11:24:19.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.597+0000 7fcb70ff9700 1 -- 172.21.15.161:0/2437168946 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7409a580 con 0x5607472062d0 2023-07-14T11:24:19.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2437168946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607472062d0 msgr2=0x5607472066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2437168946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607472062d0 0x5607472066a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb740cb680 tx=0x7fcb7409abd0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2437168946 shutdown_connections 2023-07-14T11:24:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2437168946 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747213eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2437168946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607472062d0 0x5607472066a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2437168946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560747206be0 0x560747211590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2437168946 >> 172.21.15.161:0/2437168946 conn(0x5607471fcda0 msgr2=0x5607472026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2437168946 shutdown_connections 2023-07-14T11:24:19.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.598+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2437168946 wait complete. 2023-07-14T11:24:19.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.599+0000 7fcb9c3f5b80 1 -- start start 2023-07-14T11:24:19.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.599+0000 7fcb9c3f5b80 1 Processor -- start 2023-07-14T11:24:19.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x560747215ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747215ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747210a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607470f8ac0 con 0x5607472062d0 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607470f87c0 con 0x560747206be0 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560746fdc920 con 0x560747211ad0 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb9c3f5b80 10 client.0 _ll_register_callbacks cb 0x5607470f6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747215ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747215ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52002/0 (socket says 172.21.15.161:52002) 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 -- 172.21.15.161:0/2728474188 learned_addr learned my addr 172.21.15.161:0/2728474188 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb82d9d700 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747210a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb8259c700 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x560747215ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 -- 172.21.15.161:0/2728474188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 msgr2=0x560747210a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747210a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 -- 172.21.15.161:0/2728474188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 msgr2=0x560747215ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x560747215ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.600+0000 7fcb81d9b700 1 -- 172.21.15.161:0/2728474188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb841a4ac0 con 0x560747206be0 2023-07-14T11:24:19.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747215ff0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb7801d8c0 tx=0x7fcb780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597fe9f053b1136c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:19.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb69ffb700 1 -- 172.21.15.161:0/2728474188 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78022070 con 0x560747206be0 2023-07-14T11:24:19.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607470b24f0 con 0x560747206be0 2023-07-14T11:24:19.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb69ffb700 1 -- 172.21.15.161:0/2728474188 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fcb7801c610 con 0x560747206be0 2023-07-14T11:24:19.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb69ffb700 1 -- 172.21.15.161:0/2728474188 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb78004830 con 0x560747206be0 2023-07-14T11:24:19.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb9c3f5b80 10 client.26099 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:19.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560746f6fc60 con 0x560747206be0 2023-07-14T11:24:19.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.601+0000 7fcb69ffb700 1 -- 172.21.15.161:0/2728474188 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607470b24f0 con 0x560747206be0 2023-07-14T11:24:19.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb69ffb700 1 -- 172.21.15.161:0/2728474188 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcb78005020 con 0x560747206be0 2023-07-14T11:24:19.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb9c3f5b80 10 client.26099 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:19.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb9c3f5b80 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:19.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb9c3f5b80 20 client.26099 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:19.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5607472210b0 0x5607472234c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb9c3f5b80 4 client.26099 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560747151eb0 con 0x5607472210b0 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb8259c700 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5607472210b0 0x5607472234c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb69ffb700 10 client.26099 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.602+0000 7fcb8259c700 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5607472210b0 0x5607472234c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76fc3f33f943a353 server_cookie=49e94886bd8d1cca in_seq=0 out_seq=0 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb69ffb700 1 -- 172.21.15.161:0/2728474188 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560747151eb0 con 0x5607472210b0 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb69ffb700 10 client.26099 handle_command_reply: tid=0 2023-07-14T11:24:19.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 client.26099 shutdown 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 msgr2=0x560747215ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747215ff0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb7801d8c0 tx=0x7fcb780047c0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 shutdown_connections 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5607472210b0 0x5607472234c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x560747215ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747210a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/2728474188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747215ff0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb69ffb700 0 client.26099 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.603+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 >> 172.21.15.161:0/2728474188 conn(0x5607471fcda0 msgr2=0x560747202290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.604+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 shutdown_connections 2023-07-14T11:24:19.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.604+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/2728474188 wait complete. 2023-07-14T11:24:19.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.604+0000 7fcb9c3f5b80 20 client.26099 trim_cache size 0 max 16384 2023-07-14T11:24:19.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.625+0000 7fcb9c3f5b80 1 Processor -- start 2023-07-14T11:24:19.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.625+0000 7fcb9c3f5b80 1 -- start start 2023-07-14T11:24:19.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.625+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x560747210210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747206be0 0x560747210750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747211ad0 0x560747215770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560747207540 con 0x5607472062d0 2023-07-14T11:24:19.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607471fc970 con 0x560747211ad0 2023-07-14T11:24:19.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607472120a0 con 0x560747206be0 2023-07-14T11:24:19.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb81d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747206be0 0x560747210750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb81d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747206be0 0x560747210750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38112/0 (socket says 172.21.15.161:38112) 2023-07-14T11:24:19.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.626+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3035169689 learned_addr learned my addr 172.21.15.161:0/3035169689 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.628+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3035169689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747211ad0 msgr2=0x560747215770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.628+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/3035169689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747211ad0 0x560747215770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.628+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3035169689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 msgr2=0x560747210210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:24:19.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.628+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/3035169689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x560747210210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.628+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3035169689 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560746fad250 con 0x560747206be0 2023-07-14T11:24:19.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.628+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/3035169689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747206be0 0x560747210750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb78000c00 tx=0x7fcb78009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6f0c98bf8c6082 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.629+0000 7fcb717fa700 1 -- 172.21.15.161:0/3035169689 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78022040 con 0x560747206be0 2023-07-14T11:24:19.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.629+0000 7fcb717fa700 1 -- 172.21.15.161:0/3035169689 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb78005630 con 0x560747206be0 2023-07-14T11:24:19.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.630+0000 7fcb717fa700 1 -- 172.21.15.161:0/3035169689 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78004e40 con 0x560747206be0 2023-07-14T11:24:19.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.631+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3035169689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747206be0 msgr2=0x560747210750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.631+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3035169689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747206be0 0x560747210750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb78000c00 tx=0x7fcb78009370 comp rx=0 tx=0).stop 2023-07-14T11:24:19.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.631+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3035169689 shutdown_connections 2023-07-14T11:24:19.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.631+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3035169689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x560747210210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.631+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3035169689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747211ad0 0x560747215770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.631+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3035169689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747206be0 0x560747210750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.631+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3035169689 >> 172.21.15.161:0/3035169689 conn(0x5607471fcda0 msgr2=0x56074720bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.632+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3035169689 shutdown_connections 2023-07-14T11:24:19.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.632+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3035169689 wait complete. 2023-07-14T11:24:19.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.633+0000 7fcb9c3f5b80 1 -- start start 2023-07-14T11:24:19.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.633+0000 7fcb9c3f5b80 1 Processor -- start 2023-07-14T11:24:19.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.634+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x56074721d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.634+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747219710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.634+0000 7fcb9c3f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747219c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.634+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074721a190 con 0x5607472062d0 2023-07-14T11:24:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.634+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074721a300 con 0x560747206be0 2023-07-14T11:24:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.634+0000 7fcb9c3f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074721a470 con 0x560747211ad0 2023-07-14T11:24:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.634+0000 7fcb9c3f5b80 10 client.0 _ll_register_callbacks cb 0x7fcb7801d4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.635+0000 7fcb81d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747219710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.635+0000 7fcb81d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747219710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52032/0 (socket says 172.21.15.161:52032) 2023-07-14T11:24:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.635+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3615844161 learned_addr learned my addr 172.21.15.161:0/3615844161 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.635+0000 7fcb82d9d700 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747219c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.636+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3615844161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 msgr2=0x560747219c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.636+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747219c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.636+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3615844161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 msgr2=0x56074721d400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:24:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.637+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x56074721d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.637+0000 7fcb81d9b700 1 -- 172.21.15.161:0/3615844161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560746f6fc60 con 0x560747206be0 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.637+0000 7fcb81d9b700 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747219710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcb7801d8c0 tx=0x7fcb78004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af75188ba36b3bfd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.637+0000 7fcb69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.638+0000 7fcb69ffb700 1 -- 172.21.15.161:0/3615844161 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78022040 con 0x560747206be0 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.638+0000 7fcb69ffb700 1 -- 172.21.15.161:0/3615844161 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fcb78003950 con 0x560747206be0 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.639+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb740a3450 con 0x560747206be0 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.641+0000 7fcb69ffb700 1 -- 172.21.15.161:0/3615844161 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb78003b10 con 0x560747206be0 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.641+0000 7fcb9c3f5b80 10 client.26102 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.641+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560746f852b0 con 0x560747206be0 2023-07-14T11:24:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb69ffb700 1 -- 172.21.15.161:0/3615844161 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb740a3450 con 0x560747206be0 2023-07-14T11:24:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb69ffb700 1 -- 172.21.15.161:0/3615844161 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcb7802e070 con 0x560747206be0 2023-07-14T11:24:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb9c3f5b80 10 client.26102 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb9c3f5b80 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb9c3f5b80 20 client.26102 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56074727d940 0x56074727fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb9c3f5b80 4 client.26102 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:24:19.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.646+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560747114050 con 0x56074727d940 2023-07-14T11:24:19.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.649+0000 7fcb8259c700 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56074727d940 0x56074727fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.661+0000 7fcb8259c700 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56074727d940 0x56074727fd50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=501ac7e55c09919b server_cookie=a4e6b76aba8f3a9f in_seq=0 out_seq=0 2023-07-14T11:24:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.661+0000 7fcb69ffb700 10 client.26102 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.665+0000 7fcb69ffb700 1 -- 172.21.15.161:0/3615844161 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560747114050 con 0x56074727d940 2023-07-14T11:24:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.665+0000 7fcb69ffb700 10 client.26102 handle_command_reply: tid=0 2023-07-14T11:24:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.665+0000 7fcb9c3f5b80 1 client.26102 shutdown 2023-07-14T11:24:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 msgr2=0x560747219710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747219710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcb7801d8c0 tx=0x7fcb78004f10 comp rx=0 tx=0).stop 2023-07-14T11:24:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 shutdown_connections 2023-07-14T11:24:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56074727d940 0x56074727fd50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607472062d0 0x56074721d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560747211ad0 0x560747219c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 --2- 172.21.15.161:0/3615844161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560747206be0 0x560747219710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 >> 172.21.15.161:0/3615844161 conn(0x5607471fcda0 msgr2=0x5607472018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.666+0000 7fcb69ffb700 0 client.26102 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:19.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.667+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 shutdown_connections 2023-07-14T11:24:19.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.668+0000 7fcb9c3f5b80 1 -- 172.21.15.161:0/3615844161 wait complete. 2023-07-14T11:24:19.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.671+0000 7fcb9c3f5b80 20 client.26102 trim_cache size 0 max 16384 2023-07-14T11:24:19.680 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:24:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:24:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:24:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "44e63868-f865-42ec-ae93-2e278538d82e": { 2023-07-14T11:24:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:24:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "44e63868-f865-42ec-ae93-2e278538d82e", 2023-07-14T11:24:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:24:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:24:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:24:19.681 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:24:19.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.678+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcb6c0a1340 msgr2=0x7fcb6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.678+0000 7fcb889ff700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcb6c0a1340 0x7fcb6c0a37f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb84199480 tx=0x7fcb78009260 comp rx=0 tx=0).stop 2023-07-14T11:24:19.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.678+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb840ffae0 msgr2=0x7fcb84197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:19.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.678+0000 7fcb889ff700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb840ffae0 0x7fcb84197e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005910 tx=0x7fcb74004710 comp rx=0 tx=0).stop 2023-07-14T11:24:19.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.679+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 shutdown_connections 2023-07-14T11:24:19.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.679+0000 7fcb889ff700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcb6c0a1340 0x7fcb6c0a37f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.679+0000 7fcb889ff700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb84105af0 0x7fcb8419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.679+0000 7fcb889ff700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcb840ffae0 0x7fcb84197e70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.679+0000 7fcb889ff700 1 --2- 172.21.15.161:0/4143417025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb84100490 0x7fcb841983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:19.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.679+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 >> 172.21.15.161:0/4143417025 conn(0x7fcb840fb5e0 msgr2=0x7fcb8410af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:19.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.681+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 shutdown_connections 2023-07-14T11:24:19.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:19.681+0000 7fcb889ff700 1 -- 172.21.15.161:0/4143417025 wait complete. 2023-07-14T11:24:19.714 INFO:tasks.cephfs.filesystem:scrub status for tag:44e63868-f865-42ec-ae93-2e278538d82e - {'path': '/', 'tag': '44e63868-f865-42ec-ae93-2e278538d82e', 'options': 'recursive,force'} 2023-07-14T11:24:20.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 130219 11.57 MB/sec warmup 62 sec latency 62.825 ms 2023-07-14T11:24:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:20 smithi175 ceph-mon[129081]: pgmap v65: 129 pgs: 129 active+clean; 78 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-07-14T11:24:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:20 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:24:20.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:20 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:20 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:24:20.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:20 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:24:20.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:20 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:20 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:20 smithi182 ceph-mon[129366]: pgmap v65: 129 pgs: 129 active+clean; 78 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-07-14T11:24:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:20 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:24:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:20 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:20 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:24:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:20 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:24:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:20 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:20 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:20 smithi161 ceph-mon[95526]: pgmap v65: 129 pgs: 129 active+clean; 78 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-07-14T11:24:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:20 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:24:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:20 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:20 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:24:20.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:20 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:24:20.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:20 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:20.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:20 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:21.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 131885 11.56 MB/sec warmup 63 sec latency 17.260 ms 2023-07-14T11:24:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:21 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:24:21.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:21 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:21.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:21 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:24:21.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:21 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:21.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:21 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:24:21.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:21 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:22.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 134298 11.58 MB/sec warmup 64 sec latency 10.269 ms 2023-07-14T11:24:22.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:22 smithi175 ceph-mon[129081]: pgmap v66: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 347 op/s 2023-07-14T11:24:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:22 smithi182 ceph-mon[129366]: pgmap v66: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 347 op/s 2023-07-14T11:24:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:22 smithi161 ceph-mon[95526]: pgmap v66: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 347 op/s 2023-07-14T11:24:23.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 135765 11.51 MB/sec warmup 65 sec latency 24.166 ms 2023-07-14T11:24:24.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 138476 11.60 MB/sec warmup 66 sec latency 20.319 ms 2023-07-14T11:24:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:23 smithi175 ceph-mon[129081]: pgmap v67: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-14T11:24:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:23 smithi182 ceph-mon[129366]: pgmap v67: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-14T11:24:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:23 smithi161 ceph-mon[95526]: pgmap v67: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-14T11:24:25.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 139405 11.47 MB/sec warmup 67 sec latency 36.489 ms 2023-07-14T11:24:25.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:24:25.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:24:25.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:24:26.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 141894 11.53 MB/sec warmup 68 sec latency 23.742 ms 2023-07-14T11:24:26.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: pgmap v68: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-14T11:24:26.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:24:26.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:24:26.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:24:26.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: pgmap v68: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-14T11:24:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:24:26.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:24:26.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:24:26.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: pgmap v68: 129 pgs: 129 active+clean; 83 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-14T11:24:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:26.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:24:26.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:24:26.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:24:26.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:24:27.167 INFO:tasks.workunit.client.0.smithi161.stdout: 1 143091 11.42 MB/sec warmup 69 sec latency 10.753 ms 2023-07-14T11:24:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:27 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:24:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:27 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:24:27.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:27 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:24:27.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:27 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:24:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:27 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:24:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:27 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:24:28.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 145695 11.50 MB/sec warmup 70 sec latency 28.627 ms 2023-07-14T11:24:28.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:28 smithi182 ceph-mon[129366]: pgmap v69: 129 pgs: 129 active+clean; 87 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 335 op/s 2023-07-14T11:24:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:28 smithi161 ceph-mon[95526]: pgmap v69: 129 pgs: 129 active+clean; 87 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 335 op/s 2023-07-14T11:24:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:28 smithi175 ceph-mon[129081]: pgmap v69: 129 pgs: 129 active+clean; 87 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 335 op/s 2023-07-14T11:24:29.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 146991 11.41 MB/sec warmup 71 sec latency 15.294 ms 2023-07-14T11:24:30.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 149120 11.43 MB/sec warmup 72 sec latency 183.571 ms 2023-07-14T11:24:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:30 smithi161 ceph-mon[95526]: pgmap v70: 129 pgs: 129 active+clean; 87 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 223 op/s 2023-07-14T11:24:30.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:30 smithi175 ceph-mon[129081]: pgmap v70: 129 pgs: 129 active+clean; 87 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 223 op/s 2023-07-14T11:24:30.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:30 smithi182 ceph-mon[129366]: pgmap v70: 129 pgs: 129 active+clean; 87 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 223 op/s 2023-07-14T11:24:31.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 149224 11.28 MB/sec warmup 73 sec latency 929.178 ms 2023-07-14T11:24:32.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 149224 11.13 MB/sec warmup 74 sec latency 1929.282 ms 2023-07-14T11:24:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:32 smithi161 ceph-mon[95526]: pgmap v71: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2023-07-14T11:24:32.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:32 smithi175 ceph-mon[129081]: pgmap v71: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2023-07-14T11:24:32.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:32 smithi182 ceph-mon[129366]: pgmap v71: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2023-07-14T11:24:33.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 149224 10.98 MB/sec warmup 75 sec latency 2929.414 ms 2023-07-14T11:24:34.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 149224 10.84 MB/sec warmup 76 sec latency 3929.554 ms 2023-07-14T11:24:34.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:33 smithi175 ceph-mon[129081]: pgmap v72: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-14T11:24:34.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:33 smithi182 ceph-mon[129366]: pgmap v72: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-14T11:24:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:33 smithi161 ceph-mon[95526]: pgmap v72: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-14T11:24:35.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 149224 10.70 MB/sec warmup 77 sec latency 4929.641 ms 2023-07-14T11:24:36.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 150262 10.59 MB/sec warmup 78 sec latency 5015.511 ms 2023-07-14T11:24:36.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:36 smithi175 ceph-mon[129081]: pgmap v73: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-14T11:24:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:36 smithi182 ceph-mon[129366]: pgmap v73: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-14T11:24:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:36 smithi161 ceph-mon[95526]: pgmap v73: 129 pgs: 129 active+clean; 90 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-14T11:24:37.168 INFO:tasks.workunit.client.0.smithi161.stdout: 1 152804 10.67 MB/sec warmup 79 sec latency 37.584 ms 2023-07-14T11:24:38.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 154123 10.60 MB/sec warmup 80 sec latency 16.361 ms 2023-07-14T11:24:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:38 smithi175 ceph-mon[129081]: pgmap v74: 129 pgs: 129 active+clean; 90 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 244 op/s 2023-07-14T11:24:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:38 smithi182 ceph-mon[129366]: pgmap v74: 129 pgs: 129 active+clean; 90 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 244 op/s 2023-07-14T11:24:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:38 smithi161 ceph-mon[95526]: pgmap v74: 129 pgs: 129 active+clean; 90 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 244 op/s 2023-07-14T11:24:39.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 156478 10.65 MB/sec warmup 81 sec latency 9.755 ms 2023-07-14T11:24:40.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 158171 10.59 MB/sec warmup 82 sec latency 41.107 ms 2023-07-14T11:24:40.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:40 smithi175 ceph-mon[129081]: pgmap v75: 129 pgs: 129 active+clean; 90 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-14T11:24:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:40 smithi182 ceph-mon[129366]: pgmap v75: 129 pgs: 129 active+clean; 90 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-14T11:24:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:40 smithi161 ceph-mon[95526]: pgmap v75: 129 pgs: 129 active+clean; 90 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-14T11:24:41.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 160080 10.62 MB/sec warmup 83 sec latency 21.722 ms 2023-07-14T11:24:42.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 162248 10.59 MB/sec warmup 84 sec latency 13.172 ms 2023-07-14T11:24:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:42 smithi175 ceph-mon[129081]: pgmap v76: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 247 op/s 2023-07-14T11:24:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:42 smithi182 ceph-mon[129366]: pgmap v76: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 247 op/s 2023-07-14T11:24:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:42 smithi161 ceph-mon[95526]: pgmap v76: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 247 op/s 2023-07-14T11:24:43.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 163928 10.60 MB/sec warmup 85 sec latency 16.782 ms 2023-07-14T11:24:44.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 166569 10.64 MB/sec warmup 86 sec latency 13.134 ms 2023-07-14T11:24:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:43 smithi175 ceph-mon[129081]: pgmap v77: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-14T11:24:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:43 smithi182 ceph-mon[129366]: pgmap v77: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-14T11:24:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:43 smithi161 ceph-mon[95526]: pgmap v77: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-14T11:24:45.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 167805 10.59 MB/sec warmup 87 sec latency 13.697 ms 2023-07-14T11:24:46.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 170257 10.65 MB/sec warmup 88 sec latency 52.529 ms 2023-07-14T11:24:46.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:46 smithi175 ceph-mon[129081]: pgmap v78: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-14T11:24:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:46 smithi182 ceph-mon[129366]: pgmap v78: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-14T11:24:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:46 smithi161 ceph-mon[95526]: pgmap v78: 129 pgs: 129 active+clean; 94 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-14T11:24:47.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 171451 10.57 MB/sec warmup 89 sec latency 11.833 ms 2023-07-14T11:24:48.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 174060 10.64 MB/sec warmup 90 sec latency 11.098 ms 2023-07-14T11:24:48.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:48 smithi175 ceph-mon[129081]: pgmap v79: 129 pgs: 129 active+clean; 99 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2023-07-14T11:24:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:48 smithi182 ceph-mon[129366]: pgmap v79: 129 pgs: 129 active+clean; 99 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2023-07-14T11:24:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:48 smithi161 ceph-mon[95526]: pgmap v79: 129 pgs: 129 active+clean; 99 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2023-07-14T11:24:49.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 175353 10.58 MB/sec warmup 91 sec latency 20.796 ms 2023-07-14T11:24:49.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:24:49.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 -- 172.21.15.161:0/932034368 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00105af0 msgr2=0x7fdf00105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 --2- 172.21.15.161:0/932034368 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00105af0 0x7fdf00105ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0098f0 tx=0x7fdefc009c00 comp rx=0 tx=0).stop 2023-07-14T11:24:49.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 -- 172.21.15.161:0/932034368 shutdown_connections 2023-07-14T11:24:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 --2- 172.21.15.161:0/932034368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00100490 0x7fdf0010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 --2- 172.21.15.161:0/932034368 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00105af0 0x7fdf00105ec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 --2- 172.21.15.161:0/932034368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf000ffae0 0x7fdf000fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 -- 172.21.15.161:0/932034368 >> 172.21.15.161:0/932034368 conn(0x7fdf000fb5e0 msgr2=0x7fdf000fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 -- 172.21.15.161:0/932034368 shutdown_connections 2023-07-14T11:24:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.930+0000 7fdf079ff700 1 -- 172.21.15.161:0/932034368 wait complete. 2023-07-14T11:24:49.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.931+0000 7fdf079ff700 1 Processor -- start 2023-07-14T11:24:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf079ff700 1 -- start start 2023-07-14T11:24:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf079ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf000ffae0 0x7fdf00197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf079ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00100490 0x7fdf00198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf079ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00105af0 0x7fdf0019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf079ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0019ce30 con 0x7fdf000ffae0 2023-07-14T11:24:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf079ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0019cfa0 con 0x7fdf00105af0 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf079ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0019d160 con 0x7fdf00100490 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf05f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00105af0 0x7fdf0019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf0579b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf000ffae0 0x7fdf00197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf05f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00105af0 0x7fdf0019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35148/0 (socket says 172.21.15.161:35148) 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf05f9c700 1 -- 172.21.15.161:0/2424586930 learned_addr learned my addr 172.21.15.161:0/2424586930 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.932+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00100490 0x7fdf00198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf05f9c700 1 -- 172.21.15.161:0/2424586930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00100490 msgr2=0x7fdf00198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00100490 0x7fdf00198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf05f9c700 1 -- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf000ffae0 msgr2=0x7fdf00197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf000ffae0 0x7fdf00197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf05f9c700 1 -- 172.21.15.161:0/2424586930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009520 con 0x7fdf00105af0 2023-07-14T11:24:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00100490 0x7fdf00198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00105af0 0x7fdf0019c7f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c110 tx=0x7fdef800c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdef67fc700 1 -- 172.21.15.161:0/2424586930 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8017070 con 0x7fdf00105af0 2023-07-14T11:24:49.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0019d3e0 con 0x7fdf00105af0 2023-07-14T11:24:49.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdef67fc700 1 -- 172.21.15.161:0/2424586930 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef800e050 con 0x7fdf00105af0 2023-07-14T11:24:49.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdef67fc700 1 -- 172.21.15.161:0/2424586930 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8007940 con 0x7fdf00105af0 2023-07-14T11:24:49.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.933+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf001a4c80 con 0x7fdf00105af0 2023-07-14T11:24:49.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.937+0000 7fdef67fc700 1 -- 172.21.15.161:0/2424586930 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdef8004ad0 con 0x7fdf00105af0 2023-07-14T11:24:49.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 Processor -- start 2023-07-14T11:24:49.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 -- start start 2023-07-14T11:24:49.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdef67fc700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdeec0a12c0 0x7fdeec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdef67fc700 1 -- 172.21.15.161:0/2424586930 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fdef80ccad0 con 0x7fdf00105af0 2023-07-14T11:24:49.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece8eecb0 con 0x55aece9e8be0 2023-07-14T11:24:49.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece941a40 con 0x55aece9fc2e0 2023-07-14T11:24:49.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.938+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece93aed0 con 0x55aece9e82d0 2023-07-14T11:24:49.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdeec0a12c0 0x7fdeec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9e86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49446/0 (socket says 172.21.15.161:49446) 2023-07-14T11:24:49.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 -- 172.21.15.161:0/2978862828 learned_addr learned my addr 172.21.15.161:0/2978862828 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:49.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 -- 172.21.15.161:0/2978862828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 msgr2=0x55aece9fe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9fe6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 -- 172.21.15.161:0/2978862828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 msgr2=0x55aece9fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 -- 172.21.15.161:0/2978862828 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aece768de0 con 0x55aece9e82d0 2023-07-14T11:24:49.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf0579b700 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9e86a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a430e1b2f9a28c5a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:49.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.939+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdeec0a12c0 0x7fdeec0a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf00199460 tx=0x7fdef0009630 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:49.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdeeb7fe700 1 -- 172.21.15.161:0/2978862828 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00f7a0 con 0x55aece9e82d0 2023-07-14T11:24:49.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdeeb7fe700 1 -- 172.21.15.161:0/2978862828 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc01d040 con 0x55aece9e82d0 2023-07-14T11:24:49.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdeeb7fe700 1 -- 172.21.15.161:0/2978862828 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0054b0 con 0x55aece9e82d0 2023-07-14T11:24:49.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 -- 172.21.15.161:0/2978862828 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 msgr2=0x55aece9e86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9e86a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc005dc0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 -- 172.21.15.161:0/2978862828 shutdown_connections 2023-07-14T11:24:49.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9e86a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/2978862828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 -- 172.21.15.161:0/2978862828 >> 172.21.15.161:0/2978862828 conn(0x55aece9deda0 msgr2=0x55aece9e46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:49.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.940+0000 7fdf1b230b80 1 -- 172.21.15.161:0/2978862828 shutdown_connections 2023-07-14T11:24:49.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.941+0000 7fdf1b230b80 1 -- 172.21.15.161:0/2978862828 wait complete. 2023-07-14T11:24:49.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.941+0000 7fdf1b230b80 1 -- start start 2023-07-14T11:24:49.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.941+0000 7fdf1b230b80 1 Processor -- start 2023-07-14T11:24:49.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece8daac0 con 0x55aece9e8be0 2023-07-14T11:24:49.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece8da7c0 con 0x55aece9fc2e0 2023-07-14T11:24:49.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece7be920 con 0x55aece9e82d0 2023-07-14T11:24:49.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf1b230b80 10 client.0 _ll_register_callbacks cb 0x55aece8d8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:49.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf04f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf05f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf04f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59074/0 (socket says 172.21.15.161:59074) 2023-07-14T11:24:49.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf04f9a700 1 -- 172.21.15.161:0/745763823 learned_addr learned my addr 172.21.15.161:0/745763823 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:49.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf0579b700 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf05f9c700 1 -- 172.21.15.161:0/745763823 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 msgr2=0x55aece9fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf05f9c700 1 -- 172.21.15.161:0/745763823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 msgr2=0x55aece9f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf05f9c700 1 -- 172.21.15.161:0/745763823 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0010d220 con 0x55aece9fc2e0 2023-07-14T11:24:49.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.942+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:49.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9f94a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdef80c9e90 tx=0x7fdef8099440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a1ceed95aaab47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:49.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdee8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:49.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdee8ff9700 1 -- 172.21.15.161:0/745763823 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8099810 con 0x55aece9fc2e0 2023-07-14T11:24:49.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdee8ff9700 1 -- 172.21.15.161:0/745763823 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fdef80b7e40 con 0x55aece9fc2e0 2023-07-14T11:24:49.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aece8944f0 con 0x55aece9fc2e0 2023-07-14T11:24:49.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdee8ff9700 1 -- 172.21.15.161:0/745763823 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef809d840 con 0x55aece9fc2e0 2023-07-14T11:24:49.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdf1b230b80 10 client.26114 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:49.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55aece751c60 con 0x55aece9fc2e0 2023-07-14T11:24:49.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdee8ff9700 1 -- 172.21.15.161:0/745763823 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aece8944f0 con 0x55aece9fc2e0 2023-07-14T11:24:49.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdee8ff9700 1 -- 172.21.15.161:0/745763823 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdef8019b90 con 0x55aece9fc2e0 2023-07-14T11:24:49.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdf1b230b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:49.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdf1b230b80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:49.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.943+0000 7fdf1b230b80 20 client.26114 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:49.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.944+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aecea0bc60 0x55aecea0e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.944+0000 7fdf1b230b80 4 client.26114 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:49.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.944+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aece933eb0 con 0x55aecea0bc60 2023-07-14T11:24:49.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.944+0000 7fdf0579b700 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aecea0bc60 0x55aecea0e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.944+0000 7fdee8ff9700 10 client.26114 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:49.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.944+0000 7fdf0579b700 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aecea0bc60 0x55aecea0e070 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29906796e8ed002 server_cookie=5766e4c36acf8b43 in_seq=0 out_seq=0 2023-07-14T11:24:49.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdee8ff9700 1 -- 172.21.15.161:0/745763823 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aece933eb0 con 0x55aecea0bc60 2023-07-14T11:24:49.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdee8ff9700 10 client.26114 handle_command_reply: tid=0 2023-07-14T11:24:49.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 client.26114 shutdown 2023-07-14T11:24:49.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 msgr2=0x55aece9f94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9f94a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdef80c9e90 tx=0x7fdef8099440 comp rx=0 tx=0).stop 2023-07-14T11:24:49.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 shutdown_connections 2023-07-14T11:24:49.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aecea0bc60 0x55aecea0e070 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9fc2e0 0x55aece9f94a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aece9fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/745763823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e8be0 0x55aece9f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdee8ff9700 0 client.26114 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:49.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.945+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 >> 172.21.15.161:0/745763823 conn(0x55aece9deda0 msgr2=0x55aece9e4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:49.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.946+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 shutdown_connections 2023-07-14T11:24:49.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.946+0000 7fdf1b230b80 1 -- 172.21.15.161:0/745763823 wait complete. 2023-07-14T11:24:49.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.946+0000 7fdf1b230b80 20 client.26114 trim_cache size 0 max 16384 2023-07-14T11:24:49.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 Processor -- start 2023-07-14T11:24:49.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 -- start start 2023-07-14T11:24:49.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9deda0 0x55aecea02cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e82d0 0x55aece9fb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aece9f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece9e0ec0 con 0x55aece9e82d0 2023-07-14T11:24:49.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece9e9e30 con 0x55aece9e8be0 2023-07-14T11:24:49.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aece9e5d20 con 0x55aece9deda0 2023-07-14T11:24:49.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.964+0000 7fdf05f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aece9f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf04f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e82d0 0x55aece9fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf0579b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9deda0 0x55aecea02cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aece9f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35170/0 (socket says 172.21.15.161:35170) 2023-07-14T11:24:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 -- 172.21.15.161:0/4158485148 learned_addr learned my addr 172.21.15.161:0/4158485148 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 -- 172.21.15.161:0/4158485148 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9deda0 msgr2=0x55aecea02cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/4158485148 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9deda0 0x55aecea02cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 -- 172.21.15.161:0/4158485148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e82d0 msgr2=0x55aece9fb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/4158485148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e82d0 0x55aece9fb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 -- 172.21.15.161:0/4158485148 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aece78f250 con 0x55aece9e8be0 2023-07-14T11:24:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/4158485148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aece9f41a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdef8004e20 tx=0x7fdef8099dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20bb5659535a397 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdeebfff700 1 -- 172.21.15.161:0/4158485148 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8099810 con 0x55aece9e8be0 2023-07-14T11:24:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdeebfff700 1 -- 172.21.15.161:0/4158485148 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef800f040 con 0x55aece9e8be0 2023-07-14T11:24:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.965+0000 7fdeebfff700 1 -- 172.21.15.161:0/4158485148 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80a17d0 con 0x55aece9e8be0 2023-07-14T11:24:49.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 -- 172.21.15.161:0/4158485148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 msgr2=0x55aece9f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/4158485148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aece9f41a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdef8004e20 tx=0x7fdef8099dd0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 -- 172.21.15.161:0/4158485148 shutdown_connections 2023-07-14T11:24:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/4158485148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aece9f41a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/4158485148 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9deda0 0x55aecea02cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/4158485148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9e82d0 0x55aece9fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 -- 172.21.15.161:0/4158485148 >> 172.21.15.161:0/4158485148 conn(0x55aece9fe830 msgr2=0x55aecea07460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 -- 172.21.15.161:0/4158485148 shutdown_connections 2023-07-14T11:24:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.966+0000 7fdf1b230b80 1 -- 172.21.15.161:0/4158485148 wait complete. 2023-07-14T11:24:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 -- start start 2023-07-14T11:24:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 Processor -- start 2023-07-14T11:24:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9deda0 0x55aecea02a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aecea00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aecea011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecea032d0 con 0x55aece9deda0 2023-07-14T11:24:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecea01700 con 0x55aece9e8be0 2023-07-14T11:24:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecea018c0 con 0x55aece9e82d0 2023-07-14T11:24:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.967+0000 7fdf1b230b80 10 client.0 _ll_register_callbacks cb 0x7fdef8095c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf0579b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9deda0 0x55aecea02a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf04f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aecea00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf0579b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9deda0 0x55aecea02a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59096/0 (socket says 172.21.15.161:59096) 2023-07-14T11:24:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf0579b700 1 -- 172.21.15.161:0/65576786 learned_addr learned my addr 172.21.15.161:0/65576786 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aecea011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf04f9a700 1 -- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 msgr2=0x55aecea011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aecea011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf04f9a700 1 -- 172.21.15.161:0/65576786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9deda0 msgr2=0x55aecea02a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9deda0 0x55aecea02a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf04f9a700 1 -- 172.21.15.161:0/65576786 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aece751c60 con 0x55aece9e82d0 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf05f9c700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aecea011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf0579b700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9deda0 0x55aecea02a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdf04f9a700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aecea00c80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55aece9e3550 tx=0x7fdef00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d59716c40fd465 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.968+0000 7fdee8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdee8ff9700 1 -- 172.21.15.161:0/65576786 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0026070 con 0x55aece9e82d0 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdefc01d040 con 0x55aece9e82d0 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdee8ff9700 1 -- 172.21.15.161:0/65576786 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fdef0020620 con 0x55aece9e82d0 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdee8ff9700 1 -- 172.21.15.161:0/65576786 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef0020800 con 0x55aece9e82d0 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 10 client.26116 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55aece7672b0 con 0x55aece9e82d0 2023-07-14T11:24:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdee8ff9700 1 -- 172.21.15.161:0/65576786 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc01d040 con 0x55aece9e82d0 2023-07-14T11:24:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdee8ff9700 1 -- 172.21.15.161:0/65576786 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdef002a990 con 0x55aece9e82d0 2023-07-14T11:24:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 20 client.26116 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aece9fc2e0 0x55aecea62ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 4 client.26116 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:24:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.969+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aece8f6050 con 0x55aece9fc2e0 2023-07-14T11:24:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf0579b700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aece9fc2e0 0x55aecea62ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdee8ff9700 10 client.26116 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf0579b700 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aece9fc2e0 0x55aecea62ff0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b05e287d2a9710 server_cookie=5e277702e950570b in_seq=0 out_seq=0 2023-07-14T11:24:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdee8ff9700 1 -- 172.21.15.161:0/65576786 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aece8f6050 con 0x55aece9fc2e0 2023-07-14T11:24:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdee8ff9700 10 client.26116 handle_command_reply: tid=0 2023-07-14T11:24:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 client.26116 shutdown 2023-07-14T11:24:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 msgr2=0x55aecea00c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aecea00c80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55aece9e3550 tx=0x7fdef00047c0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 shutdown_connections 2023-07-14T11:24:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aece9fc2e0 0x55aecea62ff0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aece9deda0 0x55aecea02a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdee8ff9700 0 client.26116 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:49.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aece9e8be0 0x55aecea011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 --2- 172.21.15.161:0/65576786 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aece9e82d0 0x55aecea00c80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.970+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 >> 172.21.15.161:0/65576786 conn(0x55aece9fe830 msgr2=0x55aece9fb850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 shutdown_connections 2023-07-14T11:24:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf1b230b80 1 -- 172.21.15.161:0/65576786 wait complete. 2023-07-14T11:24:49.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf1b230b80 20 client.26116 trim_cache size 0 max 16384 2023-07-14T11:24:49.987 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:24:49.988 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:24:49.988 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:24:49.988 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:24:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdeec0a12c0 msgr2=0x7fdeec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdeec0a12c0 0x7fdeec0a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf00199460 tx=0x7fdef0009630 comp rx=0 tx=0).stop 2023-07-14T11:24:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00105af0 msgr2=0x7fdf0019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00105af0 0x7fdf0019c7f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c110 tx=0x7fdef800c420 comp rx=0 tx=0).stop 2023-07-14T11:24:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 shutdown_connections 2023-07-14T11:24:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdeec0a12c0 0x7fdeec0a3770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf000ffae0 0x7fdf00197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf00105af0 0x7fdf0019c7f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 --2- 172.21.15.161:0/2424586930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdf00100490 0x7fdf00198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.971+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 >> 172.21.15.161:0/2424586930 conn(0x7fdf000fb5e0 msgr2=0x7fdf0010aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.972+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 shutdown_connections 2023-07-14T11:24:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:49.972+0000 7fdf079ff700 1 -- 172.21.15.161:0/2424586930 wait complete. 2023-07-14T11:24:49.994 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:24:49.994 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:24:50.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 177637 10.62 MB/sec warmup 92 sec latency 21.931 ms 2023-07-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 -- 172.21.15.161:0/4146929949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 msgr2=0x7fc8180fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 --2- 172.21.15.161:0/4146929949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 0x7fc8180fff60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0098f0 tx=0x7fc80c009c00 comp rx=0 tx=0).stop 2023-07-14T11:24:50.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 -- 172.21.15.161:0/4146929949 shutdown_connections 2023-07-14T11:24:50.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 --2- 172.21.15.161:0/4146929949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc8181004a0 0x7fc81810c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 --2- 172.21.15.161:0/4146929949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc818105b00 0x7fc818105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 --2- 172.21.15.161:0/4146929949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 0x7fc8180fff60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 -- 172.21.15.161:0/4146929949 >> 172.21.15.161:0/4146929949 conn(0x7fc8180fb630 msgr2=0x7fc8180fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 -- 172.21.15.161:0/4146929949 shutdown_connections 2023-07-14T11:24:50.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.205+0000 7fc81cdff700 1 -- 172.21.15.161:0/4146929949 wait complete. 2023-07-14T11:24:50.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 Processor -- start 2023-07-14T11:24:50.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 -- start start 2023-07-14T11:24:50.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 0x7fc81819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8181004a0 0x7fc81819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc818105b00 0x7fc818195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818196110 con 0x7fc8180ffaf0 2023-07-14T11:24:50.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818196280 con 0x7fc8181004a0 2023-07-14T11:24:50.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81cdff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818196440 con 0x7fc818105b00 2023-07-14T11:24:50.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8181004a0 0x7fc81819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc81659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 0x7fc81819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8181004a0 0x7fc81819bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35194/0 (socket says 172.21.15.161:35194) 2023-07-14T11:24:50.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc815d9b700 1 -- 172.21.15.161:0/455172949 learned_addr learned my addr 172.21.15.161:0/455172949 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:50.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.207+0000 7fc816d9d700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc818105b00 0x7fc818195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc815d9b700 1 -- 172.21.15.161:0/455172949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc818105b00 msgr2=0x7fc818195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc815d9b700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc818105b00 0x7fc818195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc815d9b700 1 -- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 msgr2=0x7fc81819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc815d9b700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 0x7fc81819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc815d9b700 1 -- 172.21.15.161:0/455172949 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080095f0 con 0x7fc8181004a0 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc81659c700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 0x7fc81819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc815d9b700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8181004a0 0x7fc81819bf10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009810 tx=0x7fc80c00fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc8077fe700 1 -- 172.21.15.161:0/455172949 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c01c070 con 0x7fc8181004a0 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc8077fe700 1 -- 172.21.15.161:0/455172949 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80c00bba0 con 0x7fc8181004a0 2023-07-14T11:24:50.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c009520 con 0x7fc8181004a0 2023-07-14T11:24:50.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc8077fe700 1 -- 172.21.15.161:0/455172949 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0049c0 con 0x7fc8181004a0 2023-07-14T11:24:50.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.208+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818072740 con 0x7fc8181004a0 2023-07-14T11:24:50.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.209+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc818067300 con 0x7fc8181004a0 2023-07-14T11:24:50.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.211+0000 7fc8077fe700 1 -- 172.21.15.161:0/455172949 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc80c020070 con 0x7fc8181004a0 2023-07-14T11:24:50.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.213+0000 7fc8077fe700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8000a1300 0x7fc8000a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.213+0000 7fc8077fe700 1 -- 172.21.15.161:0/455172949 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc80c0cdff0 con 0x7fc8181004a0 2023-07-14T11:24:50.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.213+0000 7fc81659c700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8000a1300 0x7fc8000a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.213+0000 7fc81659c700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8000a1300 0x7fc8000a37b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc808005fd0 tx=0x7fc808009310 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.215+0000 7fc8077fe700 1 -- 172.21.15.161:0/455172949 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc80c097140 con 0x7fc8181004a0 2023-07-14T11:24:50.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:50 smithi175 ceph-mon[129081]: pgmap v80: 129 pgs: 129 active+clean; 99 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-14T11:24:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:50 smithi182 ceph-mon[129366]: pgmap v80: 129 pgs: 129 active+clean; 99 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-14T11:24:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:50 smithi161 ceph-mon[95526]: pgmap v80: 129 pgs: 129 active+clean; 99 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-14T11:24:50.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.519+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc818072990 con 0x7fc8181004a0 2023-07-14T11:24:50.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.520+0000 7fc8077fe700 1 -- 172.21.15.161:0/455172949 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10867 (secure 0 0 0) 0x7fc80c09a760 con 0x7fc8181004a0 2023-07-14T11:24:50.524 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:24:50.524 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:23:41.117589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:24:50.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8000a1300 msgr2=0x7fc8000a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8000a1300 0x7fc8000a37b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc808005fd0 tx=0x7fc808009310 comp rx=0 tx=0).stop 2023-07-14T11:24:50.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8181004a0 msgr2=0x7fc81819bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8181004a0 0x7fc81819bf10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009810 tx=0x7fc80c00fa70 comp rx=0 tx=0).stop 2023-07-14T11:24:50.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 shutdown_connections 2023-07-14T11:24:50.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8000a1300 0x7fc8000a37b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8180ffaf0 0x7fc81819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc818105b00 0x7fc818195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 --2- 172.21.15.161:0/455172949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8181004a0 0x7fc81819bf10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 >> 172.21.15.161:0/455172949 conn(0x7fc8180fb630 msgr2=0x7fc818103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 shutdown_connections 2023-07-14T11:24:50.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.523+0000 7fc81cdff700 1 -- 172.21.15.161:0/455172949 wait complete. 2023-07-14T11:24:50.529 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2023-07-14T11:24:50.546 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:24:50.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.760+0000 7f9708dff700 1 -- 172.21.15.161:0/1025346759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704072850 msgr2=0x7f9704072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.760+0000 7f9708dff700 1 --2- 172.21.15.161:0/1025346759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704072850 0x7f9704072c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b0a0 tx=0x7f96f400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.760+0000 7f9708dff700 1 -- 172.21.15.161:0/1025346759 shutdown_connections 2023-07-14T11:24:50.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.760+0000 7f9708dff700 1 --2- 172.21.15.161:0/1025346759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970410c3b0 0x7f970410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.760+0000 7f9708dff700 1 --2- 172.21.15.161:0/1025346759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704072850 0x7f9704072c20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.760+0000 7f9708dff700 1 --2- 172.21.15.161:0/1025346759 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704073160 0x7f970410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.760+0000 7f9708dff700 1 -- 172.21.15.161:0/1025346759 >> 172.21.15.161:0/1025346759 conn(0x7f97040fb690 msgr2=0x7f97040fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.761+0000 7f9708dff700 1 -- 172.21.15.161:0/1025346759 shutdown_connections 2023-07-14T11:24:50.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.761+0000 7f9708dff700 1 -- 172.21.15.161:0/1025346759 wait complete. 2023-07-14T11:24:50.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 Processor -- start 2023-07-14T11:24:50.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 -- start start 2023-07-14T11:24:50.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704072850 0x7f9704198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9704073160 0x7f9704198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970410c3b0 0x7f970419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970419d000 con 0x7f9704073160 2023-07-14T11:24:50.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970419d170 con 0x7f970410c3b0 2023-07-14T11:24:50.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f9708dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970419d330 con 0x7f9704072850 2023-07-14T11:24:50.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.762+0000 7f970259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704072850 0x7f9704198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f96fbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9704073160 0x7f9704198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704072850 0x7f9704198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49498/0 (socket says 172.21.15.161:49498) 2023-07-14T11:24:50.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 -- 172.21.15.161:0/1898739637 learned_addr learned my addr 172.21.15.161:0/1898739637 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:50.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f9702d9d700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970410c3b0 0x7f970419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 -- 172.21.15.161:0/1898739637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970410c3b0 msgr2=0x7f970419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970410c3b0 0x7f970419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 -- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9704073160 msgr2=0x7f9704198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9704073160 0x7f9704198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 -- 172.21.15.161:0/1898739637 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc009520 con 0x7f9704072850 2023-07-14T11:24:50.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f970259c700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704072850 0x7f9704198020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b070 tx=0x7f96f400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f96fb7fe700 1 -- 172.21.15.161:0/1898739637 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f400e040 con 0x7f9704072850 2023-07-14T11:24:50.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f96fb7fe700 1 -- 172.21.15.161:0/1898739637 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96f4009480 con 0x7f9704072850 2023-07-14T11:24:50.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f96fb7fe700 1 -- 172.21.15.161:0/1898739637 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f401b6c0 con 0x7f9704072850 2023-07-14T11:24:50.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.763+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f4009d00 con 0x7f9704072850 2023-07-14T11:24:50.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.764+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970419d840 con 0x7f9704072850 2023-07-14T11:24:50.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.767+0000 7f96fb7fe700 1 -- 172.21.15.161:0/1898739637 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f96f401d470 con 0x7f9704072850 2023-07-14T11:24:50.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.768+0000 7f971c762b80 1 Processor -- start 2023-07-14T11:24:50.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.768+0000 7f971c762b80 1 -- start start 2023-07-14T11:24:50.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.768+0000 7f971c762b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f971c762b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f971c762b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f971c762b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc222ebcb0 con 0x55bc223e5be0 2023-07-14T11:24:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f971c762b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2233ea40 con 0x55bc223e52d0 2023-07-14T11:24:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f971c762b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc22337ed0 con 0x55bc223f0ad0 2023-07-14T11:24:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223e56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35234/0 (socket says 172.21.15.161:35234) 2023-07-14T11:24:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 -- 172.21.15.161:0/3786551120 learned_addr learned my addr 172.21.15.161:0/3786551120 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f970259c700 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f96fbfff700 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 -- 172.21.15.161:0/3786551120 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 msgr2=0x55bc223f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f96fb7fe700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f00a1340 0x7f96f00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 -- 172.21.15.161:0/3786551120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 msgr2=0x55bc223f0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223f0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f9702d9d700 1 -- 172.21.15.161:0/3786551120 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc22165de0 con 0x55bc223e52d0 2023-07-14T11:24:50.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f970259c700 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223f0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:50.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.769+0000 7f96fb7fe700 1 -- 172.21.15.161:0/1898739637 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f96f40cdff0 con 0x7f9704072850 2023-07-14T11:24:50.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.770+0000 7f9702d9d700 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223e56a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009db0 tx=0x7f96fc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba6517c29c6473c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.770+0000 7f9702d9d700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f00a1340 0x7f96f00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.770+0000 7f96f8ff9700 1 -- 172.21.15.161:0/3786551120 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0043b0 con 0x55bc223e52d0 2023-07-14T11:24:50.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.770+0000 7f96f8ff9700 1 -- 172.21.15.161:0/3786551120 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96fc004510 con 0x55bc223e52d0 2023-07-14T11:24:50.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.770+0000 7f96f8ff9700 1 -- 172.21.15.161:0/3786551120 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc01ac30 con 0x55bc223e52d0 2023-07-14T11:24:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 -- 172.21.15.161:0/3786551120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 msgr2=0x55bc223e56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223e56a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009db0 tx=0x7f96fc011720 comp rx=0 tx=0).stop 2023-07-14T11:24:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f9702d9d700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f00a1340 0x7f96f00a37f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96fc004020 tx=0x7f96fc01ad90 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 -- 172.21.15.161:0/3786551120 shutdown_connections 2023-07-14T11:24:50.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223e56a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 --2- 172.21.15.161:0/3786551120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223f0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 -- 172.21.15.161:0/3786551120 >> 172.21.15.161:0/3786551120 conn(0x55bc223dbda0 msgr2=0x55bc223e16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 -- 172.21.15.161:0/3786551120 shutdown_connections 2023-07-14T11:24:50.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.771+0000 7f971c762b80 1 -- 172.21.15.161:0/3786551120 wait complete. 2023-07-14T11:24:50.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.772+0000 7f971c762b80 1 -- start start 2023-07-14T11:24:50.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.772+0000 7f971c762b80 1 Processor -- start 2023-07-14T11:24:50.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.772+0000 7f971c762b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.772+0000 7f971c762b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223ed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f971c762b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f971c762b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc222d7ac0 con 0x55bc223e5be0 2023-07-14T11:24:50.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f971c762b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc222d77c0 con 0x55bc223e52d0 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f971c762b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc221bb920 con 0x55bc223f0ad0 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f971c762b80 10 client.0 _ll_register_callbacks cb 0x55bc222d5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223f0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35238/0 (socket says 172.21.15.161:35238) 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 -- 172.21.15.161:0/3653909278 learned_addr learned my addr 172.21.15.161:0/3653909278 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f96fbfff700 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f9702d9d700 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 -- 172.21.15.161:0/3653909278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 msgr2=0x55bc223ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 -- 172.21.15.161:0/3653909278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 msgr2=0x55bc223ed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223ed710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.773+0000 7f970259c700 1 -- 172.21.15.161:0/3653909278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc009a60 con 0x55bc223e52d0 2023-07-14T11:24:50.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f970259c700 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223f0500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96f4097870 tx=0x7f96f4009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0000ea36785ae14 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f96e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:50.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f96e9ffb700 1 -- 172.21.15.161:0/3653909278 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f409b260 con 0x55bc223e52d0 2023-07-14T11:24:50.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc222914f0 con 0x55bc223e52d0 2023-07-14T11:24:50.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f96e9ffb700 1 -- 172.21.15.161:0/3653909278 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f96f4003e10 con 0x55bc223e52d0 2023-07-14T11:24:50.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f96e9ffb700 1 -- 172.21.15.161:0/3653909278 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96f409a940 con 0x55bc223e52d0 2023-07-14T11:24:50.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f971c762b80 10 client.26138 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:50.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bc2214ec60 con 0x55bc223e52d0 2023-07-14T11:24:50.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f96e9ffb700 1 -- 172.21.15.161:0/3653909278 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc222914f0 con 0x55bc223e52d0 2023-07-14T11:24:50.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.774+0000 7f96e9ffb700 1 -- 172.21.15.161:0/3653909278 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f96f409a2f0 con 0x55bc223e52d0 2023-07-14T11:24:50.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f971c762b80 10 client.26138 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:50.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f971c762b80 10 client.26138 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:24:50.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f971c762b80 20 client.26138 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:50.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f971c762b80 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc223fffe0 0x55bc224023f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f971c762b80 4 client.26138 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:50.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc22330eb0 con 0x55bc223fffe0 2023-07-14T11:24:50.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f96fbfff700 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc223fffe0 0x55bc224023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f96fbfff700 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc223fffe0 0x55bc224023f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ff751f30c6e0fcd server_cookie=944dd526b294f5f1 in_seq=0 out_seq=0 2023-07-14T11:24:50.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.775+0000 7f96e9ffb700 10 client.26138 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:50.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f96e9ffb700 1 -- 172.21.15.161:0/3653909278 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bc22330eb0 con 0x55bc223fffe0 2023-07-14T11:24:50.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f96e9ffb700 10 client.26138 handle_command_reply: tid=0 2023-07-14T11:24:50.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 client.26138 shutdown 2023-07-14T11:24:50.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 msgr2=0x55bc223f0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223f0500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96f4097870 tx=0x7f96f4009c80 comp rx=0 tx=0).stop 2023-07-14T11:24:50.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 shutdown_connections 2023-07-14T11:24:50.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc223fffe0 0x55bc224023f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e52d0 0x55bc223f0500 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 --2- 172.21.15.161:0/3653909278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223e5be0 0x55bc223ed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 >> 172.21.15.161:0/3653909278 conn(0x55bc223dbda0 msgr2=0x55bc223e1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.776+0000 7f96e9ffb700 0 client.26138 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:50.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.777+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 shutdown_connections 2023-07-14T11:24:50.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.777+0000 7f971c762b80 1 -- 172.21.15.161:0/3653909278 wait complete. 2023-07-14T11:24:50.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.777+0000 7f971c762b80 20 client.26138 trim_cache size 0 max 16384 2023-07-14T11:24:50.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 Processor -- start 2023-07-14T11:24:50.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 -- start start 2023-07-14T11:24:50.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223e5be0 0x55bc223f0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0ad0 0x55bc223e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223f0570 0x55bc223e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc223e84e0 con 0x55bc223f0ad0 2023-07-14T11:24:50.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc223e61c0 con 0x55bc223f0570 2023-07-14T11:24:50.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.796+0000 7f971c762b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc223ecc80 con 0x55bc223e5be0 2023-07-14T11:24:50.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223e5be0 0x55bc223f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223f0570 0x55bc223e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223e5be0 0x55bc223f0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49526/0 (socket says 172.21.15.161:49526) 2023-07-14T11:24:50.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 -- 172.21.15.161:0/2019775807 learned_addr learned my addr 172.21.15.161:0/2019775807 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:50.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223f0570 0x55bc223e8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35240/0 (socket says 172.21.15.161:35240) 2023-07-14T11:24:50.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f96fbfff700 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0ad0 0x55bc223e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 -- 172.21.15.161:0/2019775807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223f0570 msgr2=0x55bc223e8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223f0570 0x55bc223e8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 -- 172.21.15.161:0/2019775807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0ad0 msgr2=0x55bc223e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0ad0 0x55bc223e89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 -- 172.21.15.161:0/2019775807 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc2218c250 con 0x55bc223e5be0 2023-07-14T11:24:50.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.797+0000 7f970259c700 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223e5be0 0x55bc223f0030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96f401f040 tx=0x7f96f4097140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950a3b7051622471 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.798+0000 7f96f97fa700 1 -- 172.21.15.161:0/2019775807 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f409a940 con 0x55bc223e5be0 2023-07-14T11:24:50.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.798+0000 7f96f97fa700 1 -- 172.21.15.161:0/2019775807 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96f4003e10 con 0x55bc223e5be0 2023-07-14T11:24:50.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.798+0000 7f96f97fa700 1 -- 172.21.15.161:0/2019775807 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f40a31c0 con 0x55bc223e5be0 2023-07-14T11:24:50.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 -- 172.21.15.161:0/2019775807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223e5be0 msgr2=0x55bc223f0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223e5be0 0x55bc223f0030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96f401f040 tx=0x7f96f4097140 comp rx=0 tx=0).stop 2023-07-14T11:24:50.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 -- 172.21.15.161:0/2019775807 shutdown_connections 2023-07-14T11:24:50.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223f0570 0x55bc223e8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223e5be0 0x55bc223f0030 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 --2- 172.21.15.161:0/2019775807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0ad0 0x55bc223e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 -- 172.21.15.161:0/2019775807 >> 172.21.15.161:0/2019775807 conn(0x55bc223dbda0 msgr2=0x55bc223fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 -- 172.21.15.161:0/2019775807 shutdown_connections 2023-07-14T11:24:50.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 -- 172.21.15.161:0/2019775807 wait complete. 2023-07-14T11:24:50.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.799+0000 7f971c762b80 1 -- start start 2023-07-14T11:24:50.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 1 Processor -- start 2023-07-14T11:24:50.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e5be0 0x55bc223f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0570 0x55bc223f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc223f5b60 con 0x55bc223f0570 2023-07-14T11:24:50.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc223f5cd0 con 0x55bc223e5be0 2023-07-14T11:24:50.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc223f5e90 con 0x55bc223f0ad0 2023-07-14T11:24:50.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.800+0000 7f971c762b80 10 client.0 _ll_register_callbacks cb 0x55bc222d5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:50.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e5be0 0x55bc223f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e5be0 0x55bc223f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35246/0 (socket says 172.21.15.161:35246) 2023-07-14T11:24:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 -- 172.21.15.161:0/2454900836 learned_addr learned my addr 172.21.15.161:0/2454900836 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f96fbfff700 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0570 0x55bc223f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 -- 172.21.15.161:0/2454900836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 msgr2=0x55bc223f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 -- 172.21.15.161:0/2454900836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0570 msgr2=0x55bc223f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0570 0x55bc223f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 -- 172.21.15.161:0/2454900836 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc2214ec60 con 0x55bc223e5be0 2023-07-14T11:24:50.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f970259c700 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e5be0 0x55bc223f6f60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b3f0 tx=0x7f96f40a30b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf972d9bbe11f5d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:50.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.801+0000 7f96e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:50.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f96e9ffb700 1 -- 172.21.15.161:0/2454900836 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f409a940 con 0x55bc223e5be0 2023-07-14T11:24:50.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f96e9ffb700 1 -- 172.21.15.161:0/2454900836 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f96f401d930 con 0x55bc223e5be0 2023-07-14T11:24:50.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96fc004510 con 0x55bc223e5be0 2023-07-14T11:24:50.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f96e9ffb700 1 -- 172.21.15.161:0/2454900836 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96f40cb2d0 con 0x55bc223e5be0 2023-07-14T11:24:50.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 10 client.26144 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:50.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bc221642b0 con 0x55bc223e5be0 2023-07-14T11:24:50.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f96e9ffb700 1 -- 172.21.15.161:0/2454900836 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96fc004510 con 0x55bc223e5be0 2023-07-14T11:24:50.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f96e9ffb700 1 -- 172.21.15.161:0/2454900836 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f96f4096e20 con 0x55bc223e5be0 2023-07-14T11:24:50.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 10 client.26144 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:50.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 10 client.26144 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:24:50.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 20 client.26144 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:50.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc22454be0 0x55bc22456fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:50.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 4 client.26144 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:24:50.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.802+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc222f3050 con 0x55bc22454be0 2023-07-14T11:24:50.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.803+0000 7f96fbfff700 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc22454be0 0x55bc22456fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:50.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.803+0000 7f96e9ffb700 10 client.26144 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:50.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.803+0000 7f96fbfff700 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc22454be0 0x55bc22456fc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f4f88b07ff098c2 server_cookie=c2a2ee6ba31d45b6 in_seq=0 out_seq=0 2023-07-14T11:24:50.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.803+0000 7f96e9ffb700 1 -- 172.21.15.161:0/2454900836 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc222f3050 con 0x55bc22454be0 2023-07-14T11:24:50.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.803+0000 7f96e9ffb700 10 client.26144 handle_command_reply: tid=0 2023-07-14T11:24:50.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.803+0000 7f971c762b80 1 client.26144 shutdown 2023-07-14T11:24:50.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e5be0 msgr2=0x55bc223f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e5be0 0x55bc223f6f60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b3f0 tx=0x7f96f40a30b0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 shutdown_connections 2023-07-14T11:24:50.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55bc22454be0 0x55bc22456fc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bc223f0ad0 0x55bc223f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc223e5be0 0x55bc223f6f60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 --2- 172.21.15.161:0/2454900836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc223f0570 0x55bc223f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f96e9ffb700 0 client.26144 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:24:50.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 >> 172.21.15.161:0/2454900836 conn(0x55bc223dbda0 msgr2=0x55bc223e89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 shutdown_connections 2023-07-14T11:24:50.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 1 -- 172.21.15.161:0/2454900836 wait complete. 2023-07-14T11:24:50.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.804+0000 7f971c762b80 20 client.26144 trim_cache size 0 max 16384 2023-07-14T11:24:50.818 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:24:50.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f00a1340 msgr2=0x7f96f00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f00a1340 0x7f96f00a37f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96fc004020 tx=0x7f96fc01ad90 comp rx=0 tx=0).stop 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704072850 msgr2=0x7f9704198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704072850 0x7f9704198020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b070 tx=0x7f96f400bce0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 shutdown_connections 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f00a1340 0x7f96f00a37f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970410c3b0 0x7f970419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9704072850 0x7f9704198020 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 --2- 172.21.15.161:0/1898739637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9704073160 0x7f9704198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:50.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 >> 172.21.15.161:0/1898739637 conn(0x7f97040fb690 msgr2=0x7f970410e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:50.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 shutdown_connections 2023-07-14T11:24:50.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:50.805+0000 7f9708dff700 1 -- 172.21.15.161:0/1898739637 wait complete. 2023-07-14T11:24:50.827 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:24:51.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.090+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/3357063013 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 msgr2=0x7fc5b40fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.090+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/3357063013 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 0x7fc5b40fee40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40098f0 tx=0x7fc5a4009c00 comp rx=0 tx=0).stop 2023-07-14T11:24:51.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.091+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/3357063013 shutdown_connections 2023-07-14T11:24:51.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.091+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/3357063013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5b40ff380 0x7fc5b410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.091+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/3357063013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b41049e0 0x7fc5b4104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.091+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/3357063013 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 0x7fc5b40fee40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.091+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/3357063013 >> 172.21.15.161:0/3357063013 conn(0x7fc5b40fa4b0 msgr2=0x7fc5b40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.091+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/3357063013 shutdown_connections 2023-07-14T11:24:51.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.091+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/3357063013 wait complete. 2023-07-14T11:24:51.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 Processor -- start 2023-07-14T11:24:51.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 -- start start 2023-07-14T11:24:51.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 0x7fc5b4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b40ff380 0x7fc5b4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5b41049e0 0x7fc5b419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419cdc0 con 0x7fc5b41049e0 2023-07-14T11:24:51.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419cf30 con 0x7fc5b40ff380 2023-07-14T11:24:51.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.092+0000 7fc5bb5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419d0f0 con 0x7fc5b40fe9d0 2023-07-14T11:24:51.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b40ff380 0x7fc5b4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b40ff380 0x7fc5b4198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35264/0 (socket says 172.21.15.161:35264) 2023-07-14T11:24:51.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 -- 172.21.15.161:0/4270098389 learned_addr learned my addr 172.21.15.161:0/4270098389 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b939b700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 0x7fc5b4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b9b9c700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5b41049e0 0x7fc5b419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 -- 172.21.15.161:0/4270098389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 msgr2=0x7fc5b4197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 0x7fc5b4197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 -- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5b41049e0 msgr2=0x7fc5b419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5b41049e0 0x7fc5b419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 -- 172.21.15.161:0/4270098389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4009520 con 0x7fc5b40ff380 2023-07-14T11:24:51.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.093+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b40ff380 0x7fc5b4198320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009810 tx=0x7fc5a400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.094+0000 7fc5aa7fc700 1 -- 172.21.15.161:0/4270098389 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a401c070 con 0x7fc5b40ff380 2023-07-14T11:24:51.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.094+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b419d370 con 0x7fc5b40ff380 2023-07-14T11:24:51.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.094+0000 7fc5aa7fc700 1 -- 172.21.15.161:0/4270098389 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a400bbd0 con 0x7fc5b40ff380 2023-07-14T11:24:51.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.094+0000 7fc5aa7fc700 1 -- 172.21.15.161:0/4270098389 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4004930 con 0x7fc5b40ff380 2023-07-14T11:24:51.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.094+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b41a4c10 con 0x7fc5b40ff380 2023-07-14T11:24:51.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.097+0000 7fc5aa7fc700 1 -- 172.21.15.161:0/4270098389 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc5a4020070 con 0x7fc5b40ff380 2023-07-14T11:24:51.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.098+0000 7fc5cee60b80 1 Processor -- start 2023-07-14T11:24:51.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5cee60b80 1 -- start start 2023-07-14T11:24:51.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff82d0 0x5606f4ff86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff8be0 0x5606f5003590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f5005eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4efecb0 con 0x5606f5003ad0 2023-07-14T11:24:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4f51a40 con 0x5606f4ff8be0 2023-07-14T11:24:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4f4aed0 con 0x5606f4ff82d0 2023-07-14T11:24:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff8be0 0x5606f5003590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5b9b9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff82d0 0x5606f4ff86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff8be0 0x5606f5003590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35272/0 (socket says 172.21.15.161:35272) 2023-07-14T11:24:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5b9b9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff82d0 0x5606f4ff86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49568/0 (socket says 172.21.15.161:49568) 2023-07-14T11:24:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5b939b700 1 -- 172.21.15.161:0/2228395517 learned_addr learned my addr 172.21.15.161:0/2228395517 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5aa7fc700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc5a00a1340 0x7fc5a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.099+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f5005eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b9b9c700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc5a00a1340 0x7fc5a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5aa7fc700 1 -- 172.21.15.161:0/4270098389 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc5a40cdc40 con 0x7fc5b40ff380 2023-07-14T11:24:51.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b939b700 1 -- 172.21.15.161:0/2228395517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff82d0 msgr2=0x5606f4ff86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b939b700 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff82d0 0x5606f4ff86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b939b700 1 -- 172.21.15.161:0/2228395517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 msgr2=0x5606f5005eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b939b700 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f5005eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b939b700 1 -- 172.21.15.161:0/2228395517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606f4d78de0 con 0x5606f4ff8be0 2023-07-14T11:24:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b939b700 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff8be0 0x5606f5003590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009d30 tx=0x7fc5b0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48bd6ec5f672a872 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc59f7fe700 1 -- 172.21.15.161:0/2228395517 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0004220 con 0x5606f4ff8be0 2023-07-14T11:24:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc59f7fe700 1 -- 172.21.15.161:0/2228395517 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b0009e80 con 0x5606f4ff8be0 2023-07-14T11:24:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc59f7fe700 1 -- 172.21.15.161:0/2228395517 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0005480 con 0x5606f4ff8be0 2023-07-14T11:24:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.100+0000 7fc5b9b9c700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc5a00a1340 0x7fc5a00a37f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009ec0 tx=0x7fc5ac004b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2228395517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff8be0 msgr2=0x5606f5003590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff8be0 0x5606f5003590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009d30 tx=0x7fc5b0011720 comp rx=0 tx=0).stop 2023-07-14T11:24:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2228395517 shutdown_connections 2023-07-14T11:24:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f5005eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff82d0 0x5606f4ff86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2228395517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff8be0 0x5606f5003590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2228395517 >> 172.21.15.161:0/2228395517 conn(0x5606f4feeda0 msgr2=0x5606f4ff46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2228395517 shutdown_connections 2023-07-14T11:24:51.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.101+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2228395517 wait complete. 2023-07-14T11:24:51.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.102+0000 7fc5cee60b80 1 -- start start 2023-07-14T11:24:51.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.102+0000 7fc5cee60b80 1 Processor -- start 2023-07-14T11:24:51.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5003550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f4ff8be0 0x5606f5000760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f5003ad0 0x5606f5000e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4eeaac0 con 0x5606f4ff8be0 2023-07-14T11:24:51.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4eea7c0 con 0x5606f4ff82d0 2023-07-14T11:24:51.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4dce920 con 0x5606f5003ad0 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5cee60b80 10 client.0 _ll_register_callbacks cb 0x5606f4ee8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5003550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5003550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35288/0 (socket says 172.21.15.161:35288) 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5b939b700 1 -- 172.21.15.161:0/572675506 learned_addr learned my addr 172.21.15.161:0/572675506 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5b9b9c700 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f5003ad0 0x5606f5000e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.103+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f4ff8be0 0x5606f5000760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc5b939b700 1 -- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f5003ad0 msgr2=0x5606f5000e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc5b939b700 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f5003ad0 0x5606f5000e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc5b939b700 1 -- 172.21.15.161:0/572675506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f4ff8be0 msgr2=0x5606f5000760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc5b939b700 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f4ff8be0 0x5606f5000760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc5b939b700 1 -- 172.21.15.161:0/572675506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b0009960 con 0x5606f4ff82d0 2023-07-14T11:24:51.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc5b939b700 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5003550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0005920 tx=0x7fc5b0005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf231aded701e4b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc59cff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:51.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc59cff9700 1 -- 172.21.15.161:0/572675506 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0005710 con 0x5606f4ff82d0 2023-07-14T11:24:51.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc59cff9700 1 -- 172.21.15.161:0/572675506 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc5b0009e80 con 0x5606f4ff82d0 2023-07-14T11:24:51.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.104+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606f4f43eb0 con 0x5606f4ff82d0 2023-07-14T11:24:51.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc59cff9700 1 -- 172.21.15.161:0/572675506 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5b0004520 con 0x5606f4ff82d0 2023-07-14T11:24:51.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 10 client.26156 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:51.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5606f4d61c60 con 0x5606f4ff82d0 2023-07-14T11:24:51.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc59cff9700 1 -- 172.21.15.161:0/572675506 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606f4f43eb0 con 0x5606f4ff82d0 2023-07-14T11:24:51.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc59cff9700 1 -- 172.21.15.161:0/572675506 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc5b0003af0 con 0x5606f4ff82d0 2023-07-14T11:24:51.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 10 client.26156 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:51.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 10 client.26156 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:24:51.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 20 client.26156 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:51.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5013230 0x5606f5015640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 4 client.26156 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:51.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606f4f06050 con 0x5606f5013230 2023-07-14T11:24:51.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.105+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5013230 0x5606f5015640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.106+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5013230 0x5606f5015640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=408275dd680771a6 server_cookie=cf9d734d158720b5 in_seq=0 out_seq=0 2023-07-14T11:24:51.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.106+0000 7fc59cff9700 10 client.26156 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:51.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.106+0000 7fc59cff9700 1 -- 172.21.15.161:0/572675506 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5606f4f06050 con 0x5606f5013230 2023-07-14T11:24:51.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.106+0000 7fc59cff9700 10 client.26156 handle_command_reply: tid=0 2023-07-14T11:24:51.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 client.26156 shutdown 2023-07-14T11:24:51.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 msgr2=0x5606f5003550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5003550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0005920 tx=0x7fc5b0005230 comp rx=0 tx=0).stop 2023-07-14T11:24:51.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 shutdown_connections 2023-07-14T11:24:51.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5013230 0x5606f5015640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f5003ad0 0x5606f5000e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5003550 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc59cff9700 0 client.26156 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:51.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/572675506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f4ff8be0 0x5606f5000760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 >> 172.21.15.161:0/572675506 conn(0x5606f4feeda0 msgr2=0x5606f4ff4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 shutdown_connections 2023-07-14T11:24:51.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 1 -- 172.21.15.161:0/572675506 wait complete. 2023-07-14T11:24:51.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.107+0000 7fc5cee60b80 20 client.26156 trim_cache size 0 max 16384 2023-07-14T11:24:51.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 Processor -- start 2023-07-14T11:24:51.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 -- start start 2023-07-14T11:24:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f500a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f500ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f5001450 con 0x5606f5003ad0 2023-07-14T11:24:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4ff9dd0 con 0x5606f4ff82d0 2023-07-14T11:24:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.121+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f500a9e0 con 0x5606f4ff8be0 2023-07-14T11:24:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5005d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35294/0 (socket says 172.21.15.161:35294) 2023-07-14T11:24:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 -- 172.21.15.161:0/2008104296 learned_addr learned my addr 172.21.15.161:0/2008104296 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b9b9c700 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f500ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f500a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 -- 172.21.15.161:0/2008104296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 msgr2=0x5606f500a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f500a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 -- 172.21.15.161:0/2008104296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 msgr2=0x5606f500ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f500ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 -- 172.21.15.161:0/2008104296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606f4d9f250 con 0x5606f4ff82d0 2023-07-14T11:24:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.122+0000 7fc5b939b700 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5005d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0000c00 tx=0x7fc5b0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b84ef509ae5154d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.123+0000 7fc59ffff700 1 -- 172.21.15.161:0/2008104296 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000e5b0 con 0x5606f4ff82d0 2023-07-14T11:24:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.123+0000 7fc59ffff700 1 -- 172.21.15.161:0/2008104296 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b0019580 con 0x5606f4ff82d0 2023-07-14T11:24:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.123+0000 7fc59ffff700 1 -- 172.21.15.161:0/2008104296 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0019770 con 0x5606f4ff82d0 2023-07-14T11:24:51.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.123+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2008104296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 msgr2=0x5606f5005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.123+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5005d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0000c00 tx=0x7fc5b0011720 comp rx=0 tx=0).stop 2023-07-14T11:24:51.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2008104296 shutdown_connections 2023-07-14T11:24:51.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f500ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f5005d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/2008104296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f500a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2008104296 >> 172.21.15.161:0/2008104296 conn(0x5606f4feeda0 msgr2=0x5606f4ff2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2008104296 shutdown_connections 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 -- 172.21.15.161:0/2008104296 wait complete. 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 -- start start 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.124+0000 7fc5cee60b80 1 Processor -- start 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f4ffde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f4ffe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5cee60b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f4ffc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4ffd0f0 con 0x5606f5003ad0 2023-07-14T11:24:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4ffd260 con 0x5606f4ff82d0 2023-07-14T11:24:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5cee60b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f4ffd420 con 0x5606f4ff8be0 2023-07-14T11:24:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5cee60b80 10 client.0 _ll_register_callbacks cb 0x7fc5ac009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f4ffde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f4ffde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35304/0 (socket says 172.21.15.161:35304) 2023-07-14T11:24:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 -- 172.21.15.161:0/3191285846 learned_addr learned my addr 172.21.15.161:0/3191285846 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f4ffe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b9b9c700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f4ffc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 -- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 msgr2=0x5606f4ffe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f4ffe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 -- 172.21.15.161:0/3191285846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 msgr2=0x5606f4ffc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f4ffc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b939b700 1 -- 172.21.15.161:0/3191285846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606f4d61c60 con 0x5606f4ff82d0 2023-07-14T11:24:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f4ffe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.125+0000 7fc5b9b9c700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f4ffc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:24:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc5b939b700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f4ffde10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009c50 tx=0x7fc5b0003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a8c145c5aea9bb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc59cff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc59cff9700 1 -- 172.21.15.161:0/3191285846 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0005350 con 0x5606f4ff82d0 2023-07-14T11:24:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc59cff9700 1 -- 172.21.15.161:0/3191285846 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc5b0000e40 con 0x5606f4ff82d0 2023-07-14T11:24:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606f4ea44f0 con 0x5606f4ff82d0 2023-07-14T11:24:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc59cff9700 1 -- 172.21.15.161:0/3191285846 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5b00054b0 con 0x5606f4ff82d0 2023-07-14T11:24:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc5cee60b80 10 client.26162 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5606f4d772b0 con 0x5606f4ff82d0 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc59cff9700 1 -- 172.21.15.161:0/3191285846 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606f4ea44f0 con 0x5606f4ff82d0 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc59cff9700 1 -- 172.21.15.161:0/3191285846 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc5b00045b0 con 0x5606f4ff82d0 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc5cee60b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc5cee60b80 10 client.26162 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.126+0000 7fc5cee60b80 20 client.26162 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5067d50 0x5606f506a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5cee60b80 4 client.26162 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:24:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5606f4d99f70 con 0x5606f5067d50 2023-07-14T11:24:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5067d50 0x5606f506a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5b8b9a700 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5067d50 0x5606f506a130 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3a5fa59dd1a3813 server_cookie=126fda7691201e51 in_seq=0 out_seq=0 2023-07-14T11:24:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc59cff9700 10 client.26162 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc59cff9700 1 -- 172.21.15.161:0/3191285846 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5606f4d99f70 con 0x5606f5067d50 2023-07-14T11:24:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc59cff9700 10 client.26162 handle_command_reply: tid=0 2023-07-14T11:24:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5cee60b80 1 client.26162 shutdown 2023-07-14T11:24:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 msgr2=0x5606f4ffde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.127+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f4ffde10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009c50 tx=0x7fc5b0003f50 comp rx=0 tx=0).stop 2023-07-14T11:24:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 shutdown_connections 2023-07-14T11:24:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5606f5067d50 0x5606f506a130 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606f5003ad0 0x5606f4ffc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f4ff82d0 0x5606f4ffde10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc59cff9700 0 client.26162 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:24:51.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 --2- 172.21.15.161:0/3191285846 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5606f4ff8be0 0x5606f4ffe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 >> 172.21.15.161:0/3191285846 conn(0x5606f4feeda0 msgr2=0x5606f500ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 shutdown_connections 2023-07-14T11:24:51.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 1 -- 172.21.15.161:0/3191285846 wait complete. 2023-07-14T11:24:51.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5cee60b80 20 client.26162 trim_cache size 0 max 16384 2023-07-14T11:24:51.141 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:24:51.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc5a00a1340 msgr2=0x7fc5a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc5a00a1340 0x7fc5a00a37f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009ec0 tx=0x7fc5ac004b40 comp rx=0 tx=0).stop 2023-07-14T11:24:51.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b40ff380 msgr2=0x7fc5b4198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.128+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b40ff380 0x7fc5b4198320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009810 tx=0x7fc5a400fa00 comp rx=0 tx=0).stop 2023-07-14T11:24:51.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 shutdown_connections 2023-07-14T11:24:51.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc5a00a1340 0x7fc5a00a37f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5b41049e0 0x7fc5b419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc5b40fe9d0 0x7fc5b4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 --2- 172.21.15.161:0/4270098389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5b40ff380 0x7fc5b4198320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 >> 172.21.15.161:0/4270098389 conn(0x7fc5b40fa4b0 msgr2=0x7fc5b40fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 shutdown_connections 2023-07-14T11:24:51.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.129+0000 7fc5bb5ff700 1 -- 172.21.15.161:0/4270098389 wait complete. 2023-07-14T11:24:51.154 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:24:51.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 178603 10.53 MB/sec warmup 93 sec latency 41.182 ms 2023-07-14T11:24:51.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.365+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1839703360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e8100ca0 msgr2=0x7fc4e8101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.365+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1839703360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e8100ca0 0x7fc4e8101070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009990 tx=0x7fc4e4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.366+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1839703360 shutdown_connections 2023-07-14T11:24:51.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.366+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1839703360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4e810a070 0x7fc4e810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.366+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1839703360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e8100ca0 0x7fc4e8101070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.366+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1839703360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4e81015b0 0x7fc4e8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.366+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1839703360 >> 172.21.15.161:0/1839703360 conn(0x7fc4e80fa510 msgr2=0x7fc4e80fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.366+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1839703360 shutdown_connections 2023-07-14T11:24:51.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.366+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1839703360 wait complete. 2023-07-14T11:24:51.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 Processor -- start 2023-07-14T11:24:51.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 -- start start 2023-07-14T11:24:51.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4e8100ca0 0x7fc4e8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e81015b0 0x7fc4e8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4e810a070 0x7fc4e819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e819cdb0 con 0x7fc4e8100ca0 2023-07-14T11:24:51.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e819cf20 con 0x7fc4e810a070 2023-07-14T11:24:51.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ef3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e819d0e0 con 0x7fc4e81015b0 2023-07-14T11:24:51.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ec99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e81015b0 0x7fc4e8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ed99c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4e810a070 0x7fc4e819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ec99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e81015b0 0x7fc4e8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49614/0 (socket says 172.21.15.161:49614) 2023-07-14T11:24:51.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.367+0000 7fc4ec99a700 1 -- 172.21.15.161:0/1164909533 learned_addr learned my addr 172.21.15.161:0/1164909533 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4e8100ca0 0x7fc4e8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ec99a700 1 -- 172.21.15.161:0/1164909533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4e810a070 msgr2=0x7fc4e819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4e810a070 0x7fc4e819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ec99a700 1 -- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4e8100ca0 msgr2=0x7fc4e8197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4e8100ca0 0x7fc4e8197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ec99a700 1 -- 172.21.15.161:0/1164909533 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40095f0 con 0x7fc4e81015b0 2023-07-14T11:24:51.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e81015b0 0x7fc4e8198310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800e9a0 tx=0x7fc4d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4de7fc700 1 -- 172.21.15.161:0/1164909533 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8009790 con 0x7fc4e81015b0 2023-07-14T11:24:51.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4de7fc700 1 -- 172.21.15.161:0/1164909533 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d8010430 con 0x7fc4e81015b0 2023-07-14T11:24:51.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e819d3c0 con 0x7fc4e81015b0 2023-07-14T11:24:51.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.368+0000 7fc4de7fc700 1 -- 172.21.15.161:0/1164909533 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8018680 con 0x7fc4e81015b0 2023-07-14T11:24:51.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.369+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e81a4c00 con 0x7fc4e81015b0 2023-07-14T11:24:51.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.372+0000 7fc4de7fc700 1 -- 172.21.15.161:0/1164909533 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc4d80105a0 con 0x7fc4e81015b0 2023-07-14T11:24:51.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.373+0000 7fc502c4eb80 1 Processor -- start 2023-07-14T11:24:51.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.373+0000 7fc502c4eb80 1 -- start start 2023-07-14T11:24:51.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b2b6cb0 con 0x56140b3b0be0 2023-07-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b309a40 con 0x56140b3b02d0 2023-07-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b302ed0 con 0x56140b3bbad0 2023-07-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc4ed99c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc4ed99c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35324/0 (socket says 172.21.15.161:35324) 2023-07-14T11:24:51.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc4ed99c700 1 -- 172.21.15.161:0/2393138210 learned_addr learned my addr 172.21.15.161:0/2393138210 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc4de7fc700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4d40a1340 0x7fc4d40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.374+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4de7fc700 1 -- 172.21.15.161:0/1164909533 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc4d8014070 con 0x7fc4e81015b0 2023-07-14T11:24:51.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ed99c700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4d40a1340 0x7fc4d40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ec99a700 1 -- 172.21.15.161:0/2393138210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 msgr2=0x56140b3b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ec99a700 1 -- 172.21.15.161:0/2393138210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 msgr2=0x56140b3bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ec99a700 1 -- 172.21.15.161:0/2393138210 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56140b130de0 con 0x56140b3bbad0 2023-07-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ed99c700 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3bdeb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80ca5c0 tx=0x7fc4d801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f9f2686f4f02f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4d37fe700 1 -- 172.21.15.161:0/2393138210 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8099f40 con 0x56140b3bbad0 2023-07-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4d37fe700 1 -- 172.21.15.161:0/2393138210 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d809df70 con 0x56140b3bbad0 2023-07-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.375+0000 7fc4d37fe700 1 -- 172.21.15.161:0/2393138210 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80cf280 con 0x56140b3bbad0 2023-07-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc4ed99c700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4d40a1340 0x7fc4d40a37f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000fcb0 tx=0x7fc4e0004b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2393138210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 msgr2=0x56140b3bdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3bdeb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80ca5c0 tx=0x7fc4d801f000 comp rx=0 tx=0).stop 2023-07-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2393138210 shutdown_connections 2023-07-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3bdeb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2393138210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2393138210 >> 172.21.15.161:0/2393138210 conn(0x56140b3a6da0 msgr2=0x56140b3ac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2393138210 shutdown_connections 2023-07-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.376+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2393138210 wait complete. 2023-07-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.377+0000 7fc502c4eb80 1 -- start start 2023-07-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.377+0000 7fc502c4eb80 1 Processor -- start 2023-07-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 0x56140b3bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b0be0 0x56140b3b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b2a2ac0 con 0x56140b3b02d0 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b2a27c0 con 0x56140b3b0be0 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b186920 con 0x56140b3bbad0 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc502c4eb80 10 client.0 _ll_register_callbacks cb 0x56140b2a0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 0x56140b3bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49644/0 (socket says 172.21.15.161:49644) 2023-07-14T11:24:51.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 -- 172.21.15.161:0/4107355705 learned_addr learned my addr 172.21.15.161:0/4107355705 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b0be0 0x56140b3b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 -- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b0be0 msgr2=0x56140b3b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b0be0 0x56140b3b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 -- 172.21.15.161:0/4107355705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 msgr2=0x56140b3bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 0x56140b3bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.378+0000 7fc4ed99c700 1 -- 172.21.15.161:0/4107355705 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e000f820 con 0x56140b3bbad0 2023-07-14T11:24:51.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc4ed99c700 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b8d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0006090 tx=0x7fc4e0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88dc8a2eea785684 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc4d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:51.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/4107355705 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000e040 con 0x56140b3bbad0 2023-07-14T11:24:51.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56140b25c4f0 con 0x56140b3bbad0 2023-07-14T11:24:51.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/4107355705 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc4e000fe40 con 0x56140b3bbad0 2023-07-14T11:24:51.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/4107355705 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4e0004db0 con 0x56140b3bbad0 2023-07-14T11:24:51.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc502c4eb80 10 client.26170 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:51.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fc4e81a4c00 con 0x56140b3bbad0 2023-07-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.379+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/4107355705 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56140b25c4f0 con 0x56140b3bbad0 2023-07-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/4107355705 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc4e0025560 con 0x56140b3bbad0 2023-07-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc502c4eb80 10 client.26170 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc502c4eb80 10 client.26170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc502c4eb80 20 client.26170 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b3cb430 0x56140b3cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc502c4eb80 4 client.26170 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56140b2fbeb0 con 0x56140b3cb430 2023-07-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b3cb430 0x56140b3cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b3cb430 0x56140b3cd840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4844403bd09a546c server_cookie=cf22f40bc2ffd2c8 in_seq=0 out_seq=0 2023-07-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.380+0000 7fc4d0ff9700 10 client.26170 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/4107355705 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56140b2fbeb0 con 0x56140b3cb430 2023-07-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc4d0ff9700 10 client.26170 handle_command_reply: tid=0 2023-07-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 client.26170 shutdown 2023-07-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 msgr2=0x56140b3b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b8d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0006090 tx=0x7fc4e0004ad0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 shutdown_connections 2023-07-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b3cb430 0x56140b3cd840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 0x56140b3bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b8d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/4107355705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b0be0 0x56140b3b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 >> 172.21.15.161:0/4107355705 conn(0x56140b3a6da0 msgr2=0x56140b3ac230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.381+0000 7fc4d0ff9700 0 client.26170 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:51.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.382+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 shutdown_connections 2023-07-14T11:24:51.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.382+0000 7fc502c4eb80 1 -- 172.21.15.161:0/4107355705 wait complete. 2023-07-14T11:24:51.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.382+0000 7fc502c4eb80 20 client.26170 trim_cache size 0 max 16384 2023-07-14T11:24:51.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.402+0000 7fc502c4eb80 1 Processor -- start 2023-07-14T11:24:51.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.402+0000 7fc502c4eb80 1 -- start start 2023-07-14T11:24:51.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.403+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.403+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.403+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.403+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b3b1570 con 0x56140b3b0be0 2023-07-14T11:24:51.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.403+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b3b9d10 con 0x56140b3b02d0 2023-07-14T11:24:51.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.403+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b3b9ab0 con 0x56140b3bbad0 2023-07-14T11:24:51.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.404+0000 7fc4ed99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.404+0000 7fc4ed99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49660/0 (socket says 172.21.15.161:49660) 2023-07-14T11:24:51.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.404+0000 7fc4ed99c700 1 -- 172.21.15.161:0/1768907118 learned_addr learned my addr 172.21.15.161:0/1768907118 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.405+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.405+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.406+0000 7fc4ec99a700 1 -- 172.21.15.161:0/1768907118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 msgr2=0x56140b3b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.406+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.406+0000 7fc4ec99a700 1 -- 172.21.15.161:0/1768907118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 msgr2=0x56140b3c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.406+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.406+0000 7fc4ec99a700 1 -- 172.21.15.161:0/1768907118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56140b119c60 con 0x56140b3b0be0 2023-07-14T11:24:51.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.406+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:51.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.406+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb050 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8099c40 tx=0x7fc4d8095c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d895af753435f2e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.407+0000 7fc4d3fff700 1 -- 172.21.15.161:0/1768907118 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8099f40 con 0x56140b3b0be0 2023-07-14T11:24:51.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.407+0000 7fc4d3fff700 1 -- 172.21.15.161:0/1768907118 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d80a2390 con 0x56140b3b0be0 2023-07-14T11:24:51.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.407+0000 7fc4d3fff700 1 -- 172.21.15.161:0/1768907118 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d809a470 con 0x56140b3b0be0 2023-07-14T11:24:51.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.408+0000 7fc502c4eb80 1 -- 172.21.15.161:0/1768907118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 msgr2=0x56140b3bb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.408+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb050 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8099c40 tx=0x7fc4d8095c80 comp rx=0 tx=0).stop 2023-07-14T11:24:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.408+0000 7fc502c4eb80 1 -- 172.21.15.161:0/1768907118 shutdown_connections 2023-07-14T11:24:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.408+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3bbad0 0x56140b3b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.408+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3b02d0 0x56140b3c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.408+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/1768907118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b0be0 0x56140b3bb050 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.408+0000 7fc502c4eb80 1 -- 172.21.15.161:0/1768907118 >> 172.21.15.161:0/1768907118 conn(0x56140b3a6da0 msgr2=0x56140b3b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.409+0000 7fc502c4eb80 1 -- 172.21.15.161:0/1768907118 shutdown_connections 2023-07-14T11:24:51.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.409+0000 7fc502c4eb80 1 -- 172.21.15.161:0/1768907118 wait complete. 2023-07-14T11:24:51.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.410+0000 7fc502c4eb80 1 -- start start 2023-07-14T11:24:51.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.410+0000 7fc502c4eb80 1 Processor -- start 2023-07-14T11:24:51.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.413+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 0x56140b41ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.413+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3b0be0 0x56140b41bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.413+0000 7fc502c4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3bbad0 0x56140b426ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.413+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b41c740 con 0x56140b3b02d0 2023-07-14T11:24:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.413+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b41c8b0 con 0x56140b3bbad0 2023-07-14T11:24:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.413+0000 7fc502c4eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140b427040 con 0x56140b3b0be0 2023-07-14T11:24:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.413+0000 7fc502c4eb80 10 client.0 _ll_register_callbacks cb 0x7fc4e00093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3b0be0 0x56140b41bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3b0be0 0x56140b41bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49662/0 (socket says 172.21.15.161:49662) 2023-07-14T11:24:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 -- 172.21.15.161:0/2686508807 learned_addr learned my addr 172.21.15.161:0/2686508807 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:51.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 -- 172.21.15.161:0/2686508807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3bbad0 msgr2=0x56140b426ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3bbad0 0x56140b426ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 -- 172.21.15.161:0/2686508807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 msgr2=0x56140b41ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:51.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 0x56140b41ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.414+0000 7fc4ec99a700 1 -- 172.21.15.161:0/2686508807 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56140b130de0 con 0x56140b3b0be0 2023-07-14T11:24:51.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:51 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/455172949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:24:51.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.415+0000 7fc4ec99a700 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3b0be0 0x56140b41bf90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8099ed0 tx=0x7fc4d8099ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f445341005e0b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.416+0000 7fc4d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:24:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.416+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/2686508807 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80ca4e0 con 0x56140b3b0be0 2023-07-14T11:24:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.416+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/2686508807 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fc4d80c8020 con 0x56140b3b0be0 2023-07-14T11:24:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.416+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d809df70 con 0x56140b3b0be0 2023-07-14T11:24:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/2686508807 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d80ca840 con 0x56140b3b0be0 2023-07-14T11:24:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc502c4eb80 10 client.26176 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56140b157250 con 0x56140b3b0be0 2023-07-14T11:24:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/2686508807 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d80c3030 con 0x56140b3b0be0 2023-07-14T11:24:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/2686508807 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc4d809a2c0 con 0x56140b3b0be0 2023-07-14T11:24:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc502c4eb80 10 client.26176 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc502c4eb80 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.417+0000 7fc502c4eb80 20 client.26176 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.418+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b428270 0x56140b42a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.418+0000 7fc502c4eb80 4 client.26176 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:24:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.418+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56140b2be050 con 0x56140b428270 2023-07-14T11:24:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.418+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b428270 0x56140b42a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.420+0000 7fc4d0ff9700 10 client.26176 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.420+0000 7fc4ed19b700 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b428270 0x56140b42a680 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f95297755577b46 server_cookie=54eaeab6420499f5 in_seq=0 out_seq=0 2023-07-14T11:24:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.421+0000 7fc4d0ff9700 1 -- 172.21.15.161:0/2686508807 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56140b2be050 con 0x56140b428270 2023-07-14T11:24:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.421+0000 7fc4d0ff9700 10 client.26176 handle_command_reply: tid=0 2023-07-14T11:24:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.421+0000 7fc502c4eb80 1 client.26176 shutdown 2023-07-14T11:24:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3b0be0 msgr2=0x56140b41bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3b0be0 0x56140b41bf90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8099ed0 tx=0x7fc4d8099ff0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 shutdown_connections 2023-07-14T11:24:51.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56140b428270 0x56140b42a680 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56140b3b02d0 0x56140b41ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56140b3bbad0 0x56140b426ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 --2- 172.21.15.161:0/2686508807 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56140b3b0be0 0x56140b41bf90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 >> 172.21.15.161:0/2686508807 conn(0x56140b3a6da0 msgr2=0x56140b3b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.422+0000 7fc4d0ff9700 0 client.26176 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:51.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.423+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 shutdown_connections 2023-07-14T11:24:51.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.423+0000 7fc502c4eb80 1 -- 172.21.15.161:0/2686508807 wait complete. 2023-07-14T11:24:51.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.424+0000 7fc502c4eb80 20 client.26176 trim_cache size 0 max 16384 2023-07-14T11:24:51.430 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:24:51.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.424+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4d40a1340 msgr2=0x7fc4d40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.424+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4d40a1340 0x7fc4d40a37f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000fcb0 tx=0x7fc4e0004b40 comp rx=0 tx=0).stop 2023-07-14T11:24:51.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.424+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e81015b0 msgr2=0x7fc4e8198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:51.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.424+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e81015b0 0x7fc4e8198310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800e9a0 tx=0x7fc4d800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.425+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 shutdown_connections 2023-07-14T11:24:51.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.425+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc4d40a1340 0x7fc4d40a37f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.425+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4e8100ca0 0x7fc4e8197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.425+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4e810a070 0x7fc4e819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.425+0000 7fc4ef3ff700 1 --2- 172.21.15.161:0/1164909533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc4e81015b0 0x7fc4e8198310 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800e9a0 tx=0x7fc4d800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:24:51.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.425+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 >> 172.21.15.161:0/1164909533 conn(0x7fc4e80fa510 msgr2=0x7fc4e810c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:51.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.425+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 shutdown_connections 2023-07-14T11:24:51.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:51.426+0000 7fc4ef3ff700 1 -- 172.21.15.161:0/1164909533 wait complete. 2023-07-14T11:24:51.457 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:51 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/455172949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:24:51.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:51 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/455172949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:24:52.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 180884 10.58 MB/sec warmup 94 sec latency 116.599 ms 2023-07-14T11:24:52.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:52 smithi175 ceph-mon[129081]: pgmap v81: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 337 op/s 2023-07-14T11:24:52.461 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:24:52.462 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:24:52.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:52 smithi182 ceph-mon[129366]: pgmap v81: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 337 op/s 2023-07-14T11:24:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:52 smithi161 ceph-mon[95526]: pgmap v81: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 337 op/s 2023-07-14T11:24:52.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 -- 172.21.15.161:0/3240273960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d4100480 msgr2=0x7f41d410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 --2- 172.21.15.161:0/3240273960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d4100480 0x7f41d410c910 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00b0a0 tx=0x7f41cc00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 -- 172.21.15.161:0/3240273960 shutdown_connections 2023-07-14T11:24:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 --2- 172.21.15.161:0/3240273960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d4105ae0 0x7f41d4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 --2- 172.21.15.161:0/3240273960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d4100480 0x7f41d410c910 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 --2- 172.21.15.161:0/3240273960 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41d40ffad0 0x7f41d40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 -- 172.21.15.161:0/3240273960 >> 172.21.15.161:0/3240273960 conn(0x7f41d40fb5f0 msgr2=0x7f41d40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 -- 172.21.15.161:0/3240273960 shutdown_connections 2023-07-14T11:24:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.674+0000 7f41d91ff700 1 -- 172.21.15.161:0/3240273960 wait complete. 2023-07-14T11:24:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.675+0000 7f41d91ff700 1 Processor -- start 2023-07-14T11:24:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.675+0000 7f41d91ff700 1 -- start start 2023-07-14T11:24:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d91ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40ffad0 0x7f41d4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d91ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41d4100480 0x7f41d4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d91ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d4105ae0 0x7f41d41a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d91ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4073580 con 0x7f41d4105ae0 2023-07-14T11:24:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d91ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41a43d0 con 0x7f41d40ffad0 2023-07-14T11:24:52.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d91ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41a4510 con 0x7f41d4100480 2023-07-14T11:24:52.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d2d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40ffad0 0x7f41d4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41d4100480 0x7f41d4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d2d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40ffad0 0x7f41d4072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35374/0 (socket says 172.21.15.161:35374) 2023-07-14T11:24:52.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d4105ae0 0x7f41d41a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d2d9d700 1 -- 172.21.15.161:0/4273327081 learned_addr learned my addr 172.21.15.161:0/4273327081 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:52.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.676+0000 7f41d259c700 1 -- 172.21.15.161:0/4273327081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40ffad0 msgr2=0x7f41d4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d259c700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40ffad0 0x7f41d4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d259c700 1 -- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d4105ae0 msgr2=0x7f41d41a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d259c700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d4105ae0 0x7f41d41a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d259c700 1 -- 172.21.15.161:0/4273327081 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc009d00 con 0x7f41d4100480 2023-07-14T11:24:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40ffad0 0x7f41d4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d359e700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d4105ae0 0x7f41d41a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d259c700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41d4100480 0x7f41d4072d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e9f0 tx=0x7f41c800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41bbfff700 1 -- 172.21.15.161:0/4273327081 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c800c9c0 con 0x7f41d4100480 2023-07-14T11:24:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d41a4770 con 0x7f41d4100480 2023-07-14T11:24:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41bbfff700 1 -- 172.21.15.161:0/4273327081 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c8010430 con 0x7f41d4100480 2023-07-14T11:24:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41bbfff700 1 -- 172.21.15.161:0/4273327081 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8018680 con 0x7f41d4100480 2023-07-14T11:24:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.677+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d41a4ca0 con 0x7f41d4100480 2023-07-14T11:24:52.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.682+0000 7f41bbfff700 1 -- 172.21.15.161:0/4273327081 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f41c8010860 con 0x7f41d4100480 2023-07-14T11:24:52.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 Processor -- start 2023-07-14T11:24:52.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 -- start start 2023-07-14T11:24:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c32d350 0x55c53c32d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c338610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c338b50 0x55c53c33af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c233d30 con 0x55c53c338b50 2023-07-14T11:24:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c286ac0 con 0x55c53c32dc60 2023-07-14T11:24:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c27ff50 con 0x55c53c32d350 2023-07-14T11:24:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.683+0000 7f41bbfff700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f41bc0a12c0 0x7f41bc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d359e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c32d350 0x55c53c32d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d359e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c32d350 0x55c53c32d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49706/0 (socket says 172.21.15.161:49706) 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d2d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c338610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d359e700 1 -- 172.21.15.161:0/343243724 learned_addr learned my addr 172.21.15.161:0/343243724 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d259c700 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c338b50 0x55c53c33af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41bbfff700 1 -- 172.21.15.161:0/4273327081 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f41c8014070 con 0x7f41d4100480 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d2d9d700 1 -- 172.21.15.161:0/343243724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c32d350 msgr2=0x55c53c32d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c32d350 0x55c53c32d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d2d9d700 1 -- 172.21.15.161:0/343243724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c338b50 msgr2=0x55c53c33af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d359e700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f41bc0a12c0 0x7f41bc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c338b50 0x55c53c33af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d2d9d700 1 -- 172.21.15.161:0/343243724 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c53c0ade60 con 0x55c53c32dc60 2023-07-14T11:24:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d259c700 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c338b50 0x55c53c33af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:52.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.684+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c338610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41c40114c0 tx=0x7f41c40117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1545bfded8d1a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:52.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41b97fa700 1 -- 172.21.15.161:0/343243724 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004230 con 0x55c53c32dc60 2023-07-14T11:24:52.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41b97fa700 1 -- 172.21.15.161:0/343243724 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c4004390 con 0x55c53c32dc60 2023-07-14T11:24:52.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41b97fa700 1 -- 172.21.15.161:0/343243724 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c400e900 con 0x55c53c32dc60 2023-07-14T11:24:52.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/343243724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 msgr2=0x55c53c338610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41d359e700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f41bc0a12c0 0x7f41bc0a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009610 tx=0x7f41cc00b3b0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c338610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41c40114c0 tx=0x7f41c40117d0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/343243724 shutdown_connections 2023-07-14T11:24:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c338b50 0x55c53c33af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c32d350 0x55c53c32d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/343243724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c338610 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/343243724 >> 172.21.15.161:0/343243724 conn(0x55c53c323e20 msgr2=0x55c53c329720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/343243724 shutdown_connections 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.685+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/343243724 wait complete. 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.686+0000 7f41ecb7bb80 1 -- start start 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.686+0000 7f41ecb7bb80 1 Processor -- start 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32d350 0x55c53c338520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c335730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c335d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c21fb40 con 0x55c53c32d350 2023-07-14T11:24:52.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c21f840 con 0x55c53c32dc60 2023-07-14T11:24:52.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c1039a0 con 0x55c53c338b50 2023-07-14T11:24:52.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41ecb7bb80 10 client.0 _ll_register_callbacks cb 0x55c53c21d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:52.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c335730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c335730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35382/0 (socket says 172.21.15.161:35382) 2023-07-14T11:24:52.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 -- 172.21.15.161:0/4051048481 learned_addr learned my addr 172.21.15.161:0/4051048481 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:52.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32d350 0x55c53c338520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d359e700 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c335d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 -- 172.21.15.161:0/4051048481 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 msgr2=0x55c53c335d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c335d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 -- 172.21.15.161:0/4051048481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32d350 msgr2=0x55c53c338520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32d350 0x55c53c338520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.687+0000 7f41d259c700 1 -- 172.21.15.161:0/4051048481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c4009ce0 con 0x55c53c32dc60 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41d259c700 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c335730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f41c8099c10 tx=0x7f41c8099f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f161b77fc72954bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:52.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:52.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41b6ff5700 1 -- 172.21.15.161:0/4051048481 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80a2360 con 0x55c53c32dc60 2023-07-14T11:24:52.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c53c1d9570 con 0x55c53c32dc60 2023-07-14T11:24:52.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41b6ff5700 1 -- 172.21.15.161:0/4051048481 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f41c80cf0f0 con 0x55c53c32dc60 2023-07-14T11:24:52.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41b6ff5700 1 -- 172.21.15.161:0/4051048481 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c80c4540 con 0x55c53c32dc60 2023-07-14T11:24:52.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41ecb7bb80 10 client.26189 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:52.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c53c096ce0 con 0x55c53c32dc60 2023-07-14T11:24:52.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41b6ff5700 1 -- 172.21.15.161:0/4051048481 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c80b1080 con 0x55c53c32dc60 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.688+0000 7f41b6ff5700 1 -- 172.21.15.161:0/4051048481 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f41c80a1f60 con 0x55c53c32dc60 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41ecb7bb80 10 client.26189 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41ecb7bb80 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41ecb7bb80 20 client.26189 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c348320 0x55c53c34a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41ecb7bb80 4 client.26189 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c53c278f30 con 0x55c53c348320 2023-07-14T11:24:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c348320 0x55c53c34a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41b6ff5700 10 client.26189 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.689+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c348320 0x55c53c34a730 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5563725412416677 server_cookie=a9a35d8a22196316 in_seq=0 out_seq=0 2023-07-14T11:24:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41b6ff5700 1 -- 172.21.15.161:0/4051048481 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c53c278f30 con 0x55c53c348320 2023-07-14T11:24:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41b6ff5700 10 client.26189 handle_command_reply: tid=0 2023-07-14T11:24:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 client.26189 shutdown 2023-07-14T11:24:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 msgr2=0x55c53c335730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c335730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f41c8099c10 tx=0x7f41c8099f10 comp rx=0 tx=0).stop 2023-07-14T11:24:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 shutdown_connections 2023-07-14T11:24:52.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c348320 0x55c53c34a730 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32d350 0x55c53c338520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41b6ff5700 0 client.26189 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:52.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c335d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/4051048481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32dc60 0x55c53c335730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.690+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 >> 172.21.15.161:0/4051048481 conn(0x55c53c323e20 msgr2=0x55c53c3292b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:52.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.691+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 shutdown_connections 2023-07-14T11:24:52.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.691+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/4051048481 wait complete. 2023-07-14T11:24:52.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.691+0000 7f41ecb7bb80 20 client.26189 trim_cache size 0 max 16384 2023-07-14T11:24:52.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 Processor -- start 2023-07-14T11:24:52.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 -- start start 2023-07-14T11:24:52.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c3356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c338050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c3329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c32e5f0 con 0x55c53c32dc60 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c344c80 con 0x55c53c32d350 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.709+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c3312c0 con 0x55c53c338b50 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c3356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d359e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c3329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c338050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c3356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35392/0 (socket says 172.21.15.161:35392) 2023-07-14T11:24:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 -- 172.21.15.161:0/3059021062 learned_addr learned my addr 172.21.15.161:0/3059021062 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d359e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c3329c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49722/0 (socket says 172.21.15.161:49722) 2023-07-14T11:24:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 -- 172.21.15.161:0/3059021062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 msgr2=0x55c53c3329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/3059021062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c3329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 -- 172.21.15.161:0/3059021062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 msgr2=0x55c53c338050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/3059021062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c338050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 -- 172.21.15.161:0/3059021062 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c53c0d42d0 con 0x55c53c32d350 2023-07-14T11:24:52.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/3059021062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c3356e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f41c40114c0 tx=0x7f41c40117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc4e2cabaf98aff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:52.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.710+0000 7f41b9ffb700 1 -- 172.21.15.161:0/3059021062 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4005d60 con 0x55c53c32d350 2023-07-14T11:24:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41b9ffb700 1 -- 172.21.15.161:0/3059021062 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c4005620 con 0x55c53c32d350 2023-07-14T11:24:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41b9ffb700 1 -- 172.21.15.161:0/3059021062 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c402ade0 con 0x55c53c32d350 2023-07-14T11:24:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/3059021062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 msgr2=0x55c53c3356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/3059021062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c3356e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f41c40114c0 tx=0x7f41c40117d0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/3059021062 shutdown_connections 2023-07-14T11:24:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/3059021062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c3329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/3059021062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c3356e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/3059021062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c338050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/3059021062 >> 172.21.15.161:0/3059021062 conn(0x55c53c323e20 msgr2=0x55c53c334fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:52.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/3059021062 shutdown_connections 2023-07-14T11:24:52.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.711+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/3059021062 wait complete. 2023-07-14T11:24:52.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.712+0000 7f41ecb7bb80 1 -- start start 2023-07-14T11:24:52.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.712+0000 7f41ecb7bb80 1 Processor -- start 2023-07-14T11:24:52.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c395530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c395a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41ecb7bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c33e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c33ebf0 con 0x55c53c32dc60 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c33ed60 con 0x55c53c32d350 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41ecb7bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53c33ef20 con 0x55c53c338b50 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41ecb7bb80 10 client.0 _ll_register_callbacks cb 0x7f41c8099a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41d259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c395a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41d2d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c395530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41d259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c395a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59298/0 (socket says 172.21.15.161:59298) 2023-07-14T11:24:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41d259c700 1 -- 172.21.15.161:0/691158058 learned_addr learned my addr 172.21.15.161:0/691158058 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.713+0000 7f41d359e700 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c33e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41d259c700 1 -- 172.21.15.161:0/691158058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 msgr2=0x55c53c33e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41d259c700 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c33e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41d259c700 1 -- 172.21.15.161:0/691158058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 msgr2=0x55c53c395530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41d259c700 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c395530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41d259c700 1 -- 172.21.15.161:0/691158058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c53c0ade60 con 0x55c53c32dc60 2023-07-14T11:24:52.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41d259c700 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c395a70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c5b0 tx=0x7f41c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df54a5f8df7b4ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:52.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:24:52.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41b6ff5700 1 -- 172.21.15.161:0/691158058 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8099600 con 0x55c53c32dc60 2023-07-14T11:24:52.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41b6ff5700 1 -- 172.21.15.161:0/691158058 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f41c80a2360 con 0x55c53c32dc60 2023-07-14T11:24:52.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41c4004390 con 0x55c53c32dc60 2023-07-14T11:24:52.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41b6ff5700 1 -- 172.21.15.161:0/691158058 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c800cb70 con 0x55c53c32dc60 2023-07-14T11:24:52.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41ecb7bb80 10 client.16146 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:52.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.714+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c53c0ac330 con 0x55c53c32dc60 2023-07-14T11:24:52.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41b6ff5700 1 -- 172.21.15.161:0/691158058 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c80b8020 con 0x55c53c32dc60 2023-07-14T11:24:52.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41b6ff5700 1 -- 172.21.15.161:0/691158058 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f41c8095f00 con 0x55c53c32dc60 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41ecb7bb80 10 client.16146 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41ecb7bb80 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41ecb7bb80 20 client.16146 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c39e150 0x55c53c3a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41ecb7bb80 4 client.16146 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c53c23b0d0 con 0x55c53c39e150 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.715+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c39e150 0x55c53c3a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:52.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.716+0000 7f41d2d9d700 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c39e150 0x55c53c3a0560 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d34611adf0f05c server_cookie=af05a826fcb150fe in_seq=0 out_seq=0 2023-07-14T11:24:52.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.716+0000 7f41b6ff5700 10 client.16146 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:52.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41b6ff5700 1 -- 172.21.15.161:0/691158058 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c53c23b0d0 con 0x55c53c39e150 2023-07-14T11:24:52.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41b6ff5700 10 client.16146 handle_command_reply: tid=0 2023-07-14T11:24:52.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 client.16146 shutdown 2023-07-14T11:24:52.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 msgr2=0x55c53c395a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c395a70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c5b0 tx=0x7f41c800cf90 comp rx=0 tx=0).stop 2023-07-14T11:24:52.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 shutdown_connections 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c53c39e150 0x55c53c3a0560 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c53c338b50 0x55c53c33e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c53c32d350 0x55c53c395530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 --2- 172.21.15.161:0/691158058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c53c32dc60 0x55c53c395a70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 >> 172.21.15.161:0/691158058 conn(0x55c53c323e20 msgr2=0x55c53c333ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41b6ff5700 0 client.16146 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 shutdown_connections 2023-07-14T11:24:52.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 1 -- 172.21.15.161:0/691158058 wait complete. 2023-07-14T11:24:52.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.717+0000 7f41ecb7bb80 20 client.16146 trim_cache size 0 max 16384 2023-07-14T11:24:52.728 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:24:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:24:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "3701b028-5ca8-48f8-8d05-8f9c5887768c", 2023-07-14T11:24:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:24:52.729 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:24:52.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f41bc0a12c0 msgr2=0x7f41bc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f41bc0a12c0 0x7f41bc0a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009610 tx=0x7f41cc00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41d4100480 msgr2=0x7f41d4072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41d4100480 0x7f41d4072d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e9f0 tx=0x7f41c800ed00 comp rx=0 tx=0).stop 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 shutdown_connections 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f41bc0a12c0 0x7f41bc0a3770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d4105ae0 0x7f41d41a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41d40ffad0 0x7f41d4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 --2- 172.21.15.161:0/4273327081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f41d4100480 0x7f41d4072d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:52.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 >> 172.21.15.161:0/4273327081 conn(0x7f41d40fb5f0 msgr2=0x7f41d40fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:52.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 shutdown_connections 2023-07-14T11:24:52.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.718+0000 7f41d91ff700 1 -- 172.21.15.161:0/4273327081 wait complete. 2023-07-14T11:24:52.741 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:24:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.998+0000 7fae08dff700 1 -- 172.21.15.161:0/854144689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae04105ae0 msgr2=0x7fae04105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.998+0000 7fae08dff700 1 --2- 172.21.15.161:0/854144689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae04105ae0 0x7fae04105eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009990 tx=0x7fadf4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.998+0000 7fae08dff700 1 -- 172.21.15.161:0/854144689 shutdown_connections 2023-07-14T11:24:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.998+0000 7fae08dff700 1 --2- 172.21.15.161:0/854144689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae04100480 0x7fae0410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.998+0000 7fae08dff700 1 --2- 172.21.15.161:0/854144689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae04105ae0 0x7fae04105eb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.998+0000 7fae08dff700 1 --2- 172.21.15.161:0/854144689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae040ffad0 0x7fae040fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.998+0000 7fae08dff700 1 -- 172.21.15.161:0/854144689 >> 172.21.15.161:0/854144689 conn(0x7fae040fb5f0 msgr2=0x7fae040fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:53.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.999+0000 7fae08dff700 1 -- 172.21.15.161:0/854144689 shutdown_connections 2023-07-14T11:24:53.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:52.999+0000 7fae08dff700 1 -- 172.21.15.161:0/854144689 wait complete. 2023-07-14T11:24:53.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 Processor -- start 2023-07-14T11:24:53.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 -- start start 2023-07-14T11:24:53.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae040ffad0 0x7fae04072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae04100480 0x7fae04072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae04105ae0 0x7fae041a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae04073580 con 0x7fae04100480 2023-07-14T11:24:53.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae041a42e0 con 0x7fae040ffad0 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae08dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae041a4420 con 0x7fae04105ae0 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae0259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae040ffad0 0x7fae04072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae0259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae040ffad0 0x7fae04072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35414/0 (socket says 172.21.15.161:35414) 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae0259c700 1 -- 172.21.15.161:0/2957016770 learned_addr learned my addr 172.21.15.161:0/2957016770 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.000+0000 7fae02d9d700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae04105ae0 0x7fae041a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae01d9b700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae04100480 0x7fae04072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae0259c700 1 -- 172.21.15.161:0/2957016770 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae04105ae0 msgr2=0x7fae041a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae0259c700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae04105ae0 0x7fae041a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae0259c700 1 -- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae04100480 msgr2=0x7fae04072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae0259c700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae04100480 0x7fae04072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae0259c700 1 -- 172.21.15.161:0/2957016770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40095f0 con 0x7fae040ffad0 2023-07-14T11:24:53.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae01d9b700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae04100480 0x7fae04072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:24:53.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae0259c700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae040ffad0 0x7fae04072810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fadf4000c00 tx=0x7fadf4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:53.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fadf37fe700 1 -- 172.21.15.161:0/2957016770 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf401d070 con 0x7fae040ffad0 2023-07-14T11:24:53.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae041a4620 con 0x7fae040ffad0 2023-07-14T11:24:53.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fadf37fe700 1 -- 172.21.15.161:0/2957016770 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf4022470 con 0x7fae040ffad0 2023-07-14T11:24:53.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fadf37fe700 1 -- 172.21.15.161:0/2957016770 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4017650 con 0x7fae040ffad0 2023-07-14T11:24:53.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.001+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae041a4a40 con 0x7fae040ffad0 2023-07-14T11:24:53.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.005+0000 7fadf37fe700 1 -- 172.21.15.161:0/2957016770 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fadf4022850 con 0x7fae040ffad0 2023-07-14T11:24:53.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 Processor -- start 2023-07-14T11:24:53.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 -- start start 2023-07-14T11:24:53.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee562a2d0 0x556ee562a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee5635590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee5635ad0 0x556ee5637eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee5530cb0 con 0x556ee562a2d0 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee5583a40 con 0x556ee562abe0 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.006+0000 7fae1c633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee557ced0 con 0x556ee5635ad0 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee5635590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae02d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee562a2d0 0x556ee562a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae01d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee5635ad0 0x556ee5637eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee5635590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35418/0 (socket says 172.21.15.161:35418) 2023-07-14T11:24:53.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 -- 172.21.15.161:0/4284409617 learned_addr learned my addr 172.21.15.161:0/4284409617 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:53.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fadf37fe700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fadec0a1340 0x7fadec0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 -- 172.21.15.161:0/4284409617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee5635ad0 msgr2=0x556ee5637eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee5635ad0 0x556ee5637eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 -- 172.21.15.161:0/4284409617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee562a2d0 msgr2=0x556ee562a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee562a2d0 0x556ee562a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae0259c700 1 -- 172.21.15.161:0/4284409617 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ee53aade0 con 0x556ee562abe0 2023-07-14T11:24:53.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae01d9b700 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee5635ad0 0x556ee5637eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:24:53.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fadf37fe700 1 -- 172.21.15.161:0/2957016770 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fadf40ce460 con 0x7fae040ffad0 2023-07-14T11:24:53.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.007+0000 7fae02d9d700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fadec0a1340 0x7fadec0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.008+0000 7fae0259c700 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee5635590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fadf40cb760 tx=0x7fadf402e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4acc4da5aacb75a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:53.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.008+0000 7fadf0ff9700 1 -- 172.21.15.161:0/4284409617 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf409b000 con 0x556ee562abe0 2023-07-14T11:24:53.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.008+0000 7fadf0ff9700 1 -- 172.21.15.161:0/4284409617 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf409ad70 con 0x556ee562abe0 2023-07-14T11:24:53.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.008+0000 7fadf0ff9700 1 -- 172.21.15.161:0/4284409617 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf409ab90 con 0x556ee562abe0 2023-07-14T11:24:53.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.008+0000 7fae1c633b80 1 -- 172.21.15.161:0/4284409617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 msgr2=0x556ee5635590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.008+0000 7fae02d9d700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fadec0a1340 0x7fadec0a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae04074920 tx=0x7fadfc009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:53.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.008+0000 7fae1c633b80 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee5635590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fadf40cb760 tx=0x7fadf402e000 comp rx=0 tx=0).stop 2023-07-14T11:24:53.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 -- 172.21.15.161:0/4284409617 shutdown_connections 2023-07-14T11:24:53.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee562a2d0 0x556ee562a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee5635ad0 0x556ee5637eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 --2- 172.21.15.161:0/4284409617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee5635590 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 -- 172.21.15.161:0/4284409617 >> 172.21.15.161:0/4284409617 conn(0x556ee5620da0 msgr2=0x556ee56266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:53.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 -- 172.21.15.161:0/4284409617 shutdown_connections 2023-07-14T11:24:53.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 -- 172.21.15.161:0/4284409617 wait complete. 2023-07-14T11:24:53.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 -- start start 2023-07-14T11:24:53.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.009+0000 7fae1c633b80 1 Processor -- start 2023-07-14T11:24:53.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee563ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee563b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee5634980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae1c633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee551cac0 con 0x556ee5635ad0 2023-07-14T11:24:53.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae1c633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee551c7c0 con 0x556ee562abe0 2023-07-14T11:24:53.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae1c633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee5400920 con 0x556ee562a2d0 2023-07-14T11:24:53.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae1c633b80 10 client.0 _ll_register_callbacks cb 0x556ee551a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:53.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae01d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee563b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae01d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee563b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35420/0 (socket says 172.21.15.161:35420) 2023-07-14T11:24:53.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae01d9b700 1 -- 172.21.15.161:0/3264278 learned_addr learned my addr 172.21.15.161:0/3264278 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:53.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae0259c700 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee563ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae02d9d700 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee5634980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae01d9b700 1 -- 172.21.15.161:0/3264278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 msgr2=0x556ee563ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.010+0000 7fae01d9b700 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee563ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fae01d9b700 1 -- 172.21.15.161:0/3264278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 msgr2=0x556ee5634980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fae01d9b700 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee5634980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fae01d9b700 1 -- 172.21.15.161:0/3264278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf80098f0 con 0x556ee562abe0 2023-07-14T11:24:53.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fae01d9b700 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee563b050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fadf8012950 tx=0x7fadf8012c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a386135460a84734 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:53.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fade9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:53.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fade9ffb700 1 -- 172.21.15.161:0/3264278 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8017070 con 0x556ee562abe0 2023-07-14T11:24:53.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ee54d64f0 con 0x556ee562abe0 2023-07-14T11:24:53.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fade9ffb700 1 -- 172.21.15.161:0/3264278 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fadf8012e40 con 0x556ee562abe0 2023-07-14T11:24:53.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fade9ffb700 1 -- 172.21.15.161:0/3264278 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadf8011ac0 con 0x556ee562abe0 2023-07-14T11:24:53.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fae1c633b80 10 client.26207 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:53.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.011+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fae041a4a40 con 0x556ee562abe0 2023-07-14T11:24:53.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fade9ffb700 1 -- 172.21.15.161:0/3264278 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ee54d64f0 con 0x556ee562abe0 2023-07-14T11:24:53.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fade9ffb700 1 -- 172.21.15.161:0/3264278 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fadf801c420 con 0x556ee562abe0 2023-07-14T11:24:53.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fae1c633b80 10 client.26207 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:53.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fae1c633b80 10 client.26207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:53.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fae1c633b80 20 client.26207 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:53.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fae1c633b80 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee56474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fae1c633b80 4 client.26207 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:24:53.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ee5575eb0 con 0x556ee56450e0 2023-07-14T11:24:53.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.012+0000 7fae0259c700 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee56474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.013+0000 7fade9ffb700 10 client.26207 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:53.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.013+0000 7fae0259c700 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee56474f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0b0044505181c5 server_cookie=c3c4d4fccfe61fc9 in_seq=0 out_seq=0 2023-07-14T11:24:53.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.013+0000 7fade9ffb700 1 -- 172.21.15.161:0/3264278 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ee5575eb0 con 0x556ee56450e0 2023-07-14T11:24:53.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.013+0000 7fade9ffb700 10 client.26207 handle_command_reply: tid=0 2023-07-14T11:24:53.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 client.26207 shutdown 2023-07-14T11:24:53.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 msgr2=0x556ee563b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee563b050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fadf8012950 tx=0x7fadf8012c60 comp rx=0 tx=0).stop 2023-07-14T11:24:53.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 shutdown_connections 2023-07-14T11:24:53.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee56474f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee5634980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee563ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fade9ffb700 0 client.26207 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:53.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 --2- 172.21.15.161:0/3264278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee563b050 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 >> 172.21.15.161:0/3264278 conn(0x556ee5620da0 msgr2=0x556ee5626290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:53.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 shutdown_connections 2023-07-14T11:24:53.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 1 -- 172.21.15.161:0/3264278 wait complete. 2023-07-14T11:24:53.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.014+0000 7fae1c633b80 20 client.26207 trim_cache size 0 max 16384 2023-07-14T11:24:53.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.033+0000 7fae1c633b80 1 Processor -- start 2023-07-14T11:24:53.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.033+0000 7fae1c633b80 1 -- start start 2023-07-14T11:24:53.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee562fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee56301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee563a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae1c633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee562c8b0 con 0x556ee5635ad0 2023-07-14T11:24:53.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae1c633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee562b540 con 0x556ee562abe0 2023-07-14T11:24:53.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae1c633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee5620970 con 0x556ee562a2d0 2023-07-14T11:24:53.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee56301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee56301e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35436/0 (socket says 172.21.15.161:35436) 2023-07-14T11:24:53.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 -- 172.21.15.161:0/15150434 learned_addr learned my addr 172.21.15.161:0/15150434 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:53.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae0259c700 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee562fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae02d9d700 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee563a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 -- 172.21.15.161:0/15150434 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 msgr2=0x556ee562fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee562fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 -- 172.21.15.161:0/15150434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 msgr2=0x556ee563a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee563a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.034+0000 7fae01d9b700 1 -- 172.21.15.161:0/15150434 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ee5635520 con 0x556ee562abe0 2023-07-14T11:24:53.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.035+0000 7fae01d9b700 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee56301e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fadf8000c00 tx=0x7fadf8009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651622c10c307eb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:53.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.035+0000 7fadf17fa700 1 -- 172.21.15.161:0/15150434 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8017040 con 0x556ee562abe0 2023-07-14T11:24:53.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.035+0000 7fadf17fa700 1 -- 172.21.15.161:0/15150434 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf8013ed0 con 0x556ee562abe0 2023-07-14T11:24:53.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.035+0000 7fadf17fa700 1 -- 172.21.15.161:0/15150434 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8005330 con 0x556ee562abe0 2023-07-14T11:24:53.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 -- 172.21.15.161:0/15150434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 msgr2=0x556ee56301e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee56301e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fadf8000c00 tx=0x7fadf8009710 comp rx=0 tx=0).stop 2023-07-14T11:24:53.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 -- 172.21.15.161:0/15150434 shutdown_connections 2023-07-14T11:24:53.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee563a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562a2d0 0x556ee562fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 --2- 172.21.15.161:0/15150434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562abe0 0x556ee56301e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 -- 172.21.15.161:0/15150434 >> 172.21.15.161:0/15150434 conn(0x556ee5620da0 msgr2=0x556ee562c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:53.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 -- 172.21.15.161:0/15150434 shutdown_connections 2023-07-14T11:24:53.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.036+0000 7fae1c633b80 1 -- 172.21.15.161:0/15150434 wait complete. 2023-07-14T11:24:53.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 -- start start 2023-07-14T11:24:53.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 Processor -- start 2023-07-14T11:24:53.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562a2d0 0x556ee56917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562abe0 0x556ee5691d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee568cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee568d640 con 0x556ee5635ad0 2023-07-14T11:24:53.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee568d7b0 con 0x556ee562a2d0 2023-07-14T11:24:53.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee568d970 con 0x556ee562abe0 2023-07-14T11:24:53.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae1c633b80 10 client.0 _ll_register_callbacks cb 0x7fadf8012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:24:53.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.037+0000 7fae0259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562a2d0 0x556ee56917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae01d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562abe0 0x556ee5691d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562a2d0 0x556ee56917f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35452/0 (socket says 172.21.15.161:35452) 2023-07-14T11:24:53.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 -- 172.21.15.161:0/1735938310 learned_addr learned my addr 172.21.15.161:0/1735938310 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:24:53.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae02d9d700 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee568cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 -- 172.21.15.161:0/1735938310 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562abe0 msgr2=0x556ee5691d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562abe0 0x556ee5691d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 -- 172.21.15.161:0/1735938310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 msgr2=0x556ee568cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee568cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 -- 172.21.15.161:0/1735938310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae041a4a40 con 0x556ee562a2d0 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae0259c700 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562a2d0 0x556ee56917f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fadf4022fd0 tx=0x7fadf4009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d5353089dcef68 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fade9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fade9ffb700 1 -- 172.21.15.161:0/1735938310 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40a10d0 con 0x556ee562a2d0 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fade9ffb700 1 -- 172.21.15.161:0/1735938310 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fadf4095e40 con 0x556ee562a2d0 2023-07-14T11:24:53.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.038+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadf409ad70 con 0x556ee562a2d0 2023-07-14T11:24:53.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fade9ffb700 1 -- 172.21.15.161:0/1735938310 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadf40aeea0 con 0x556ee562a2d0 2023-07-14T11:24:53.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 10 client.26213 fetch_fsmap learned FSMap version 23 2023-07-14T11:24:53.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556ee5393c60 con 0x556ee562a2d0 2023-07-14T11:24:53.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fade9ffb700 1 -- 172.21.15.161:0/1735938310 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf409ad70 con 0x556ee562a2d0 2023-07-14T11:24:53.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fade9ffb700 1 -- 172.21.15.161:0/1735938310 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fadf40970a0 con 0x556ee562a2d0 2023-07-14T11:24:53.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 10 client.26213 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:24:53.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:24:53.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 20 client.26213 populate_metadata read hostname 'smithi161' 2023-07-14T11:24:53.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee569c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:24:53.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 4 client.26213 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:24:53.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ee5538050 con 0x556ee56450e0 2023-07-14T11:24:53.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.039+0000 7fae01d9b700 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee569c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:24:53.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae01d9b700 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee569c3e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db09e5076125b78 server_cookie=98f49bc7e26ec840 in_seq=0 out_seq=0 2023-07-14T11:24:53.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fade9ffb700 10 client.26213 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:53.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fade9ffb700 1 -- 172.21.15.161:0/1735938310 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556ee5538050 con 0x556ee56450e0 2023-07-14T11:24:53.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fade9ffb700 10 client.26213 handle_command_reply: tid=0 2023-07-14T11:24:53.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 client.26213 shutdown 2023-07-14T11:24:53.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562a2d0 msgr2=0x556ee56917f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562a2d0 0x556ee56917f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fadf4022fd0 tx=0x7fadf4009290 comp rx=0 tx=0).stop 2023-07-14T11:24:53.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 shutdown_connections 2023-07-14T11:24:53.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ee56450e0 0x556ee569c3e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ee5635ad0 0x556ee568cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ee562a2d0 0x556ee56917f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fade9ffb700 0 client.26213 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:24:53.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 --2- 172.21.15.161:0/1735938310 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ee562abe0 0x556ee5691d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.040+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 >> 172.21.15.161:0/1735938310 conn(0x556ee5620da0 msgr2=0x556ee562fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:53.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.041+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 shutdown_connections 2023-07-14T11:24:53.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.041+0000 7fae1c633b80 1 -- 172.21.15.161:0/1735938310 wait complete. 2023-07-14T11:24:53.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.041+0000 7fae1c633b80 20 client.26213 trim_cache size 0 max 16384 2023-07-14T11:24:53.055 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:24:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:24:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:24:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "3701b028-5ca8-48f8-8d05-8f9c5887768c": { 2023-07-14T11:24:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:24:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "3701b028-5ca8-48f8-8d05-8f9c5887768c", 2023-07-14T11:24:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:24:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:24:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:24:53.057 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:24:53.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fadec0a1340 msgr2=0x7fadec0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fadec0a1340 0x7fadec0a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae04074920 tx=0x7fadfc009040 comp rx=0 tx=0).stop 2023-07-14T11:24:53.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae040ffad0 msgr2=0x7fae04072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:24:53.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae040ffad0 0x7fae04072810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fadf4000c00 tx=0x7fadf4005720 comp rx=0 tx=0).stop 2023-07-14T11:24:53.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 shutdown_connections 2023-07-14T11:24:53.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fadec0a1340 0x7fadec0a37f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae04105ae0 0x7fae041a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae040ffad0 0x7fae04072810 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 --2- 172.21.15.161:0/2957016770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae04100480 0x7fae04072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:24:53.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 >> 172.21.15.161:0/2957016770 conn(0x7fae040fb5f0 msgr2=0x7fae040fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:24:53.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 shutdown_connections 2023-07-14T11:24:53.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:24:53.042+0000 7fae08dff700 1 -- 172.21.15.161:0/2957016770 wait complete. 2023-07-14T11:24:53.063 INFO:tasks.cephfs.filesystem:scrub status for tag:3701b028-5ca8-48f8-8d05-8f9c5887768c - {'path': '/', 'tag': '3701b028-5ca8-48f8-8d05-8f9c5887768c', 'options': 'recursive,force'} 2023-07-14T11:24:53.169 INFO:tasks.workunit.client.0.smithi161.stdout: 1 181985 10.51 MB/sec warmup 95 sec latency 19.452 ms 2023-07-14T11:24:53.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:53 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:24:53.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:53 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:53.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:53 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:24:53.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:53 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:24:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:53 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:53.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:53 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:24:53.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:53 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:24:53.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:53 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:53.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:53 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:24:54.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 184659 10.57 MB/sec warmup 96 sec latency 43.670 ms 2023-07-14T11:24:54.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:54 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:24:54.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:54 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:24:54.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:54 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:54 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:54 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:54 smithi175 ceph-mon[129081]: pgmap v82: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-14T11:24:54.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:54 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:24:54.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:54 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:24:54.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:54 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:54 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:54 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:54 smithi182 ceph-mon[129366]: pgmap v82: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-14T11:24:54.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:54 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:24:54.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:54 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:24:54.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:54 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:54 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:54 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:24:54.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:54 smithi161 ceph-mon[95526]: pgmap v82: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-14T11:24:55.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 185952 10.52 MB/sec warmup 97 sec latency 17.654 ms 2023-07-14T11:24:56.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 188293 10.55 MB/sec warmup 98 sec latency 21.548 ms 2023-07-14T11:24:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:56 smithi175 ceph-mon[129081]: pgmap v83: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-14T11:24:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:56 smithi182 ceph-mon[129366]: pgmap v83: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-14T11:24:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:56 smithi161 ceph-mon[95526]: pgmap v83: 129 pgs: 129 active+clean; 111 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-14T11:24:57.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 189644 10.50 MB/sec warmup 99 sec latency 46.639 ms 2023-07-14T11:24:58.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 191935 10.54 MB/sec warmup 100 sec latency 14.370 ms 2023-07-14T11:24:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:24:58 smithi175 ceph-mon[129081]: pgmap v84: 129 pgs: 129 active+clean; 113 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2023-07-14T11:24:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:24:58 smithi182 ceph-mon[129366]: pgmap v84: 129 pgs: 129 active+clean; 113 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2023-07-14T11:24:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:24:58 smithi161 ceph-mon[95526]: pgmap v84: 129 pgs: 129 active+clean; 113 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2023-07-14T11:24:59.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 193885 10.50 MB/sec warmup 101 sec latency 11.823 ms 2023-07-14T11:25:00.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 195653 10.52 MB/sec warmup 102 sec latency 24.296 ms 2023-07-14T11:25:00.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:00 smithi175 ceph-mon[129081]: pgmap v85: 129 pgs: 129 active+clean; 113 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:00 smithi182 ceph-mon[129366]: pgmap v85: 129 pgs: 129 active+clean; 113 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:00 smithi161 ceph-mon[95526]: pgmap v85: 129 pgs: 129 active+clean; 113 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:01.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 197361 10.47 MB/sec warmup 103 sec latency 60.149 ms 2023-07-14T11:25:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:01 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:25:01.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:01 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:25:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:01 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:25:02.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 198469 10.45 MB/sec warmup 104 sec latency 79.365 ms 2023-07-14T11:25:02.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:02 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:25:02.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:02 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:25:02.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:02 smithi175 ceph-mon[129081]: pgmap v86: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2023-07-14T11:25:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:02 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:25:02.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:02 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:25:02.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:02 smithi182 ceph-mon[129366]: pgmap v86: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2023-07-14T11:25:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:02 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:25:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:02 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:25:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:02 smithi161 ceph-mon[95526]: pgmap v86: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2023-07-14T11:25:03.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 199827 10.42 MB/sec warmup 105 sec latency 12.286 ms 2023-07-14T11:25:04.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:03 smithi175 ceph-mon[129081]: pgmap v87: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2023-07-14T11:25:04.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 202433 10.48 MB/sec warmup 106 sec latency 48.424 ms 2023-07-14T11:25:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:03 smithi182 ceph-mon[129366]: pgmap v87: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2023-07-14T11:25:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:03 smithi161 ceph-mon[95526]: pgmap v87: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2023-07-14T11:25:05.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 203780 10.43 MB/sec warmup 107 sec latency 15.920 ms 2023-07-14T11:25:06.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 206121 10.46 MB/sec warmup 108 sec latency 18.256 ms 2023-07-14T11:25:06.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:06 smithi175 ceph-mon[129081]: pgmap v88: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2023-07-14T11:25:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:06 smithi182 ceph-mon[129366]: pgmap v88: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2023-07-14T11:25:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:06 smithi161 ceph-mon[95526]: pgmap v88: 129 pgs: 129 active+clean; 120 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2023-07-14T11:25:07.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 207870 10.42 MB/sec warmup 109 sec latency 13.255 ms 2023-07-14T11:25:08.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 209746 10.45 MB/sec warmup 110 sec latency 20.912 ms 2023-07-14T11:25:08.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:08 smithi175 ceph-mon[129081]: pgmap v89: 129 pgs: 129 active+clean; 117 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 344 op/s 2023-07-14T11:25:08.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:08 smithi182 ceph-mon[129366]: pgmap v89: 129 pgs: 129 active+clean; 117 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 344 op/s 2023-07-14T11:25:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:08 smithi161 ceph-mon[95526]: pgmap v89: 129 pgs: 129 active+clean; 117 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 344 op/s 2023-07-14T11:25:09.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 211680 10.42 MB/sec warmup 111 sec latency 68.754 ms 2023-07-14T11:25:10.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 213424 10.43 MB/sec warmup 112 sec latency 13.391 ms 2023-07-14T11:25:10.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:10 smithi175 ceph-mon[129081]: pgmap v90: 129 pgs: 129 active+clean; 117 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2023-07-14T11:25:10.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:10 smithi182 ceph-mon[129366]: pgmap v90: 129 pgs: 129 active+clean; 117 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2023-07-14T11:25:10.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:10 smithi161 ceph-mon[95526]: pgmap v90: 129 pgs: 129 active+clean; 117 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2023-07-14T11:25:11.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 215470 10.41 MB/sec warmup 113 sec latency 28.214 ms 2023-07-14T11:25:12.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 217074 10.42 MB/sec warmup 114 sec latency 32.763 ms 2023-07-14T11:25:12.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:12 smithi175 ceph-mon[129081]: pgmap v91: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 339 op/s 2023-07-14T11:25:12.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:12 smithi182 ceph-mon[129366]: pgmap v91: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 339 op/s 2023-07-14T11:25:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:12 smithi161 ceph-mon[95526]: pgmap v91: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 339 op/s 2023-07-14T11:25:13.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 219342 10.43 MB/sec warmup 115 sec latency 64.849 ms 2023-07-14T11:25:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:13 smithi175 ceph-mon[129081]: pgmap v92: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2023-07-14T11:25:14.170 INFO:tasks.workunit.client.0.smithi161.stdout: 1 220746 10.40 MB/sec warmup 116 sec latency 34.480 ms 2023-07-14T11:25:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:13 smithi182 ceph-mon[129366]: pgmap v92: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2023-07-14T11:25:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:13 smithi161 ceph-mon[95526]: pgmap v92: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2023-07-14T11:25:15.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 223511 10.46 MB/sec warmup 117 sec latency 20.145 ms 2023-07-14T11:25:16.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 224639 10.40 MB/sec warmup 118 sec latency 64.841 ms 2023-07-14T11:25:16.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:16 smithi175 ceph-mon[129081]: pgmap v93: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2023-07-14T11:25:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:16 smithi182 ceph-mon[129366]: pgmap v93: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2023-07-14T11:25:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:16 smithi161 ceph-mon[95526]: pgmap v93: 129 pgs: 129 active+clean; 132 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2023-07-14T11:25:17.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 227202 10.45 MB/sec warmup 119 sec latency 33.812 ms 2023-07-14T11:25:18.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:18 smithi175 ceph-mon[129081]: pgmap v94: 129 pgs: 129 active+clean; 135 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2023-07-14T11:25:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:18 smithi182 ceph-mon[129366]: pgmap v94: 129 pgs: 129 active+clean; 135 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2023-07-14T11:25:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:18 smithi161 ceph-mon[95526]: pgmap v94: 129 pgs: 129 active+clean; 135 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2023-07-14T11:25:19.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 230849 13.75 MB/sec execute 1 sec latency 25.345 ms 2023-07-14T11:25:20.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 232296 9.66 MB/sec execute 2 sec latency 19.484 ms 2023-07-14T11:25:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:20 smithi175 ceph-mon[129081]: pgmap v95: 129 pgs: 129 active+clean; 135 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 214 op/s 2023-07-14T11:25:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:20 smithi182 ceph-mon[129366]: pgmap v95: 129 pgs: 129 active+clean; 135 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 214 op/s 2023-07-14T11:25:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:20 smithi161 ceph-mon[95526]: pgmap v95: 129 pgs: 129 active+clean; 135 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 214 op/s 2023-07-14T11:25:21.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 234530 11.06 MB/sec execute 3 sec latency 35.181 ms 2023-07-14T11:25:22.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 236331 9.74 MB/sec execute 4 sec latency 22.073 ms 2023-07-14T11:25:22.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:22 smithi175 ceph-mon[129081]: pgmap v96: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-07-14T11:25:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:22 smithi182 ceph-mon[129366]: pgmap v96: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-07-14T11:25:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:22 smithi161 ceph-mon[95526]: pgmap v96: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-07-14T11:25:23.064 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:25:23.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 238178 10.46 MB/sec execute 5 sec latency 21.079 ms 2023-07-14T11:25:23.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 -- 172.21.15.161:0/3027400259 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230105ae0 msgr2=0x7fe230105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 --2- 172.21.15.161:0/3027400259 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230105ae0 0x7fe230105eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009990 tx=0x7fe22c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 -- 172.21.15.161:0/3027400259 shutdown_connections 2023-07-14T11:25:23.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 --2- 172.21.15.161:0/3027400259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230100480 0x7fe23010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 --2- 172.21.15.161:0/3027400259 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230105ae0 0x7fe230105eb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 --2- 172.21.15.161:0/3027400259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2300ffad0 0x7fe2300fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 -- 172.21.15.161:0/3027400259 >> 172.21.15.161:0/3027400259 conn(0x7fe2300fb5f0 msgr2=0x7fe2300fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 -- 172.21.15.161:0/3027400259 shutdown_connections 2023-07-14T11:25:23.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.275+0000 7fe2383ff700 1 -- 172.21.15.161:0/3027400259 wait complete. 2023-07-14T11:25:23.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.276+0000 7fe2383ff700 1 Processor -- start 2023-07-14T11:25:23.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.276+0000 7fe2383ff700 1 -- start start 2023-07-14T11:25:23.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe2383ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2300ffad0 0x7fe230197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe2383ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230100480 0x7fe230198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe2383ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230105ae0 0x7fe23019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe2383ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019cdf0 con 0x7fe2300ffad0 2023-07-14T11:25:23.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe2383ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019cf60 con 0x7fe230105ae0 2023-07-14T11:25:23.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe2383ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019d120 con 0x7fe230100480 2023-07-14T11:25:23.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23619b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2300ffad0 0x7fe230197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230100480 0x7fe230198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230100480 0x7fe230198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56386/0 (socket says 172.21.15.161:56386) 2023-07-14T11:25:23.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23619b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2300ffad0 0x7fe230197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44046/0 (socket says 172.21.15.161:44046) 2023-07-14T11:25:23.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 -- 172.21.15.161:0/1715721165 learned_addr learned my addr 172.21.15.161:0/1715721165 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:23.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23699c700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230105ae0 0x7fe23019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 -- 172.21.15.161:0/1715721165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230105ae0 msgr2=0x7fe23019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230105ae0 0x7fe23019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 -- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2300ffad0 msgr2=0x7fe230197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2300ffad0 0x7fe230197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23599a700 1 -- 172.21.15.161:0/1715721165 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0095f0 con 0x7fe230100480 2023-07-14T11:25:23.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.277+0000 7fe23699c700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230105ae0 0x7fe23019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:25:23.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.278+0000 7fe23599a700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230100480 0x7fe230198350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe22000e9a0 tx=0x7fe22000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.278+0000 7fe2277fe700 1 -- 172.21.15.161:0/1715721165 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220009790 con 0x7fe230100480 2023-07-14T11:25:23.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.278+0000 7fe2277fe700 1 -- 172.21.15.161:0/1715721165 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe22000cbd0 con 0x7fe230100480 2023-07-14T11:25:23.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.278+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23019d400 con 0x7fe230100480 2023-07-14T11:25:23.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.278+0000 7fe2277fe700 1 -- 172.21.15.161:0/1715721165 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220018680 con 0x7fe230100480 2023-07-14T11:25:23.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.278+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301a4c40 con 0x7fe230100480 2023-07-14T11:25:23.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.281+0000 7fe2277fe700 1 -- 172.21.15.161:0/1715721165 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe22000cd40 con 0x7fe230100480 2023-07-14T11:25:23.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.282+0000 7fe24bcbfb80 1 Processor -- start 2023-07-14T11:25:23.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe24bcbfb80 1 -- start start 2023-07-14T11:25:23.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640b2d0 0x563f3640b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f36416590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36416ad0 0x563f36418eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f36311cb0 con 0x563f3640bbe0 2023-07-14T11:25:23.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f36364a40 con 0x563f3640b2d0 2023-07-14T11:25:23.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3635ded0 con 0x563f36416ad0 2023-07-14T11:25:23.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe2277fe700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe21c0a12c0 0x7fe21c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe2277fe700 1 -- 172.21.15.161:0/1715721165 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe220014070 con 0x7fe230100480 2023-07-14T11:25:23.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe23699c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640b2d0 0x563f3640b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe23699c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640b2d0 0x563f3640b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44970/0 (socket says 172.21.15.161:44970) 2023-07-14T11:25:23.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe23699c700 1 -- 172.21.15.161:0/1778983272 learned_addr learned my addr 172.21.15.161:0/1778983272 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:23.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe23619b700 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f36416590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe23599a700 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36416ad0 0x563f36418eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.283+0000 7fe23699c700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe21c0a12c0 0x7fe21c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23619b700 1 -- 172.21.15.161:0/1778983272 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36416ad0 msgr2=0x563f36418eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23619b700 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36416ad0 0x563f36418eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23619b700 1 -- 172.21.15.161:0/1778983272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640b2d0 msgr2=0x563f3640b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23619b700 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640b2d0 0x563f3640b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23619b700 1 -- 172.21.15.161:0/1778983272 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f3618bde0 con 0x563f3640bbe0 2023-07-14T11:25:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23699c700 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640b2d0 0x563f3640b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:23.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23699c700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe21c0a12c0 0x7fe21c0a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2280060b0 tx=0x7fe22800b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe23619b700 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f36416590 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe22c000c00 tx=0x7fe22c005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c809c712d3bbc9c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe224bfe700 1 -- 172.21.15.161:0/1778983272 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c01d070 con 0x563f3640bbe0 2023-07-14T11:25:23.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe224bfe700 1 -- 172.21.15.161:0/1778983272 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe22c01e040 con 0x563f3640bbe0 2023-07-14T11:25:23.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.284+0000 7fe224bfe700 1 -- 172.21.15.161:0/1778983272 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0047d0 con 0x563f3640bbe0 2023-07-14T11:25:23.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1778983272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 msgr2=0x563f36416590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f36416590 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe22c000c00 tx=0x7fe22c005ea0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1778983272 shutdown_connections 2023-07-14T11:25:23.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36416ad0 0x563f36418eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640b2d0 0x563f3640b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1778983272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f36416590 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1778983272 >> 172.21.15.161:0/1778983272 conn(0x563f36401da0 msgr2=0x563f364076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1778983272 shutdown_connections 2023-07-14T11:25:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.285+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1778983272 wait complete. 2023-07-14T11:25:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.286+0000 7fe24bcbfb80 1 -- start start 2023-07-14T11:25:23.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.286+0000 7fe24bcbfb80 1 Processor -- start 2023-07-14T11:25:23.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36416420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f364136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f36416ad0 0x563f36413c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f362fdac0 con 0x563f3640bbe0 2023-07-14T11:25:23.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f362fd7c0 con 0x563f36416ad0 2023-07-14T11:25:23.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f361e1920 con 0x563f3640b2d0 2023-07-14T11:25:23.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe24bcbfb80 10 client.0 _ll_register_callbacks cb 0x563f362fb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:23.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe23699c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f36416ad0 0x563f36413c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe23599a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f364136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe23599a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f364136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44062/0 (socket says 172.21.15.161:44062) 2023-07-14T11:25:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe23599a700 1 -- 172.21.15.161:0/3476102873 learned_addr learned my addr 172.21.15.161:0/3476102873 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.287+0000 7fe23619b700 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36416420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe23619b700 1 -- 172.21.15.161:0/3476102873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f36416ad0 msgr2=0x563f36413c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe23619b700 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f36416ad0 0x563f36413c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe23619b700 1 -- 172.21.15.161:0/3476102873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 msgr2=0x563f364136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe23619b700 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f364136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe23619b700 1 -- 172.21.15.161:0/3476102873 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe228011b40 con 0x563f3640b2d0 2023-07-14T11:25:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe23619b700 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36416420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b6f0 tx=0x7fe22c009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=894b45c5b47893f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe219ffb700 1 -- 172.21.15.161:0/3476102873 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c01d040 con 0x563f3640b2d0 2023-07-14T11:25:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f362b74f0 con 0x563f3640b2d0 2023-07-14T11:25:23.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe219ffb700 1 -- 172.21.15.161:0/3476102873 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe22c00f870 con 0x563f3640b2d0 2023-07-14T11:25:23.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe219ffb700 1 -- 172.21.15.161:0/3476102873 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe22c00fa30 con 0x563f3640b2d0 2023-07-14T11:25:23.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe24bcbfb80 10 client.26224 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:23.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.288+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe23010d210 con 0x563f3640b2d0 2023-07-14T11:25:23.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe219ffb700 1 -- 172.21.15.161:0/3476102873 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f362b74f0 con 0x563f3640b2d0 2023-07-14T11:25:23.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe219ffb700 1 -- 172.21.15.161:0/3476102873 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe22c031020 con 0x563f3640b2d0 2023-07-14T11:25:23.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe24bcbfb80 10 client.26224 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:23.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe24bcbfb80 10 client.26224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:23.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe24bcbfb80 20 client.26224 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:23.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36425e20 0x563f36428200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe24bcbfb80 4 client.26224 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:23.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f36356eb0 con 0x563f36425e20 2023-07-14T11:25:23.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.289+0000 7fe23599a700 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36425e20 0x563f36428200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.290+0000 7fe23599a700 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36425e20 0x563f36428200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e89e85c05c5f839 server_cookie=e2103a94a6aea672 in_seq=0 out_seq=0 2023-07-14T11:25:23.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.290+0000 7fe219ffb700 10 client.26224 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.292+0000 7fe219ffb700 1 -- 172.21.15.161:0/3476102873 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f36356eb0 con 0x563f36425e20 2023-07-14T11:25:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.292+0000 7fe219ffb700 10 client.26224 handle_command_reply: tid=0 2023-07-14T11:25:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.292+0000 7fe24bcbfb80 1 client.26224 shutdown 2023-07-14T11:25:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 msgr2=0x563f36416420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36416420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b6f0 tx=0x7fe22c009530 comp rx=0 tx=0).stop 2023-07-14T11:25:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 shutdown_connections 2023-07-14T11:25:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36425e20 0x563f36428200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f36416ad0 0x563f36413c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36416420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe219ffb700 0 client.26224 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:23.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/3476102873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640bbe0 0x563f364136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 >> 172.21.15.161:0/3476102873 conn(0x563f36401da0 msgr2=0x563f36407230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 shutdown_connections 2023-07-14T11:25:23.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/3476102873 wait complete. 2023-07-14T11:25:23.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.293+0000 7fe24bcbfb80 20 client.26224 trim_cache size 0 max 16384 2023-07-14T11:25:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 Processor -- start 2023-07-14T11:25:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 -- start start 2023-07-14T11:25:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f36401da0 0x563f36415fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36412e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f364133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f36403ec0 con 0x563f36401da0 2023-07-14T11:25:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f36408d20 con 0x563f3640bbe0 2023-07-14T11:25:23.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f36408330 con 0x563f3640b2d0 2023-07-14T11:25:23.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe23599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36412e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe23599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36412e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56404/0 (socket says 172.21.15.161:56404) 2023-07-14T11:25:23.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe23619b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f36401da0 0x563f36415fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe23599a700 1 -- 172.21.15.161:0/1627301182 learned_addr learned my addr 172.21.15.161:0/1627301182 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:23.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.312+0000 7fe23699c700 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f364133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe23599a700 1 -- 172.21.15.161:0/1627301182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 msgr2=0x563f364133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe23599a700 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f364133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe23599a700 1 -- 172.21.15.161:0/1627301182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f36401da0 msgr2=0x563f36415fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe23599a700 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f36401da0 0x563f36415fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe23599a700 1 -- 172.21.15.161:0/1627301182 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f36174c60 con 0x563f3640b2d0 2023-07-14T11:25:23.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe23619b700 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f36401da0 0x563f36415fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:25:23.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe23599a700 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36412e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe220096540 tx=0x7fe220011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c09190de2741884 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe2253ff700 1 -- 172.21.15.161:0/1627301182 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22009e070 con 0x563f3640b2d0 2023-07-14T11:25:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe2253ff700 1 -- 172.21.15.161:0/1627301182 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2200a2490 con 0x563f3640b2d0 2023-07-14T11:25:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.313+0000 7fe2253ff700 1 -- 172.21.15.161:0/1627301182 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220099530 con 0x563f3640b2d0 2023-07-14T11:25:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.314+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1627301182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 msgr2=0x563f36412e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.314+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36412e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe220096540 tx=0x7fe220011f90 comp rx=0 tx=0).stop 2023-07-14T11:25:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.314+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1627301182 shutdown_connections 2023-07-14T11:25:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.314+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f36401da0 0x563f36415fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.314+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f364133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.314+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1627301182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f3640b2d0 0x563f36412e60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.314+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1627301182 >> 172.21.15.161:0/1627301182 conn(0x563f36419020 msgr2=0x563f3640e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.315+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1627301182 shutdown_connections 2023-07-14T11:25:23.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.315+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1627301182 wait complete. 2023-07-14T11:25:23.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.315+0000 7fe24bcbfb80 1 -- start start 2023-07-14T11:25:23.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.315+0000 7fe24bcbfb80 1 Processor -- start 2023-07-14T11:25:23.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36401da0 0x563f3641d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640b2d0 0x563f364198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe24bcbfb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f36419e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3641a570 con 0x563f3640b2d0 2023-07-14T11:25:23.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3641a6e0 con 0x563f3640bbe0 2023-07-14T11:25:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe24bcbfb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f36479c70 con 0x563f36401da0 2023-07-14T11:25:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe24bcbfb80 10 client.0 _ll_register_callbacks cb 0x7fe2280113f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f36419e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f36419e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45006/0 (socket says 172.21.15.161:45006) 2023-07-14T11:25:23.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 -- 172.21.15.161:0/1287177158 learned_addr learned my addr 172.21.15.161:0/1287177158 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:23.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23599a700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640b2d0 0x563f364198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23619b700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36401da0 0x563f3641d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 -- 172.21.15.161:0/1287177158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36401da0 msgr2=0x563f3641d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36401da0 0x563f3641d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 -- 172.21.15.161:0/1287177158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640b2d0 msgr2=0x563f364198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:23.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640b2d0 0x563f364198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23699c700 1 -- 172.21.15.161:0/1287177158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301a4c40 con 0x563f3640bbe0 2023-07-14T11:25:23.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.316+0000 7fe23599a700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640b2d0 0x563f364198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:25:23.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.317+0000 7fe23699c700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f36419e30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe228009440 tx=0x7fe228004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc66efea62b0018 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.317+0000 7fe219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.317+0000 7fe219ffb700 1 -- 172.21.15.161:0/1287177158 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22802b070 con 0x563f3640bbe0 2023-07-14T11:25:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.317+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe22c01e040 con 0x563f3640bbe0 2023-07-14T11:25:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.317+0000 7fe219ffb700 1 -- 172.21.15.161:0/1287177158 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe228004bb0 con 0x563f3640bbe0 2023-07-14T11:25:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.317+0000 7fe219ffb700 1 -- 172.21.15.161:0/1287177158 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe228022070 con 0x563f3640bbe0 2023-07-14T11:25:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 10 client.26228 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563f361b2250 con 0x563f3640bbe0 2023-07-14T11:25:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe219ffb700 1 -- 172.21.15.161:0/1287177158 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe22c01e040 con 0x563f3640bbe0 2023-07-14T11:25:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe219ffb700 1 -- 172.21.15.161:0/1287177158 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe22802e020 con 0x563f3640bbe0 2023-07-14T11:25:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 10 client.26228 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 20 client.26228 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36416ad0 0x563f3647d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 4 client.26228 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:25:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f36319050 con 0x563f36416ad0 2023-07-14T11:25:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.318+0000 7fe23619b700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36416ad0 0x563f3647d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.319+0000 7fe23619b700 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36416ad0 0x563f3647d420 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1daabb62f1ad3e server_cookie=7e0c45a237301203 in_seq=0 out_seq=0 2023-07-14T11:25:23.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.319+0000 7fe219ffb700 10 client.26228 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:23.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.319+0000 7fe219ffb700 1 -- 172.21.15.161:0/1287177158 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563f36319050 con 0x563f36416ad0 2023-07-14T11:25:23.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.319+0000 7fe219ffb700 10 client.26228 handle_command_reply: tid=0 2023-07-14T11:25:23.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.319+0000 7fe24bcbfb80 1 client.26228 shutdown 2023-07-14T11:25:23.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 msgr2=0x563f36419e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f36419e30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe228009440 tx=0x7fe228004b40 comp rx=0 tx=0).stop 2023-07-14T11:25:23.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 shutdown_connections 2023-07-14T11:25:23.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563f36416ad0 0x563f3647d420 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f3640bbe0 0x563f36419e30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563f36401da0 0x563f3641d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 --2- 172.21.15.161:0/1287177158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f3640b2d0 0x563f364198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 >> 172.21.15.161:0/1287177158 conn(0x563f36419020 msgr2=0x563f3640d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe219ffb700 0 client.26228 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 shutdown_connections 2023-07-14T11:25:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 1 -- 172.21.15.161:0/1287177158 wait complete. 2023-07-14T11:25:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.320+0000 7fe24bcbfb80 20 client.26228 trim_cache size 0 max 16384 2023-07-14T11:25:23.350 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:25:23.350 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:25:23.350 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:25:23.350 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:25:23.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe21c0a12c0 msgr2=0x7fe21c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe21c0a12c0 0x7fe21c0a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2280060b0 tx=0x7fe22800b040 comp rx=0 tx=0).stop 2023-07-14T11:25:23.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230100480 msgr2=0x7fe230198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230100480 0x7fe230198350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe22000e9a0 tx=0x7fe22000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 shutdown_connections 2023-07-14T11:25:23.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe21c0a12c0 0x7fe21c0a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2300ffad0 0x7fe230197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230105ae0 0x7fe23019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 --2- 172.21.15.161:0/1715721165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe230100480 0x7fe230198350 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 >> 172.21.15.161:0/1715721165 conn(0x7fe2300fb5f0 msgr2=0x7fe2300fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 shutdown_connections 2023-07-14T11:25:23.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.321+0000 7fe2383ff700 1 -- 172.21.15.161:0/1715721165 wait complete. 2023-07-14T11:25:23.353 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:25:23.353 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:25:23.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/2791227353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 msgr2=0x7f6c28105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/2791227353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 0x7f6c28105ed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009990 tx=0x7f6c24009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/2791227353 shutdown_connections 2023-07-14T11:25:23.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/2791227353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281004a0 0x7f6c2810c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/2791227353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 0x7f6c28105ed0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/2791227353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c280ffaf0 0x7f6c280fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/2791227353 >> 172.21.15.161:0/2791227353 conn(0x7f6c280fb630 msgr2=0x7f6c280fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/2791227353 shutdown_connections 2023-07-14T11:25:23.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.567+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/2791227353 wait complete. 2023-07-14T11:25:23.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 Processor -- start 2023-07-14T11:25:23.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 -- start start 2023-07-14T11:25:23.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c280ffaf0 0x7f6c28197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281004a0 0x7f6c28198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 0x7f6c2819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2819ce00 con 0x7f6c281004a0 2023-07-14T11:25:23.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2819cf70 con 0x7f6c28105b00 2023-07-14T11:25:23.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.568+0000 7f6c2f5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2819d130 con 0x7f6c280ffaf0 2023-07-14T11:25:23.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2cb9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281004a0 0x7f6c28198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2db9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 0x7f6c2819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2cb9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281004a0 0x7f6c28198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44096/0 (socket says 172.21.15.161:44096) 2023-07-14T11:25:23.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2cb9a700 1 -- 172.21.15.161:0/153536922 learned_addr learned my addr 172.21.15.161:0/153536922 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:23.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2d39b700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c280ffaf0 0x7f6c28197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2db9c700 1 -- 172.21.15.161:0/153536922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c280ffaf0 msgr2=0x7f6c28197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2db9c700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c280ffaf0 0x7f6c28197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2db9c700 1 -- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281004a0 msgr2=0x7f6c28198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2db9c700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281004a0 0x7f6c28198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2db9c700 1 -- 172.21.15.161:0/153536922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20009c40 con 0x7f6c28105b00 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c2db9c700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 0x7f6c2819c7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000efb0 tx=0x7f6c2000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.569+0000 7f6c1e7fc700 1 -- 172.21.15.161:0/153536922 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20003a00 con 0x7f6c28105b00 2023-07-14T11:25:23.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.570+0000 7f6c1e7fc700 1 -- 172.21.15.161:0/153536922 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c2000f040 con 0x7f6c28105b00 2023-07-14T11:25:23.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.570+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c240095f0 con 0x7f6c28105b00 2023-07-14T11:25:23.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.570+0000 7f6c1e7fc700 1 -- 172.21.15.161:0/153536922 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004540 con 0x7f6c28105b00 2023-07-14T11:25:23.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.570+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c281a4a70 con 0x7f6c28105b00 2023-07-14T11:25:23.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.571+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c28067300 con 0x7f6c28105b00 2023-07-14T11:25:23.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.573+0000 7f6c1e7fc700 1 -- 172.21.15.161:0/153536922 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6c20003bf0 con 0x7f6c28105b00 2023-07-14T11:25:23.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.575+0000 7f6c1e7fc700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c140a1380 0x7f6c140a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:23.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.575+0000 7f6c1e7fc700 1 -- 172.21.15.161:0/153536922 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f6c200ccb60 con 0x7f6c28105b00 2023-07-14T11:25:23.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.575+0000 7f6c2d39b700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c140a1380 0x7f6c140a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:23.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.576+0000 7f6c2d39b700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c140a1380 0x7f6c140a3830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400b2e0 tx=0x7f6c24005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:23.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.576+0000 7f6c1e7fc700 1 -- 172.21.15.161:0/153536922 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c20095c70 con 0x7f6c28105b00 2023-07-14T11:25:23.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.845+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c28199170 con 0x7f6c28105b00 2023-07-14T11:25:23.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.846+0000 7f6c1e7fc700 1 -- 172.21.15.161:0/153536922 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f6c20099290 con 0x7f6c28105b00 2023-07-14T11:25:23.849 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:25:23.849 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:23:41.117589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:25:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.848+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c140a1380 msgr2=0x7f6c140a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.848+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c140a1380 0x7f6c140a3830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400b2e0 tx=0x7f6c24005fd0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.848+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 msgr2=0x7f6c2819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:23.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.848+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 0x7f6c2819c7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000efb0 tx=0x7f6c2000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 shutdown_connections 2023-07-14T11:25:23.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c140a1380 0x7f6c140a3830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c28105b00 0x7f6c2819c7c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c280ffaf0 0x7f6c28197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 --2- 172.21.15.161:0/153536922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281004a0 0x7f6c28198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:23.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 >> 172.21.15.161:0/153536922 conn(0x7f6c280fb630 msgr2=0x7f6c2810ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:23.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 shutdown_connections 2023-07-14T11:25:23.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:23.849+0000 7f6c2f5ff700 1 -- 172.21.15.161:0/153536922 wait complete. 2023-07-14T11:25:23.853 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2023-07-14T11:25:23.870 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:25:24.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 -- 172.21.15.161:0/3112253712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 msgr2=0x7f3c48109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 --2- 172.21.15.161:0/3112253712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 0x7f3c48109b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009990 tx=0x7f3c38009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 -- 172.21.15.161:0/3112253712 shutdown_connections 2023-07-14T11:25:24.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 --2- 172.21.15.161:0/3112253712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 0x7f3c481010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 --2- 172.21.15.161:0/3112253712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c4810a040 0x7f3c4810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 --2- 172.21.15.161:0/3112253712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 0x7f3c48109b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 -- 172.21.15.161:0/3112253712 >> 172.21.15.161:0/3112253712 conn(0x7f3c480fa4b0 msgr2=0x7f3c480fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 -- 172.21.15.161:0/3112253712 shutdown_connections 2023-07-14T11:25:24.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.080+0000 7f3c505ff700 1 -- 172.21.15.161:0/3112253712 wait complete. 2023-07-14T11:25:24.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.081+0000 7f3c505ff700 1 Processor -- start 2023-07-14T11:25:24.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c505ff700 1 -- start start 2023-07-14T11:25:24.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c505ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 0x7f3c48197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c505ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 0x7f3c481983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c505ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c4810a040 0x7f3c4819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c505ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4819ce40 con 0x7f3c48100d00 2023-07-14T11:25:24.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c505ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4819cfb0 con 0x7f3c48101610 2023-07-14T11:25:24.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c505ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4819d170 con 0x7f3c4810a040 2023-07-14T11:25:24.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c4e39b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 0x7f3c48197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c4db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 0x7f3c481983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c4e39b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 0x7f3c48197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44118/0 (socket says 172.21.15.161:44118) 2023-07-14T11:25:24.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c4e39b700 1 -- 172.21.15.161:0/2737646988 learned_addr learned my addr 172.21.15.161:0/2737646988 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.082+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c4810a040 0x7f3c4819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c4db9a700 1 -- 172.21.15.161:0/2737646988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c4810a040 msgr2=0x7f3c4819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c4810a040 0x7f3c4819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c4db9a700 1 -- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 msgr2=0x7f3c48197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 0x7f3c48197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c4db9a700 1 -- 172.21.15.161:0/2737646988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c380095f0 con 0x7f3c48101610 2023-07-14T11:25:24.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 0x7f3c48197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:24.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 0x7f3c481983a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c38009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c3f7fe700 1 -- 172.21.15.161:0/2737646988 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3801d070 con 0x7f3c48101610 2023-07-14T11:25:24.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c3f7fe700 1 -- 172.21.15.161:0/2737646988 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c38022470 con 0x7f3c48101610 2023-07-14T11:25:24.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c3f7fe700 1 -- 172.21.15.161:0/2737646988 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38017650 con 0x7f3c48101610 2023-07-14T11:25:24.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.083+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4819d3f0 con 0x7f3c48101610 2023-07-14T11:25:24.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.084+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4819d670 con 0x7f3c48101610 2023-07-14T11:25:24.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.087+0000 7f3c3f7fe700 1 -- 172.21.15.161:0/2737646988 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f3c380225e0 con 0x7f3c48101610 2023-07-14T11:25:24.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 Processor -- start 2023-07-14T11:25:24.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 -- start start 2023-07-14T11:25:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f7582d0 0x55c16f7586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f758be0 0x55c16f763590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f763ad0 0x55c16f765eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f65ecb0 con 0x55c16f758be0 2023-07-14T11:25:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f6b1a40 con 0x55c16f763ad0 2023-07-14T11:25:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f6aaed0 con 0x55c16f7582d0 2023-07-14T11:25:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.088+0000 7f3c3f7fe700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3c340a1340 0x7f3c340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c3f7fe700 1 -- 172.21.15.161:0/2737646988 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f3c380cdfc0 con 0x7f3c48101610 2023-07-14T11:25:24.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f7582d0 0x55c16f7586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f7582d0 0x55c16f7586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56470/0 (socket says 172.21.15.161:56470) 2023-07-14T11:25:24.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 -- 172.21.15.161:0/3795414790 learned_addr learned my addr 172.21.15.161:0/3795414790 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f763ad0 0x55c16f765eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3c340a1340 0x7f3c340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f758be0 0x55c16f763590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 -- 172.21.15.161:0/3795414790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f763ad0 msgr2=0x55c16f765eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f763ad0 0x55c16f765eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 -- 172.21.15.161:0/3795414790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f758be0 msgr2=0x55c16f763590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f758be0 0x55c16f763590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.089+0000 7f3c4eb9c700 1 -- 172.21.15.161:0/3795414790 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c16f4d8de0 con 0x55c16f7582d0 2023-07-14T11:25:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.090+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f7582d0 0x55c16f7586a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009fd0 tx=0x7f3c40008dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda8f670c8f5df66 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.090+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3c340a1340 0x7f3c340a37f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c40011000 tx=0x7f3c4001d000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.090+0000 7f3c3cbfe700 1 -- 172.21.15.161:0/3795414790 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000e070 con 0x55c16f7582d0 2023-07-14T11:25:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.090+0000 7f3c3cbfe700 1 -- 172.21.15.161:0/3795414790 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4000f040 con 0x55c16f7582d0 2023-07-14T11:25:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.090+0000 7f3c3cbfe700 1 -- 172.21.15.161:0/3795414790 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400247c0 con 0x55c16f7582d0 2023-07-14T11:25:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3795414790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f7582d0 msgr2=0x55c16f7586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f7582d0 0x55c16f7586a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009fd0 tx=0x7f3c40008dd0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3795414790 shutdown_connections 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f763ad0 0x55c16f765eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f7582d0 0x55c16f7586a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3795414790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f758be0 0x55c16f763590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3795414790 >> 172.21.15.161:0/3795414790 conn(0x55c16f74eda0 msgr2=0x55c16f7546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3795414790 shutdown_connections 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3795414790 wait complete. 2023-07-14T11:25:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.091+0000 7f3c63f28b80 1 -- start start 2023-07-14T11:25:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 1 Processor -- start 2023-07-14T11:25:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f760850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f758be0 0x55c16f760d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f763ad0 0x55c16f7612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f64aac0 con 0x55c16f7582d0 2023-07-14T11:25:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f64a7c0 con 0x55c16f758be0 2023-07-14T11:25:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f52e920 con 0x55c16f763ad0 2023-07-14T11:25:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c63f28b80 10 client.0 _ll_register_callbacks cb 0x55c16f648760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c4db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f758be0 0x55c16f760d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c4eb9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f763ad0 0x55c16f7612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c4db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f758be0 0x55c16f760d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45066/0 (socket says 172.21.15.161:45066) 2023-07-14T11:25:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.092+0000 7f3c4db9a700 1 -- 172.21.15.161:0/2040293787 learned_addr learned my addr 172.21.15.161:0/2040293787 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f760850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4eb9c700 1 -- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f758be0 msgr2=0x55c16f760d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f758be0 0x55c16f760d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4eb9c700 1 -- 172.21.15.161:0/2040293787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 msgr2=0x55c16f760850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f760850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4eb9c700 1 -- 172.21.15.161:0/2040293787 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c400086d0 con 0x55c16f763ad0 2023-07-14T11:25:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f758be0 0x55c16f760d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f763ad0 0x55c16f7612d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009960 tx=0x7f3c40024650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b889c1c1d784f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:24.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c31ffb700 1 -- 172.21.15.161:0/2040293787 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000e050 con 0x55c16f763ad0 2023-07-14T11:25:24.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c31ffb700 1 -- 172.21.15.161:0/2040293787 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f3c400249c0 con 0x55c16f763ad0 2023-07-14T11:25:24.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.093+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c16f6044f0 con 0x55c16f763ad0 2023-07-14T11:25:24.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c31ffb700 1 -- 172.21.15.161:0/2040293787 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c40009550 con 0x55c16f763ad0 2023-07-14T11:25:24.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 10 client.26248 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:24.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c16f4c1c60 con 0x55c16f763ad0 2023-07-14T11:25:24.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c31ffb700 1 -- 172.21.15.161:0/2040293787 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c16f6044f0 con 0x55c16f763ad0 2023-07-14T11:25:24.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c31ffb700 1 -- 172.21.15.161:0/2040293787 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3c40034060 con 0x55c16f763ad0 2023-07-14T11:25:24.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 10 client.26248 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:24.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 10 client.26248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:25:24.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 20 client.26248 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:24.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f773350 0x55c16f775730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 4 client.26248 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:24.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.094+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c16f6a3eb0 con 0x55c16f773350 2023-07-14T11:25:24.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.095+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f773350 0x55c16f775730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.095+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f773350 0x55c16f775730 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cffd140aa06cd21 server_cookie=69ccb1f4d43d42f1 in_seq=0 out_seq=0 2023-07-14T11:25:24.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.095+0000 7f3c31ffb700 10 client.26248 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c31ffb700 1 -- 172.21.15.161:0/2040293787 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c16f6a3eb0 con 0x55c16f773350 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c31ffb700 10 client.26248 handle_command_reply: tid=0 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 client.26248 shutdown 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f763ad0 msgr2=0x55c16f7612d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f763ad0 0x55c16f7612d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009960 tx=0x7f3c40024650 comp rx=0 tx=0).stop 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 shutdown_connections 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f773350 0x55c16f775730 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f760850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c31ffb700 0 client.26248 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f763ad0 0x55c16f7612d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/2040293787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f758be0 0x55c16f760d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.096+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 >> 172.21.15.161:0/2040293787 conn(0x55c16f74eda0 msgr2=0x55c16f754290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.097+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 shutdown_connections 2023-07-14T11:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.097+0000 7f3c63f28b80 1 -- 172.21.15.161:0/2040293787 wait complete. 2023-07-14T11:25:24.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.097+0000 7f3c63f28b80 20 client.26248 trim_cache size 0 max 16384 2023-07-14T11:25:24.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 Processor -- start 2023-07-14T11:25:24.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 -- start start 2023-07-14T11:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f74eda0 0x55c16f76a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f7582d0 0x55c16f75dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f75e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f6044f0 con 0x55c16f74eda0 2023-07-14T11:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f773180 con 0x55c16f7582d0 2023-07-14T11:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f750ec0 con 0x55c16f758be0 2023-07-14T11:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c4eb9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f75e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c4db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f7582d0 0x55c16f75dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c4e39b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f74eda0 0x55c16f76a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c4db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f7582d0 0x55c16f75dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45072/0 (socket says 172.21.15.161:45072) 2023-07-14T11:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c4eb9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f75e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56488/0 (socket says 172.21.15.161:56488) 2023-07-14T11:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c4db9a700 1 -- 172.21.15.161:0/1691033542 learned_addr learned my addr 172.21.15.161:0/1691033542 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.113+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f74eda0 0x55c16f76a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44148/0 (socket says 172.21.15.161:44148) 2023-07-14T11:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c4db9a700 1 -- 172.21.15.161:0/1691033542 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 msgr2=0x55c16f75e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f75e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c4db9a700 1 -- 172.21.15.161:0/1691033542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f74eda0 msgr2=0x55c16f76a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f74eda0 0x55c16f76a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c4db9a700 1 -- 172.21.15.161:0/1691033542 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c16f7603c0 con 0x55c16f7582d0 2023-07-14T11:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f7582d0 0x55c16f75dac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c38022fd0 tx=0x7f3c38075040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e23855943826d79 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c3d3ff700 1 -- 172.21.15.161:0/1691033542 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380970d0 con 0x55c16f7582d0 2023-07-14T11:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c3d3ff700 1 -- 172.21.15.161:0/1691033542 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3809ad90 con 0x55c16f7582d0 2023-07-14T11:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.114+0000 7f3c3d3ff700 1 -- 172.21.15.161:0/1691033542 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3809b210 con 0x55c16f7582d0 2023-07-14T11:25:24.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 -- 172.21.15.161:0/1691033542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f7582d0 msgr2=0x55c16f75dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f7582d0 0x55c16f75dac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c38022fd0 tx=0x7f3c38075040 comp rx=0 tx=0).stop 2023-07-14T11:25:24.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 -- 172.21.15.161:0/1691033542 shutdown_connections 2023-07-14T11:25:24.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f74eda0 0x55c16f76a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f75e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/1691033542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f7582d0 0x55c16f75dac0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 -- 172.21.15.161:0/1691033542 >> 172.21.15.161:0/1691033542 conn(0x55c16f750690 msgr2=0x55c16f75bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 -- 172.21.15.161:0/1691033542 shutdown_connections 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 -- 172.21.15.161:0/1691033542 wait complete. 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.115+0000 7f3c63f28b80 1 -- start start 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 1 Processor -- start 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f74eda0 0x55c16f768300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f768840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f7c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f7c6720 con 0x55c16f7582d0 2023-07-14T11:25:24.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f7c6860 con 0x55c16f74eda0 2023-07-14T11:25:24.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16f7c69d0 con 0x55c16f758be0 2023-07-14T11:25:24.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c63f28b80 10 client.0 _ll_register_callbacks cb 0x7f3c440095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:24.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c4e39b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f74eda0 0x55c16f768300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c4db9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f768840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c4e39b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f74eda0 0x55c16f768300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45082/0 (socket says 172.21.15.161:45082) 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c4e39b700 1 -- 172.21.15.161:0/3612530971 learned_addr learned my addr 172.21.15.161:0/3612530971 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.116+0000 7f3c4eb9c700 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f7c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c4e39b700 1 -- 172.21.15.161:0/3612530971 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 msgr2=0x55c16f7c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f7c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c4e39b700 1 -- 172.21.15.161:0/3612530971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 msgr2=0x55c16f768840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f768840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c4e39b700 1 -- 172.21.15.161:0/3612530971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4819d3f0 con 0x55c16f74eda0 2023-07-14T11:25:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c4e39b700 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f74eda0 0x55c16f768300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005d90 tx=0x7f3c44005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde64607fa69ca6a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c31ffb700 1 -- 172.21.15.161:0/3612530971 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44017040 con 0x55c16f74eda0 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c4000f040 con 0x55c16f74eda0 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.117+0000 7f3c31ffb700 1 -- 172.21.15.161:0/3612530971 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f3c440210b0 con 0x55c16f74eda0 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c31ffb700 1 -- 172.21.15.161:0/3612530971 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c440122d0 con 0x55c16f74eda0 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 10 client.26252 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c16f4ff250 con 0x55c16f74eda0 2023-07-14T11:25:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c31ffb700 1 -- 172.21.15.161:0/3612530971 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4000f040 con 0x55c16f74eda0 2023-07-14T11:25:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c31ffb700 1 -- 172.21.15.161:0/3612530971 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3c44010ce0 con 0x55c16f74eda0 2023-07-14T11:25:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 10 client.26252 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 10 client.26252 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:25:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 20 client.26252 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f763ad0 0x55c16f7ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 4 client.26252 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:25:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c16f666050 con 0x55c16f763ad0 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.118+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f763ad0 0x55c16f7ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c4db9a700 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f763ad0 0x55c16f7ca130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f08da17582939559 server_cookie=f1cec6fcb59c84a8 in_seq=0 out_seq=0 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c31ffb700 10 client.26252 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c31ffb700 1 -- 172.21.15.161:0/3612530971 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c16f666050 con 0x55c16f763ad0 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c31ffb700 10 client.26252 handle_command_reply: tid=0 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 client.26252 shutdown 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f74eda0 msgr2=0x55c16f768300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f74eda0 0x55c16f768300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005d90 tx=0x7f3c44005e40 comp rx=0 tx=0).stop 2023-07-14T11:25:24.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 shutdown_connections 2023-07-14T11:25:24.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55c16f763ad0 0x55c16f7ca130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c16f758be0 0x55c16f7c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16f74eda0 0x55c16f768300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c31ffb700 0 client.26252 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:24.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 --2- 172.21.15.161:0/3612530971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c16f7582d0 0x55c16f768840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.119+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 >> 172.21.15.161:0/3612530971 conn(0x55c16f750690 msgr2=0x55c16f75f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 shutdown_connections 2023-07-14T11:25:24.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c63f28b80 1 -- 172.21.15.161:0/3612530971 wait complete. 2023-07-14T11:25:24.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c63f28b80 20 client.26252 trim_cache size 0 max 16384 2023-07-14T11:25:24.131 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3c340a1340 msgr2=0x7f3c340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3c340a1340 0x7f3c340a37f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c40011000 tx=0x7f3c4001d000 comp rx=0 tx=0).stop 2023-07-14T11:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 msgr2=0x7f3c481983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 0x7f3c481983a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c38009580 comp rx=0 tx=0).stop 2023-07-14T11:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 shutdown_connections 2023-07-14T11:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3c340a1340 0x7f3c340a37f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c48100d00 0x7f3c48197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c4810a040 0x7f3c4819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 --2- 172.21.15.161:0/2737646988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c48101610 0x7f3c481983a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 >> 172.21.15.161:0/2737646988 conn(0x7f3c480fa4b0 msgr2=0x7f3c4810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.120+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 shutdown_connections 2023-07-14T11:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.121+0000 7f3c505ff700 1 -- 172.21.15.161:0/2737646988 wait complete. 2023-07-14T11:25:24.140 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:23 smithi161 ceph-mon[95526]: pgmap v97: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-14T11:25:24.143 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:25:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:23 smithi175 ceph-mon[129081]: pgmap v97: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-14T11:25:24.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 239896 9.68 MB/sec execute 6 sec latency 25.542 ms 2023-07-14T11:25:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:23 smithi182 ceph-mon[129366]: pgmap v97: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-14T11:25:24.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 -- 172.21.15.161:0/777782251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c073140 msgr2=0x7fac9c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 --2- 172.21.15.161:0/777782251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c073140 0x7fac9c10be60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fac90009990 tx=0x7fac90009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 -- 172.21.15.161:0/777782251 shutdown_connections 2023-07-14T11:25:24.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 --2- 172.21.15.161:0/777782251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c10c3a0 0x7fac9c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 --2- 172.21.15.161:0/777782251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac9c072830 0x7fac9c072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 --2- 172.21.15.161:0/777782251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c073140 0x7fac9c10be60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 -- 172.21.15.161:0/777782251 >> 172.21.15.161:0/777782251 conn(0x7fac9c0fb5e0 msgr2=0x7fac9c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 -- 172.21.15.161:0/777782251 shutdown_connections 2023-07-14T11:25:24.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.351+0000 7faca15ff700 1 -- 172.21.15.161:0/777782251 wait complete. 2023-07-14T11:25:24.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.352+0000 7faca15ff700 1 Processor -- start 2023-07-14T11:25:24.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.352+0000 7faca15ff700 1 -- start start 2023-07-14T11:25:24.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7faca15ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c072830 0x7fac9c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7faca15ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac9c073140 0x7fac9c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7faca15ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c10c3a0 0x7fac9c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7faca15ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c19ce10 con 0x7fac9c072830 2023-07-14T11:25:24.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7faca15ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c19cf80 con 0x7fac9c10c3a0 2023-07-14T11:25:24.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7faca15ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c19d140 con 0x7fac9c073140 2023-07-14T11:25:24.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c072830 0x7fac9c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c072830 0x7fac9c197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44192/0 (socket says 172.21.15.161:44192) 2023-07-14T11:25:24.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9affd700 1 -- 172.21.15.161:0/1307447932 learned_addr learned my addr 172.21.15.161:0/1307447932 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c10c3a0 0x7fac9c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac9c073140 0x7fac9c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9b7fe700 1 -- 172.21.15.161:0/1307447932 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac9c073140 msgr2=0x7fac9c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac9c073140 0x7fac9c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9b7fe700 1 -- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c072830 msgr2=0x7fac9c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c072830 0x7fac9c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9b7fe700 1 -- 172.21.15.161:0/1307447932 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac94009c40 con 0x7fac9c10c3a0 2023-07-14T11:25:24.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9affd700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c072830 0x7fac9c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:24.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.353+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac9c073140 0x7fac9c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:24.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.354+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c10c3a0 0x7fac9c19c7d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fac9400efb0 tx=0x7fac9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.354+0000 7fac83fff700 1 -- 172.21.15.161:0/1307447932 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94003a00 con 0x7fac9c10c3a0 2023-07-14T11:25:24.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.354+0000 7fac83fff700 1 -- 172.21.15.161:0/1307447932 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9400f040 con 0x7fac9c10c3a0 2023-07-14T11:25:24.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.354+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac900095f0 con 0x7fac9c10c3a0 2023-07-14T11:25:24.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.354+0000 7fac83fff700 1 -- 172.21.15.161:0/1307447932 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94004540 con 0x7fac9c10c3a0 2023-07-14T11:25:24.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.354+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c19d610 con 0x7fac9c10c3a0 2023-07-14T11:25:24.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.357+0000 7fac83fff700 1 -- 172.21.15.161:0/1307447932 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fac94003bf0 con 0x7fac9c10c3a0 2023-07-14T11:25:24.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7fac83fff700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fac840a1340 0x7fac840a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 Processor -- start 2023-07-14T11:25:24.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7fac83fff700 1 -- 172.21.15.161:0/1307447932 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fac940ccba0 con 0x7fac9c10c3a0 2023-07-14T11:25:24.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 -- start start 2023-07-14T11:25:24.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7fac9affd700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fac840a1340 0x7fac840a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad75b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad75bad0 0x55f0ad75deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad656cb0 con 0x55f0ad75bad0 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad6a9a40 con 0x55f0ad7502d0 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.359+0000 7facb4f98b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad6a2ed0 con 0x55f0ad750be0 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45116/0 (socket says 172.21.15.161:45116) 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 -- 172.21.15.161:0/1431325938 learned_addr learned my addr 172.21.15.161:0/1431325938 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad75bad0 0x55f0ad75deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad75b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fac840a1340 0x7fac840a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0115a0 tx=0x7fac8c009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 -- 172.21.15.161:0/1431325938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 msgr2=0x55f0ad75b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad75b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 -- 172.21.15.161:0/1431325938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad75bad0 msgr2=0x55f0ad75deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad75bad0 0x55f0ad75deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 -- 172.21.15.161:0/1431325938 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0ad4d0de0 con 0x55f0ad7502d0 2023-07-14T11:25:24.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad75bad0 0x55f0ad75deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:25:24.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.360+0000 7fac9affd700 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7506a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac8c01c6d0 tx=0x7fac8c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1ea160ed7206c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.361+0000 7fac817fa700 1 -- 172.21.15.161:0/1431325938 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c024430 con 0x55f0ad7502d0 2023-07-14T11:25:24.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.361+0000 7fac817fa700 1 -- 172.21.15.161:0/1431325938 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac8c024a50 con 0x55f0ad7502d0 2023-07-14T11:25:24.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.361+0000 7fac817fa700 1 -- 172.21.15.161:0/1431325938 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00fc20 con 0x55f0ad7502d0 2023-07-14T11:25:24.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.361+0000 7facb4f98b80 1 -- 172.21.15.161:0/1431325938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 msgr2=0x55f0ad7506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.361+0000 7facb4f98b80 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7506a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac8c01c6d0 tx=0x7fac8c01c000 comp rx=0 tx=0).stop 2023-07-14T11:25:24.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 -- 172.21.15.161:0/1431325938 shutdown_connections 2023-07-14T11:25:24.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad75bad0 0x55f0ad75deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7506a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 --2- 172.21.15.161:0/1431325938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad75b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 -- 172.21.15.161:0/1431325938 >> 172.21.15.161:0/1431325938 conn(0x55f0ad746da0 msgr2=0x55f0ad74c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 -- 172.21.15.161:0/1431325938 shutdown_connections 2023-07-14T11:25:24.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 -- 172.21.15.161:0/1431325938 wait complete. 2023-07-14T11:25:24.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 -- start start 2023-07-14T11:25:24.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.362+0000 7facb4f98b80 1 Processor -- start 2023-07-14T11:25:24.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad75b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad7587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad75bad0 0x55f0ad758f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7facb4f98b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad642ac0 con 0x55f0ad7502d0 2023-07-14T11:25:24.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7facb4f98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad6427c0 con 0x55f0ad75bad0 2023-07-14T11:25:24.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7facb4f98b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad526920 con 0x55f0ad750be0 2023-07-14T11:25:24.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7facb4f98b80 10 client.0 _ll_register_callbacks cb 0x55f0ad640760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:24.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad75b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad75b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44218/0 (socket says 172.21.15.161:44218) 2023-07-14T11:25:24.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9affd700 1 -- 172.21.15.161:0/32139713 learned_addr learned my addr 172.21.15.161:0/32139713 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad75bad0 0x55f0ad758f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad7587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9b7fe700 1 -- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 msgr2=0x55f0ad7587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad7587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9b7fe700 1 -- 172.21.15.161:0/32139713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 msgr2=0x55f0ad75b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad75b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.363+0000 7fac9b7fe700 1 -- 172.21.15.161:0/32139713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8c017050 con 0x55f0ad75bad0 2023-07-14T11:25:24.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad7587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:25:24.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad75bad0 0x55f0ad758f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac940c9f60 tx=0x7fac940992d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfab8ee131b7837b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:24.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac7eff5700 1 -- 172.21.15.161:0/32139713 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac940b20e0 con 0x55f0ad75bad0 2023-07-14T11:25:24.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac7eff5700 1 -- 172.21.15.161:0/32139713 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fac940a1d30 con 0x55f0ad75bad0 2023-07-14T11:25:24.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0ad5fc4f0 con 0x55f0ad75bad0 2023-07-14T11:25:24.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac7eff5700 1 -- 172.21.15.161:0/32139713 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac940046a0 con 0x55f0ad75bad0 2023-07-14T11:25:24.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7facb4f98b80 10 client.26264 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f0ad4b9c60 con 0x55f0ad75bad0 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac7eff5700 1 -- 172.21.15.161:0/32139713 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0ad5fc4f0 con 0x55f0ad75bad0 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.364+0000 7fac7eff5700 1 -- 172.21.15.161:0/32139713 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fac94098ef0 con 0x55f0ad75bad0 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7facb4f98b80 10 client.26264 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7facb4f98b80 10 client.26264 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7facb4f98b80 20 client.26264 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7facb4f98b80 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad76b370 0x55f0ad76d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7facb4f98b80 4 client.26264 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:24.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0ad69beb0 con 0x55f0ad76b370 2023-07-14T11:25:24.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7fac9affd700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad76b370 0x55f0ad76d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7fac9affd700 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad76b370 0x55f0ad76d750 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=876c03d26b3028cd server_cookie=20977b8770006ec0 in_seq=0 out_seq=0 2023-07-14T11:25:24.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.365+0000 7fac7eff5700 10 client.26264 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:24.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7fac7eff5700 1 -- 172.21.15.161:0/32139713 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0ad69beb0 con 0x55f0ad76b370 2023-07-14T11:25:24.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7fac7eff5700 10 client.26264 handle_command_reply: tid=0 2023-07-14T11:25:24.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 client.26264 shutdown 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad75bad0 msgr2=0x55f0ad758f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad75bad0 0x55f0ad758f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac940c9f60 tx=0x7fac940992d0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 shutdown_connections 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad76b370 0x55f0ad76d750 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad75b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad75bad0 0x55f0ad758f30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 --2- 172.21.15.161:0/32139713 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad750be0 0x55f0ad7587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7fac7eff5700 0 client.26264 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:24.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 >> 172.21.15.161:0/32139713 conn(0x55f0ad746da0 msgr2=0x55f0ad74c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 shutdown_connections 2023-07-14T11:25:24.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.366+0000 7facb4f98b80 1 -- 172.21.15.161:0/32139713 wait complete. 2023-07-14T11:25:24.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.367+0000 7facb4f98b80 20 client.26264 trim_cache size 0 max 16384 2023-07-14T11:25:24.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 Processor -- start 2023-07-14T11:25:24.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 -- start start 2023-07-14T11:25:24.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad750be0 0x55f0ad753af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad754030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad751e70 con 0x55f0ad750be0 2023-07-14T11:25:24.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad751540 con 0x55f0ad7502d0 2023-07-14T11:25:24.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.381+0000 7facb4f98b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad74dd00 con 0x55f0ad75bad0 2023-07-14T11:25:24.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad750be0 0x55f0ad753af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad750be0 0x55f0ad753af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44230/0 (socket says 172.21.15.161:44230) 2023-07-14T11:25:24.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 -- 172.21.15.161:0/947073739 learned_addr learned my addr 172.21.15.161:0/947073739 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9affd700 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad754030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 -- 172.21.15.161:0/947073739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 msgr2=0x55f0ad754030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad754030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 -- 172.21.15.161:0/947073739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 msgr2=0x55f0ad7531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 -- 172.21.15.161:0/947073739 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0ad74b0e0 con 0x55f0ad750be0 2023-07-14T11:25:24.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.382+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad750be0 0x55f0ad753af0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fac900098e0 tx=0x7fac90005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5b9f0eaae1b996 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7fac81ffb700 1 -- 172.21.15.161:0/947073739 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9001d070 con 0x55f0ad750be0 2023-07-14T11:25:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7fac81ffb700 1 -- 172.21.15.161:0/947073739 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9001e040 con 0x55f0ad750be0 2023-07-14T11:25:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7fac81ffb700 1 -- 172.21.15.161:0/947073739 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000f460 con 0x55f0ad750be0 2023-07-14T11:25:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 -- 172.21.15.161:0/947073739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad750be0 msgr2=0x55f0ad753af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad750be0 0x55f0ad753af0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fac900098e0 tx=0x7fac90005c70 comp rx=0 tx=0).stop 2023-07-14T11:25:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 -- 172.21.15.161:0/947073739 shutdown_connections 2023-07-14T11:25:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad754030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad7502d0 0x55f0ad7531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 --2- 172.21.15.161:0/947073739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad750be0 0x55f0ad753af0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 -- 172.21.15.161:0/947073739 >> 172.21.15.161:0/947073739 conn(0x55f0ad746da0 msgr2=0x55f0ad767720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 -- 172.21.15.161:0/947073739 shutdown_connections 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.383+0000 7facb4f98b80 1 -- 172.21.15.161:0/947073739 wait complete. 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.384+0000 7facb4f98b80 1 -- start start 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.384+0000 7facb4f98b80 1 Processor -- start 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad762110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad750be0 0x55f0ad762650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7facb4f98b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad7605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7facb4f98b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad760ce0 con 0x55f0ad7502d0 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7facb4f98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad760e50 con 0x55f0ad750be0 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7facb4f98b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ad761010 con 0x55f0ad75bad0 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7facb4f98b80 10 client.0 _ll_register_callbacks cb 0x55f0ad640760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad762110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad750be0 0x55f0ad762650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad762110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44236/0 (socket says 172.21.15.161:44236) 2023-07-14T11:25:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 -- 172.21.15.161:0/404455282 learned_addr learned my addr 172.21.15.161:0/404455282 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9b7fe700 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad7605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 -- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 msgr2=0x55f0ad7605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad7605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 -- 172.21.15.161:0/404455282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad750be0 msgr2=0x55f0ad762650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad750be0 0x55f0ad762650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 -- 172.21.15.161:0/404455282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c10f380 con 0x55f0ad7502d0 2023-07-14T11:25:24.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac9affd700 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad762110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fac8c01c6d0 tx=0x7fac8c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1926a30a0c4931 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.385+0000 7fac7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:25:24.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7fac7eff5700 1 -- 172.21.15.161:0/404455282 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c035040 con 0x55f0ad7502d0 2023-07-14T11:25:24.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac8c024a50 con 0x55f0ad7502d0 2023-07-14T11:25:24.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7fac7eff5700 1 -- 172.21.15.161:0/404455282 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fac8c038590 con 0x55f0ad7502d0 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7fac7eff5700 1 -- 172.21.15.161:0/404455282 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac8c024dc0 con 0x55f0ad7502d0 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 10 client.16224 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f0ad4f7250 con 0x55f0ad7502d0 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7fac7eff5700 1 -- 172.21.15.161:0/404455282 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac8c024a50 con 0x55f0ad7502d0 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7fac7eff5700 1 -- 172.21.15.161:0/404455282 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fac8c00f7f0 con 0x55f0ad7502d0 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 10 client.16224 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 10 client.16224 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:25:24.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 20 client.16224 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:24.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad7c0400 0x55f0ad7c2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 4 client.16224 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:25:24.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.386+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f0ad65e050 con 0x55f0ad7c0400 2023-07-14T11:25:24.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad7c0400 0x55f0ad7c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7fac7eff5700 10 client.16224 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:24.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7fac9a7fc700 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad7c0400 0x55f0ad7c2810 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72d610c2bc765f68 server_cookie=e3f77831880e522 in_seq=0 out_seq=0 2023-07-14T11:25:24.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7fac7eff5700 1 -- 172.21.15.161:0/404455282 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f0ad65e050 con 0x55f0ad7c0400 2023-07-14T11:25:24.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7fac7eff5700 10 client.16224 handle_command_reply: tid=0 2023-07-14T11:25:24.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7facb4f98b80 1 client.16224 shutdown 2023-07-14T11:25:24.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 msgr2=0x55f0ad762110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.387+0000 7facb4f98b80 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad762110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fac8c01c6d0 tx=0x7fac8c00f6a0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 shutdown_connections 2023-07-14T11:25:24.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55f0ad7c0400 0x55f0ad7c2810 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0ad7502d0 0x55f0ad762110 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f0ad75bad0 0x55f0ad7605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7fac7eff5700 0 client.16224 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:24.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 --2- 172.21.15.161:0/404455282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0ad750be0 0x55f0ad762650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 >> 172.21.15.161:0/404455282 conn(0x55f0ad746da0 msgr2=0x55f0ad753af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 shutdown_connections 2023-07-14T11:25:24.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 1 -- 172.21.15.161:0/404455282 wait complete. 2023-07-14T11:25:24.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7facb4f98b80 20 client.16224 trim_cache size 0 max 16384 2023-07-14T11:25:24.400 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:25:24.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fac840a1340 msgr2=0x7fac840a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7faca15ff700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fac840a1340 0x7fac840a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0115a0 tx=0x7fac8c009260 comp rx=0 tx=0).stop 2023-07-14T11:25:24.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c10c3a0 msgr2=0x7fac9c19c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.388+0000 7faca15ff700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c10c3a0 0x7fac9c19c7d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fac9400efb0 tx=0x7fac9400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 shutdown_connections 2023-07-14T11:25:24.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fac840a1340 0x7fac840a37f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac9c072830 0x7fac9c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac9c10c3a0 0x7fac9c19c7d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 --2- 172.21.15.161:0/1307447932 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fac9c073140 0x7fac9c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 >> 172.21.15.161:0/1307447932 conn(0x7fac9c0fb5e0 msgr2=0x7fac9c10e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 shutdown_connections 2023-07-14T11:25:24.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.389+0000 7faca15ff700 1 -- 172.21.15.161:0/1307447932 wait complete. 2023-07-14T11:25:24.410 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:25:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 -- 172.21.15.161:0/1342823831 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f35280728b0 msgr2=0x7f3528072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 --2- 172.21.15.161:0/1342823831 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f35280728b0 0x7f3528072c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f35180098f0 tx=0x7f3518009c00 comp rx=0 tx=0).stop 2023-07-14T11:25:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 -- 172.21.15.161:0/1342823831 shutdown_connections 2023-07-14T11:25:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 --2- 172.21.15.161:0/1342823831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f352810c310 0x7f352810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 --2- 172.21.15.161:0/1342823831 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f35280728b0 0x7f3528072c80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 --2- 172.21.15.161:0/1342823831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35280731c0 0x7f352810bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 -- 172.21.15.161:0/1342823831 >> 172.21.15.161:0/1342823831 conn(0x7f35280fb5f0 msgr2=0x7f35280fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 -- 172.21.15.161:0/1342823831 shutdown_connections 2023-07-14T11:25:24.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.663+0000 7f352dbff700 1 -- 172.21.15.161:0/1342823831 wait complete. 2023-07-14T11:25:24.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.664+0000 7f352dbff700 1 Processor -- start 2023-07-14T11:25:24.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f352dbff700 1 -- start start 2023-07-14T11:25:24.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f352dbff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280728b0 0x7f35280ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f352dbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35280731c0 0x7f35280fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f352dbff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f352810c310 0x7f35281004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f352dbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35281a85e0 con 0x7f35280731c0 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f352dbff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35281a8750 con 0x7f35280728b0 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f352dbff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35281a8910 con 0x7f352810c310 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f35277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280728b0 0x7f35280ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f35277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280728b0 0x7f35280ffa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45146/0 (socket says 172.21.15.161:45146) 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f35277fe700 1 -- 172.21.15.161:0/348462099 learned_addr learned my addr 172.21.15.161:0/348462099 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f3527fff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f352810c310 0x7f35281004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.665+0000 7f3526ffd700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35280731c0 0x7f35280fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f35277fe700 1 -- 172.21.15.161:0/348462099 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f352810c310 msgr2=0x7f35281004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f35277fe700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f352810c310 0x7f35281004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f35277fe700 1 -- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35280731c0 msgr2=0x7f35280fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f35277fe700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35280731c0 0x7f35280fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f35277fe700 1 -- 172.21.15.161:0/348462099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3520009d00 con 0x7f35280728b0 2023-07-14T11:25:24.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f3527fff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f352810c310 0x7f35281004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:24.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f35277fe700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280728b0 0x7f35280ffa30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3518009810 tx=0x7f351800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f3524ff9700 1 -- 172.21.15.161:0/348462099 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351801c070 con 0x7f35280728b0 2023-07-14T11:25:24.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f3524ff9700 1 -- 172.21.15.161:0/348462099 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f351800bb20 con 0x7f35280728b0 2023-07-14T11:25:24.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.666+0000 7f3524ff9700 1 -- 172.21.15.161:0/348462099 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518004930 con 0x7f35280728b0 2023-07-14T11:25:24.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.667+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3518009520 con 0x7f35280728b0 2023-07-14T11:25:24.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.667+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35281a8e60 con 0x7f35280728b0 2023-07-14T11:25:24.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.670+0000 7f3524ff9700 1 -- 172.21.15.161:0/348462099 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f3518020070 con 0x7f35280728b0 2023-07-14T11:25:24.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.671+0000 7f3541517b80 1 Processor -- start 2023-07-14T11:25:24.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.671+0000 7f3541517b80 1 -- start start 2023-07-14T11:25:24.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3524ff9700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f35100a1340 0x7f35100a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3524ff9700 1 -- 172.21.15.161:0/348462099 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f35180cdcc0 con 0x7f35280728b0 2023-07-14T11:25:24.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3541517b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcc16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3541517b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3541517b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcccad0 0x55f91bcceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3541517b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bbc7cb0 con 0x55f91bcc1be0 2023-07-14T11:25:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3541517b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bc1aa40 con 0x55f91bcccad0 2023-07-14T11:25:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3541517b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bc13ed0 con 0x55f91bcc12d0 2023-07-14T11:25:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3527fff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f35100a1340 0x7f35100a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3526ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3527fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcccad0 0x55f91bcceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3526ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44266/0 (socket says 172.21.15.161:44266) 2023-07-14T11:25:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3526ffd700 1 -- 172.21.15.161:0/430167569 learned_addr learned my addr 172.21.15.161:0/430167569 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f3527fff700 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcccad0 0x55f91bcceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45154/0 (socket says 172.21.15.161:45154) 2023-07-14T11:25:24.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.672+0000 7f35277fe700 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcc16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f35277fe700 1 -- 172.21.15.161:0/430167569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcccad0 msgr2=0x55f91bcceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f35277fe700 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcccad0 0x55f91bcceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f35277fe700 1 -- 172.21.15.161:0/430167569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 msgr2=0x55f91bccc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f35277fe700 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f35277fe700 1 -- 172.21.15.161:0/430167569 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91ba41de0 con 0x55f91bcc12d0 2023-07-14T11:25:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f3526ffd700 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f3527fff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f35100a1340 0x7f35100a37f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3520009fd0 tx=0x7f352000e9a0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f35277fe700 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcc16a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35180cb080 tx=0x7f35180054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb7a05c8a23f5b31 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f350dffb700 1 -- 172.21.15.161:0/430167569 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351809aa90 con 0x55f91bcc12d0 2023-07-14T11:25:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f350dffb700 1 -- 172.21.15.161:0/430167569 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f351809eac0 con 0x55f91bcc12d0 2023-07-14T11:25:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.673+0000 7f350dffb700 1 -- 172.21.15.161:0/430167569 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518099f80 con 0x55f91bcc12d0 2023-07-14T11:25:24.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 -- 172.21.15.161:0/430167569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 msgr2=0x55f91bcc16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcc16a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35180cb080 tx=0x7f35180054e0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 -- 172.21.15.161:0/430167569 shutdown_connections 2023-07-14T11:25:24.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcccad0 0x55f91bcceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcc16a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 --2- 172.21.15.161:0/430167569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 -- 172.21.15.161:0/430167569 >> 172.21.15.161:0/430167569 conn(0x55f91bcb7da0 msgr2=0x55f91bcbd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 -- 172.21.15.161:0/430167569 shutdown_connections 2023-07-14T11:25:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.674+0000 7f3541517b80 1 -- 172.21.15.161:0/430167569 wait complete. 2023-07-14T11:25:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.675+0000 7f3541517b80 1 -- start start 2023-07-14T11:25:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.675+0000 7f3541517b80 1 Processor -- start 2023-07-14T11:25:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3541517b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bcccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3541517b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3541517b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcccad0 0x55f91bccd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3541517b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bbb3ac0 con 0x55f91bcc1be0 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3541517b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bbb37c0 con 0x55f91bcc12d0 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3541517b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ba97920 con 0x55f91bcccad0 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3541517b80 10 client.0 _ll_register_callbacks cb 0x55f91bbb1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bcccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bcccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45162/0 (socket says 172.21.15.161:45162) 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 -- 172.21.15.161:0/2278240804 learned_addr learned my addr 172.21.15.161:0/2278240804 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3526ffd700 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3527fff700 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcccad0 0x55f91bccd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 -- 172.21.15.161:0/2278240804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcccad0 msgr2=0x55f91bccd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcccad0 0x55f91bccd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 -- 172.21.15.161:0/2278240804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 msgr2=0x55f91bccd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 -- 172.21.15.161:0/2278240804 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c009920 con 0x55f91bcc12d0 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f35277fe700 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bcccea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f351809a790 tx=0x7f35180973d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d02d8b1ffb5efd47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.676+0000 7f3506ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:24.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3506ffd700 1 -- 172.21.15.161:0/2278240804 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351809aa90 con 0x55f91bcc12d0 2023-07-14T11:25:24.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3506ffd700 1 -- 172.21.15.161:0/2278240804 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f351809acf0 con 0x55f91bcc12d0 2023-07-14T11:25:24.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f91bb6d4f0 con 0x55f91bcc12d0 2023-07-14T11:25:24.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3506ffd700 1 -- 172.21.15.161:0/2278240804 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f351809aeb0 con 0x55f91bcc12d0 2023-07-14T11:25:24.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 10 client.26282 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:24.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f35281a8e60 con 0x55f91bcc12d0 2023-07-14T11:25:24.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3506ffd700 1 -- 172.21.15.161:0/2278240804 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f91bb6d4f0 con 0x55f91bcc12d0 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3506ffd700 1 -- 172.21.15.161:0/2278240804 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f351809ed20 con 0x55f91bcc12d0 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 10 client.26282 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 20 client.26282 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bcdbc20 0x55f91bcde000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 4 client.26282 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.677+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f91bc0ceb0 con 0x55f91bcdbc20 2023-07-14T11:25:24.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.678+0000 7f3526ffd700 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bcdbc20 0x55f91bcde000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3526ffd700 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bcdbc20 0x55f91bcde000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7d72234dac3c2c server_cookie=1f62f8da37f42082 in_seq=0 out_seq=0 2023-07-14T11:25:24.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3506ffd700 10 client.26282 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:24.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3506ffd700 1 -- 172.21.15.161:0/2278240804 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f91bc0ceb0 con 0x55f91bcdbc20 2023-07-14T11:25:24.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3506ffd700 10 client.26282 handle_command_reply: tid=0 2023-07-14T11:25:24.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3541517b80 1 client.26282 shutdown 2023-07-14T11:25:24.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 msgr2=0x55f91bcccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3541517b80 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bcccea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f351809a790 tx=0x7f35180973d0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 shutdown_connections 2023-07-14T11:25:24.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.679+0000 7f3541517b80 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bcdbc20 0x55f91bcde000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3541517b80 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcccad0 0x55f91bccd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3541517b80 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bcccea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3541517b80 1 --2- 172.21.15.161:0/2278240804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcc1be0 0x55f91bccd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3506ffd700 0 client.26282 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:24.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 >> 172.21.15.161:0/2278240804 conn(0x55f91bcb7da0 msgr2=0x55f91bcbd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 shutdown_connections 2023-07-14T11:25:24.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3541517b80 1 -- 172.21.15.161:0/2278240804 wait complete. 2023-07-14T11:25:24.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.680+0000 7f3541517b80 20 client.26282 trim_cache size 0 max 16384 2023-07-14T11:25:24.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.693+0000 7f3541517b80 1 Processor -- start 2023-07-14T11:25:24.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3541517b80 1 -- start start 2023-07-14T11:25:24.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3541517b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcd1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3541517b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc1be0 0x55f91bcd7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3541517b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcbc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3541517b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bcc1fe0 con 0x55f91bcccad0 2023-07-14T11:25:24.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3541517b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bcc2d60 con 0x55f91bcc1be0 2023-07-14T11:25:24.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3541517b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bcb79a0 con 0x55f91bcc12d0 2023-07-14T11:25:24.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f35277fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcd1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3526ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc1be0 0x55f91bcd7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f35277fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcd1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56598/0 (socket says 172.21.15.161:56598) 2023-07-14T11:25:24.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f35277fe700 1 -- 172.21.15.161:0/977221653 learned_addr learned my addr 172.21.15.161:0/977221653 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3527fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcbc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f3526ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc1be0 0x55f91bcd7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45170/0 (socket says 172.21.15.161:45170) 2023-07-14T11:25:24.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f35277fe700 1 -- 172.21.15.161:0/977221653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc1be0 msgr2=0x55f91bcd7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f35277fe700 1 --2- 172.21.15.161:0/977221653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc1be0 0x55f91bcd7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.694+0000 7f35277fe700 1 -- 172.21.15.161:0/977221653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 msgr2=0x55f91bcbc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.695+0000 7f35277fe700 1 --2- 172.21.15.161:0/977221653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcbc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.695+0000 7f35277fe700 1 -- 172.21.15.161:0/977221653 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91bcb95e0 con 0x55f91bcc12d0 2023-07-14T11:25:24.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.695+0000 7f35277fe700 1 --2- 172.21.15.161:0/977221653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcd1eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3518098de0 tx=0x7f351800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8855346702ae686 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.695+0000 7f350e7fc700 1 -- 172.21.15.161:0/977221653 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518096880 con 0x55f91bcc12d0 2023-07-14T11:25:24.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.695+0000 7f350e7fc700 1 -- 172.21.15.161:0/977221653 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f351809aa90 con 0x55f91bcc12d0 2023-07-14T11:25:24.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.695+0000 7f350e7fc700 1 -- 172.21.15.161:0/977221653 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180a2c20 con 0x55f91bcc12d0 2023-07-14T11:25:24.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 -- 172.21.15.161:0/977221653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 msgr2=0x55f91bcd1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 --2- 172.21.15.161:0/977221653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcd1eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3518098de0 tx=0x7f351800fa70 comp rx=0 tx=0).stop 2023-07-14T11:25:24.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 -- 172.21.15.161:0/977221653 shutdown_connections 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 --2- 172.21.15.161:0/977221653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcbc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 --2- 172.21.15.161:0/977221653 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc12d0 0x55f91bcd1eb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 --2- 172.21.15.161:0/977221653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc1be0 0x55f91bcd7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 -- 172.21.15.161:0/977221653 >> 172.21.15.161:0/977221653 conn(0x55f91bcb7da0 msgr2=0x55f91bcc6460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 -- 172.21.15.161:0/977221653 shutdown_connections 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 -- 172.21.15.161:0/977221653 wait complete. 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 -- start start 2023-07-14T11:25:24.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.696+0000 7f3541517b80 1 Processor -- start 2023-07-14T11:25:24.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3541517b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bccd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3541517b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc1be0 0x55f91bccdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3541517b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3541517b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bd2ebf0 con 0x55f91bcccad0 2023-07-14T11:25:24.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3541517b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bd2ed60 con 0x55f91bcc12d0 2023-07-14T11:25:24.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3541517b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91bd2ef20 con 0x55f91bcc1be0 2023-07-14T11:25:24.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3541517b80 10 client.0 _ll_register_callbacks cb 0x7f351c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bccd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bccd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45186/0 (socket says 172.21.15.161:45186) 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 -- 172.21.15.161:0/366317897 learned_addr learned my addr 172.21.15.161:0/366317897 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3526ffd700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc1be0 0x55f91bccdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3527fff700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 -- 172.21.15.161:0/366317897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc1be0 msgr2=0x55f91bccdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc1be0 0x55f91bccdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 -- 172.21.15.161:0/366317897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 msgr2=0x55f91bcce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:24.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f35277fe700 1 -- 172.21.15.161:0/366317897 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35281a8e60 con 0x55f91bcc12d0 2023-07-14T11:25:24.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.697+0000 7f3526ffd700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc1be0 0x55f91bccdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:24.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f35277fe700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bccd700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3518005e90 tx=0x7f35180054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d0bf60b645e5774 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:24.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3506ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:24.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3506ffd700 1 -- 172.21.15.161:0/366317897 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518096880 con 0x55f91bcc12d0 2023-07-14T11:25:24.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3506ffd700 1 -- 172.21.15.161:0/366317897 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f35180a2bd0 con 0x55f91bcc12d0 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f351809eac0 con 0x55f91bcc12d0 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3506ffd700 1 -- 172.21.15.161:0/366317897 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f351809a3f0 con 0x55f91bcc12d0 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 10 client.26288 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f91ba2ac60 con 0x55f91bcc12d0 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3506ffd700 1 -- 172.21.15.161:0/366317897 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f351809eac0 con 0x55f91bcc12d0 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3506ffd700 1 -- 172.21.15.161:0/366317897 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f35180cb300 con 0x55f91bcc12d0 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 10 client.26288 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:24.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:24.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 20 client.26288 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:24.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bd30150 0x55f91bd32530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:24.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 4 client.26288 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:25:24.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.698+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f91bbcf050 con 0x55f91bd30150 2023-07-14T11:25:24.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.699+0000 7f3526ffd700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bd30150 0x55f91bd32530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:24.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.699+0000 7f3506ffd700 10 client.26288 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:24.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.699+0000 7f3526ffd700 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bd30150 0x55f91bd32530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de45c483f40206 server_cookie=cecab1467f51b67c in_seq=0 out_seq=0 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.699+0000 7f3506ffd700 1 -- 172.21.15.161:0/366317897 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f91bbcf050 con 0x55f91bd30150 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.699+0000 7f3506ffd700 10 client.26288 handle_command_reply: tid=0 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.699+0000 7f3541517b80 1 client.26288 shutdown 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 msgr2=0x55f91bccd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bccd700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3518005e90 tx=0x7f35180054e0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 shutdown_connections 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f91bd30150 0x55f91bd32530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f91bcccad0 0x55f91bcce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3506ffd700 0 client.26288 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:24.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f91bcc12d0 0x55f91bccd700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 --2- 172.21.15.161:0/366317897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f91bcc1be0 0x55f91bccdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 >> 172.21.15.161:0/366317897 conn(0x55f91bcb7da0 msgr2=0x55f91bcca340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 shutdown_connections 2023-07-14T11:25:24.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 1 -- 172.21.15.161:0/366317897 wait complete. 2023-07-14T11:25:24.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f3541517b80 20 client.26288 trim_cache size 0 max 16384 2023-07-14T11:25:24.712 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:25:24.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f35100a1340 msgr2=0x7f35100a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f352dbff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f35100a1340 0x7f35100a37f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3520009fd0 tx=0x7f352000e9a0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280728b0 msgr2=0x7f35280ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:24.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.700+0000 7f352dbff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280728b0 0x7f35280ffa30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3518009810 tx=0x7f351800fa00 comp rx=0 tx=0).stop 2023-07-14T11:25:24.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 shutdown_connections 2023-07-14T11:25:24.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f35100a1340 0x7f35100a37f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f352810c310 0x7f35281004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280728b0 0x7f35280ffa30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 --2- 172.21.15.161:0/348462099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35280731c0 0x7f35280fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:24.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 >> 172.21.15.161:0/348462099 conn(0x7f35280fb5f0 msgr2=0x7f352810e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:24.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 shutdown_connections 2023-07-14T11:25:24.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:24.701+0000 7f352dbff700 1 -- 172.21.15.161:0/348462099 wait complete. 2023-07-14T11:25:24.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:24 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/153536922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:25:25.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 241691 10.20 MB/sec execute 7 sec latency 22.799 ms 2023-07-14T11:25:25.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:24 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/153536922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:25:25.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:24 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/153536922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:25:25.722 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:25:25.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:25:25.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.940+0000 7f98433ff700 1 -- 172.21.15.161:0/3221815177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c0ffad0 msgr2=0x7f983c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.940+0000 7f98433ff700 1 --2- 172.21.15.161:0/3221815177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c0ffad0 0x7f983c0fff40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f982c009990 tx=0x7f982c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.940+0000 7f98433ff700 1 -- 172.21.15.161:0/3221815177 shutdown_connections 2023-07-14T11:25:25.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.940+0000 7f98433ff700 1 --2- 172.21.15.161:0/3221815177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f983c105ae0 0x7f983c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.940+0000 7f98433ff700 1 --2- 172.21.15.161:0/3221815177 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f983c100480 0x7f983c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.940+0000 7f98433ff700 1 --2- 172.21.15.161:0/3221815177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c0ffad0 0x7f983c0fff40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.940+0000 7f98433ff700 1 -- 172.21.15.161:0/3221815177 >> 172.21.15.161:0/3221815177 conn(0x7f983c0fb5f0 msgr2=0x7f983c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:25.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.941+0000 7f98433ff700 1 -- 172.21.15.161:0/3221815177 shutdown_connections 2023-07-14T11:25:25.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.941+0000 7f98433ff700 1 -- 172.21.15.161:0/3221815177 wait complete. 2023-07-14T11:25:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 Processor -- start 2023-07-14T11:25:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 -- start start 2023-07-14T11:25:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f983c0ffad0 0x7f983c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f983c100480 0x7f983c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c105ae0 0x7f983c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c19cdd0 con 0x7f983c0ffad0 2023-07-14T11:25:25.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c19cf40 con 0x7f983c105ae0 2023-07-14T11:25:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.942+0000 7f98433ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c19d100 con 0x7f983c100480 2023-07-14T11:25:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c105ae0 0x7f983c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c105ae0 0x7f983c19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45198/0 (socket says 172.21.15.161:45198) 2023-07-14T11:25:25.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 -- 172.21.15.161:0/1513089688 learned_addr learned my addr 172.21.15.161:0/1513089688 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:25.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984119b700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f983c0ffad0 0x7f983c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984099a700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f983c100480 0x7f983c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 -- 172.21.15.161:0/1513089688 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f983c100480 msgr2=0x7f983c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f983c100480 0x7f983c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 -- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f983c0ffad0 msgr2=0x7f983c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f983c0ffad0 0x7f983c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 -- 172.21.15.161:0/1513089688 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0095f0 con 0x7f983c105ae0 2023-07-14T11:25:25.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984099a700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f983c100480 0x7f983c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:25.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984119b700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f983c0ffad0 0x7f983c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:25.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.943+0000 7f984199c700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c105ae0 0x7f983c19c790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f983400ef30 tx=0x7f983400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:25.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.944+0000 7f98327fc700 1 -- 172.21.15.161:0/1513089688 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834018070 con 0x7f983c105ae0 2023-07-14T11:25:25.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.944+0000 7f98327fc700 1 -- 172.21.15.161:0/1513089688 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983400f040 con 0x7f983c105ae0 2023-07-14T11:25:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.944+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c19d380 con 0x7f983c105ae0 2023-07-14T11:25:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.944+0000 7f98327fc700 1 -- 172.21.15.161:0/1513089688 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400cd00 con 0x7f983c105ae0 2023-07-14T11:25:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.944+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c1a4c20 con 0x7f983c105ae0 2023-07-14T11:25:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.947+0000 7f98327fc700 1 -- 172.21.15.161:0/1513089688 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9834003a00 con 0x7f983c105ae0 2023-07-14T11:25:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 Processor -- start 2023-07-14T11:25:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 -- start start 2023-07-14T11:25:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be853720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be85e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be860f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be759d30 con 0x55c8be85eb50 2023-07-14T11:25:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be7acac0 con 0x55c8be853350 2023-07-14T11:25:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f9856c59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be7a5f50 con 0x55c8be853c60 2023-07-14T11:25:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.949+0000 7f98327fc700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98280a12c0 0x7f98280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be85e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be85e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56642/0 (socket says 172.21.15.161:56642) 2023-07-14T11:25:25.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 -- 172.21.15.161:0/3258898435 learned_addr learned my addr 172.21.15.161:0/3258898435 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:25.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984199c700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98280a12c0 0x7f98280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984099a700 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be853720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f98327fc700 1 -- 172.21.15.161:0/1513089688 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f98340ccbd0 con 0x7f983c105ae0 2023-07-14T11:25:25.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984099a700 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be860f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 -- 172.21.15.161:0/3258898435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 msgr2=0x55c8be853720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be853720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 -- 172.21.15.161:0/3258898435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 msgr2=0x55c8be860f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be860f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 -- 172.21.15.161:0/3258898435 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8be5d3e60 con 0x55c8be853c60 2023-07-14T11:25:25.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984099a700 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be860f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:25:25.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.950+0000 7f984119b700 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be85e610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98380115f0 tx=0x7f9838011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e84e4947b6248597 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:25.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.951+0000 7f984199c700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98280a12c0 0x7f98280a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9834099930 tx=0x7f983400fdc0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:25.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.951+0000 7f98277fe700 1 -- 172.21.15.161:0/3258898435 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838019630 con 0x55c8be853c60 2023-07-14T11:25:25.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.951+0000 7f98277fe700 1 -- 172.21.15.161:0/3258898435 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9838019c50 con 0x55c8be853c60 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.951+0000 7f98277fe700 1 -- 172.21.15.161:0/3258898435 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983800ebc0 con 0x55c8be853c60 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 -- 172.21.15.161:0/3258898435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 msgr2=0x55c8be85e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be85e610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98380115f0 tx=0x7f9838011900 comp rx=0 tx=0).stop 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 -- 172.21.15.161:0/3258898435 shutdown_connections 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be860f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be853720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 --2- 172.21.15.161:0/3258898435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be85e610 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 -- 172.21.15.161:0/3258898435 >> 172.21.15.161:0/3258898435 conn(0x55c8be849e20 msgr2=0x55c8be84f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 -- 172.21.15.161:0/3258898435 shutdown_connections 2023-07-14T11:25:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.952+0000 7f9856c59b80 1 -- 172.21.15.161:0/3258898435 wait complete. 2023-07-14T11:25:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 -- start start 2023-07-14T11:25:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 Processor -- start 2023-07-14T11:25:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be85e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853c60 0x55c8be85b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be85eb50 0x55c8be85bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be745b40 con 0x55c8be853350 2023-07-14T11:25:25.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be745840 con 0x55c8be853c60 2023-07-14T11:25:25.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be6299a0 con 0x55c8be85eb50 2023-07-14T11:25:25.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.953+0000 7f9856c59b80 10 client.0 _ll_register_callbacks cb 0x55c8be7437e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:25.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853c60 0x55c8be85b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853c60 0x55c8be85b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45210/0 (socket says 172.21.15.161:45210) 2023-07-14T11:25:25.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984099a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be85e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 -- 172.21.15.161:0/2923481409 learned_addr learned my addr 172.21.15.161:0/2923481409 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984099a700 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be85eb50 0x55c8be85bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 -- 172.21.15.161:0/2923481409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be85eb50 msgr2=0x55c8be85bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be85eb50 0x55c8be85bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 -- 172.21.15.161:0/2923481409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 msgr2=0x55c8be85e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be85e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 -- 172.21.15.161:0/2923481409 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9838009d50 con 0x55c8be853c60 2023-07-14T11:25:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f984119b700 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853c60 0x55c8be85b810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4066e7734905c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.954+0000 7f9824ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9824ff9700 1 -- 172.21.15.161:0/2923481409 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838004250 con 0x55c8be853c60 2023-07-14T11:25:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9824ff9700 1 -- 172.21.15.161:0/2923481409 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f98380043b0 con 0x55c8be853c60 2023-07-14T11:25:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8be6ff570 con 0x55c8be853c60 2023-07-14T11:25:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9824ff9700 1 -- 172.21.15.161:0/2923481409 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9838004570 con 0x55c8be853c60 2023-07-14T11:25:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9856c59b80 10 client.26300 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c8be5bcce0 con 0x55c8be853c60 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9824ff9700 1 -- 172.21.15.161:0/2923481409 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8be6ff570 con 0x55c8be853c60 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9824ff9700 1 -- 172.21.15.161:0/2923481409 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9838004a40 con 0x55c8be853c60 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9856c59b80 10 client.26300 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9856c59b80 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9856c59b80 20 client.26300 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.955+0000 7f9856c59b80 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be86e330 0x55c8be870740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.956+0000 7f9856c59b80 4 client.26300 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:25.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.956+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8be79ef30 con 0x55c8be86e330 2023-07-14T11:25:25.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.956+0000 7f984099a700 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be86e330 0x55c8be870740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.956+0000 7f984099a700 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be86e330 0x55c8be870740 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c171cf2d27f054c server_cookie=a22844fc7af3da03 in_seq=0 out_seq=0 2023-07-14T11:25:25.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.956+0000 7f9824ff9700 10 client.26300 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:25.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9824ff9700 1 -- 172.21.15.161:0/2923481409 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c8be79ef30 con 0x55c8be86e330 2023-07-14T11:25:25.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9824ff9700 10 client.26300 handle_command_reply: tid=0 2023-07-14T11:25:25.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 client.26300 shutdown 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853c60 msgr2=0x55c8be85b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853c60 0x55c8be85b810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838019660 comp rx=0 tx=0).stop 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 shutdown_connections 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be86e330 0x55c8be870740 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be85e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9824ff9700 0 client.26300 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be85eb50 0x55c8be85bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 --2- 172.21.15.161:0/2923481409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853c60 0x55c8be85b810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 >> 172.21.15.161:0/2923481409 conn(0x55c8be849e20 msgr2=0x55c8be84f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:25.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 shutdown_connections 2023-07-14T11:25:25.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.957+0000 7f9856c59b80 1 -- 172.21.15.161:0/2923481409 wait complete. 2023-07-14T11:25:25.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.958+0000 7f9856c59b80 20 client.26300 trim_cache size 0 max 16384 2023-07-14T11:25:25.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 Processor -- start 2023-07-14T11:25:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 -- start start 2023-07-14T11:25:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be865570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be855f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be854600 con 0x55c8be85eb50 2023-07-14T11:25:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be854030 con 0x55c8be853350 2023-07-14T11:25:25.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f9856c59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be86aef0 con 0x55c8be853c60 2023-07-14T11:25:25.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f984119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f984119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56660/0 (socket says 172.21.15.161:56660) 2023-07-14T11:25:25.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f984119b700 1 -- 172.21.15.161:0/4104637646 learned_addr learned my addr 172.21.15.161:0/4104637646 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:25.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.976+0000 7f984099a700 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be855f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f984099a700 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be865570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f984119b700 1 -- 172.21.15.161:0/4104637646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 msgr2=0x55c8be865570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f984119b700 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be865570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f984119b700 1 -- 172.21.15.161:0/4104637646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 msgr2=0x55c8be855f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f984119b700 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be855f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f984119b700 1 -- 172.21.15.161:0/4104637646 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8be5fa2d0 con 0x55c8be853c60 2023-07-14T11:25:25.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f984119b700 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8559d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f983800fc90 tx=0x7f9838011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2239b2e36b97eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:25.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f9827fff700 1 -- 172.21.15.161:0/4104637646 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983800e830 con 0x55c8be853c60 2023-07-14T11:25:25.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f9827fff700 1 -- 172.21.15.161:0/4104637646 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9838019780 con 0x55c8be853c60 2023-07-14T11:25:25.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.977+0000 7f9827fff700 1 -- 172.21.15.161:0/4104637646 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98380055b0 con 0x55c8be853c60 2023-07-14T11:25:25.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 -- 172.21.15.161:0/4104637646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 msgr2=0x55c8be8559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8559d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f983800fc90 tx=0x7f9838011900 comp rx=0 tx=0).stop 2023-07-14T11:25:25.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 -- 172.21.15.161:0/4104637646 shutdown_connections 2023-07-14T11:25:25.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be85eb50 0x55c8be855f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be853350 0x55c8be865570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4104637646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8559d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 -- 172.21.15.161:0/4104637646 >> 172.21.15.161:0/4104637646 conn(0x55c8be849e20 msgr2=0x55c8be8552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:25.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 -- 172.21.15.161:0/4104637646 shutdown_connections 2023-07-14T11:25:25.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 -- 172.21.15.161:0/4104637646 wait complete. 2023-07-14T11:25:25.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.978+0000 7f9856c59b80 1 -- start start 2023-07-14T11:25:25.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 1 Processor -- start 2023-07-14T11:25:25.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be863fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be85eb50 0x55c8be8c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be864df0 con 0x55c8be853350 2023-07-14T11:25:25.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be8c2740 con 0x55c8be85eb50 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8be8c2880 con 0x55c8be853c60 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f9856c59b80 10 client.0 _ll_register_callbacks cb 0x7f9838011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f984119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f984099a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be863fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f984119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56666/0 (socket says 172.21.15.161:56666) 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.979+0000 7f984119b700 1 -- 172.21.15.161:0/4149859414 learned_addr learned my addr 172.21.15.161:0/4149859414 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984099a700 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be85eb50 0x55c8be8c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984119b700 1 -- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be85eb50 msgr2=0x55c8be8c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984119b700 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be85eb50 0x55c8be8c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984119b700 1 -- 172.21.15.161:0/4149859414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 msgr2=0x55c8be863fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984119b700 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be863fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984119b700 1 -- 172.21.15.161:0/4149859414 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8be5bcce0 con 0x55c8be853c60 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984099a700 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be863fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f984119b700 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8644e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9838005a80 tx=0x7f9838019bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd422c4031e1088 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f9824ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:25.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f9824ff9700 1 -- 172.21.15.161:0/4149859414 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838005560 con 0x55c8be853c60 2023-07-14T11:25:25.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.980+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9838019c50 con 0x55c8be853c60 2023-07-14T11:25:25.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9824ff9700 1 -- 172.21.15.161:0/4149859414 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f983803d030 con 0x55c8be853c60 2023-07-14T11:25:25.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9824ff9700 1 -- 172.21.15.161:0/4149859414 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f983800fed0 con 0x55c8be853c60 2023-07-14T11:25:25.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 10 client.26308 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:25.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c8be5d2330 con 0x55c8be853c60 2023-07-14T11:25:25.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9824ff9700 1 -- 172.21.15.161:0/4149859414 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9838019c50 con 0x55c8be853c60 2023-07-14T11:25:25.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9824ff9700 1 -- 172.21.15.161:0/4149859414 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f983800f720 con 0x55c8be853c60 2023-07-14T11:25:25.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 10 client.26308 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:25.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 10 client.26308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:25.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 20 client.26308 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:25.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be8c3ae0 0x55c8be8c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:25.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 4 client.26308 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:25:25.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.981+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c8be7610d0 con 0x55c8be8c3ae0 2023-07-14T11:25:25.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.982+0000 7f984099a700 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be8c3ae0 0x55c8be8c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:25.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.982+0000 7f984099a700 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be8c3ae0 0x55c8be8c5ef0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b648042b8bdc46 server_cookie=406137b75b9e0888 in_seq=0 out_seq=0 2023-07-14T11:25:25.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.982+0000 7f9824ff9700 10 client.26308 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:25.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9824ff9700 1 -- 172.21.15.161:0/4149859414 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c8be7610d0 con 0x55c8be8c3ae0 2023-07-14T11:25:25.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9824ff9700 10 client.26308 handle_command_reply: tid=0 2023-07-14T11:25:25.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 client.26308 shutdown 2023-07-14T11:25:25.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 msgr2=0x55c8be8644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8644e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9838005a80 tx=0x7f9838019bb0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 shutdown_connections 2023-07-14T11:25:25.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c8be8c3ae0 0x55c8be8c5ef0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8be853350 0x55c8be863fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8be85eb50 0x55c8be8c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9824ff9700 0 client.26308 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:25.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 --2- 172.21.15.161:0/4149859414 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c8be853c60 0x55c8be8644e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 >> 172.21.15.161:0/4149859414 conn(0x55c8be849e20 msgr2=0x55c8be855ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:25.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.983+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 shutdown_connections 2023-07-14T11:25:25.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f9856c59b80 1 -- 172.21.15.161:0/4149859414 wait complete. 2023-07-14T11:25:25.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f9856c59b80 20 client.26308 trim_cache size 0 max 16384 2023-07-14T11:25:25.996 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:25:25.996 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:25:25.996 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "8ded668a-1fcd-4540-9cea-dcbfc3e4198c", 2023-07-14T11:25:25.997 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:25:25.997 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:25:25.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98280a12c0 msgr2=0x7f98280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98280a12c0 0x7f98280a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9834099930 tx=0x7f983400fdc0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c105ae0 msgr2=0x7f983c19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:25.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c105ae0 0x7f983c19c790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f983400ef30 tx=0x7f983400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 shutdown_connections 2023-07-14T11:25:25.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98280a12c0 0x7f98280a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f983c0ffad0 0x7f983c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983c105ae0 0x7f983c19c790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 --2- 172.21.15.161:0/1513089688 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f983c100480 0x7f983c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:25.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.984+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 >> 172.21.15.161:0/1513089688 conn(0x7f983c0fb5f0 msgr2=0x7f983c10ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:25.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.985+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 shutdown_connections 2023-07-14T11:25:25.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:25.985+0000 7f98433ff700 1 -- 172.21.15.161:0/1513089688 wait complete. 2023-07-14T11:25:26.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:25:26.081 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:25 smithi161 ceph-mon[95526]: pgmap v98: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-14T11:25:26.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:25 smithi175 ceph-mon[129081]: pgmap v98: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-14T11:25:26.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 243617 9.80 MB/sec execute 8 sec latency 26.415 ms 2023-07-14T11:25:26.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:25 smithi182 ceph-mon[129366]: pgmap v98: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-14T11:25:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.285+0000 7fa8015ff700 1 -- 172.21.15.161:0/3769203421 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc073160 msgr2=0x7fa7fc10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.285+0000 7fa8015ff700 1 --2- 172.21.15.161:0/3769203421 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc073160 0x7fa7fc10be40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009990 tx=0x7fa7e4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.286+0000 7fa8015ff700 1 -- 172.21.15.161:0/3769203421 shutdown_connections 2023-07-14T11:25:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.286+0000 7fa8015ff700 1 --2- 172.21.15.161:0/3769203421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc10c380 0x7fa7fc10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.286+0000 7fa8015ff700 1 --2- 172.21.15.161:0/3769203421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7fc072850 0x7fa7fc072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.286+0000 7fa8015ff700 1 --2- 172.21.15.161:0/3769203421 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc073160 0x7fa7fc10be40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.286+0000 7fa8015ff700 1 -- 172.21.15.161:0/3769203421 >> 172.21.15.161:0/3769203421 conn(0x7fa7fc0fb5e0 msgr2=0x7fa7fc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:26.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.286+0000 7fa8015ff700 1 -- 172.21.15.161:0/3769203421 shutdown_connections 2023-07-14T11:25:26.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.286+0000 7fa8015ff700 1 -- 172.21.15.161:0/3769203421 wait complete. 2023-07-14T11:25:26.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 Processor -- start 2023-07-14T11:25:26.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 -- start start 2023-07-14T11:25:26.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc072850 0x7fa7fc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc073160 0x7fa7fc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7fc10c380 0x7fa7fc19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc19cf10 con 0x7fa7fc073160 2023-07-14T11:25:26.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc19d080 con 0x7fa7fc10c380 2023-07-14T11:25:26.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.287+0000 7fa8015ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc19d240 con 0x7fa7fc072850 2023-07-14T11:25:26.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc072850 0x7fa7fc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc072850 0x7fa7fc197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56688/0 (socket says 172.21.15.161:56688) 2023-07-14T11:25:26.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 -- 172.21.15.161:0/68006361 learned_addr learned my addr 172.21.15.161:0/68006361 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:26.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc073160 0x7fa7fc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7fc10c380 0x7fa7fc19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 -- 172.21.15.161:0/68006361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7fc10c380 msgr2=0x7fa7fc19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7fc10c380 0x7fa7fc19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 -- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc073160 msgr2=0x7fa7fc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc073160 0x7fa7fc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.288+0000 7fa7faffd700 1 -- 172.21.15.161:0/68006361 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e40095f0 con 0x7fa7fc072850 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7fc10c380 0x7fa7fc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc073160 0x7fa7fc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa7faffd700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc072850 0x7fa7fc197f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00ef10 tx=0x7fa7ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa7f8ff9700 1 -- 172.21.15.161:0/68006361 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00c9b0 con 0x7fa7fc072850 2023-07-14T11:25:26.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc19d4c0 con 0x7fa7fc072850 2023-07-14T11:25:26.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa7f8ff9700 1 -- 172.21.15.161:0/68006361 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7ec00f040 con 0x7fa7fc072850 2023-07-14T11:25:26.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa7f8ff9700 1 -- 172.21.15.161:0/68006361 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec014480 con 0x7fa7fc072850 2023-07-14T11:25:26.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.289+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc06a110 con 0x7fa7fc072850 2023-07-14T11:25:26.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.293+0000 7fa7f8ff9700 1 -- 172.21.15.161:0/68006361 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa7ec007690 con 0x7fa7fc072850 2023-07-14T11:25:26.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.294+0000 7fa814e2bb80 1 Processor -- start 2023-07-14T11:25:26.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa814e2bb80 1 -- start start 2023-07-14T11:25:26.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa7f8ff9700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa7e80a1340 0x7fa7e80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb72d0 0x556393cb76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393ccada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393ccb2e0 0x556393ccd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393bbdcb0 con 0x556393cb7be0 2023-07-14T11:25:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393c10a40 con 0x556393ccb2e0 2023-07-14T11:25:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393c09ed0 con 0x556393cb72d0 2023-07-14T11:25:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.295+0000 7fa7f8ff9700 1 -- 172.21.15.161:0/68006361 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa7ec0cd2d0 con 0x7fa7fc072850 2023-07-14T11:25:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393ccada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393ccada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44390/0 (socket says 172.21.15.161:44390) 2023-07-14T11:25:26.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 -- 172.21.15.161:0/2998490383 learned_addr learned my addr 172.21.15.161:0/2998490383 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:26.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7faffd700 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb72d0 0x556393cb76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa7e80a1340 0x7fa7e80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393ccb2e0 0x556393ccd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 -- 172.21.15.161:0/2998490383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb72d0 msgr2=0x556393cb76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb72d0 0x556393cb76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 -- 172.21.15.161:0/2998490383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393ccb2e0 msgr2=0x556393ccd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393ccb2e0 0x556393ccd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7f27fc700 1 -- 172.21.15.161:0/2998490383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556393a37de0 con 0x556393cb7be0 2023-07-14T11:25:26.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.296+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393ccb2e0 0x556393ccd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:25:26.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.297+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa7e80a1340 0x7fa7e80a37f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4007610 tx=0x7fa7f40075a0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:26.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.297+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393ccada0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4015000 tx=0x7fa7e400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65971493ad351d96 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:26.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.297+0000 7fa7f17fa700 1 -- 172.21.15.161:0/2998490383 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e401d050 con 0x556393cb7be0 2023-07-14T11:25:26.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.297+0000 7fa7f17fa700 1 -- 172.21.15.161:0/2998490383 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7e401e040 con 0x556393cb7be0 2023-07-14T11:25:26.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.297+0000 7fa7f17fa700 1 -- 172.21.15.161:0/2998490383 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40055c0 con 0x556393cb7be0 2023-07-14T11:25:26.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.298+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2998490383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 msgr2=0x556393ccada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.298+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393ccada0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4015000 tx=0x7fa7e400b610 comp rx=0 tx=0).stop 2023-07-14T11:25:26.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.298+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2998490383 shutdown_connections 2023-07-14T11:25:26.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.298+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393ccb2e0 0x556393ccd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.298+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb72d0 0x556393cb76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.298+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2998490383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393ccada0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.298+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2998490383 >> 172.21.15.161:0/2998490383 conn(0x556393cadda0 msgr2=0x556393cb36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.299+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2998490383 shutdown_connections 2023-07-14T11:25:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.299+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2998490383 wait complete. 2023-07-14T11:25:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.299+0000 7fa814e2bb80 1 -- start start 2023-07-14T11:25:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.299+0000 7fa814e2bb80 1 Processor -- start 2023-07-14T11:25:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393ccacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb7be0 0x556393cc7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393ccb2e0 0x556393cc8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393ba9ac0 con 0x556393ccb2e0 2023-07-14T11:25:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393ba97c0 con 0x556393cb72d0 2023-07-14T11:25:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393a8d920 con 0x556393cb7be0 2023-07-14T11:25:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa814e2bb80 10 client.0 _ll_register_callbacks cb 0x556393ba7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa7faffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393ccacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa7faffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393ccacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45272/0 (socket says 172.21.15.161:45272) 2023-07-14T11:25:26.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa7faffd700 1 -- 172.21.15.161:0/1885833644 learned_addr learned my addr 172.21.15.161:0/1885833644 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:26.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb7be0 0x556393cc7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.300+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393ccb2e0 0x556393cc8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7faffd700 1 -- 172.21.15.161:0/1885833644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb7be0 msgr2=0x556393cc7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7faffd700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb7be0 0x556393cc7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7faffd700 1 -- 172.21.15.161:0/1885833644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393ccb2e0 msgr2=0x556393cc8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7faffd700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393ccb2e0 0x556393cc8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7faffd700 1 -- 172.21.15.161:0/1885833644 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc10f3a0 con 0x556393cb72d0 2023-07-14T11:25:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7faffd700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393ccacf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec007fd0 tx=0x7fa7ec099bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee033f8276aa470 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7e27fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393ccb2e0 0x556393cc8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:25:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7e27fc700 1 -- 172.21.15.161:0/1885833644 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec099700 con 0x556393cb72d0 2023-07-14T11:25:26.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa7e27fc700 1 -- 172.21.15.161:0/1885833644 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa7ec0112c0 con 0x556393cb72d0 2023-07-14T11:25:26.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.301+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556393b634f0 con 0x556393cb72d0 2023-07-14T11:25:26.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa7e27fc700 1 -- 172.21.15.161:0/1885833644 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7ec09a010 con 0x556393cb72d0 2023-07-14T11:25:26.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 10 client.26315 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:26.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556393a20c60 con 0x556393cb72d0 2023-07-14T11:25:26.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa7e27fc700 1 -- 172.21.15.161:0/1885833644 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556393b634f0 con 0x556393cb72d0 2023-07-14T11:25:26.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa7e27fc700 1 -- 172.21.15.161:0/1885833644 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa7ec00cb10 con 0x556393cb72d0 2023-07-14T11:25:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 10 client.26315 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 20 client.26315 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393cdab00 0x556393cdcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 4 client.26315 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556393c02eb0 con 0x556393cdab00 2023-07-14T11:25:26.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.302+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393cdab00 0x556393cdcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.303+0000 7fa7e27fc700 10 client.26315 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:26.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.303+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393cdab00 0x556393cdcf10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f524e666beae7f9 server_cookie=49bcd386f0d48c42 in_seq=0 out_seq=0 2023-07-14T11:25:26.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa7e27fc700 1 -- 172.21.15.161:0/1885833644 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556393c02eb0 con 0x556393cdab00 2023-07-14T11:25:26.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa7e27fc700 10 client.26315 handle_command_reply: tid=0 2023-07-14T11:25:26.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 client.26315 shutdown 2023-07-14T11:25:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 msgr2=0x556393ccacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393ccacf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec007fd0 tx=0x7fa7ec099bf0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 shutdown_connections 2023-07-14T11:25:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393cdab00 0x556393cdcf10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393ccb2e0 0x556393cc8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393ccacf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1885833644 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393cb7be0 0x556393cc7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa7e27fc700 0 client.26315 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 >> 172.21.15.161:0/1885833644 conn(0x556393cadda0 msgr2=0x556393cb3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 shutdown_connections 2023-07-14T11:25:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.304+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1885833644 wait complete. 2023-07-14T11:25:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.305+0000 7fa814e2bb80 20 client.26315 trim_cache size 0 max 16384 2023-07-14T11:25:26.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.323+0000 7fa814e2bb80 1 Processor -- start 2023-07-14T11:25:26.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.323+0000 7fa814e2bb80 1 -- start start 2023-07-14T11:25:26.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.323+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb72d0 0x556393cd1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.323+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb7be0 0x556393cb9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.323+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393cc5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.323+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393cb7a20 con 0x556393cb72d0 2023-07-14T11:25:26.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.323+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393cad970 con 0x556393cb7be0 2023-07-14T11:25:26.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393cc89f0 con 0x556393ccb2e0 2023-07-14T11:25:26.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7f27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb7be0 0x556393cb9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7fb7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393cc5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7f27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb7be0 0x556393cb9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45280/0 (socket says 172.21.15.161:45280) 2023-07-14T11:25:26.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7f27fc700 1 -- 172.21.15.161:0/1818432805 learned_addr learned my addr 172.21.15.161:0/1818432805 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:26.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7faffd700 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb72d0 0x556393cd1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7fb7fe700 1 -- 172.21.15.161:0/1818432805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb7be0 msgr2=0x556393cb9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb7be0 0x556393cb9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7fb7fe700 1 -- 172.21.15.161:0/1818432805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb72d0 msgr2=0x556393cd1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb72d0 0x556393cd1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7fb7fe700 1 -- 172.21.15.161:0/1818432805 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556393a5e250 con 0x556393ccb2e0 2023-07-14T11:25:26.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.324+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393cc5250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc069a80 tx=0x7fa7f4014070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f39b3bc4da82e24 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:26.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.325+0000 7fa7f1ffb700 1 -- 172.21.15.161:0/1818432805 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4028640 con 0x556393ccb2e0 2023-07-14T11:25:26.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.325+0000 7fa7f1ffb700 1 -- 172.21.15.161:0/1818432805 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7f4016070 con 0x556393ccb2e0 2023-07-14T11:25:26.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.325+0000 7fa7f1ffb700 1 -- 172.21.15.161:0/1818432805 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f401ba20 con 0x556393ccb2e0 2023-07-14T11:25:26.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.325+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1818432805 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 msgr2=0x556393cc5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.325+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393cc5250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc069a80 tx=0x7fa7f4014070 comp rx=0 tx=0).stop 2023-07-14T11:25:26.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1818432805 shutdown_connections 2023-07-14T11:25:26.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb72d0 0x556393cd1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393cc5250 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/1818432805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb7be0 0x556393cb9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1818432805 >> 172.21.15.161:0/1818432805 conn(0x556393cadda0 msgr2=0x556393cb93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1818432805 shutdown_connections 2023-07-14T11:25:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 -- 172.21.15.161:0/1818432805 wait complete. 2023-07-14T11:25:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.326+0000 7fa814e2bb80 1 -- start start 2023-07-14T11:25:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 1 Processor -- start 2023-07-14T11:25:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393d282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393d28830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393d22950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393d23030 con 0x556393cb7be0 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393d231a0 con 0x556393cb72d0 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556393d23360 con 0x556393ccb2e0 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa814e2bb80 10 client.0 _ll_register_callbacks cb 0x556393ba7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.327+0000 7fa7fb7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393d22950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393d22950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56712/0 (socket says 172.21.15.161:56712) 2023-07-14T11:25:26.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 -- 172.21.15.161:0/2052972487 learned_addr learned my addr 172.21.15.161:0/2052972487 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:26.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393d28830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7faffd700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393d282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 -- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 msgr2=0x556393d282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393d282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 -- 172.21.15.161:0/2052972487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 msgr2=0x556393d28830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:26.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393d28830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 -- 172.21.15.161:0/2052972487 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f401f420 con 0x556393ccb2e0 2023-07-14T11:25:26.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7faffd700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393d282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:26.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7f27fc700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393d28830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:26.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7fb7fe700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393d22950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa7f401b520 tx=0x7fa7f401b550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6228af2531643ca0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:26.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.328+0000 7fa7e27fc700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:26.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa7e27fc700 1 -- 172.21.15.161:0/2052972487 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4004220 con 0x556393ccb2e0 2023-07-14T11:25:26.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa7e27fc700 1 -- 172.21.15.161:0/2052972487 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa7f4004380 con 0x556393ccb2e0 2023-07-14T11:25:26.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7e401e040 con 0x556393ccb2e0 2023-07-14T11:25:26.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa7e27fc700 1 -- 172.21.15.161:0/2052972487 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7f4004540 con 0x556393ccb2e0 2023-07-14T11:25:26.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa814e2bb80 10 client.26326 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:26.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556393a362b0 con 0x556393ccb2e0 2023-07-14T11:25:26.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa7e27fc700 1 -- 172.21.15.161:0/2052972487 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7e401e040 con 0x556393ccb2e0 2023-07-14T11:25:26.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.329+0000 7fa7e27fc700 1 -- 172.21.15.161:0/2052972487 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa7f4003ac0 con 0x556393ccb2e0 2023-07-14T11:25:26.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa814e2bb80 10 client.26326 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:26.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa814e2bb80 10 client.26326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa814e2bb80 20 client.26326 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393d30080 0x556393d32490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa814e2bb80 4 client.26326 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556393bc5050 con 0x556393d30080 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa7faffd700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393d30080 0x556393d32490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa7faffd700 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393d30080 0x556393d32490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=398f7be10fdeb061 server_cookie=add5c94c25d0d378 in_seq=0 out_seq=0 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.330+0000 7fa7e27fc700 10 client.26326 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:26.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa7e27fc700 1 -- 172.21.15.161:0/2052972487 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556393bc5050 con 0x556393d30080 2023-07-14T11:25:26.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa7e27fc700 10 client.26326 handle_command_reply: tid=0 2023-07-14T11:25:26.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 client.26326 shutdown 2023-07-14T11:25:26.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 msgr2=0x556393d22950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393d22950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa7f401b520 tx=0x7fa7f401b550 comp rx=0 tx=0).stop 2023-07-14T11:25:26.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 shutdown_connections 2023-07-14T11:25:26.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556393d30080 0x556393d32490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556393ccb2e0 0x556393d22950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556393cb72d0 0x556393d282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 --2- 172.21.15.161:0/2052972487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556393cb7be0 0x556393d28830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 >> 172.21.15.161:0/2052972487 conn(0x556393cadda0 msgr2=0x556393cb9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:26.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.331+0000 7fa7e27fc700 0 client.26326 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:26.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.332+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 shutdown_connections 2023-07-14T11:25:26.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.332+0000 7fa814e2bb80 1 -- 172.21.15.161:0/2052972487 wait complete. 2023-07-14T11:25:26.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.332+0000 7fa814e2bb80 20 client.26326 trim_cache size 0 max 16384 2023-07-14T11:25:26.342 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:25:26.342 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:25:26.342 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:25:26.342 INFO:teuthology.orchestra.run.smithi161.stdout: "8ded668a-1fcd-4540-9cea-dcbfc3e4198c": { 2023-07-14T11:25:26.342 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:25:26.342 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "8ded668a-1fcd-4540-9cea-dcbfc3e4198c", 2023-07-14T11:25:26.343 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:25:26.343 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:25:26.343 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:25:26.343 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:25:26.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.332+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa7e80a1340 msgr2=0x7fa7e80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.332+0000 7fa8015ff700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa7e80a1340 0x7fa7e80a37f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4007610 tx=0x7fa7f40075a0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.332+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc072850 msgr2=0x7fa7fc197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:26.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.332+0000 7fa8015ff700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc072850 0x7fa7fc197f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00ef10 tx=0x7fa7ec00c3a0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 shutdown_connections 2023-07-14T11:25:26.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa7e80a1340 0x7fa7e80a37f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7fc10c380 0x7fa7fc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa7fc072850 0x7fa7fc197f30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 --2- 172.21.15.161:0/68006361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc073160 0x7fa7fc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:26.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 >> 172.21.15.161:0/68006361 conn(0x7fa7fc0fb5e0 msgr2=0x7fa7fc10e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:26.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 shutdown_connections 2023-07-14T11:25:26.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:26.333+0000 7fa8015ff700 1 -- 172.21.15.161:0/68006361 wait complete. 2023-07-14T11:25:26.357 INFO:tasks.cephfs.filesystem:scrub status for tag:8ded668a-1fcd-4540-9cea-dcbfc3e4198c - {'path': '/', 'tag': '8ded668a-1fcd-4540-9cea-dcbfc3e4198c', 'options': 'recursive,force'} 2023-07-14T11:25:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:25:27.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:25:27.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:25:27.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:25:27.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:25:27.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 245150 10.06 MB/sec execute 9 sec latency 93.958 ms 2023-07-14T11:25:27.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:25:27.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:25:27.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:25:27.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:25:27.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:25:27.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:25:27.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:25:27.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:25:27.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:25:27.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:25:27.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:25:28.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:27 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:25:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:27 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:25:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:27 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:25:28.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:27 smithi175 ceph-mon[129081]: pgmap v99: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2023-07-14T11:25:28.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 246990 9.63 MB/sec execute 10 sec latency 21.399 ms 2023-07-14T11:25:28.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:27 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:25:28.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:27 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:25:28.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:27 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:25:28.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:27 smithi182 ceph-mon[129366]: pgmap v99: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2023-07-14T11:25:28.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:27 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:25:28.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:27 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:25:28.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:27 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:25:28.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:27 smithi161 ceph-mon[95526]: pgmap v99: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2023-07-14T11:25:29.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 248787 9.97 MB/sec execute 11 sec latency 28.375 ms 2023-07-14T11:25:30.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 250830 9.79 MB/sec execute 12 sec latency 10.922 ms 2023-07-14T11:25:30.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:30 smithi175 ceph-mon[129081]: pgmap v100: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2023-07-14T11:25:30.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:30 smithi182 ceph-mon[129366]: pgmap v100: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2023-07-14T11:25:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:30 smithi161 ceph-mon[95526]: pgmap v100: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2023-07-14T11:25:31.171 INFO:tasks.workunit.client.0.smithi161.stdout: 1 252437 9.91 MB/sec execute 13 sec latency 22.108 ms 2023-07-14T11:25:32.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 254638 9.98 MB/sec execute 14 sec latency 59.905 ms 2023-07-14T11:25:32.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:32 smithi175 ceph-mon[129081]: pgmap v101: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2023-07-14T11:25:32.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:32 smithi182 ceph-mon[129366]: pgmap v101: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2023-07-14T11:25:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:32 smithi161 ceph-mon[95526]: pgmap v101: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2023-07-14T11:25:33.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 256294 9.90 MB/sec execute 15 sec latency 11.373 ms 2023-07-14T11:25:34.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:33 smithi175 ceph-mon[129081]: pgmap v102: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2023-07-14T11:25:34.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 258967 10.34 MB/sec execute 16 sec latency 8.846 ms 2023-07-14T11:25:34.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:33 smithi182 ceph-mon[129366]: pgmap v102: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2023-07-14T11:25:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:33 smithi161 ceph-mon[95526]: pgmap v102: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2023-07-14T11:25:35.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 260176 9.95 MB/sec execute 17 sec latency 12.970 ms 2023-07-14T11:25:36.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 262515 10.25 MB/sec execute 18 sec latency 129.576 ms 2023-07-14T11:25:36.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:36 smithi175 ceph-mon[129081]: pgmap v103: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 204 op/s 2023-07-14T11:25:36.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:36 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:25:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:36 smithi182 ceph-mon[129366]: pgmap v103: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 204 op/s 2023-07-14T11:25:36.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:36 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:25:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:36 smithi161 ceph-mon[95526]: pgmap v103: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 204 op/s 2023-07-14T11:25:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:36 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:25:37.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 263502 9.85 MB/sec execute 19 sec latency 50.910 ms 2023-07-14T11:25:37.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:37 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:25:37.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:37 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:25:37.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:37 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:25:37.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:37 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:25:37.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:37 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:25:37.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:37 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:25:38.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 265868 10.03 MB/sec execute 20 sec latency 144.511 ms 2023-07-14T11:25:38.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:38 smithi175 ceph-mon[129081]: pgmap v104: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 311 op/s 2023-07-14T11:25:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:38 smithi182 ceph-mon[129366]: pgmap v104: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 311 op/s 2023-07-14T11:25:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:38 smithi161 ceph-mon[95526]: pgmap v104: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 311 op/s 2023-07-14T11:25:39.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 267119 9.87 MB/sec execute 21 sec latency 13.402 ms 2023-07-14T11:25:40.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 269675 10.13 MB/sec execute 22 sec latency 48.923 ms 2023-07-14T11:25:40.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:40 smithi175 ceph-mon[129081]: pgmap v105: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-07-14T11:25:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:40 smithi182 ceph-mon[129366]: pgmap v105: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-07-14T11:25:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:40 smithi161 ceph-mon[95526]: pgmap v105: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-07-14T11:25:41.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 270671 9.84 MB/sec execute 23 sec latency 35.705 ms 2023-07-14T11:25:42.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 273145 10.08 MB/sec execute 24 sec latency 46.028 ms 2023-07-14T11:25:42.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:42 smithi175 ceph-mon[129081]: pgmap v106: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2023-07-14T11:25:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:42 smithi182 ceph-mon[129366]: pgmap v106: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2023-07-14T11:25:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:42 smithi161 ceph-mon[95526]: pgmap v106: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2023-07-14T11:25:43.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 274307 9.82 MB/sec execute 25 sec latency 15.904 ms 2023-07-14T11:25:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:43 smithi175 ceph-mon[129081]: pgmap v107: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-14T11:25:44.172 INFO:tasks.workunit.client.0.smithi161.stdout: 1 276905 10.09 MB/sec execute 26 sec latency 12.050 ms 2023-07-14T11:25:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:43 smithi182 ceph-mon[129366]: pgmap v107: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-14T11:25:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:43 smithi161 ceph-mon[95526]: pgmap v107: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-14T11:25:45.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 278119 9.91 MB/sec execute 27 sec latency 21.605 ms 2023-07-14T11:25:46.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 280525 10.05 MB/sec execute 28 sec latency 11.074 ms 2023-07-14T11:25:46.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:46 smithi175 ceph-mon[129081]: pgmap v108: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-14T11:25:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:46 smithi182 ceph-mon[129366]: pgmap v108: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-14T11:25:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:46 smithi161 ceph-mon[95526]: pgmap v108: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-14T11:25:47.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 281735 9.89 MB/sec execute 29 sec latency 39.719 ms 2023-07-14T11:25:48.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 284107 10.02 MB/sec execute 30 sec latency 19.220 ms 2023-07-14T11:25:48.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:48 smithi175 ceph-mon[129081]: pgmap v109: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 317 op/s 2023-07-14T11:25:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:48 smithi182 ceph-mon[129366]: pgmap v109: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 317 op/s 2023-07-14T11:25:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:48 smithi161 ceph-mon[95526]: pgmap v109: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 317 op/s 2023-07-14T11:25:49.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 285232 9.87 MB/sec execute 31 sec latency 150.982 ms 2023-07-14T11:25:50.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 287621 9.99 MB/sec execute 32 sec latency 10.884 ms 2023-07-14T11:25:50.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:50 smithi175 ceph-mon[129081]: pgmap v110: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-14T11:25:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:50 smithi182 ceph-mon[129366]: pgmap v110: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-14T11:25:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:50 smithi161 ceph-mon[95526]: pgmap v110: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-14T11:25:51.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 289036 9.86 MB/sec execute 33 sec latency 14.742 ms 2023-07-14T11:25:52.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 291250 9.97 MB/sec execute 34 sec latency 35.489 ms 2023-07-14T11:25:52.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:52 smithi175 ceph-mon[129081]: pgmap v111: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-14T11:25:52.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:52 smithi182 ceph-mon[129366]: pgmap v111: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-14T11:25:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:52 smithi161 ceph-mon[95526]: pgmap v111: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-14T11:25:53.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 292567 9.84 MB/sec execute 35 sec latency 54.947 ms 2023-07-14T11:25:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:53 smithi175 ceph-mon[129081]: pgmap v112: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:54.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 294842 9.95 MB/sec execute 36 sec latency 11.629 ms 2023-07-14T11:25:54.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:53 smithi182 ceph-mon[129366]: pgmap v112: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:53 smithi161 ceph-mon[95526]: pgmap v112: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:55.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 296603 9.84 MB/sec execute 37 sec latency 14.817 ms 2023-07-14T11:25:56.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 298419 9.93 MB/sec execute 38 sec latency 25.809 ms 2023-07-14T11:25:56.357 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:25:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:56 smithi175 ceph-mon[129081]: pgmap v113: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:56.424 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:56 smithi161 ceph-mon[95526]: pgmap v113: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:56 smithi182 ceph-mon[129366]: pgmap v113: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-07-14T11:25:56.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 -- 172.21.15.161:0/4242775198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffae0 msgr2=0x7f40280fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 --2- 172.21.15.161:0/4242775198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffae0 0x7f40280fff50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f401c0098f0 tx=0x7f401c009c00 comp rx=0 tx=0).stop 2023-07-14T11:25:56.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 -- 172.21.15.161:0/4242775198 shutdown_connections 2023-07-14T11:25:56.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 --2- 172.21.15.161:0/4242775198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100490 0x7f402810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 --2- 172.21.15.161:0/4242775198 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028105af0 0x7f4028105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 --2- 172.21.15.161:0/4242775198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffae0 0x7f40280fff50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 -- 172.21.15.161:0/4242775198 >> 172.21.15.161:0/4242775198 conn(0x7f40280fb5e0 msgr2=0x7f40280fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:56.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 -- 172.21.15.161:0/4242775198 shutdown_connections 2023-07-14T11:25:56.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.579+0000 7f402e7ff700 1 -- 172.21.15.161:0/4242775198 wait complete. 2023-07-14T11:25:56.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.580+0000 7f402e7ff700 1 Processor -- start 2023-07-14T11:25:56.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.580+0000 7f402e7ff700 1 -- start start 2023-07-14T11:25:56.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402e7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40280ffae0 0x7f4028197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402e7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028100490 0x7f4028198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402e7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105af0 0x7f402819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402e7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402819cea0 con 0x7f40280ffae0 2023-07-14T11:25:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402e7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402819d010 con 0x7f4028105af0 2023-07-14T11:25:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402e7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402819d1d0 con 0x7f4028100490 2023-07-14T11:25:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40280ffae0 0x7f4028197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105af0 0x7f402819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40280ffae0 0x7f4028197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60400/0 (socket says 172.21.15.161:60400) 2023-07-14T11:25:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105af0 0x7f402819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44802/0 (socket says 172.21.15.161:44802) 2023-07-14T11:25:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 -- 172.21.15.161:0/2617932742 learned_addr learned my addr 172.21.15.161:0/2617932742 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f40277fe700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028100490 0x7f4028198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 -- 172.21.15.161:0/2617932742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028100490 msgr2=0x7f4028198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028100490 0x7f4028198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 -- 172.21.15.161:0/2617932742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105af0 msgr2=0x7f402819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105af0 0x7f402819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f4027fff700 1 -- 172.21.15.161:0/2617932742 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020009d00 con 0x7f40280ffae0 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f40277fe700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028100490 0x7f4028198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.581+0000 7f402cd9c700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105af0 0x7f402819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:56.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.582+0000 7f4027fff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40280ffae0 0x7f4028197ec0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f401800e9a0 tx=0x7f401800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.582+0000 7f40257fa700 1 -- 172.21.15.161:0/2617932742 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018009790 con 0x7f40280ffae0 2023-07-14T11:25:56.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.582+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c009520 con 0x7f40280ffae0 2023-07-14T11:25:56.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.582+0000 7f40257fa700 1 -- 172.21.15.161:0/2617932742 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4018010430 con 0x7f40280ffae0 2023-07-14T11:25:56.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.582+0000 7f40257fa700 1 -- 172.21.15.161:0/2617932742 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018018680 con 0x7f40280ffae0 2023-07-14T11:25:56.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.582+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40281a4b10 con 0x7f40280ffae0 2023-07-14T11:25:56.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.585+0000 7f40257fa700 1 -- 172.21.15.161:0/2617932742 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f40180105a0 con 0x7f40280ffae0 2023-07-14T11:25:56.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 Processor -- start 2023-07-14T11:25:56.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f40257fa700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40100a12c0 0x7f40100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 -- start start 2023-07-14T11:25:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f40257fa700 1 -- 172.21.15.161:0/2617932742 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f4018014070 con 0x7f40280ffae0 2023-07-14T11:25:56.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f402cd9c700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40100a12c0 0x7f40100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0710982d0 0x55a0710986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710a3ad0 0x55a0710a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070f9ecb0 con 0x55a0710982d0 2023-07-14T11:25:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070ff1a40 con 0x55a071098be0 2023-07-14T11:25:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f404203eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070feaed0 con 0x55a0710a3ad0 2023-07-14T11:25:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f40277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44812/0 (socket says 172.21.15.161:44812) 2023-07-14T11:25:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 -- 172.21.15.161:0/3186867602 learned_addr learned my addr 172.21.15.161:0/3186867602 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.588+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710a3ad0 0x55a0710a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f4027fff700 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0710982d0 0x55a0710986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 -- 172.21.15.161:0/3186867602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710a3ad0 msgr2=0x55a0710a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710a3ad0 0x55a0710a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 -- 172.21.15.161:0/3186867602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0710982d0 msgr2=0x55a0710986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0710982d0 0x55a0710986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 -- 172.21.15.161:0/3186867602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a070e18de0 con 0x55a071098be0 2023-07-14T11:25:56.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f402cd9c700 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710a3ad0 0x55a0710a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:25:56.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f402cd9c700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40100a12c0 0x7f40100a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f4020018640 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.589+0000 7f40277fe700 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a3590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f401c000c00 tx=0x7f401c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbf4ff44b3a5a03 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.590+0000 7f400e7fc700 1 -- 172.21.15.161:0/3186867602 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c01c050 con 0x55a071098be0 2023-07-14T11:25:56.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.590+0000 7f400e7fc700 1 -- 172.21.15.161:0/3186867602 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c01d040 con 0x55a071098be0 2023-07-14T11:25:56.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.590+0000 7f400e7fc700 1 -- 172.21.15.161:0/3186867602 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0052a0 con 0x55a071098be0 2023-07-14T11:25:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.590+0000 7f404203eb80 1 -- 172.21.15.161:0/3186867602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 msgr2=0x55a0710a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.590+0000 7f404203eb80 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a3590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f401c000c00 tx=0x7f401c005dc0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 -- 172.21.15.161:0/3186867602 shutdown_connections 2023-07-14T11:25:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0710982d0 0x55a0710986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710a3ad0 0x55a0710a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 --2- 172.21.15.161:0/3186867602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a3590 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 -- 172.21.15.161:0/3186867602 >> 172.21.15.161:0/3186867602 conn(0x55a07108eda0 msgr2=0x55a0710946a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:56.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 -- 172.21.15.161:0/3186867602 shutdown_connections 2023-07-14T11:25:56.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 -- 172.21.15.161:0/3186867602 wait complete. 2023-07-14T11:25:56.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 -- start start 2023-07-14T11:25:56.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.591+0000 7f404203eb80 1 Processor -- start 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f404203eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f404203eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a0710a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f404203eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710a3ad0 0x55a0710a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f404203eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070f8aac0 con 0x55a071098be0 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f404203eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070f8a7c0 con 0x55a0710a3ad0 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f404203eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070e6e920 con 0x55a0710982d0 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f404203eb80 10 client.0 _ll_register_callbacks cb 0x55a070f88760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:56.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710a3ad0 0x55a0710a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710a3ad0 0x55a0710a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44824/0 (socket says 172.21.15.161:44824) 2023-07-14T11:25:56.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f402cd9c700 1 -- 172.21.15.161:0/1864108343 learned_addr learned my addr 172.21.15.161:0/1864108343 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:56.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f40277fe700 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a0710a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.592+0000 7f4027fff700 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f402cd9c700 1 -- 172.21.15.161:0/1864108343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 msgr2=0x55a0710a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f402cd9c700 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f402cd9c700 1 -- 172.21.15.161:0/1864108343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 msgr2=0x55a0710a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f402cd9c700 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a0710a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f402cd9c700 1 -- 172.21.15.161:0/1864108343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402810d220 con 0x55a0710a3ad0 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f402cd9c700 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710a3ad0 0x55a0710a0ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f40200095b0 tx=0x7f4020004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c604298d5eb2f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f40077fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f40077fe700 1 -- 172.21.15.161:0/1864108343 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020029070 con 0x55a0710a3ad0 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a070f444f0 con 0x55a0710a3ad0 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f40077fe700 1 -- 172.21.15.161:0/1864108343 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f4020003a00 con 0x55a0710a3ad0 2023-07-14T11:25:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f40077fe700 1 -- 172.21.15.161:0/1864108343 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f402000e040 con 0x55a0710a3ad0 2023-07-14T11:25:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f404203eb80 10 client.26333 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.593+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a070e01c60 con 0x55a0710a3ad0 2023-07-14T11:25:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f40077fe700 1 -- 172.21.15.161:0/1864108343 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a070f444f0 con 0x55a0710a3ad0 2023-07-14T11:25:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f40077fe700 1 -- 172.21.15.161:0/1864108343 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f402002c020 con 0x55a0710a3ad0 2023-07-14T11:25:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f404203eb80 10 client.26333 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f404203eb80 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:56.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f404203eb80 20 client.26333 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:56.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f404203eb80 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710b3430 0x55a0710b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f404203eb80 4 client.26333 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:56.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a070fe3eb0 con 0x55a0710b3430 2023-07-14T11:25:56.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.594+0000 7f4027fff700 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710b3430 0x55a0710b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.595+0000 7f4027fff700 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710b3430 0x55a0710b5840 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da15ccceb21e9107 server_cookie=7e65829efb30e8db in_seq=0 out_seq=0 2023-07-14T11:25:56.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.595+0000 7f40077fe700 10 client.26333 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:56.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.595+0000 7f40077fe700 1 -- 172.21.15.161:0/1864108343 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a070fe3eb0 con 0x55a0710b3430 2023-07-14T11:25:56.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.595+0000 7f40077fe700 10 client.26333 handle_command_reply: tid=0 2023-07-14T11:25:56.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.595+0000 7f404203eb80 1 client.26333 shutdown 2023-07-14T11:25:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710a3ad0 msgr2=0x55a0710a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710a3ad0 0x55a0710a0ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f40200095b0 tx=0x7f4020004b40 comp rx=0 tx=0).stop 2023-07-14T11:25:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 shutdown_connections 2023-07-14T11:25:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710b3430 0x55a0710b5840 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710a3ad0 0x55a0710a0ed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 --2- 172.21.15.161:0/1864108343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a0710a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f40077fe700 0 client.26333 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 >> 172.21.15.161:0/1864108343 conn(0x55a07108eda0 msgr2=0x55a071094290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 shutdown_connections 2023-07-14T11:25:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 1 -- 172.21.15.161:0/1864108343 wait complete. 2023-07-14T11:25:56.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.596+0000 7f404203eb80 20 client.26333 trim_cache size 0 max 16384 2023-07-14T11:25:56.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 Processor -- start 2023-07-14T11:25:56.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 -- start start 2023-07-14T11:25:56.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a07108eda0 0x55a0710af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710982d0 0x55a07109ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a07109bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a071090f00 con 0x55a071098be0 2023-07-14T11:25:56.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a071095500 con 0x55a0710982d0 2023-07-14T11:25:56.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f404203eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07109ed20 con 0x55a07108eda0 2023-07-14T11:25:56.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f40277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710982d0 0x55a07109ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a07109bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f4027fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a07108eda0 0x55a0710af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f40277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710982d0 0x55a07109ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44836/0 (socket says 172.21.15.161:44836) 2023-07-14T11:25:56.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.615+0000 7f40277fe700 1 -- 172.21.15.161:0/3833010671 learned_addr learned my addr 172.21.15.161:0/3833010671 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:56.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f402cd9c700 1 -- 172.21.15.161:0/3833010671 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a07108eda0 msgr2=0x55a0710af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f402cd9c700 1 --2- 172.21.15.161:0/3833010671 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a07108eda0 0x55a0710af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f402cd9c700 1 -- 172.21.15.161:0/3833010671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710982d0 msgr2=0x55a07109ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f402cd9c700 1 --2- 172.21.15.161:0/3833010671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710982d0 0x55a07109ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f402cd9c700 1 -- 172.21.15.161:0/3833010671 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a070e3f250 con 0x55a071098be0 2023-07-14T11:25:56.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f402cd9c700 1 --2- 172.21.15.161:0/3833010671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a07109bf90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4020009a20 tx=0x7f4020006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab66604e6ee2bf32 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f400effd700 1 -- 172.21.15.161:0/3833010671 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000e030 con 0x55a071098be0 2023-07-14T11:25:56.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f400effd700 1 -- 172.21.15.161:0/3833010671 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f402000f040 con 0x55a071098be0 2023-07-14T11:25:56.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.616+0000 7f400effd700 1 -- 172.21.15.161:0/3833010671 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020029040 con 0x55a071098be0 2023-07-14T11:25:56.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 -- 172.21.15.161:0/3833010671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 msgr2=0x55a07109bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 --2- 172.21.15.161:0/3833010671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a07109bf90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4020009a20 tx=0x7f4020006060 comp rx=0 tx=0).stop 2023-07-14T11:25:56.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 -- 172.21.15.161:0/3833010671 shutdown_connections 2023-07-14T11:25:56.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 --2- 172.21.15.161:0/3833010671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a071098be0 0x55a07109bf90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 --2- 172.21.15.161:0/3833010671 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a07108eda0 0x55a0710af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 --2- 172.21.15.161:0/3833010671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0710982d0 0x55a07109ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 -- 172.21.15.161:0/3833010671 >> 172.21.15.161:0/3833010671 conn(0x55a0710a1040 msgr2=0x55a071092e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:56.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 -- 172.21.15.161:0/3833010671 shutdown_connections 2023-07-14T11:25:56.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.617+0000 7f404203eb80 1 -- 172.21.15.161:0/3833010671 wait complete. 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.618+0000 7f404203eb80 1 -- start start 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.618+0000 7f404203eb80 1 Processor -- start 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f404203eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a07108eda0 0x55a0710aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f404203eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f404203eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f404203eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0710a9530 con 0x55a07108eda0 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f404203eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0710a96a0 con 0x55a071098be0 2023-07-14T11:25:56.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f404203eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a071107300 con 0x55a0710982d0 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f404203eb80 10 client.0 _ll_register_callbacks cb 0x7f4020018f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f40277fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f4027fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a07108eda0 0x55a0710aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44838/0 (socket says 172.21.15.161:44838) 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f40277fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34512/0 (socket says 172.21.15.161:34512) 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f402cd9c700 1 -- 172.21.15.161:0/3707194497 learned_addr learned my addr 172.21.15.161:0/3707194497 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f40277fe700 1 -- 172.21.15.161:0/3707194497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 msgr2=0x55a0710a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f40277fe700 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f40277fe700 1 -- 172.21.15.161:0/3707194497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a07108eda0 msgr2=0x55a0710aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:56.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f40277fe700 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a07108eda0 0x55a0710aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.619+0000 7f40277fe700 1 -- 172.21.15.161:0/3707194497 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a070e01c60 con 0x55a0710982d0 2023-07-14T11:25:56.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f40277fe700 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a88c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a070fe3f20 tx=0x7f401c0052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b450420b3464c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f40077fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:56.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f40077fe700 1 -- 172.21.15.161:0/3707194497 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c01c040 con 0x55a0710982d0 2023-07-14T11:25:56.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f401c01d040 con 0x55a0710982d0 2023-07-14T11:25:56.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f40077fe700 1 -- 172.21.15.161:0/3707194497 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f401c00b560 con 0x55a0710982d0 2023-07-14T11:25:56.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f40077fe700 1 -- 172.21.15.161:0/3707194497 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f401c00b720 con 0x55a0710982d0 2023-07-14T11:25:56.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f404203eb80 10 client.26344 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:56.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a070e172b0 con 0x55a0710982d0 2023-07-14T11:25:56.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.620+0000 7f40077fe700 1 -- 172.21.15.161:0/3707194497 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c01d040 con 0x55a0710982d0 2023-07-14T11:25:56.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f40077fe700 1 -- 172.21.15.161:0/3707194497 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f401c009c70 con 0x55a0710982d0 2023-07-14T11:25:56.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f404203eb80 10 client.26344 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:56.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f404203eb80 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:56.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f404203eb80 20 client.26344 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:56.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f404203eb80 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710a3ad0 0x55a07110a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f404203eb80 4 client.26344 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a070fa6050 con 0x55a0710a3ad0 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f4027fff700 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710a3ad0 0x55a07110a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f40077fe700 10 client.26344 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.621+0000 7f4027fff700 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710a3ad0 0x55a07110a630 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb082fe48113ae5 server_cookie=68c43eabb4cd93c8 in_seq=0 out_seq=0 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f40077fe700 1 -- 172.21.15.161:0/3707194497 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a070fa6050 con 0x55a0710a3ad0 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f40077fe700 10 client.26344 handle_command_reply: tid=0 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 client.26344 shutdown 2023-07-14T11:25:56.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 msgr2=0x55a0710a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a88c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a070fe3f20 tx=0x7f401c0052e0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 shutdown_connections 2023-07-14T11:25:56.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a0710a3ad0 0x55a07110a630 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a07108eda0 0x55a0710aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a071098be0 0x55a0710a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f40077fe700 0 client.26344 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:56.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 --2- 172.21.15.161:0/3707194497 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a0710982d0 0x55a0710a88c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 >> 172.21.15.161:0/3707194497 conn(0x55a0710a1040 msgr2=0x55a0710af700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:56.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 shutdown_connections 2023-07-14T11:25:56.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 1 -- 172.21.15.161:0/3707194497 wait complete. 2023-07-14T11:25:56.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.622+0000 7f404203eb80 20 client.26344 trim_cache size 0 max 16384 2023-07-14T11:25:56.633 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:25:56.633 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:25:56.634 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:25:56.634 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:25:56.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40100a12c0 msgr2=0x7f40100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40100a12c0 0x7f40100a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f4020018640 comp rx=0 tx=0).stop 2023-07-14T11:25:56.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40280ffae0 msgr2=0x7f4028197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40280ffae0 0x7f4028197ec0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f401800e9a0 tx=0x7f401800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 shutdown_connections 2023-07-14T11:25:56.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40100a12c0 0x7f40100a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40280ffae0 0x7f4028197ec0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105af0 0x7f402819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 --2- 172.21.15.161:0/2617932742 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028100490 0x7f4028198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 >> 172.21.15.161:0/2617932742 conn(0x7f40280fb5e0 msgr2=0x7f402810ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:56.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 shutdown_connections 2023-07-14T11:25:56.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.623+0000 7f402e7ff700 1 -- 172.21.15.161:0/2617932742 wait complete. 2023-07-14T11:25:56.644 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:25:56.645 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:25:56.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.854+0000 7f15395ff700 1 -- 172.21.15.161:0/1028744677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 msgr2=0x7f1534072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.854+0000 7f15395ff700 1 --2- 172.21.15.161:0/1028744677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f1534072c20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1524009990 tx=0x7f1524009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.855+0000 7f15395ff700 1 -- 172.21.15.161:0/1028744677 shutdown_connections 2023-07-14T11:25:56.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.855+0000 7f15395ff700 1 --2- 172.21.15.161:0/1028744677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f153410c390 0x7f153410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.855+0000 7f15395ff700 1 --2- 172.21.15.161:0/1028744677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f1534072c20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.855+0000 7f15395ff700 1 --2- 172.21.15.161:0/1028744677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1534073160 0x7f153410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.855+0000 7f15395ff700 1 -- 172.21.15.161:0/1028744677 >> 172.21.15.161:0/1028744677 conn(0x7f15340fb630 msgr2=0x7f15340fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:56.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.855+0000 7f15395ff700 1 -- 172.21.15.161:0/1028744677 shutdown_connections 2023-07-14T11:25:56.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.855+0000 7f15395ff700 1 -- 172.21.15.161:0/1028744677 wait complete. 2023-07-14T11:25:56.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 Processor -- start 2023-07-14T11:25:56.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 -- start start 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f153410b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1534073160 0x7f1534109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f153410c390 0x7f1534109f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153410a4a0 con 0x7f153410c390 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153410a610 con 0x7f1534072850 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f15395ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153410a7d0 con 0x7f1534073160 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f1532ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f153410b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f1532ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f153410b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44850/0 (socket says 172.21.15.161:44850) 2023-07-14T11:25:56.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.856+0000 7f1532ffd700 1 -- 172.21.15.161:0/1786219216 learned_addr learned my addr 172.21.15.161:0/1786219216 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:56.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f15327fc700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1534073160 0x7f1534109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f15337fe700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f153410c390 0x7f1534109f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f1532ffd700 1 -- 172.21.15.161:0/1786219216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1534073160 msgr2=0x7f1534109a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f1532ffd700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1534073160 0x7f1534109a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f1532ffd700 1 -- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f153410c390 msgr2=0x7f1534109f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:56.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f1532ffd700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f153410c390 0x7f1534109f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f1532ffd700 1 -- 172.21.15.161:0/1786219216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240095f0 con 0x7f1534072850 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f15327fc700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1534073160 0x7f1534109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f1532ffd700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f153410b9b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1524005910 tx=0x7f1524005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f151bfff700 1 -- 172.21.15.161:0/1786219216 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152401d070 con 0x7f1534072850 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f151bfff700 1 -- 172.21.15.161:0/1786219216 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1524022470 con 0x7f1534072850 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.857+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15341a8aa0 con 0x7f1534072850 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.858+0000 7f151bfff700 1 -- 172.21.15.161:0/1786219216 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524017650 con 0x7f1534072850 2023-07-14T11:25:56.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.858+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15341a8f70 con 0x7f1534072850 2023-07-14T11:25:56.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.859+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1534104320 con 0x7f1534072850 2023-07-14T11:25:56.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.861+0000 7f151bfff700 1 -- 172.21.15.161:0/1786219216 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f152400fad0 con 0x7f1534072850 2023-07-14T11:25:56.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.862+0000 7f151bfff700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f151c0a13c0 0x7f151c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:56.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.862+0000 7f151bfff700 1 -- 172.21.15.161:0/1786219216 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f15240ce560 con 0x7f1534072850 2023-07-14T11:25:56.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.862+0000 7f15327fc700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f151c0a13c0 0x7f151c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:56.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.863+0000 7f15327fc700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f151c0a13c0 0x7f151c0a3870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f153410b210 tx=0x7f152800b410 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:56.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:56.865+0000 7f151bfff700 1 -- 172.21.15.161:0/1786219216 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1524097670 con 0x7f1534072850 2023-07-14T11:25:57.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 300151 9.82 MB/sec execute 39 sec latency 56.038 ms 2023-07-14T11:25:57.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.185+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f153410af50 con 0x7f1534072850 2023-07-14T11:25:57.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.186+0000 7f151bfff700 1 -- 172.21.15.161:0/1786219216 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f153410af50 con 0x7f1534072850 2023-07-14T11:25:57.189 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:25:57.190 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:23:41.117589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:25:57.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f151c0a13c0 msgr2=0x7f151c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f151c0a13c0 0x7f151c0a3870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f153410b210 tx=0x7f152800b410 comp rx=0 tx=0).stop 2023-07-14T11:25:57.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 msgr2=0x7f153410b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f153410b9b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1524005910 tx=0x7f1524005740 comp rx=0 tx=0).stop 2023-07-14T11:25:57.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 shutdown_connections 2023-07-14T11:25:57.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f151c0a13c0 0x7f151c0a3870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f153410c390 0x7f1534109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1534072850 0x7f153410b9b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 --2- 172.21.15.161:0/1786219216 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1534073160 0x7f1534109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 >> 172.21.15.161:0/1786219216 conn(0x7f15340fb630 msgr2=0x7f153410e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 shutdown_connections 2023-07-14T11:25:57.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.189+0000 7f15395ff700 1 -- 172.21.15.161:0/1786219216 wait complete. 2023-07-14T11:25:57.194 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2023-07-14T11:25:57.211 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:25:57.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 -- 172.21.15.161:0/437050693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4518073160 msgr2=0x7f451810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 --2- 172.21.15.161:0/437050693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4518073160 0x7f451810be60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4504009990 tx=0x7f4504009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 -- 172.21.15.161:0/437050693 shutdown_connections 2023-07-14T11:25:57.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 --2- 172.21.15.161:0/437050693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4518072850 0x7f4518072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 --2- 172.21.15.161:0/437050693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f451810c3a0 0x7f451810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 --2- 172.21.15.161:0/437050693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4518073160 0x7f451810be60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 -- 172.21.15.161:0/437050693 >> 172.21.15.161:0/437050693 conn(0x7f45180fb600 msgr2=0x7f45180fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 -- 172.21.15.161:0/437050693 shutdown_connections 2023-07-14T11:25:57.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.423+0000 7f451efff700 1 -- 172.21.15.161:0/437050693 wait complete. 2023-07-14T11:25:57.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.424+0000 7f451efff700 1 Processor -- start 2023-07-14T11:25:57.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451efff700 1 -- start start 2023-07-14T11:25:57.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451efff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4518072850 0x7f4518197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451efff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4518073160 0x7f4518198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451efff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f451810c3a0 0x7f451819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451efff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819ce10 con 0x7f4518072850 2023-07-14T11:25:57.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451efff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819cf80 con 0x7f4518073160 2023-07-14T11:25:57.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451efff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819d140 con 0x7f451810c3a0 2023-07-14T11:25:57.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f4517fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4518073160 0x7f4518198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451d59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f451810c3a0 0x7f451819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f4517fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4518073160 0x7f4518198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44854/0 (socket says 172.21.15.161:44854) 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f4517fff700 1 -- 172.21.15.161:0/2456236679 learned_addr learned my addr 172.21.15.161:0/2456236679 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.425+0000 7f451cd9b700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4518072850 0x7f4518197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4517fff700 1 -- 172.21.15.161:0/2456236679 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f451810c3a0 msgr2=0x7f451819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4517fff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f451810c3a0 0x7f451819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4517fff700 1 -- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4518072850 msgr2=0x7f4518197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4517fff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4518072850 0x7f4518197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4517fff700 1 -- 172.21.15.161:0/2456236679 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040095f0 con 0x7f4518073160 2023-07-14T11:25:57.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f451d59c700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f451810c3a0 0x7f451819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:57.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4517fff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4518073160 0x7f4518198370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4504005910 tx=0x7f4504005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4515ffb700 1 -- 172.21.15.161:0/2456236679 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450401d070 con 0x7f4518073160 2023-07-14T11:25:57.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451819d3c0 con 0x7f4518073160 2023-07-14T11:25:57.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4515ffb700 1 -- 172.21.15.161:0/2456236679 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4504022470 con 0x7f4518073160 2023-07-14T11:25:57.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f4515ffb700 1 -- 172.21.15.161:0/2456236679 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504017650 con 0x7f4518073160 2023-07-14T11:25:57.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.426+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45181a4c60 con 0x7f4518073160 2023-07-14T11:25:57.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.430+0000 7f4515ffb700 1 -- 172.21.15.161:0/2456236679 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f450400fad0 con 0x7f4518073160 2023-07-14T11:25:57.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.432+0000 7f4515ffb700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45080a12c0 0x7f45080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.432+0000 7f4515ffb700 1 -- 172.21.15.161:0/2456236679 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f45040ce4a0 con 0x7f4518073160 2023-07-14T11:25:57.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.432+0000 7f4532939b80 1 Processor -- start 2023-07-14T11:25:57.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.432+0000 7f451cd9b700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45080a12c0 0x7f45080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.432+0000 7f4532939b80 1 -- start start 2023-07-14T11:25:57.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f4532939b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd273e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f4532939b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd2751da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f4532939b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f4532939b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd2644cb0 con 0x55ecd273e2d0 2023-07-14T11:25:57.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f4532939b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd2697a40 con 0x55ecd27522e0 2023-07-14T11:25:57.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f4532939b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd2690ed0 con 0x55ecd273ebe0 2023-07-14T11:25:57.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f451d59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f451d59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44858/0 (socket says 172.21.15.161:44858) 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.433+0000 7f451d59c700 1 -- 172.21.15.161:0/505381160 learned_addr learned my addr 172.21.15.161:0/505381160 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f4517fff700 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd2751da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451cd9b700 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd273e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451cd9b700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45080a12c0 0x7f45080a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f450c005950 tx=0x7f450c00b410 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451d59c700 1 -- 172.21.15.161:0/505381160 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 msgr2=0x55ecd2751da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451d59c700 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd2751da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451d59c700 1 -- 172.21.15.161:0/505381160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 msgr2=0x55ecd273e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451d59c700 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd273e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451d59c700 1 -- 172.21.15.161:0/505381160 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd24bede0 con 0x55ecd27522e0 2023-07-14T11:25:57.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.434+0000 7f451d59c700 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27546c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4510003e30 tx=0x7f4510009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ec0453aa725ad5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.435+0000 7f4502ffd700 1 -- 172.21.15.161:0/505381160 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100092d0 con 0x55ecd27522e0 2023-07-14T11:25:57.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.435+0000 7f4502ffd700 1 -- 172.21.15.161:0/505381160 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451000f040 con 0x55ecd27522e0 2023-07-14T11:25:57.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.435+0000 7f4502ffd700 1 -- 172.21.15.161:0/505381160 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451000e7e0 con 0x55ecd27522e0 2023-07-14T11:25:57.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 -- 172.21.15.161:0/505381160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 msgr2=0x55ecd27546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27546c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4510003e30 tx=0x7f4510009920 comp rx=0 tx=0).stop 2023-07-14T11:25:57.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 -- 172.21.15.161:0/505381160 shutdown_connections 2023-07-14T11:25:57.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd273e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27546c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 --2- 172.21.15.161:0/505381160 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd2751da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 -- 172.21.15.161:0/505381160 >> 172.21.15.161:0/505381160 conn(0x55ecd2734da0 msgr2=0x55ecd273a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 -- 172.21.15.161:0/505381160 shutdown_connections 2023-07-14T11:25:57.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.436+0000 7f4532939b80 1 -- 172.21.15.161:0/505381160 wait complete. 2023-07-14T11:25:57.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.437+0000 7f4532939b80 1 -- start start 2023-07-14T11:25:57.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.437+0000 7f4532939b80 1 Processor -- start 2023-07-14T11:25:57.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4532939b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273e2d0 0x55ecd2751d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4532939b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd274ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4532939b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd27522e0 0x55ecd274f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4532939b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd2630ac0 con 0x55ecd27522e0 2023-07-14T11:25:57.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4532939b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd26307c0 con 0x55ecd273e2d0 2023-07-14T11:25:57.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4532939b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd2514920 con 0x55ecd273ebe0 2023-07-14T11:25:57.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4532939b80 10 client.0 _ll_register_callbacks cb 0x55ecd262e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:57.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd274ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd274ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34560/0 (socket says 172.21.15.161:34560) 2023-07-14T11:25:57.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 -- 172.21.15.161:0/880963190 learned_addr learned my addr 172.21.15.161:0/880963190 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f451cd9b700 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273e2d0 0x55ecd2751d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f451d59c700 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd27522e0 0x55ecd274f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 -- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273e2d0 msgr2=0x55ecd2751d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273e2d0 0x55ecd2751d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 -- 172.21.15.161:0/880963190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd27522e0 msgr2=0x55ecd274f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd27522e0 0x55ecd274f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.438+0000 7f4517fff700 1 -- 172.21.15.161:0/880963190 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4510003ae0 con 0x55ecd273ebe0 2023-07-14T11:25:57.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4517fff700 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd274ef50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4504000c00 tx=0x7f4504009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee1b12caae98bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f44f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:57.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f44f7fff700 1 -- 172.21.15.161:0/880963190 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450409b2d0 con 0x55ecd273ebe0 2023-07-14T11:25:57.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f44f7fff700 1 -- 172.21.15.161:0/880963190 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f450409f300 con 0x55ecd273ebe0 2023-07-14T11:25:57.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecd25ea4f0 con 0x55ecd273ebe0 2023-07-14T11:25:57.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f44f7fff700 1 -- 172.21.15.161:0/880963190 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45040a3690 con 0x55ecd273ebe0 2023-07-14T11:25:57.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 10 client.26362 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:57.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ecd24a7c60 con 0x55ecd273ebe0 2023-07-14T11:25:57.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f44f7fff700 1 -- 172.21.15.161:0/880963190 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecd25ea4f0 con 0x55ecd273ebe0 2023-07-14T11:25:57.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f44f7fff700 1 -- 172.21.15.161:0/880963190 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f45040ce130 con 0x55ecd273ebe0 2023-07-14T11:25:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 10 client.26362 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 10 client.26362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:25:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 20 client.26362 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27616e0 0x55ecd2763af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 4 client.26362 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:57.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.439+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecd2689eb0 con 0x55ecd27616e0 2023-07-14T11:25:57.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.440+0000 7f451cd9b700 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27616e0 0x55ecd2763af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.440+0000 7f451cd9b700 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27616e0 0x55ecd2763af0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=543195c3b5f9f39e server_cookie=5f201a1a6ca48355 in_seq=0 out_seq=0 2023-07-14T11:25:57.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.440+0000 7f44f7fff700 10 client.26362 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:57.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f44f7fff700 1 -- 172.21.15.161:0/880963190 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ecd2689eb0 con 0x55ecd27616e0 2023-07-14T11:25:57.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f44f7fff700 10 client.26362 handle_command_reply: tid=0 2023-07-14T11:25:57.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 client.26362 shutdown 2023-07-14T11:25:57.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 msgr2=0x55ecd274ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd274ef50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4504000c00 tx=0x7f4504009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 shutdown_connections 2023-07-14T11:25:57.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27616e0 0x55ecd2763af0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd27522e0 0x55ecd274f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273e2d0 0x55ecd2751d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 --2- 172.21.15.161:0/880963190 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273ebe0 0x55ecd274ef50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f44f7fff700 0 client.26362 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.441+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 >> 172.21.15.161:0/880963190 conn(0x55ecd2734da0 msgr2=0x55ecd273a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.442+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 shutdown_connections 2023-07-14T11:25:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.442+0000 7f4532939b80 1 -- 172.21.15.161:0/880963190 wait complete. 2023-07-14T11:25:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.442+0000 7f4532939b80 20 client.26362 trim_cache size 0 max 16384 2023-07-14T11:25:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 Processor -- start 2023-07-14T11:25:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 -- start start 2023-07-14T11:25:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd274f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273ebe0 0x55ecd2758cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd27522e0 0x55ecd275deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd273f160 con 0x55ecd273e2d0 2023-07-14T11:25:57.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd2759200 con 0x55ecd273ebe0 2023-07-14T11:25:57.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.456+0000 7f4532939b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd275e3f0 con 0x55ecd27522e0 2023-07-14T11:25:57.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f451cd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd274f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273ebe0 0x55ecd2758cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273ebe0 0x55ecd2758cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44876/0 (socket says 172.21.15.161:44876) 2023-07-14T11:25:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 -- 172.21.15.161:0/1573411922 learned_addr learned my addr 172.21.15.161:0/1573411922 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f451d59c700 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd27522e0 0x55ecd275deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 -- 172.21.15.161:0/1573411922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd27522e0 msgr2=0x55ecd275deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd27522e0 0x55ecd275deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 -- 172.21.15.161:0/1573411922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 msgr2=0x55ecd274f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd274f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 -- 172.21.15.161:0/1573411922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd24e5250 con 0x55ecd273ebe0 2023-07-14T11:25:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f4517fff700 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273ebe0 0x55ecd2758cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45040976c0 tx=0x7f45040094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e220e51032be690 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f45037fe700 1 -- 172.21.15.161:0/1573411922 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450409aa20 con 0x55ecd273ebe0 2023-07-14T11:25:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f45037fe700 1 -- 172.21.15.161:0/1573411922 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450409a810 con 0x55ecd273ebe0 2023-07-14T11:25:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.457+0000 7f45037fe700 1 -- 172.21.15.161:0/1573411922 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504022850 con 0x55ecd273ebe0 2023-07-14T11:25:57.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 -- 172.21.15.161:0/1573411922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273ebe0 msgr2=0x55ecd2758cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273ebe0 0x55ecd2758cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45040976c0 tx=0x7f45040094f0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 -- 172.21.15.161:0/1573411922 shutdown_connections 2023-07-14T11:25:57.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273e2d0 0x55ecd274f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd27522e0 0x55ecd275deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 --2- 172.21.15.161:0/1573411922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd273ebe0 0x55ecd2758cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 -- 172.21.15.161:0/1573411922 >> 172.21.15.161:0/1573411922 conn(0x55ecd2734da0 msgr2=0x55ecd2738e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 -- 172.21.15.161:0/1573411922 shutdown_connections 2023-07-14T11:25:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.458+0000 7f4532939b80 1 -- 172.21.15.161:0/1573411922 wait complete. 2023-07-14T11:25:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 -- start start 2023-07-14T11:25:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 Processor -- start 2023-07-14T11:25:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273e2d0 0x55ecd2756e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273ebe0 0x55ecd27573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd2757a90 con 0x55ecd273ebe0 2023-07-14T11:25:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd27b4ae0 con 0x55ecd27522e0 2023-07-14T11:25:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd27b4c20 con 0x55ecd273e2d0 2023-07-14T11:25:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.459+0000 7f4532939b80 10 client.0 _ll_register_callbacks cb 0x55ecd262e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273e2d0 0x55ecd2756e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273e2d0 0x55ecd2756e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34586/0 (socket says 172.21.15.161:34586) 2023-07-14T11:25:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 -- 172.21.15.161:0/1577723076 learned_addr learned my addr 172.21.15.161:0/1577723076 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451d59c700 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f4517fff700 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273ebe0 0x55ecd27573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 -- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 msgr2=0x55ecd27b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 -- 172.21.15.161:0/1577723076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273ebe0 msgr2=0x55ecd27573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273ebe0 0x55ecd27573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 -- 172.21.15.161:0/1577723076 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecd24a7c60 con 0x55ecd273e2d0 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f451cd9b700 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273e2d0 0x55ecd2756e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55ecd27643d0 tx=0x7f450c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ac55c5dfb78041 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.460+0000 7f44f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f44f7fff700 1 -- 172.21.15.161:0/1577723076 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c024040 con 0x55ecd273e2d0 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f44f7fff700 1 -- 172.21.15.161:0/1577723076 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f450c017070 con 0x55ecd273e2d0 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f451000f040 con 0x55ecd273e2d0 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f44f7fff700 1 -- 172.21.15.161:0/1577723076 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f450c011640 con 0x55ecd273e2d0 2023-07-14T11:25:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 10 client.26368 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ecd24bd2b0 con 0x55ecd273e2d0 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f44f7fff700 1 -- 172.21.15.161:0/1577723076 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451000f040 con 0x55ecd273e2d0 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f44f7fff700 1 -- 172.21.15.161:0/1577723076 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f450c01f990 con 0x55ecd273e2d0 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 10 client.26368 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 10 client.26368 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 20 client.26368 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27b63b0 0x55ecd27b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 4 client.26368 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:25:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.461+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ecd264c050 con 0x55ecd27b63b0 2023-07-14T11:25:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f4517fff700 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27b63b0 0x55ecd27b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f4517fff700 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27b63b0 0x55ecd27b87c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdcdcb87cca6d0e8 server_cookie=e6dddc8b4400a020 in_seq=0 out_seq=0 2023-07-14T11:25:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f44f7fff700 10 client.26368 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f44f7fff700 1 -- 172.21.15.161:0/1577723076 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ecd264c050 con 0x55ecd27b63b0 2023-07-14T11:25:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f44f7fff700 10 client.26368 handle_command_reply: tid=0 2023-07-14T11:25:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f4532939b80 1 client.26368 shutdown 2023-07-14T11:25:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273e2d0 msgr2=0x55ecd2756e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.462+0000 7f4532939b80 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273e2d0 0x55ecd2756e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55ecd27643d0 tx=0x7f450c0047c0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 shutdown_connections 2023-07-14T11:25:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ecd27b63b0 0x55ecd27b87c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecd27522e0 0x55ecd27b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ecd273e2d0 0x55ecd2756e90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f44f7fff700 0 client.26368 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:25:57.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 --2- 172.21.15.161:0/1577723076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ecd273ebe0 0x55ecd27573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 >> 172.21.15.161:0/1577723076 conn(0x55ecd2734da0 msgr2=0x55ecd274a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 shutdown_connections 2023-07-14T11:25:57.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 1 -- 172.21.15.161:0/1577723076 wait complete. 2023-07-14T11:25:57.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f4532939b80 20 client.26368 trim_cache size 0 max 16384 2023-07-14T11:25:57.474 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:25:57.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45080a12c0 msgr2=0x7f45080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45080a12c0 0x7f45080a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f450c005950 tx=0x7f450c00b410 comp rx=0 tx=0).stop 2023-07-14T11:25:57.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4518073160 msgr2=0x7f4518198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4518073160 0x7f4518198370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4504005910 tx=0x7f4504005740 comp rx=0 tx=0).stop 2023-07-14T11:25:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 shutdown_connections 2023-07-14T11:25:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45080a12c0 0x7f45080a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4518072850 0x7f4518197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f451810c3a0 0x7f451819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.463+0000 7f451efff700 1 --2- 172.21.15.161:0/2456236679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4518073160 0x7f4518198370 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.464+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 >> 172.21.15.161:0/2456236679 conn(0x7f45180fb600 msgr2=0x7f4518103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.464+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 shutdown_connections 2023-07-14T11:25:57.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.464+0000 7f451efff700 1 -- 172.21.15.161:0/2456236679 wait complete. 2023-07-14T11:25:57.488 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:25:57.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 -- 172.21.15.161:0/349723705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c100480 msgr2=0x7fee1c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 --2- 172.21.15.161:0/349723705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c100480 0x7fee1c10c910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee1400b0a0 tx=0x7fee1400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 -- 172.21.15.161:0/349723705 shutdown_connections 2023-07-14T11:25:57.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 --2- 172.21.15.161:0/349723705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee1c105ae0 0x7fee1c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 --2- 172.21.15.161:0/349723705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c100480 0x7fee1c10c910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 --2- 172.21.15.161:0/349723705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fee1c0ffad0 0x7fee1c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 -- 172.21.15.161:0/349723705 >> 172.21.15.161:0/349723705 conn(0x7fee1c0fb5f0 msgr2=0x7fee1c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 -- 172.21.15.161:0/349723705 shutdown_connections 2023-07-14T11:25:57.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.737+0000 7fee213ff700 1 -- 172.21.15.161:0/349723705 wait complete. 2023-07-14T11:25:57.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.738+0000 7fee213ff700 1 Processor -- start 2023-07-14T11:25:57.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee213ff700 1 -- start start 2023-07-14T11:25:57.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee213ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee1c0ffad0 0x7fee1c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee213ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fee1c100480 0x7fee1c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee213ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c105ae0 0x7fee1c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee213ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c19cf20 con 0x7fee1c0ffad0 2023-07-14T11:25:57.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee213ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c19d090 con 0x7fee1c105ae0 2023-07-14T11:25:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee213ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c19d250 con 0x7fee1c100480 2023-07-14T11:25:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee1affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee1c0ffad0 0x7fee1c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee1a7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fee1c100480 0x7fee1c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee1b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c105ae0 0x7fee1c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee1affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee1c0ffad0 0x7fee1c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60510/0 (socket says 172.21.15.161:60510) 2023-07-14T11:25:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.739+0000 7fee1affd700 1 -- 172.21.15.161:0/2758561092 learned_addr learned my addr 172.21.15.161:0/2758561092 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee1b7fe700 1 -- 172.21.15.161:0/2758561092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fee1c100480 msgr2=0x7fee1c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee1b7fe700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fee1c100480 0x7fee1c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee1b7fe700 1 -- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee1c0ffad0 msgr2=0x7fee1c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee1b7fe700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee1c0ffad0 0x7fee1c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee1b7fe700 1 -- 172.21.15.161:0/2758561092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c009520 con 0x7fee1c105ae0 2023-07-14T11:25:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee1b7fe700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c105ae0 0x7fee1c19c8e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee14009fd0 tx=0x7fee14007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee03fff700 1 -- 172.21.15.161:0/2758561092 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400e040 con 0x7fee1c105ae0 2023-07-14T11:25:57.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee03fff700 1 -- 172.21.15.161:0/2758561092 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee1401d470 con 0x7fee1c105ae0 2023-07-14T11:25:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee14009d00 con 0x7fee1c105ae0 2023-07-14T11:25:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee03fff700 1 -- 172.21.15.161:0/2758561092 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee140052a0 con 0x7fee1c105ae0 2023-07-14T11:25:57.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.740+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c1a4b90 con 0x7fee1c105ae0 2023-07-14T11:25:57.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.745+0000 7fee03fff700 1 -- 172.21.15.161:0/2758561092 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fee1401d5e0 con 0x7fee1c105ae0 2023-07-14T11:25:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 Processor -- start 2023-07-14T11:25:57.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 -- start start 2023-07-14T11:25:57.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e945872d0 0x564e945876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94587be0 0x564e94592590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94592ad0 0x564e94594eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9448dcb0 con 0x564e945872d0 2023-07-14T11:25:57.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e944e0a40 con 0x564e94592ad0 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.746+0000 7fee34d8db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e944d9ed0 con 0x564e94587be0 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94587be0 0x564e94592590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94587be0 0x564e94592590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34612/0 (socket says 172.21.15.161:34612) 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 -- 172.21.15.161:0/3787790068 learned_addr learned my addr 172.21.15.161:0/3787790068 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e945872d0 0x564e945876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94592ad0 0x564e94594eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 -- 172.21.15.161:0/3787790068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94592ad0 msgr2=0x564e94594eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94592ad0 0x564e94594eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 -- 172.21.15.161:0/3787790068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e945872d0 msgr2=0x564e945876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e945872d0 0x564e945876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 -- 172.21.15.161:0/3787790068 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e94307de0 con 0x564e94587be0 2023-07-14T11:25:57.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94592ad0 0x564e94594eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:25:57.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee03fff700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fee040a12c0 0x7fee040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1affd700 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94587be0 0x564e94592590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009fb0 tx=0x7fee0c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff7040f5c7b48143 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee03fff700 1 -- 172.21.15.161:0/2758561092 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fee140ce450 con 0x7fee1c105ae0 2023-07-14T11:25:57.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.747+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fee040a12c0 0x7fee040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.748+0000 7fee017fa700 1 -- 172.21.15.161:0/3787790068 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c019630 con 0x564e94587be0 2023-07-14T11:25:57.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.748+0000 7fee017fa700 1 -- 172.21.15.161:0/3787790068 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0c019c50 con 0x564e94587be0 2023-07-14T11:25:57.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.748+0000 7fee017fa700 1 -- 172.21.15.161:0/3787790068 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00fc80 con 0x564e94587be0 2023-07-14T11:25:57.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.748+0000 7fee34d8db80 1 -- 172.21.15.161:0/3787790068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94587be0 msgr2=0x564e94592590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.748+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fee040a12c0 0x7fee040a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee100097c0 tx=0x7fee10009750 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.748+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94587be0 0x564e94592590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009fb0 tx=0x7fee0c011720 comp rx=0 tx=0).stop 2023-07-14T11:25:57.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 -- 172.21.15.161:0/3787790068 shutdown_connections 2023-07-14T11:25:57.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e945872d0 0x564e945876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94592ad0 0x564e94594eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3787790068 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94587be0 0x564e94592590 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 -- 172.21.15.161:0/3787790068 >> 172.21.15.161:0/3787790068 conn(0x564e9457dda0 msgr2=0x564e945836a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 -- 172.21.15.161:0/3787790068 shutdown_connections 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 -- 172.21.15.161:0/3787790068 wait complete. 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 -- start start 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.749+0000 7fee34d8db80 1 Processor -- start 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e94592540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e9458f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e9458feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee34d8db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94479ac0 con 0x564e94587be0 2023-07-14T11:25:57.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee34d8db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e944797c0 con 0x564e945872d0 2023-07-14T11:25:57.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee34d8db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9435d920 con 0x564e94592ad0 2023-07-14T11:25:57.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee34d8db80 10 client.0 _ll_register_callbacks cb 0x564e94477760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:57.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee1affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e94592540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee1affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e94592540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44916/0 (socket says 172.21.15.161:44916) 2023-07-14T11:25:57.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee1affd700 1 -- 172.21.15.161:0/2780011326 learned_addr learned my addr 172.21.15.161:0/2780011326 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e9458f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee1b7fe700 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e9458feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee1affd700 1 -- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 msgr2=0x564e9458feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.750+0000 7fee1affd700 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e9458feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fee1affd700 1 -- 172.21.15.161:0/2780011326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 msgr2=0x564e9458f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fee1affd700 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e9458f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fee1affd700 1 -- 172.21.15.161:0/2780011326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c009c10 con 0x564e945872d0 2023-07-14T11:25:57.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fee1affd700 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e94592540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee0c000c00 tx=0x7fee0c004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0b4f76e2ef90f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fedfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:57.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fedfeff5700 1 -- 172.21.15.161:0/2780011326 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00fcd0 con 0x564e945872d0 2023-07-14T11:25:57.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e944334f0 con 0x564e945872d0 2023-07-14T11:25:57.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fedfeff5700 1 -- 172.21.15.161:0/2780011326 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fee0c019dc0 con 0x564e945872d0 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fedfeff5700 1 -- 172.21.15.161:0/2780011326 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee0c00f5e0 con 0x564e945872d0 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fee34d8db80 10 client.26372 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.751+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564e942f0c60 con 0x564e945872d0 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fedfeff5700 1 -- 172.21.15.161:0/2780011326 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e944334f0 con 0x564e945872d0 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fedfeff5700 1 -- 172.21.15.161:0/2780011326 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fee0c028020 con 0x564e945872d0 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fee34d8db80 10 client.26372 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fee34d8db80 10 client.26372 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:25:57.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fee34d8db80 20 client.26372 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:57.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fee34d8db80 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945a22e0 0x564e945a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fee34d8db80 4 client.26372 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:57.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e944d2eb0 con 0x564e945a22e0 2023-07-14T11:25:57.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.752+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945a22e0 0x564e945a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.753+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945a22e0 0x564e945a46f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=273b0250093e46aa server_cookie=c1dd340e3b978d64 in_seq=0 out_seq=0 2023-07-14T11:25:57.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.753+0000 7fedfeff5700 10 client.26372 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:57.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.753+0000 7fedfeff5700 1 -- 172.21.15.161:0/2780011326 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e944d2eb0 con 0x564e945a22e0 2023-07-14T11:25:57.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.753+0000 7fedfeff5700 10 client.26372 handle_command_reply: tid=0 2023-07-14T11:25:57.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.753+0000 7fee34d8db80 1 client.26372 shutdown 2023-07-14T11:25:57.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 msgr2=0x564e94592540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e94592540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee0c000c00 tx=0x7fee0c004510 comp rx=0 tx=0).stop 2023-07-14T11:25:57.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 shutdown_connections 2023-07-14T11:25:57.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945a22e0 0x564e945a46f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e9458feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e94592540 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fedfeff5700 0 client.26372 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:57.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 --2- 172.21.15.161:0/2780011326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e9458f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 >> 172.21.15.161:0/2780011326 conn(0x564e9457dda0 msgr2=0x564e94583230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 shutdown_connections 2023-07-14T11:25:57.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 1 -- 172.21.15.161:0/2780011326 wait complete. 2023-07-14T11:25:57.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.754+0000 7fee34d8db80 20 client.26372 trim_cache size 0 max 16384 2023-07-14T11:25:57.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 Processor -- start 2023-07-14T11:25:57.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 -- start start 2023-07-14T11:25:57.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e945872d0 0x564e9458fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94587be0 0x564e945994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94592ad0 0x564e945892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94584d20 con 0x564e94592ad0 2023-07-14T11:25:57.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94588540 con 0x564e94587be0 2023-07-14T11:25:57.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee34d8db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9458dd60 con 0x564e945872d0 2023-07-14T11:25:57.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e945872d0 0x564e9458fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94592ad0 0x564e945892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e945872d0 0x564e9458fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34638/0 (socket says 172.21.15.161:34638) 2023-07-14T11:25:57.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 -- 172.21.15.161:0/760191432 learned_addr learned my addr 172.21.15.161:0/760191432 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.769+0000 7fee1a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94587be0 0x564e945994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 -- 172.21.15.161:0/760191432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94587be0 msgr2=0x564e945994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 --2- 172.21.15.161:0/760191432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94587be0 0x564e945994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 -- 172.21.15.161:0/760191432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94592ad0 msgr2=0x564e945892c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 --2- 172.21.15.161:0/760191432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94592ad0 0x564e945892c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 -- 172.21.15.161:0/760191432 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e9432e250 con 0x564e945872d0 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee1affd700 1 --2- 172.21.15.161:0/760191432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e945872d0 0x564e9458fe70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee0c011760 tx=0x7fee0c011bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99f440b87df4f99 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee01ffb700 1 -- 172.21.15.161:0/760191432 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c019dc0 con 0x564e945872d0 2023-07-14T11:25:57.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee01ffb700 1 -- 172.21.15.161:0/760191432 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0c0056a0 con 0x564e945872d0 2023-07-14T11:25:57.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.770+0000 7fee01ffb700 1 -- 172.21.15.161:0/760191432 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0228d0 con 0x564e945872d0 2023-07-14T11:25:57.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 -- 172.21.15.161:0/760191432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e945872d0 msgr2=0x564e9458fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 --2- 172.21.15.161:0/760191432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e945872d0 0x564e9458fe70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee0c011760 tx=0x7fee0c011bd0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 -- 172.21.15.161:0/760191432 shutdown_connections 2023-07-14T11:25:57.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 --2- 172.21.15.161:0/760191432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94592ad0 0x564e945892c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 --2- 172.21.15.161:0/760191432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e945872d0 0x564e9458fe70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 --2- 172.21.15.161:0/760191432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e94587be0 0x564e945994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 -- 172.21.15.161:0/760191432 >> 172.21.15.161:0/760191432 conn(0x564e9457dda0 msgr2=0x564e9459e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 -- 172.21.15.161:0/760191432 shutdown_connections 2023-07-14T11:25:57.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.771+0000 7fee34d8db80 1 -- 172.21.15.161:0/760191432 wait complete. 2023-07-14T11:25:57.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 -- start start 2023-07-14T11:25:57.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 Processor -- start 2023-07-14T11:25:57.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e945974d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e94597a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e94597f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e945f52e0 con 0x564e94587be0 2023-07-14T11:25:57.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e945f5450 con 0x564e945872d0 2023-07-14T11:25:57.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.772+0000 7fee34d8db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e945f5610 con 0x564e94592ad0 2023-07-14T11:25:57.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee34d8db80 10 client.0 _ll_register_callbacks cb 0x7fee0c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:57.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e945974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e945974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44946/0 (socket says 172.21.15.161:44946) 2023-07-14T11:25:57.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e94597f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 -- 172.21.15.161:0/3520687196 learned_addr learned my addr 172.21.15.161:0/3520687196 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:57.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e94597a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 -- 172.21.15.161:0/3520687196 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 msgr2=0x564e94597f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e94597f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 -- 172.21.15.161:0/3520687196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 msgr2=0x564e94597a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:57.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e94597a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 -- 172.21.15.161:0/3520687196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e942f0c60 con 0x564e945872d0 2023-07-14T11:25:57.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fee1affd700 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e945974d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee0c011b20 tx=0x7fee0c019e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf100f34726ed24 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:57.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.773+0000 7fedfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:57.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fedfeff5700 1 -- 172.21.15.161:0/3520687196 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c03d070 con 0x564e945872d0 2023-07-14T11:25:57.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fedfeff5700 1 -- 172.21.15.161:0/3520687196 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fee0c000e40 con 0x564e945872d0 2023-07-14T11:25:57.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee0c019c50 con 0x564e945872d0 2023-07-14T11:25:57.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fedfeff5700 1 -- 172.21.15.161:0/3520687196 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee0c0225c0 con 0x564e945872d0 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fee34d8db80 10 client.26378 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564e943062b0 con 0x564e945872d0 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fedfeff5700 1 -- 172.21.15.161:0/3520687196 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0c019c50 con 0x564e945872d0 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.774+0000 7fedfeff5700 1 -- 172.21.15.161:0/3520687196 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fee0c040020 con 0x564e945872d0 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee34d8db80 10 client.26378 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee34d8db80 10 client.26378 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee34d8db80 20 client.26378 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:57.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945f68c0 0x564e945f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:57.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee34d8db80 4 client.26378 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:25:57.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e94495050 con 0x564e945f68c0 2023-07-14T11:25:57.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945f68c0 0x564e945f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:57.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fee1a7fc700 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945f68c0 0x564e945f8cd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=988fcb96aec186af server_cookie=df8eff2dcbedc31a in_seq=0 out_seq=0 2023-07-14T11:25:57.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fedfeff5700 10 client.26378 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:57.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fedfeff5700 1 -- 172.21.15.161:0/3520687196 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e94495050 con 0x564e945f68c0 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.775+0000 7fedfeff5700 10 client.26378 handle_command_reply: tid=0 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 client.26378 shutdown 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 msgr2=0x564e945974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e945974d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee0c011b20 tx=0x7fee0c019e40 comp rx=0 tx=0).stop 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 shutdown_connections 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x564e945f68c0 0x564e945f8cd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564e94592ad0 0x564e94597f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fedfeff5700 0 client.26378 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:25:57.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e945872d0 0x564e945974d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 --2- 172.21.15.161:0/3520687196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e94587be0 0x564e94597a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 >> 172.21.15.161:0/3520687196 conn(0x564e9457dda0 msgr2=0x564e945891a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 shutdown_connections 2023-07-14T11:25:57.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 1 -- 172.21.15.161:0/3520687196 wait complete. 2023-07-14T11:25:57.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.776+0000 7fee34d8db80 20 client.26378 trim_cache size 0 max 16384 2023-07-14T11:25:57.788 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:25:57.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fee040a12c0 msgr2=0x7fee040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fee040a12c0 0x7fee040a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee100097c0 tx=0x7fee10009750 comp rx=0 tx=0).stop 2023-07-14T11:25:57.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c105ae0 msgr2=0x7fee1c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:57.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c105ae0 0x7fee1c19c8e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee14009fd0 tx=0x7fee14007930 comp rx=0 tx=0).stop 2023-07-14T11:25:57.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 shutdown_connections 2023-07-14T11:25:57.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fee040a12c0 0x7fee040a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee1c0ffad0 0x7fee1c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee1c105ae0 0x7fee1c19c8e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 --2- 172.21.15.161:0/2758561092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fee1c100480 0x7fee1c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:57.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 >> 172.21.15.161:0/2758561092 conn(0x7fee1c0fb5f0 msgr2=0x7fee1c10af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:57.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 shutdown_connections 2023-07-14T11:25:57.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:57.777+0000 7fee213ff700 1 -- 172.21.15.161:0/2758561092 wait complete. 2023-07-14T11:25:57.798 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:25:58.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.051+0000 7f258f7ff700 1 -- 172.21.15.161:0/4186463531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25880fe770 msgr2=0x7f258810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.051+0000 7f258f7ff700 1 --2- 172.21.15.161:0/4186463531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25880fe770 0x7f258810bbf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f258000b0a0 tx=0x7f258000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.053+0000 7f258f7ff700 1 -- 172.21.15.161:0/4186463531 shutdown_connections 2023-07-14T11:25:58.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.053+0000 7f258f7ff700 1 --2- 172.21.15.161:0/4186463531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f258810c260 0x7f258810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.053+0000 7f258f7ff700 1 --2- 172.21.15.161:0/4186463531 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fde60 0x7f25880fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.053+0000 7f258f7ff700 1 --2- 172.21.15.161:0/4186463531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25880fe770 0x7f258810bbf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.053+0000 7f258f7ff700 1 -- 172.21.15.161:0/4186463531 >> 172.21.15.161:0/4186463531 conn(0x7f2588075b70 msgr2=0x7f2588075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:58.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.053+0000 7f258f7ff700 1 -- 172.21.15.161:0/4186463531 shutdown_connections 2023-07-14T11:25:58.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.054+0000 7f258f7ff700 1 -- 172.21.15.161:0/4186463531 wait complete. 2023-07-14T11:25:58.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.055+0000 7f258f7ff700 1 Processor -- start 2023-07-14T11:25:58.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.055+0000 7f258f7ff700 1 -- start start 2023-07-14T11:25:58.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.055+0000 7f258f7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25880fde60 0x7f258810ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.055+0000 7f258f7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fe770 0x7f2588070e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258f7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f258810c260 0x7f25880713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258f7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588071b40 con 0x7f258810c260 2023-07-14T11:25:58.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258f7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588071cb0 con 0x7f25880fde60 2023-07-14T11:25:58.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258f7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25881b11c0 con 0x7f25880fe770 2023-07-14T11:25:58.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fe770 0x7f2588070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258d59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25880fde60 0x7f258810ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258dd9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f258810c260 0x7f25880713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fe770 0x7f2588070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56430/0 (socket says 172.21.15.161:56430) 2023-07-14T11:25:58.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 -- 172.21.15.161:0/165840825 learned_addr learned my addr 172.21.15.161:0/165840825 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:58.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 -- 172.21.15.161:0/165840825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25880fde60 msgr2=0x7f258810ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25880fde60 0x7f258810ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 -- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f258810c260 msgr2=0x7f25880713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f258810c260 0x7f25880713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.056+0000 7f258cd9a700 1 -- 172.21.15.161:0/165840825 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580009d00 con 0x7f25880fe770 2023-07-14T11:25:58.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.057+0000 7f258cd9a700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fe770 0x7f2588070e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2580000f80 tx=0x7f2580007be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:58.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.057+0000 7f257e7fc700 1 -- 172.21.15.161:0/165840825 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000e040 con 0x7f25880fe770 2023-07-14T11:25:58.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.057+0000 7f257e7fc700 1 -- 172.21.15.161:0/165840825 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2580003e10 con 0x7f25880fe770 2023-07-14T11:25:58.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.057+0000 7f257e7fc700 1 -- 172.21.15.161:0/165840825 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258001d9d0 con 0x7f25880fe770 2023-07-14T11:25:58.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.057+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25881b13f0 con 0x7f25880fe770 2023-07-14T11:25:58.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.057+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881b18d0 con 0x7f25880fe770 2023-07-14T11:25:58.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.060+0000 7f257e7fc700 1 -- 172.21.15.161:0/165840825 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2580005630 con 0x7f25880fe770 2023-07-14T11:25:58.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 Processor -- start 2023-07-14T11:25:58.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f257e7fc700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f25740a1340 0x7f25740a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 -- start start 2023-07-14T11:25:58.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f257e7fc700 1 -- 172.21.15.161:0/165840825 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f25800ce3f0 con 0x7f25880fe770 2023-07-14T11:25:58.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5605943202d0 0x5605943206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560594320be0 0x56059432b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x56059432deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594226cb0 con 0x56059432bad0 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594279a40 con 0x560594320be0 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f258dd9c700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f25740a1340 0x7f25740a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.062+0000 7f25a3191b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594272ed0 con 0x5605943202d0 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5605943202d0 0x5605943206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258cd9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560594320be0 0x56059432b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5605943202d0 0x5605943206a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56436/0 (socket says 172.21.15.161:56436) 2023-07-14T11:25:58.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258cd9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560594320be0 0x56059432b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34496/0 (socket says 172.21.15.161:34496) 2023-07-14T11:25:58.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 -- 172.21.15.161:0/1881121252 learned_addr learned my addr 172.21.15.161:0/1881121252 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:58.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258dd9c700 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x56059432deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 -- 172.21.15.161:0/1881121252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560594320be0 msgr2=0x56059432b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560594320be0 0x56059432b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 -- 172.21.15.161:0/1881121252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 msgr2=0x56059432deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x56059432deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 -- 172.21.15.161:0/1881121252 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605940a0de0 con 0x5605943202d0 2023-07-14T11:25:58.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258dd9c700 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x56059432deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:25:58.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258cd9a700 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560594320be0 0x56059432b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:58.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258dd9c700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f25740a1340 0x7f25740a37f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f258810c100 tx=0x7f2584009e90 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:58.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f258d59b700 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5605943202d0 0x5605943206a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2578009ad0 tx=0x7f2578009de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad959a05bb46a75 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:58.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.063+0000 7f25737fe700 1 -- 172.21.15.161:0/1881121252 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780095d0 con 0x5605943202d0 2023-07-14T11:25:58.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25737fe700 1 -- 172.21.15.161:0/1881121252 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257800ed30 con 0x5605943202d0 2023-07-14T11:25:58.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25737fe700 1 -- 172.21.15.161:0/1881121252 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578013e50 con 0x5605943202d0 2023-07-14T11:25:58.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 -- 172.21.15.161:0/1881121252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5605943202d0 msgr2=0x5605943206a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5605943202d0 0x5605943206a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2578009ad0 tx=0x7f2578009de0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 -- 172.21.15.161:0/1881121252 shutdown_connections 2023-07-14T11:25:58.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x56059432deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5605943202d0 0x5605943206a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 --2- 172.21.15.161:0/1881121252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560594320be0 0x56059432b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 -- 172.21.15.161:0/1881121252 >> 172.21.15.161:0/1881121252 conn(0x560594316da0 msgr2=0x56059431c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.064+0000 7f25a3191b80 1 -- 172.21.15.161:0/1881121252 shutdown_connections 2023-07-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.065+0000 7f25a3191b80 1 -- 172.21.15.161:0/1881121252 wait complete. 2023-07-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.065+0000 7f25a3191b80 1 -- start start 2023-07-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.065+0000 7f25a3191b80 1 Processor -- start 2023-07-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x56059432b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594328710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x560594328e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f25a3191b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594212ac0 con 0x56059432bad0 2023-07-14T11:25:58.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f25a3191b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605942127c0 con 0x5605943202d0 2023-07-14T11:25:58.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f25a3191b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605940f6920 con 0x560594320be0 2023-07-14T11:25:58.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f25a3191b80 10 client.0 _ll_register_callbacks cb 0x560594210760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:58.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.066+0000 7f258dd9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x560594328e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x560594328e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38164/0 (socket says 172.21.15.161:38164) 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 -- 172.21.15.161:0/3041152615 learned_addr learned my addr 172.21.15.161:0/3041152615 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258d59b700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x56059432b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 -- 172.21.15.161:0/3041152615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 msgr2=0x560594328710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258cd9a700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594328710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594328710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 -- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 msgr2=0x56059432b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x56059432b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 -- 172.21.15.161:0/3041152615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881b18d0 con 0x56059432bad0 2023-07-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258d59b700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x56059432b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258cd9a700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594328710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f258dd9c700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x560594328e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2584015460 tx=0x7f2584004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0b3578819b73f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f2570ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.067+0000 7f2570ff9700 1 -- 172.21.15.161:0/3041152615 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584027070 con 0x56059432bad0 2023-07-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605941cc4f0 con 0x56059432bad0 2023-07-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f2570ff9700 1 -- 172.21.15.161:0/3041152615 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f2584004020 con 0x56059432bad0 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f2570ff9700 1 -- 172.21.15.161:0/3041152615 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2584004220 con 0x56059432bad0 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 10 client.16344 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f2578009730 con 0x56059432bad0 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f2570ff9700 1 -- 172.21.15.161:0/3041152615 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605941cc4f0 con 0x56059432bad0 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f2570ff9700 1 -- 172.21.15.161:0/3041152615 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f258402a020 con 0x56059432bad0 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 10 client.16344 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 20 client.16344 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56059433b350 0x56059433d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 4 client.16344 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.068+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56059426beb0 con 0x56059433b350 2023-07-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.069+0000 7f258d59b700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56059433b350 0x56059433d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.069+0000 7f258d59b700 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56059433b350 0x56059433d760 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f07060c882065c39 server_cookie=c6018e858864deaf in_seq=0 out_seq=0 2023-07-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.069+0000 7f2570ff9700 10 client.16344 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f2570ff9700 1 -- 172.21.15.161:0/3041152615 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56059426beb0 con 0x56059433b350 2023-07-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f2570ff9700 10 client.16344 handle_command_reply: tid=0 2023-07-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 client.16344 shutdown 2023-07-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 msgr2=0x560594328e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x560594328e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2584015460 tx=0x7f2584004790 comp rx=0 tx=0).stop 2023-07-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 shutdown_connections 2023-07-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56059433b350 0x56059433d760 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x560594328e50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x56059432b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f2570ff9700 0 client.16344 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3041152615 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594328710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 >> 172.21.15.161:0/3041152615 conn(0x560594316da0 msgr2=0x56059431c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 shutdown_connections 2023-07-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.070+0000 7f25a3191b80 1 -- 172.21.15.161:0/3041152615 wait complete. 2023-07-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.071+0000 7f25a3191b80 20 client.16344 trim_cache size 0 max 16384 2023-07-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 Processor -- start 2023-07-14T11:25:58.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 -- start start 2023-07-14T11:25:58.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605943202d0 0x560594327fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594337760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059432bad0 0x56059432b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605943284f0 con 0x5605943202d0 2023-07-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594337ca0 con 0x56059432bad0 2023-07-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.085+0000 7f25a3191b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594337e10 con 0x560594320be0 2023-07-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594337760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258dd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059432bad0 0x56059432b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594337760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56448/0 (socket says 172.21.15.161:56448) 2023-07-14T11:25:58.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258dd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059432bad0 0x56059432b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34506/0 (socket says 172.21.15.161:34506) 2023-07-14T11:25:58.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 -- 172.21.15.161:0/225196110 learned_addr learned my addr 172.21.15.161:0/225196110 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:58.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258d59b700 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605943202d0 0x560594327fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 -- 172.21.15.161:0/225196110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059432bad0 msgr2=0x56059432b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059432bad0 0x56059432b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 -- 172.21.15.161:0/225196110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605943202d0 msgr2=0x560594327fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605943202d0 0x560594327fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 -- 172.21.15.161:0/225196110 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560594089c60 con 0x560594320be0 2023-07-14T11:25:58.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.086+0000 7f258cd9a700 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594337760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2580097850 tx=0x7f2580009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf7add0078e7b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.087+0000 7f2573fff700 1 -- 172.21.15.161:0/225196110 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258009f380 con 0x560594320be0 2023-07-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.087+0000 7f2573fff700 1 -- 172.21.15.161:0/225196110 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25800a3710 con 0x560594320be0 2023-07-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.087+0000 7f2573fff700 1 -- 172.21.15.161:0/225196110 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580097a60 con 0x560594320be0 2023-07-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 -- 172.21.15.161:0/225196110 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 msgr2=0x560594337760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594337760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2580097850 tx=0x7f2580009040 comp rx=0 tx=0).stop 2023-07-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 -- 172.21.15.161:0/225196110 shutdown_connections 2023-07-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605943202d0 0x560594327fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059432bad0 0x56059432b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 --2- 172.21.15.161:0/225196110 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594337760 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 -- 172.21.15.161:0/225196110 >> 172.21.15.161:0/225196110 conn(0x560594316da0 msgr2=0x5605943420c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:58.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 -- 172.21.15.161:0/225196110 shutdown_connections 2023-07-14T11:25:58.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 -- 172.21.15.161:0/225196110 wait complete. 2023-07-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.088+0000 7f25a3191b80 1 -- start start 2023-07-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.089+0000 7f25a3191b80 1 Processor -- start 2023-07-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x560594330410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594330950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f25a3191b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x5605943881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f25a3191b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594388750 con 0x56059432bad0 2023-07-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f25a3191b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605943888c0 con 0x5605943202d0 2023-07-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f25a3191b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560594388a80 con 0x560594320be0 2023-07-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f25a3191b80 10 client.0 _ll_register_callbacks cb 0x7f2584012a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x560594330410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x560594330410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34510/0 (socket says 172.21.15.161:34510) 2023-07-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 -- 172.21.15.161:0/3072509988 learned_addr learned my addr 172.21.15.161:0/3072509988 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258dd9c700 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x5605943881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258cd9a700 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594330950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 -- 172.21.15.161:0/3072509988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 msgr2=0x560594330950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594330950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 -- 172.21.15.161:0/3072509988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 msgr2=0x5605943881e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x5605943881e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.090+0000 7f258d59b700 1 -- 172.21.15.161:0/3072509988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881b13f0 con 0x5605943202d0 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.091+0000 7f258d59b700 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x560594330410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2578006a90 tx=0x7f2578009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a04e805c826e28c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.091+0000 7f2570ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.091+0000 7f2570ff9700 1 -- 172.21.15.161:0/3072509988 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578021040 con 0x5605943202d0 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.091+0000 7f2570ff9700 1 -- 172.21.15.161:0/3072509988 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f257800eaf0 con 0x5605943202d0 2023-07-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.091+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f257800ed30 con 0x5605943202d0 2023-07-14T11:25:58.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f2570ff9700 1 -- 172.21.15.161:0/3072509988 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2578013a60 con 0x5605943202d0 2023-07-14T11:25:58.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 10 client.26393 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:58.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5605940c7250 con 0x5605943202d0 2023-07-14T11:25:58.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f2570ff9700 1 -- 172.21.15.161:0/3072509988 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257800ed30 con 0x5605943202d0 2023-07-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f2570ff9700 1 -- 172.21.15.161:0/3072509988 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2578009b00 con 0x5605943202d0 2023-07-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 10 client.26393 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 20 client.26393 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560594390230 0x560594392640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 4 client.26393 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56059422e050 con 0x560594390230 2023-07-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.092+0000 7f258cd9a700 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560594390230 0x560594392640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f258cd9a700 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560594390230 0x560594392640 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacb13f4c9eda526 server_cookie=cdb5ade46e83950c in_seq=0 out_seq=0 2023-07-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f2570ff9700 10 client.26393 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f2570ff9700 1 -- 172.21.15.161:0/3072509988 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56059422e050 con 0x560594390230 2023-07-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f2570ff9700 10 client.26393 handle_command_reply: tid=0 2023-07-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 client.26393 shutdown 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 msgr2=0x560594330410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x560594330410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2578006a90 tx=0x7f2578009320 comp rx=0 tx=0).stop 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 shutdown_connections 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560594390230 0x560594392640 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059432bad0 0x5605943881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5605943202d0 0x560594330410 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 --2- 172.21.15.161:0/3072509988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560594320be0 0x560594330950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f2570ff9700 0 client.26393 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:58.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.093+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 >> 172.21.15.161:0/3072509988 conn(0x560594316da0 msgr2=0x5605943239f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:58.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 shutdown_connections 2023-07-14T11:25:58.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f25a3191b80 1 -- 172.21.15.161:0/3072509988 wait complete. 2023-07-14T11:25:58.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f25a3191b80 20 client.26393 trim_cache size 0 max 16384 2023-07-14T11:25:58.103 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:25:58.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f25740a1340 msgr2=0x7f25740a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f258f7ff700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f25740a1340 0x7f25740a37f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f258810c100 tx=0x7f2584009e90 comp rx=0 tx=0).stop 2023-07-14T11:25:58.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fe770 msgr2=0x7f2588070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:58.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f258f7ff700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fe770 0x7f2588070e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2580000f80 tx=0x7f2580007be0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.094+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 shutdown_connections 2023-07-14T11:25:58.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.095+0000 7f258f7ff700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f25740a1340 0x7f25740a37f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.095+0000 7f258f7ff700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f258810c260 0x7f25880713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.095+0000 7f258f7ff700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25880fde60 0x7f258810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.095+0000 7f258f7ff700 1 --2- 172.21.15.161:0/165840825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f25880fe770 0x7f2588070e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:58.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.095+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 >> 172.21.15.161:0/165840825 conn(0x7f2588075b70 msgr2=0x7f25880fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:58.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.095+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 shutdown_connections 2023-07-14T11:25:58.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:58.095+0000 7f258f7ff700 1 -- 172.21.15.161:0/165840825 wait complete. 2023-07-14T11:25:58.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 301883 9.91 MB/sec execute 40 sec latency 52.089 ms 2023-07-14T11:25:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:58 smithi175 ceph-mon[129081]: pgmap v114: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-07-14T11:25:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:25:58 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1786219216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:25:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:58 smithi182 ceph-mon[129366]: pgmap v114: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-07-14T11:25:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:25:58 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1786219216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:25:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:58 smithi161 ceph-mon[95526]: pgmap v114: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-07-14T11:25:58.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:25:58 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1786219216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:25:59.117 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:25:59.118 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:25:59.173 INFO:tasks.workunit.client.0.smithi161.stdout: 1 303782 9.81 MB/sec execute 41 sec latency 23.648 ms 2023-07-14T11:25:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 -- 172.21.15.161:0/2536572855 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce0101610 msgr2=0x7f0ce0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/2536572855 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce0101610 0x7f0ce0109b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009990 tx=0x7f0cd4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 -- 172.21.15.161:0/2536572855 shutdown_connections 2023-07-14T11:25:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/2536572855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce010a040 0x7f0ce010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/2536572855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 0x7f0ce01010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/2536572855 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce0101610 0x7f0ce0109b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 -- 172.21.15.161:0/2536572855 >> 172.21.15.161:0/2536572855 conn(0x7f0ce00fa4b0 msgr2=0x7f0ce00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 -- 172.21.15.161:0/2536572855 shutdown_connections 2023-07-14T11:25:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.337+0000 7f0ce57ff700 1 -- 172.21.15.161:0/2536572855 wait complete. 2023-07-14T11:25:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.338+0000 7f0ce57ff700 1 Processor -- start 2023-07-14T11:25:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.338+0000 7f0ce57ff700 1 -- start start 2023-07-14T11:25:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.338+0000 7f0ce57ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 0x7f0ce0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0ce57ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce0101610 0x7f0ce0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0ce57ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce010a040 0x7f0ce019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0ce57ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019ce30 con 0x7f0ce0101610 2023-07-14T11:25:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0ce57ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019cfa0 con 0x7f0ce0100d00 2023-07-14T11:25:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0ce57ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019d160 con 0x7f0ce010a040 2023-07-14T11:25:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 0x7f0ce0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 0x7f0ce0197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34520/0 (socket says 172.21.15.161:34520) 2023-07-14T11:25:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 -- 172.21.15.161:0/1272618744 learned_addr learned my addr 172.21.15.161:0/1272618744 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce010a040 0x7f0ce019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cde7fc700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce0101610 0x7f0ce0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 -- 172.21.15.161:0/1272618744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce010a040 msgr2=0x7f0ce019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce010a040 0x7f0ce019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 -- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce0101610 msgr2=0x7f0ce0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce0101610 0x7f0ce0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 -- 172.21.15.161:0/1272618744 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40095f0 con 0x7f0ce0100d00 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.339+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 0x7f0ce0197e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000ef10 tx=0x7f0cd000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.340+0000 7f0cc7fff700 1 -- 172.21.15.161:0/1272618744 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000c9c0 con 0x7f0ce0100d00 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.340+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce019d3e0 con 0x7f0ce0100d00 2023-07-14T11:25:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.340+0000 7f0cc7fff700 1 -- 172.21.15.161:0/1272618744 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd000f040 con 0x7f0ce0100d00 2023-07-14T11:25:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.340+0000 7f0cc7fff700 1 -- 172.21.15.161:0/1272618744 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0014480 con 0x7f0ce0100d00 2023-07-14T11:25:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.340+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce01a4c80 con 0x7f0ce0100d00 2023-07-14T11:25:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.343+0000 7f0cc7fff700 1 -- 172.21.15.161:0/1272618744 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0cd0007690 con 0x7f0ce0100d00 2023-07-14T11:25:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 Processor -- start 2023-07-14T11:25:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 -- start start 2023-07-14T11:25:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cc7fff700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0cc80a12c0 0x7f0cc80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cc7fff700 1 -- 172.21.15.161:0/1272618744 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f0cd00cd310 con 0x7f0ce0100d00 2023-07-14T11:25:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042350 0x55fce4042720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042c60 0x55fce404d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce3f48d30 con 0x55fce404db50 2023-07-14T11:25:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce3f9bac0 con 0x55fce4042c60 2023-07-14T11:25:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.345+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce3f94f50 con 0x55fce4042350 2023-07-14T11:25:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042350 0x55fce4042720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042c60 0x55fce404d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042350 0x55fce4042720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56468/0 (socket says 172.21.15.161:56468) 2023-07-14T11:25:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 -- 172.21.15.161:0/2238837156 learned_addr learned my addr 172.21.15.161:0/2238837156 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0cc80a12c0 0x7f0cc80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 -- 172.21.15.161:0/2238837156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042c60 msgr2=0x55fce404d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042c60 0x55fce404d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 -- 172.21.15.161:0/2238837156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 msgr2=0x55fce404ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 -- 172.21.15.161:0/2238837156 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce3dc2e60 con 0x55fce4042350 2023-07-14T11:25:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042350 0x55fce4042720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00a1fd0 tx=0x7f0cd0096520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca371a77a67c2cf4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0cc80a12c0 0x7f0cc80a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097a0 tx=0x7f0cd8009990 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cc57fa700 1 -- 172.21.15.161:0/2238837156 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00112c0 con 0x55fce4042350 2023-07-14T11:25:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cc57fa700 1 -- 172.21.15.161:0/2238837156 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd009a0b0 con 0x55fce4042350 2023-07-14T11:25:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.346+0000 7f0cc57fa700 1 -- 172.21.15.161:0/2238837156 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00995a0 con 0x55fce4042350 2023-07-14T11:25:59.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2238837156 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042350 msgr2=0x55fce4042720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042350 0x55fce4042720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00a1fd0 tx=0x7f0cd0096520 comp rx=0 tx=0).stop 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2238837156 shutdown_connections 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042350 0x55fce4042720 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2238837156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042c60 0x55fce404d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2238837156 >> 172.21.15.161:0/2238837156 conn(0x55fce4038e20 msgr2=0x55fce403e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2238837156 shutdown_connections 2023-07-14T11:25:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.347+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2238837156 wait complete. 2023-07-14T11:25:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.348+0000 7f0cf906fb80 1 -- start start 2023-07-14T11:25:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.348+0000 7f0cf906fb80 1 Processor -- start 2023-07-14T11:25:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce404a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce404ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce3f34b40 con 0x55fce404db50 2023-07-14T11:25:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce3f34840 con 0x55fce4042350 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce3e189a0 con 0x55fce4042c60 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cf906fb80 10 client.0 _ll_register_callbacks cb 0x55fce3f327e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce404a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce404ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce404a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34546/0 (socket says 172.21.15.161:34546) 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cdeffd700 1 -- 172.21.15.161:0/2293426506 learned_addr learned my addr 172.21.15.161:0/2293426506 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cde7fc700 1 -- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 msgr2=0x55fce404a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cde7fc700 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce404a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cde7fc700 1 -- 172.21.15.161:0/2293426506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 msgr2=0x55fce404b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cde7fc700 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cde7fc700 1 -- 172.21.15.161:0/2293426506 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce01a4c80 con 0x55fce4042c60 2023-07-14T11:25:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.349+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce404a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cde7fc700 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce404ade0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400b2e0 tx=0x7f0cd4009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a117f4a5d344434 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2293426506 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd401d030 con 0x55fce4042c60 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fce3eee570 con 0x55fce4042c60 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2293426506 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f0cd4004500 con 0x55fce4042c60 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2293426506 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd40046c0 con 0x55fce4042c60 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cf906fb80 10 client.26410 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.350+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fce3dabce0 con 0x55fce4042c60 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2293426506 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fce3eee570 con 0x55fce4042c60 2023-07-14T11:25:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2293426506 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0cd400f8a0 con 0x55fce4042c60 2023-07-14T11:25:59.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cf906fb80 10 client.26410 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:59.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cf906fb80 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:59.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cf906fb80 20 client.26410 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:59.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce405d580 0x55fce405f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cf906fb80 4 client.26410 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:59.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fce3f8df30 con 0x55fce405d580 2023-07-14T11:25:59.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.351+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce405d580 0x55fce405f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.352+0000 7f0cc2ff5700 10 client.26410 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.352+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce405d580 0x55fce405f990 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90cc98420d102926 server_cookie=ad333ee049ff8b99 in_seq=0 out_seq=0 2023-07-14T11:25:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.353+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2293426506 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fce3f8df30 con 0x55fce405d580 2023-07-14T11:25:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.353+0000 7f0cc2ff5700 10 client.26410 handle_command_reply: tid=0 2023-07-14T11:25:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.353+0000 7f0cf906fb80 1 client.26410 shutdown 2023-07-14T11:25:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.353+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 msgr2=0x55fce404ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.353+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce404ade0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400b2e0 tx=0x7f0cd4009570 comp rx=0 tx=0).stop 2023-07-14T11:25:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 shutdown_connections 2023-07-14T11:25:59.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce405d580 0x55fce405f990 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce404db50 0x55fce404b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce404a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2293426506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce404ade0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cc2ff5700 0 client.26410 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 >> 172.21.15.161:0/2293426506 conn(0x55fce4038e20 msgr2=0x55fce403e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 shutdown_connections 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2293426506 wait complete. 2023-07-14T11:25:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.354+0000 7f0cf906fb80 20 client.26410 trim_cache size 0 max 16384 2023-07-14T11:25:59.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.372+0000 7f0cf906fb80 1 Processor -- start 2023-07-14T11:25:59.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cf906fb80 1 -- start start 2023-07-14T11:25:59.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042350 0x55fce404b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce4047b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce404db50 0x55fce4048050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce4044220 con 0x55fce4042350 2023-07-14T11:25:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce403f3b0 con 0x55fce404db50 2023-07-14T11:25:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce4046440 con 0x55fce4042c60 2023-07-14T11:25:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042350 0x55fce404b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce404db50 0x55fce4048050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce404db50 0x55fce4048050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34552/0 (socket says 172.21.15.161:34552) 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042350 0x55fce404b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38214/0 (socket says 172.21.15.161:38214) 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/3618726343 learned_addr learned my addr 172.21.15.161:0/3618726343 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.373+0000 7f0cde7fc700 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce4047b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/3618726343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 msgr2=0x55fce4047b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce4047b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/3618726343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042350 msgr2=0x55fce404b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042350 0x55fce404b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/3618726343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce3de92d0 con 0x55fce404db50 2023-07-14T11:25:59.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce404db50 0x55fce4048050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80132d0 tx=0x7f0cd8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a71b2f5d3740713 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cc5ffb700 1 -- 172.21.15.161:0/3618726343 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8012b40 con 0x55fce404db50 2023-07-14T11:25:59.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cc5ffb700 1 -- 172.21.15.161:0/3618726343 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd8012ca0 con 0x55fce404db50 2023-07-14T11:25:59.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.374+0000 7f0cc5ffb700 1 -- 172.21.15.161:0/3618726343 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80116b0 con 0x55fce404db50 2023-07-14T11:25:59.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 -- 172.21.15.161:0/3618726343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce404db50 msgr2=0x55fce4048050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce404db50 0x55fce4048050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80132d0 tx=0x7f0cd8009260 comp rx=0 tx=0).stop 2023-07-14T11:25:59.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 -- 172.21.15.161:0/3618726343 shutdown_connections 2023-07-14T11:25:59.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042350 0x55fce404b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce404db50 0x55fce4048050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/3618726343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce4042c60 0x55fce4047b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 -- 172.21.15.161:0/3618726343 >> 172.21.15.161:0/3618726343 conn(0x55fce4038e20 msgr2=0x55fce403e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 -- 172.21.15.161:0/3618726343 shutdown_connections 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.375+0000 7f0cf906fb80 1 -- 172.21.15.161:0/3618726343 wait complete. 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.376+0000 7f0cf906fb80 1 -- start start 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.376+0000 7f0cf906fb80 1 Processor -- start 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce4052fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042c60 0x55fce4053510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cf906fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce404db50 0x55fce40b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce4053de0 con 0x55fce4042c60 2023-07-14T11:25:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce40b1990 con 0x55fce4042350 2023-07-14T11:25:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cf906fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce40b1ad0 con 0x55fce404db50 2023-07-14T11:25:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cf906fb80 10 client.0 _ll_register_callbacks cb 0x7f0cd400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce404db50 0x55fce40b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce404db50 0x55fce40b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56498/0 (socket says 172.21.15.161:56498) 2023-07-14T11:25:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/2547938051 learned_addr learned my addr 172.21.15.161:0/2547938051 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce4052fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.377+0000 7f0cde7fc700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042c60 0x55fce4053510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 msgr2=0x55fce4052fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce4052fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/2547938051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042c60 msgr2=0x55fce4053510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042c60 0x55fce4053510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cdf7fe700 1 -- 172.21.15.161:0/2547938051 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd8013040 con 0x55fce404db50 2023-07-14T11:25:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce4052fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cdf7fe700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce404db50 0x55fce40b1450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8012a10 tx=0x7f0cd802aac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e66e5f1d1827e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2547938051 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd802f070 con 0x55fce404db50 2023-07-14T11:25:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2547938051 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f0cd8012e10 con 0x55fce404db50 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cd009a0b0 con 0x55fce404db50 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2547938051 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd8012b40 con 0x55fce404db50 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.378+0000 7f0cf906fb80 10 client.26416 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fce3dc1330 con 0x55fce404db50 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2547938051 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd009a0b0 con 0x55fce404db50 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2547938051 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0cd8034420 con 0x55fce404db50 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cf906fb80 10 client.26416 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cf906fb80 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cf906fb80 20 client.26416 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce40b31c0 0x55fce40b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cf906fb80 4 client.26416 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:25:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fce3f500d0 con 0x55fce40b31c0 2023-07-14T11:25:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.379+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce40b31c0 0x55fce40b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.380+0000 7f0cdeffd700 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce40b31c0 0x55fce40b55d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad5acdc593a937b server_cookie=768c779325cc8b56 in_seq=0 out_seq=0 2023-07-14T11:25:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.380+0000 7f0cc2ff5700 10 client.26416 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cc2ff5700 1 -- 172.21.15.161:0/2547938051 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fce3f500d0 con 0x55fce40b31c0 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cc2ff5700 10 client.26416 handle_command_reply: tid=0 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 client.26416 shutdown 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce404db50 msgr2=0x55fce40b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce404db50 0x55fce40b1450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8012a10 tx=0x7f0cd802aac0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 shutdown_connections 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fce40b31c0 0x55fce40b55d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fce404db50 0x55fce40b1450 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fce4042350 0x55fce4052fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 --2- 172.21.15.161:0/2547938051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fce4042c60 0x55fce4053510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cc2ff5700 0 client.26416 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 >> 172.21.15.161:0/2547938051 conn(0x55fce4038e20 msgr2=0x55fce4044460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 shutdown_connections 2023-07-14T11:25:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.382+0000 7f0cf906fb80 1 -- 172.21.15.161:0/2547938051 wait complete. 2023-07-14T11:25:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0cf906fb80 20 client.26416 trim_cache size 0 max 16384 2023-07-14T11:25:59.391 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:25:59.391 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:25:59.391 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "bfd99564-b789-41c5-852d-3e47e013eded", 2023-07-14T11:25:59.391 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:25:59.391 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:25:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0cc80a12c0 msgr2=0x7f0cc80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0cc80a12c0 0x7f0cc80a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097a0 tx=0x7f0cd8009990 comp rx=0 tx=0).stop 2023-07-14T11:25:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 msgr2=0x7f0ce0197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 0x7f0ce0197e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000ef10 tx=0x7f0cd000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 shutdown_connections 2023-07-14T11:25:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0cc80a12c0 0x7f0cc80a3770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ce010a040 0x7f0ce019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce0100d00 0x7f0ce0197e50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 --2- 172.21.15.161:0/1272618744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce0101610 0x7f0ce0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.383+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 >> 172.21.15.161:0/1272618744 conn(0x7f0ce00fa4b0 msgr2=0x7f0ce010c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.384+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 shutdown_connections 2023-07-14T11:25:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.384+0000 7f0ce57ff700 1 -- 172.21.15.161:0/1272618744 wait complete. 2023-07-14T11:25:59.407 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:25:59.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 -- 172.21.15.161:0/2490205098 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 msgr2=0x7ff7bc0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/2490205098 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 0x7ff7bc0fff70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098f0 tx=0x7ff7ac009c00 comp rx=0 tx=0).stop 2023-07-14T11:25:59.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 -- 172.21.15.161:0/2490205098 shutdown_connections 2023-07-14T11:25:59.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/2490205098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc1004b0 0x7ff7bc10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/2490205098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc105b10 0x7ff7bc105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/2490205098 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 0x7ff7bc0fff70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 -- 172.21.15.161:0/2490205098 >> 172.21.15.161:0/2490205098 conn(0x7ff7bc0fb600 msgr2=0x7ff7bc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 -- 172.21.15.161:0/2490205098 shutdown_connections 2023-07-14T11:25:59.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.621+0000 7ff7c37ff700 1 -- 172.21.15.161:0/2490205098 wait complete. 2023-07-14T11:25:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.622+0000 7ff7c37ff700 1 Processor -- start 2023-07-14T11:25:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c37ff700 1 -- start start 2023-07-14T11:25:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c37ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 0x7ff7bc197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c37ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc1004b0 0x7ff7bc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c37ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc105b10 0x7ff7bc19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c37ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc19ced0 con 0x7ff7bc105b10 2023-07-14T11:25:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c37ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc19d040 con 0x7ff7bc1004b0 2023-07-14T11:25:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c37ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc19d200 con 0x7ff7bc0ffb00 2023-07-14T11:25:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc1004b0 0x7ff7bc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c159b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 0x7ff7bc197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc1004b0 0x7ff7bc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34588/0 (socket says 172.21.15.161:34588) 2023-07-14T11:25:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/4020889038 learned_addr learned my addr 172.21.15.161:0/4020889038 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.623+0000 7ff7c1d9c700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc105b10 0x7ff7bc19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/4020889038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 msgr2=0x7ff7bc197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 0x7ff7bc197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc105b10 msgr2=0x7ff7bc19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc105b10 0x7ff7bc19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/4020889038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80095f0 con 0x7ff7bc1004b0 2023-07-14T11:25:59.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c159b700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 0x7ff7bc197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:59.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc1004b0 0x7ff7bc198430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009810 tx=0x7ff7ac00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7b27fc700 1 -- 172.21.15.161:0/4020889038 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac01c070 con 0x7ff7bc1004b0 2023-07-14T11:25:59.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac009520 con 0x7ff7bc1004b0 2023-07-14T11:25:59.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7b27fc700 1 -- 172.21.15.161:0/4020889038 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ac00bb20 con 0x7ff7bc1004b0 2023-07-14T11:25:59.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7b27fc700 1 -- 172.21.15.161:0/4020889038 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004930 con 0x7ff7bc1004b0 2023-07-14T11:25:59.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.624+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc1a4b40 con 0x7ff7bc1004b0 2023-07-14T11:25:59.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.628+0000 7ff7b27fc700 1 -- 172.21.15.161:0/4020889038 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff7ac020070 con 0x7ff7bc1004b0 2023-07-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.629+0000 7ff7d70c5b80 1 Processor -- start 2023-07-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7d70c5b80 1 -- start start 2023-07-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f72d0 0x5617b63f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7b27fc700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff7a80a12c0 0x7ff7a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b6402590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6404eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b62fdcb0 con 0x5617b6402ad0 2023-07-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b6350a40 con 0x5617b63f72d0 2023-07-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b6349ed0 con 0x5617b63f7be0 2023-07-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7b27fc700 1 -- 172.21.15.161:0/4020889038 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7ff7ac0cdd60 con 0x7ff7bc1004b0 2023-07-14T11:25:59.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7c1d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f72d0 0x5617b63f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7c1d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f72d0 0x5617b63f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34592/0 (socket says 172.21.15.161:34592) 2023-07-14T11:25:59.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7c1d9c700 1 -- 172.21.15.161:0/3641183469 learned_addr learned my addr 172.21.15.161:0/3641183469 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7c1d9c700 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6404eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7c159b700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff7a80a12c0 0x7ff7a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.630+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b6402590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c1d9c700 1 -- 172.21.15.161:0/3641183469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 msgr2=0x5617b6402590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c1d9c700 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b6402590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c1d9c700 1 -- 172.21.15.161:0/3641183469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 msgr2=0x5617b6404eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c1d9c700 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6404eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c1d9c700 1 -- 172.21.15.161:0/3641183469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617b6177de0 con 0x5617b63f72d0 2023-07-14T11:25:59.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b6402590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:25:59.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c1d9c700 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f72d0 0x5617b63f76a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4008cd0 tx=0x7ff7b4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57e09f2d589b522 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7c159b700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff7a80a12c0 0x7ff7a80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005fd0 tx=0x7ff7b8009310 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7a77fe700 1 -- 172.21.15.161:0/3641183469 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b401f070 con 0x5617b63f72d0 2023-07-14T11:25:59.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7a77fe700 1 -- 172.21.15.161:0/3641183469 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b400f040 con 0x5617b63f72d0 2023-07-14T11:25:59.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.631+0000 7ff7a77fe700 1 -- 172.21.15.161:0/3641183469 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400e050 con 0x5617b63f72d0 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/3641183469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f72d0 msgr2=0x5617b63f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f72d0 0x5617b63f76a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4008cd0 tx=0x7ff7b4009920 comp rx=0 tx=0).stop 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/3641183469 shutdown_connections 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6404eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f72d0 0x5617b63f76a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/3641183469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b6402590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/3641183469 >> 172.21.15.161:0/3641183469 conn(0x5617b63edda0 msgr2=0x5617b63f36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/3641183469 shutdown_connections 2023-07-14T11:25:59.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.632+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/3641183469 wait complete. 2023-07-14T11:25:59.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.633+0000 7ff7d70c5b80 1 -- start start 2023-07-14T11:25:59.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.633+0000 7ff7d70c5b80 1 Processor -- start 2023-07-14T11:25:59.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.633+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b6402500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b62e9ac0 con 0x5617b63f72d0 2023-07-14T11:25:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b62e97c0 con 0x5617b6402ad0 2023-07-14T11:25:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b61cd920 con 0x5617b63f7be0 2023-07-14T11:25:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7d70c5b80 10 client.0 _ll_register_callbacks cb 0x5617b62e7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:59.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56534/0 (socket says 172.21.15.161:56534) 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/472728165 learned_addr learned my addr 172.21.15.161:0/472728165 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c1d9c700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c159b700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b6402500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 msgr2=0x5617b63ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/472728165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 msgr2=0x5617b6402500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b6402500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/472728165 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc1a4b40 con 0x5617b63f7be0 2023-07-14T11:25:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c159b700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b6402500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:25:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63ff780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0975a0 tx=0x7ff7ac09a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46b7c6eefbed1d39 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/472728165 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0bf650 con 0x5617b63f7be0 2023-07-14T11:25:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617b62a34f0 con 0x5617b63f7be0 2023-07-14T11:25:59.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.634+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/472728165 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7ff7ac09eb60 con 0x5617b63f7be0 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/472728165 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ac0a2ef0 con 0x5617b63f7be0 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 10 client.26428 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff7b40089b0 con 0x5617b63f7be0 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/472728165 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617b62a34f0 con 0x5617b63f7be0 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/472728165 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff7ac09a020 con 0x5617b63f7be0 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 10 client.26428 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:59.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 20 client.26428 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6411ff0 0x5617b6414400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 4 client.26428 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617b6342eb0 con 0x5617b6411ff0 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.635+0000 7ff7c159b700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6411ff0 0x5617b6414400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.636+0000 7ff7a4ff9700 10 client.26428 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.636+0000 7ff7c159b700 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6411ff0 0x5617b6414400 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0c00eb69f21c34 server_cookie=2dbc652ca03cfee in_seq=0 out_seq=0 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/472728165 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5617b6342eb0 con 0x5617b6411ff0 2023-07-14T11:25:59.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7a4ff9700 10 client.26428 handle_command_reply: tid=0 2023-07-14T11:25:59.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 client.26428 shutdown 2023-07-14T11:25:59.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 msgr2=0x5617b63ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63ff780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0975a0 tx=0x7ff7ac09a510 comp rx=0 tx=0).stop 2023-07-14T11:25:59.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 shutdown_connections 2023-07-14T11:25:59.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6411ff0 0x5617b6414400 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b6402500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7a4ff9700 0 client.26428 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/472728165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63ff780 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.637+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 >> 172.21.15.161:0/472728165 conn(0x5617b63edda0 msgr2=0x5617b63f3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.638+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 shutdown_connections 2023-07-14T11:25:59.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.638+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/472728165 wait complete. 2023-07-14T11:25:59.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.638+0000 7ff7d70c5b80 20 client.26428 trim_cache size 0 max 16384 2023-07-14T11:25:59.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.656+0000 7ff7d70c5b80 1 Processor -- start 2023-07-14T11:25:59.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.656+0000 7ff7d70c5b80 1 -- start start 2023-07-14T11:25:59.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b64094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63faa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b63efec0 con 0x5617b63f72d0 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b63f8540 con 0x5617b6402ad0 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b63f4d00 con 0x5617b63f7be0 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b64094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b64094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38274/0 (socket says 172.21.15.161:38274) 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 -- 172.21.15.161:0/2829032000 learned_addr learned my addr 172.21.15.161:0/2829032000 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c1d9c700 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 -- 172.21.15.161:0/2829032000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 msgr2=0x5617b63f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 -- 172.21.15.161:0/2829032000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 msgr2=0x5617b63faa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63faa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.657+0000 7ff7c159b700 1 -- 172.21.15.161:0/2829032000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617b6160c60 con 0x5617b63f72d0 2023-07-14T11:25:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.658+0000 7ff7c159b700 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b64094b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5617b6414c70 tx=0x7ff7b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9ab8fb1d64f7cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.658+0000 7ff7a7fff700 1 -- 172.21.15.161:0/2829032000 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8020040 con 0x5617b63f72d0 2023-07-14T11:25:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.658+0000 7ff7a7fff700 1 -- 172.21.15.161:0/2829032000 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b8028e90 con 0x5617b63f72d0 2023-07-14T11:25:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.658+0000 7ff7a7fff700 1 -- 172.21.15.161:0/2829032000 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80038e0 con 0x5617b63f72d0 2023-07-14T11:25:59.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/2829032000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 msgr2=0x5617b64094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b64094b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5617b6414c70 tx=0x7ff7b8014000 comp rx=0 tx=0).stop 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/2829032000 shutdown_connections 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b63f72d0 0x5617b64094b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b6402ad0 0x5617b63faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/2829032000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f7be0 0x5617b63f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/2829032000 >> 172.21.15.161:0/2829032000 conn(0x5617b63edda0 msgr2=0x5617b640e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/2829032000 shutdown_connections 2023-07-14T11:25:59.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/2829032000 wait complete. 2023-07-14T11:25:59.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.659+0000 7ff7d70c5b80 1 -- start start 2023-07-14T11:25:59.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 1 Processor -- start 2023-07-14T11:25:59.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f72d0 0x5617b6409520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f7be0 0x5617b6407850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6407d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b6408460 con 0x5617b6402ad0 2023-07-14T11:25:59.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b64085d0 con 0x5617b63f7be0 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617b6465e80 con 0x5617b63f72d0 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7d70c5b80 10 client.0 _ll_register_callbacks cb 0x7ff7ac09a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f7be0 0x5617b6407850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7c1d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6407d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7c159b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f72d0 0x5617b6409520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f7be0 0x5617b6407850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34624/0 (socket says 172.21.15.161:34624) 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.660+0000 7ff7c0d9a700 1 -- 172.21.15.161:0/1750664467 learned_addr learned my addr 172.21.15.161:0/1750664467 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:25:59.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7c159b700 1 -- 172.21.15.161:0/1750664467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f7be0 msgr2=0x5617b6407850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7c159b700 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f7be0 0x5617b6407850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7c159b700 1 -- 172.21.15.161:0/1750664467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 msgr2=0x5617b6407d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:25:59.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7c159b700 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6407d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7c159b700 1 -- 172.21.15.161:0/1750664467 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b8017050 con 0x5617b63f72d0 2023-07-14T11:25:59.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7c159b700 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f72d0 0x5617b6409520 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80095c0 tx=0x7ff7b8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b42f14662d191f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:25:59.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:25:59.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/1750664467 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8020040 con 0x5617b63f72d0 2023-07-14T11:25:59.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/1750664467 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7ff7b8028bd0 con 0x5617b63f72d0 2023-07-14T11:25:59.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7b400f040 con 0x5617b63f72d0 2023-07-14T11:25:59.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.661+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/1750664467 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b80285d0 con 0x5617b63f72d0 2023-07-14T11:25:59.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 10 client.26434 fetch_fsmap learned FSMap version 23 2023-07-14T11:25:59.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5617b619e250 con 0x5617b63f72d0 2023-07-14T11:25:59.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/1750664467 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b400f040 con 0x5617b63f72d0 2023-07-14T11:25:59.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/1750664467 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 23) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff7b802f050 con 0x5617b63f72d0 2023-07-14T11:25:59.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 10 client.26434 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T11:25:59.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 10 client.26434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:25:59.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 20 client.26434 populate_metadata read hostname 'smithi161' 2023-07-14T11:25:59.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6467390 0x5617b6469770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 4 client.26434 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5617b6305050 con 0x5617b6467390 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.662+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6467390 0x5617b6469770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7c0d9a700 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6467390 0x5617b6469770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0120d1fa1f1ab66 server_cookie=51c8e8ba732c0fe6 in_seq=0 out_seq=0 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7a4ff9700 10 client.26434 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7a4ff9700 1 -- 172.21.15.161:0/1750664467 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5617b6305050 con 0x5617b6467390 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7a4ff9700 10 client.26434 handle_command_reply: tid=0 2023-07-14T11:25:59.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 client.26434 shutdown 2023-07-14T11:25:59.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f72d0 msgr2=0x5617b6409520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f72d0 0x5617b6409520 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80095c0 tx=0x7ff7b8004600 comp rx=0 tx=0).stop 2023-07-14T11:25:59.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 shutdown_connections 2023-07-14T11:25:59.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617b6467390 0x5617b6469770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b6402ad0 0x5617b6407d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7a4ff9700 0 client.26434 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:25:59.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617b63f72d0 0x5617b6409520 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 --2- 172.21.15.161:0/1750664467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617b63f7be0 0x5617b6407850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 >> 172.21.15.161:0/1750664467 conn(0x5617b63edda0 msgr2=0x5617b63f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 shutdown_connections 2023-07-14T11:25:59.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.663+0000 7ff7d70c5b80 1 -- 172.21.15.161:0/1750664467 wait complete. 2023-07-14T11:25:59.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7d70c5b80 20 client.26434 trim_cache size 0 max 16384 2023-07-14T11:25:59.674 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: "bfd99564-b789-41c5-852d-3e47e013eded": { 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bfd99564-b789-41c5-852d-3e47e013eded", 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:25:59.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:25:59.676 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:25:59.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff7a80a12c0 msgr2=0x7ff7a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff7a80a12c0 0x7ff7a80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005fd0 tx=0x7ff7b8009310 comp rx=0 tx=0).stop 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc1004b0 msgr2=0x7ff7bc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc1004b0 0x7ff7bc198430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009810 tx=0x7ff7ac00fa00 comp rx=0 tx=0).stop 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 shutdown_connections 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff7a80a12c0 0x7ff7a80a3770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc105b10 0x7ff7bc19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff7bc0ffb00 0x7ff7bc197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 --2- 172.21.15.161:0/4020889038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc1004b0 0x7ff7bc198430 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:25:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 >> 172.21.15.161:0/4020889038 conn(0x7ff7bc0fb600 msgr2=0x7ff7bc10af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:25:59.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 shutdown_connections 2023-07-14T11:25:59.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:25:59.664+0000 7ff7c37ff700 1 -- 172.21.15.161:0/4020889038 wait complete. 2023-07-14T11:25:59.686 INFO:tasks.cephfs.filesystem:scrub status for tag:bfd99564-b789-41c5-852d-3e47e013eded - {'path': '/', 'tag': 'bfd99564-b789-41c5-852d-3e47e013eded', 'options': 'recursive,force'} 2023-07-14T11:26:00.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 305517 9.89 MB/sec execute 42 sec latency 28.186 ms 2023-07-14T11:26:00.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:00 smithi175 ceph-mon[129081]: pgmap v115: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2023-07-14T11:26:00.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:00 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:26:00.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:00 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:00 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:00.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:00 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:00.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:00 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:00 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:00 smithi182 ceph-mon[129366]: pgmap v115: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2023-07-14T11:26:00.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:00 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:26:00.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:00 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:00 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:00.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:00 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:00.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:00 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:00 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:00 smithi161 ceph-mon[95526]: pgmap v115: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2023-07-14T11:26:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:00 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:26:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:00 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:00 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:00.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:00 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:00.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:00 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:00.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:00 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 306099 9.71 MB/sec execute 43 sec latency 62.756 ms 2023-07-14T11:26:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:01 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:01 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:01 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:01 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:01 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:01.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:01 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:01 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:01.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:01 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:01 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:01.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:01 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:01 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:01.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:01 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:01 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:01.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:01 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:01 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:01.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:01 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:01.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:01 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:01.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:01 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:02.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 308031 9.73 MB/sec execute 44 sec latency 67.449 ms 2023-07-14T11:26:02.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:02 smithi175 ceph-mon[129081]: pgmap v116: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 333 op/s 2023-07-14T11:26:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:02 smithi182 ceph-mon[129366]: pgmap v116: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 333 op/s 2023-07-14T11:26:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:02 smithi161 ceph-mon[95526]: pgmap v116: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 333 op/s 2023-07-14T11:26:03.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 309335 9.67 MB/sec execute 45 sec latency 21.847 ms 2023-07-14T11:26:04.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:03 smithi175 ceph-mon[129081]: pgmap v117: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 229 op/s 2023-07-14T11:26:04.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 311988 9.76 MB/sec execute 46 sec latency 15.932 ms 2023-07-14T11:26:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:03 smithi182 ceph-mon[129366]: pgmap v117: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 229 op/s 2023-07-14T11:26:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:03 smithi161 ceph-mon[95526]: pgmap v117: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 229 op/s 2023-07-14T11:26:05.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 313310 9.70 MB/sec execute 47 sec latency 13.935 ms 2023-07-14T11:26:06.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 315860 9.83 MB/sec execute 48 sec latency 10.418 ms 2023-07-14T11:26:06.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:06 smithi175 ceph-mon[129081]: pgmap v118: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 229 op/s 2023-07-14T11:26:06.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:06 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:26:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:06 smithi182 ceph-mon[129366]: pgmap v118: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 229 op/s 2023-07-14T11:26:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:06 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:26:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:06 smithi161 ceph-mon[95526]: pgmap v118: 129 pgs: 129 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 229 op/s 2023-07-14T11:26:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:06 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:26:07.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 316942 9.69 MB/sec execute 49 sec latency 23.271 ms 2023-07-14T11:26:07.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:07 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:26:07.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:07 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:26:07.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:07 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:26:07.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:07 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:26:07.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:07 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:26:07.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:07 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:26:07.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:07 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:26:07.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:07 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:26:07.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:07 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:26:08.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 319406 9.81 MB/sec execute 50 sec latency 25.696 ms 2023-07-14T11:26:08.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:08 smithi182 ceph-mon[129366]: pgmap v119: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-07-14T11:26:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:08 smithi161 ceph-mon[95526]: pgmap v119: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-07-14T11:26:08.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:08 smithi175 ceph-mon[129081]: pgmap v119: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-07-14T11:26:09.174 INFO:tasks.workunit.client.0.smithi161.stdout: 1 320715 9.75 MB/sec execute 51 sec latency 17.208 ms 2023-07-14T11:26:10.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 323135 9.83 MB/sec execute 52 sec latency 19.637 ms 2023-07-14T11:26:10.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:10 smithi182 ceph-mon[129366]: pgmap v120: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 231 op/s 2023-07-14T11:26:10.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:10 smithi161 ceph-mon[95526]: pgmap v120: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 231 op/s 2023-07-14T11:26:10.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:10 smithi175 ceph-mon[129081]: pgmap v120: 129 pgs: 129 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 231 op/s 2023-07-14T11:26:11.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 324463 9.74 MB/sec execute 53 sec latency 47.840 ms 2023-07-14T11:26:12.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 326703 9.82 MB/sec execute 54 sec latency 10.470 ms 2023-07-14T11:26:12.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:12 smithi182 ceph-mon[129366]: pgmap v121: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2023-07-14T11:26:12.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:12 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:26:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:12 smithi161 ceph-mon[95526]: pgmap v121: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2023-07-14T11:26:12.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:12 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:26:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:12 smithi175 ceph-mon[129081]: pgmap v121: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2023-07-14T11:26:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:12 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:26:13.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 327977 9.74 MB/sec execute 55 sec latency 71.820 ms 2023-07-14T11:26:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:13 smithi175 ceph-mon[129081]: pgmap v122: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2023-07-14T11:26:14.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 330284 9.81 MB/sec execute 56 sec latency 16.830 ms 2023-07-14T11:26:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:13 smithi182 ceph-mon[129366]: pgmap v122: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2023-07-14T11:26:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:13 smithi161 ceph-mon[95526]: pgmap v122: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2023-07-14T11:26:15.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 331544 9.73 MB/sec execute 57 sec latency 55.824 ms 2023-07-14T11:26:16.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 333830 9.80 MB/sec execute 58 sec latency 15.428 ms 2023-07-14T11:26:16.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:16 smithi175 ceph-mon[129081]: pgmap v123: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2023-07-14T11:26:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:16 smithi182 ceph-mon[129366]: pgmap v123: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2023-07-14T11:26:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:16 smithi161 ceph-mon[95526]: pgmap v123: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2023-07-14T11:26:17.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 335628 9.73 MB/sec execute 59 sec latency 10.925 ms 2023-07-14T11:26:18.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 337371 9.79 MB/sec execute 60 sec latency 28.617 ms 2023-07-14T11:26:18.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:18 smithi175 ceph-mon[129081]: pgmap v124: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 324 op/s 2023-07-14T11:26:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:18 smithi182 ceph-mon[129366]: pgmap v124: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 324 op/s 2023-07-14T11:26:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:18 smithi161 ceph-mon[95526]: pgmap v124: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 324 op/s 2023-07-14T11:26:19.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 339432 9.75 MB/sec execute 61 sec latency 12.993 ms 2023-07-14T11:26:20.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 341111 9.79 MB/sec execute 62 sec latency 49.832 ms 2023-07-14T11:26:20.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:20 smithi175 ceph-mon[129081]: pgmap v125: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 217 op/s 2023-07-14T11:26:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:20 smithi182 ceph-mon[129366]: pgmap v125: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 217 op/s 2023-07-14T11:26:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:20 smithi161 ceph-mon[95526]: pgmap v125: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 217 op/s 2023-07-14T11:26:21.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 343700 9.84 MB/sec execute 63 sec latency 20.997 ms 2023-07-14T11:26:22.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 345135 9.81 MB/sec execute 64 sec latency 10.583 ms 2023-07-14T11:26:22.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:22 smithi175 ceph-mon[129081]: pgmap v126: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 325 op/s 2023-07-14T11:26:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:22 smithi182 ceph-mon[129366]: pgmap v126: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 325 op/s 2023-07-14T11:26:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:22 smithi161 ceph-mon[95526]: pgmap v126: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 325 op/s 2023-07-14T11:26:23.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 347592 9.90 MB/sec execute 65 sec latency 52.262 ms 2023-07-14T11:26:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:23 smithi175 ceph-mon[129081]: pgmap v127: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-14T11:26:24.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 348798 9.80 MB/sec execute 66 sec latency 29.299 ms 2023-07-14T11:26:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:23 smithi182 ceph-mon[129366]: pgmap v127: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-14T11:26:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:23 smithi161 ceph-mon[95526]: pgmap v127: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-14T11:26:25.175 INFO:tasks.workunit.client.0.smithi161.stdout: 1 351300 9.89 MB/sec execute 67 sec latency 34.515 ms 2023-07-14T11:26:26.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 352664 9.84 MB/sec execute 68 sec latency 20.499 ms 2023-07-14T11:26:26.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:26 smithi175 ceph-mon[129081]: pgmap v128: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-14T11:26:26.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:26 smithi182 ceph-mon[129366]: pgmap v128: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-14T11:26:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:26 smithi161 ceph-mon[95526]: pgmap v128: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-14T11:26:27.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 355059 9.90 MB/sec execute 69 sec latency 20.531 ms 2023-07-14T11:26:28.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 355693 9.77 MB/sec execute 70 sec latency 390.393 ms 2023-07-14T11:26:28.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:28 smithi175 ceph-mon[129081]: pgmap v129: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 326 op/s 2023-07-14T11:26:28.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:28 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:26:28.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:28 smithi182 ceph-mon[129366]: pgmap v129: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 326 op/s 2023-07-14T11:26:28.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:28 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:26:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:28 smithi161 ceph-mon[95526]: pgmap v129: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 326 op/s 2023-07-14T11:26:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:28 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:26:29.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 358353 9.87 MB/sec execute 71 sec latency 20.834 ms 2023-07-14T11:26:29.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:29 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:26:29.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:29 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:26:29.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:29 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:26:29.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:29 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:26:29.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:29 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:26:29.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:29 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:26:29.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:29 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:26:29.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:29 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:26:29.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:29 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:26:29.687 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:26:29.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 -- 172.21.15.161:0/2639487401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c0ffad0 msgr2=0x7f4c1c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 --2- 172.21.15.161:0/2639487401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c0ffad0 0x7f4c1c0fff40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009990 tx=0x7f4c10009ca0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 -- 172.21.15.161:0/2639487401 shutdown_connections 2023-07-14T11:26:29.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 --2- 172.21.15.161:0/2639487401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c1c100480 0x7f4c1c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 --2- 172.21.15.161:0/2639487401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4c1c105ae0 0x7f4c1c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 --2- 172.21.15.161:0/2639487401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c0ffad0 0x7f4c1c0fff40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 -- 172.21.15.161:0/2639487401 >> 172.21.15.161:0/2639487401 conn(0x7f4c1c0fb5f0 msgr2=0x7f4c1c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:29.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 -- 172.21.15.161:0/2639487401 shutdown_connections 2023-07-14T11:26:29.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.909+0000 7f4c21bff700 1 -- 172.21.15.161:0/2639487401 wait complete. 2023-07-14T11:26:29.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 Processor -- start 2023-07-14T11:26:29.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 -- start start 2023-07-14T11:26:29.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c1c0ffad0 0x7f4c1c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c100480 0x7f4c1c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4c1c105ae0 0x7f4c1c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c19cdb0 con 0x7f4c1c0ffad0 2023-07-14T11:26:29.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c19cf20 con 0x7f4c1c100480 2023-07-14T11:26:29.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.910+0000 7f4c21bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c19d0e0 con 0x7f4c1c105ae0 2023-07-14T11:26:29.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c1c0ffad0 0x7f4c1c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c100480 0x7f4c1c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c100480 0x7f4c1c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55764/0 (socket says 172.21.15.161:55764) 2023-07-14T11:26:29.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 -- 172.21.15.161:0/1708365988 learned_addr learned my addr 172.21.15.161:0/1708365988 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:29.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4c1c105ae0 0x7f4c1c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 -- 172.21.15.161:0/1708365988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4c1c105ae0 msgr2=0x7f4c1c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4c1c105ae0 0x7f4c1c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 -- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c1c0ffad0 msgr2=0x7f4c1c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c1c0ffad0 0x7f4c1c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 -- 172.21.15.161:0/1708365988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c100095f0 con 0x7f4c1c100480 2023-07-14T11:26:29.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.911+0000 7f4c1affd700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c100480 0x7f4c1c198310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:29.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.912+0000 7f4c18ff9700 1 -- 172.21.15.161:0/1708365988 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1001d070 con 0x7f4c1c100480 2023-07-14T11:26:29.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.912+0000 7f4c18ff9700 1 -- 172.21.15.161:0/1708365988 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c10021410 con 0x7f4c1c100480 2023-07-14T11:26:29.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.912+0000 7f4c18ff9700 1 -- 172.21.15.161:0/1708365988 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10017650 con 0x7f4c1c100480 2023-07-14T11:26:29.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.912+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c19d360 con 0x7f4c1c100480 2023-07-14T11:26:29.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.912+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1a4c00 con 0x7f4c1c100480 2023-07-14T11:26:29.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.915+0000 7f4c18ff9700 1 -- 172.21.15.161:0/1708365988 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4c10021580 con 0x7f4c1c100480 2023-07-14T11:26:29.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.917+0000 7f4c18ff9700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4c040a1340 0x7f4c040a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.917+0000 7f4c35544b80 1 Processor -- start 2023-07-14T11:26:29.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.917+0000 7f4c18ff9700 1 -- 172.21.15.161:0/1708365988 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f4c100ce370 con 0x7f4c1c100480 2023-07-14T11:26:29.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.917+0000 7f4c35544b80 1 -- start start 2023-07-14T11:26:29.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.917+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4c040a1340 0x7f4c040a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c35544b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe0accb0 con 0x55fefe1b1ad0 2023-07-14T11:26:29.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c35544b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe0ffa40 con 0x55fefe1a62d0 2023-07-14T11:26:29.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c35544b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe0f8ed0 con 0x55fefe1a6be0 2023-07-14T11:26:29.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50180/0 (socket says 172.21.15.161:50180) 2023-07-14T11:26:29.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 -- 172.21.15.161:0/684065467 learned_addr learned my addr 172.21.15.161:0/684065467 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:29.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4c040a1340 0x7f4c040a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009be0 tx=0x7f4c0c009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 -- 172.21.15.161:0/684065467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 msgr2=0x55fefe1a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 -- 172.21.15.161:0/684065467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1b1ad0 msgr2=0x55fefe1b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1affd700 1 -- 172.21.15.161:0/684065467 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefdf26de0 con 0x55fefe1a6be0 2023-07-14T11:26:29.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.918+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:26:29.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.919+0000 7f4c1affd700 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b1590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c10097bb0 tx=0x7f4c100975b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c14df332c41f8f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:29.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.919+0000 7f4c01ffb700 1 -- 172.21.15.161:0/684065467 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1009f170 con 0x55fefe1a6be0 2023-07-14T11:26:29.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.919+0000 7f4c01ffb700 1 -- 172.21.15.161:0/684065467 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c100a3500 con 0x55fefe1a6be0 2023-07-14T11:26:29.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.919+0000 7f4c01ffb700 1 -- 172.21.15.161:0/684065467 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1009b600 con 0x55fefe1a6be0 2023-07-14T11:26:29.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.919+0000 7f4c35544b80 1 -- 172.21.15.161:0/684065467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 msgr2=0x55fefe1b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.919+0000 7f4c35544b80 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b1590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c10097bb0 tx=0x7f4c100975b0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 -- 172.21.15.161:0/684065467 shutdown_connections 2023-07-14T11:26:29.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 --2- 172.21.15.161:0/684065467 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b1590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 -- 172.21.15.161:0/684065467 >> 172.21.15.161:0/684065467 conn(0x55fefe19cda0 msgr2=0x55fefe1a26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:29.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 -- 172.21.15.161:0/684065467 shutdown_connections 2023-07-14T11:26:29.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 -- 172.21.15.161:0/684065467 wait complete. 2023-07-14T11:26:29.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.920+0000 7f4c35544b80 1 -- start start 2023-07-14T11:26:29.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 1 Processor -- start 2023-07-14T11:26:29.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a62d0 0x55fefe1b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe098ac0 con 0x55fefe1a62d0 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe0987c0 con 0x55fefe1b1ad0 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefdf7c920 con 0x55fefe1a6be0 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c35544b80 10 client.0 _ll_register_callbacks cb 0x55fefe096760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55788/0 (socket says 172.21.15.161:55788) 2023-07-14T11:26:29.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50194/0 (socket says 172.21.15.161:50194) 2023-07-14T11:26:29.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c1bfff700 1 -- 172.21.15.161:0/715723548 learned_addr learned my addr 172.21.15.161:0/715723548 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:29.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.921+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a62d0 0x55fefe1b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c1bfff700 1 -- 172.21.15.161:0/715723548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 msgr2=0x55fefe1b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c1bfff700 1 -- 172.21.15.161:0/715723548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a62d0 msgr2=0x55fefe1b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a62d0 0x55fefe1b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c1bfff700 1 -- 172.21.15.161:0/715723548 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1a4c00 con 0x55fefe1b1ad0 2023-07-14T11:26:29.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c1affd700 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:29.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b2920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007bf0 tx=0x7f4c14007f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f432c94297abb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:29.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:26:29.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4bfaffd700 1 -- 172.21.15.161:0/715723548 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400e070 con 0x55fefe1b1ad0 2023-07-14T11:26:29.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4bfaffd700 1 -- 172.21.15.161:0/715723548 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f4c14012b20 con 0x55fefe1b1ad0 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.922+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fefe0524f0 con 0x55fefe1b1ad0 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4bfaffd700 1 -- 172.21.15.161:0/715723548 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c140090b0 con 0x55fefe1b1ad0 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 10 client.26441 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fefdf0fc60 con 0x55fefe1b1ad0 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4bfaffd700 1 -- 172.21.15.161:0/715723548 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fefe0524f0 con 0x55fefe1b1ad0 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4bfaffd700 1 -- 172.21.15.161:0/715723548 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c14014040 con 0x55fefe1b1ad0 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 10 client.26441 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 10 client.26441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:29.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 20 client.26441 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:29.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe1c9990 0x55fefe1cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 4 client.26441 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:26:29.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.923+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fefe0f1eb0 con 0x55fefe1c9990 2023-07-14T11:26:29.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.924+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe1c9990 0x55fefe1cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.924+0000 7f4bfaffd700 10 client.26441 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:29.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.924+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe1c9990 0x55fefe1cbda0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e252eeda5c36534d server_cookie=9c3777ff0a72987e in_seq=0 out_seq=0 2023-07-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4bfaffd700 1 -- 172.21.15.161:0/715723548 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fefe0f1eb0 con 0x55fefe1c9990 2023-07-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4bfaffd700 10 client.26441 handle_command_reply: tid=0 2023-07-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 client.26441 shutdown 2023-07-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1b1ad0 msgr2=0x55fefe1b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b2920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007bf0 tx=0x7f4c14007f00 comp rx=0 tx=0).stop 2023-07-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 shutdown_connections 2023-07-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe1c9990 0x55fefe1cbda0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a62d0 0x55fefe1b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1b1ad0 0x55fefe1b2920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 --2- 172.21.15.161:0/715723548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1a6be0 0x55fefe1b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4bfaffd700 0 client.26441 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:29.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 >> 172.21.15.161:0/715723548 conn(0x55fefe19cda0 msgr2=0x55fefe1a2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:29.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.925+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 shutdown_connections 2023-07-14T11:26:29.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.926+0000 7f4c35544b80 1 -- 172.21.15.161:0/715723548 wait complete. 2023-07-14T11:26:29.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.926+0000 7f4c35544b80 20 client.26441 trim_cache size 0 max 16384 2023-07-14T11:26:29.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 Processor -- start 2023-07-14T11:26:29.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 -- start start 2023-07-14T11:26:29.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe1b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe1abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe1c9770 con 0x55fefe1a6be0 2023-07-14T11:26:29.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe19c970 con 0x55fefe1a62d0 2023-07-14T11:26:29.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.944+0000 7f4c35544b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe1b1180 con 0x55fefe1b1ad0 2023-07-14T11:26:29.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55804/0 (socket says 172.21.15.161:55804) 2023-07-14T11:26:29.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/1765868002 learned_addr learned my addr 172.21.15.161:0/1765868002 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:29.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe1abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1affd700 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe1b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/1765868002 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 msgr2=0x55fefe1abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe1abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/1765868002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 msgr2=0x55fefe1b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe1b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/1765868002 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefe1a10e0 con 0x55fefe1a62d0 2023-07-14T11:26:29.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe1abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:29.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1ab400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55fefe1cc680 tx=0x7f4c0c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d0034961cf8a6d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:29.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c027fc700 1 -- 172.21.15.161:0/1765868002 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c024040 con 0x55fefe1a62d0 2023-07-14T11:26:29.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c027fc700 1 -- 172.21.15.161:0/1765868002 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0c02a4d0 con 0x55fefe1a62d0 2023-07-14T11:26:29.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.945+0000 7f4c027fc700 1 -- 172.21.15.161:0/1765868002 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c004030 con 0x55fefe1a62d0 2023-07-14T11:26:29.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 -- 172.21.15.161:0/1765868002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 msgr2=0x55fefe1ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1ab400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55fefe1cc680 tx=0x7f4c0c014000 comp rx=0 tx=0).stop 2023-07-14T11:26:29.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 -- 172.21.15.161:0/1765868002 shutdown_connections 2023-07-14T11:26:29.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe1abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe1ab400 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 --2- 172.21.15.161:0/1765868002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe1b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 -- 172.21.15.161:0/1765868002 >> 172.21.15.161:0/1765868002 conn(0x55fefe19cda0 msgr2=0x55fefe1d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:29.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 -- 172.21.15.161:0/1765868002 shutdown_connections 2023-07-14T11:26:29.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.946+0000 7f4c35544b80 1 -- 172.21.15.161:0/1765868002 wait complete. 2023-07-14T11:26:29.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.947+0000 7f4c35544b80 1 -- start start 2023-07-14T11:26:29.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.947+0000 7f4c35544b80 1 Processor -- start 2023-07-14T11:26:29.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.947+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe216d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.947+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe217280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.947+0000 7f4c35544b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe2117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.947+0000 7f4c35544b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe211e00 con 0x55fefe1a6be0 2023-07-14T11:26:29.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c35544b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe211f70 con 0x55fefe1a62d0 2023-07-14T11:26:29.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c35544b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefe212130 con 0x55fefe1b1ad0 2023-07-14T11:26:29.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c35544b80 10 client.0 _ll_register_callbacks cb 0x7f4c14007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:29.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe217280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe2117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe216d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe216d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55812/0 (socket says 172.21.15.161:55812) 2023-07-14T11:26:29.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe2117e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50222/0 (socket says 172.21.15.161:50222) 2023-07-14T11:26:29.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe217280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47812/0 (socket says 172.21.15.161:47812) 2023-07-14T11:26:29.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/105347738 learned_addr learned my addr 172.21.15.161:0/105347738 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:29.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/105347738 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 msgr2=0x55fefe2117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe2117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/105347738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 msgr2=0x55fefe217280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:29.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe217280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1b7fe700 1 -- 172.21.15.161:0/105347738 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fefdf0fc60 con 0x55fefe1a62d0 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.948+0000 7f4c1bfff700 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe2117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4c1b7fe700 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe216d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0117d0 tx=0x7f4c0c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5620655779ae9c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4bfaffd700 1 -- 172.21.15.161:0/105347738 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c024040 con 0x55fefe1a62d0 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4bfaffd700 1 -- 172.21.15.161:0/105347738 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f4c0c02ac30 con 0x55fefe1a62d0 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c100a3500 con 0x55fefe1a62d0 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4bfaffd700 1 -- 172.21.15.161:0/105347738 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c0c0212e0 con 0x55fefe1a62d0 2023-07-14T11:26:29.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4c35544b80 10 client.26447 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f4c0c017070 con 0x55fefe1a62d0 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.949+0000 7f4bfaffd700 1 -- 172.21.15.161:0/105347738 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c100a3500 con 0x55fefe1a62d0 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4bfaffd700 1 -- 172.21.15.161:0/105347738 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c0c030020 con 0x55fefe1a62d0 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c35544b80 10 client.26447 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c35544b80 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c35544b80 20 client.26447 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c35544b80 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe21eac0 0x55fefe220ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:29.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c35544b80 4 client.26447 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:26:29.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fefe0b4050 con 0x55fefe21eac0 2023-07-14T11:26:29.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c1affd700 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe21eac0 0x55fefe220ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:29.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4c1affd700 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe21eac0 0x55fefe220ed0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d551612e4e5706a server_cookie=df79ac5b5c621aab in_seq=0 out_seq=0 2023-07-14T11:26:29.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.950+0000 7f4bfaffd700 10 client.26447 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:29.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4bfaffd700 1 -- 172.21.15.161:0/105347738 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fefe0b4050 con 0x55fefe21eac0 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4bfaffd700 10 client.26447 handle_command_reply: tid=0 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 client.26447 shutdown 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 msgr2=0x55fefe216d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe216d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0117d0 tx=0x7f4c0c005dc0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 shutdown_connections 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55fefe21eac0 0x55fefe220ed0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55fefe1b1ad0 0x55fefe2117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fefe1a62d0 0x55fefe216d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 --2- 172.21.15.161:0/105347738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fefe1a6be0 0x55fefe217280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4bfaffd700 0 client.26447 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:29.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 >> 172.21.15.161:0/105347738 conn(0x55fefe19cda0 msgr2=0x55fefe1abe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:29.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 shutdown_connections 2023-07-14T11:26:29.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 1 -- 172.21.15.161:0/105347738 wait complete. 2023-07-14T11:26:29.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.951+0000 7f4c35544b80 20 client.26447 trim_cache size 0 max 16384 2023-07-14T11:26:29.970 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:26:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:26:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:26:29.970 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:26:29.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4c040a1340 msgr2=0x7f4c040a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4c040a1340 0x7f4c040a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009be0 tx=0x7f4c0c009260 comp rx=0 tx=0).stop 2023-07-14T11:26:29.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c100480 msgr2=0x7f4c1c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:29.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c100480 0x7f4c1c198310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10004790 comp rx=0 tx=0).stop 2023-07-14T11:26:29.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 shutdown_connections 2023-07-14T11:26:29.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4c040a1340 0x7f4c040a37f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c1c0ffad0 0x7f4c1c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4c1c105ae0 0x7f4c1c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 --2- 172.21.15.161:0/1708365988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c1c100480 0x7f4c1c198310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 >> 172.21.15.161:0/1708365988 conn(0x7f4c1c0fb5f0 msgr2=0x7f4c1c10ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 shutdown_connections 2023-07-14T11:26:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:29.952+0000 7f4c21bff700 1 -- 172.21.15.161:0/1708365988 wait complete. 2023-07-14T11:26:29.974 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:26:29.974 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:26:30.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 359634 9.82 MB/sec execute 72 sec latency 13.254 ms 2023-07-14T11:26:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 -- 172.21.15.161:0/2534870595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 msgr2=0x7fb9580ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 --2- 172.21.15.161:0/2534870595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 0x7fb9580ffe40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009990 tx=0x7fb94c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 -- 172.21.15.161:0/2534870595 shutdown_connections 2023-07-14T11:26:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 --2- 172.21.15.161:0/2534870595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb958100380 0x7fb95810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 --2- 172.21.15.161:0/2534870595 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9581059e0 0x7fb958105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 --2- 172.21.15.161:0/2534870595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 0x7fb9580ffe40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 -- 172.21.15.161:0/2534870595 >> 172.21.15.161:0/2534870595 conn(0x7fb9580fb4d0 msgr2=0x7fb9580fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 -- 172.21.15.161:0/2534870595 shutdown_connections 2023-07-14T11:26:30.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.187+0000 7fb95d79d700 1 -- 172.21.15.161:0/2534870595 wait complete. 2023-07-14T11:26:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.188+0000 7fb95d79d700 1 Processor -- start 2023-07-14T11:26:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.188+0000 7fb95d79d700 1 -- start start 2023-07-14T11:26:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb95d79d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 0x7fb958072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb95d79d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb958100380 0x7fb958072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb95d79d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9581059e0 0x7fb9581a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb95d79d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb958073580 con 0x7fb9580ff9d0 2023-07-14T11:26:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb95d79d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581a3fa0 con 0x7fb9581059e0 2023-07-14T11:26:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb95d79d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581a40e0 con 0x7fb958100380 2023-07-14T11:26:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb958100380 0x7fb958072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb957fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 0x7fb958072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb958100380 0x7fb958072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50232/0 (socket says 172.21.15.161:50232) 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 -- 172.21.15.161:0/3937097207 learned_addr learned my addr 172.21.15.161:0/3937097207 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb95cf9c700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9581059e0 0x7fb9581a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 -- 172.21.15.161:0/3937097207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9581059e0 msgr2=0x7fb9581a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9581059e0 0x7fb9581a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 -- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 msgr2=0x7fb958072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 0x7fb958072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.189+0000 7fb9577fe700 1 -- 172.21.15.161:0/3937097207 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950009c40 con 0x7fb958100380 2023-07-14T11:26:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.190+0000 7fb957fff700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 0x7fb958072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.190+0000 7fb9577fe700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb958100380 0x7fb958072d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb94c005910 tx=0x7fb94c004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.190+0000 7fb9557fa700 1 -- 172.21.15.161:0/3937097207 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c01d070 con 0x7fb958100380 2023-07-14T11:26:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.190+0000 7fb9557fa700 1 -- 172.21.15.161:0/3937097207 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb94c022470 con 0x7fb958100380 2023-07-14T11:26:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.190+0000 7fb9557fa700 1 -- 172.21.15.161:0/3937097207 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c017650 con 0x7fb958100380 2023-07-14T11:26:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.190+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0095f0 con 0x7fb958100380 2023-07-14T11:26:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.190+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9581a4520 con 0x7fb958100380 2023-07-14T11:26:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.191+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb958067220 con 0x7fb958100380 2023-07-14T11:26:30.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.193+0000 7fb9557fa700 1 -- 172.21.15.161:0/3937097207 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb94c0225e0 con 0x7fb958100380 2023-07-14T11:26:30.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.194+0000 7fb9557fa700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9400a1430 0x7fb9400a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.194+0000 7fb9557fa700 1 -- 172.21.15.161:0/3937097207 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fb94c0ce220 con 0x7fb958100380 2023-07-14T11:26:30.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.195+0000 7fb957fff700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9400a1430 0x7fb9400a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.195+0000 7fb957fff700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9400a1430 0x7fb9400a38e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb958100b10 tx=0x7fb948009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.200+0000 7fb9557fa700 1 -- 172.21.15.161:0/3937097207 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb94c05c790 con 0x7fb958100380 2023-07-14T11:26:30.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:30 smithi182 ceph-mon[129366]: pgmap v130: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2023-07-14T11:26:30.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.469+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb958073d40 con 0x7fb958100380 2023-07-14T11:26:30.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.470+0000 7fb9557fa700 1 -- 172.21.15.161:0/3937097207 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10864 (secure 0 0 0) 0x7fb94c097330 con 0x7fb958100380 2023-07-14T11:26:30.474 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:26:30.474 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:26:11.184716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:26:30.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9400a1430 msgr2=0x7fb9400a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9400a1430 0x7fb9400a38e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb958100b10 tx=0x7fb948009260 comp rx=0 tx=0).stop 2023-07-14T11:26:30.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb958100380 msgr2=0x7fb958072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb958100380 0x7fb958072d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb94c005910 tx=0x7fb94c004710 comp rx=0 tx=0).stop 2023-07-14T11:26:30.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 shutdown_connections 2023-07-14T11:26:30.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9400a1430 0x7fb9400a38e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9580ff9d0 0x7fb958072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9581059e0 0x7fb9581a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 --2- 172.21.15.161:0/3937097207 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb958100380 0x7fb958072d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.472+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 >> 172.21.15.161:0/3937097207 conn(0x7fb9580fb4d0 msgr2=0x7fb958103520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.473+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 shutdown_connections 2023-07-14T11:26:30.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.473+0000 7fb95d79d700 1 -- 172.21.15.161:0/3937097207 wait complete. 2023-07-14T11:26:30.478 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 25 2023-07-14T11:26:30.494 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:30 smithi161 ceph-mon[95526]: pgmap v130: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2023-07-14T11:26:30.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:26:30.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:30 smithi175 ceph-mon[129081]: pgmap v130: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2023-07-14T11:26:30.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 -- 172.21.15.161:0/1656438887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 msgr2=0x7f1c9c072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/1656438887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c072c20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009990 tx=0x7f1c8c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 -- 172.21.15.161:0/1656438887 shutdown_connections 2023-07-14T11:26:30.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/1656438887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c072c20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/1656438887 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c9c10c380 0x7f1c9c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/1656438887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9c073160 0x7f1c9c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 -- 172.21.15.161:0/1656438887 >> 172.21.15.161:0/1656438887 conn(0x7f1c9c0fb5e0 msgr2=0x7f1c9c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 -- 172.21.15.161:0/1656438887 shutdown_connections 2023-07-14T11:26:30.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.718+0000 7f1ca25ff700 1 -- 172.21.15.161:0/1656438887 wait complete. 2023-07-14T11:26:30.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.719+0000 7f1ca25ff700 1 Processor -- start 2023-07-14T11:26:30.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.719+0000 7f1ca25ff700 1 -- start start 2023-07-14T11:26:30.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1ca25ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1ca25ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9c073160 0x7f1c9c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1ca25ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c9c10c380 0x7f1c9c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1ca25ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c19ce50 con 0x7f1c9c072850 2023-07-14T11:26:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1ca25ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c19cfc0 con 0x7f1c9c073160 2023-07-14T11:26:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1ca25ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c19d180 con 0x7f1c9c10c380 2023-07-14T11:26:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9c073160 0x7f1c9c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47846/0 (socket says 172.21.15.161:47846) 2023-07-14T11:26:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 -- 172.21.15.161:0/2426636399 learned_addr learned my addr 172.21.15.161:0/2426636399 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1ca0b9c700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c9c10c380 0x7f1c9c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 -- 172.21.15.161:0/2426636399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c9c10c380 msgr2=0x7f1c9c19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c9c10c380 0x7f1c9c19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 -- 172.21.15.161:0/2426636399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9c073160 msgr2=0x7f1c9c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9c073160 0x7f1c9c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.720+0000 7f1c9bfff700 1 -- 172.21.15.161:0/2426636399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c0095f0 con 0x7f1c9c072850 2023-07-14T11:26:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.721+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c197e70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005910 tx=0x7f1c8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.721+0000 7f1c997fa700 1 -- 172.21.15.161:0/2426636399 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c01d070 con 0x7f1c9c072850 2023-07-14T11:26:30.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.721+0000 7f1c997fa700 1 -- 172.21.15.161:0/2426636399 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8c022470 con 0x7f1c9c072850 2023-07-14T11:26:30.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.721+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c19d400 con 0x7f1c9c072850 2023-07-14T11:26:30.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.721+0000 7f1c997fa700 1 -- 172.21.15.161:0/2426636399 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c017650 con 0x7f1c9c072850 2023-07-14T11:26:30.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.721+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c1a4ca0 con 0x7f1c9c072850 2023-07-14T11:26:30.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.724+0000 7f1c997fa700 1 -- 172.21.15.161:0/2426636399 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1c8c00fad0 con 0x7f1c9c072850 2023-07-14T11:26:30.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.726+0000 7f1c997fa700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c840a12c0 0x7f1c840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.726+0000 7f1cb5e9cb80 1 Processor -- start 2023-07-14T11:26:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1c997fa700 1 -- 172.21.15.161:0/2426636399 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f1c8c0ce4b0 con 0x7f1c9c072850 2023-07-14T11:26:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c840a12c0 0x7f1c840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1cb5e9cb80 1 -- start start 2023-07-14T11:26:30.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c02edcb0 con 0x5641c03e7be0 2023-07-14T11:26:30.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c0340a40 con 0x5641c03f2ad0 2023-07-14T11:26:30.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.727+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c0339ed0 con 0x5641c03e72d0 2023-07-14T11:26:30.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1ca0b9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03e76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50242/0 (socket says 172.21.15.161:50242) 2023-07-14T11:26:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 -- 172.21.15.161:0/280773031 learned_addr learned my addr 172.21.15.161:0/280773031 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c840a12c0 0x7f1c840a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c199480 tx=0x7f1c9000b410 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 -- 172.21.15.161:0/280773031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 msgr2=0x5641c03f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 -- 172.21.15.161:0/280773031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 msgr2=0x5641c03f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 -- 172.21.15.161:0/280773031 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c0167de0 con 0x5641c03e72d0 2023-07-14T11:26:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03e76a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0977e0 tx=0x7f1c8c09b1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b16e6878cd35f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c827fc700 1 -- 172.21.15.161:0/280773031 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c09b2e0 con 0x5641c03e72d0 2023-07-14T11:26:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c827fc700 1 -- 172.21.15.161:0/280773031 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8c09f310 con 0x5641c03e72d0 2023-07-14T11:26:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.728+0000 7f1c827fc700 1 -- 172.21.15.161:0/280773031 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c09f020 con 0x5641c03e72d0 2023-07-14T11:26:30.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/280773031 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 msgr2=0x5641c03e76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03e76a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0977e0 tx=0x7f1c8c09b1a0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/280773031 shutdown_connections 2023-07-14T11:26:30.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03e76a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/280773031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/280773031 >> 172.21.15.161:0/280773031 conn(0x5641c03ddda0 msgr2=0x5641c03e36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/280773031 shutdown_connections 2023-07-14T11:26:30.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.729+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/280773031 wait complete. 2023-07-14T11:26:30.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.730+0000 7f1cb5e9cb80 1 -- start start 2023-07-14T11:26:30.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.730+0000 7f1cb5e9cb80 1 Processor -- start 2023-07-14T11:26:30.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c02d9ac0 con 0x5641c03e7be0 2023-07-14T11:26:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c02d97c0 con 0x5641c03f2ad0 2023-07-14T11:26:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c01bd920 con 0x5641c03e72d0 2023-07-14T11:26:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1cb5e9cb80 10 client.0 _ll_register_callbacks cb 0x5641c02d7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50252/0 (socket says 172.21.15.161:50252) 2023-07-14T11:26:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1614871532 learned_addr learned my addr 172.21.15.161:0/1614871532 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1ca0b9c700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 msgr2=0x5641c03f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1614871532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 msgr2=0x5641c03f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1614871532 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c1a4ca0 con 0x5641c03e72d0 2023-07-14T11:26:30.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03f2ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005f50 tx=0x7f1c8c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe8213042162e5f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.731+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1614871532 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c0a10d0 con 0x5641c03e72d0 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1614871532 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f1c8c0b9640 con 0x5641c03e72d0 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641c02934f0 con 0x5641c03e72d0 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1614871532 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c8c0225e0 con 0x5641c03e72d0 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 10 client.26470 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:30.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5641c0150c60 con 0x5641c03e72d0 2023-07-14T11:26:30.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1614871532 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641c02934f0 con 0x5641c03e72d0 2023-07-14T11:26:30.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1614871532 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c8c09b540 con 0x5641c03e72d0 2023-07-14T11:26:30.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 10 client.26470 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:30.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 10 client.26470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:26:30.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 20 client.26470 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:30.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c0401c50 0x5641c0404030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 4 client.26470 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.732+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641c0332eb0 con 0x5641c0401c50 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.733+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c0401c50 0x5641c0404030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.733+0000 7f1c7b7fe700 10 client.26470 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.733+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c0401c50 0x5641c0404030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=344b0aa30423350e server_cookie=8bfe3ad88f9c2739 in_seq=0 out_seq=0 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1614871532 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5641c0332eb0 con 0x5641c0401c50 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1c7b7fe700 10 client.26470 handle_command_reply: tid=0 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 client.26470 shutdown 2023-07-14T11:26:30.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 msgr2=0x5641c03f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03f2ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005f50 tx=0x7f1c8c005c70 comp rx=0 tx=0).stop 2023-07-14T11:26:30.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 shutdown_connections 2023-07-14T11:26:30.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c0401c50 0x5641c0404030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03f2ad0 0x5641c03f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e72d0 0x5641c03f2ea0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1614871532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03e7be0 0x5641c03f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1c7b7fe700 0 client.26470 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:26:30.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 >> 172.21.15.161:0/1614871532 conn(0x5641c03ddda0 msgr2=0x5641c03e3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 shutdown_connections 2023-07-14T11:26:30.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1614871532 wait complete. 2023-07-14T11:26:30.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.734+0000 7f1cb5e9cb80 20 client.26470 trim_cache size 0 max 16384 2023-07-14T11:26:30.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 Processor -- start 2023-07-14T11:26:30.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 -- start start 2023-07-14T11:26:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c03f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c03e85b0 con 0x5641c03f2ad0 2023-07-14T11:26:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c03e8d90 con 0x5641c03e72d0 2023-07-14T11:26:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c03dd9f0 con 0x5641c03e7be0 2023-07-14T11:26:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.749+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1ca0b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c03f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55894/0 (socket says 172.21.15.161:55894) 2023-07-14T11:26:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 -- 172.21.15.161:0/944890690 learned_addr learned my addr 172.21.15.161:0/944890690 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 -- 172.21.15.161:0/944890690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 msgr2=0x5641c03fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 -- 172.21.15.161:0/944890690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 msgr2=0x5641c03f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c03f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 -- 172.21.15.161:0/944890690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c018e250 con 0x5641c03e72d0 2023-07-14T11:26:30.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f7ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006010 tx=0x7f1c8c00ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c5490a9d68193e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c82ffd700 1 -- 172.21.15.161:0/944890690 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c09ae20 con 0x5641c03e72d0 2023-07-14T11:26:30.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c82ffd700 1 -- 172.21.15.161:0/944890690 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8c09b0b0 con 0x5641c03e72d0 2023-07-14T11:26:30.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.750+0000 7f1c82ffd700 1 -- 172.21.15.161:0/944890690 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c0b92b0 con 0x5641c03e72d0 2023-07-14T11:26:30.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/944890690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 msgr2=0x5641c03f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f7ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006010 tx=0x7f1c8c00ff90 comp rx=0 tx=0).stop 2023-07-14T11:26:30.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/944890690 shutdown_connections 2023-07-14T11:26:30.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c03f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f7ee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/944890690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/944890690 >> 172.21.15.161:0/944890690 conn(0x5641c03ddda0 msgr2=0x5641c03e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/944890690 shutdown_connections 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.751+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/944890690 wait complete. 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 -- start start 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 Processor -- start 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03f6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c044ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c044efb0 con 0x5641c03f2ad0 2023-07-14T11:26:30.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c044f120 con 0x5641c03e72d0 2023-07-14T11:26:30.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c044f2e0 con 0x5641c03e7be0 2023-07-14T11:26:30.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.752+0000 7f1cb5e9cb80 10 client.0 _ll_register_callbacks cb 0x7f1c94009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:30.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55902/0 (socket says 172.21.15.161:55902) 2023-07-14T11:26:30.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1504278020 learned_addr learned my addr 172.21.15.161:0/1504278020 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:30.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1ca0b9c700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c044ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1504278020 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 msgr2=0x5641c03f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1504278020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 msgr2=0x5641c044ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c044ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 -- 172.21.15.161:0/1504278020 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641c0150c60 con 0x5641c03e72d0 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c9bfff700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f5e20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0977e0 tx=0x7f1c8c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8065785b0a24cd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:30.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:26:30.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1504278020 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c0b9620 con 0x5641c03e72d0 2023-07-14T11:26:30.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c8c09f310 con 0x5641c03e72d0 2023-07-14T11:26:30.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.753+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1504278020 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f1c8c0a36a0 con 0x5641c03e72d0 2023-07-14T11:26:30.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1504278020 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c8c022890 con 0x5641c03e72d0 2023-07-14T11:26:30.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 10 client.26471 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:30.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5641c01662b0 con 0x5641c03e72d0 2023-07-14T11:26:30.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1504278020 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8c09f310 con 0x5641c03e72d0 2023-07-14T11:26:30.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1504278020 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c8c09a240 con 0x5641c03e72d0 2023-07-14T11:26:30.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 10 client.26471 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:30.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 10 client.26471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:26:30.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 20 client.26471 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:30.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c04565b0 0x5641c0458990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:30.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 4 client.26471 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:26:30.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5641c02f5050 con 0x5641c04565b0 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.754+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c04565b0 0x5641c0458990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1c9b7fe700 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c04565b0 0x5641c0458990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcfb2f5229d6bafb server_cookie=f717431d54154e9e in_seq=0 out_seq=0 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1c7b7fe700 10 client.26471 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1c7b7fe700 1 -- 172.21.15.161:0/1504278020 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5641c02f5050 con 0x5641c04565b0 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1c7b7fe700 10 client.26471 handle_command_reply: tid=0 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 client.26471 shutdown 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 msgr2=0x5641c03f5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f5e20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0977e0 tx=0x7f1c8c005c00 comp rx=0 tx=0).stop 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 shutdown_connections 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5641c04565b0 0x5641c0458990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641c03f2ad0 0x5641c044ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c03e72d0 0x5641c03f5e20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 --2- 172.21.15.161:0/1504278020 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5641c03e7be0 0x5641c03f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1c7b7fe700 0 client.26471 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 >> 172.21.15.161:0/1504278020 conn(0x5641c03ddda0 msgr2=0x5641c03f2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 shutdown_connections 2023-07-14T11:26:30.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.755+0000 7f1cb5e9cb80 1 -- 172.21.15.161:0/1504278020 wait complete. 2023-07-14T11:26:30.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1cb5e9cb80 20 client.26471 trim_cache size 0 max 16384 2023-07-14T11:26:30.768 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:26:30.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c840a12c0 msgr2=0x7f1c840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c840a12c0 0x7f1c840a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c199480 tx=0x7f1c9000b410 comp rx=0 tx=0).stop 2023-07-14T11:26:30.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 msgr2=0x7f1c9c197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:30.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c197e70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005910 tx=0x7f1c8c004730 comp rx=0 tx=0).stop 2023-07-14T11:26:30.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 shutdown_connections 2023-07-14T11:26:30.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c840a12c0 0x7f1c840a3770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c9c072850 0x7f1c9c197e70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c9c10c380 0x7f1c9c19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 --2- 172.21.15.161:0/2426636399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9c073160 0x7f1c9c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:30.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 >> 172.21.15.161:0/2426636399 conn(0x7f1c9c0fb5e0 msgr2=0x7f1c9c10e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:30.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 shutdown_connections 2023-07-14T11:26:30.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.756+0000 7f1ca25ff700 1 -- 172.21.15.161:0/2426636399 wait complete. 2023-07-14T11:26:30.779 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:26:30.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 -- 172.21.15.161:0/1375622377 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938102910 msgr2=0x7f59381068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 --2- 172.21.15.161:0/1375622377 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938102910 0x7f59381068d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f59280099c0 tx=0x7f5928009cd0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 -- 172.21.15.161:0/1375622377 shutdown_connections 2023-07-14T11:26:31.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 --2- 172.21.15.161:0/1375622377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5938101f70 0x7f5938102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 --2- 172.21.15.161:0/1375622377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938106fb0 0x7f593810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 --2- 172.21.15.161:0/1375622377 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938102910 0x7f59381068d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 -- 172.21.15.161:0/1375622377 >> 172.21.15.161:0/1375622377 conn(0x7f59380fd7e0 msgr2=0x7f59380ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 -- 172.21.15.161:0/1375622377 shutdown_connections 2023-07-14T11:26:31.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.995+0000 7f59405ff700 1 -- 172.21.15.161:0/1375622377 wait complete. 2023-07-14T11:26:31.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.996+0000 7f59405ff700 1 Processor -- start 2023-07-14T11:26:31.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.996+0000 7f59405ff700 1 -- start start 2023-07-14T11:26:31.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f59405ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5938101f70 0x7f5938197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f59405ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938102910 0x7f59381983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f59405ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938106fb0 0x7f593819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f59405ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593819ce50 con 0x7f5938101f70 2023-07-14T11:26:31.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f59405ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593819cfc0 con 0x7f5938102910 2023-07-14T11:26:31.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f59405ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593819d180 con 0x7f5938106fb0 2023-07-14T11:26:31.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938102910 0x7f59381983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593e39b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5938101f70 0x7f5938197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938102910 0x7f59381983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55920/0 (socket says 172.21.15.161:55920) 2023-07-14T11:26:31.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 -- 172.21.15.161:0/268363522 learned_addr learned my addr 172.21.15.161:0/268363522 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593eb9c700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938106fb0 0x7f593819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 -- 172.21.15.161:0/268363522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938106fb0 msgr2=0x7f593819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938106fb0 0x7f593819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 -- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5938101f70 msgr2=0x7f5938197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5938101f70 0x7f5938197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.997+0000 7f593db9a700 1 -- 172.21.15.161:0/268363522 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59280095f0 con 0x7f5938102910 2023-07-14T11:26:31.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.998+0000 7f593db9a700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938102910 0x7f59381983b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f59280098e0 tx=0x7f592800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.998+0000 7f592f7fe700 1 -- 172.21.15.161:0/268363522 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592801d070 con 0x7f5938102910 2023-07-14T11:26:31.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.998+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593819d400 con 0x7f5938102910 2023-07-14T11:26:31.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.998+0000 7f592f7fe700 1 -- 172.21.15.161:0/268363522 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5928005050 con 0x7f5938102910 2023-07-14T11:26:31.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.998+0000 7f592f7fe700 1 -- 172.21.15.161:0/268363522 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928004970 con 0x7f5938102910 2023-07-14T11:26:31.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:30.998+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59381a4ca0 con 0x7f5938102910 2023-07-14T11:26:31.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.001+0000 7f592f7fe700 1 -- 172.21.15.161:0/268363522 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5928005470 con 0x7f5938102910 2023-07-14T11:26:31.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 Processor -- start 2023-07-14T11:26:31.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 -- start start 2023-07-14T11:26:31.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f592f7fe700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59240a12c0 0x7f59240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f592f7fe700 1 -- 172.21.15.161:0/268363522 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f59280cdd40 con 0x7f5938102910 2023-07-14T11:26:31.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f593eb9c700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59240a12c0 0x7f59240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622de929cb0 con 0x5622dea232d0 2023-07-14T11:26:31.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622de97ca40 con 0x5622dea23be0 2023-07-14T11:26:31.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.003+0000 7f5953f59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622de975ed0 con 0x5622dea2ead0 2023-07-14T11:26:31.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea2e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55930/0 (socket says 172.21.15.161:55930) 2023-07-14T11:26:31.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 -- 172.21.15.161:0/2907234569 learned_addr learned my addr 172.21.15.161:0/2907234569 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593eb9c700 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593e39b700 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 -- 172.21.15.161:0/2907234569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 msgr2=0x5622dea30eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 -- 172.21.15.161:0/2907234569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 msgr2=0x5622dea236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea236a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 -- 172.21.15.161:0/2907234569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622de7a3de0 con 0x5622dea23be0 2023-07-14T11:26:31.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593e39b700 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:26:31.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593db9a700 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea2e590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f59280a29a0 tx=0x7f592809ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef9e113926b5e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f593eb9c700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59240a12c0 0x7f59240a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5930009ec0 tx=0x7f593000fc40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f592cbfe700 1 -- 172.21.15.161:0/2907234569 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280051c0 con 0x5622dea23be0 2023-07-14T11:26:31.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f592cbfe700 1 -- 172.21.15.161:0/2907234569 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f592809ad70 con 0x5622dea23be0 2023-07-14T11:26:31.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.004+0000 7f592cbfe700 1 -- 172.21.15.161:0/2907234569 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280a2ca0 con 0x5622dea23be0 2023-07-14T11:26:31.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 -- 172.21.15.161:0/2907234569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 msgr2=0x5622dea2e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea2e590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f59280a29a0 tx=0x7f592809ab90 comp rx=0 tx=0).stop 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 -- 172.21.15.161:0/2907234569 shutdown_connections 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2907234569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea2e590 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 -- 172.21.15.161:0/2907234569 >> 172.21.15.161:0/2907234569 conn(0x5622dea19da0 msgr2=0x5622dea1f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 -- 172.21.15.161:0/2907234569 shutdown_connections 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.005+0000 7f5953f59b80 1 -- 172.21.15.161:0/2907234569 wait complete. 2023-07-14T11:26:31.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.006+0000 7f5953f59b80 1 -- start start 2023-07-14T11:26:31.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.006+0000 7f5953f59b80 1 Processor -- start 2023-07-14T11:26:31.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea2eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea23be0 0x5622dea2f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea2f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f5953f59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622de915ac0 con 0x5622dea232d0 2023-07-14T11:26:31.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f5953f59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622de9157c0 con 0x5622dea2ead0 2023-07-14T11:26:31.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f5953f59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622de7f9920 con 0x5622dea23be0 2023-07-14T11:26:31.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f5953f59b80 10 client.0 _ll_register_callbacks cb 0x5622de913760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:31.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f593e39b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea2eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f593e39b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea2eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47908/0 (socket says 172.21.15.161:47908) 2023-07-14T11:26:31.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f593e39b700 1 -- 172.21.15.161:0/1491583984 learned_addr learned my addr 172.21.15.161:0/1491583984 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.007+0000 7f593db9a700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea23be0 0x5622dea2f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593eb9c700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea2f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593db9a700 1 -- 172.21.15.161:0/1491583984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 msgr2=0x5622dea2f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593db9a700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea2f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593db9a700 1 -- 172.21.15.161:0/1491583984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 msgr2=0x5622dea2eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593db9a700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea2eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593db9a700 1 -- 172.21.15.161:0/1491583984 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59381a4ca0 con 0x5622dea23be0 2023-07-14T11:26:31.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593e39b700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea2eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:31.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f593db9a700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea23be0 0x5622dea2f3e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5622dea24050 tx=0x7f592809a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b4a9b5311259c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f5921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:26:31.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f5921ffb700 1 -- 172.21.15.161:0/1491583984 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280b3220 con 0x5622dea23be0 2023-07-14T11:26:31.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f5921ffb700 1 -- 172.21.15.161:0/1491583984 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f59280a2a40 con 0x5622dea23be0 2023-07-14T11:26:31.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622de8cf4f0 con 0x5622dea23be0 2023-07-14T11:26:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.008+0000 7f5921ffb700 1 -- 172.21.15.161:0/1491583984 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59280a2c00 con 0x5622dea23be0 2023-07-14T11:26:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 10 client.26488 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5622de78cc60 con 0x5622dea23be0 2023-07-14T11:26:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5921ffb700 1 -- 172.21.15.161:0/1491583984 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622de8cf4f0 con 0x5622dea23be0 2023-07-14T11:26:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5921ffb700 1 -- 172.21.15.161:0/1491583984 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f59280c6050 con 0x5622dea23be0 2023-07-14T11:26:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 10 client.26488 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 10 client.26488 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 20 client.26488 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea3dc00 0x5622dea40010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 4 client.26488 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622de96eeb0 con 0x5622dea3dc00 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.009+0000 7f593e39b700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea3dc00 0x5622dea40010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.010+0000 7f5921ffb700 10 client.26488 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.010+0000 7f593e39b700 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea3dc00 0x5622dea40010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec5a4c38871eb668 server_cookie=1ae8d9b86eb8b616 in_seq=0 out_seq=0 2023-07-14T11:26:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5921ffb700 1 -- 172.21.15.161:0/1491583984 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5622de96eeb0 con 0x5622dea3dc00 2023-07-14T11:26:31.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5921ffb700 10 client.26488 handle_command_reply: tid=0 2023-07-14T11:26:31.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 client.26488 shutdown 2023-07-14T11:26:31.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea23be0 msgr2=0x5622dea2f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea23be0 0x5622dea2f3e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5622dea24050 tx=0x7f592809a650 comp rx=0 tx=0).stop 2023-07-14T11:26:31.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 shutdown_connections 2023-07-14T11:26:31.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea3dc00 0x5622dea40010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea2eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea2f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 --2- 172.21.15.161:0/1491583984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea23be0 0x5622dea2f3e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5921ffb700 0 client.26488 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:26:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 >> 172.21.15.161:0/1491583984 conn(0x5622dea19da0 msgr2=0x5622dea1f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 shutdown_connections 2023-07-14T11:26:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 1 -- 172.21.15.161:0/1491583984 wait complete. 2023-07-14T11:26:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.011+0000 7f5953f59b80 20 client.26488 trim_cache size 0 max 16384 2023-07-14T11:26:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 Processor -- start 2023-07-14T11:26:31.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 -- start start 2023-07-14T11:26:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea232d0 0x5622dea32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea23be0 0x5622dea1de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea28f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dea19970 con 0x5622dea23be0 2023-07-14T11:26:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dea2e180 con 0x5622dea2ead0 2023-07-14T11:26:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f5953f59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dea33410 con 0x5622dea232d0 2023-07-14T11:26:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f593e39b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea232d0 0x5622dea32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.025+0000 7f593eb9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea28f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea232d0 0x5622dea32ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50300/0 (socket says 172.21.15.161:50300) 2023-07-14T11:26:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593eb9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea28f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55952/0 (socket says 172.21.15.161:55952) 2023-07-14T11:26:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 -- 172.21.15.161:0/2695560277 learned_addr learned my addr 172.21.15.161:0/2695560277 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593db9a700 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea23be0 0x5622dea1de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 -- 172.21.15.161:0/2695560277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 msgr2=0x5622dea28f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea28f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 -- 172.21.15.161:0/2695560277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea23be0 msgr2=0x5622dea1de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea23be0 0x5622dea1de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 -- 172.21.15.161:0/2695560277 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622dea1b5e0 con 0x5622dea232d0 2023-07-14T11:26:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f593e39b700 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea232d0 0x5622dea32ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5934000c00 tx=0x7f5934015980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90acc5503aeb5911 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f592d3ff700 1 -- 172.21.15.161:0/2695560277 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934018040 con 0x5622dea232d0 2023-07-14T11:26:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f592d3ff700 1 -- 172.21.15.161:0/2695560277 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5934013310 con 0x5622dea232d0 2023-07-14T11:26:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.026+0000 7f592d3ff700 1 -- 172.21.15.161:0/2695560277 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593401de50 con 0x5622dea232d0 2023-07-14T11:26:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 -- 172.21.15.161:0/2695560277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea232d0 msgr2=0x5622dea32ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea232d0 0x5622dea32ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5934000c00 tx=0x7f5934015980 comp rx=0 tx=0).stop 2023-07-14T11:26:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 -- 172.21.15.161:0/2695560277 shutdown_connections 2023-07-14T11:26:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea2ead0 0x5622dea28f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea232d0 0x5622dea32ed0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 --2- 172.21.15.161:0/2695560277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea23be0 0x5622dea1de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 -- 172.21.15.161:0/2695560277 >> 172.21.15.161:0/2695560277 conn(0x5622dea19da0 msgr2=0x5622dea28420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 -- 172.21.15.161:0/2695560277 shutdown_connections 2023-07-14T11:26:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 -- 172.21.15.161:0/2695560277 wait complete. 2023-07-14T11:26:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.027+0000 7f5953f59b80 1 -- start start 2023-07-14T11:26:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 1 Processor -- start 2023-07-14T11:26:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea32730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea32c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dea31200 con 0x5622dea232d0 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dea31370 con 0x5622dea23be0 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dea31530 con 0x5622dea2ead0 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f5953f59b80 10 client.0 _ll_register_callbacks cb 0x5622de913760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f593db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea32c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f593eb9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f593db9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea32c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55954/0 (socket says 172.21.15.161:55954) 2023-07-14T11:26:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.028+0000 7f593db9a700 1 -- 172.21.15.161:0/616455524 learned_addr learned my addr 172.21.15.161:0/616455524 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593e39b700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea32730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593db9a700 1 -- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 msgr2=0x5622dea30b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593db9a700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593db9a700 1 -- 172.21.15.161:0/616455524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 msgr2=0x5622dea32730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593db9a700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea32730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593db9a700 1 -- 172.21.15.161:0/616455524 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5934013040 con 0x5622dea23be0 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593eb9c700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593e39b700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea32730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f593db9a700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea32c70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f592809aaa0 tx=0x7f59280a2cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5c9aedd40ff7fd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f5921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f5921ffb700 1 -- 172.21.15.161:0/616455524 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592809a8e0 con 0x5622dea23be0 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f5921ffb700 1 -- 172.21.15.161:0/616455524 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f592800fce0 con 0x5622dea23be0 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f592809ad70 con 0x5622dea23be0 2023-07-14T11:26:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f5921ffb700 1 -- 172.21.15.161:0/616455524 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f592809ebc0 con 0x5622dea23be0 2023-07-14T11:26:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f5953f59b80 10 client.26489 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.029+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5622de7ca250 con 0x5622dea23be0 2023-07-14T11:26:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5921ffb700 1 -- 172.21.15.161:0/616455524 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f592809ad70 con 0x5622dea23be0 2023-07-14T11:26:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5921ffb700 1 -- 172.21.15.161:0/616455524 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f59280c6020 con 0x5622dea23be0 2023-07-14T11:26:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5953f59b80 10 client.26489 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5953f59b80 10 client.26489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:26:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5953f59b80 20 client.26489 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5953f59b80 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea92700 0x5622dea94ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5953f59b80 4 client.26489 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622de931050 con 0x5622dea92700 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.030+0000 7f593e39b700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea92700 0x5622dea94ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f593e39b700 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea92700 0x5622dea94ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43090c2401d539c4 server_cookie=d93d1a5310258450 in_seq=0 out_seq=0 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5921ffb700 10 client.26489 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5921ffb700 1 -- 172.21.15.161:0/616455524 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622de931050 con 0x5622dea92700 2023-07-14T11:26:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5921ffb700 10 client.26489 handle_command_reply: tid=0 2023-07-14T11:26:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 client.26489 shutdown 2023-07-14T11:26:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 msgr2=0x5622dea32c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea32c70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f592809aaa0 tx=0x7f59280a2cf0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 shutdown_connections 2023-07-14T11:26:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5622dea92700 0x5622dea94ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622dea232d0 0x5622dea32730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5622dea2ead0 0x5622dea30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 --2- 172.21.15.161:0/616455524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622dea23be0 0x5622dea32c70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5921ffb700 0 client.26489 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:26:31.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 >> 172.21.15.161:0/616455524 conn(0x5622dea19da0 msgr2=0x5622dea2c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 shutdown_connections 2023-07-14T11:26:31.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 1 -- 172.21.15.161:0/616455524 wait complete. 2023-07-14T11:26:31.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.031+0000 7f5953f59b80 20 client.26489 trim_cache size 0 max 16384 2023-07-14T11:26:31.045 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:26:31.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59240a12c0 msgr2=0x7f59240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59240a12c0 0x7f59240a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5930009ec0 tx=0x7f593000fc40 comp rx=0 tx=0).stop 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938102910 msgr2=0x7f59381983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938102910 0x7f59381983b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f59280098e0 tx=0x7f592800f690 comp rx=0 tx=0).stop 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 shutdown_connections 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f59240a12c0 0x7f59240a3770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5938101f70 0x7f5938197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5938106fb0 0x7f593819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 --2- 172.21.15.161:0/268363522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5938102910 0x7f59381983b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 >> 172.21.15.161:0/268363522 conn(0x7f59380fd7e0 msgr2=0x7f59380fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 shutdown_connections 2023-07-14T11:26:31.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.032+0000 7f59405ff700 1 -- 172.21.15.161:0/268363522 wait complete. 2023-07-14T11:26:31.055 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:26:31.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 361951 9.87 MB/sec execute 73 sec latency 14.554 ms 2023-07-14T11:26:31.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 -- 172.21.15.161:0/997883556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 msgr2=0x7f32dc10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 --2- 172.21.15.161:0/997883556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc10c920 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b0a0 tx=0x7f32d400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 -- 172.21.15.161:0/997883556 shutdown_connections 2023-07-14T11:26:31.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 --2- 172.21.15.161:0/997883556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc10c920 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 --2- 172.21.15.161:0/997883556 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32dc105af0 0x7f32dc105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 --2- 172.21.15.161:0/997883556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32dc0ffae0 0x7f32dc0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 -- 172.21.15.161:0/997883556 >> 172.21.15.161:0/997883556 conn(0x7f32dc0fb5e0 msgr2=0x7f32dc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 -- 172.21.15.161:0/997883556 shutdown_connections 2023-07-14T11:26:31.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.263+0000 7f32e33ff700 1 -- 172.21.15.161:0/997883556 wait complete. 2023-07-14T11:26:31.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.264+0000 7f32e33ff700 1 Processor -- start 2023-07-14T11:26:31.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.264+0000 7f32e33ff700 1 -- start start 2023-07-14T11:26:31.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e33ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32dc0ffae0 0x7f32dc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e33ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e33ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32dc105af0 0x7f32dc19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e33ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc19cf20 con 0x7f32dc105af0 2023-07-14T11:26:31.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e33ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc19d090 con 0x7f32dc100490 2023-07-14T11:26:31.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e33ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc19d250 con 0x7f32dc0ffae0 2023-07-14T11:26:31.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e099a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e099a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55978/0 (socket says 172.21.15.161:55978) 2023-07-14T11:26:31.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e099a700 1 -- 172.21.15.161:0/2279509409 learned_addr learned my addr 172.21.15.161:0/2279509409 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e119b700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32dc0ffae0 0x7f32dc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.265+0000 7f32e199c700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32dc105af0 0x7f32dc19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e119b700 1 -- 172.21.15.161:0/2279509409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 msgr2=0x7f32dc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e119b700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e119b700 1 -- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32dc105af0 msgr2=0x7f32dc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e119b700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32dc105af0 0x7f32dc19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e119b700 1 -- 172.21.15.161:0/2279509409 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4009d00 con 0x7f32dc0ffae0 2023-07-14T11:26:31.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e099a700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:31.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e119b700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32dc0ffae0 0x7f32dc197f40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e9f0 tx=0x7f32d800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32d27fc700 1 -- 172.21.15.161:0/2279509409 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d800c9c0 con 0x7f32dc0ffae0 2023-07-14T11:26:31.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc19d530 con 0x7f32dc0ffae0 2023-07-14T11:26:31.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32d27fc700 1 -- 172.21.15.161:0/2279509409 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32d8010430 con 0x7f32dc0ffae0 2023-07-14T11:26:31.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32d27fc700 1 -- 172.21.15.161:0/2279509409 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8018680 con 0x7f32dc0ffae0 2023-07-14T11:26:31.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.266+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc1a4d70 con 0x7f32dc0ffae0 2023-07-14T11:26:31.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.270+0000 7f32d27fc700 1 -- 172.21.15.161:0/2279509409 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f32d8010860 con 0x7f32dc0ffae0 2023-07-14T11:26:31.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.271+0000 7f32d27fc700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32c80a12c0 0x7f32c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.271+0000 7f32f6d00b80 1 Processor -- start 2023-07-14T11:26:31.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.271+0000 7f32d27fc700 1 -- 172.21.15.161:0/2279509409 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f32d8014070 con 0x7f32dc0ffae0 2023-07-14T11:26:31.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.271+0000 7f32e099a700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32c80a12c0 0x7f32c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.271+0000 7f32f6d00b80 1 -- start start 2023-07-14T11:26:31.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a17c7cb0 con 0x5571a18c12d0 2023-07-14T11:26:31.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a181aa40 con 0x5571a18ccad0 2023-07-14T11:26:31.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a1813ed0 con 0x5571a18c1be0 2023-07-14T11:26:31.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e099a700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32c80a12c0 0x7f32c80a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32dc199550 tx=0x7f32cc009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e099a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55980/0 (socket says 172.21.15.161:55980) 2023-07-14T11:26:31.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 -- 172.21.15.161:0/2664101127 learned_addr learned my addr 172.21.15.161:0/2664101127 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 -- 172.21.15.161:0/2664101127 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 msgr2=0x5571a18cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 -- 172.21.15.161:0/2664101127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 msgr2=0x5571a18c16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e199c700 1 -- 172.21.15.161:0/2664101127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a1641de0 con 0x5571a18ccad0 2023-07-14T11:26:31.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.272+0000 7f32e099a700 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:31.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.273+0000 7f32e199c700 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18ceeb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32d4015000 tx=0x7f32d4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a4e5996b2809479 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.273+0000 7f32c77fe700 1 -- 172.21.15.161:0/2664101127 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d400e040 con 0x5571a18ccad0 2023-07-14T11:26:31.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.273+0000 7f32c77fe700 1 -- 172.21.15.161:0/2664101127 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32d400bcb0 con 0x5571a18ccad0 2023-07-14T11:26:31.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.273+0000 7f32c77fe700 1 -- 172.21.15.161:0/2664101127 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d401c6a0 con 0x5571a18ccad0 2023-07-14T11:26:31.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2664101127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 msgr2=0x5571a18ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18ceeb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32d4015000 tx=0x7f32d4009c20 comp rx=0 tx=0).stop 2023-07-14T11:26:31.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2664101127 shutdown_connections 2023-07-14T11:26:31.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18ceeb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2664101127 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2664101127 >> 172.21.15.161:0/2664101127 conn(0x5571a18b7da0 msgr2=0x5571a18bd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2664101127 shutdown_connections 2023-07-14T11:26:31.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2664101127 wait complete. 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.274+0000 7f32f6d00b80 1 -- start start 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 1 Processor -- start 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c12d0 0x5571a18cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c1be0 0x5571a18c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a17b3ac0 con 0x5571a18c1be0 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a17b37c0 con 0x5571a18ccad0 2023-07-14T11:26:31.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a1697920 con 0x5571a18c12d0 2023-07-14T11:26:31.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32f6d00b80 10 client.0 _ll_register_callbacks cb 0x5571a17b1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:31.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32e119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c12d0 0x5571a18cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32e199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32e119b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c12d0 0x5571a18cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50362/0 (socket says 172.21.15.161:50362) 2023-07-14T11:26:31.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32e119b700 1 -- 172.21.15.161:0/2443297126 learned_addr learned my addr 172.21.15.161:0/2443297126 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.275+0000 7f32e099a700 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c1be0 0x5571a18c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32e119b700 1 -- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 msgr2=0x5571a18c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32e119b700 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32e119b700 1 -- 172.21.15.161:0/2443297126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c1be0 msgr2=0x5571a18c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32e119b700 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c1be0 0x5571a18c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32e119b700 1 -- 172.21.15.161:0/2443297126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc017070 con 0x5571a18c12d0 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32e199c700 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32e119b700 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c12d0 0x5571a18cc550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e9c0 tx=0x7f32d8099f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b0c085179e0a1f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:26:31.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32c4ff9700 1 -- 172.21.15.161:0/2443297126 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8099570 con 0x5571a18c12d0 2023-07-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32c4ff9700 1 -- 172.21.15.161:0/2443297126 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f32d80a22d0 con 0x5571a18c12d0 2023-07-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a176d4f0 con 0x5571a18c12d0 2023-07-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32c4ff9700 1 -- 172.21.15.161:0/2443297126 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32d80a1e40 con 0x5571a18c12d0 2023-07-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32f6d00b80 10 client.26506 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.276+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f32dc10d220 con 0x5571a18c12d0 2023-07-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32c4ff9700 1 -- 172.21.15.161:0/2443297126 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571a176d4f0 con 0x5571a18c12d0 2023-07-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32c4ff9700 1 -- 172.21.15.161:0/2443297126 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f32d8095c70 con 0x5571a18c12d0 2023-07-14T11:26:31.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32f6d00b80 10 client.26506 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:31.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32f6d00b80 10 client.26506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:31.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32f6d00b80 20 client.26506 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:31.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a18dc370 0x5571a18de780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32f6d00b80 4 client.26506 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:26:31.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571a180ceb0 con 0x5571a18dc370 2023-07-14T11:26:31.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.277+0000 7f32e099a700 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a18dc370 0x5571a18de780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.278+0000 7f32e099a700 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a18dc370 0x5571a18de780 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc71bf3d4dca1c3 server_cookie=ffcb62c9045b8344 in_seq=0 out_seq=0 2023-07-14T11:26:31.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.278+0000 7f32c4ff9700 10 client.26506 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:31.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.278+0000 7f32c4ff9700 1 -- 172.21.15.161:0/2443297126 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571a180ceb0 con 0x5571a18dc370 2023-07-14T11:26:31.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.278+0000 7f32c4ff9700 10 client.26506 handle_command_reply: tid=0 2023-07-14T11:26:31.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.278+0000 7f32f6d00b80 1 client.26506 shutdown 2023-07-14T11:26:31.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c12d0 msgr2=0x5571a18cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c12d0 0x5571a18cc550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e9c0 tx=0x7f32d8099f00 comp rx=0 tx=0).stop 2023-07-14T11:26:31.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 shutdown_connections 2023-07-14T11:26:31.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a18dc370 0x5571a18de780 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32c4ff9700 0 client.26506 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:31.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c12d0 0x5571a18cc550 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/2443297126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c1be0 0x5571a18c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 >> 172.21.15.161:0/2443297126 conn(0x5571a18b7da0 msgr2=0x5571a18bd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 shutdown_connections 2023-07-14T11:26:31.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 1 -- 172.21.15.161:0/2443297126 wait complete. 2023-07-14T11:26:31.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.279+0000 7f32f6d00b80 20 client.26506 trim_cache size 0 max 16384 2023-07-14T11:26:31.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.293+0000 7f32f6d00b80 1 Processor -- start 2023-07-14T11:26:31.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.293+0000 7f32f6d00b80 1 -- start start 2023-07-14T11:26:31.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18c1be0 0x5571a18cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18ccad0 0x5571a18c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a18bc490 con 0x5571a18c12d0 2023-07-14T11:26:31.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a18c8d40 con 0x5571a18c1be0 2023-07-14T11:26:31.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a18c8f00 con 0x5571a18ccad0 2023-07-14T11:26:31.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18c1be0 0x5571a18cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e199c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18ccad0 0x5571a18c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18c1be0 0x5571a18cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56006/0 (socket says 172.21.15.161:56006) 2023-07-14T11:26:31.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e199c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18ccad0 0x5571a18c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50376/0 (socket says 172.21.15.161:50376) 2023-07-14T11:26:31.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 -- 172.21.15.161:0/1140215435 learned_addr learned my addr 172.21.15.161:0/1140215435 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 -- 172.21.15.161:0/1140215435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18ccad0 msgr2=0x5571a18c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 --2- 172.21.15.161:0/1140215435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18ccad0 0x5571a18c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 -- 172.21.15.161:0/1140215435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 msgr2=0x5571a18c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 --2- 172.21.15.161:0/1140215435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 -- 172.21.15.161:0/1140215435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a162ac60 con 0x5571a18c1be0 2023-07-14T11:26:31.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.294+0000 7f32e099a700 1 --2- 172.21.15.161:0/1140215435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18c1be0 0x5571a18cc080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009cf0 tx=0x7f32cc013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c184095080f488a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32c7fff700 1 -- 172.21.15.161:0/1140215435 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc022070 con 0x5571a18c1be0 2023-07-14T11:26:31.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32c7fff700 1 -- 172.21.15.161:0/1140215435 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32cc023040 con 0x5571a18c1be0 2023-07-14T11:26:31.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32c7fff700 1 -- 172.21.15.161:0/1140215435 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc004d30 con 0x5571a18c1be0 2023-07-14T11:26:31.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1140215435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18c1be0 msgr2=0x5571a18cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1140215435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18c1be0 0x5571a18cc080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009cf0 tx=0x7f32cc013040 comp rx=0 tx=0).stop 2023-07-14T11:26:31.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1140215435 shutdown_connections 2023-07-14T11:26:31.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1140215435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1140215435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18ccad0 0x5571a18c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1140215435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18c1be0 0x5571a18cc080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1140215435 >> 172.21.15.161:0/1140215435 conn(0x5571a18b7da0 msgr2=0x5571a18c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1140215435 shutdown_connections 2023-07-14T11:26:31.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1140215435 wait complete. 2023-07-14T11:26:31.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 -- start start 2023-07-14T11:26:31.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.296+0000 7f32f6d00b80 1 Processor -- start 2023-07-14T11:26:31.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32f6d00b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a18d1a90 con 0x5571a18c12d0 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a18d1c00 con 0x5571a18ccad0 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32f6d00b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a18d1dc0 con 0x5571a18c1be0 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32f6d00b80 10 client.0 _ll_register_callbacks cb 0x7f32d8099a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56008/0 (socket says 172.21.15.161:56008) 2023-07-14T11:26:31.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 -- 172.21.15.161:0/1291266957 learned_addr learned my addr 172.21.15.161:0/1291266957 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:31.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e099a700 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 -- 172.21.15.161:0/1291266957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 msgr2=0x5571a18d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18d32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 -- 172.21.15.161:0/1291266957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 msgr2=0x5571a18d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:31.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.297+0000 7f32e199c700 1 -- 172.21.15.161:0/1291266957 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc02bab0 con 0x5571a18ccad0 2023-07-14T11:26:31.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32e199c700 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18d1490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009fd0 tx=0x7f32d4003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a500211dc2fe6177 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:31.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:26:31.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 1 -- 172.21.15.161:0/1291266957 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d400e040 con 0x5571a18ccad0 2023-07-14T11:26:31.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32d400bcb0 con 0x5571a18ccad0 2023-07-14T11:26:31.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 1 -- 172.21.15.161:0/1291266957 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f32d4029070 con 0x5571a18ccad0 2023-07-14T11:26:31.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 1 -- 172.21.15.161:0/1291266957 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32d400b7b0 con 0x5571a18ccad0 2023-07-14T11:26:31.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 10 client.26507 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:31.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5571a1668250 con 0x5571a18ccad0 2023-07-14T11:26:31.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 1 -- 172.21.15.161:0/1291266957 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32d400bcb0 con 0x5571a18ccad0 2023-07-14T11:26:31.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 1 -- 172.21.15.161:0/1291266957 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f32d4019030 con 0x5571a18ccad0 2023-07-14T11:26:31.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 10 client.26507 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:31.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 10 client.26507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:31.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 20 client.26507 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:31.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a1939730 0x5571a193bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:31.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 4 client.26507 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5571a17cf050 con 0x5571a1939730 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32e119b700 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a1939730 0x5571a193bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32e119b700 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a1939730 0x5571a193bb40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27fdb37ee2308c08 server_cookie=7d0a6dc27b300a3a in_seq=0 out_seq=0 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 10 client.26507 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 1 -- 172.21.15.161:0/1291266957 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5571a17cf050 con 0x5571a1939730 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32c4ff9700 10 client.26507 handle_command_reply: tid=0 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.299+0000 7f32f6d00b80 1 client.26507 shutdown 2023-07-14T11:26:31.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 msgr2=0x5571a18d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18d1490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009fd0 tx=0x7f32d4003f60 comp rx=0 tx=0).stop 2023-07-14T11:26:31.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 shutdown_connections 2023-07-14T11:26:31.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5571a1939730 0x5571a193bb40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5571a18c12d0 0x5571a18d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32c4ff9700 0 client.26507 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:31.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571a18ccad0 0x5571a18d1490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 --2- 172.21.15.161:0/1291266957 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5571a18c1be0 0x5571a18d32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 >> 172.21.15.161:0/1291266957 conn(0x5571a18b7da0 msgr2=0x5571a18c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 shutdown_connections 2023-07-14T11:26:31.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 1 -- 172.21.15.161:0/1291266957 wait complete. 2023-07-14T11:26:31.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32f6d00b80 20 client.26507 trim_cache size 0 max 16384 2023-07-14T11:26:31.313 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:26:31.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32c80a12c0 msgr2=0x7f32c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32c80a12c0 0x7f32c80a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32dc199550 tx=0x7f32cc009260 comp rx=0 tx=0).stop 2023-07-14T11:26:31.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32dc0ffae0 msgr2=0x7f32dc197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:31.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32dc0ffae0 0x7f32dc197f40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e9f0 tx=0x7f32d800ed00 comp rx=0 tx=0).stop 2023-07-14T11:26:31.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 shutdown_connections 2023-07-14T11:26:31.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32c80a12c0 0x7f32c80a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32dc105af0 0x7f32dc19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32dc0ffae0 0x7f32dc197f40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 --2- 172.21.15.161:0/2279509409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc100490 0x7f32dc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:31.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 >> 172.21.15.161:0/2279509409 conn(0x7f32dc0fb5e0 msgr2=0x7f32dc10b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:31.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 shutdown_connections 2023-07-14T11:26:31.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:31.301+0000 7f32e33ff700 1 -- 172.21.15.161:0/2279509409 wait complete. 2023-07-14T11:26:31.321 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:31 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3937097207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:26:31.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:31 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3937097207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:26:31.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:31 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3937097207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:26:32.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 362975 9.78 MB/sec execute 74 sec latency 56.119 ms 2023-07-14T11:26:32.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:26:32.325 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:26:32.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:32 smithi182 ceph-mon[129366]: pgmap v131: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 318 op/s 2023-07-14T11:26:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:32 smithi161 ceph-mon[95526]: pgmap v131: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 318 op/s 2023-07-14T11:26:32.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 -- 172.21.15.161:0/3366963338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 msgr2=0x7fbb5810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/3366963338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb5810c910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009f90 tx=0x7fbb5000b330 comp rx=0 tx=0).stop 2023-07-14T11:26:32.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 -- 172.21.15.161:0/3366963338 shutdown_connections 2023-07-14T11:26:32.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/3366963338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb58105ae0 0x7fbb58105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/3366963338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb5810c910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/3366963338 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbb580ffad0 0x7fbb580fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 -- 172.21.15.161:0/3366963338 >> 172.21.15.161:0/3366963338 conn(0x7fbb580fb5f0 msgr2=0x7fbb580fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 -- 172.21.15.161:0/3366963338 shutdown_connections 2023-07-14T11:26:32.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.540+0000 7fbb5dfff700 1 -- 172.21.15.161:0/3366963338 wait complete. 2023-07-14T11:26:32.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.541+0000 7fbb5dfff700 1 Processor -- start 2023-07-14T11:26:32.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.541+0000 7fbb5dfff700 1 -- start start 2023-07-14T11:26:32.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.541+0000 7fbb5dfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb580ffad0 0x7fbb58197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb5dfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb58198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb5dfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbb58105ae0 0x7fbb5819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb5dfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5819cf20 con 0x7fbb580ffad0 2023-07-14T11:26:32.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb5dfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5819d090 con 0x7fbb58100480 2023-07-14T11:26:32.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb5dfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5819d250 con 0x7fbb58105ae0 2023-07-14T11:26:32.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb58198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb58198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56026/0 (socket says 172.21.15.161:56026) 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2684096981 learned_addr learned my addr 172.21.15.161:0/2684096981 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbb58105ae0 0x7fbb5819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb577fe700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb580ffad0 0x7fbb58197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2684096981 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbb58105ae0 msgr2=0x7fbb5819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbb58105ae0 0x7fbb5819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb580ffad0 msgr2=0x7fbb58197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb580ffad0 0x7fbb58197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2684096981 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c0095f0 con 0x7fbb58100480 2023-07-14T11:26:32.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb577fe700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb580ffad0 0x7fbb58197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:26:32.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbb58105ae0 0x7fbb5819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:32.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.542+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb58198480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00ec10 tx=0x7fbb4c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.543+0000 7fbb54ff9700 1 -- 172.21.15.161:0/2684096981 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00cd40 con 0x7fbb58100480 2023-07-14T11:26:32.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.543+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb50009c40 con 0x7fbb58100480 2023-07-14T11:26:32.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.543+0000 7fbb54ff9700 1 -- 172.21.15.161:0/2684096981 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4c0044e0 con 0x7fbb58100480 2023-07-14T11:26:32.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.543+0000 7fbb54ff9700 1 -- 172.21.15.161:0/2684096981 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c010690 con 0x7fbb58100480 2023-07-14T11:26:32.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.543+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5819d720 con 0x7fbb58100480 2023-07-14T11:26:32.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.546+0000 7fbb54ff9700 1 -- 172.21.15.161:0/2684096981 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fbb4c003710 con 0x7fbb58100480 2023-07-14T11:26:32.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.547+0000 7fbb71a16b80 1 Processor -- start 2023-07-14T11:26:32.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.547+0000 7fbb71a16b80 1 -- start start 2023-07-14T11:26:32.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb54ff9700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbb400a1340 0x7fbb400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed82720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaed8d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed8db50 0x55faaed8ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaec88d30 con 0x55faaed8db50 2023-07-14T11:26:32.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaecdbac0 con 0x55faaed82350 2023-07-14T11:26:32.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaecd4f50 con 0x55faaed82c60 2023-07-14T11:26:32.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb54ff9700 1 -- 172.21.15.161:0/2684096981 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fbb4c014070 con 0x7fbb58100480 2023-07-14T11:26:32.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaed8d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb577fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed82720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaed8d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50418/0 (socket says 172.21.15.161:50418) 2023-07-14T11:26:32.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb577fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed82720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56038/0 (socket says 172.21.15.161:56038) 2023-07-14T11:26:32.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2781076328 learned_addr learned my addr 172.21.15.161:0/2781076328 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbb400a1340 0x7fbb400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed8db50 0x55faaed8ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2781076328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 msgr2=0x55faaed82720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed82720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2781076328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed8db50 msgr2=0x55faaed8ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed8db50 0x55faaed8ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.548+0000 7fbb56ffd700 1 -- 172.21.15.161:0/2781076328 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faaeb02e60 con 0x55faaed82c60 2023-07-14T11:26:32.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.549+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaed8d610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0a2210 tx=0x7fbb4c099e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e873f23c067c79eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.549+0000 7fbb3dffb700 1 -- 172.21.15.161:0/2781076328 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c077ba0 con 0x55faaed82c60 2023-07-14T11:26:32.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.549+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbb400a1340 0x7fbb400a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb500074d0 tx=0x7fbb50007420 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.549+0000 7fbb3dffb700 1 -- 172.21.15.161:0/2781076328 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4c0779c0 con 0x55faaed82c60 2023-07-14T11:26:32.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.549+0000 7fbb3dffb700 1 -- 172.21.15.161:0/2781076328 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c09e390 con 0x55faaed82c60 2023-07-14T11:26:32.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2781076328 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 msgr2=0x55faaed8d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaed8d610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0a2210 tx=0x7fbb4c099e30 comp rx=0 tx=0).stop 2023-07-14T11:26:32.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2781076328 shutdown_connections 2023-07-14T11:26:32.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed8db50 0x55faaed8ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed82720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2781076328 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaed8d610 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2781076328 >> 172.21.15.161:0/2781076328 conn(0x55faaed78e20 msgr2=0x55faaed7e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2781076328 shutdown_connections 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2781076328 wait complete. 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.550+0000 7fbb71a16b80 1 -- start start 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 1 Processor -- start 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaed8d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82c60 0x55faaed8a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed8ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaec74b40 con 0x55faaed82350 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaec74840 con 0x55faaed82c60 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaeb589a0 con 0x55faaed8db50 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb71a16b80 10 client.0 _ll_register_callbacks cb 0x55faaec727e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb57fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed8ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaed8d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82c60 0x55faaed8a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb57fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed8ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50430/0 (socket says 172.21.15.161:50430) 2023-07-14T11:26:32.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.551+0000 7fbb57fff700 1 -- 172.21.15.161:0/2816138528 learned_addr learned my addr 172.21.15.161:0/2816138528 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb57fff700 1 -- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82c60 msgr2=0x55faaed8a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82c60 0x55faaed8a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb57fff700 1 -- 172.21.15.161:0/2816138528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 msgr2=0x55faaed8d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaed8d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb57fff700 1 -- 172.21.15.161:0/2816138528 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5819d720 con 0x55faaed8db50 2023-07-14T11:26:32.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82c60 0x55faaed8a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:32.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb57fff700 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed8ae80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbb50015000 tx=0x7fbb5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57fc8ba92a797c59 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:26:32.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb36ffd700 1 -- 172.21.15.161:0/2816138528 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50027070 con 0x55faaed8db50 2023-07-14T11:26:32.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb36ffd700 1 -- 172.21.15.161:0/2816138528 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fbb50005660 con 0x55faaed8db50 2023-07-14T11:26:32.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.552+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faaec2e570 con 0x55faaed8db50 2023-07-14T11:26:32.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb36ffd700 1 -- 172.21.15.161:0/2816138528 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb5000e040 con 0x55faaed8db50 2023-07-14T11:26:32.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 10 client.26524 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:32.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55faaeaebce0 con 0x55faaed8db50 2023-07-14T11:26:32.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb36ffd700 1 -- 172.21.15.161:0/2816138528 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55faaec2e570 con 0x55faaed8db50 2023-07-14T11:26:32.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb36ffd700 1 -- 172.21.15.161:0/2816138528 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb5002cbf0 con 0x55faaed8db50 2023-07-14T11:26:32.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 10 client.26524 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:32.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 10 client.26524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:32.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 20 client.26524 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:32.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaed9d440 0x55faaed9f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 4 client.26524 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:26:32.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.553+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faaeccdf30 con 0x55faaed9d440 2023-07-14T11:26:32.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.554+0000 7fbb577fe700 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaed9d440 0x55faaed9f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.554+0000 7fbb36ffd700 10 client.26524 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.554+0000 7fbb577fe700 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaed9d440 0x55faaed9f850 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aabba0f35db2aff server_cookie=126acf4f7af5f006 in_seq=0 out_seq=0 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb36ffd700 1 -- 172.21.15.161:0/2816138528 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55faaeccdf30 con 0x55faaed9d440 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb36ffd700 10 client.26524 handle_command_reply: tid=0 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 client.26524 shutdown 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 msgr2=0x55faaed8ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed8ae80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbb50015000 tx=0x7fbb5000b330 comp rx=0 tx=0).stop 2023-07-14T11:26:32.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 shutdown_connections 2023-07-14T11:26:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaed9d440 0x55faaed9f850 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaed8d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb36ffd700 0 client.26524 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed8ae80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/2816138528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82c60 0x55faaed8a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 >> 172.21.15.161:0/2816138528 conn(0x55faaed78e20 msgr2=0x55faaed7e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 shutdown_connections 2023-07-14T11:26:32.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 1 -- 172.21.15.161:0/2816138528 wait complete. 2023-07-14T11:26:32.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.555+0000 7fbb71a16b80 20 client.26524 trim_cache size 0 max 16384 2023-07-14T11:26:32.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 Processor -- start 2023-07-14T11:26:32.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 -- start start 2023-07-14T11:26:32.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed94570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82c60 0x55faaed8c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed89fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaed84220 con 0x55faaed82c60 2023-07-14T11:26:32.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaed88d80 con 0x55faaed82350 2023-07-14T11:26:32.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaed94ab0 con 0x55faaed8db50 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb577fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed94570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb57fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed89fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb577fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed94570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56050/0 (socket says 172.21.15.161:56050) 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb577fe700 1 -- 172.21.15.161:0/1515267574 learned_addr learned my addr 172.21.15.161:0/1515267574 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.574+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82c60 0x55faaed8c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb577fe700 1 -- 172.21.15.161:0/1515267574 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 msgr2=0x55faaed89fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb577fe700 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed89fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb577fe700 1 -- 172.21.15.161:0/1515267574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82c60 msgr2=0x55faaed8c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb577fe700 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82c60 0x55faaed8c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb577fe700 1 -- 172.21.15.161:0/1515267574 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faaeb292d0 con 0x55faaed82350 2023-07-14T11:26:32.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82c60 0x55faaed8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:26:32.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb577fe700 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed94570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55faaeda00f0 tx=0x7fbb48009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0608bb8636b9589 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb3e7fc700 1 -- 172.21.15.161:0/1515267574 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48016070 con 0x55faaed82350 2023-07-14T11:26:32.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb3e7fc700 1 -- 172.21.15.161:0/1515267574 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb48017040 con 0x55faaed82350 2023-07-14T11:26:32.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.575+0000 7fbb3e7fc700 1 -- 172.21.15.161:0/1515267574 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48011440 con 0x55faaed82350 2023-07-14T11:26:32.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 -- 172.21.15.161:0/1515267574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 msgr2=0x55faaed94570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed94570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55faaeda00f0 tx=0x7fbb48009260 comp rx=0 tx=0).stop 2023-07-14T11:26:32.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 -- 172.21.15.161:0/1515267574 shutdown_connections 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed8db50 0x55faaed89fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed82350 0x55faaed94570 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/1515267574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82c60 0x55faaed8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 -- 172.21.15.161:0/1515267574 >> 172.21.15.161:0/1515267574 conn(0x55faaed78e20 msgr2=0x55faaed8c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 -- 172.21.15.161:0/1515267574 shutdown_connections 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 -- 172.21.15.161:0/1515267574 wait complete. 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 -- start start 2023-07-14T11:26:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.576+0000 7fbb71a16b80 1 Processor -- start 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaedea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaedeabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb71a16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed8db50 0x55faaed93260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaed93940 con 0x55faaed82350 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaed93ab0 con 0x55faaed8db50 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb71a16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faaed93c70 con 0x55faaed82c60 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb71a16b80 10 client.0 _ll_register_callbacks cb 0x7fbb50003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:32.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaedeabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaedeabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50446/0 (socket says 172.21.15.161:50446) 2023-07-14T11:26:32.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb56ffd700 1 -- 172.21.15.161:0/375206406 learned_addr learned my addr 172.21.15.161:0/375206406 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb57fff700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed8db50 0x55faaed93260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.577+0000 7fbb577fe700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaedea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb56ffd700 1 -- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed8db50 msgr2=0x55faaed93260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed8db50 0x55faaed93260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb56ffd700 1 -- 172.21.15.161:0/375206406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 msgr2=0x55faaedea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaedea6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb56ffd700 1 -- 172.21.15.161:0/375206406 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4801fab0 con 0x55faaed82c60 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb57fff700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed8db50 0x55faaed93260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb56ffd700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaedeabe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c099ff0 tx=0x7fbb4c003b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e6765aa52f3d2b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb36ffd700 1 -- 172.21.15.161:0/375206406 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c016330 con 0x55faaed82c60 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb36ffd700 1 -- 172.21.15.161:0/375206406 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fbb4c0a2740 con 0x55faaed82c60 2023-07-14T11:26:32.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb4c0779c0 con 0x55faaed82c60 2023-07-14T11:26:32.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb36ffd700 1 -- 172.21.15.161:0/375206406 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb4c09a2f0 con 0x55faaed82c60 2023-07-14T11:26:32.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb71a16b80 10 client.26530 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:32.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55faaeb01330 con 0x55faaed82c60 2023-07-14T11:26:32.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.578+0000 7fbb36ffd700 1 -- 172.21.15.161:0/375206406 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4c0779c0 con 0x55faaed82c60 2023-07-14T11:26:32.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb36ffd700 1 -- 172.21.15.161:0/375206406 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb4c0b4d90 con 0x55faaed82c60 2023-07-14T11:26:32.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb71a16b80 10 client.26530 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:32.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb71a16b80 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:32.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb71a16b80 20 client.26530 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:32.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaedf3360 0x55faaedf5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb71a16b80 4 client.26530 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:26:32.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55faaec900d0 con 0x55faaedf3360 2023-07-14T11:26:32.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb577fe700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaedf3360 0x55faaedf5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb36ffd700 10 client.26530 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.579+0000 7fbb577fe700 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaedf3360 0x55faaedf5770 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b204f88600f557 server_cookie=5fee03dc06c85c3d in_seq=0 out_seq=0 2023-07-14T11:26:32.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.580+0000 7fbb36ffd700 1 -- 172.21.15.161:0/375206406 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55faaec900d0 con 0x55faaedf3360 2023-07-14T11:26:32.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.580+0000 7fbb36ffd700 10 client.26530 handle_command_reply: tid=0 2023-07-14T11:26:32.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.580+0000 7fbb71a16b80 1 client.26530 shutdown 2023-07-14T11:26:32.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 msgr2=0x55faaedeabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaedeabe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c099ff0 tx=0x7fbb4c003b10 comp rx=0 tx=0).stop 2023-07-14T11:26:32.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 shutdown_connections 2023-07-14T11:26:32.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55faaedf3360 0x55faaedf5770 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faaed82350 0x55faaedea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55faaed8db50 0x55faaed93260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb36ffd700 0 client.26530 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 --2- 172.21.15.161:0/375206406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55faaed82c60 0x55faaedeabe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 >> 172.21.15.161:0/375206406 conn(0x55faaed78e20 msgr2=0x55faaed8c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 shutdown_connections 2023-07-14T11:26:32.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 1 -- 172.21.15.161:0/375206406 wait complete. 2023-07-14T11:26:32.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.581+0000 7fbb71a16b80 20 client.26530 trim_cache size 0 max 16384 2023-07-14T11:26:32.592 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:26:32.593 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:26:32.593 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "d4d8cc56-a768-4a6d-81e3-91e3152d2fea", 2023-07-14T11:26:32.593 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:26:32.593 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:26:32.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbb400a1340 msgr2=0x7fbb400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbb400a1340 0x7fbb400a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb500074d0 tx=0x7fbb50007420 comp rx=0 tx=0).stop 2023-07-14T11:26:32.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 msgr2=0x7fbb58198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb58198480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00ec10 tx=0x7fbb4c00ef20 comp rx=0 tx=0).stop 2023-07-14T11:26:32.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 shutdown_connections 2023-07-14T11:26:32.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbb400a1340 0x7fbb400a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb580ffad0 0x7fbb58197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbb58105ae0 0x7fbb5819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 --2- 172.21.15.161:0/2684096981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb58100480 0x7fbb58198480 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 >> 172.21.15.161:0/2684096981 conn(0x7fbb580fb5f0 msgr2=0x7fbb5810aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 shutdown_connections 2023-07-14T11:26:32.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.582+0000 7fbb5dfff700 1 -- 172.21.15.161:0/2684096981 wait complete. 2023-07-14T11:26:32.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:26:32.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:32 smithi175 ceph-mon[129081]: pgmap v131: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 318 op/s 2023-07-14T11:26:32.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 -- 172.21.15.161:0/2992868376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac10c400 msgr2=0x7fe8ac10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/2992868376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac10c400 0x7fe8ac10e7e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40098f0 tx=0x7fe8a4009c00 comp rx=0 tx=0).stop 2023-07-14T11:26:32.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 -- 172.21.15.161:0/2992868376 shutdown_connections 2023-07-14T11:26:32.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/2992868376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac0fe850 0x7fe8ac0fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/2992868376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac10c400 0x7fe8ac10e7e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/2992868376 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8ac0ff1f0 0x7fe8ac10bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 -- 172.21.15.161:0/2992868376 >> 172.21.15.161:0/2992868376 conn(0x7fe8ac0fa4e0 msgr2=0x7fe8ac0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 -- 172.21.15.161:0/2992868376 shutdown_connections 2023-07-14T11:26:32.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.837+0000 7fe8b41ff700 1 -- 172.21.15.161:0/2992868376 wait complete. 2023-07-14T11:26:32.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.838+0000 7fe8b41ff700 1 Processor -- start 2023-07-14T11:26:32.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.838+0000 7fe8b41ff700 1 -- start start 2023-07-14T11:26:32.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b41ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac0fe850 0x7fe8ac197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b41ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8ac0ff1f0 0x7fe8ac198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b41ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac10c400 0x7fe8ac19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b41ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac19cd90 con 0x7fe8ac10c400 2023-07-14T11:26:32.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b41ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac19cf00 con 0x7fe8ac0fe850 2023-07-14T11:26:32.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b41ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac19d0c0 con 0x7fe8ac0ff1f0 2023-07-14T11:26:32.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac0fe850 0x7fe8ac197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac0fe850 0x7fe8ac197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56086/0 (socket says 172.21.15.161:56086) 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3496654812 learned_addr learned my addr 172.21.15.161:0/3496654812 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b279c700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac10c400 0x7fe8ac19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b179a700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8ac0ff1f0 0x7fe8ac198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3496654812 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8ac0ff1f0 msgr2=0x7fe8ac198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8ac0ff1f0 0x7fe8ac198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac10c400 msgr2=0x7fe8ac19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac10c400 0x7fe8ac19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.839+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3496654812 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009520 con 0x7fe8ac0fe850 2023-07-14T11:26:32.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.840+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac0fe850 0x7fe8ac197e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c0c0 tx=0x7fe89c00c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.840+0000 7fe8aaffd700 1 -- 172.21.15.161:0/3496654812 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c018070 con 0x7fe8ac0fe850 2023-07-14T11:26:32.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.840+0000 7fe8aaffd700 1 -- 172.21.15.161:0/3496654812 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89c00f040 con 0x7fe8ac0fe850 2023-07-14T11:26:32.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.840+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac19d340 con 0x7fe8ac0fe850 2023-07-14T11:26:32.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.840+0000 7fe8aaffd700 1 -- 172.21.15.161:0/3496654812 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00e020 con 0x7fe8ac0fe850 2023-07-14T11:26:32.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.840+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac1a4be0 con 0x7fe8ac0fe850 2023-07-14T11:26:32.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.844+0000 7fe8aaffd700 1 -- 172.21.15.161:0/3496654812 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe89c00ca50 con 0x7fe8ac0fe850 2023-07-14T11:26:32.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8aaffd700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8940a12c0 0x7fe8940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 Processor -- start 2023-07-14T11:26:32.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8aaffd700 1 -- 172.21.15.161:0/3496654812 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe89c0ccdd0 con 0x7fe8ac0fe850 2023-07-14T11:26:32.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 -- start start 2023-07-14T11:26:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8b179a700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8940a12c0 0x7fe8940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb61eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfa5acb0 con 0x5634dfb5fad0 2023-07-14T11:26:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfaada40 con 0x5634dfb542d0 2023-07-14T11:26:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfaa6ed0 con 0x5634dfb54be0 2023-07-14T11:26:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.846+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56096/0 (socket says 172.21.15.161:56096) 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/689054939 learned_addr learned my addr 172.21.15.161:0/689054939 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b279c700 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb61eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8940a12c0 0x7fe8940a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1993c0 tx=0x7fe8a0009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 -- 172.21.15.161:0/689054939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 msgr2=0x5634dfb546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 -- 172.21.15.161:0/689054939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 msgr2=0x5634dfb61eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb61eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 -- 172.21.15.161:0/689054939 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634df8d4de0 con 0x5634dfb54be0 2023-07-14T11:26:32.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:32.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe8b179a700 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5f590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00117d0 tx=0x7fe8a001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1e753502c84c5a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe893fff700 1 -- 172.21.15.161:0/689054939 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0004050 con 0x5634dfb54be0 2023-07-14T11:26:32.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe893fff700 1 -- 172.21.15.161:0/689054939 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a001c890 con 0x5634dfb54be0 2023-07-14T11:26:32.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.847+0000 7fe893fff700 1 -- 172.21.15.161:0/689054939 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0003c50 con 0x5634dfb54be0 2023-07-14T11:26:32.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/689054939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 msgr2=0x5634dfb5f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5f590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00117d0 tx=0x7fe8a001c000 comp rx=0 tx=0).stop 2023-07-14T11:26:32.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/689054939 shutdown_connections 2023-07-14T11:26:32.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb61eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/689054939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5f590 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/689054939 >> 172.21.15.161:0/689054939 conn(0x5634dfb4ada0 msgr2=0x5634dfb506a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.848+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/689054939 shutdown_connections 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.849+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/689054939 wait complete. 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.849+0000 7fe8c7a5eb80 1 -- start start 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.849+0000 7fe8c7a5eb80 1 Processor -- start 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb5cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfa46ac0 con 0x5634dfb5fad0 2023-07-14T11:26:32.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfa467c0 con 0x5634dfb542d0 2023-07-14T11:26:32.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634df92a920 con 0x5634dfb54be0 2023-07-14T11:26:32.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8c7a5eb80 10 client.0 _ll_register_callbacks cb 0x5634dfa44760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:32.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b179a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b279c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb5cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56110/0 (socket says 172.21.15.161:56110) 2023-07-14T11:26:32.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b179a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50490/0 (socket says 172.21.15.161:50490) 2023-07-14T11:26:32.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3039005806 learned_addr learned my addr 172.21.15.161:0/3039005806 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3039005806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 msgr2=0x5634dfb5c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3039005806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 msgr2=0x5634dfb5cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.850+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb5cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/3039005806 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac1a4be0 con 0x5634dfb542d0 2023-07-14T11:26:32.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f560 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe89c000f80 tx=0x7fe89c099550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34212a048a172b92 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8917fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:26:32.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8917fa700 1 -- 172.21.15.161:0/3039005806 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c09dba0 con 0x5634dfb542d0 2023-07-14T11:26:32.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634dfa004f0 con 0x5634dfb542d0 2023-07-14T11:26:32.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8917fa700 1 -- 172.21.15.161:0/3039005806 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe89c0a1fc0 con 0x5634dfb542d0 2023-07-14T11:26:32.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8917fa700 1 -- 172.21.15.161:0/3039005806 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89c007d70 con 0x5634dfb542d0 2023-07-14T11:26:32.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8c7a5eb80 10 client.26537 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:32.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe8a0017050 con 0x5634dfb542d0 2023-07-14T11:26:32.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.851+0000 7fe8917fa700 1 -- 172.21.15.161:0/3039005806 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634dfa004f0 con 0x5634dfb542d0 2023-07-14T11:26:32.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8917fa700 1 -- 172.21.15.161:0/3039005806 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe89c0a1b30 con 0x5634dfb542d0 2023-07-14T11:26:32.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8c7a5eb80 10 client.26537 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:32.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8c7a5eb80 10 client.26537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:32.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8c7a5eb80 20 client.26537 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:32.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfb6f3d0 0x5634dfb717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8c7a5eb80 4 client.26537 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:26:32.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634dfa9feb0 con 0x5634dfb6f3d0 2023-07-14T11:26:32.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8b179a700 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfb6f3d0 0x5634dfb717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8b179a700 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfb6f3d0 0x5634dfb717e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73135525a82a8b9f server_cookie=f7e4704e8d84729a in_seq=0 out_seq=0 2023-07-14T11:26:32.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.852+0000 7fe8917fa700 10 client.26537 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8917fa700 1 -- 172.21.15.161:0/3039005806 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5634dfa9feb0 con 0x5634dfb6f3d0 2023-07-14T11:26:32.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8917fa700 10 client.26537 handle_command_reply: tid=0 2023-07-14T11:26:32.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 client.26537 shutdown 2023-07-14T11:26:32.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 msgr2=0x5634dfb5f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f560 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe89c000f80 tx=0x7fe89c099550 comp rx=0 tx=0).stop 2023-07-14T11:26:32.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 shutdown_connections 2023-07-14T11:26:32.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfb6f3d0 0x5634dfb717e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb5cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f560 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8917fa700 0 client.26537 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/3039005806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb54be0 0x5634dfb5c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 >> 172.21.15.161:0/3039005806 conn(0x5634dfb4ada0 msgr2=0x5634dfb50230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 shutdown_connections 2023-07-14T11:26:32.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.853+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/3039005806 wait complete. 2023-07-14T11:26:32.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.854+0000 7fe8c7a5eb80 20 client.26537 trim_cache size 0 max 16384 2023-07-14T11:26:32.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 Processor -- start 2023-07-14T11:26:32.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 -- start start 2023-07-14T11:26:32.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb54be0 0x5634dfb5bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb5fad0 0x5634dfb5c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfb55540 con 0x5634dfb54be0 2023-07-14T11:26:32.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfb561a0 con 0x5634dfb542d0 2023-07-14T11:26:32.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfb5ad60 con 0x5634dfb5fad0 2023-07-14T11:26:32.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.872+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb54be0 0x5634dfb5bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb5fad0 0x5634dfb5c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb54be0 0x5634dfb5bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48058/0 (socket says 172.21.15.161:48058) 2023-07-14T11:26:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb5fad0 0x5634dfb5c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50500/0 (socket says 172.21.15.161:50500) 2023-07-14T11:26:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b179a700 1 -- 172.21.15.161:0/1468870222 learned_addr learned my addr 172.21.15.161:0/1468870222 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56114/0 (socket says 172.21.15.161:56114) 2023-07-14T11:26:32.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 -- 172.21.15.161:0/1468870222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 msgr2=0x5634dfb5f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 -- 172.21.15.161:0/1468870222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb54be0 msgr2=0x5634dfb5bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb54be0 0x5634dfb5bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 -- 172.21.15.161:0/1468870222 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634df8bdc60 con 0x5634dfb5fad0 2023-07-14T11:26:32.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:26:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8b279c700 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb5fad0 0x5634dfb5c500 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009810 tx=0x7fe8a4005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd33e806e0f45a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.873+0000 7fe8a8ff9700 1 -- 172.21.15.161:0/1468870222 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400f770 con 0x5634dfb5fad0 2023-07-14T11:26:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8a8ff9700 1 -- 172.21.15.161:0/1468870222 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a401d040 con 0x5634dfb5fad0 2023-07-14T11:26:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8a8ff9700 1 -- 172.21.15.161:0/1468870222 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40054e0 con 0x5634dfb5fad0 2023-07-14T11:26:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/1468870222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb5fad0 msgr2=0x5634dfb5c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb5fad0 0x5634dfb5c500 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009810 tx=0x7fe8a4005de0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/1468870222 shutdown_connections 2023-07-14T11:26:32.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb5fad0 0x5634dfb5c500 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb542d0 0x5634dfb5f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/1468870222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb54be0 0x5634dfb5bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/1468870222 >> 172.21.15.161:0/1468870222 conn(0x5634dfb4ada0 msgr2=0x5634dfb57b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/1468870222 shutdown_connections 2023-07-14T11:26:32.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.874+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/1468870222 wait complete. 2023-07-14T11:26:32.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 -- start start 2023-07-14T11:26:32.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 Processor -- start 2023-07-14T11:26:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb542d0 0x5634dfb662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb54be0 0x5634dfb645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb64b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfb65070 con 0x5634dfb5fad0 2023-07-14T11:26:32.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfb651e0 con 0x5634dfb54be0 2023-07-14T11:26:32.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.875+0000 7fe8c7a5eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dfb653a0 con 0x5634dfb542d0 2023-07-14T11:26:32.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8c7a5eb80 10 client.0 _ll_register_callbacks cb 0x7fe89c099940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:26:32.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb542d0 0x5634dfb662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb542d0 0x5634dfb662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50508/0 (socket says 172.21.15.161:50508) 2023-07-14T11:26:32.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/2341152360 learned_addr learned my addr 172.21.15.161:0/2341152360 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:26:32.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b279c700 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb64b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b179a700 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb54be0 0x5634dfb645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb54be0 msgr2=0x5634dfb645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb54be0 0x5634dfb645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/2341152360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 msgr2=0x5634dfb64b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:26:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb64b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 -- 172.21.15.161:0/2341152360 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a0017050 con 0x5634dfb542d0 2023-07-14T11:26:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8b1f9b700 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb542d0 0x5634dfb662c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0965e0 tx=0x7fe89c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a29b1d2de0c76ba2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:26:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.876+0000 7fe8917fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:26:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8917fa700 1 -- 172.21.15.161:0/2341152360 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c063070 con 0x5634dfb542d0 2023-07-14T11:26:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8917fa700 1 -- 172.21.15.161:0/2341152360 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe89c0a1fc0 con 0x5634dfb542d0 2023-07-14T11:26:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8a001c890 con 0x5634dfb542d0 2023-07-14T11:26:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8917fa700 1 -- 172.21.15.161:0/2341152360 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89c09dba0 con 0x5634dfb542d0 2023-07-14T11:26:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 10 client.26548 fetch_fsmap learned FSMap version 25 2023-07-14T11:26:32.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe8ac10f420 con 0x5634dfb542d0 2023-07-14T11:26:32.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8917fa700 1 -- 172.21.15.161:0/2341152360 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a001c890 con 0x5634dfb542d0 2023-07-14T11:26:32.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8917fa700 1 -- 172.21.15.161:0/2341152360 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe89c098ad0 con 0x5634dfb542d0 2023-07-14T11:26:32.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 10 client.26548 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:26:32.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 10 client.26548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:26:32.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 20 client.26548 populate_metadata read hostname 'smithi161' 2023-07-14T11:26:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfbc4260 0x5634dfbc6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:26:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 4 client.26548 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:26:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5634dfa62050 con 0x5634dfbc4260 2023-07-14T11:26:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.877+0000 7fe8b179a700 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfbc4260 0x5634dfbc6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:26:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8b179a700 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfbc4260 0x5634dfbc6640 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b7111d3cad7778 server_cookie=a376b2c6fdce3763 in_seq=0 out_seq=0 2023-07-14T11:26:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8917fa700 10 client.26548 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8917fa700 1 -- 172.21.15.161:0/2341152360 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5634dfa62050 con 0x5634dfbc4260 2023-07-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8917fa700 10 client.26548 handle_command_reply: tid=0 2023-07-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 client.26548 shutdown 2023-07-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb542d0 msgr2=0x5634dfb662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb542d0 0x5634dfb662c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0965e0 tx=0x7fe89c01d000 comp rx=0 tx=0).stop 2023-07-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 shutdown_connections 2023-07-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5634dfbc4260 0x5634dfbc6640 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634dfb5fad0 0x5634dfb64b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5634dfb542d0 0x5634dfb662c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 --2- 172.21.15.161:0/2341152360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634dfb54be0 0x5634dfb645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8917fa700 0 client.26548 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:26:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 >> 172.21.15.161:0/2341152360 conn(0x5634dfb4ada0 msgr2=0x5634dfb563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 shutdown_connections 2023-07-14T11:26:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.878+0000 7fe8c7a5eb80 1 -- 172.21.15.161:0/2341152360 wait complete. 2023-07-14T11:26:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8c7a5eb80 20 client.26548 trim_cache size 0 max 16384 2023-07-14T11:26:32.899 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:26:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:26:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:26:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "d4d8cc56-a768-4a6d-81e3-91e3152d2fea": { 2023-07-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "d4d8cc56-a768-4a6d-81e3-91e3152d2fea", 2023-07-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:26:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8940a12c0 msgr2=0x7fe8940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8940a12c0 0x7fe8940a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1993c0 tx=0x7fe8a0009260 comp rx=0 tx=0).stop 2023-07-14T11:26:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac0fe850 msgr2=0x7fe8ac197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:26:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac0fe850 0x7fe8ac197e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c0c0 tx=0x7fe89c00c3d0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 shutdown_connections 2023-07-14T11:26:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8940a12c0 0x7fe8940a3770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac10c400 0x7fe8ac19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8ac0fe850 0x7fe8ac197e20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 --2- 172.21.15.161:0/3496654812 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8ac0ff1f0 0x7fe8ac198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:26:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 >> 172.21.15.161:0/3496654812 conn(0x7fe8ac0fa4e0 msgr2=0x7fe8ac10d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:26:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 shutdown_connections 2023-07-14T11:26:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:26:32.879+0000 7fe8b41ff700 1 -- 172.21.15.161:0/3496654812 wait complete. 2023-07-14T11:26:32.904 INFO:tasks.cephfs.filesystem:scrub status for tag:d4d8cc56-a768-4a6d-81e3-91e3152d2fea - {'path': '/', 'tag': 'd4d8cc56-a768-4a6d-81e3-91e3152d2fea', 'options': 'recursive,force'} 2023-07-14T11:26:33.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 365301 9.85 MB/sec execute 75 sec latency 45.028 ms 2023-07-14T11:26:33.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:33 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:26:33.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:33 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:33.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:33 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:33.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:33 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:33.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:33 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:26:33.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:33 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:33.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:33 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:33.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:33 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:33.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:33 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:26:33.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:33 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:33.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:33 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:33.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:33 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:34.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 366495 9.77 MB/sec execute 76 sec latency 16.661 ms 2023-07-14T11:26:34.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:34 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:34.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:34 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:26:34.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:34 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:34.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:34 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:34.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:34 smithi182 ceph-mon[129366]: pgmap v132: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-14T11:26:34.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:34 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:34.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:34 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:26:34.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:34 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:34.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:34 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:34.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:34 smithi161 ceph-mon[95526]: pgmap v132: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-14T11:26:34.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:34 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:34.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:34 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:26:34.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:34 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:34.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:34 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:26:34.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:34 smithi175 ceph-mon[129081]: pgmap v132: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-14T11:26:35.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 369126 9.86 MB/sec execute 77 sec latency 12.092 ms 2023-07-14T11:26:36.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 370413 9.81 MB/sec execute 78 sec latency 15.001 ms 2023-07-14T11:26:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:36 smithi182 ceph-mon[129366]: pgmap v133: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-14T11:26:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:36 smithi161 ceph-mon[95526]: pgmap v133: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-14T11:26:36.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:36 smithi175 ceph-mon[129081]: pgmap v133: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-14T11:26:37.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 372692 9.85 MB/sec execute 79 sec latency 57.872 ms 2023-07-14T11:26:38.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 373937 9.80 MB/sec execute 80 sec latency 38.317 ms 2023-07-14T11:26:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:38 smithi182 ceph-mon[129366]: pgmap v134: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-07-14T11:26:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:38 smithi161 ceph-mon[95526]: pgmap v134: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-07-14T11:26:38.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:38 smithi175 ceph-mon[129081]: pgmap v134: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-07-14T11:26:39.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 376278 9.85 MB/sec execute 81 sec latency 15.657 ms 2023-07-14T11:26:40.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 377750 9.80 MB/sec execute 82 sec latency 13.710 ms 2023-07-14T11:26:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:40 smithi182 ceph-mon[129366]: pgmap v135: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2023-07-14T11:26:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:40 smithi161 ceph-mon[95526]: pgmap v135: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2023-07-14T11:26:40.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:40 smithi175 ceph-mon[129081]: pgmap v135: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2023-07-14T11:26:41.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 379946 9.84 MB/sec execute 83 sec latency 14.933 ms 2023-07-14T11:26:41.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:41 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:26:41.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:41 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:26:41.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:41 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:26:42.176 INFO:tasks.workunit.client.0.smithi161.stdout: 1 381339 9.79 MB/sec execute 84 sec latency 18.609 ms 2023-07-14T11:26:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:42 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:26:42.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:42 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:26:42.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:42 smithi182 ceph-mon[129366]: pgmap v136: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2023-07-14T11:26:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:42 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:26:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:42 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:26:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:42 smithi161 ceph-mon[95526]: pgmap v136: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2023-07-14T11:26:42.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:42 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:26:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:42 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:26:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:42 smithi175 ceph-mon[129081]: pgmap v136: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2023-07-14T11:26:43.177 INFO:tasks.workunit.client.0.smithi161.stdout: 1 383391 9.83 MB/sec execute 85 sec latency 57.547 ms 2023-07-14T11:26:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:43 smithi175 ceph-mon[129081]: pgmap v137: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2023-07-14T11:26:44.177 INFO:tasks.workunit.client.0.smithi161.stdout: 1 384797 9.78 MB/sec execute 86 sec latency 25.198 ms 2023-07-14T11:26:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:43 smithi182 ceph-mon[129366]: pgmap v137: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2023-07-14T11:26:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:43 smithi161 ceph-mon[95526]: pgmap v137: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2023-07-14T11:26:45.177 INFO:tasks.workunit.client.0.smithi161.stdout: 1 387061 9.83 MB/sec execute 87 sec latency 24.871 ms 2023-07-14T11:26:46.177 INFO:tasks.workunit.client.0.smithi161.stdout: 1 388767 9.79 MB/sec execute 88 sec latency 10.318 ms 2023-07-14T11:26:46.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:46 smithi175 ceph-mon[129081]: pgmap v138: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2023-07-14T11:26:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:46 smithi182 ceph-mon[129366]: pgmap v138: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2023-07-14T11:26:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:46 smithi161 ceph-mon[95526]: pgmap v138: 129 pgs: 129 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2023-07-14T11:26:47.177 INFO:tasks.workunit.client.0.smithi161.stdout: 1 390683 9.83 MB/sec execute 89 sec latency 37.796 ms 2023-07-14T11:26:48.177 INFO:tasks.workunit.client.0.smithi161.stdout: 1 392801 9.81 MB/sec execute 90 sec latency 24.170 ms 2023-07-14T11:26:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:48 smithi182 ceph-mon[129366]: pgmap v139: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2023-07-14T11:26:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:48 smithi161 ceph-mon[95526]: pgmap v139: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2023-07-14T11:26:48.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:48 smithi175 ceph-mon[129081]: pgmap v139: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2023-07-14T11:26:49.177 INFO:tasks.workunit.client.0.smithi161.stdout: 1 394359 9.82 MB/sec execute 91 sec latency 26.749 ms 2023-07-14T11:26:50.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 396697 9.85 MB/sec execute 92 sec latency 42.501 ms 2023-07-14T11:26:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:50 smithi182 ceph-mon[129366]: pgmap v140: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2023-07-14T11:26:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:50 smithi161 ceph-mon[95526]: pgmap v140: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2023-07-14T11:26:50.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:50 smithi175 ceph-mon[129081]: pgmap v140: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2023-07-14T11:26:51.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 398049 9.82 MB/sec execute 93 sec latency 14.193 ms 2023-07-14T11:26:52.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 400494 9.85 MB/sec execute 94 sec latency 10.128 ms 2023-07-14T11:26:52.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:52 smithi182 ceph-mon[129366]: pgmap v141: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2023-07-14T11:26:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:52 smithi161 ceph-mon[95526]: pgmap v141: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2023-07-14T11:26:52.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:52 smithi175 ceph-mon[129081]: pgmap v141: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2023-07-14T11:26:53.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 401795 9.82 MB/sec execute 95 sec latency 10.930 ms 2023-07-14T11:26:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:53 smithi175 ceph-mon[129081]: pgmap v142: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-07-14T11:26:54.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 404030 9.84 MB/sec execute 96 sec latency 63.527 ms 2023-07-14T11:26:54.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:53 smithi182 ceph-mon[129366]: pgmap v142: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-07-14T11:26:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:53 smithi161 ceph-mon[95526]: pgmap v142: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-07-14T11:26:55.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 405400 9.81 MB/sec execute 97 sec latency 14.189 ms 2023-07-14T11:26:56.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 408063 9.89 MB/sec execute 98 sec latency 9.608 ms 2023-07-14T11:26:56.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:56 smithi175 ceph-mon[129081]: pgmap v143: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-07-14T11:26:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:56 smithi182 ceph-mon[129366]: pgmap v143: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-07-14T11:26:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:56 smithi161 ceph-mon[95526]: pgmap v143: 129 pgs: 129 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-07-14T11:26:57.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 409201 9.82 MB/sec execute 99 sec latency 21.430 ms 2023-07-14T11:26:58.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 411746 9.89 MB/sec execute 100 sec latency 10.384 ms 2023-07-14T11:26:58.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:26:58 smithi175 ceph-mon[129081]: pgmap v144: 129 pgs: 129 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 318 op/s 2023-07-14T11:26:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:26:58 smithi182 ceph-mon[129366]: pgmap v144: 129 pgs: 129 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 318 op/s 2023-07-14T11:26:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:26:58 smithi161 ceph-mon[95526]: pgmap v144: 129 pgs: 129 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 318 op/s 2023-07-14T11:26:59.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 412633 9.81 MB/sec execute 101 sec latency 78.854 ms 2023-07-14T11:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 415208 9.87 MB/sec execute 102 sec latency 28.712 ms 2023-07-14T11:27:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:00 smithi182 ceph-mon[129366]: pgmap v145: 129 pgs: 129 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2023-07-14T11:27:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:00 smithi161 ceph-mon[95526]: pgmap v145: 129 pgs: 129 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2023-07-14T11:27:00.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:00 smithi175 ceph-mon[129081]: pgmap v145: 129 pgs: 129 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2023-07-14T11:27:01.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 415456 9.79 MB/sec execute 103 sec latency 63.902 ms 2023-07-14T11:27:02.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 417369 9.76 MB/sec execute 104 sec latency 10.023 ms 2023-07-14T11:27:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:02 smithi182 ceph-mon[129366]: pgmap v146: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2023-07-14T11:27:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:02 smithi161 ceph-mon[95526]: pgmap v146: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2023-07-14T11:27:02.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:02 smithi175 ceph-mon[129081]: pgmap v146: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2023-07-14T11:27:02.904 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:27:03.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.159+0000 7f7a763ff700 1 -- 172.21.15.161:0/740638363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7010e280 msgr2=0x7f7a7010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.159+0000 7f7a763ff700 1 --2- 172.21.15.161:0/740638363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7010e280 0x7f7a7010e650 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009990 tx=0x7f7a60009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.160+0000 7f7a763ff700 1 -- 172.21.15.161:0/740638363 shutdown_connections 2023-07-14T11:27:03.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.160+0000 7f7a763ff700 1 --2- 172.21.15.161:0/740638363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7010e280 0x7f7a7010e650 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.160+0000 7f7a763ff700 1 --2- 172.21.15.161:0/740638363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7006d3a0 0x7f7a7011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.160+0000 7f7a763ff700 1 --2- 172.21.15.161:0/740638363 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 0x7f7a7006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.160+0000 7f7a763ff700 1 -- 172.21.15.161:0/740638363 >> 172.21.15.161:0/740638363 conn(0x7f7a7006bf70 msgr2=0x7f7a70070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.160+0000 7f7a763ff700 1 -- 172.21.15.161:0/740638363 shutdown_connections 2023-07-14T11:27:03.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.160+0000 7f7a763ff700 1 -- 172.21.15.161:0/740638363 wait complete. 2023-07-14T11:27:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.161+0000 7f7a763ff700 1 Processor -- start 2023-07-14T11:27:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.161+0000 7f7a763ff700 1 -- start start 2023-07-14T11:27:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a763ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 0x7f7a701aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a763ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7006d3a0 0x7f7a701aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a763ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7010e280 0x7f7a701a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a763ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701a4eb0 con 0x7f7a7006d3a0 2023-07-14T11:27:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a763ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701a5020 con 0x7f7a7010e280 2023-07-14T11:27:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a763ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701a51e0 con 0x7f7a7006c9a0 2023-07-14T11:27:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 0x7f7a701aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a7499c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7010e280 0x7f7a701a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 0x7f7a701aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58374/0 (socket says 172.21.15.161:58374) 2023-07-14T11:27:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a6ffff700 1 -- 172.21.15.161:0/2467483739 learned_addr learned my addr 172.21.15.161:0/2467483739 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a6f7fe700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7006d3a0 0x7f7a701aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a6ffff700 1 -- 172.21.15.161:0/2467483739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7010e280 msgr2=0x7f7a701a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.162+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7010e280 0x7f7a701a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a6ffff700 1 -- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7006d3a0 msgr2=0x7f7a701aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7006d3a0 0x7f7a701aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a6ffff700 1 -- 172.21.15.161:0/2467483739 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68009c40 con 0x7f7a7006c9a0 2023-07-14T11:27:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 0x7f7a701aa7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a60000c00 tx=0x7f7a60005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a6d7fa700 1 -- 172.21.15.161:0/2467483739 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6001d070 con 0x7f7a7006c9a0 2023-07-14T11:27:03.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a600095f0 con 0x7f7a7006c9a0 2023-07-14T11:27:03.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a701b15e0 con 0x7f7a7006c9a0 2023-07-14T11:27:03.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a6d7fa700 1 -- 172.21.15.161:0/2467483739 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a60022470 con 0x7f7a7006c9a0 2023-07-14T11:27:03.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.163+0000 7f7a6d7fa700 1 -- 172.21.15.161:0/2467483739 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60017650 con 0x7f7a7006c9a0 2023-07-14T11:27:03.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.167+0000 7f7a6d7fa700 1 -- 172.21.15.161:0/2467483739 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7a6000fa50 con 0x7f7a7006c9a0 2023-07-14T11:27:03.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 Processor -- start 2023-07-14T11:27:03.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 -- start start 2023-07-14T11:27:03.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a303d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a3048590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3048ad0 0x55a6a304aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a2f43cb0 con 0x55a6a303dbe0 2023-07-14T11:27:03.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a2f96a40 con 0x55a6a303d2d0 2023-07-14T11:27:03.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a2f8fed0 con 0x55a6a3048ad0 2023-07-14T11:27:03.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.168+0000 7f7a6d7fa700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f7a580a13c0 0x7f7a580a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3048ad0 0x55a6a304aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a303d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a6d7fa700 1 -- 172.21.15.161:0/2467483739 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f7a600ce2c0 con 0x7f7a7006c9a0 2023-07-14T11:27:03.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3048ad0 0x55a6a304aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58386/0 (socket says 172.21.15.161:58386) 2023-07-14T11:27:03.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a303d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37382/0 (socket says 172.21.15.161:37382) 2023-07-14T11:27:03.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a6f7fe700 1 -- 172.21.15.161:0/3358397882 learned_addr learned my addr 172.21.15.161:0/3358397882 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:03.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a3048590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f7a580a13c0 0x7f7a580a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 -- 172.21.15.161:0/3358397882 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3048ad0 msgr2=0x55a6a304aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3048ad0 0x55a6a304aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 -- 172.21.15.161:0/3358397882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 msgr2=0x55a6a3048590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a3048590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 -- 172.21.15.161:0/3358397882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a2dbdde0 con 0x55a6a303d2d0 2023-07-14T11:27:03.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a3048590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:03.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.169+0000 7f7a7499c700 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a303d6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a701ab4d0 tx=0x7f7a68008e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3669bf9ad79e6eeb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.170+0000 7f7a7499c700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f7a580a13c0 0x7f7a580a3870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011000 tx=0x7f7a6801d000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.170+0000 7f7a567fc700 1 -- 172.21.15.161:0/3358397882 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800e070 con 0x55a6a303d2d0 2023-07-14T11:27:03.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.170+0000 7f7a567fc700 1 -- 172.21.15.161:0/3358397882 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6800f040 con 0x55a6a303d2d0 2023-07-14T11:27:03.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.170+0000 7f7a567fc700 1 -- 172.21.15.161:0/3358397882 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68004730 con 0x55a6a303d2d0 2023-07-14T11:27:03.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.170+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/3358397882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 msgr2=0x55a6a303d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.170+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a303d6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a701ab4d0 tx=0x7f7a68008e50 comp rx=0 tx=0).stop 2023-07-14T11:27:03.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/3358397882 shutdown_connections 2023-07-14T11:27:03.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3048ad0 0x55a6a304aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a303d6a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/3358397882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a3048590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/3358397882 >> 172.21.15.161:0/3358397882 conn(0x55a6a3033da0 msgr2=0x55a6a30396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/3358397882 shutdown_connections 2023-07-14T11:27:03.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/3358397882 wait complete. 2023-07-14T11:27:03.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 -- start start 2023-07-14T11:27:03.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.171+0000 7f7a89cd1b80 1 Processor -- start 2023-07-14T11:27:03.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a303dbe0 0x55a6a3045720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a3048ad0 0x55a6a3045da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a2f2fac0 con 0x55a6a303d2d0 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a2f2f7c0 con 0x55a6a3048ad0 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a2e13920 con 0x55a6a303dbe0 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a89cd1b80 10 client.0 _ll_register_callbacks cb 0x55a6a2f2d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a7499c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a3048ad0 0x55a6a3045da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a7499c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a3048ad0 0x55a6a3045da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37396/0 (socket says 172.21.15.161:37396) 2023-07-14T11:27:03.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a7499c700 1 -- 172.21.15.161:0/2955084906 learned_addr learned my addr 172.21.15.161:0/2955084906 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:03.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a6f7fe700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a303dbe0 0x55a6a3045720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.172+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a7499c700 1 -- 172.21.15.161:0/2955084906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a303dbe0 msgr2=0x55a6a3045720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a7499c700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a303dbe0 0x55a6a3045720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a7499c700 1 -- 172.21.15.161:0/2955084906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 msgr2=0x55a6a30484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a7499c700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a7499c700 1 -- 172.21.15.161:0/2955084906 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a701b15e0 con 0x55a6a3048ad0 2023-07-14T11:27:03.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:27:03.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a7499c700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a3048ad0 0x55a6a3045da0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7a68004350 tx=0x7f7a680055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b991b5d1344888e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:03.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2955084906 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800e030 con 0x55a6a3048ad0 2023-07-14T11:27:03.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6a2ee94f0 con 0x55a6a3048ad0 2023-07-14T11:27:03.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.173+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2955084906 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f7a68007700 con 0x55a6a3048ad0 2023-07-14T11:27:03.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2955084906 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a68005aa0 con 0x55a6a3048ad0 2023-07-14T11:27:03.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 10 client.26555 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:03.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7a68008750 con 0x55a6a3048ad0 2023-07-14T11:27:03.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2955084906 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6a2ee94f0 con 0x55a6a3048ad0 2023-07-14T11:27:03.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2955084906 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7a6801a4e0 con 0x55a6a3048ad0 2023-07-14T11:27:03.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 10 client.26555 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:03.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 10 client.26555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:03.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 20 client.26555 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:03.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a30582b0 0x55a6a305a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 4 client.26555 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:03.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6a2f88eb0 con 0x55a6a30582b0 2023-07-14T11:27:03.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 419069 9.79 MB/sec execute 105 sec latency 32.496 ms 2023-07-14T11:27:03.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.174+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a30582b0 0x55a6a305a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.175+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a30582b0 0x55a6a305a6c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b732af5ae4650a69 server_cookie=3a5e889abf3b8cfa in_seq=0 out_seq=0 2023-07-14T11:27:03.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.175+0000 7f7a4f7fe700 10 client.26555 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:03.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2955084906 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6a2f88eb0 con 0x55a6a30582b0 2023-07-14T11:27:03.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a4f7fe700 10 client.26555 handle_command_reply: tid=0 2023-07-14T11:27:03.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 client.26555 shutdown 2023-07-14T11:27:03.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a3048ad0 msgr2=0x55a6a3045da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a3048ad0 0x55a6a3045da0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7a68004350 tx=0x7f7a680055f0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 shutdown_connections 2023-07-14T11:27:03.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a30582b0 0x55a6a305a6c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a3048ad0 0x55a6a3045da0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2955084906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a303dbe0 0x55a6a3045720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 >> 172.21.15.161:0/2955084906 conn(0x55a6a3033da0 msgr2=0x55a6a3039230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a4f7fe700 0 client.26555 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:03.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 shutdown_connections 2023-07-14T11:27:03.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2955084906 wait complete. 2023-07-14T11:27:03.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.176+0000 7f7a89cd1b80 20 client.26555 trim_cache size 0 max 16384 2023-07-14T11:27:03.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 Processor -- start 2023-07-14T11:27:03.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 -- start start 2023-07-14T11:27:03.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a304f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a3048040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a30429b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a3035ec0 con 0x55a6a303dbe0 2023-07-14T11:27:03.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a3046680 con 0x55a6a303d2d0 2023-07-14T11:27:03.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.195+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a3046130 con 0x55a6a3033da0 2023-07-14T11:27:03.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a3048040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a3048040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37402/0 (socket says 172.21.15.161:37402) 2023-07-14T11:27:03.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 -- 172.21.15.161:0/429820867 learned_addr learned my addr 172.21.15.161:0/429820867 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:03.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a7499c700 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a30429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a304f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 -- 172.21.15.161:0/429820867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 msgr2=0x55a6a304f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a304f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 -- 172.21.15.161:0/429820867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 msgr2=0x55a6a30429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a30429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 -- 172.21.15.161:0/429820867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a2da6c60 con 0x55a6a303d2d0 2023-07-14T11:27:03.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a7499c700 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a30429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:03.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.196+0000 7f7a6f7fe700 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a3048040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009ea0 tx=0x7f7a64009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45dbee77bcd61cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a56ffd700 1 -- 172.21.15.161:0/429820867 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64020070 con 0x55a6a303d2d0 2023-07-14T11:27:03.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a56ffd700 1 -- 172.21.15.161:0/429820867 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a64018040 con 0x55a6a303d2d0 2023-07-14T11:27:03.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a56ffd700 1 -- 172.21.15.161:0/429820867 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64017050 con 0x55a6a303d2d0 2023-07-14T11:27:03.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/429820867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 msgr2=0x55a6a3048040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a3048040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009ea0 tx=0x7f7a64009260 comp rx=0 tx=0).stop 2023-07-14T11:27:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/429820867 shutdown_connections 2023-07-14T11:27:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303dbe0 0x55a6a30429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a304f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/429820867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303d2d0 0x55a6a3048040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/429820867 >> 172.21.15.161:0/429820867 conn(0x55a6a3035600 msgr2=0x55a6a3040a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/429820867 shutdown_connections 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.197+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/429820867 wait complete. 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.198+0000 7f7a89cd1b80 1 -- start start 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.198+0000 7f7a89cd1b80 1 Processor -- start 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.198+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a30a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.198+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a89cd1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303dbe0 0x55a6a30a0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a30a0700 con 0x55a6a303d2d0 2023-07-14T11:27:03.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a30a0870 con 0x55a6a303dbe0 2023-07-14T11:27:03.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a89cd1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a30a0a30 con 0x55a6a3033da0 2023-07-14T11:27:03.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a89cd1b80 10 client.0 _ll_register_callbacks cb 0x7f7a64009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:03.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a7499c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303dbe0 0x55a6a30a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a30a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34446/0 (socket says 172.21.15.161:34446) 2023-07-14T11:27:03.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a30a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58406/0 (socket says 172.21.15.161:58406) 2023-07-14T11:27:03.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a7499c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303dbe0 0x55a6a30a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37408/0 (socket says 172.21.15.161:37408) 2023-07-14T11:27:03.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6f7fe700 1 -- 172.21.15.161:0/2447337441 learned_addr learned my addr 172.21.15.161:0/2447337441 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:03.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 -- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303dbe0 msgr2=0x55a6a30a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303dbe0 0x55a6a30a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 -- 172.21.15.161:0/2447337441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 msgr2=0x55a6a30a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:03.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 -- 172.21.15.161:0/2447337441 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68008750 con 0x55a6a3033da0 2023-07-14T11:27:03.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a7499c700 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303dbe0 0x55a6a30a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:03.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a6ffff700 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a30a56c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a600cb680 tx=0x7f7a6006d100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e15728c2bd26938 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.199+0000 7f7a4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:03.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2447337441 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6009b090 con 0x55a6a3033da0 2023-07-14T11:27:03.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2447337441 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f7a6009f0c0 con 0x55a6a3033da0 2023-07-14T11:27:03.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a6800f040 con 0x55a6a3033da0 2023-07-14T11:27:03.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2447337441 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a600a3450 con 0x55a6a3033da0 2023-07-14T11:27:03.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 10 client.26566 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:03.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7a64009b20 con 0x55a6a3033da0 2023-07-14T11:27:03.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2447337441 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6800f040 con 0x55a6a3033da0 2023-07-14T11:27:03.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2447337441 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7a6009b410 con 0x55a6a3033da0 2023-07-14T11:27:03.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 10 client.26566 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:03.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 10 client.26566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:03.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 20 client.26566 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:03.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a3048ad0 0x55a6a30af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 4 client.26566 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:27:03.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6a2f4b050 con 0x55a6a3048ad0 2023-07-14T11:27:03.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.200+0000 7f7a6f7fe700 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a3048ad0 0x55a6a30af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.201+0000 7f7a6f7fe700 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a3048ad0 0x55a6a30af330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c261f8ce53234f server_cookie=518944a7cfd12acc in_seq=0 out_seq=0 2023-07-14T11:27:03.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.201+0000 7f7a4f7fe700 10 client.26566 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:03.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a4f7fe700 1 -- 172.21.15.161:0/2447337441 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a6a2f4b050 con 0x55a6a3048ad0 2023-07-14T11:27:03.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a4f7fe700 10 client.26566 handle_command_reply: tid=0 2023-07-14T11:27:03.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 client.26566 shutdown 2023-07-14T11:27:03.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 msgr2=0x55a6a30a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a30a56c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a600cb680 tx=0x7f7a6006d100 comp rx=0 tx=0).stop 2023-07-14T11:27:03.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 shutdown_connections 2023-07-14T11:27:03.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a6a3048ad0 0x55a6a30af330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6a303dbe0 0x55a6a30a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a6a3033da0 0x55a6a30a56c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 --2- 172.21.15.161:0/2447337441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6a303d2d0 0x55a6a30a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a4f7fe700 0 client.26566 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:03.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 >> 172.21.15.161:0/2447337441 conn(0x55a6a3035600 msgr2=0x55a6a30388b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 shutdown_connections 2023-07-14T11:27:03.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.202+0000 7f7a89cd1b80 1 -- 172.21.15.161:0/2447337441 wait complete. 2023-07-14T11:27:03.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a89cd1b80 20 client.26566 trim_cache size 0 max 16384 2023-07-14T11:27:03.218 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:27:03.218 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:27:03.218 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:27:03.218 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:27:03.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f7a580a13c0 msgr2=0x7f7a580a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f7a580a13c0 0x7f7a580a3870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011000 tx=0x7f7a6801d000 comp rx=0 tx=0).stop 2023-07-14T11:27:03.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 msgr2=0x7f7a701aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 0x7f7a701aa7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a60000c00 tx=0x7f7a60005740 comp rx=0 tx=0).stop 2023-07-14T11:27:03.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 shutdown_connections 2023-07-14T11:27:03.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f7a580a13c0 0x7f7a580a3870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a7010e280 0x7f7a701a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7a7006c9a0 0x7f7a701aa7d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 --2- 172.21.15.161:0/2467483739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7006d3a0 0x7f7a701aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.203+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 >> 172.21.15.161:0/2467483739 conn(0x7f7a7006bf70 msgr2=0x7f7a7006f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.204+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 shutdown_connections 2023-07-14T11:27:03.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.204+0000 7f7a763ff700 1 -- 172.21.15.161:0/2467483739 wait complete. 2023-07-14T11:27:03.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:27:03.226 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:27:03.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 -- 172.21.15.161:0/2164922802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 msgr2=0x7fe60c0ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 --2- 172.21.15.161:0/2164922802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 0x7fe60c0ffe40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe600009990 tx=0x7fe600009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 -- 172.21.15.161:0/2164922802 shutdown_connections 2023-07-14T11:27:03.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 --2- 172.21.15.161:0/2164922802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c100380 0x7fe60c10c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 --2- 172.21.15.161:0/2164922802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe60c1059e0 0x7fe60c105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 --2- 172.21.15.161:0/2164922802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 0x7fe60c0ffe40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 -- 172.21.15.161:0/2164922802 >> 172.21.15.161:0/2164922802 conn(0x7fe60c0fb4d0 msgr2=0x7fe60c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 -- 172.21.15.161:0/2164922802 shutdown_connections 2023-07-14T11:27:03.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.438+0000 7fe60b59e700 1 -- 172.21.15.161:0/2164922802 wait complete. 2023-07-14T11:27:03.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.439+0000 7fe60b59e700 1 Processor -- start 2023-07-14T11:27:03.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.439+0000 7fe60b59e700 1 -- start start 2023-07-14T11:27:03.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 0x7fe60c19b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60b59e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe60c100380 0x7fe60c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c1059e0 0x7fe60c195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c195da0 con 0x7fe60c0ff9d0 2023-07-14T11:27:03.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c195f10 con 0x7fe60c1059e0 2023-07-14T11:27:03.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60b59e700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c1960d0 con 0x7fe60c100380 2023-07-14T11:27:03.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 0x7fe60c19b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c1059e0 0x7fe60c195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c1059e0 0x7fe60c195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37422/0 (socket says 172.21.15.161:37422) 2023-07-14T11:27:03.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 0x7fe60c19b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34464/0 (socket says 172.21.15.161:34464) 2023-07-14T11:27:03.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 -- 172.21.15.161:0/1619438074 learned_addr learned my addr 172.21.15.161:0/1619438074 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:03.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe609d9b700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe60c100380 0x7fe60c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 -- 172.21.15.161:0/1619438074 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe60c100380 msgr2=0x7fe60c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe60c100380 0x7fe60c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 -- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 msgr2=0x7fe60c19b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 0x7fe60c19b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.440+0000 7fe60ad9d700 1 -- 172.21.15.161:0/1619438074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6000095f0 con 0x7fe60c1059e0 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.441+0000 7fe60ad9d700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c1059e0 0x7fe60c195730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe60400ef30 tx=0x7fe60400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.441+0000 7fe5fb7fe700 1 -- 172.21.15.161:0/1619438074 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604018070 con 0x7fe60c1059e0 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.441+0000 7fe5fb7fe700 1 -- 172.21.15.161:0/1619438074 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe60400f040 con 0x7fe60c1059e0 2023-07-14T11:27:03.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.441+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c196350 con 0x7fe60c1059e0 2023-07-14T11:27:03.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.441+0000 7fe5fb7fe700 1 -- 172.21.15.161:0/1619438074 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60400cd00 con 0x7fe60c1059e0 2023-07-14T11:27:03.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.441+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c1a26b0 con 0x7fe60c1059e0 2023-07-14T11:27:03.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.442+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c067220 con 0x7fe60c1059e0 2023-07-14T11:27:03.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.444+0000 7fe5fb7fe700 1 -- 172.21.15.161:0/1619438074 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe604003a00 con 0x7fe60c1059e0 2023-07-14T11:27:03.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.445+0000 7fe5fb7fe700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe5f40a13b0 0x7fe5f40a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:03.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.445+0000 7fe5fb7fe700 1 -- 172.21.15.161:0/1619438074 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe6040ccbd0 con 0x7fe60c1059e0 2023-07-14T11:27:03.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.446+0000 7fe60a59c700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe5f40a13b0 0x7fe5f40a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:03.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.446+0000 7fe60a59c700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe5f40a13b0 0x7fe5f40a3860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe60c100b10 tx=0x7fe5fc009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:03.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.449+0000 7fe5fb7fe700 1 -- 172.21.15.161:0/1619438074 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe604095ce0 con 0x7fe60c1059e0 2023-07-14T11:27:03.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.793+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe60c196c40 con 0x7fe60c1059e0 2023-07-14T11:27:03.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.800+0000 7fe5fb7fe700 1 -- 172.21.15.161:0/1619438074 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10864 (secure 0 0 0) 0x7fe604099300 con 0x7fe60c1059e0 2023-07-14T11:27:03.804 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:27:03.804 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:26:11.184716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:27:03.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe5f40a13b0 msgr2=0x7fe5f40a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe5f40a13b0 0x7fe5f40a3860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe60c100b10 tx=0x7fe5fc009260 comp rx=0 tx=0).stop 2023-07-14T11:27:03.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c1059e0 msgr2=0x7fe60c195730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:03.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c1059e0 0x7fe60c195730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe60400ef30 tx=0x7fe60400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 shutdown_connections 2023-07-14T11:27:03.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe5f40a13b0 0x7fe5f40a3860 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0ff9d0 0x7fe60c19b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c1059e0 0x7fe60c195730 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 --2- 172.21.15.161:0/1619438074 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe60c100380 0x7fe60c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:03.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 >> 172.21.15.161:0/1619438074 conn(0x7fe60c0fb4d0 msgr2=0x7fe60c103520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:03.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 shutdown_connections 2023-07-14T11:27:03.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:03.803+0000 7fe60b59e700 1 -- 172.21.15.161:0/1619438074 wait complete. 2023-07-14T11:27:03.809 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 25 2023-07-14T11:27:03.827 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:27:04.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1979402843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc073160 msgr2=0x7fcbbc10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1979402843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc073160 0x7fcbbc10be40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009990 tx=0x7fcbb0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1979402843 shutdown_connections 2023-07-14T11:27:04.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1979402843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc072850 0x7fcbbc072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1979402843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc10c380 0x7fcbbc10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1979402843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc073160 0x7fcbbc10be40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1979402843 >> 172.21.15.161:0/1979402843 conn(0x7fcbbc0fb5e0 msgr2=0x7fcbbc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1979402843 shutdown_connections 2023-07-14T11:27:04.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.058+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1979402843 wait complete. 2023-07-14T11:27:04.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.059+0000 7fcbc0dff700 1 Processor -- start 2023-07-14T11:27:04.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbc0dff700 1 -- start start 2023-07-14T11:27:04.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbc0dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc072850 0x7fcbbc197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbc0dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc073160 0x7fcbbc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbc0dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc10c380 0x7fcbbc19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbc0dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc19cdd0 con 0x7fcbbc073160 2023-07-14T11:27:04.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbc0dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc19cf40 con 0x7fcbbc10c380 2023-07-14T11:27:04.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbc0dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc19d100 con 0x7fcbbc072850 2023-07-14T11:27:04.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc072850 0x7fcbbc197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbbad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc10c380 0x7fcbbc19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbb9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc073160 0x7fcbbc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc072850 0x7fcbbc197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58432/0 (socket says 172.21.15.161:58432) 2023-07-14T11:27:04.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.060+0000 7fcbba59c700 1 -- 172.21.15.161:0/1139162996 learned_addr learned my addr 172.21.15.161:0/1139162996 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbba59c700 1 -- 172.21.15.161:0/1139162996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc10c380 msgr2=0x7fcbbc19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbba59c700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc10c380 0x7fcbbc19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbba59c700 1 -- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc073160 msgr2=0x7fcbbc198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbba59c700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc073160 0x7fcbbc198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbba59c700 1 -- 172.21.15.161:0/1139162996 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb00095f0 con 0x7fcbbc072850 2023-07-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbba59c700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc072850 0x7fcbbc197df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00e9a0 tx=0x7fcbac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbab7fe700 1 -- 172.21.15.161:0/1139162996 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac009790 con 0x7fcbbc072850 2023-07-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc19d3e0 con 0x7fcbbc072850 2023-07-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbab7fe700 1 -- 172.21.15.161:0/1139162996 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac010430 con 0x7fcbbc072850 2023-07-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbab7fe700 1 -- 172.21.15.161:0/1139162996 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac018680 con 0x7fcbbc072850 2023-07-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.061+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc1a4c20 con 0x7fcbbc072850 2023-07-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.065+0000 7fcbab7fe700 1 -- 172.21.15.161:0/1139162996 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcbac0105a0 con 0x7fcbbc072850 2023-07-14T11:27:04.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.066+0000 7fcbd46bbb80 1 Processor -- start 2023-07-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.066+0000 7fcbd46bbb80 1 -- start start 2023-07-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.066+0000 7fcbab7fe700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcba40a12c0 0x7fcba40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.066+0000 7fcbab7fe700 1 -- 172.21.15.161:0/1139162996 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fcbac014070 con 0x7fcbbc072850 2023-07-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.066+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a54fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fc3f0 0x55f8a5506da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a55072e0 0x55f8a55096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a53f9cb0 con 0x55f8a54fc3f0 2023-07-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a544ca40 con 0x55f8a55072e0 2023-07-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a5445ed0 con 0x55f8a54fbae0 2023-07-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a54fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a54fbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58434/0 (socket says 172.21.15.161:58434) 2023-07-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 -- 172.21.15.161:0/1042883484 learned_addr learned my addr 172.21.15.161:0/1042883484 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbbad9d700 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a55072e0 0x55f8a55096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fc3f0 0x55f8a5506da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbbad9d700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcba40a12c0 0x7fcba40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 -- 172.21.15.161:0/1042883484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a55072e0 msgr2=0x55f8a55096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a55072e0 0x55f8a55096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 -- 172.21.15.161:0/1042883484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fc3f0 msgr2=0x55f8a5506da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fc3f0 0x55f8a5506da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbba59c700 1 -- 172.21.15.161:0/1042883484 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8a5273de0 con 0x55f8a54fbae0 2023-07-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.067+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fc3f0 0x55f8a5506da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.068+0000 7fcbba59c700 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a54fbeb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbac096a10 tx=0x7fcbac099c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c7eb0d25144df2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.068+0000 7fcbbad9d700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcba40a12c0 0x7fcba40a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009ec0 tx=0x7fcbb400b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.068+0000 7fcba8ff9700 1 -- 172.21.15.161:0/1042883484 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0a23f0 con 0x55f8a54fbae0 2023-07-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.068+0000 7fcba8ff9700 1 -- 172.21.15.161:0/1042883484 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac010880 con 0x55f8a54fbae0 2023-07-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.068+0000 7fcba8ff9700 1 -- 172.21.15.161:0/1042883484 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0a1f60 con 0x55f8a54fbae0 2023-07-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.068+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/1042883484 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 msgr2=0x55f8a54fbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.068+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a54fbeb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbac096a10 tx=0x7fcbac099c10 comp rx=0 tx=0).stop 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/1042883484 shutdown_connections 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a55072e0 0x55f8a55096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a54fbeb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/1042883484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fc3f0 0x55f8a5506da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/1042883484 >> 172.21.15.161:0/1042883484 conn(0x55f8a54e9da0 msgr2=0x55f8a54ef6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/1042883484 shutdown_connections 2023-07-14T11:27:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/1042883484 wait complete. 2023-07-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 -- start start 2023-07-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.069+0000 7fcbd46bbb80 1 Processor -- start 2023-07-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fbae0 0x55f8a5504090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fc3f0 0x55f8a55045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5504b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a53e5ac0 con 0x55f8a55072e0 2023-07-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a53e57c0 con 0x55f8a54fbae0 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a52c9920 con 0x55f8a54fc3f0 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbd46bbb80 10 client.0 _ll_register_callbacks cb 0x55f8a53e3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fbae0 0x55f8a5504090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbb9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fc3f0 0x55f8a55045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fbae0 0x55f8a5504090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37452/0 (socket says 172.21.15.161:37452) 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbb9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fc3f0 0x55f8a55045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58444/0 (socket says 172.21.15.161:58444) 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbba59c700 1 -- 172.21.15.161:0/920171268 learned_addr learned my addr 172.21.15.161:0/920171268 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbbad9d700 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5504b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbb9d9b700 1 -- 172.21.15.161:0/920171268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fbae0 msgr2=0x55f8a5504090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fbae0 0x55f8a5504090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbb9d9b700 1 -- 172.21.15.161:0/920171268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 msgr2=0x55f8a5504b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5504b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbb9d9b700 1 -- 172.21.15.161:0/920171268 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc1a4c20 con 0x55f8a54fc3f0 2023-07-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.070+0000 7fcbbad9d700 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5504b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fc3f0 0x55f8a55045d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000b2e0 tx=0x7fcbb000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59405ee9bd57979 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcba1ffb700 1 -- 172.21.15.161:0/920171268 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb001d070 con 0x55f8a54fc3f0 2023-07-14T11:27:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8a539f4f0 con 0x55f8a54fc3f0 2023-07-14T11:27:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcba1ffb700 1 -- 172.21.15.161:0/920171268 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fcbb00056b0 con 0x55f8a54fc3f0 2023-07-14T11:27:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcba1ffb700 1 -- 172.21.15.161:0/920171268 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbb0004700 con 0x55f8a54fc3f0 2023-07-14T11:27:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcbd46bbb80 10 client.26584 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.071+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8a525cc60 con 0x55f8a54fc3f0 2023-07-14T11:27:04.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcba1ffb700 1 -- 172.21.15.161:0/920171268 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8a539f4f0 con 0x55f8a54fc3f0 2023-07-14T11:27:04.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcba1ffb700 1 -- 172.21.15.161:0/920171268 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbb0024780 con 0x55f8a54fc3f0 2023-07-14T11:27:04.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcbd46bbb80 10 client.26584 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:04.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcbd46bbb80 10 client.26584 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:27:04.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcbd46bbb80 20 client.26584 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:04.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a5516cb0 0x55f8a55190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcbd46bbb80 4 client.26584 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:04.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8a543eeb0 con 0x55f8a5516cb0 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.072+0000 7fcbba59c700 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a5516cb0 0x55f8a55190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.073+0000 7fcba1ffb700 10 client.26584 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.073+0000 7fcbba59c700 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a5516cb0 0x55f8a55190c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f49e9ffbe1bbffe6 server_cookie=6c8c6fefbde1a151 in_seq=0 out_seq=0 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.073+0000 7fcba1ffb700 1 -- 172.21.15.161:0/920171268 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f8a543eeb0 con 0x55f8a5516cb0 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.073+0000 7fcba1ffb700 10 client.26584 handle_command_reply: tid=0 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.073+0000 7fcbd46bbb80 1 client.26584 shutdown 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fc3f0 msgr2=0x55f8a55045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fc3f0 0x55f8a55045d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000b2e0 tx=0x7fcbb000bf60 comp rx=0 tx=0).stop 2023-07-14T11:27:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 shutdown_connections 2023-07-14T11:27:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a5516cb0 0x55f8a55190c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5504b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fbae0 0x55f8a5504090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/920171268 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fc3f0 0x55f8a55045d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcba1ffb700 0 client.26584 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 >> 172.21.15.161:0/920171268 conn(0x55f8a54e9da0 msgr2=0x55f8a54ef290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 shutdown_connections 2023-07-14T11:27:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/920171268 wait complete. 2023-07-14T11:27:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.074+0000 7fcbd46bbb80 20 client.26584 trim_cache size 0 max 16384 2023-07-14T11:27:04.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.088+0000 7fcbd46bbb80 1 Processor -- start 2023-07-14T11:27:04.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.088+0000 7fcbd46bbb80 1 -- start start 2023-07-14T11:27:04.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fbae0 0x55f8a550dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a5504940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a55072e0 0x55f8a5501300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a54ebec0 con 0x55f8a54fbae0 2023-07-14T11:27:04.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a54fcd50 con 0x55f8a54fc3f0 2023-07-14T11:27:04.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a54fd570 con 0x55f8a55072e0 2023-07-14T11:27:04.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a5504940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fbae0 0x55f8a550dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a5504940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37462/0 (socket says 172.21.15.161:37462) 2023-07-14T11:27:04.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 -- 172.21.15.161:0/3551348817 learned_addr learned my addr 172.21.15.161:0/3551348817 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbbad9d700 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a55072e0 0x55f8a5501300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 -- 172.21.15.161:0/3551348817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a55072e0 msgr2=0x55f8a5501300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a55072e0 0x55f8a5501300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 -- 172.21.15.161:0/3551348817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fbae0 msgr2=0x55f8a550dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fbae0 0x55f8a550dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.089+0000 7fcbb9d9b700 1 -- 172.21.15.161:0/3551348817 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8a54eb5e0 con 0x55f8a54fc3f0 2023-07-14T11:27:04.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.090+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a5504940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0024b90 tx=0x7fcbb0024d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36a565b2eedaba18 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.090+0000 7fcba97fa700 1 -- 172.21.15.161:0/3551348817 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb001d070 con 0x55f8a54fc3f0 2023-07-14T11:27:04.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.090+0000 7fcba97fa700 1 -- 172.21.15.161:0/3551348817 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbb001e040 con 0x55f8a54fc3f0 2023-07-14T11:27:04.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.090+0000 7fcba97fa700 1 -- 172.21.15.161:0/3551348817 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000e7f0 con 0x55f8a54fc3f0 2023-07-14T11:27:04.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.090+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/3551348817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 msgr2=0x55f8a5504940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.090+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a5504940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0024b90 tx=0x7fcbb0024d30 comp rx=0 tx=0).stop 2023-07-14T11:27:04.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/3551348817 shutdown_connections 2023-07-14T11:27:04.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a54fbae0 0x55f8a550dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a55072e0 0x55f8a5501300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/3551348817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a5504940 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/3551348817 >> 172.21.15.161:0/3551348817 conn(0x55f8a54e9da0 msgr2=0x55f8a54ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/3551348817 shutdown_connections 2023-07-14T11:27:04.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/3551348817 wait complete. 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 -- start start 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.091+0000 7fcbd46bbb80 1 Processor -- start 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a550b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a550bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbd46bbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5569b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a550c510 con 0x55f8a55072e0 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a556a0b0 con 0x55f8a54fc3f0 2023-07-14T11:27:04.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbd46bbb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a556a1f0 con 0x55f8a54fbae0 2023-07-14T11:27:04.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbd46bbb80 10 client.0 _ll_register_callbacks cb 0x7fcbb0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:04.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a550b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a550b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58468/0 (socket says 172.21.15.161:58468) 2023-07-14T11:27:04.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 -- 172.21.15.161:0/62215252 learned_addr learned my addr 172.21.15.161:0/62215252 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a550bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbbad9d700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5569b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 -- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 msgr2=0x55f8a550bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a550bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 -- 172.21.15.161:0/62215252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 msgr2=0x55f8a5569b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5569b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbba59c700 1 -- 172.21.15.161:0/62215252 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8a525cc60 con 0x55f8a54fbae0 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.092+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a550bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbba59c700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a550b6c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcbac096a10 tx=0x7fcbac0cd160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7bf9d996b23deb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcba1ffb700 1 -- 172.21.15.161:0/62215252 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac099690 con 0x55f8a54fbae0 2023-07-14T11:27:04.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcba1ffb700 1 -- 172.21.15.161:0/62215252 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fcbac09a200 con 0x55f8a54fbae0 2023-07-14T11:27:04.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbac010880 con 0x55f8a54fbae0 2023-07-14T11:27:04.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcba1ffb700 1 -- 172.21.15.161:0/62215252 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbac09a3c0 con 0x55f8a54fbae0 2023-07-14T11:27:04.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbd46bbb80 10 client.26590 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:04.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8a529a250 con 0x55f8a54fbae0 2023-07-14T11:27:04.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcba1ffb700 1 -- 172.21.15.161:0/62215252 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac010880 con 0x55f8a54fbae0 2023-07-14T11:27:04.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcba1ffb700 1 -- 172.21.15.161:0/62215252 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbac094bb0 con 0x55f8a54fbae0 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbd46bbb80 10 client.26590 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbd46bbb80 10 client.26590 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbd46bbb80 20 client.26590 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.093+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a556b5b0 0x55f8a556d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.094+0000 7fcbd46bbb80 4 client.26590 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.094+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f8a5401050 con 0x55f8a556b5b0 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.094+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a556b5b0 0x55f8a556d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.094+0000 7fcba1ffb700 10 client.26590 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:04.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.094+0000 7fcbb9d9b700 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a556b5b0 0x55f8a556d9c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fcb251624b4a4dd server_cookie=36449373474774da in_seq=0 out_seq=0 2023-07-14T11:27:04.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcba1ffb700 1 -- 172.21.15.161:0/62215252 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f8a5401050 con 0x55f8a556b5b0 2023-07-14T11:27:04.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcba1ffb700 10 client.26590 handle_command_reply: tid=0 2023-07-14T11:27:04.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 client.26590 shutdown 2023-07-14T11:27:04.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 msgr2=0x55f8a550b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a550b6c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcbac096a10 tx=0x7fcbac0cd160 comp rx=0 tx=0).stop 2023-07-14T11:27:04.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 shutdown_connections 2023-07-14T11:27:04.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55f8a556b5b0 0x55f8a556d9c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f8a55072e0 0x55f8a5569b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcba1ffb700 0 client.26590 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f8a54fbae0 0x55f8a550b6c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 --2- 172.21.15.161:0/62215252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f8a54fc3f0 0x55f8a550bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 >> 172.21.15.161:0/62215252 conn(0x55f8a54e9da0 msgr2=0x55f8a54fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 shutdown_connections 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 1 -- 172.21.15.161:0/62215252 wait complete. 2023-07-14T11:27:04.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.095+0000 7fcbd46bbb80 20 client.26590 trim_cache size 0 max 16384 2023-07-14T11:27:04.108 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:27:04.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcba40a12c0 msgr2=0x7fcba40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcba40a12c0 0x7fcba40a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009ec0 tx=0x7fcbb400b040 comp rx=0 tx=0).stop 2023-07-14T11:27:04.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc072850 msgr2=0x7fcbbc197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc072850 0x7fcbbc197df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00e9a0 tx=0x7fcbac00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 shutdown_connections 2023-07-14T11:27:04.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcba40a12c0 0x7fcba40a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc10c380 0x7fcbbc19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcbbc072850 0x7fcbbc197df0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 --2- 172.21.15.161:0/1139162996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc073160 0x7fcbbc198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 >> 172.21.15.161:0/1139162996 conn(0x7fcbbc0fb5e0 msgr2=0x7fcbbc103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 shutdown_connections 2023-07-14T11:27:04.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.096+0000 7fcbc0dff700 1 -- 172.21.15.161:0/1139162996 wait complete. 2023-07-14T11:27:04.120 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:03 smithi161 ceph-mon[95526]: pgmap v147: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2023-07-14T11:27:04.121 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:27:04.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:03 smithi175 ceph-mon[129081]: pgmap v147: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2023-07-14T11:27:04.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 420473 9.74 MB/sec execute 106 sec latency 69.511 ms 2023-07-14T11:27:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:03 smithi182 ceph-mon[129366]: pgmap v147: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2023-07-14T11:27:04.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 -- 172.21.15.161:0/3806778017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa270100490 msgr2=0x7fa27010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3806778017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa270100490 0x7fa27010c920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b0a0 tx=0x7fa26800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 -- 172.21.15.161:0/3806778017 shutdown_connections 2023-07-14T11:27:04.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3806778017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa270100490 0x7fa27010c920 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3806778017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270105af0 0x7fa270105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3806778017 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2700ffae0 0x7fa2700fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 -- 172.21.15.161:0/3806778017 >> 172.21.15.161:0/3806778017 conn(0x7fa2700fb5e0 msgr2=0x7fa2700fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 -- 172.21.15.161:0/3806778017 shutdown_connections 2023-07-14T11:27:04.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.345+0000 7fa2759ff700 1 -- 172.21.15.161:0/3806778017 wait complete. 2023-07-14T11:27:04.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.346+0000 7fa2759ff700 1 Processor -- start 2023-07-14T11:27:04.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.346+0000 7fa2759ff700 1 -- start start 2023-07-14T11:27:04.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa2759ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ffae0 0x7fa270072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa2759ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270100490 0x7fa270072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa2759ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa270105af0 0x7fa2701a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa2759ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270073580 con 0x7fa2700ffae0 2023-07-14T11:27:04.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa2759ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2701a43f0 con 0x7fa270100490 2023-07-14T11:27:04.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa2759ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2701a4530 con 0x7fa270105af0 2023-07-14T11:27:04.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270100490 0x7fa270072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ffae0 0x7fa270072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270100490 0x7fa270072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37488/0 (socket says 172.21.15.161:37488) 2023-07-14T11:27:04.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26e7fc700 1 -- 172.21.15.161:0/3851380302 learned_addr learned my addr 172.21.15.161:0/3851380302 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa270105af0 0x7fa2701a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26f7fe700 1 -- 172.21.15.161:0/3851380302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270100490 msgr2=0x7fa270072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270100490 0x7fa270072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26f7fe700 1 -- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ffae0 msgr2=0x7fa270072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ffae0 0x7fa270072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26f7fe700 1 -- 172.21.15.161:0/3851380302 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268009d00 con 0x7fa270105af0 2023-07-14T11:27:04.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.347+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270100490 0x7fa270072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:04.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.348+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa270105af0 0x7fa2701a3eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa268009fd0 tx=0x7fa26800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.348+0000 7fa2749fd700 1 -- 172.21.15.161:0/3851380302 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800e040 con 0x7fa270105af0 2023-07-14T11:27:04.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.348+0000 7fa2749fd700 1 -- 172.21.15.161:0/3851380302 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2680094c0 con 0x7fa270105af0 2023-07-14T11:27:04.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.348+0000 7fa2749fd700 1 -- 172.21.15.161:0/3851380302 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268005040 con 0x7fa270105af0 2023-07-14T11:27:04.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.348+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2701a4730 con 0x7fa270105af0 2023-07-14T11:27:04.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.348+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2701a4b60 con 0x7fa270105af0 2023-07-14T11:27:04.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.351+0000 7fa2749fd700 1 -- 172.21.15.161:0/3851380302 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa26801d470 con 0x7fa270105af0 2023-07-14T11:27:04.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.353+0000 7fa289311b80 1 Processor -- start 2023-07-14T11:27:04.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.353+0000 7fa289311b80 1 -- start start 2023-07-14T11:27:04.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa2749fd700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2580a12c0 0x7fa2580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa289311b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba53c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa289311b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba547590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa289311b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba549eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa289311b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba442cb0 con 0x561fba53c2d0 2023-07-14T11:27:04.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa289311b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba495a40 con 0x561fba547ad0 2023-07-14T11:27:04.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa289311b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba48eed0 con 0x561fba53cbe0 2023-07-14T11:27:04.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa2749fd700 1 -- 172.21.15.161:0/3851380302 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa2680ce710 con 0x7fa270105af0 2023-07-14T11:27:04.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa26f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba549eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2580a12c0 0x7fa2580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa26f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba549eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37504/0 (socket says 172.21.15.161:37504) 2023-07-14T11:27:04.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa26f7fe700 1 -- 172.21.15.161:0/2429859084 learned_addr learned my addr 172.21.15.161:0/2429859084 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa26effd700 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba53c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.354+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba547590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa26f7fe700 1 -- 172.21.15.161:0/2429859084 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 msgr2=0x561fba547590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba547590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa26f7fe700 1 -- 172.21.15.161:0/2429859084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 msgr2=0x561fba53c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba53c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa26f7fe700 1 -- 172.21.15.161:0/2429859084 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fba2bcde0 con 0x561fba547ad0 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba547590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba549eb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa2680979e0 tx=0x7fa268004e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6808c00f8cd125f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa255ffb700 1 -- 172.21.15.161:0/2429859084 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26809ae40 con 0x561fba547ad0 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa255ffb700 1 -- 172.21.15.161:0/2429859084 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa268078d90 con 0x561fba547ad0 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.355+0000 7fa255ffb700 1 -- 172.21.15.161:0/2429859084 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26809b550 con 0x561fba547ad0 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 -- 172.21.15.161:0/2429859084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 msgr2=0x561fba549eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba549eb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa2680979e0 tx=0x7fa268004e60 comp rx=0 tx=0).stop 2023-07-14T11:27:04.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2580a12c0 0x7fa2580a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa270074000 tx=0x7fa264007c10 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 -- 172.21.15.161:0/2429859084 shutdown_connections 2023-07-14T11:27:04.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba53c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba549eb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 --2- 172.21.15.161:0/2429859084 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba547590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 -- 172.21.15.161:0/2429859084 >> 172.21.15.161:0/2429859084 conn(0x561fba532da0 msgr2=0x561fba5386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 -- 172.21.15.161:0/2429859084 shutdown_connections 2023-07-14T11:27:04.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.356+0000 7fa289311b80 1 -- 172.21.15.161:0/2429859084 wait complete. 2023-07-14T11:27:04.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 -- start start 2023-07-14T11:27:04.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 Processor -- start 2023-07-14T11:27:04.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba547ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba5483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba548920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba42eac0 con 0x561fba53c2d0 2023-07-14T11:27:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba42e7c0 con 0x561fba547ad0 2023-07-14T11:27:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba312920 con 0x561fba53cbe0 2023-07-14T11:27:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.357+0000 7fa289311b80 10 client.0 _ll_register_callbacks cb 0x561fba42c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba548920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba5483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba547ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba548920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37520/0 (socket says 172.21.15.161:37520) 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 -- 172.21.15.161:0/3204121790 learned_addr learned my addr 172.21.15.161:0/3204121790 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 -- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 msgr2=0x561fba5483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba5483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 -- 172.21.15.161:0/3204121790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 msgr2=0x561fba547ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba547ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 -- 172.21.15.161:0/3204121790 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2701a4b60 con 0x561fba547ad0 2023-07-14T11:27:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba5483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba548920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa2680a3370 tx=0x7fa26809b0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4998de4bfff2780f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa24effd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa24effd700 1 -- 172.21.15.161:0/3204121790 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268078bb0 con 0x561fba547ad0 2023-07-14T11:27:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa24effd700 1 -- 172.21.15.161:0/3204121790 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa268078990 con 0x561fba547ad0 2023-07-14T11:27:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.358+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fba3e84f0 con 0x561fba547ad0 2023-07-14T11:27:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa24effd700 1 -- 172.21.15.161:0/3204121790 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa26802e350 con 0x561fba547ad0 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 10 client.26597 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561fba2a5c60 con 0x561fba547ad0 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa24effd700 1 -- 172.21.15.161:0/3204121790 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2680c9020 con 0x561fba547ad0 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa24effd700 1 -- 172.21.15.161:0/3204121790 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2680cbd50 con 0x561fba547ad0 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 10 client.26597 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 10 client.26597 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 20 client.26597 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba556d60 0x561fba559140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 4 client.26597 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:04.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fba487eb0 con 0x561fba556d60 2023-07-14T11:27:04.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.359+0000 7fa26effd700 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba556d60 0x561fba559140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.360+0000 7fa26effd700 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba556d60 0x561fba559140 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5896e8f0d4d40a22 server_cookie=ce1b56471110a96d in_seq=0 out_seq=0 2023-07-14T11:27:04.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.360+0000 7fa24effd700 10 client.26597 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:04.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa24effd700 1 -- 172.21.15.161:0/3204121790 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561fba487eb0 con 0x561fba556d60 2023-07-14T11:27:04.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa24effd700 10 client.26597 handle_command_reply: tid=0 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 client.26597 shutdown 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 msgr2=0x561fba548920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba548920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa2680a3370 tx=0x7fa26809b0e0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 shutdown_connections 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba556d60 0x561fba559140 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba547ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba548920 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 --2- 172.21.15.161:0/3204121790 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba5483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 >> 172.21.15.161:0/3204121790 conn(0x561fba532da0 msgr2=0x561fba538290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa24effd700 0 client.26597 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 shutdown_connections 2023-07-14T11:27:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 1 -- 172.21.15.161:0/3204121790 wait complete. 2023-07-14T11:27:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.361+0000 7fa289311b80 20 client.26597 trim_cache size 0 max 16384 2023-07-14T11:27:04.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 Processor -- start 2023-07-14T11:27:04.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 -- start start 2023-07-14T11:27:04.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba55daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba54cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba553450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba55e010 con 0x561fba53c2d0 2023-07-14T11:27:04.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba54d3e0 con 0x561fba547ad0 2023-07-14T11:27:04.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.376+0000 7fa289311b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba559140 con 0x561fba53cbe0 2023-07-14T11:27:04.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba54cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba553450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba54cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58510/0 (socket says 172.21.15.161:58510) 2023-07-14T11:27:04.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 -- 172.21.15.161:0/1946913149 learned_addr learned my addr 172.21.15.161:0/1946913149 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26effd700 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba55daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 -- 172.21.15.161:0/1946913149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 msgr2=0x561fba553450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba553450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 -- 172.21.15.161:0/1946913149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 msgr2=0x561fba55daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba55daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 -- 172.21.15.161:0/1946913149 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fba2e3250 con 0x561fba53cbe0 2023-07-14T11:27:04.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26effd700 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba55daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:04.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba54cea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa264012210 tx=0x7fa264014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd6216c170036e7a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.377+0000 7fa2567fc700 1 -- 172.21.15.161:0/1946913149 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26402a630 con 0x561fba53cbe0 2023-07-14T11:27:04.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa2567fc700 1 -- 172.21.15.161:0/1946913149 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa264025070 con 0x561fba53cbe0 2023-07-14T11:27:04.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa2567fc700 1 -- 172.21.15.161:0/1946913149 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264021650 con 0x561fba53cbe0 2023-07-14T11:27:04.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 -- 172.21.15.161:0/1946913149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 msgr2=0x561fba54cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba54cea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa264012210 tx=0x7fa264014040 comp rx=0 tx=0).stop 2023-07-14T11:27:04.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 -- 172.21.15.161:0/1946913149 shutdown_connections 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba53c2d0 0x561fba55daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba547ad0 0x561fba553450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 --2- 172.21.15.161:0/1946913149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53cbe0 0x561fba54cea0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 -- 172.21.15.161:0/1946913149 >> 172.21.15.161:0/1946913149 conn(0x561fba532da0 msgr2=0x561fba548730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 -- 172.21.15.161:0/1946913149 shutdown_connections 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.378+0000 7fa289311b80 1 -- 172.21.15.161:0/1946913149 wait complete. 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.379+0000 7fa289311b80 1 -- start start 2023-07-14T11:27:04.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.379+0000 7fa289311b80 1 Processor -- start 2023-07-14T11:27:04.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa289311b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53c2d0 0x561fba54c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa289311b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba53cbe0 0x561fba548e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa289311b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba547ad0 0x561fba5493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa289311b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba54d280 con 0x561fba547ad0 2023-07-14T11:27:04.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa289311b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba5498e0 con 0x561fba53cbe0 2023-07-14T11:27:04.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa289311b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fba549aa0 con 0x561fba53c2d0 2023-07-14T11:27:04.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa289311b80 10 client.0 _ll_register_callbacks cb 0x7fa260009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa26e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba53cbe0 0x561fba548e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa26e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba53cbe0 0x561fba548e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37536/0 (socket says 172.21.15.161:37536) 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa26e7fc700 1 -- 172.21.15.161:0/592394704 learned_addr learned my addr 172.21.15.161:0/592394704 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa26effd700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53c2d0 0x561fba54c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.380+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba547ad0 0x561fba5493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa26e7fc700 1 -- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53c2d0 msgr2=0x561fba54c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53c2d0 0x561fba54c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa26e7fc700 1 -- 172.21.15.161:0/592394704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba547ad0 msgr2=0x561fba5493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba547ad0 0x561fba5493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa26e7fc700 1 -- 172.21.15.161:0/592394704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2640126d0 con 0x561fba53cbe0 2023-07-14T11:27:04.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa26f7fe700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba547ad0 0x561fba5493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:04.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa26e7fc700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba53cbe0 0x561fba548e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa264007be0 tx=0x7fa264012590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4823b5b74697 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa24effd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:04.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa24effd700 1 -- 172.21.15.161:0/592394704 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264010420 con 0x561fba53cbe0 2023-07-14T11:27:04.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa268078d90 con 0x561fba53cbe0 2023-07-14T11:27:04.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa24effd700 1 -- 172.21.15.161:0/592394704 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa264010580 con 0x561fba53cbe0 2023-07-14T11:27:04.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa24effd700 1 -- 172.21.15.161:0/592394704 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa264010740 con 0x561fba53cbe0 2023-07-14T11:27:04.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa289311b80 10 client.26603 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:04.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.381+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561fba2bb2b0 con 0x561fba53cbe0 2023-07-14T11:27:04.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa24effd700 1 -- 172.21.15.161:0/592394704 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa268078d90 con 0x561fba53cbe0 2023-07-14T11:27:04.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa24effd700 1 -- 172.21.15.161:0/592394704 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa264004910 con 0x561fba53cbe0 2023-07-14T11:27:04.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa289311b80 10 client.26603 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:04.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa289311b80 10 client.26603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:27:04.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa289311b80 20 client.26603 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:04.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa289311b80 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba5abec0 0x561fba5ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa289311b80 4 client.26603 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:27:04.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561fba44a050 con 0x561fba5abec0 2023-07-14T11:27:04.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.382+0000 7fa26effd700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba5abec0 0x561fba5ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa26effd700 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba5abec0 0x561fba5ae2d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d55cad451977d04a server_cookie=d6ecba5dc3cf661f in_seq=0 out_seq=0 2023-07-14T11:27:04.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa24effd700 10 client.26603 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:04.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa24effd700 1 -- 172.21.15.161:0/592394704 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561fba44a050 con 0x561fba5abec0 2023-07-14T11:27:04.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa24effd700 10 client.26603 handle_command_reply: tid=0 2023-07-14T11:27:04.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 client.26603 shutdown 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba53cbe0 msgr2=0x561fba548e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba53cbe0 0x561fba548e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa264007be0 tx=0x7fa264012590 comp rx=0 tx=0).stop 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 shutdown_connections 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x561fba5abec0 0x561fba5ae2d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fba547ad0 0x561fba5493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa24effd700 0 client.26603 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561fba53c2d0 0x561fba54c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 --2- 172.21.15.161:0/592394704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fba53cbe0 0x561fba548e60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 >> 172.21.15.161:0/592394704 conn(0x561fba532da0 msgr2=0x561fba548730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 shutdown_connections 2023-07-14T11:27:04.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 1 -- 172.21.15.161:0/592394704 wait complete. 2023-07-14T11:27:04.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.383+0000 7fa289311b80 20 client.26603 trim_cache size 0 max 16384 2023-07-14T11:27:04.395 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:27:04.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2580a12c0 msgr2=0x7fa2580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2580a12c0 0x7fa2580a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa270074000 tx=0x7fa264007c10 comp rx=0 tx=0).stop 2023-07-14T11:27:04.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa270105af0 msgr2=0x7fa2701a3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa270105af0 0x7fa2701a3eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa268009fd0 tx=0x7fa26800bce0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 shutdown_connections 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2580a12c0 0x7fa2580a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ffae0 0x7fa270072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa270105af0 0x7fa2701a3eb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 --2- 172.21.15.161:0/3851380302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa270100490 0x7fa270072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 >> 172.21.15.161:0/3851380302 conn(0x7fa2700fb5e0 msgr2=0x7fa2700fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 shutdown_connections 2023-07-14T11:27:04.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.384+0000 7fa2759ff700 1 -- 172.21.15.161:0/3851380302 wait complete. 2023-07-14T11:27:04.406 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:27:04.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.641+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3502645143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 msgr2=0x7fdcec105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.641+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3502645143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 0x7fdcec105ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009990 tx=0x7fdcdc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.641+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3502645143 shutdown_connections 2023-07-14T11:27:04.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.641+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3502645143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec1004b0 0x7fdcec10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.641+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3502645143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 0x7fdcec105ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.641+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3502645143 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 0x7fdcec0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.641+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3502645143 >> 172.21.15.161:0/3502645143 conn(0x7fdcec0fb600 msgr2=0x7fdcec0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.642+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3502645143 shutdown_connections 2023-07-14T11:27:04.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.642+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3502645143 wait complete. 2023-07-14T11:27:04.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 Processor -- start 2023-07-14T11:27:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 -- start start 2023-07-14T11:27:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 0x7fdcec197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec1004b0 0x7fdcec198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 0x7fdcec19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec19cde0 con 0x7fdcec1004b0 2023-07-14T11:27:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec19cf50 con 0x7fdcec105b10 2023-07-14T11:27:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf27ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec19d110 con 0x7fdcec0ffb00 2023-07-14T11:27:04.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 0x7fdcec197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 0x7fdcec197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58540/0 (socket says 172.21.15.161:58540) 2023-07-14T11:27:04.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcf0d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 0x7fdcec19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdcebfff700 1 -- 172.21.15.161:0/3177747206 learned_addr learned my addr 172.21.15.161:0/3177747206 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.643+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec1004b0 0x7fdcec198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcebfff700 1 -- 172.21.15.161:0/3177747206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 msgr2=0x7fdcec19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcebfff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 0x7fdcec19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcebfff700 1 -- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec1004b0 msgr2=0x7fdcec198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcebfff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec1004b0 0x7fdcec198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcebfff700 1 -- 172.21.15.161:0/3177747206 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0095f0 con 0x7fdcec0ffb00 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcebfff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 0x7fdcec197e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc005910 tx=0x7fdcdc004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdce97fa700 1 -- 172.21.15.161:0/3177747206 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc01d070 con 0x7fdcec0ffb00 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec19d390 con 0x7fdcec0ffb00 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdce97fa700 1 -- 172.21.15.161:0/3177747206 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcdc022470 con 0x7fdcec0ffb00 2023-07-14T11:27:04.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdce97fa700 1 -- 172.21.15.161:0/3177747206 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc017650 con 0x7fdcec0ffb00 2023-07-14T11:27:04.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.644+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec1a4c30 con 0x7fdcec0ffb00 2023-07-14T11:27:04.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.648+0000 7fdce97fa700 1 -- 172.21.15.161:0/3177747206 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdcdc0225e0 con 0x7fdcec0ffb00 2023-07-14T11:27:04.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.649+0000 7fdd060b2b80 1 Processor -- start 2023-07-14T11:27:04.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.649+0000 7fdd060b2b80 1 -- start start 2023-07-14T11:27:04.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.649+0000 7fdce97fa700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdcd40a12c0 0x7fdcd40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdce97fa700 1 -- 172.21.15.161:0/3177747206 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fdcdc0ce2c0 con 0x7fdcec0ffb00 2023-07-14T11:27:04.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdcf0d9c700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdcd40a12c0 0x7fdcd40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648186ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648185d6cb0 con 0x5648186d0be0 2023-07-14T11:27:04.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564818629a40 con 0x5648186d02d0 2023-07-14T11:27:04.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564818622ed0 con 0x5648186dbad0 2023-07-14T11:27:04.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37554/0 (socket says 172.21.15.161:37554) 2023-07-14T11:27:04.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdcebfff700 1 -- 172.21.15.161:0/2855469501 learned_addr learned my addr 172.21.15.161:0/2855469501 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.650+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcebfff700 1 -- 172.21.15.161:0/2855469501 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 msgr2=0x5648186ddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcebfff700 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648186ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcebfff700 1 -- 172.21.15.161:0/2855469501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 msgr2=0x5648186db590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcebfff700 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186db590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcebfff700 1 -- 172.21.15.161:0/2855469501 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564818450de0 con 0x5648186d02d0 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcf0d9c700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdcd40a12c0 0x7fdcd40a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdce400f540 tx=0x7fdce4009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcebfff700 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d06a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc006010 tx=0x7fdcdc09ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa43e3aff54fac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcd27fc700 1 -- 172.21.15.161:0/2855469501 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc09f0c0 con 0x5648186d02d0 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcd27fc700 1 -- 172.21.15.161:0/2855469501 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcdc0a3450 con 0x5648186d02d0 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.651+0000 7fdcd27fc700 1 -- 172.21.15.161:0/2855469501 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc09a7e0 con 0x5648186d02d0 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 -- 172.21.15.161:0/2855469501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 msgr2=0x5648186d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d06a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc006010 tx=0x7fdcdc09ac50 comp rx=0 tx=0).stop 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 -- 172.21.15.161:0/2855469501 shutdown_connections 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648186ddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d06a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/2855469501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186db590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 -- 172.21.15.161:0/2855469501 >> 172.21.15.161:0/2855469501 conn(0x5648186c6da0 msgr2=0x5648186cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 -- 172.21.15.161:0/2855469501 shutdown_connections 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.652+0000 7fdd060b2b80 1 -- 172.21.15.161:0/2855469501 wait complete. 2023-07-14T11:27:04.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.653+0000 7fdd060b2b80 1 -- start start 2023-07-14T11:27:04.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.653+0000 7fdd060b2b80 1 Processor -- start 2023-07-14T11:27:04.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648186daa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648185c2ac0 con 0x5648186d0be0 2023-07-14T11:27:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648185c27c0 con 0x5648186d02d0 2023-07-14T11:27:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648184a6920 con 0x5648186dbad0 2023-07-14T11:27:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdd060b2b80 10 client.0 _ll_register_callbacks cb 0x5648185c0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdceb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186e0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37570/0 (socket says 172.21.15.161:37570) 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 -- 172.21.15.161:0/1643131081 learned_addr learned my addr 172.21.15.161:0/1643131081 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcf0d9c700 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648186daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 -- 172.21.15.161:0/1643131081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 msgr2=0x5648186daa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648186daa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 -- 172.21.15.161:0/1643131081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 msgr2=0x5648186e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 -- 172.21.15.161:0/1643131081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec1a4c30 con 0x5648186d02d0 2023-07-14T11:27:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdcebfff700 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186e0b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ce0 tx=0x7fdcdc022b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2faa333ddf93bc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.654+0000 7fdccb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:04.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdccb7fe700 1 -- 172.21.15.161:0/1643131081 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc09f0c0 con 0x5648186d02d0 2023-07-14T11:27:04.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdccb7fe700 1 -- 172.21.15.161:0/1643131081 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fdcdc09b500 con 0x5648186d02d0 2023-07-14T11:27:04.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56481857c4f0 con 0x5648186d02d0 2023-07-14T11:27:04.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdccb7fe700 1 -- 172.21.15.161:0/1643131081 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcdc0b9250 con 0x5648186d02d0 2023-07-14T11:27:04.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 10 client.26615 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:04.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564818439c60 con 0x5648186d02d0 2023-07-14T11:27:04.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdccb7fe700 1 -- 172.21.15.161:0/1643131081 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56481857c4f0 con 0x5648186d02d0 2023-07-14T11:27:04.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdccb7fe700 1 -- 172.21.15.161:0/1643131081 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdcdc096e80 con 0x5648186d02d0 2023-07-14T11:27:04.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 10 client.26615 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:04.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 10 client.26615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:04.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 20 client.26615 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:04.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5648186eaa70 0x5648186ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 4 client.26615 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.655+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56481861beb0 con 0x5648186eaa70 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.656+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5648186eaa70 0x5648186ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.656+0000 7fdccb7fe700 10 client.26615 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.656+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5648186eaa70 0x5648186ece50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6cc1a60a89b1ce server_cookie=ea9f5fed602c05f4 in_seq=0 out_seq=0 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdccb7fe700 1 -- 172.21.15.161:0/1643131081 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56481861beb0 con 0x5648186eaa70 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdccb7fe700 10 client.26615 handle_command_reply: tid=0 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 client.26615 shutdown 2023-07-14T11:27:04.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 msgr2=0x5648186e0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186e0b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ce0 tx=0x7fdcdc022b30 comp rx=0 tx=0).stop 2023-07-14T11:27:04.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 shutdown_connections 2023-07-14T11:27:04.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5648186eaa70 0x5648186ece50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648186daa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186e0b20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdccb7fe700 0 client.26615 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:04.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/1643131081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x5648186e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 >> 172.21.15.161:0/1643131081 conn(0x5648186c6da0 msgr2=0x5648186cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 shutdown_connections 2023-07-14T11:27:04.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.657+0000 7fdd060b2b80 1 -- 172.21.15.161:0/1643131081 wait complete. 2023-07-14T11:27:04.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.658+0000 7fdd060b2b80 20 client.26615 trim_cache size 0 max 16384 2023-07-14T11:27:04.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 Processor -- start 2023-07-14T11:27:04.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 -- start start 2023-07-14T11:27:04.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186d0be0 0x5648186d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186dbad0 0x5648186d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648186d6650 con 0x5648186dbad0 2023-07-14T11:27:04.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648186c6970 con 0x5648186d02d0 2023-07-14T11:27:04.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.674+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648186ed410 con 0x5648186d0be0 2023-07-14T11:27:04.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37578/0 (socket says 172.21.15.161:37578) 2023-07-14T11:27:04.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 -- 172.21.15.161:0/3293891664 learned_addr learned my addr 172.21.15.161:0/3293891664 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcf0d9c700 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186dbad0 0x5648186d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186d0be0 0x5648186d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 -- 172.21.15.161:0/3293891664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186d0be0 msgr2=0x5648186d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186d0be0 0x5648186d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 -- 172.21.15.161:0/3293891664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186dbad0 msgr2=0x5648186d81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186dbad0 0x5648186d81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 -- 172.21.15.161:0/3293891664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564818477250 con 0x5648186d02d0 2023-07-14T11:27:04.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.675+0000 7fdcebfff700 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d98c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc09ad90 tx=0x7fdcdc09ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b46b59771735f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdcd2ffd700 1 -- 172.21.15.161:0/3293891664 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004780 con 0x5648186d02d0 2023-07-14T11:27:04.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdcd2ffd700 1 -- 172.21.15.161:0/3293891664 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcdc0a3200 con 0x5648186d02d0 2023-07-14T11:27:04.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdcd2ffd700 1 -- 172.21.15.161:0/3293891664 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc096e80 con 0x5648186d02d0 2023-07-14T11:27:04.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 -- 172.21.15.161:0/3293891664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 msgr2=0x5648186d98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d98c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc09ad90 tx=0x7fdcdc09ac50 comp rx=0 tx=0).stop 2023-07-14T11:27:04.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 -- 172.21.15.161:0/3293891664 shutdown_connections 2023-07-14T11:27:04.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186dbad0 0x5648186d81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x5648186d98c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/3293891664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186d0be0 0x5648186d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 -- 172.21.15.161:0/3293891664 >> 172.21.15.161:0/3293891664 conn(0x5648186c6da0 msgr2=0x5648186d91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.676+0000 7fdd060b2b80 1 -- 172.21.15.161:0/3293891664 shutdown_connections 2023-07-14T11:27:04.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.677+0000 7fdd060b2b80 1 -- 172.21.15.161:0/3293891664 wait complete. 2023-07-14T11:27:04.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.677+0000 7fdd060b2b80 1 -- start start 2023-07-14T11:27:04.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.677+0000 7fdd060b2b80 1 Processor -- start 2023-07-14T11:27:04.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x564818736c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x564818737140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdd060b2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648187327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564818732e90 con 0x5648186d0be0 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564818733000 con 0x5648186d02d0 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdd060b2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648187331c0 con 0x5648186dbad0 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdd060b2b80 10 client.0 _ll_register_callbacks cb 0x7fdcdc09aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x564818736c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdceb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x564818737140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x564818736c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37580/0 (socket says 172.21.15.161:37580) 2023-07-14T11:27:04.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 -- 172.21.15.161:0/15916191 learned_addr learned my addr 172.21.15.161:0/15916191 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcf0d9c700 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648187327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 -- 172.21.15.161:0/15916191 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 msgr2=0x5648187327b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648187327b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 -- 172.21.15.161:0/15916191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 msgr2=0x564818737140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x564818737140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 -- 172.21.15.161:0/15916191 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564818439c60 con 0x5648186d02d0 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x564818737140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:04.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdcebfff700 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x564818736c00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc097240 tx=0x7fdcdc0973d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f10208b3b64266 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:04.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.678+0000 7fdccb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:04.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdccb7fe700 1 -- 172.21.15.161:0/15916191 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004780 con 0x5648186d02d0 2023-07-14T11:27:04.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdccb7fe700 1 -- 172.21.15.161:0/15916191 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fdcdc09f0c0 con 0x5648186d02d0 2023-07-14T11:27:04.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcdc0a3450 con 0x5648186d02d0 2023-07-14T11:27:04.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdccb7fe700 1 -- 172.21.15.161:0/15916191 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcdc097090 con 0x5648186d02d0 2023-07-14T11:27:04.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 10 client.26621 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:04.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56481844f2b0 con 0x5648186d02d0 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdccb7fe700 1 -- 172.21.15.161:0/15916191 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcdc0a3450 con 0x5648186d02d0 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdccb7fe700 1 -- 172.21.15.161:0/15916191 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdcdc0cb410 con 0x5648186d02d0 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 10 client.26621 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 20 client.26621 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56481873f520 0x564818741900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 4 client.26621 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:27:04.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.679+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5648185de050 con 0x56481873f520 2023-07-14T11:27:04.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.680+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56481873f520 0x564818741900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:04.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.680+0000 7fdceb7fe700 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56481873f520 0x564818741900 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977d46cde574487b server_cookie=39083abf3271b7a8 in_seq=0 out_seq=0 2023-07-14T11:27:04.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.680+0000 7fdccb7fe700 10 client.26621 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:04.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdccb7fe700 1 -- 172.21.15.161:0/15916191 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5648185de050 con 0x56481873f520 2023-07-14T11:27:04.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdccb7fe700 10 client.26621 handle_command_reply: tid=0 2023-07-14T11:27:04.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 client.26621 shutdown 2023-07-14T11:27:04.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 msgr2=0x564818736c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x564818736c00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc097240 tx=0x7fdcdc0973d0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 shutdown_connections 2023-07-14T11:27:04.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56481873f520 0x564818741900 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5648186dbad0 0x5648187327b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648186d02d0 0x564818736c00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 --2- 172.21.15.161:0/15916191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648186d0be0 0x564818737140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 >> 172.21.15.161:0/15916191 conn(0x5648186c6da0 msgr2=0x5648186d9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdccb7fe700 0 client.26621 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 shutdown_connections 2023-07-14T11:27:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 1 -- 172.21.15.161:0/15916191 wait complete. 2023-07-14T11:27:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.681+0000 7fdd060b2b80 20 client.26621 trim_cache size 0 max 16384 2023-07-14T11:27:04.693 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:27:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdcd40a12c0 msgr2=0x7fdcd40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdcd40a12c0 0x7fdcd40a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdce400f540 tx=0x7fdce4009920 comp rx=0 tx=0).stop 2023-07-14T11:27:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 msgr2=0x7fdcec197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 0x7fdcec197e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc005910 tx=0x7fdcdc004710 comp rx=0 tx=0).stop 2023-07-14T11:27:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 shutdown_connections 2023-07-14T11:27:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fdcd40a12c0 0x7fdcd40a3770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec105b10 0x7fdcec19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fdcec0ffb00 0x7fdcec197e00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 --2- 172.21.15.161:0/3177747206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec1004b0 0x7fdcec198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 >> 172.21.15.161:0/3177747206 conn(0x7fdcec0fb600 msgr2=0x7fdcec10ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 shutdown_connections 2023-07-14T11:27:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:04.682+0000 7fdcf27ff700 1 -- 172.21.15.161:0/3177747206 wait complete. 2023-07-14T11:27:04.996 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:04 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1619438074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:27:05.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:04 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1619438074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:27:05.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 422541 9.78 MB/sec execute 107 sec latency 25.754 ms 2023-07-14T11:27:05.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:04 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1619438074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:27:05.704 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:27:05.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:27:05.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 -- 172.21.15.161:0/171298134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 msgr2=0x7fe37c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 --2- 172.21.15.161:0/171298134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 0x7fe37c0fff50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009990 tx=0x7fe36c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 -- 172.21.15.161:0/171298134 shutdown_connections 2023-07-14T11:27:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 --2- 172.21.15.161:0/171298134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe37c105af0 0x7fe37c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 --2- 172.21.15.161:0/171298134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe37c100490 0x7fe37c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 --2- 172.21.15.161:0/171298134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 0x7fe37c0fff50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 -- 172.21.15.161:0/171298134 >> 172.21.15.161:0/171298134 conn(0x7fe37c0fb5e0 msgr2=0x7fe37c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 -- 172.21.15.161:0/171298134 shutdown_connections 2023-07-14T11:27:05.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.917+0000 7fe3843ff700 1 -- 172.21.15.161:0/171298134 wait complete. 2023-07-14T11:27:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.918+0000 7fe3843ff700 1 Processor -- start 2023-07-14T11:27:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe3843ff700 1 -- start start 2023-07-14T11:27:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe3843ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 0x7fe37c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe3843ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe37c100490 0x7fe37c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe3843ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe37c105af0 0x7fe37c1a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe3843ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c073580 con 0x7fe37c105af0 2023-07-14T11:27:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe3843ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1a4310 con 0x7fe37c0ffae0 2023-07-14T11:27:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe3843ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1a4450 con 0x7fe37c100490 2023-07-14T11:27:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe38219b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 0x7fe37c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe38299c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe37c105af0 0x7fe37c1a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe38199a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe37c100490 0x7fe37c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe38219b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 0x7fe37c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37596/0 (socket says 172.21.15.161:37596) 2023-07-14T11:27:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.919+0000 7fe38219b700 1 -- 172.21.15.161:0/2574502279 learned_addr learned my addr 172.21.15.161:0/2574502279 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe38199a700 1 -- 172.21.15.161:0/2574502279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 msgr2=0x7fe37c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe38199a700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 0x7fe37c072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe38199a700 1 -- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe37c105af0 msgr2=0x7fe37c1a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe38199a700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe37c105af0 0x7fe37c1a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe38199a700 1 -- 172.21.15.161:0/2574502279 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0095f0 con 0x7fe37c100490 2023-07-14T11:27:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe38199a700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe37c100490 0x7fe37c072d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000c00 tx=0x7fe36c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe3737fe700 1 -- 172.21.15.161:0/2574502279 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c01d070 con 0x7fe37c100490 2023-07-14T11:27:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c1a4650 con 0x7fe37c100490 2023-07-14T11:27:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe3737fe700 1 -- 172.21.15.161:0/2574502279 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36c021410 con 0x7fe37c100490 2023-07-14T11:27:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe3737fe700 1 -- 172.21.15.161:0/2574502279 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c017650 con 0x7fe37c100490 2023-07-14T11:27:05.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.920+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c1a4ad0 con 0x7fe37c100490 2023-07-14T11:27:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.925+0000 7fe3737fe700 1 -- 172.21.15.161:0/2574502279 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe36c0217f0 con 0x7fe37c100490 2023-07-14T11:27:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 Processor -- start 2023-07-14T11:27:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 -- start start 2023-07-14T11:27:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453a350 0x558eb453a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb4545610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4547f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb4440d30 con 0x558eb453a350 2023-07-14T11:27:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb4493ac0 con 0x558eb453ac60 2023-07-14T11:27:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.926+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb448cf50 con 0x558eb4545b50 2023-07-14T11:27:05.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb4545610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38299c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453a350 0x558eb453a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe3737fe700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe3680a12c0 0x7fe3680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb4545610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37598/0 (socket says 172.21.15.161:37598) 2023-07-14T11:27:05.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 -- 172.21.15.161:0/842374105 learned_addr learned my addr 172.21.15.161:0/842374105 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:05.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38199a700 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4547f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe3737fe700 1 -- 172.21.15.161:0/2574502279 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe36c0ce370 con 0x7fe37c100490 2023-07-14T11:27:05.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 -- 172.21.15.161:0/842374105 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 msgr2=0x558eb4547f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4547f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38299c700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe3680a12c0 0x7fe3680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 -- 172.21.15.161:0/842374105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453a350 msgr2=0x558eb453a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453a350 0x558eb453a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 -- 172.21.15.161:0/842374105 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eb42bae60 con 0x558eb453ac60 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38199a700 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4547f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.927+0000 7fe38219b700 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb4545610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe378009dd0 tx=0x7fe378011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6979ada2c1080ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe370bfe700 1 -- 172.21.15.161:0/842374105 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004a30 con 0x558eb453ac60 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe370bfe700 1 -- 172.21.15.161:0/842374105 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3780054e0 con 0x558eb453ac60 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe370bfe700 1 -- 172.21.15.161:0/842374105 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378019890 con 0x558eb453ac60 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe38299c700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe3680a12c0 0x7fe3680a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe374009ec0 tx=0x7fe374004b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 -- 172.21.15.161:0/842374105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 msgr2=0x558eb4545610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb4545610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe378009dd0 tx=0x7fe378011720 comp rx=0 tx=0).stop 2023-07-14T11:27:05.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 -- 172.21.15.161:0/842374105 shutdown_connections 2023-07-14T11:27:05.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453a350 0x558eb453a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4547f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/842374105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb4545610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 -- 172.21.15.161:0/842374105 >> 172.21.15.161:0/842374105 conn(0x558eb4530e20 msgr2=0x558eb4536720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:05.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.928+0000 7fe397c6fb80 1 -- 172.21.15.161:0/842374105 shutdown_connections 2023-07-14T11:27:05.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.929+0000 7fe397c6fb80 1 -- 172.21.15.161:0/842374105 wait complete. 2023-07-14T11:27:05.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.929+0000 7fe397c6fb80 1 -- start start 2023-07-14T11:27:05.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.929+0000 7fe397c6fb80 1 Processor -- start 2023-07-14T11:27:05.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453a350 0x558eb4545570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb45427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4542e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb442cb40 con 0x558eb453ac60 2023-07-14T11:27:05.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb442c840 con 0x558eb453a350 2023-07-14T11:27:05.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb43109a0 con 0x558eb4545b50 2023-07-14T11:27:05.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe397c6fb80 10 client.0 _ll_register_callbacks cb 0x558eb442a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:05.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453a350 0x558eb4545570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38299c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4542e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453a350 0x558eb4545570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37614/0 (socket says 172.21.15.161:37614) 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 -- 172.21.15.161:0/707933537 learned_addr learned my addr 172.21.15.161:0/707933537 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38199a700 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb45427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 -- 172.21.15.161:0/707933537 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 msgr2=0x558eb4542e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4542e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 -- 172.21.15.161:0/707933537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 msgr2=0x558eb45427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb45427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 -- 172.21.15.161:0/707933537 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe378009a30 con 0x558eb453a350 2023-07-14T11:27:05.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe38219b700 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453a350 0x558eb4545570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe37801b040 tx=0x7fe378004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e46edca0c4e829 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:05.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.930+0000 7fe365ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:05.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe365ffb700 1 -- 172.21.15.161:0/707933537 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004040 con 0x558eb453a350 2023-07-14T11:27:05.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558eb43e6570 con 0x558eb453a350 2023-07-14T11:27:05.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe365ffb700 1 -- 172.21.15.161:0/707933537 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe3780041a0 con 0x558eb453a350 2023-07-14T11:27:05.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe365ffb700 1 -- 172.21.15.161:0/707933537 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe378004360 con 0x558eb453a350 2023-07-14T11:27:05.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe397c6fb80 10 client.26633 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:05.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558eb42a3ce0 con 0x558eb453a350 2023-07-14T11:27:05.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe365ffb700 1 -- 172.21.15.161:0/707933537 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558eb43e6570 con 0x558eb453a350 2023-07-14T11:27:05.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe365ffb700 1 -- 172.21.15.161:0/707933537 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe378028020 con 0x558eb453a350 2023-07-14T11:27:05.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe397c6fb80 10 client.26633 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:05.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe397c6fb80 10 client.26633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:05.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.931+0000 7fe397c6fb80 20 client.26633 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:05.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.932+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45552f0 0x558eb4557700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.932+0000 7fe397c6fb80 4 client.26633 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:05.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.932+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558eb4485f30 con 0x558eb45552f0 2023-07-14T11:27:05.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.932+0000 7fe38199a700 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45552f0 0x558eb4557700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.932+0000 7fe365ffb700 10 client.26633 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:05.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.932+0000 7fe38199a700 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45552f0 0x558eb4557700 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479b6754b8cd6bf9 server_cookie=1af9b62394636fbb in_seq=0 out_seq=0 2023-07-14T11:27:05.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe365ffb700 1 -- 172.21.15.161:0/707933537 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558eb4485f30 con 0x558eb45552f0 2023-07-14T11:27:05.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe365ffb700 10 client.26633 handle_command_reply: tid=0 2023-07-14T11:27:05.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 client.26633 shutdown 2023-07-14T11:27:05.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453a350 msgr2=0x558eb4545570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453a350 0x558eb4545570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe37801b040 tx=0x7fe378004e20 comp rx=0 tx=0).stop 2023-07-14T11:27:05.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 shutdown_connections 2023-07-14T11:27:05.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45552f0 0x558eb4557700 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb4545b50 0x558eb4542e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453a350 0x558eb4545570 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/707933537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb45427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe365ffb700 0 client.26633 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:05.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 >> 172.21.15.161:0/707933537 conn(0x558eb4530e20 msgr2=0x558eb45362b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:05.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 shutdown_connections 2023-07-14T11:27:05.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.934+0000 7fe397c6fb80 1 -- 172.21.15.161:0/707933537 wait complete. 2023-07-14T11:27:05.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.935+0000 7fe397c6fb80 20 client.26633 trim_cache size 0 max 16384 2023-07-14T11:27:05.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 Processor -- start 2023-07-14T11:27:05.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 -- start start 2023-07-14T11:27:05.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb454c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb45450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb4545b50 0x558eb4542060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb4535510 con 0x558eb4545b50 2023-07-14T11:27:05.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb454ca60 con 0x558eb453ac60 2023-07-14T11:27:05.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb455c2d0 con 0x558eb453a350 2023-07-14T11:27:05.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe38219b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb454c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe38199a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb45450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe38219b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb454c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58618/0 (socket says 172.21.15.161:58618) 2023-07-14T11:27:05.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe38219b700 1 -- 172.21.15.161:0/3397989706 learned_addr learned my addr 172.21.15.161:0/3397989706 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:05.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.952+0000 7fe38299c700 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb4545b50 0x558eb4542060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe38219b700 1 -- 172.21.15.161:0/3397989706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 msgr2=0x558eb45450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe38219b700 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb45450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe38219b700 1 -- 172.21.15.161:0/3397989706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb4545b50 msgr2=0x558eb4542060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe38219b700 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb4545b50 0x558eb4542060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe38219b700 1 -- 172.21.15.161:0/3397989706 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eb42e12d0 con 0x558eb453a350 2023-07-14T11:27:05.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe38199a700 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb45450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:05.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe38219b700 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb454c520 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe378000c00 tx=0x7fe378011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf03e160d3772de0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:05.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe3713ff700 1 -- 172.21.15.161:0/3397989706 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004e90 con 0x558eb453a350 2023-07-14T11:27:05.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe3713ff700 1 -- 172.21.15.161:0/3397989706 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3780226a0 con 0x558eb453a350 2023-07-14T11:27:05.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.953+0000 7fe3713ff700 1 -- 172.21.15.161:0/3397989706 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780213b0 con 0x558eb453a350 2023-07-14T11:27:05.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 -- 172.21.15.161:0/3397989706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 msgr2=0x558eb454c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb454c520 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe378000c00 tx=0x7fe378011720 comp rx=0 tx=0).stop 2023-07-14T11:27:05.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 -- 172.21.15.161:0/3397989706 shutdown_connections 2023-07-14T11:27:05.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb4545b50 0x558eb4542060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb454c520 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/3397989706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb453ac60 0x558eb45450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 -- 172.21.15.161:0/3397989706 >> 172.21.15.161:0/3397989706 conn(0x558eb4530e20 msgr2=0x558eb453db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 -- 172.21.15.161:0/3397989706 shutdown_connections 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 -- 172.21.15.161:0/3397989706 wait complete. 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.954+0000 7fe397c6fb80 1 -- start start 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 1 Processor -- start 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb453f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb453f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb4545b50 0x558eb453fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb454ca40 con 0x558eb453ac60 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb45a91d0 con 0x558eb4545b50 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb45a9340 con 0x558eb453a350 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe397c6fb80 10 client.0 _ll_register_callbacks cb 0x7fe378009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe38219b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb453f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe38219b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb453f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58626/0 (socket says 172.21.15.161:58626) 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe38219b700 1 -- 172.21.15.161:0/2327666756 learned_addr learned my addr 172.21.15.161:0/2327666756 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe38299c700 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb4545b50 0x558eb453fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.955+0000 7fe38199a700 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb453f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe38219b700 1 -- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb4545b50 msgr2=0x558eb453fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe38219b700 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb4545b50 0x558eb453fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe38219b700 1 -- 172.21.15.161:0/2327666756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 msgr2=0x558eb453f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe38219b700 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb453f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe38219b700 1 -- 172.21.15.161:0/2327666756 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c1a4ad0 con 0x558eb453a350 2023-07-14T11:27:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe38219b700 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb453f0d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe378011ba0 tx=0x7fe378009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b0e300334c4f94 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe365ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe365ffb700 1 -- 172.21.15.161:0/2327666756 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004e60 con 0x558eb453a350 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3780054e0 con 0x558eb453a350 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe365ffb700 1 -- 172.21.15.161:0/2327666756 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fe378000e40 con 0x558eb453a350 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe365ffb700 1 -- 172.21.15.161:0/2327666756 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe378004000 con 0x558eb453a350 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe397c6fb80 10 client.26641 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558eb42a3ce0 con 0x558eb453a350 2023-07-14T11:27:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.956+0000 7fe365ffb700 1 -- 172.21.15.161:0/2327666756 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3780054e0 con 0x558eb453a350 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe365ffb700 1 -- 172.21.15.161:0/2327666756 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe378011840 con 0x558eb453a350 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe397c6fb80 10 client.26641 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe397c6fb80 10 client.26641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe397c6fb80 20 client.26641 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45aa550 0x558eb45ac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe397c6fb80 4 client.26641 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558eb44480d0 con 0x558eb45aa550 2023-07-14T11:27:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe38199a700 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45aa550 0x558eb45ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:05.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe38199a700 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45aa550 0x558eb45ac960 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91d38d6f68b3e38 server_cookie=1c8568fb03492876 in_seq=0 out_seq=0 2023-07-14T11:27:05.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.957+0000 7fe365ffb700 10 client.26641 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:05.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.958+0000 7fe365ffb700 1 -- 172.21.15.161:0/2327666756 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558eb44480d0 con 0x558eb45aa550 2023-07-14T11:27:05.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.958+0000 7fe365ffb700 10 client.26641 handle_command_reply: tid=0 2023-07-14T11:27:05.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.958+0000 7fe397c6fb80 1 client.26641 shutdown 2023-07-14T11:27:05.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 msgr2=0x558eb453f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb453f0d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe378011ba0 tx=0x7fe378009800 comp rx=0 tx=0).stop 2023-07-14T11:27:05.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 shutdown_connections 2023-07-14T11:27:05.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558eb45aa550 0x558eb45ac960 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558eb4545b50 0x558eb453fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558eb453a350 0x558eb453f0d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 --2- 172.21.15.161:0/2327666756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558eb453ac60 0x558eb453f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe365ffb700 0 client.26641 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 >> 172.21.15.161:0/2327666756 conn(0x558eb4530e20 msgr2=0x558eb453c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 shutdown_connections 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 1 -- 172.21.15.161:0/2327666756 wait complete. 2023-07-14T11:27:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.959+0000 7fe397c6fb80 20 client.26641 trim_cache size 0 max 16384 2023-07-14T11:27:05.973 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:27:05.973 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:27:05.973 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "79d7cfd5-9d78-46fc-86ad-90c3b6d875eb", 2023-07-14T11:27:05.973 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:27:05.973 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:27:05.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe3680a12c0 msgr2=0x7fe3680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe3680a12c0 0x7fe3680a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe374009ec0 tx=0x7fe374004b40 comp rx=0 tx=0).stop 2023-07-14T11:27:05.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe37c100490 msgr2=0x7fe37c072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe37c100490 0x7fe37c072d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000c00 tx=0x7fe36c005740 comp rx=0 tx=0).stop 2023-07-14T11:27:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 shutdown_connections 2023-07-14T11:27:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe3680a12c0 0x7fe3680a3770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe37c105af0 0x7fe37c1a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe37c0ffae0 0x7fe37c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 --2- 172.21.15.161:0/2574502279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe37c100490 0x7fe37c072d50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 >> 172.21.15.161:0/2574502279 conn(0x7fe37c0fb5e0 msgr2=0x7fe37c0fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:05.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 shutdown_connections 2023-07-14T11:27:05.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:05.960+0000 7fe3843ff700 1 -- 172.21.15.161:0/2574502279 wait complete. 2023-07-14T11:27:05.983 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:27:06.046 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:05 smithi161 ceph-mon[95526]: pgmap v148: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2023-07-14T11:27:06.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:05 smithi175 ceph-mon[129081]: pgmap v148: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2023-07-14T11:27:06.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 424224 9.74 MB/sec execute 108 sec latency 15.291 ms 2023-07-14T11:27:06.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 -- 172.21.15.161:0/1151973366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 msgr2=0x7f160c109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 --2- 172.21.15.161:0/1151973366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 0x7f160c109aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16000098f0 tx=0x7f1600009c00 comp rx=0 tx=0).stop 2023-07-14T11:27:06.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 -- 172.21.15.161:0/1151973366 shutdown_connections 2023-07-14T11:27:06.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 --2- 172.21.15.161:0/1151973366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f160c100ca0 0x7f160c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 --2- 172.21.15.161:0/1151973366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c10a070 0x7f160c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 --2- 172.21.15.161:0/1151973366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 0x7f160c109aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 -- 172.21.15.161:0/1151973366 >> 172.21.15.161:0/1151973366 conn(0x7f160c0fa510 msgr2=0x7f160c0fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:06.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 -- 172.21.15.161:0/1151973366 shutdown_connections 2023-07-14T11:27:06.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.197+0000 7f16119ff700 1 -- 172.21.15.161:0/1151973366 wait complete. 2023-07-14T11:27:06.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.198+0000 7f16119ff700 1 Processor -- start 2023-07-14T11:27:06.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f16119ff700 1 -- start start 2023-07-14T11:27:06.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f16119ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c100ca0 0x7f160c0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f16119ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 0x7f160c0fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f16119ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f160c10a070 0x7f160c1a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f16119ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0ff660 con 0x7f160c10a070 2023-07-14T11:27:06.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f16119ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c1a43a0 con 0x7f160c1015b0 2023-07-14T11:27:06.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f16119ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c1a44e0 con 0x7f160c100ca0 2023-07-14T11:27:06.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f160affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c100ca0 0x7f160c0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f160affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c100ca0 0x7f160c0fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58634/0 (socket says 172.21.15.161:58634) 2023-07-14T11:27:06.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f160affd700 1 -- 172.21.15.161:0/3432753313 learned_addr learned my addr 172.21.15.161:0/3432753313 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:06.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.199+0000 7f160a7fc700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 0x7f160c0fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160b7fe700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f160c10a070 0x7f160c1a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160affd700 1 -- 172.21.15.161:0/3432753313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 msgr2=0x7f160c0fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160affd700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 0x7f160c0fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160affd700 1 -- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f160c10a070 msgr2=0x7f160c1a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160affd700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f160c10a070 0x7f160c1a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160affd700 1 -- 172.21.15.161:0/3432753313 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600009520 con 0x7f160c100ca0 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160b7fe700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f160c10a070 0x7f160c1a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f160affd700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c100ca0 0x7f160c0fe8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00e9a0 tx=0x7f15fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:06.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f16109fd700 1 -- 172.21.15.161:0/3432753313 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc009790 con 0x7f160c100ca0 2023-07-14T11:27:06.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f16109fd700 1 -- 172.21.15.161:0/3432753313 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc00cbd0 con 0x7f160c100ca0 2023-07-14T11:27:06.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f16109fd700 1 -- 172.21.15.161:0/3432753313 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc018680 con 0x7f160c100ca0 2023-07-14T11:27:06.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.200+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c1a4740 con 0x7f160c100ca0 2023-07-14T11:27:06.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.201+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c1a4bd0 con 0x7f160c100ca0 2023-07-14T11:27:06.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.204+0000 7f16109fd700 1 -- 172.21.15.161:0/3432753313 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f15fc00cd40 con 0x7f160c100ca0 2023-07-14T11:27:06.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.205+0000 7f16253cdb80 1 Processor -- start 2023-07-14T11:27:06.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.205+0000 7f16253cdb80 1 -- start start 2023-07-14T11:27:06.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.205+0000 7f16109fd700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f15f40a1300 0x7f15f40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f16109fd700 1 -- 172.21.15.161:0/3432753313 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f15fc014070 con 0x7f160c100ca0 2023-07-14T11:27:06.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c57b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160b7fe700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f15f40a1300 0x7f15f40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c586590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c588eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f16253cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c481cb0 con 0x56294c586ad0 2023-07-14T11:27:06.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f16253cdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c4d4a40 con 0x56294c57bbe0 2023-07-14T11:27:06.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f16253cdb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c4cded0 con 0x56294c57b2d0 2023-07-14T11:27:06.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c57b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c586590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c57b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58638/0 (socket says 172.21.15.161:58638) 2023-07-14T11:27:06.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 -- 172.21.15.161:0/2953630506 learned_addr learned my addr 172.21.15.161:0/2953630506 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:06.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160b7fe700 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c588eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 -- 172.21.15.161:0/2953630506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 msgr2=0x56294c586590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c586590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 -- 172.21.15.161:0/2953630506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 msgr2=0x56294c588eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c588eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160affd700 1 -- 172.21.15.161:0/2953630506 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294c2fbde0 con 0x56294c57b2d0 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.206+0000 7f160a7fc700 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c586590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.207+0000 7f160affd700 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c57b6a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15fc096ab0 tx=0x7f15fc099a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dfd71ba9c7653f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.207+0000 7f160b7fe700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f15f40a1300 0x7f15f40a37b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1604009fd0 tx=0x7f1604007b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.207+0000 7f15f1ffb700 1 -- 172.21.15.161:0/2953630506 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc09e070 con 0x56294c57b2d0 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.207+0000 7f15f1ffb700 1 -- 172.21.15.161:0/2953630506 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc0a2490 con 0x56294c57b2d0 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.207+0000 7f15f1ffb700 1 -- 172.21.15.161:0/2953630506 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0a2000 con 0x56294c57b2d0 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 -- 172.21.15.161:0/2953630506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 msgr2=0x56294c57b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c57b6a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15fc096ab0 tx=0x7f15fc099a20 comp rx=0 tx=0).stop 2023-07-14T11:27:06.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 -- 172.21.15.161:0/2953630506 shutdown_connections 2023-07-14T11:27:06.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c588eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c57b6a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2953630506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c586590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 -- 172.21.15.161:0/2953630506 >> 172.21.15.161:0/2953630506 conn(0x56294c571da0 msgr2=0x56294c5776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 -- 172.21.15.161:0/2953630506 shutdown_connections 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 -- 172.21.15.161:0/2953630506 wait complete. 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.208+0000 7f16253cdb80 1 -- start start 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 1 Processor -- start 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c583870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c583db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c5842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c46dac0 con 0x56294c586ad0 2023-07-14T11:27:06.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c46d7c0 con 0x56294c57bbe0 2023-07-14T11:27:06.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c351920 con 0x56294c57b2d0 2023-07-14T11:27:06.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f16253cdb80 10 client.0 _ll_register_callbacks cb 0x56294c46b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:06.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.209+0000 7f160affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c583870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c583db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c583870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:58640/0 (socket says 172.21.15.161:58640) 2023-07-14T11:27:06.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 -- 172.21.15.161:0/2716257896 learned_addr learned my addr 172.21.15.161:0/2716257896 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:06.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160b7fe700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c5842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 -- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 msgr2=0x56294c583db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c583db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 -- 172.21.15.161:0/2716257896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 msgr2=0x56294c5842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c5842f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 -- 172.21.15.161:0/2716257896 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c10d170 con 0x56294c57b2d0 2023-07-14T11:27:06.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160a7fc700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c583db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:06.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160b7fe700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c5842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:06.220 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:05 smithi182 ceph-mon[129366]: pgmap v148: 129 pgs: 129 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2023-07-14T11:27:06.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f160affd700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c583870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0cac40 tx=0x7f15fc0cd230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b7a9352aae0878 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:06.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f15eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:06.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f15eaffd700 1 -- 172.21.15.161:0/2716257896 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc09e070 con 0x56294c57b2d0 2023-07-14T11:27:06.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f15eaffd700 1 -- 172.21.15.161:0/2716257896 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f15fc0a2210 con 0x56294c57b2d0 2023-07-14T11:27:06.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.210+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56294c4274f0 con 0x56294c57b2d0 2023-07-14T11:27:06.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f15eaffd700 1 -- 172.21.15.161:0/2716257896 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15fc0b8250 con 0x56294c57b2d0 2023-07-14T11:27:06.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 10 client.26653 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:06.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f160c1a4bd0 con 0x56294c57b2d0 2023-07-14T11:27:06.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f15eaffd700 1 -- 172.21.15.161:0/2716257896 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56294c4274f0 con 0x56294c57b2d0 2023-07-14T11:27:06.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f15eaffd700 1 -- 172.21.15.161:0/2716257896 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15fc099030 con 0x56294c57b2d0 2023-07-14T11:27:06.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 10 client.26653 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:06.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 10 client.26653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:06.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 20 client.26653 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:06.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c596090 0x56294c5984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 4 client.26653 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:06.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.211+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56294c4c6eb0 con 0x56294c596090 2023-07-14T11:27:06.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.212+0000 7f160a7fc700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c596090 0x56294c5984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.212+0000 7f15eaffd700 10 client.26653 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:06.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.212+0000 7f160a7fc700 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c596090 0x56294c5984a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45c8a56fd8e4ef0 server_cookie=40f47329dd98cfb8 in_seq=0 out_seq=0 2023-07-14T11:27:06.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.212+0000 7f15eaffd700 1 -- 172.21.15.161:0/2716257896 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56294c4c6eb0 con 0x56294c596090 2023-07-14T11:27:06.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f15eaffd700 10 client.26653 handle_command_reply: tid=0 2023-07-14T11:27:06.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 client.26653 shutdown 2023-07-14T11:27:06.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 msgr2=0x56294c583870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c583870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0cac40 tx=0x7f15fc0cd230 comp rx=0 tx=0).stop 2023-07-14T11:27:06.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 shutdown_connections 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c596090 0x56294c5984a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c586ad0 0x56294c5842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f15eaffd700 0 client.26653 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57b2d0 0x56294c583870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 --2- 172.21.15.161:0/2716257896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57bbe0 0x56294c583db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 >> 172.21.15.161:0/2716257896 conn(0x56294c571da0 msgr2=0x56294c577230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 shutdown_connections 2023-07-14T11:27:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 1 -- 172.21.15.161:0/2716257896 wait complete. 2023-07-14T11:27:06.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.213+0000 7f16253cdb80 20 client.26653 trim_cache size 0 max 16384 2023-07-14T11:27:06.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.231+0000 7f16253cdb80 1 Processor -- start 2023-07-14T11:27:06.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f16253cdb80 1 -- start start 2023-07-14T11:27:06.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57b2d0 0x56294c58d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57bbe0 0x56294c584120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c586ad0 0x56294c580ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f16253cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c57c380 con 0x56294c57b2d0 2023-07-14T11:27:06.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f16253cdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c578310 con 0x56294c586ad0 2023-07-14T11:27:06.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f16253cdb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c57e6a0 con 0x56294c57bbe0 2023-07-14T11:27:06.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f160affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57b2d0 0x56294c58d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f160affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57b2d0 0x56294c58d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34764/0 (socket says 172.21.15.161:34764) 2023-07-14T11:27:06.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f160affd700 1 -- 172.21.15.161:0/1394708533 learned_addr learned my addr 172.21.15.161:0/1394708533 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:06.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.232+0000 7f160a7fc700 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57bbe0 0x56294c584120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160b7fe700 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c586ad0 0x56294c580ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160affd700 1 -- 172.21.15.161:0/1394708533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57bbe0 msgr2=0x56294c584120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160affd700 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57bbe0 0x56294c584120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160affd700 1 -- 172.21.15.161:0/1394708533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c586ad0 msgr2=0x56294c580ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160affd700 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c586ad0 0x56294c580ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160affd700 1 -- 172.21.15.161:0/1394708533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294c2e4c60 con 0x56294c57b2d0 2023-07-14T11:27:06.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160a7fc700 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57bbe0 0x56294c584120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:06.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f160affd700 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57b2d0 0x56294c58d4e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f15fc096ab0 tx=0x7f15fc09a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777edfd77739bd9c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:06.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f15f27fc700 1 -- 172.21.15.161:0/1394708533 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc09aec0 con 0x56294c57b2d0 2023-07-14T11:27:06.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f15f27fc700 1 -- 172.21.15.161:0/1394708533 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc09e070 con 0x56294c57b2d0 2023-07-14T11:27:06.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.233+0000 7f15f27fc700 1 -- 172.21.15.161:0/1394708533 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0b8490 con 0x56294c57b2d0 2023-07-14T11:27:06.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 -- 172.21.15.161:0/1394708533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57b2d0 msgr2=0x56294c58d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57b2d0 0x56294c58d4e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f15fc096ab0 tx=0x7f15fc09a040 comp rx=0 tx=0).stop 2023-07-14T11:27:06.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 -- 172.21.15.161:0/1394708533 shutdown_connections 2023-07-14T11:27:06.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57b2d0 0x56294c58d4e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c586ad0 0x56294c580ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 --2- 172.21.15.161:0/1394708533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c57bbe0 0x56294c584120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 -- 172.21.15.161:0/1394708533 >> 172.21.15.161:0/1394708533 conn(0x56294c571da0 msgr2=0x56294c57eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:06.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 -- 172.21.15.161:0/1394708533 shutdown_connections 2023-07-14T11:27:06.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 -- 172.21.15.161:0/1394708533 wait complete. 2023-07-14T11:27:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.234+0000 7f16253cdb80 1 -- start start 2023-07-14T11:27:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 1 Processor -- start 2023-07-14T11:27:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57b2d0 0x56294c58b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57bbe0 0x56294c58b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c586ad0 0x56294c5e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c5e9490 con 0x56294c57bbe0 2023-07-14T11:27:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c5e95d0 con 0x56294c57b2d0 2023-07-14T11:27:06.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294c5e9710 con 0x56294c586ad0 2023-07-14T11:27:06.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f16253cdb80 10 client.0 _ll_register_callbacks cb 0x7f15fc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:06.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f160affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57b2d0 0x56294c58b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f160affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57b2d0 0x56294c58b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37672/0 (socket says 172.21.15.161:37672) 2023-07-14T11:27:06.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f160affd700 1 -- 172.21.15.161:0/4057829088 learned_addr learned my addr 172.21.15.161:0/4057829088 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:06.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.235+0000 7f160b7fe700 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c586ad0 0x56294c5e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f160a7fc700 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57bbe0 0x56294c58b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f160affd700 1 -- 172.21.15.161:0/4057829088 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c586ad0 msgr2=0x56294c5e8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f160affd700 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c586ad0 0x56294c5e8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f160affd700 1 -- 172.21.15.161:0/4057829088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57bbe0 msgr2=0x56294c58b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f160affd700 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57bbe0 0x56294c58b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f160affd700 1 -- 172.21.15.161:0/4057829088 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160000f8d0 con 0x56294c57b2d0 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f160affd700 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57b2d0 0x56294c58b210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15fc098390 tx=0x7f15fc09a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9704ad748c1f27a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f15eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f15eaffd700 1 -- 172.21.15.161:0/4057829088 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc09b280 con 0x56294c57b2d0 2023-07-14T11:27:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f15eaffd700 1 -- 172.21.15.161:0/4057829088 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f15fc098fa0 con 0x56294c57b2d0 2023-07-14T11:27:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15fc0a2490 con 0x56294c57b2d0 2023-07-14T11:27:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f15eaffd700 1 -- 172.21.15.161:0/4057829088 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15fc09aec0 con 0x56294c57b2d0 2023-07-14T11:27:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f16253cdb80 10 client.26657 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.236+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f160c1a4bd0 con 0x56294c57b2d0 2023-07-14T11:27:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f15eaffd700 1 -- 172.21.15.161:0/4057829088 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc0a2490 con 0x56294c57b2d0 2023-07-14T11:27:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f15eaffd700 1 -- 172.21.15.161:0/4057829088 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15fc0b2ba0 con 0x56294c57b2d0 2023-07-14T11:27:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f16253cdb80 10 client.26657 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f16253cdb80 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f16253cdb80 20 client.26657 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f16253cdb80 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c5ea990 0x56294c5ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:06.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f16253cdb80 4 client.26657 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:27:06.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56294c489050 con 0x56294c5ea990 2023-07-14T11:27:06.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.237+0000 7f160a7fc700 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c5ea990 0x56294c5ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:06.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.238+0000 7f15eaffd700 10 client.26657 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:06.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.238+0000 7f160a7fc700 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c5ea990 0x56294c5ecda0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06b208f70a3ec32 server_cookie=61c3e570810a908c in_seq=0 out_seq=0 2023-07-14T11:27:06.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f15eaffd700 1 -- 172.21.15.161:0/4057829088 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56294c489050 con 0x56294c5ea990 2023-07-14T11:27:06.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f15eaffd700 10 client.26657 handle_command_reply: tid=0 2023-07-14T11:27:06.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 client.26657 shutdown 2023-07-14T11:27:06.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57b2d0 msgr2=0x56294c58b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57b2d0 0x56294c58b210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15fc098390 tx=0x7f15fc09a0b0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 shutdown_connections 2023-07-14T11:27:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56294c5ea990 0x56294c5ecda0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56294c586ad0 0x56294c5e8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56294c57b2d0 0x56294c58b210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 --2- 172.21.15.161:0/4057829088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56294c57bbe0 0x56294c58b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f15eaffd700 0 client.26657 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 >> 172.21.15.161:0/4057829088 conn(0x56294c571da0 msgr2=0x56294c57d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 shutdown_connections 2023-07-14T11:27:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 1 -- 172.21.15.161:0/4057829088 wait complete. 2023-07-14T11:27:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.239+0000 7f16253cdb80 20 client.26657 trim_cache size 0 max 16384 2023-07-14T11:27:06.252 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:27:06.252 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:27:06.253 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:27:06.253 INFO:teuthology.orchestra.run.smithi161.stdout: "79d7cfd5-9d78-46fc-86ad-90c3b6d875eb": { 2023-07-14T11:27:06.253 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:27:06.253 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "79d7cfd5-9d78-46fc-86ad-90c3b6d875eb", 2023-07-14T11:27:06.253 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:27:06.253 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:27:06.253 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:27:06.254 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:27:06.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f15f40a1300 msgr2=0x7f15f40a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f15f40a1300 0x7f15f40a37b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1604009fd0 tx=0x7f1604007b40 comp rx=0 tx=0).stop 2023-07-14T11:27:06.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c100ca0 msgr2=0x7f160c0fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:06.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c100ca0 0x7f160c0fe8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00e9a0 tx=0x7f15fc00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 shutdown_connections 2023-07-14T11:27:06.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f15f40a1300 0x7f15f40a37b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f160c10a070 0x7f160c1a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f160c100ca0 0x7f160c0fe8f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 --2- 172.21.15.161:0/3432753313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160c1015b0 0x7f160c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:06.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 >> 172.21.15.161:0/3432753313 conn(0x7f160c0fa510 msgr2=0x7f160c10c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 shutdown_connections 2023-07-14T11:27:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:06.240+0000 7f16119ff700 1 -- 172.21.15.161:0/3432753313 wait complete. 2023-07-14T11:27:06.263 INFO:tasks.cephfs.filesystem:scrub status for tag:79d7cfd5-9d78-46fc-86ad-90c3b6d875eb - {'path': '/', 'tag': '79d7cfd5-9d78-46fc-86ad-90c3b6d875eb', 'options': 'recursive,force'} 2023-07-14T11:27:07.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:27:07.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:27:07.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:27:07.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:27:07.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:06 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.178 INFO:tasks.workunit.client.0.smithi161.stdout: 1 426112 9.78 MB/sec execute 109 sec latency 17.884 ms 2023-07-14T11:27:07.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:27:07.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:27:07.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:27:07.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:27:07.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:06 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:27:07.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:27:07.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:27:07.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:27:07.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:07.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:06 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:08.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:07 smithi175 ceph-mon[129081]: pgmap v149: 129 pgs: 129 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2023-07-14T11:27:08.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 428082 9.75 MB/sec execute 110 sec latency 13.343 ms 2023-07-14T11:27:08.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:07 smithi182 ceph-mon[129366]: pgmap v149: 129 pgs: 129 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2023-07-14T11:27:08.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:07 smithi161 ceph-mon[95526]: pgmap v149: 129 pgs: 129 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2023-07-14T11:27:09.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 429681 9.77 MB/sec execute 111 sec latency 67.265 ms 2023-07-14T11:27:10.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 431707 9.76 MB/sec execute 112 sec latency 25.513 ms 2023-07-14T11:27:10.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:10 smithi182 ceph-mon[129366]: pgmap v150: 129 pgs: 129 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 231 op/s 2023-07-14T11:27:10.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:10 smithi161 ceph-mon[95526]: pgmap v150: 129 pgs: 129 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 231 op/s 2023-07-14T11:27:10.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:10 smithi175 ceph-mon[129081]: pgmap v150: 129 pgs: 129 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 231 op/s 2023-07-14T11:27:11.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 433487 9.77 MB/sec execute 113 sec latency 14.291 ms 2023-07-14T11:27:12.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 435705 9.79 MB/sec execute 114 sec latency 10.049 ms 2023-07-14T11:27:12.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:12 smithi182 ceph-mon[129366]: pgmap v151: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-07-14T11:27:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:12 smithi161 ceph-mon[95526]: pgmap v151: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-07-14T11:27:12.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:12 smithi175 ceph-mon[129081]: pgmap v151: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-07-14T11:27:13.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 437097 9.77 MB/sec execute 115 sec latency 24.688 ms 2023-07-14T11:27:14.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:13 smithi175 ceph-mon[129081]: pgmap v152: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-07-14T11:27:14.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 439242 9.79 MB/sec execute 116 sec latency 40.387 ms 2023-07-14T11:27:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:13 smithi182 ceph-mon[129366]: pgmap v152: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-07-14T11:27:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:13 smithi161 ceph-mon[95526]: pgmap v152: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-07-14T11:27:15.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 440577 9.76 MB/sec execute 117 sec latency 34.112 ms 2023-07-14T11:27:16.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 443212 9.80 MB/sec execute 118 sec latency 35.541 ms 2023-07-14T11:27:16.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:16 smithi175 ceph-mon[129081]: pgmap v153: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-07-14T11:27:16.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:16 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:27:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:16 smithi182 ceph-mon[129366]: pgmap v153: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-07-14T11:27:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:16 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:27:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:16 smithi161 ceph-mon[95526]: pgmap v153: 129 pgs: 129 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-07-14T11:27:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:16 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:27:17.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 444546 9.78 MB/sec execute 119 sec latency 13.841 ms 2023-07-14T11:27:17.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:17 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:27:17.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:17 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:27:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:17 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:27:17.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:17 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:27:17.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:17 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:27:17.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:17 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:27:18.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 447107 9.83 MB/sec execute 120 sec latency 10.163 ms 2023-07-14T11:27:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:18 smithi182 ceph-mon[129366]: pgmap v154: 129 pgs: 129 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-07-14T11:27:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:18 smithi161 ceph-mon[95526]: pgmap v154: 129 pgs: 129 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-07-14T11:27:18.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:18 smithi175 ceph-mon[129081]: pgmap v154: 129 pgs: 129 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-07-14T11:27:19.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 447984 9.77 MB/sec execute 121 sec latency 58.267 ms 2023-07-14T11:27:20.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 450554 9.82 MB/sec execute 122 sec latency 35.271 ms 2023-07-14T11:27:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:20 smithi182 ceph-mon[129366]: pgmap v155: 129 pgs: 129 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2023-07-14T11:27:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:20 smithi161 ceph-mon[95526]: pgmap v155: 129 pgs: 129 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2023-07-14T11:27:20.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:20 smithi175 ceph-mon[129081]: pgmap v155: 129 pgs: 129 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2023-07-14T11:27:21.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 451321 9.76 MB/sec execute 123 sec latency 118.049 ms 2023-07-14T11:27:22.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 453863 9.79 MB/sec execute 124 sec latency 29.401 ms 2023-07-14T11:27:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:22 smithi182 ceph-mon[129366]: pgmap v156: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2023-07-14T11:27:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:22 smithi161 ceph-mon[95526]: pgmap v156: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2023-07-14T11:27:22.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:22 smithi175 ceph-mon[129081]: pgmap v156: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2023-07-14T11:27:23.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 455106 9.76 MB/sec execute 125 sec latency 11.725 ms 2023-07-14T11:27:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:23 smithi175 ceph-mon[129081]: pgmap v157: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-07-14T11:27:24.179 INFO:tasks.workunit.client.0.smithi161.stdout: 1 457473 9.80 MB/sec execute 126 sec latency 33.714 ms 2023-07-14T11:27:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:23 smithi182 ceph-mon[129366]: pgmap v157: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-07-14T11:27:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:23 smithi161 ceph-mon[95526]: pgmap v157: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-07-14T11:27:25.180 INFO:tasks.workunit.client.0.smithi161.stdout: 1 458322 9.75 MB/sec execute 127 sec latency 68.347 ms 2023-07-14T11:27:26.180 INFO:tasks.workunit.client.0.smithi161.stdout: 1 459479 9.85 MB/sec execute 128 sec latency 50.175 ms 2023-07-14T11:27:26.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:26 smithi182 ceph-mon[129366]: pgmap v158: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-07-14T11:27:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:26 smithi161 ceph-mon[95526]: pgmap v158: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-07-14T11:27:26.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:26 smithi175 ceph-mon[129081]: pgmap v158: 129 pgs: 129 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-07-14T11:27:27.180 INFO:tasks.workunit.client.0.smithi161.stdout: 1 462051 9.90 MB/sec execute 129 sec latency 18.184 ms 2023-07-14T11:27:28.180 INFO:tasks.workunit.client.0.smithi161.stdout: 1 463375 9.87 MB/sec execute 130 sec latency 40.229 ms 2023-07-14T11:27:28.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:28 smithi182 ceph-mon[129366]: pgmap v159: 129 pgs: 129 active+clean; 277 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 316 op/s 2023-07-14T11:27:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:28 smithi161 ceph-mon[95526]: pgmap v159: 129 pgs: 129 active+clean; 277 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 316 op/s 2023-07-14T11:27:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:28 smithi175 ceph-mon[129081]: pgmap v159: 129 pgs: 129 active+clean; 277 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 316 op/s 2023-07-14T11:27:29.181 INFO:tasks.workunit.client.0.smithi161.stdout: 1 465656 9.90 MB/sec execute 131 sec latency 54.588 ms 2023-07-14T11:27:29.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:29 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:27:29.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:29 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:27:29.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:29 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:27:30.181 INFO:tasks.workunit.client.0.smithi161.stdout: 1 466856 9.87 MB/sec execute 132 sec latency 36.835 ms 2023-07-14T11:27:30.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:30 smithi182 ceph-mon[129366]: pgmap v160: 129 pgs: 129 active+clean; 277 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2023-07-14T11:27:30.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:30 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:27:30.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:30 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:27:30.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:30 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:27:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:30 smithi161 ceph-mon[95526]: pgmap v160: 129 pgs: 129 active+clean; 277 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2023-07-14T11:27:30.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:30 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:27:30.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:30 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:27:30.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:30 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:27:30.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:30 smithi175 ceph-mon[129081]: pgmap v160: 129 pgs: 129 active+clean; 277 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2023-07-14T11:27:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:30 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:27:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:30 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:27:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:30 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:27:31.181 INFO:tasks.workunit.client.0.smithi161.stdout: 1 469234 9.89 MB/sec execute 133 sec latency 31.274 ms 2023-07-14T11:27:32.181 INFO:tasks.workunit.client.0.smithi161.stdout: 1 470499 9.86 MB/sec execute 134 sec latency 24.845 ms 2023-07-14T11:27:32.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:32 smithi182 ceph-mon[129366]: pgmap v161: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 316 op/s 2023-07-14T11:27:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:32 smithi161 ceph-mon[95526]: pgmap v161: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 316 op/s 2023-07-14T11:27:32.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:32 smithi175 ceph-mon[129081]: pgmap v161: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 316 op/s 2023-07-14T11:27:33.181 INFO:tasks.workunit.client.0.smithi161.stdout: 1 472835 9.89 MB/sec execute 135 sec latency 14.665 ms 2023-07-14T11:27:34.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:33 smithi175 ceph-mon[129081]: pgmap v162: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2023-07-14T11:27:34.183 INFO:tasks.workunit.client.0.smithi161.stdout: 1 474234 9.86 MB/sec execute 136 sec latency 17.470 ms 2023-07-14T11:27:34.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:33 smithi182 ceph-mon[129366]: pgmap v162: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2023-07-14T11:27:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:33 smithi161 ceph-mon[95526]: pgmap v162: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2023-07-14T11:27:35.183 INFO:tasks.workunit.client.0.smithi161.stdout: 1 476405 9.88 MB/sec execute 137 sec latency 46.822 ms 2023-07-14T11:27:36.183 INFO:tasks.workunit.client.0.smithi161.stdout: 1 477783 9.85 MB/sec execute 138 sec latency 27.623 ms 2023-07-14T11:27:36.264 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:27:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:36 smithi182 ceph-mon[129366]: pgmap v163: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2023-07-14T11:27:36.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 -- 172.21.15.161:0/1009507322 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f010c3a0 msgr2=0x7f46f010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 --2- 172.21.15.161:0/1009507322 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f010c3a0 0x7f46f010e780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009f90 tx=0x7f46e800b330 comp rx=0 tx=0).stop 2023-07-14T11:27:36.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 -- 172.21.15.161:0/1009507322 shutdown_connections 2023-07-14T11:27:36.480 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 --2- 172.21.15.161:0/1009507322 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f010c3a0 0x7f46f010e780 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 --2- 172.21.15.161:0/1009507322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f0072850 0x7f46f0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 --2- 172.21.15.161:0/1009507322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f0073160 0x7f46f010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 -- 172.21.15.161:0/1009507322 >> 172.21.15.161:0/1009507322 conn(0x7f46f00fb600 msgr2=0x7f46f00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:36.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 -- 172.21.15.161:0/1009507322 shutdown_connections 2023-07-14T11:27:36.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.477+0000 7f46f67ff700 1 -- 172.21.15.161:0/1009507322 wait complete. 2023-07-14T11:27:36.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.478+0000 7f46f67ff700 1 Processor -- start 2023-07-14T11:27:36.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.478+0000 7f46f67ff700 1 -- start start 2023-07-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f67ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f0072850 0x7f46f0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f67ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f0073160 0x7f46f0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f67ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f010c3a0 0x7f46f019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f67ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f019cfd0 con 0x7f46f0073160 2023-07-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f67ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f019d140 con 0x7f46f010c3a0 2023-07-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f67ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f019d300 con 0x7f46f0072850 2023-07-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46effff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f0072850 0x7f46f0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f4d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f010c3a0 0x7f46f019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46effff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f0072850 0x7f46f0197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60812/0 (socket says 172.21.15.161:60812) 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46effff700 1 -- 172.21.15.161:0/3783060432 learned_addr learned my addr 172.21.15.161:0/3783060432 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f0073160 0x7f46f0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f4d9c700 1 -- 172.21.15.161:0/3783060432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f0072850 msgr2=0x7f46f0197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f4d9c700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f0072850 0x7f46f0197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f4d9c700 1 -- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f0073160 msgr2=0x7f46f0198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f4d9c700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f0073160 0x7f46f0198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46f4d9c700 1 -- 172.21.15.161:0/3783060432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8009c40 con 0x7f46f010c3a0 2023-07-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.479+0000 7f46effff700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f0072850 0x7f46f0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.480+0000 7f46f4d9c700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f010c3a0 0x7f46f019c990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b6c0 tx=0x7f46e80094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.480+0000 7f46ed7fa700 1 -- 172.21.15.161:0/3783060432 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e800e040 con 0x7f46f010c3a0 2023-07-14T11:27:36.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.480+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f019d580 con 0x7f46f010c3a0 2023-07-14T11:27:36.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.480+0000 7f46ed7fa700 1 -- 172.21.15.161:0/3783060432 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e801d470 con 0x7f46f010c3a0 2023-07-14T11:27:36.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.480+0000 7f46ed7fa700 1 -- 172.21.15.161:0/3783060432 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e80052a0 con 0x7f46f010c3a0 2023-07-14T11:27:36.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.480+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f01a4e20 con 0x7f46f010c3a0 2023-07-14T11:27:36.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.483+0000 7f46ed7fa700 1 -- 172.21.15.161:0/3783060432 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f46e801d5e0 con 0x7f46f010c3a0 2023-07-14T11:27:36.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.485+0000 7f470a190b80 1 Processor -- start 2023-07-14T11:27:36.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.485+0000 7f470a190b80 1 -- start start 2023-07-14T11:27:36.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f470a190b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f470a190b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f470a190b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d63eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f470a190b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8c5ccb0 con 0x55e0f8d562d0 2023-07-14T11:27:36.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f470a190b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8cafa40 con 0x55e0f8d61ad0 2023-07-14T11:27:36.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f470a190b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8ca8ed0 con 0x55e0f8d56be0 2023-07-14T11:27:36.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46ed7fa700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46d80a12c0 0x7f46d80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46effff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46ed7fa700 1 -- 172.21.15.161:0/3783060432 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f46e80ce4b0 con 0x7f46f010c3a0 2023-07-14T11:27:36.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46effff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d61590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60824/0 (socket says 172.21.15.161:60824) 2023-07-14T11:27:36.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46effff700 1 -- 172.21.15.161:0/3965229923 learned_addr learned my addr 172.21.15.161:0/3965229923 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:36.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d63eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.486+0000 7f46f4d9c700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46d80a12c0 0x7f46d80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46effff700 1 -- 172.21.15.161:0/3965229923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 msgr2=0x55e0f8d63eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46effff700 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d63eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46effff700 1 -- 172.21.15.161:0/3965229923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 msgr2=0x55e0f8d566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46effff700 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46effff700 1 -- 172.21.15.161:0/3965229923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f8ad6de0 con 0x55e0f8d56be0 2023-07-14T11:27:36.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:36.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46effff700 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d61590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f46e0011650 tx=0x7f46e0011960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a5591776658ab4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46f4d9c700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46d80a12c0 0x7f46d80a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46e809b270 tx=0x7f46e8009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46d67fc700 1 -- 172.21.15.161:0/3965229923 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000f680 con 0x55e0f8d56be0 2023-07-14T11:27:36.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46d67fc700 1 -- 172.21.15.161:0/3965229923 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e000fca0 con 0x55e0f8d56be0 2023-07-14T11:27:36.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.487+0000 7f46d67fc700 1 -- 172.21.15.161:0/3965229923 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0019e10 con 0x55e0f8d56be0 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 -- 172.21.15.161:0/3965229923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 msgr2=0x55e0f8d61590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d61590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f46e0011650 tx=0x7f46e0011960 comp rx=0 tx=0).stop 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 -- 172.21.15.161:0/3965229923 shutdown_connections 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d63eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 --2- 172.21.15.161:0/3965229923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d61590 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 -- 172.21.15.161:0/3965229923 >> 172.21.15.161:0/3965229923 conn(0x55e0f8d4cda0 msgr2=0x55e0f8d526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:36.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 -- 172.21.15.161:0/3965229923 shutdown_connections 2023-07-14T11:27:36.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.488+0000 7f470a190b80 1 -- 172.21.15.161:0/3965229923 wait complete. 2023-07-14T11:27:36.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 -- start start 2023-07-14T11:27:36.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 Processor -- start 2023-07-14T11:27:36.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d562d0 0x55e0f8d614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d56be0 0x55e0f8d5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d5edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8c48ac0 con 0x55e0f8d61ad0 2023-07-14T11:27:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8c487c0 con 0x55e0f8d56be0 2023-07-14T11:27:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8b2c920 con 0x55e0f8d562d0 2023-07-14T11:27:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.489+0000 7f470a190b80 10 client.0 _ll_register_callbacks cb 0x55e0f8c46760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d56be0 0x55e0f8d5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d56be0 0x55e0f8d5e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57950/0 (socket says 172.21.15.161:57950) 2023-07-14T11:27:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 -- 172.21.15.161:0/454669464 learned_addr learned my addr 172.21.15.161:0/454669464 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d562d0 0x55e0f8d614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d5edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 -- 172.21.15.161:0/454669464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d562d0 msgr2=0x55e0f8d614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d562d0 0x55e0f8d614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 -- 172.21.15.161:0/454669464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d61ad0 msgr2=0x55e0f8d5edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d5edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 -- 172.21.15.161:0/454669464 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e0009e40 con 0x55e0f8d56be0 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d5edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46effff700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d56be0 0x55e0f8d5e750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f46e00097d0 tx=0x7f46e0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d62d85653478ec5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.490+0000 7f46cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f46cf7fe700 1 -- 172.21.15.161:0/454669464 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000f890 con 0x55e0f8d56be0 2023-07-14T11:27:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f46cf7fe700 1 -- 172.21.15.161:0/454669464 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f46e0026070 con 0x55e0f8d56be0 2023-07-14T11:27:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0f8c024f0 con 0x55e0f8d56be0 2023-07-14T11:27:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f46cf7fe700 1 -- 172.21.15.161:0/454669464 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46e0019430 con 0x55e0f8d56be0 2023-07-14T11:27:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 10 client.26669 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e0f8abfc60 con 0x55e0f8d56be0 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f46cf7fe700 1 -- 172.21.15.161:0/454669464 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0f8c024f0 con 0x55e0f8d56be0 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f46cf7fe700 1 -- 172.21.15.161:0/454669464 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46e0004750 con 0x55e0f8d56be0 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 10 client.26669 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 20 client.26669 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8d713d0 0x55e0f8d737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 4 client.26669 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:36.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.491+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0f8ca1eb0 con 0x55e0f8d713d0 2023-07-14T11:27:36.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.492+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8d713d0 0x55e0f8d737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.492+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8d713d0 0x55e0f8d737e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5292b1d6ef69c872 server_cookie=d998c46c12af7ae8 in_seq=0 out_seq=0 2023-07-14T11:27:36.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.492+0000 7f46cf7fe700 10 client.26669 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:36.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f46cf7fe700 1 -- 172.21.15.161:0/454669464 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e0f8ca1eb0 con 0x55e0f8d713d0 2023-07-14T11:27:36.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f46cf7fe700 10 client.26669 handle_command_reply: tid=0 2023-07-14T11:27:36.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 client.26669 shutdown 2023-07-14T11:27:36.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d56be0 msgr2=0x55e0f8d5e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d56be0 0x55e0f8d5e750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f46e00097d0 tx=0x7f46e0005e80 comp rx=0 tx=0).stop 2023-07-14T11:27:36.503 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:36 smithi161 ceph-mon[95526]: pgmap v163: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2023-07-14T11:27:36.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 shutdown_connections 2023-07-14T11:27:36.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8d713d0 0x55e0f8d737e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d5edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d562d0 0x55e0f8d614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 --2- 172.21.15.161:0/454669464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d56be0 0x55e0f8d5e750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f46cf7fe700 0 client.26669 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:36.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 >> 172.21.15.161:0/454669464 conn(0x55e0f8d4cda0 msgr2=0x55e0f8d52230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:36.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 shutdown_connections 2023-07-14T11:27:36.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.493+0000 7f470a190b80 1 -- 172.21.15.161:0/454669464 wait complete. 2023-07-14T11:27:36.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.494+0000 7f470a190b80 20 client.26669 trim_cache size 0 max 16384 2023-07-14T11:27:36.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.512+0000 7f470a190b80 1 Processor -- start 2023-07-14T11:27:36.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.512+0000 7f470a190b80 1 -- start start 2023-07-14T11:27:36.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f470a190b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f470a190b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d59a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f470a190b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d59f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f470a190b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8d57580 con 0x55e0f8d562d0 2023-07-14T11:27:36.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f470a190b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8d5f6e0 con 0x55e0f8d61ad0 2023-07-14T11:27:36.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f470a190b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8d5ccb0 con 0x55e0f8d56be0 2023-07-14T11:27:36.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f46ef7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f46effff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d59a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f46ef7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45782/0 (socket says 172.21.15.161:45782) 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f46ef7fe700 1 -- 172.21.15.161:0/557247421 learned_addr learned my addr 172.21.15.161:0/557247421 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d59f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.513+0000 7f46ef7fe700 1 -- 172.21.15.161:0/557247421 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 msgr2=0x55e0f8d59a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d59a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46ef7fe700 1 -- 172.21.15.161:0/557247421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 msgr2=0x55e0f8d59f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d59f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46ef7fe700 1 -- 172.21.15.161:0/557247421 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f8afd250 con 0x55e0f8d562d0 2023-07-14T11:27:36.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46effff700 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d59a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:36.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d591b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f46e4000c00 tx=0x7f46e4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e63b1c6accb4de2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46d6ffd700 1 -- 172.21.15.161:0/557247421 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4010bb0 con 0x55e0f8d562d0 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46d6ffd700 1 -- 172.21.15.161:0/557247421 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e4019070 con 0x55e0f8d562d0 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.514+0000 7f46d6ffd700 1 -- 172.21.15.161:0/557247421 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4020a00 con 0x55e0f8d562d0 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 -- 172.21.15.161:0/557247421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 msgr2=0x55e0f8d591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d591b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f46e4000c00 tx=0x7f46e4009260 comp rx=0 tx=0).stop 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 -- 172.21.15.161:0/557247421 shutdown_connections 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d562d0 0x55e0f8d591b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d59f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 --2- 172.21.15.161:0/557247421 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d56be0 0x55e0f8d59a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 -- 172.21.15.161:0/557247421 >> 172.21.15.161:0/557247421 conn(0x55e0f8d4cda0 msgr2=0x55e0f8d6d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:36.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 -- 172.21.15.161:0/557247421 shutdown_connections 2023-07-14T11:27:36.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.515+0000 7f470a190b80 1 -- 172.21.15.161:0/557247421 wait complete. 2023-07-14T11:27:36.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 -- start start 2023-07-14T11:27:36.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 Processor -- start 2023-07-14T11:27:36.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d562d0 0x55e0f8d682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d56be0 0x55e0f8d66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8d67080 con 0x55e0f8d56be0 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8d671f0 con 0x55e0f8d562d0 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f8d673b0 con 0x55e0f8d61ad0 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.516+0000 7f470a190b80 10 client.0 _ll_register_callbacks cb 0x7f46e00114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d562d0 0x55e0f8d682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d562d0 0x55e0f8d682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57968/0 (socket says 172.21.15.161:57968) 2023-07-14T11:27:36.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 -- 172.21.15.161:0/1761452767 learned_addr learned my addr 172.21.15.161:0/1761452767 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:36.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46effff700 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d56be0 0x55e0f8d66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 -- 172.21.15.161:0/1761452767 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d61ad0 msgr2=0x55e0f8d66b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d66b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 -- 172.21.15.161:0/1761452767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d56be0 msgr2=0x55e0f8d66600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:36.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d56be0 0x55e0f8d66600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 -- 172.21.15.161:0/1761452767 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0f8abfc60 con 0x55e0f8d562d0 2023-07-14T11:27:36.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46ef7fe700 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d562d0 0x55e0f8d682d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46e4010fb0 tx=0x7f46e4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a34cc09c7ac2776f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.517+0000 7f46cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:36.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.518+0000 7f46cf7fe700 1 -- 172.21.15.161:0/1761452767 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4022a30 con 0x55e0f8d562d0 2023-07-14T11:27:36.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.518+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46e000fca0 con 0x55e0f8d562d0 2023-07-14T11:27:36.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.518+0000 7f46cf7fe700 1 -- 172.21.15.161:0/1761452767 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f46e4022b90 con 0x55e0f8d562d0 2023-07-14T11:27:36.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.518+0000 7f46cf7fe700 1 -- 172.21.15.161:0/1761452767 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46e4022d50 con 0x55e0f8d562d0 2023-07-14T11:27:36.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.518+0000 7f470a190b80 10 client.26675 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:36.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.518+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f46e401b040 con 0x55e0f8d562d0 2023-07-14T11:27:36.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.518+0000 7f46cf7fe700 1 -- 172.21.15.161:0/1761452767 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e000fca0 con 0x55e0f8d562d0 2023-07-14T11:27:36.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f46cf7fe700 1 -- 172.21.15.161:0/1761452767 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46e4038020 con 0x55e0f8d562d0 2023-07-14T11:27:36.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f470a190b80 10 client.26675 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:36.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f470a190b80 10 client.26675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:36.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f470a190b80 20 client.26675 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:36.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f470a190b80 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8dc6580 0x55e0f8dc8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f470a190b80 4 client.26675 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:27:36.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0f8c64050 con 0x55e0f8dc6580 2023-07-14T11:27:36.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f46effff700 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8dc6580 0x55e0f8dc8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f46effff700 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8dc6580 0x55e0f8dc8960 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b414f044c572d5 server_cookie=47ac415bdd3e0710 in_seq=0 out_seq=0 2023-07-14T11:27:36.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f46cf7fe700 10 client.26675 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f46cf7fe700 1 -- 172.21.15.161:0/1761452767 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e0f8c64050 con 0x55e0f8dc6580 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f46cf7fe700 10 client.26675 handle_command_reply: tid=0 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.519+0000 7f470a190b80 1 client.26675 shutdown 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d562d0 msgr2=0x55e0f8d682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d562d0 0x55e0f8d682d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46e4010fb0 tx=0x7f46e4004750 comp rx=0 tx=0).stop 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 shutdown_connections 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e0f8dc6580 0x55e0f8dc8960 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e0f8d61ad0 0x55e0f8d66b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0f8d562d0 0x55e0f8d682d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f46cf7fe700 0 client.26675 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:36.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 --2- 172.21.15.161:0/1761452767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0f8d56be0 0x55e0f8d66600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 >> 172.21.15.161:0/1761452767 conn(0x55e0f8d4cda0 msgr2=0x55e0f8d59a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:36.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 shutdown_connections 2023-07-14T11:27:36.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 1 -- 172.21.15.161:0/1761452767 wait complete. 2023-07-14T11:27:36.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f470a190b80 20 client.26675 trim_cache size 0 max 16384 2023-07-14T11:27:36.531 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:27:36.531 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:27:36.531 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:27:36.531 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:27:36.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46d80a12c0 msgr2=0x7f46d80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.520+0000 7f46f67ff700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46d80a12c0 0x7f46d80a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46e809b270 tx=0x7f46e8009920 comp rx=0 tx=0).stop 2023-07-14T11:27:36.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f010c3a0 msgr2=0x7f46f019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f010c3a0 0x7f46f019c990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b6c0 tx=0x7f46e80094b0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 shutdown_connections 2023-07-14T11:27:36.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46d80a12c0 0x7f46d80a3770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f010c3a0 0x7f46f019c990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f46f0072850 0x7f46f0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 --2- 172.21.15.161:0/3783060432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f0073160 0x7f46f0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 >> 172.21.15.161:0/3783060432 conn(0x7f46f00fb600 msgr2=0x7f46f0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:36.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 shutdown_connections 2023-07-14T11:27:36.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.521+0000 7f46f67ff700 1 -- 172.21.15.161:0/3783060432 wait complete. 2023-07-14T11:27:36.542 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:27:36.543 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:27:36.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:36 smithi175 ceph-mon[129081]: pgmap v163: 129 pgs: 129 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2023-07-14T11:27:36.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3964242671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 msgr2=0x7fa42810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3964242671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 0x7fa42810e770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa420009f90 tx=0x7fa42000b330 comp rx=0 tx=0).stop 2023-07-14T11:27:36.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3964242671 shutdown_connections 2023-07-14T11:27:36.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3964242671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 0x7fa42810e770 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3964242671 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa428072850 0x7fa428072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3964242671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 0x7fa42810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3964242671 >> 172.21.15.161:0/3964242671 conn(0x7fa4280fb630 msgr2=0x7fa4280fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:36.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3964242671 shutdown_connections 2023-07-14T11:27:36.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.759+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3964242671 wait complete. 2023-07-14T11:27:36.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.760+0000 7fa42e9ff700 1 Processor -- start 2023-07-14T11:27:36.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.760+0000 7fa42e9ff700 1 -- start start 2023-07-14T11:27:36.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42e9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa428072850 0x7fa428197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42e9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 0x7fa428198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42e9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 0x7fa42819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42e9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819cf00 con 0x7fa428073160 2023-07-14T11:27:36.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42e9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819d070 con 0x7fa42810c390 2023-07-14T11:27:36.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42e9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819d230 con 0x7fa428072850 2023-07-14T11:27:36.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa4277fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 0x7fa428198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa427fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa428072850 0x7fa428197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa4277fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 0x7fa428198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45800/0 (socket says 172.21.15.161:45800) 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42cf9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 0x7fa42819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa4277fe700 1 -- 172.21.15.161:0/3560168866 learned_addr learned my addr 172.21.15.161:0/3560168866 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42cf9c700 1 -- 172.21.15.161:0/3560168866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa428072850 msgr2=0x7fa428197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42cf9c700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa428072850 0x7fa428197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42cf9c700 1 -- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 msgr2=0x7fa428198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42cf9c700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 0x7fa428198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:36.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa42cf9c700 1 -- 172.21.15.161:0/3560168866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009c40 con 0x7fa42810c390 2023-07-14T11:27:36.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa427fff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa428072850 0x7fa428197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:36.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.761+0000 7fa4277fe700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 0x7fa428198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:36.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.762+0000 7fa42cf9c700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 0x7fa42819c8c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b6c0 tx=0x7fa4200094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.762+0000 7fa4257fa700 1 -- 172.21.15.161:0/3560168866 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42000e040 con 0x7fa42810c390 2023-07-14T11:27:36.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.762+0000 7fa4257fa700 1 -- 172.21.15.161:0/3560168866 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa420004e70 con 0x7fa42810c390 2023-07-14T11:27:36.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.762+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42819d4b0 con 0x7fa42810c390 2023-07-14T11:27:36.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.762+0000 7fa4257fa700 1 -- 172.21.15.161:0/3560168866 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420005040 con 0x7fa42810c390 2023-07-14T11:27:36.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.762+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4281a4d50 con 0x7fa42810c390 2023-07-14T11:27:36.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.763+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa428106cb0 con 0x7fa42810c390 2023-07-14T11:27:36.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.765+0000 7fa4257fa700 1 -- 172.21.15.161:0/3560168866 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa42001d470 con 0x7fa42810c390 2023-07-14T11:27:36.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.766+0000 7fa4257fa700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa4100a1300 0x7fa4100a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:36.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.767+0000 7fa4257fa700 1 -- 172.21.15.161:0/3560168866 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7fa4200ce480 con 0x7fa42810c390 2023-07-14T11:27:36.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.767+0000 7fa427fff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa4100a1300 0x7fa4100a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:36.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.767+0000 7fa427fff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa4100a1300 0x7fa4100a37b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa418005fd0 tx=0x7fa418009310 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:36.769+0000 7fa4257fa700 1 -- 172.21.15.161:0/3560168866 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa420097590 con 0x7fa42810c390 2023-07-14T11:27:37.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.040+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa428199380 con 0x7fa42810c390 2023-07-14T11:27:37.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.040+0000 7fa4257fa700 1 -- 172.21.15.161:0/3560168866 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10864 (secure 0 0 0) 0x7fa420095e60 con 0x7fa42810c390 2023-07-14T11:27:37.044 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:27:37.044 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:26:11.184716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:27:37.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa4100a1300 msgr2=0x7fa4100a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa4100a1300 0x7fa4100a37b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa418005fd0 tx=0x7fa418009310 comp rx=0 tx=0).stop 2023-07-14T11:27:37.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 msgr2=0x7fa42819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 0x7fa42819c8c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b6c0 tx=0x7fa4200094b0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 shutdown_connections 2023-07-14T11:27:37.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa4100a1300 0x7fa4100a37b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa42810c390 0x7fa42819c8c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa428072850 0x7fa428197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 --2- 172.21.15.161:0/3560168866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa428073160 0x7fa428198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 >> 172.21.15.161:0/3560168866 conn(0x7fa4280fb630 msgr2=0x7fa428103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 shutdown_connections 2023-07-14T11:27:37.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.043+0000 7fa42e9ff700 1 -- 172.21.15.161:0/3560168866 wait complete. 2023-07-14T11:27:37.048 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 25 2023-07-14T11:27:37.066 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:27:37.183 INFO:tasks.workunit.client.0.smithi161.stdout: 1 480026 9.88 MB/sec execute 139 sec latency 33.660 ms 2023-07-14T11:27:37.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.283+0000 7f78da5ff700 1 -- 172.21.15.161:0/2557293235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 msgr2=0x7f78d4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.283+0000 7f78da5ff700 1 --2- 172.21.15.161:0/2557293235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4102340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009990 tx=0x7f78bc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.284+0000 7f78da5ff700 1 -- 172.21.15.161:0/2557293235 shutdown_connections 2023-07-14T11:27:37.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.284+0000 7f78da5ff700 1 --2- 172.21.15.161:0/2557293235 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78d4106fb0 0x7f78d410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.284+0000 7f78da5ff700 1 --2- 172.21.15.161:0/2557293235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4102340 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.284+0000 7f78da5ff700 1 --2- 172.21.15.161:0/2557293235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d4102910 0x7f78d41068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.284+0000 7f78da5ff700 1 -- 172.21.15.161:0/2557293235 >> 172.21.15.161:0/2557293235 conn(0x7f78d40fd7e0 msgr2=0x7f78d40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.284+0000 7f78da5ff700 1 -- 172.21.15.161:0/2557293235 shutdown_connections 2023-07-14T11:27:37.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.284+0000 7f78da5ff700 1 -- 172.21.15.161:0/2557293235 wait complete. 2023-07-14T11:27:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.285+0000 7f78da5ff700 1 Processor -- start 2023-07-14T11:27:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.285+0000 7f78da5ff700 1 -- start start 2023-07-14T11:27:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78da5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78da5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d4102910 0x7f78d4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78da5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78d4106fb0 0x7f78d419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78da5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d419ced0 con 0x7f78d4102910 2023-07-14T11:27:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78da5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d419d040 con 0x7f78d4101f70 2023-07-14T11:27:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78da5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d419d200 con 0x7f78d4106fb0 2023-07-14T11:27:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57990/0 (socket says 172.21.15.161:57990) 2023-07-14T11:27:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78d3fff700 1 -- 172.21.15.161:0/3772983900 learned_addr learned my addr 172.21.15.161:0/3772983900 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78d4106fb0 0x7f78d419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.286+0000 7f78d37fe700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d4102910 0x7f78d4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d3fff700 1 -- 172.21.15.161:0/3772983900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78d4106fb0 msgr2=0x7f78d419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78d4106fb0 0x7f78d419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d3fff700 1 -- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d4102910 msgr2=0x7f78d4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d4102910 0x7f78d4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d3fff700 1 -- 172.21.15.161:0/3772983900 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc009c40 con 0x7f78d4101f70 2023-07-14T11:27:37.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4197ef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78bc000c00 tx=0x7f78bc004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d17fa700 1 -- 172.21.15.161:0/3772983900 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc01d070 con 0x7f78d4101f70 2023-07-14T11:27:37.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc0095f0 con 0x7f78d4101f70 2023-07-14T11:27:37.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d17fa700 1 -- 172.21.15.161:0/3772983900 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78bc021410 con 0x7f78d4101f70 2023-07-14T11:27:37.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78d17fa700 1 -- 172.21.15.161:0/3772983900 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc017650 con 0x7f78d4101f70 2023-07-14T11:27:37.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.287+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d41a4b40 con 0x7f78d4101f70 2023-07-14T11:27:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.292+0000 7f78d17fa700 1 -- 172.21.15.161:0/3772983900 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f78bc021580 con 0x7f78d4101f70 2023-07-14T11:27:37.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.293+0000 7f78ee00cb80 1 Processor -- start 2023-07-14T11:27:37.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.293+0000 7f78ee00cb80 1 -- start start 2023-07-14T11:27:37.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.293+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f2a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.293+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f35590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.293+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f37eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9e30cb0 con 0x55ccd9f2a2d0 2023-07-14T11:27:37.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9e83a40 con 0x55ccd9f35ad0 2023-07-14T11:27:37.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9e7ced0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d8b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f2a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f37eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f37eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57998/0 (socket says 172.21.15.161:57998) 2023-07-14T11:27:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 -- 172.21.15.161:0/3434811245 learned_addr learned my addr 172.21.15.161:0/3434811245 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f35590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 -- 172.21.15.161:0/3434811245 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f2abe0 msgr2=0x55ccd9f35590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f35590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 -- 172.21.15.161:0/3434811245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 msgr2=0x55ccd9f2a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f2a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 -- 172.21.15.161:0/3434811245 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccd9caade0 con 0x55ccd9f35ad0 2023-07-14T11:27:37.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d17fa700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78c00a12c0 0x7f78c00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.294+0000 7f78d37fe700 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f37eb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009cb0 tx=0x7f78c4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e726eb13bb8ead60 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78d17fa700 1 -- 172.21.15.161:0/3772983900 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f78bc0ce130 con 0x7f78d4101f70 2023-07-14T11:27:37.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78c00a12c0 0x7f78c00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78ca7fc700 1 -- 172.21.15.161:0/3434811245 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4011d80 con 0x55ccd9f35ad0 2023-07-14T11:27:37.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78ca7fc700 1 -- 172.21.15.161:0/3434811245 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78c4004510 con 0x55ccd9f35ad0 2023-07-14T11:27:37.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78ca7fc700 1 -- 172.21.15.161:0/3434811245 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4005280 con 0x55ccd9f35ad0 2023-07-14T11:27:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78c00a12c0 0x7f78c00a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009fd0 tx=0x7f78cc00b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3434811245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f35ad0 msgr2=0x55ccd9f37eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.295+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f37eb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009cb0 tx=0x7f78c4011440 comp rx=0 tx=0).stop 2023-07-14T11:27:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3434811245 shutdown_connections 2023-07-14T11:27:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f2a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f37eb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3434811245 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f35590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3434811245 >> 172.21.15.161:0/3434811245 conn(0x55ccd9f20da0 msgr2=0x55ccd9f266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3434811245 shutdown_connections 2023-07-14T11:27:37.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3434811245 wait complete. 2023-07-14T11:27:37.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.296+0000 7f78ee00cb80 1 -- start start 2023-07-14T11:27:37.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 1 Processor -- start 2023-07-14T11:27:37.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f327a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f32e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9e1cac0 con 0x55ccd9f2a2d0 2023-07-14T11:27:37.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9e1c7c0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9d00920 con 0x55ccd9f35ad0 2023-07-14T11:27:37.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78ee00cb80 10 client.0 _ll_register_callbacks cb 0x55ccd9e1a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:37.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f327a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:58006/0 (socket says 172.21.15.161:58006) 2023-07-14T11:27:37.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78d37fe700 1 -- 172.21.15.161:0/3733386384 learned_addr learned my addr 172.21.15.161:0/3733386384 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.297+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f32e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78d37fe700 1 -- 172.21.15.161:0/3733386384 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 msgr2=0x55ccd9f32e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78d37fe700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f32e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78d37fe700 1 -- 172.21.15.161:0/3733386384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 msgr2=0x55ccd9f354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78d37fe700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78d37fe700 1 -- 172.21.15.161:0/3733386384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c40098e0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f32e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:37.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78d37fe700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f327a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005ce0 tx=0x7f78c4005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bcdd9dcfffec7b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:37.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78b37fe700 1 -- 172.21.15.161:0/3733386384 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4004050 con 0x55ccd9f2abe0 2023-07-14T11:27:37.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccd9dd64f0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.298+0000 7f78b37fe700 1 -- 172.21.15.161:0/3733386384 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f78c40041b0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78b37fe700 1 -- 172.21.15.161:0/3733386384 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78c4004370 con 0x55ccd9f2abe0 2023-07-14T11:27:37.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 10 client.26693 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:37.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ccd9c93c60 con 0x55ccd9f2abe0 2023-07-14T11:27:37.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78b37fe700 1 -- 172.21.15.161:0/3733386384 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ccd9dd64f0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78b37fe700 1 -- 172.21.15.161:0/3733386384 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f78c40050b0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 10 client.26693 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:37.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 10 client.26693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:27:37.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 20 client.26693 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:37.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f45340 0x55ccd9f47750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 4 client.26693 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:37.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccd9e75eb0 con 0x55ccd9f45340 2023-07-14T11:27:37.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.299+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f45340 0x55ccd9f47750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.300+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f45340 0x55ccd9f47750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e835918588f7216a server_cookie=d47c11a468e7b17f in_seq=0 out_seq=0 2023-07-14T11:27:37.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.300+0000 7f78b37fe700 10 client.26693 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:37.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78b37fe700 1 -- 172.21.15.161:0/3733386384 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ccd9e75eb0 con 0x55ccd9f45340 2023-07-14T11:27:37.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78b37fe700 10 client.26693 handle_command_reply: tid=0 2023-07-14T11:27:37.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 client.26693 shutdown 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 msgr2=0x55ccd9f327a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f327a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005ce0 tx=0x7f78c4005d10 comp rx=0 tx=0).stop 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 shutdown_connections 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f45340 0x55ccd9f47750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f32e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3733386384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f327a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 >> 172.21.15.161:0/3733386384 conn(0x55ccd9f20da0 msgr2=0x55ccd9f26290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78b37fe700 0 client.26693 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:37.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 shutdown_connections 2023-07-14T11:27:37.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.301+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3733386384 wait complete. 2023-07-14T11:27:37.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.302+0000 7f78ee00cb80 20 client.26693 trim_cache size 0 max 16384 2023-07-14T11:27:37.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 Processor -- start 2023-07-14T11:27:37.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 -- start start 2023-07-14T11:27:37.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f2daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9f27d20 con 0x55ccd9f2a2d0 2023-07-14T11:27:37.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9f2b540 con 0x55ccd9f2abe0 2023-07-14T11:27:37.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9f1dfa0 con 0x55ccd9f35ad0 2023-07-14T11:27:37.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78d8b9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f2daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78d8b9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f2e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60946/0 (socket says 172.21.15.161:60946) 2023-07-14T11:27:37.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.317+0000 7f78d8b9c700 1 -- 172.21.15.161:0/3303793555 learned_addr learned my addr 172.21.15.161:0/3303793555 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d8b9c700 1 -- 172.21.15.161:0/3303793555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 msgr2=0x55ccd9f2daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f2daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d8b9c700 1 -- 172.21.15.161:0/3303793555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 msgr2=0x55ccd9f47350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f47350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d8b9c700 1 -- 172.21.15.161:0/3303793555 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccd9cd1250 con 0x55ccd9f35ad0 2023-07-14T11:27:37.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d3fff700 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f47350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:37.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78d8b9c700 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f2e030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f78cc004380 tx=0x7f78cc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9ef33c866bfdad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78caffd700 1 -- 172.21.15.161:0/3303793555 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00e040 con 0x55ccd9f35ad0 2023-07-14T11:27:37.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78caffd700 1 -- 172.21.15.161:0/3303793555 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78cc00f040 con 0x55ccd9f35ad0 2023-07-14T11:27:37.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.318+0000 7f78caffd700 1 -- 172.21.15.161:0/3303793555 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc014a90 con 0x55ccd9f35ad0 2023-07-14T11:27:37.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3303793555 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 msgr2=0x55ccd9f2e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f2e030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f78cc004380 tx=0x7f78cc017000 comp rx=0 tx=0).stop 2023-07-14T11:27:37.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3303793555 shutdown_connections 2023-07-14T11:27:37.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f47350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f2e030 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/3303793555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f2daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3303793555 >> 172.21.15.161:0/3303793555 conn(0x55ccd9f20da0 msgr2=0x55ccd9f41740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3303793555 shutdown_connections 2023-07-14T11:27:37.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.319+0000 7f78ee00cb80 1 -- 172.21.15.161:0/3303793555 wait complete. 2023-07-14T11:27:37.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.320+0000 7f78ee00cb80 1 -- start start 2023-07-14T11:27:37.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.320+0000 7f78ee00cb80 1 Processor -- start 2023-07-14T11:27:37.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f3a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f3a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78ee00cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9f3b2e0 con 0x55ccd9f2a2d0 2023-07-14T11:27:37.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9f98700 con 0x55ccd9f2abe0 2023-07-14T11:27:37.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78ee00cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd9f98840 con 0x55ccd9f35ad0 2023-07-14T11:27:37.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78ee00cb80 10 client.0 _ll_register_callbacks cb 0x7f78cc00e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:37.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f3a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78d8b9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f3a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:58020/0 (socket says 172.21.15.161:58020) 2023-07-14T11:27:37.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78d37fe700 1 -- 172.21.15.161:0/28292505 learned_addr learned my addr 172.21.15.161:0/28292505 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.321+0000 7f78d3fff700 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f3a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78d37fe700 1 -- 172.21.15.161:0/28292505 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 msgr2=0x55ccd9f981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78d37fe700 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78d37fe700 1 -- 172.21.15.161:0/28292505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 msgr2=0x55ccd9f3a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78d37fe700 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f3a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78d37fe700 1 -- 172.21.15.161:0/28292505 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc004030 con 0x55ccd9f2abe0 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78d37fe700 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f3a9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78c401c040 tx=0x7f78c4005860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a2e600bcc9678c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:37.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78b37fe700 1 -- 172.21.15.161:0/28292505 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4011b80 con 0x55ccd9f2abe0 2023-07-14T11:27:37.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78c4004510 con 0x55ccd9f2abe0 2023-07-14T11:27:37.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78b37fe700 1 -- 172.21.15.161:0/28292505 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f78c4000e40 con 0x55ccd9f2abe0 2023-07-14T11:27:37.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78b37fe700 1 -- 172.21.15.161:0/28292505 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78c4004eb0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78ee00cb80 10 client.26699 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:37.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.322+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ccd9ca92b0 con 0x55ccd9f2abe0 2023-07-14T11:27:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78b37fe700 1 -- 172.21.15.161:0/28292505 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78c4040020 con 0x55ccd9f2abe0 2023-07-14T11:27:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78b37fe700 1 -- 172.21.15.161:0/28292505 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f78c403d070 con 0x55ccd9f2abe0 2023-07-14T11:27:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78ee00cb80 10 client.26699 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78ee00cb80 10 client.26699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:27:37.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78ee00cb80 20 client.26699 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:37.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f99c30 0x55ccd9f9c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78ee00cb80 4 client.26699 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:27:37.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ccd9e38050 con 0x55ccd9f99c30 2023-07-14T11:27:37.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.323+0000 7f78d3fff700 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f99c30 0x55ccd9f9c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.324+0000 7f78d3fff700 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f99c30 0x55ccd9f9c040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83ecaab03dc608a server_cookie=89c15039bdf0f5f7 in_seq=0 out_seq=0 2023-07-14T11:27:37.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.324+0000 7f78b37fe700 10 client.26699 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:37.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78b37fe700 1 -- 172.21.15.161:0/28292505 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ccd9e38050 con 0x55ccd9f99c30 2023-07-14T11:27:37.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78b37fe700 10 client.26699 handle_command_reply: tid=0 2023-07-14T11:27:37.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 client.26699 shutdown 2023-07-14T11:27:37.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 msgr2=0x55ccd9f3a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f3a9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78c401c040 tx=0x7f78c4005860 comp rx=0 tx=0).stop 2023-07-14T11:27:37.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 shutdown_connections 2023-07-14T11:27:37.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55ccd9f99c30 0x55ccd9f9c040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccd9f2a2d0 0x55ccd9f3a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ccd9f35ad0 0x55ccd9f981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 --2- 172.21.15.161:0/28292505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccd9f2abe0 0x55ccd9f3a9d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78b37fe700 0 client.26699 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:27:37.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 >> 172.21.15.161:0/28292505 conn(0x55ccd9f20da0 msgr2=0x55ccd9f350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 shutdown_connections 2023-07-14T11:27:37.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 1 -- 172.21.15.161:0/28292505 wait complete. 2023-07-14T11:27:37.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.325+0000 7f78ee00cb80 20 client.26699 trim_cache size 0 max 16384 2023-07-14T11:27:37.340 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:27:37.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78c00a12c0 msgr2=0x7f78c00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78c00a12c0 0x7f78c00a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009fd0 tx=0x7f78cc00b040 comp rx=0 tx=0).stop 2023-07-14T11:27:37.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 msgr2=0x7f78d4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4197ef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78bc000c00 tx=0x7f78bc004790 comp rx=0 tx=0).stop 2023-07-14T11:27:37.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 shutdown_connections 2023-07-14T11:27:37.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f78c00a12c0 0x7f78c00a3770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f78d4106fb0 0x7f78d419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d4101f70 0x7f78d4197ef0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 --2- 172.21.15.161:0/3772983900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d4102910 0x7f78d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 >> 172.21.15.161:0/3772983900 conn(0x7f78d40fd7e0 msgr2=0x7f78d4105910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 shutdown_connections 2023-07-14T11:27:37.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.326+0000 7f78da5ff700 1 -- 172.21.15.161:0/3772983900 wait complete. 2023-07-14T11:27:37.346 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3560168866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:27:37.348 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:27:37.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:37 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3560168866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:27:37.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.560+0000 7f07c03ff700 1 -- 172.21.15.161:0/3578235712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b8105c00 msgr2=0x7f07b8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.560+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3578235712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b8105c00 0x7f07b8105fd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009990 tx=0x7f07b4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.561+0000 7f07c03ff700 1 -- 172.21.15.161:0/3578235712 shutdown_connections 2023-07-14T11:27:37.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.561+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3578235712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07b80feb90 0x7f07b810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.561+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3578235712 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b8105c00 0x7f07b8105fd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.561+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3578235712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07b80fe200 0x7f07b80fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.561+0000 7f07c03ff700 1 -- 172.21.15.161:0/3578235712 >> 172.21.15.161:0/3578235712 conn(0x7f07b8076200 msgr2=0x7f07b8076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.561+0000 7f07c03ff700 1 -- 172.21.15.161:0/3578235712 shutdown_connections 2023-07-14T11:27:37.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.561+0000 7f07c03ff700 1 -- 172.21.15.161:0/3578235712 wait complete. 2023-07-14T11:27:37.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 Processor -- start 2023-07-14T11:27:37.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 -- start start 2023-07-14T11:27:37.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07b80fe200 0x7f07b8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b80feb90 0x7f07b8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07b8105c00 0x7f07b819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b819cee0 con 0x7f07b8105c00 2023-07-14T11:27:37.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b819d050 con 0x7f07b80fe200 2023-07-14T11:27:37.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.562+0000 7f07c03ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b819d210 con 0x7f07b80feb90 2023-07-14T11:27:37.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b80feb90 0x7f07b8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07be19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07b80fe200 0x7f07b8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07be19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07b80fe200 0x7f07b8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:58032/0 (socket says 172.21.15.161:58032) 2023-07-14T11:27:37.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b80feb90 0x7f07b8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60974/0 (socket says 172.21.15.161:60974) 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07be19b700 1 -- 172.21.15.161:0/3220110719 learned_addr learned my addr 172.21.15.161:0/3220110719 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07be99c700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07b8105c00 0x7f07b819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 -- 172.21.15.161:0/3220110719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07b80fe200 msgr2=0x7f07b8197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07b80fe200 0x7f07b8197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 -- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07b8105c00 msgr2=0x7f07b819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07b8105c00 0x7f07b819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 -- 172.21.15.161:0/3220110719 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40095f0 con 0x7f07b80feb90 2023-07-14T11:27:37.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07be99c700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07b8105c00 0x7f07b819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:37.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07bd99a700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b80feb90 0x7f07b8198440 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f07a800e9f0 tx=0x7f07a800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.563+0000 7f07af7fe700 1 -- 172.21.15.161:0/3220110719 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800eee0 con 0x7f07b80feb90 2023-07-14T11:27:37.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.564+0000 7f07af7fe700 1 -- 172.21.15.161:0/3220110719 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a800cbd0 con 0x7f07b80feb90 2023-07-14T11:27:37.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.564+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b819d4f0 con 0x7f07b80feb90 2023-07-14T11:27:37.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.564+0000 7f07af7fe700 1 -- 172.21.15.161:0/3220110719 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8018680 con 0x7f07b80feb90 2023-07-14T11:27:37.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.564+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b81a4d30 con 0x7f07b80feb90 2023-07-14T11:27:37.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.567+0000 7f07af7fe700 1 -- 172.21.15.161:0/3220110719 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f07a800cd40 con 0x7f07b80feb90 2023-07-14T11:27:37.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 Processor -- start 2023-07-14T11:27:37.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 -- start start 2023-07-14T11:27:37.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d1f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07af7fe700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f07a40a1340 0x7f07a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299c25cb0 con 0x55e299d1f2d0 2023-07-14T11:27:37.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299c78a40 con 0x55e299d2aad0 2023-07-14T11:27:37.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.568+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299c71ed0 con 0x55e299d1fbe0 2023-07-14T11:27:37.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07af7fe700 1 -- 172.21.15.161:0/3220110719 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f07a8014070 con 0x7f07b80feb90 2023-07-14T11:27:37.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:58036/0 (socket says 172.21.15.161:58036) 2023-07-14T11:27:37.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07be19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 -- 172.21.15.161:0/1841458871 learned_addr learned my addr 172.21.15.161:0/1841458871 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07be99c700 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d1f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07be99c700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f07a40a1340 0x7f07a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 -- 172.21.15.161:0/1841458871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 msgr2=0x55e299d2a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d2a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 -- 172.21.15.161:0/1841458871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 msgr2=0x55e299d1f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d1f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 -- 172.21.15.161:0/1841458871 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e299a9fde0 con 0x55e299d2aad0 2023-07-14T11:27:37.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.569+0000 7f07bd99a700 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2ceb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f07a8099ef0 tx=0x7f07a800af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c9331a834ffc24 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07acbfe700 1 -- 172.21.15.161:0/1841458871 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8099d30 con 0x55e299d2aad0 2023-07-14T11:27:37.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07acbfe700 1 -- 172.21.15.161:0/1841458871 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a8099aa0 con 0x55e299d2aad0 2023-07-14T11:27:37.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07be99c700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f07a40a1340 0x7f07a40a37f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07b0003ac0 tx=0x7f07b0009670 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07acbfe700 1 -- 172.21.15.161:0/1841458871 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a809a420 con 0x55e299d2aad0 2023-07-14T11:27:37.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1841458871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 msgr2=0x55e299d2ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2ceb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f07a8099ef0 tx=0x7f07a800af00 comp rx=0 tx=0).stop 2023-07-14T11:27:37.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1841458871 shutdown_connections 2023-07-14T11:27:37.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d1f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2ceb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1841458871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.570+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1841458871 >> 172.21.15.161:0/1841458871 conn(0x55e299d15da0 msgr2=0x55e299d1b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.571+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1841458871 shutdown_connections 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.571+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1841458871 wait complete. 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.571+0000 7f07d3c7ab80 1 -- start start 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.571+0000 7f07d3c7ab80 1 Processor -- start 2023-07-14T11:27:37.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1f2d0 0x55e299d2a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1fbe0 0x55e299d27770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d27da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299c11ac0 con 0x55e299d2aad0 2023-07-14T11:27:37.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299c117c0 con 0x55e299d1fbe0 2023-07-14T11:27:37.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299af5920 con 0x55e299d1f2d0 2023-07-14T11:27:37.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07d3c7ab80 10 client.0 _ll_register_callbacks cb 0x55e299c0f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:37.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1fbe0 0x55e299d27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1fbe0 0x55e299d27770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:58050/0 (socket says 172.21.15.161:58050) 2023-07-14T11:27:37.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 -- 172.21.15.161:0/470725389 learned_addr learned my addr 172.21.15.161:0/470725389 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07be99c700 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d27da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07be19b700 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1f2d0 0x55e299d2a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 -- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1f2d0 msgr2=0x55e299d2a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1f2d0 0x55e299d2a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 -- 172.21.15.161:0/470725389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 msgr2=0x55e299d27da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d27da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.572+0000 7f07bd99a700 1 -- 172.21.15.161:0/470725389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b81a4d30 con 0x55e299d1fbe0 2023-07-14T11:27:37.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07bd99a700 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1fbe0 0x55e299d27770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000c00 tx=0x7f07a809a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef83885b8b0c068 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:37.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07a1ffb700 1 -- 172.21.15.161:0/470725389 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a809df90 con 0x55e299d1fbe0 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e299bcb4f0 con 0x55e299d1fbe0 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07a1ffb700 1 -- 172.21.15.161:0/470725389 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f07a801c1c0 con 0x55e299d1fbe0 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07a1ffb700 1 -- 172.21.15.161:0/470725389 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a80a23b0 con 0x55e299d1fbe0 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07d3c7ab80 10 client.26711 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e299a88c60 con 0x55e299d1fbe0 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.573+0000 7f07a1ffb700 1 -- 172.21.15.161:0/470725389 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a80ca020 con 0x55e299d1fbe0 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07a1ffb700 1 -- 172.21.15.161:0/470725389 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f07a80ca190 con 0x55e299d1fbe0 2023-07-14T11:27:37.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07d3c7ab80 10 client.26711 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:37.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07d3c7ab80 10 client.26711 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:27:37.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07d3c7ab80 20 client.26711 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:37.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d3a130 0x55e299d3c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07d3c7ab80 4 client.26711 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:37.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e299c6aeb0 con 0x55e299d3a130 2023-07-14T11:27:37.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07be19b700 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d3a130 0x55e299d3c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07be19b700 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d3a130 0x55e299d3c540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3de573cc4ec3453 server_cookie=7b0f765ffc0d1e39 in_seq=0 out_seq=0 2023-07-14T11:27:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.574+0000 7f07a1ffb700 10 client.26711 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.575+0000 7f07a1ffb700 1 -- 172.21.15.161:0/470725389 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e299c6aeb0 con 0x55e299d3a130 2023-07-14T11:27:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.575+0000 7f07a1ffb700 10 client.26711 handle_command_reply: tid=0 2023-07-14T11:27:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.575+0000 7f07d3c7ab80 1 client.26711 shutdown 2023-07-14T11:27:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1fbe0 msgr2=0x55e299d27770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1fbe0 0x55e299d27770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000c00 tx=0x7f07a809a3d0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 shutdown_connections 2023-07-14T11:27:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d3a130 0x55e299d3c540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d27da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1f2d0 0x55e299d2a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/470725389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1fbe0 0x55e299d27770 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07a1ffb700 0 client.26711 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 >> 172.21.15.161:0/470725389 conn(0x55e299d15da0 msgr2=0x55e299d1b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 shutdown_connections 2023-07-14T11:27:37.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/470725389 wait complete. 2023-07-14T11:27:37.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.576+0000 7f07d3c7ab80 20 client.26711 trim_cache size 0 max 16384 2023-07-14T11:27:37.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.590+0000 7f07d3c7ab80 1 Processor -- start 2023-07-14T11:27:37.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07d3c7ab80 1 -- start start 2023-07-14T11:27:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1f2d0 0x55e299d27d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d19e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299d20160 con 0x55e299d2aad0 2023-07-14T11:27:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299d20320 con 0x55e299d1f2d0 2023-07-14T11:27:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299d20540 con 0x55e299d1fbe0 2023-07-14T11:27:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d19e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07be19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1f2d0 0x55e299d27d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d19e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60984/0 (socket says 172.21.15.161:60984) 2023-07-14T11:27:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07bd99a700 1 -- 172.21.15.161:0/1379227475 learned_addr learned my addr 172.21.15.161:0/1379227475 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.591+0000 7f07be99c700 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07bd99a700 1 -- 172.21.15.161:0/1379227475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1f2d0 msgr2=0x55e299d27d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07bd99a700 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1f2d0 0x55e299d27d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07bd99a700 1 -- 172.21.15.161:0/1379227475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 msgr2=0x55e299d314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07bd99a700 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07bd99a700 1 -- 172.21.15.161:0/1379227475 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e299ac6250 con 0x55e299d1fbe0 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07be99c700 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07bd99a700 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d19e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07a8099c60 tx=0x7f07a80b8250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f2bebc7fc5ae808 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07ad3ff700 1 -- 172.21.15.161:0/1379227475 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80a23b0 con 0x55e299d1fbe0 2023-07-14T11:27:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07ad3ff700 1 -- 172.21.15.161:0/1379227475 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a80ca040 con 0x55e299d1fbe0 2023-07-14T11:27:37.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.592+0000 7f07ad3ff700 1 -- 172.21.15.161:0/1379227475 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80ca1b0 con 0x55e299d1fbe0 2023-07-14T11:27:37.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1379227475 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 msgr2=0x55e299d19e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d19e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07a8099c60 tx=0x7f07a80b8250 comp rx=0 tx=0).stop 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1379227475 shutdown_connections 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d2aad0 0x55e299d314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d1f2d0 0x55e299d27d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/1379227475 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d19e10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1379227475 >> 172.21.15.161:0/1379227475 conn(0x55e299d15da0 msgr2=0x55e299d40e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1379227475 shutdown_connections 2023-07-14T11:27:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/1379227475 wait complete. 2023-07-14T11:27:37.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.593+0000 7f07d3c7ab80 1 -- start start 2023-07-14T11:27:37.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 1 Processor -- start 2023-07-14T11:27:37.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d30ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299d2dbd0 con 0x55e299d1f2d0 2023-07-14T11:27:37.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299d2dd40 con 0x55e299d2aad0 2023-07-14T11:27:37.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e299d2df00 con 0x55e299d1fbe0 2023-07-14T11:27:37.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07d3c7ab80 10 client.0 _ll_register_callbacks cb 0x7f07a8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:37.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.594+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07be99c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d31410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:32768/0 (socket says 172.21.15.161:32768) 2023-07-14T11:27:37.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 -- 172.21.15.161:0/201831698 learned_addr learned my addr 172.21.15.161:0/201831698 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07be19b700 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d30ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 -- 172.21.15.161:0/201831698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 msgr2=0x55e299d2d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 -- 172.21.15.161:0/201831698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 msgr2=0x55e299d30ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d30ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 -- 172.21.15.161:0/201831698 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e299a88c60 con 0x55e299d1fbe0 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07bd99a700 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d31410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07a8099c10 tx=0x7f07a80961e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12bdad9c7aad239 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:37.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07a1ffb700 1 -- 172.21.15.161:0/201831698 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80b8370 con 0x55e299d1fbe0 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07a1ffb700 1 -- 172.21.15.161:0/201831698 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f07a8099f60 con 0x55e299d1fbe0 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.595+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07a8099aa0 con 0x55e299d1fbe0 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07a1ffb700 1 -- 172.21.15.161:0/201831698 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a80962a0 con 0x55e299d1fbe0 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 10 client.26719 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e299a9e2b0 con 0x55e299d1fbe0 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07a1ffb700 1 -- 172.21.15.161:0/201831698 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a8099aa0 con 0x55e299d1fbe0 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07a1ffb700 1 -- 172.21.15.161:0/201831698 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f07a807e020 con 0x55e299d1fbe0 2023-07-14T11:27:37.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 10 client.26719 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:37.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 10 client.26719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:27:37.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 20 client.26719 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:37.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d8eb70 0x55e299d90f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 4 client.26719 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:27:37.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e299c2d050 con 0x55e299d8eb70 2023-07-14T11:27:37.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.596+0000 7f07be19b700 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d8eb70 0x55e299d90f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07be19b700 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d8eb70 0x55e299d90f50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8817e2228fa15c9d server_cookie=210b3690ca2a5c76 in_seq=0 out_seq=0 2023-07-14T11:27:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07a1ffb700 10 client.26719 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07a1ffb700 1 -- 172.21.15.161:0/201831698 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e299c2d050 con 0x55e299d8eb70 2023-07-14T11:27:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07a1ffb700 10 client.26719 handle_command_reply: tid=0 2023-07-14T11:27:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 client.26719 shutdown 2023-07-14T11:27:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 msgr2=0x55e299d31410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d31410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07a8099c10 tx=0x7f07a80961e0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 shutdown_connections 2023-07-14T11:27:37.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55e299d8eb70 0x55e299d90f50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e299d1f2d0 0x55e299d30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e299d2aad0 0x55e299d2d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 --2- 172.21.15.161:0/201831698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e299d1fbe0 0x55e299d31410 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07a1ffb700 0 client.26719 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:27:37.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 >> 172.21.15.161:0/201831698 conn(0x55e299d15da0 msgr2=0x55e299d19e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.597+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 shutdown_connections 2023-07-14T11:27:37.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07d3c7ab80 1 -- 172.21.15.161:0/201831698 wait complete. 2023-07-14T11:27:37.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07d3c7ab80 20 client.26719 trim_cache size 0 max 16384 2023-07-14T11:27:37.609 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:27:37.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f07a40a1340 msgr2=0x7f07a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f07a40a1340 0x7f07a40a37f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07b0003ac0 tx=0x7f07b0009670 comp rx=0 tx=0).stop 2023-07-14T11:27:37.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b80feb90 msgr2=0x7f07b8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b80feb90 0x7f07b8198440 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f07a800e9f0 tx=0x7f07a800ed00 comp rx=0 tx=0).stop 2023-07-14T11:27:37.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 shutdown_connections 2023-07-14T11:27:37.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f07a40a1340 0x7f07a40a37f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07b8105c00 0x7f07b819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07b80fe200 0x7f07b8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 --2- 172.21.15.161:0/3220110719 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f07b80feb90 0x7f07b8198440 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.598+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 >> 172.21.15.161:0/3220110719 conn(0x7f07b8076200 msgr2=0x7f07b81016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.599+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 shutdown_connections 2023-07-14T11:27:37.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.599+0000 7f07c03ff700 1 -- 172.21.15.161:0/3220110719 wait complete. 2023-07-14T11:27:37.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:27:37.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:37 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3560168866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:27:37.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.854+0000 7f49e59ff700 1 -- 172.21.15.161:0/3766042002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 msgr2=0x7f49e010c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.854+0000 7f49e59ff700 1 --2- 172.21.15.161:0/3766042002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 0x7f49e010c4e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009f90 tx=0x7f49d800b330 comp rx=0 tx=0).stop 2023-07-14T11:27:37.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.855+0000 7f49e59ff700 1 -- 172.21.15.161:0/3766042002 shutdown_connections 2023-07-14T11:27:37.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.855+0000 7f49e59ff700 1 --2- 172.21.15.161:0/3766042002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 0x7f49e010c4e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.855+0000 7f49e59ff700 1 --2- 172.21.15.161:0/3766042002 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e0100cd0 0x7f49e01010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.855+0000 7f49e59ff700 1 --2- 172.21.15.161:0/3766042002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e01015e0 0x7f49e0109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.855+0000 7f49e59ff700 1 -- 172.21.15.161:0/3766042002 >> 172.21.15.161:0/3766042002 conn(0x7f49e00fa4e0 msgr2=0x7f49e00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.855+0000 7f49e59ff700 1 -- 172.21.15.161:0/3766042002 shutdown_connections 2023-07-14T11:27:37.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.855+0000 7f49e59ff700 1 -- 172.21.15.161:0/3766042002 wait complete. 2023-07-14T11:27:37.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.856+0000 7f49e59ff700 1 Processor -- start 2023-07-14T11:27:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.856+0000 7f49e59ff700 1 -- start start 2023-07-14T11:27:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49e59ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e0100cd0 0x7f49e00fe920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49e59ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e01015e0 0x7f49e00fee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49e59ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 0x7f49e01a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49e59ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00ff690 con 0x7f49e010a0a0 2023-07-14T11:27:37.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49e59ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01a4520 con 0x7f49e0100cd0 2023-07-14T11:27:37.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49e59ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01a4660 con 0x7f49e01015e0 2023-07-14T11:27:37.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49de7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e01015e0 0x7f49e00fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49deffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e0100cd0 0x7f49e00fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49deffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e0100cd0 0x7f49e00fe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48734/0 (socket says 172.21.15.161:48734) 2023-07-14T11:27:37.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49de7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e01015e0 0x7f49e00fee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45316/0 (socket says 172.21.15.161:45316) 2023-07-14T11:27:37.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49deffd700 1 -- 172.21.15.161:0/2789187593 learned_addr learned my addr 172.21.15.161:0/2789187593 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.857+0000 7f49df7fe700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 0x7f49e01a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49de7fc700 1 -- 172.21.15.161:0/2789187593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e0100cd0 msgr2=0x7f49e00fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49de7fc700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e0100cd0 0x7f49e00fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49de7fc700 1 -- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 msgr2=0x7f49e01a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49de7fc700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 0x7f49e01a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49de7fc700 1 -- 172.21.15.161:0/2789187593 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40095f0 con 0x7f49e01015e0 2023-07-14T11:27:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49df7fe700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 0x7f49e01a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49de7fc700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e01015e0 0x7f49e00fee60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f49d400eb90 tx=0x7f49d400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49e49fd700 1 -- 172.21.15.161:0/2789187593 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400ccf0 con 0x7f49e01015e0 2023-07-14T11:27:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d8009c40 con 0x7f49e01015e0 2023-07-14T11:27:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49e49fd700 1 -- 172.21.15.161:0/2789187593 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d400ce50 con 0x7f49e01015e0 2023-07-14T11:27:37.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49e49fd700 1 -- 172.21.15.161:0/2789187593 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4010430 con 0x7f49e01015e0 2023-07-14T11:27:37.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.858+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e01a4c10 con 0x7f49e01015e0 2023-07-14T11:27:37.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.862+0000 7f49e49fd700 1 -- 172.21.15.161:0/2789187593 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f49d4003680 con 0x7f49e01015e0 2023-07-14T11:27:37.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.863+0000 7f49e49fd700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f49c80a12c0 0x7f49c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.863+0000 7f49f933ab80 1 Processor -- start 2023-07-14T11:27:37.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49e49fd700 1 -- 172.21.15.161:0/2789187593 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f49d4014070 con 0x7f49e01015e0 2023-07-14T11:27:37.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49f933ab80 1 -- start start 2023-07-14T11:27:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49deffd700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f49c80a12c0 0x7f49c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49f933ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c29f3cb0 con 0x5572c2aed2d0 2023-07-14T11:27:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49f933ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2a46a40 con 0x5572c2af8ad0 2023-07-14T11:27:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.864+0000 7f49f933ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2a3fed0 con 0x5572c2aedbe0 2023-07-14T11:27:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49de7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48736/0 (socket says 172.21.15.161:48736) 2023-07-14T11:27:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 -- 172.21.15.161:0/3008309637 learned_addr learned my addr 172.21.15.161:0/3008309637 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49deffd700 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 -- 172.21.15.161:0/3008309637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 msgr2=0x5572c2af8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49deffd700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f49c80a12c0 0x7f49c80a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d70 tx=0x7f49d0011720 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 -- 172.21.15.161:0/3008309637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 msgr2=0x5572c2aed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 -- 172.21.15.161:0/3008309637 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c286dde0 con 0x5572c2af8ad0 2023-07-14T11:27:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49de7fc700 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:37.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49df7fe700 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afaeb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49d80093d0 tx=0x7f49d8009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2df5407e48e61282 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49c5ffb700 1 -- 172.21.15.161:0/3008309637 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800e070 con 0x5572c2af8ad0 2023-07-14T11:27:37.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49c5ffb700 1 -- 172.21.15.161:0/3008309637 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d8007bd0 con 0x5572c2af8ad0 2023-07-14T11:27:37.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.865+0000 7f49c5ffb700 1 -- 172.21.15.161:0/3008309637 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800bbf0 con 0x5572c2af8ad0 2023-07-14T11:27:37.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 -- 172.21.15.161:0/3008309637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 msgr2=0x5572c2afaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afaeb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49d80093d0 tx=0x7f49d8009400 comp rx=0 tx=0).stop 2023-07-14T11:27:37.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 -- 172.21.15.161:0/3008309637 shutdown_connections 2023-07-14T11:27:37.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afaeb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3008309637 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 -- 172.21.15.161:0/3008309637 >> 172.21.15.161:0/3008309637 conn(0x5572c2ae3da0 msgr2=0x5572c2ae96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 -- 172.21.15.161:0/3008309637 shutdown_connections 2023-07-14T11:27:37.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.866+0000 7f49f933ab80 1 -- 172.21.15.161:0/3008309637 wait complete. 2023-07-14T11:27:37.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.867+0000 7f49f933ab80 1 -- start start 2023-07-14T11:27:37.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.867+0000 7f49f933ab80 1 Processor -- start 2023-07-14T11:27:37.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.867+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2af8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aedbe0 0x5572c2af5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2af8ad0 0x5572c2af5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49f933ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c29dfac0 con 0x5572c2aed2d0 2023-07-14T11:27:37.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49f933ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c29df7c0 con 0x5572c2aedbe0 2023-07-14T11:27:37.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49f933ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c28c3920 con 0x5572c2af8ad0 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49f933ab80 10 client.0 _ll_register_callbacks cb 0x5572c29dd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49deffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2af8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49deffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2af8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57212/0 (socket says 172.21.15.161:57212) 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49deffd700 1 -- 172.21.15.161:0/4234312113 learned_addr learned my addr 172.21.15.161:0/4234312113 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49de7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aedbe0 0x5572c2af5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49df7fe700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2af8ad0 0x5572c2af5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49de7fc700 1 -- 172.21.15.161:0/4234312113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2af8ad0 msgr2=0x5572c2af5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49de7fc700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2af8ad0 0x5572c2af5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49de7fc700 1 -- 172.21.15.161:0/4234312113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 msgr2=0x5572c2af8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49de7fc700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2af8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49de7fc700 1 -- 172.21.15.161:0/4234312113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e01a4c10 con 0x5572c2aedbe0 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49df7fe700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2af8ad0 0x5572c2af5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.868+0000 7f49deffd700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2af8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49de7fc700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aedbe0 0x5572c2af5740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49d40ca8d0 tx=0x7f49d40163b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e242ecaaa6776302 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49beffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:37.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49beffd700 1 -- 172.21.15.161:0/4234312113 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d409a6b0 con 0x5572c2aedbe0 2023-07-14T11:27:37.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49beffd700 1 -- 172.21.15.161:0/4234312113 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f49d409e280 con 0x5572c2aedbe0 2023-07-14T11:27:37.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572c29994f0 con 0x5572c2aedbe0 2023-07-14T11:27:37.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49beffd700 1 -- 172.21.15.161:0/4234312113 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d40a26a0 con 0x5572c2aedbe0 2023-07-14T11:27:37.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49f933ab80 10 client.26729 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:37.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f49e010d1a0 con 0x5572c2aedbe0 2023-07-14T11:27:37.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.869+0000 7f49beffd700 1 -- 172.21.15.161:0/4234312113 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d40c8080 con 0x5572c2aedbe0 2023-07-14T11:27:37.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49beffd700 1 -- 172.21.15.161:0/4234312113 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49d40a22a0 con 0x5572c2aedbe0 2023-07-14T11:27:37.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49f933ab80 10 client.26729 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:37.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49f933ab80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:37.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49f933ab80 20 client.26729 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:37.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49f933ab80 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b08330 0x5572c2b0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49f933ab80 4 client.26729 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:37.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572c2a38eb0 con 0x5572c2b08330 2023-07-14T11:27:37.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.870+0000 7f49deffd700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b08330 0x5572c2b0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.871+0000 7f49beffd700 10 client.26729 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:37.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.871+0000 7f49deffd700 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b08330 0x5572c2b0a710 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfef00fb689a1f6 server_cookie=3cecc454c4e9015d in_seq=0 out_seq=0 2023-07-14T11:27:37.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.871+0000 7f49beffd700 1 -- 172.21.15.161:0/4234312113 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5572c2a38eb0 con 0x5572c2b08330 2023-07-14T11:27:37.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.871+0000 7f49beffd700 10 client.26729 handle_command_reply: tid=0 2023-07-14T11:27:37.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.871+0000 7f49f933ab80 1 client.26729 shutdown 2023-07-14T11:27:37.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aedbe0 msgr2=0x5572c2af5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aedbe0 0x5572c2af5740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49d40ca8d0 tx=0x7f49d40163b0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 shutdown_connections 2023-07-14T11:27:37.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b08330 0x5572c2b0a710 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2af8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2af8ad0 0x5572c2af5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 --2- 172.21.15.161:0/4234312113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aedbe0 0x5572c2af5740 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49beffd700 0 client.26729 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:37.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 >> 172.21.15.161:0/4234312113 conn(0x5572c2ae3da0 msgr2=0x5572c2ae9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 shutdown_connections 2023-07-14T11:27:37.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 1 -- 172.21.15.161:0/4234312113 wait complete. 2023-07-14T11:27:37.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.872+0000 7f49f933ab80 20 client.26729 trim_cache size 0 max 16384 2023-07-14T11:27:37.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 Processor -- start 2023-07-14T11:27:37.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 -- start start 2023-07-14T11:27:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aed2d0 0x5572c2af7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2af8ad0 0x5572c2af5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2ae8550 con 0x5572c2af8ad0 2023-07-14T11:27:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2aea2a0 con 0x5572c2aed2d0 2023-07-14T11:27:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49f933ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2af5980 con 0x5572c2aedbe0 2023-07-14T11:27:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49deffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aed2d0 0x5572c2af7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49de7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49deffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aed2d0 0x5572c2af7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48742/0 (socket says 172.21.15.161:48742) 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49deffd700 1 -- 172.21.15.161:0/2546042004 learned_addr learned my addr 172.21.15.161:0/2546042004 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.887+0000 7f49df7fe700 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2af8ad0 0x5572c2af5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49deffd700 1 -- 172.21.15.161:0/2546042004 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 msgr2=0x5572c2af4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49deffd700 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49deffd700 1 -- 172.21.15.161:0/2546042004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2af8ad0 msgr2=0x5572c2af5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49deffd700 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2af8ad0 0x5572c2af5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49deffd700 1 -- 172.21.15.161:0/2546042004 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c2856c60 con 0x5572c2aed2d0 2023-07-14T11:27:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49df7fe700 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2af8ad0 0x5572c2af5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49de7fc700 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49deffd700 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aed2d0 0x5572c2af7ff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49d00046b0 tx=0x7f49d001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d1a32a571955bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49c67fc700 1 -- 172.21.15.161:0/2546042004 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0023040 con 0x5572c2aed2d0 2023-07-14T11:27:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49c67fc700 1 -- 172.21.15.161:0/2546042004 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d0025040 con 0x5572c2aed2d0 2023-07-14T11:27:37.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.888+0000 7f49c67fc700 1 -- 172.21.15.161:0/2546042004 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d000f620 con 0x5572c2aed2d0 2023-07-14T11:27:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 -- 172.21.15.161:0/2546042004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aed2d0 msgr2=0x5572c2af7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aed2d0 0x5572c2af7ff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49d00046b0 tx=0x7f49d001b000 comp rx=0 tx=0).stop 2023-07-14T11:27:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 -- 172.21.15.161:0/2546042004 shutdown_connections 2023-07-14T11:27:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2af8ad0 0x5572c2af5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2aed2d0 0x5572c2af7ff0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 --2- 172.21.15.161:0/2546042004 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2af4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 -- 172.21.15.161:0/2546042004 >> 172.21.15.161:0/2546042004 conn(0x5572c2ae3da0 msgr2=0x5572c2af0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 -- 172.21.15.161:0/2546042004 shutdown_connections 2023-07-14T11:27:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 -- 172.21.15.161:0/2546042004 wait complete. 2023-07-14T11:27:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.889+0000 7f49f933ab80 1 -- start start 2023-07-14T11:27:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 1 Processor -- start 2023-07-14T11:27:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2afd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2affae0 con 0x5572c2aed2d0 2023-07-14T11:27:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2afdfb0 con 0x5572c2af8ad0 2023-07-14T11:27:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c2afe170 con 0x5572c2aedbe0 2023-07-14T11:27:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49f933ab80 10 client.0 _ll_register_callbacks cb 0x7f49d4099bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49de7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2afd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49deffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49de7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2afd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45356/0 (socket says 172.21.15.161:45356) 2023-07-14T11:27:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49df7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.890+0000 7f49de7fc700 1 -- 172.21.15.161:0/3623329361 learned_addr learned my addr 172.21.15.161:0/3623329361 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49de7fc700 1 -- 172.21.15.161:0/3623329361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 msgr2=0x5572c2afda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49de7fc700 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49de7fc700 1 -- 172.21.15.161:0/3623329361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 msgr2=0x5572c2aff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49de7fc700 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49de7fc700 1 -- 172.21.15.161:0/3623329361 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e010d1a0 con 0x5572c2aedbe0 2023-07-14T11:27:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49de7fc700 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2afd500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5572c2aea4a0 tx=0x7f49d4009a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bb17aea533cdf1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:37.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49beffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49beffd700 1 -- 172.21.15.161:0/3623329361 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40cd380 con 0x5572c2aedbe0 2023-07-14T11:27:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49beffd700 1 -- 172.21.15.161:0/3623329361 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f49d409e280 con 0x5572c2aedbe0 2023-07-14T11:27:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49d8007bd0 con 0x5572c2aedbe0 2023-07-14T11:27:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49beffd700 1 -- 172.21.15.161:0/3623329361 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d4099440 con 0x5572c2aedbe0 2023-07-14T11:27:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49f933ab80 10 client.26737 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.891+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f49d001e1d0 con 0x5572c2aedbe0 2023-07-14T11:27:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49beffd700 1 -- 172.21.15.161:0/3623329361 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d8007bd0 con 0x5572c2aedbe0 2023-07-14T11:27:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49beffd700 1 -- 172.21.15.161:0/3623329361 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49d40b8330 con 0x5572c2aedbe0 2023-07-14T11:27:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49f933ab80 10 client.26737 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49f933ab80 10 client.26737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49f933ab80 20 client.26737 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b5d410 0x5572c2b5f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49f933ab80 4 client.26737 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:27:37.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572c29fb050 con 0x5572c2b5d410 2023-07-14T11:27:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.892+0000 7f49deffd700 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b5d410 0x5572c2b5f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49deffd700 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b5d410 0x5572c2b5f7f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f69862f542bdc93 server_cookie=1f34ec811a93a4 in_seq=0 out_seq=0 2023-07-14T11:27:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49beffd700 10 client.26737 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49beffd700 1 -- 172.21.15.161:0/3623329361 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572c29fb050 con 0x5572c2b5d410 2023-07-14T11:27:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49beffd700 10 client.26737 handle_command_reply: tid=0 2023-07-14T11:27:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 client.26737 shutdown 2023-07-14T11:27:37.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 msgr2=0x5572c2afd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2afd500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5572c2aea4a0 tx=0x7f49d4009a30 comp rx=0 tx=0).stop 2023-07-14T11:27:37.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 shutdown_connections 2023-07-14T11:27:37.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5572c2b5d410 0x5572c2b5f7f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572c2aed2d0 0x5572c2aff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572c2af8ad0 0x5572c2afda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 --2- 172.21.15.161:0/3623329361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5572c2aedbe0 0x5572c2afd500 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49beffd700 0 client.26737 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:37.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 >> 172.21.15.161:0/3623329361 conn(0x5572c2ae3da0 msgr2=0x5572c2ae88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 shutdown_connections 2023-07-14T11:27:37.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 1 -- 172.21.15.161:0/3623329361 wait complete. 2023-07-14T11:27:37.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.893+0000 7f49f933ab80 20 client.26737 trim_cache size 0 max 16384 2023-07-14T11:27:37.907 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:27:37.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f49c80a12c0 msgr2=0x7f49c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f49c80a12c0 0x7f49c80a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d70 tx=0x7f49d0011720 comp rx=0 tx=0).stop 2023-07-14T11:27:37.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e01015e0 msgr2=0x7f49e00fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:37.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e01015e0 0x7f49e00fee60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f49d400eb90 tx=0x7f49d400eea0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 shutdown_connections 2023-07-14T11:27:37.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f49c80a12c0 0x7f49c80a3770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49e010a0a0 0x7f49e01a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e0100cd0 0x7f49e00fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 --2- 172.21.15.161:0/2789187593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f49e01015e0 0x7f49e00fee60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:37.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 >> 172.21.15.161:0/2789187593 conn(0x7f49e00fa4e0 msgr2=0x7f49e0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:37.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 shutdown_connections 2023-07-14T11:27:37.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:37.894+0000 7f49e59ff700 1 -- 172.21.15.161:0/2789187593 wait complete. 2023-07-14T11:27:38.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:38 smithi161 ceph-mon[95526]: pgmap v164: 129 pgs: 129 active+clean; 290 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2023-07-14T11:27:38.183 INFO:tasks.workunit.client.0.smithi161.stdout: 1 481360 9.85 MB/sec execute 140 sec latency 50.088 ms 2023-07-14T11:27:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:38 smithi182 ceph-mon[129366]: pgmap v164: 129 pgs: 129 active+clean; 290 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2023-07-14T11:27:38.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:38 smithi175 ceph-mon[129081]: pgmap v164: 129 pgs: 129 active+clean; 290 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2023-07-14T11:27:38.917 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:27:38.917 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:27:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.134+0000 7f6e31bff700 1 -- 172.21.15.161:0/1046319038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 msgr2=0x7f6e2c109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.134+0000 7f6e31bff700 1 --2- 172.21.15.161:0/1046319038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 0x7f6e2c109b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009990 tx=0x7f6e20009ca0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.134+0000 7f6e31bff700 1 -- 172.21.15.161:0/1046319038 shutdown_connections 2023-07-14T11:27:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.134+0000 7f6e31bff700 1 --2- 172.21.15.161:0/1046319038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 0x7f6e2c100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.134+0000 7f6e31bff700 1 --2- 172.21.15.161:0/1046319038 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6e2c10a0a0 0x7f6e2c10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.134+0000 7f6e31bff700 1 --2- 172.21.15.161:0/1046319038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 0x7f6e2c109b60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.134+0000 7f6e31bff700 1 -- 172.21.15.161:0/1046319038 >> 172.21.15.161:0/1046319038 conn(0x7f6e2c0fa4b0 msgr2=0x7f6e2c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.135+0000 7f6e31bff700 1 -- 172.21.15.161:0/1046319038 shutdown_connections 2023-07-14T11:27:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.135+0000 7f6e31bff700 1 -- 172.21.15.161:0/1046319038 wait complete. 2023-07-14T11:27:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.136+0000 7f6e31bff700 1 Processor -- start 2023-07-14T11:27:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.136+0000 7f6e31bff700 1 -- start start 2023-07-14T11:27:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e31bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 0x7f6e2c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e31bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 0x7f6e2c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e31bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6e2c10a0a0 0x7f6e2c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e31bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c1960c0 con 0x7f6e2c100b80 2023-07-14T11:27:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e31bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c196230 con 0x7f6e2c101490 2023-07-14T11:27:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e31bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c1963f0 con 0x7f6e2c10a0a0 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 0x7f6e2c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 0x7f6e2c19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57250/0 (socket says 172.21.15.161:57250) 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/3853125274 learned_addr learned my addr 172.21.15.161:0/3853125274 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6e2c10a0a0 0x7f6e2c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 0x7f6e2c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/3853125274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6e2c10a0a0 msgr2=0x7f6e2c195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6e2c10a0a0 0x7f6e2c195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/3853125274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 msgr2=0x7f6e2c19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 0x7f6e2c19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/3853125274 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c009520 con 0x7f6e2c100b80 2023-07-14T11:27:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.137+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6e2c10a0a0 0x7f6e2c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.138+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 0x7f6e2c19b9e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00eb30 tx=0x7f6e1c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.138+0000 7f6e28ff9700 1 -- 172.21.15.161:0/3853125274 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c009880 con 0x7f6e2c100b80 2023-07-14T11:27:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.138+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e200095f0 con 0x7f6e2c100b80 2023-07-14T11:27:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.138+0000 7f6e28ff9700 1 -- 172.21.15.161:0/3853125274 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e1c0044e0 con 0x7f6e2c100b80 2023-07-14T11:27:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.138+0000 7f6e28ff9700 1 -- 172.21.15.161:0/3853125274 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c003d80 con 0x7f6e2c100b80 2023-07-14T11:27:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.138+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c0feac0 con 0x7f6e2c100b80 2023-07-14T11:27:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.141+0000 7f6e28ff9700 1 -- 172.21.15.161:0/3853125274 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6e1c003f20 con 0x7f6e2c100b80 2023-07-14T11:27:39.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.142+0000 7f6e45548b80 1 Processor -- start 2023-07-14T11:27:39.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e45548b80 1 -- start start 2023-07-14T11:27:39.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e28ff9700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6e140a12c0 0x7f6e140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3350 0x561d59ae3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59aee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e28ff9700 1 -- 172.21.15.161:0/3853125274 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f6e1c014070 con 0x7f6e2c100b80 2023-07-14T11:27:39.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e45548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d599e9d30 con 0x561d59aeeb50 2023-07-14T11:27:39.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e45548b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59a3cac0 con 0x561d59ae3c60 2023-07-14T11:27:39.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e45548b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59a35f50 con 0x561d59ae3350 2023-07-14T11:27:39.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6e140a12c0 0x7f6e140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59aee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59aee610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48774/0 (socket says 172.21.15.161:48774) 2023-07-14T11:27:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e2affd700 1 -- 172.21.15.161:0/2080329041 learned_addr learned my addr 172.21.15.161:0/2080329041 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3350 0x561d59ae3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.143+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2affd700 1 -- 172.21.15.161:0/2080329041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3350 msgr2=0x561d59ae3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2affd700 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3350 0x561d59ae3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2affd700 1 -- 172.21.15.161:0/2080329041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 msgr2=0x561d59af0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2affd700 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2affd700 1 -- 172.21.15.161:0/2080329041 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d59863e60 con 0x561d59ae3c60 2023-07-14T11:27:39.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3350 0x561d59ae3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:39.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:39.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2affd700 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59aee610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6e20015000 tx=0x7f6e20005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7471f1d5f7de037 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6e140a12c0 0x7f6e140a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009ec0 tx=0x7f6e24009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e11ffb700 1 -- 172.21.15.161:0/2080329041 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20021050 con 0x561d59ae3c60 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e11ffb700 1 -- 172.21.15.161:0/2080329041 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e20022040 con 0x561d59ae3c60 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.144+0000 7f6e11ffb700 1 -- 172.21.15.161:0/2080329041 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20005430 con 0x561d59ae3c60 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 -- 172.21.15.161:0/2080329041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 msgr2=0x561d59aee610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59aee610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6e20015000 tx=0x7f6e20005ea0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 -- 172.21.15.161:0/2080329041 shutdown_connections 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3350 0x561d59ae3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2080329041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59aee610 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 -- 172.21.15.161:0/2080329041 >> 172.21.15.161:0/2080329041 conn(0x561d59ad9e20 msgr2=0x561d59adf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 -- 172.21.15.161:0/2080329041 shutdown_connections 2023-07-14T11:27:39.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.145+0000 7f6e45548b80 1 -- 172.21.15.161:0/2080329041 wait complete. 2023-07-14T11:27:39.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.146+0000 7f6e45548b80 1 -- start start 2023-07-14T11:27:39.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.146+0000 7f6e45548b80 1 Processor -- start 2023-07-14T11:27:39.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59aee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59aeb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59aebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e45548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d599d5b40 con 0x561d59aeeb50 2023-07-14T11:27:39.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e45548b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d599d5840 con 0x561d59ae3350 2023-07-14T11:27:39.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e45548b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d598b99a0 con 0x561d59ae3c60 2023-07-14T11:27:39.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e45548b80 10 client.0 _ll_register_callbacks cb 0x561d599d37e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:39.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59aeb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59aee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59aeb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45396/0 (socket says 172.21.15.161:45396) 2023-07-14T11:27:39.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2affd700 1 -- 172.21.15.161:0/2163057705 learned_addr learned my addr 172.21.15.161:0/2163057705 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59aebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/2163057705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 msgr2=0x561d59aeb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59aeb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/2163057705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 msgr2=0x561d59aebf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.147+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59aebf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/2163057705 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c0feac0 con 0x561d59ae3350 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59aee590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0ca9a0 tx=0x7f6e1c099d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbe059b21cd4418 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e0affd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:39.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e0affd700 1 -- 172.21.15.161:0/2163057705 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0cd370 con 0x561d59ae3350 2023-07-14T11:27:39.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e0affd700 1 -- 172.21.15.161:0/2163057705 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f6e1c09a320 con 0x561d59ae3350 2023-07-14T11:27:39.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d5998f570 con 0x561d59ae3350 2023-07-14T11:27:39.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e0affd700 1 -- 172.21.15.161:0/2163057705 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1c09e160 con 0x561d59ae3350 2023-07-14T11:27:39.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e45548b80 10 client.26747 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:39.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f6e2c10d180 con 0x561d59ae3350 2023-07-14T11:27:39.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.148+0000 7f6e0affd700 1 -- 172.21.15.161:0/2163057705 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d5998f570 con 0x561d59ae3350 2023-07-14T11:27:39.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e0affd700 1 -- 172.21.15.161:0/2163057705 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6e1c099810 con 0x561d59ae3350 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e45548b80 10 client.26747 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e45548b80 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e45548b80 20 client.26747 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59afe280 0x561d59b00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e45548b80 4 client.26747 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d59a2ef30 con 0x561d59afe280 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e2affd700 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59afe280 0x561d59b00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e0affd700 10 client.26747 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.149+0000 7f6e2affd700 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59afe280 0x561d59b00690 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccfd388dd52665ce server_cookie=741beb9359a0527b in_seq=0 out_seq=0 2023-07-14T11:27:39.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e0affd700 1 -- 172.21.15.161:0/2163057705 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561d59a2ef30 con 0x561d59afe280 2023-07-14T11:27:39.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e0affd700 10 client.26747 handle_command_reply: tid=0 2023-07-14T11:27:39.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 client.26747 shutdown 2023-07-14T11:27:39.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 msgr2=0x561d59aee590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59aee590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0ca9a0 tx=0x7f6e1c099d00 comp rx=0 tx=0).stop 2023-07-14T11:27:39.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 shutdown_connections 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59afe280 0x561d59b00690 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59aebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e0affd700 0 client.26747 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59aee590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 --2- 172.21.15.161:0/2163057705 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59aeb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.150+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 >> 172.21.15.161:0/2163057705 conn(0x561d59ad9e20 msgr2=0x561d59adf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.151+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 shutdown_connections 2023-07-14T11:27:39.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.151+0000 7f6e45548b80 1 -- 172.21.15.161:0/2163057705 wait complete. 2023-07-14T11:27:39.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.151+0000 7f6e45548b80 20 client.26747 trim_cache size 0 max 16384 2023-07-14T11:27:39.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 Processor -- start 2023-07-14T11:27:39.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 -- start start 2023-07-14T11:27:39.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59ae3350 0x561d59adf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59ae6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59aeeb50 0x561d59af5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59aec7c0 con 0x561d59ae3350 2023-07-14T11:27:39.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59ae45c0 con 0x561d59ae3c60 2023-07-14T11:27:39.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e45548b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59afe090 con 0x561d59aeeb50 2023-07-14T11:27:39.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59ae3350 0x561d59adf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59ae6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59ae3350 0x561d59adf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57286/0 (socket says 172.21.15.161:57286) 2023-07-14T11:27:39.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/1415950806 learned_addr learned my addr 172.21.15.161:0/1415950806 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.170+0000 7f6e2bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59aeeb50 0x561d59af5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e2affd700 1 -- 172.21.15.161:0/1415950806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59aeeb50 msgr2=0x561d59af5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e2affd700 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59aeeb50 0x561d59af5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e2affd700 1 -- 172.21.15.161:0/1415950806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59ae3350 msgr2=0x561d59adf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e2affd700 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59ae3350 0x561d59adf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e2affd700 1 -- 172.21.15.161:0/1415950806 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d5984cce0 con 0x561d59ae3c60 2023-07-14T11:27:39.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59ae3350 0x561d59adf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:39.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e2affd700 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59ae6230 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005920 tx=0x7f6e2001e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc55bf6f91c318e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e127fc700 1 -- 172.21.15.161:0/1415950806 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20039570 con 0x561d59ae3c60 2023-07-14T11:27:39.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e127fc700 1 -- 172.21.15.161:0/1415950806 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e20036070 con 0x561d59ae3c60 2023-07-14T11:27:39.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.171+0000 7f6e127fc700 1 -- 172.21.15.161:0/1415950806 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20039e90 con 0x561d59ae3c60 2023-07-14T11:27:39.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 -- 172.21.15.161:0/1415950806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 msgr2=0x561d59ae6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59ae6230 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005920 tx=0x7f6e2001e040 comp rx=0 tx=0).stop 2023-07-14T11:27:39.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 -- 172.21.15.161:0/1415950806 shutdown_connections 2023-07-14T11:27:39.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59ae3350 0x561d59adf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59aeeb50 0x561d59af5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 --2- 172.21.15.161:0/1415950806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3c60 0x561d59ae6230 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 -- 172.21.15.161:0/1415950806 >> 172.21.15.161:0/1415950806 conn(0x561d59ad9e20 msgr2=0x561d59adf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 -- 172.21.15.161:0/1415950806 shutdown_connections 2023-07-14T11:27:39.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 -- 172.21.15.161:0/1415950806 wait complete. 2023-07-14T11:27:39.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.172+0000 7f6e45548b80 1 -- start start 2023-07-14T11:27:39.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 1 Processor -- start 2023-07-14T11:27:39.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59b4b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59b4be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59af4f80 con 0x561d59aeeb50 2023-07-14T11:27:39.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59af50f0 con 0x561d59ae3350 2023-07-14T11:27:39.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59af52b0 con 0x561d59ae3c60 2023-07-14T11:27:39.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e45548b80 10 client.0 _ll_register_callbacks cb 0x7f6e1c099e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:39.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59b4b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59b4b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48816/0 (socket says 172.21.15.161:48816) 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/862155592 learned_addr learned my addr 172.21.15.161:0/862155592 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e2bfff700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.173+0000 7f6e2affd700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59b4be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/862155592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 msgr2=0x561d59b4be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59b4be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/862155592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 msgr2=0x561d59af49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e2b7fe700 1 -- 172.21.15.161:0/862155592 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c10d180 con 0x561d59ae3350 2023-07-14T11:27:39.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e2affd700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59b4be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:39.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e2b7fe700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59b4b940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c096d90 tx=0x7f6e1c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b2aa4d4acda65e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e0affd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:27:39.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e0affd700 1 -- 172.21.15.161:0/862155592 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c09a790 con 0x561d59ae3350 2023-07-14T11:27:39.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e20022040 con 0x561d59ae3350 2023-07-14T11:27:39.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e0affd700 1 -- 172.21.15.161:0/862155592 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f6e1c061080 con 0x561d59ae3350 2023-07-14T11:27:39.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e0affd700 1 -- 172.21.15.161:0/862155592 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1c09e160 con 0x561d59ae3350 2023-07-14T11:27:39.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e45548b80 10 client.26753 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:39.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.174+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561d59863e60 con 0x561d59ae3350 2023-07-14T11:27:39.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e0affd700 1 -- 172.21.15.161:0/862155592 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e20022040 con 0x561d59ae3350 2023-07-14T11:27:39.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e0affd700 1 -- 172.21.15.161:0/862155592 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6e1c0c4b50 con 0x561d59ae3350 2023-07-14T11:27:39.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e45548b80 10 client.26753 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:39.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e45548b80 10 client.26753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e45548b80 20 client.26753 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e45548b80 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59b545b0 0x561d59b56990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e45548b80 4 client.26753 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561d599f10d0 con 0x561d59b545b0 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.175+0000 7f6e2affd700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59b545b0 0x561d59b56990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.176+0000 7f6e2affd700 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59b545b0 0x561d59b56990 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a103c3e47856fe41 server_cookie=2de9078b04390192 in_seq=0 out_seq=0 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.176+0000 7f6e0affd700 10 client.26753 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e0affd700 1 -- 172.21.15.161:0/862155592 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561d599f10d0 con 0x561d59b545b0 2023-07-14T11:27:39.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e0affd700 10 client.26753 handle_command_reply: tid=0 2023-07-14T11:27:39.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 client.26753 shutdown 2023-07-14T11:27:39.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 msgr2=0x561d59b4b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59b4b940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c096d90 tx=0x7f6e1c003680 comp rx=0 tx=0).stop 2023-07-14T11:27:39.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 shutdown_connections 2023-07-14T11:27:39.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561d59b545b0 0x561d59b56990 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d59aeeb50 0x561d59af49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d59ae3350 0x561d59b4b940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 --2- 172.21.15.161:0/862155592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561d59ae3c60 0x561d59b4be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 >> 172.21.15.161:0/862155592 conn(0x561d59ad9e20 msgr2=0x561d59ae5220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e0affd700 0 client.26753 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 shutdown_connections 2023-07-14T11:27:39.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 1 -- 172.21.15.161:0/862155592 wait complete. 2023-07-14T11:27:39.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.177+0000 7f6e45548b80 20 client.26753 trim_cache size 0 max 16384 2023-07-14T11:27:39.191 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:27:39.192 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:27:39.192 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:27:39.192 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:27:39.192 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:27:39.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6e140a12c0 msgr2=0x7f6e140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6e140a12c0 0x7f6e140a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009ec0 tx=0x7f6e24009920 comp rx=0 tx=0).stop 2023-07-14T11:27:39.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 msgr2=0x7f6e2c19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 0x7f6e2c19b9e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00eb30 tx=0x7f6e1c00ee40 comp rx=0 tx=0).stop 2023-07-14T11:27:39.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 shutdown_connections 2023-07-14T11:27:39.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6e140a12c0 0x7f6e140a3770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c100b80 0x7f6e2c19b9e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6e2c10a0a0 0x7f6e2c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 --2- 172.21.15.161:0/3853125274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101490 0x7f6e2c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 >> 172.21.15.161:0/3853125274 conn(0x7f6e2c0fa4b0 msgr2=0x7f6e2c10c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.178+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 shutdown_connections 2023-07-14T11:27:39.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.179+0000 7f6e31bff700 1 -- 172.21.15.161:0/3853125274 wait complete. 2023-07-14T11:27:39.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 483586 9.88 MB/sec execute 141 sec latency 28.735 ms 2023-07-14T11:27:39.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:27:39.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.448+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/240876249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c64105ae0 msgr2=0x7f6c64105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.448+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/240876249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c64105ae0 0x7f6c64105eb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6c600098f0 tx=0x7f6c60009c00 comp rx=0 tx=0).stop 2023-07-14T11:27:39.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.449+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/240876249 shutdown_connections 2023-07-14T11:27:39.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.449+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/240876249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c64105ae0 0x7f6c64105eb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.449+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/240876249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c64100480 0x7f6c6410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.449+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/240876249 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c640ffad0 0x7f6c640fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.449+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/240876249 >> 172.21.15.161:0/240876249 conn(0x7f6c640fb5f0 msgr2=0x7f6c640fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.449+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/240876249 shutdown_connections 2023-07-14T11:27:39.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.449+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/240876249 wait complete. 2023-07-14T11:27:39.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 Processor -- start 2023-07-14T11:27:39.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 -- start start 2023-07-14T11:27:39.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c640ffad0 0x7f6c64072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c64100480 0x7f6c64072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c64105ae0 0x7f6c641a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64073580 con 0x7f6c640ffad0 2023-07-14T11:27:39.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c641a4340 con 0x7f6c64105ae0 2023-07-14T11:27:39.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6b9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c641a4480 con 0x7f6c64100480 2023-07-14T11:27:39.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c68f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c64100480 0x7f6c64072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c6979b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c640ffad0 0x7f6c64072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.450+0000 7f6c68f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c64100480 0x7f6c64072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45448/0 (socket says 172.21.15.161:45448) 2023-07-14T11:27:39.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c68f9a700 1 -- 172.21.15.161:0/480103059 learned_addr learned my addr 172.21.15.161:0/480103059 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c64105ae0 0x7f6c641a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c6979b700 1 -- 172.21.15.161:0/480103059 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c64100480 msgr2=0x7f6c64072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c6979b700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c64100480 0x7f6c64072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c6979b700 1 -- 172.21.15.161:0/480103059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c64105ae0 msgr2=0x7f6c641a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c6979b700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c64105ae0 0x7f6c641a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c6979b700 1 -- 172.21.15.161:0/480103059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540095f0 con 0x7f6c640ffad0 2023-07-14T11:27:39.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c64105ae0 0x7f6c641a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:39.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c64100480 0x7f6c64072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:39.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c6979b700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c640ffad0 0x7f6c64072810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009810 tx=0x7f6c6000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c5a7fc700 1 -- 172.21.15.161:0/480103059 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6001c070 con 0x7f6c640ffad0 2023-07-14T11:27:39.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c5a7fc700 1 -- 172.21.15.161:0/480103059 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c60005040 con 0x7f6c640ffad0 2023-07-14T11:27:39.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.451+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c60009520 con 0x7f6c640ffad0 2023-07-14T11:27:39.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.452+0000 7f6c5a7fc700 1 -- 172.21.15.161:0/480103059 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60003970 con 0x7f6c640ffad0 2023-07-14T11:27:39.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.452+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c641a4910 con 0x7f6c640ffad0 2023-07-14T11:27:39.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.455+0000 7f6c5a7fc700 1 -- 172.21.15.161:0/480103059 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6c60020030 con 0x7f6c640ffad0 2023-07-14T11:27:39.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.456+0000 7f6c7f243b80 1 Processor -- start 2023-07-14T11:27:39.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c5a7fc700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c500a12c0 0x7f6c500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c7f243b80 1 -- start start 2023-07-14T11:27:39.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c5a7fc700 1 -- 172.21.15.161:0/480103059 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f6c600cdfa0 con 0x7f6c640ffad0 2023-07-14T11:27:39.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c500a12c0 0x7f6c500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f37590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f39eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294e32cb0 con 0x55b294f2c2d0 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294e85a40 con 0x55b294f2cbe0 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294e7eed0 con 0x55b294f37ad0 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.457+0000 7f6c6979b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f2c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57332/0 (socket says 172.21.15.161:57332) 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 -- 172.21.15.161:0/2487139685 learned_addr learned my addr 172.21.15.161:0/2487139685 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f37590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f39eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 -- 172.21.15.161:0/2487139685 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 msgr2=0x55b294f39eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f39eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 -- 172.21.15.161:0/2487139685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 msgr2=0x55b294f37590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f37590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 -- 172.21.15.161:0/2487139685 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b294cacde0 con 0x55b294f2c2d0 2023-07-14T11:27:39.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c500a12c0 0x7f6c500a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c64074920 tx=0x7f6c5c009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c6979b700 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f2c6a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6c60097270 tx=0x7f6c6009a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38aea255bfb52a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.458+0000 7f6c4f7fe700 1 -- 172.21.15.161:0/2487139685 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c600a3130 con 0x55b294f2c2d0 2023-07-14T11:27:39.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c4f7fe700 1 -- 172.21.15.161:0/2487139685 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c60099110 con 0x55b294f2c2d0 2023-07-14T11:27:39.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c4f7fe700 1 -- 172.21.15.161:0/2487139685 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60005460 con 0x55b294f2c2d0 2023-07-14T11:27:39.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 -- 172.21.15.161:0/2487139685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 msgr2=0x55b294f2c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f2c6a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6c60097270 tx=0x7f6c6009a750 comp rx=0 tx=0).stop 2023-07-14T11:27:39.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 -- 172.21.15.161:0/2487139685 shutdown_connections 2023-07-14T11:27:39.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f2c6a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f39eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/2487139685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f37590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 -- 172.21.15.161:0/2487139685 >> 172.21.15.161:0/2487139685 conn(0x55b294f22da0 msgr2=0x55b294f286a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.459+0000 7f6c7f243b80 1 -- 172.21.15.161:0/2487139685 shutdown_connections 2023-07-14T11:27:39.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.460+0000 7f6c7f243b80 1 -- 172.21.15.161:0/2487139685 wait complete. 2023-07-14T11:27:39.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.460+0000 7f6c7f243b80 1 -- start start 2023-07-14T11:27:39.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.460+0000 7f6c7f243b80 1 Processor -- start 2023-07-14T11:27:39.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2c2d0 0x55b294f37ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2cbe0 0x55b294f383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f37ad0 0x55b294f38920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294e1eac0 con 0x55b294f37ad0 2023-07-14T11:27:39.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294e1e7c0 con 0x55b294f2c2d0 2023-07-14T11:27:39.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294d02920 con 0x55b294f2cbe0 2023-07-14T11:27:39.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c7f243b80 10 client.0 _ll_register_callbacks cb 0x55b294e1c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:39.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2cbe0 0x55b294f383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2cbe0 0x55b294f383e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45466/0 (socket says 172.21.15.161:45466) 2023-07-14T11:27:39.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c6979b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2c2d0 0x55b294f37ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 -- 172.21.15.161:0/1110824158 learned_addr learned my addr 172.21.15.161:0/1110824158 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f37ad0 0x55b294f38920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 -- 172.21.15.161:0/1110824158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2c2d0 msgr2=0x55b294f37ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2c2d0 0x55b294f37ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 -- 172.21.15.161:0/1110824158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f37ad0 msgr2=0x55b294f38920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f37ad0 0x55b294f38920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.461+0000 7f6c68f9a700 1 -- 172.21.15.161:0/1110824158 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c641a4910 con 0x55b294f2cbe0 2023-07-14T11:27:39.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f37ad0 0x55b294f38920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:39.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2cbe0 0x55b294f383e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6c54012e20 tx=0x7f6c5400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b0ae82cc5f83c87 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:39.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c4cff9700 1 -- 172.21.15.161:0/1110824158 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54017070 con 0x55b294f2cbe0 2023-07-14T11:27:39.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c4cff9700 1 -- 172.21.15.161:0/1110824158 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f6c54012070 con 0x55b294f2cbe0 2023-07-14T11:27:39.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b294dd84f0 con 0x55b294f2cbe0 2023-07-14T11:27:39.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c4cff9700 1 -- 172.21.15.161:0/1110824158 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c54012230 con 0x55b294f2cbe0 2023-07-14T11:27:39.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c7f243b80 10 client.26767 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:39.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.462+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b294c95c60 con 0x55b294f2cbe0 2023-07-14T11:27:39.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c4cff9700 1 -- 172.21.15.161:0/1110824158 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b294dd84f0 con 0x55b294f2cbe0 2023-07-14T11:27:39.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c4cff9700 1 -- 172.21.15.161:0/1110824158 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c54003c10 con 0x55b294f2cbe0 2023-07-14T11:27:39.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c7f243b80 10 client.26767 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:39.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c7f243b80 10 client.26767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:39.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c7f243b80 20 client.26767 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:39.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c7f243b80 4 client.26767 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:27:39.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b294e77eb0 con 0x55b294f47260 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c6979b700 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c6979b700 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f49670 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb2ff450461dde0 server_cookie=435e1c7aa1c8369c in_seq=0 out_seq=0 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.463+0000 7f6c4cff9700 10 client.26767 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.464+0000 7f6c4cff9700 1 -- 172.21.15.161:0/1110824158 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b294e77eb0 con 0x55b294f47260 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.464+0000 7f6c4cff9700 10 client.26767 handle_command_reply: tid=0 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.464+0000 7f6c7f243b80 1 client.26767 shutdown 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.464+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2cbe0 msgr2=0x55b294f383e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.464+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2cbe0 0x55b294f383e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6c54012e20 tx=0x7f6c5400f6a0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.464+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 shutdown_connections 2023-07-14T11:27:39.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.464+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f49670 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c4cff9700 0 client.26767 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f37ad0 0x55b294f38920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2c2d0 0x55b294f37ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/1110824158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2cbe0 0x55b294f383e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 >> 172.21.15.161:0/1110824158 conn(0x55b294f22da0 msgr2=0x55b294f28230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 shutdown_connections 2023-07-14T11:27:39.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c7f243b80 1 -- 172.21.15.161:0/1110824158 wait complete. 2023-07-14T11:27:39.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.465+0000 7f6c7f243b80 20 client.26767 trim_cache size 0 max 16384 2023-07-14T11:27:39.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 Processor -- start 2023-07-14T11:27:39.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 -- start start 2023-07-14T11:27:39.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f43390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f31e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294dd84f0 con 0x55b294f2c2d0 2023-07-14T11:27:39.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294f2d1c0 con 0x55b294f2cbe0 2023-07-14T11:27:39.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294f22970 con 0x55b294f37ad0 2023-07-14T11:27:39.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c68f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f31e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c6979b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f43390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c68f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f31e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48840/0 (socket says 172.21.15.161:48840) 2023-07-14T11:27:39.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c68f9a700 1 -- 172.21.15.161:0/3675967064 learned_addr learned my addr 172.21.15.161:0/3675967064 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.483+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c68f9a700 1 -- 172.21.15.161:0/3675967064 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 msgr2=0x55b294f281d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f281d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c68f9a700 1 -- 172.21.15.161:0/3675967064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 msgr2=0x55b294f43390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f43390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c68f9a700 1 -- 172.21.15.161:0/3675967064 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b294cd3250 con 0x55b294f2cbe0 2023-07-14T11:27:39.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c6979b700 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f43390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f31e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6c54010430 tx=0x7f6c5400ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d01a6fb3b1c02a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c4ffff700 1 -- 172.21.15.161:0/3675967064 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54017040 con 0x55b294f2cbe0 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c4ffff700 1 -- 172.21.15.161:0/3675967064 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c54012310 con 0x55b294f2cbe0 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.484+0000 7f6c4ffff700 1 -- 172.21.15.161:0/3675967064 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540042f0 con 0x55b294f2cbe0 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3675967064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 msgr2=0x55b294f31e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f31e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6c54010430 tx=0x7f6c5400ff00 comp rx=0 tx=0).stop 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3675967064 shutdown_connections 2023-07-14T11:27:39.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2c2d0 0x55b294f43390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f37ad0 0x55b294f281d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3675967064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f2cbe0 0x55b294f31e80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3675967064 >> 172.21.15.161:0/3675967064 conn(0x55b294f22da0 msgr2=0x55b294f31460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3675967064 shutdown_connections 2023-07-14T11:27:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.485+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3675967064 wait complete. 2023-07-14T11:27:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 -- start start 2023-07-14T11:27:39.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 Processor -- start 2023-07-14T11:27:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2c2d0 0x55b294f3bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2cbe0 0x55b294f3c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f37ad0 0x55b294f3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294f3a7a0 con 0x55b294f2cbe0 2023-07-14T11:27:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294f3a910 con 0x55b294f37ad0 2023-07-14T11:27:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.486+0000 7f6c7f243b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b294f3aad0 con 0x55b294f2c2d0 2023-07-14T11:27:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c7f243b80 10 client.0 _ll_register_callbacks cb 0x7f6c540128f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:27:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2c2d0 0x55b294f3bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2c2d0 0x55b294f3bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45480/0 (socket says 172.21.15.161:45480) 2023-07-14T11:27:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 -- 172.21.15.161:0/3088018124 learned_addr learned my addr 172.21.15.161:0/3088018124 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:27:39.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c69f9c700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f37ad0 0x55b294f3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2cbe0 0x55b294f3c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 -- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f37ad0 msgr2=0x55b294f3a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f37ad0 0x55b294f3a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 -- 172.21.15.161:0/3088018124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2cbe0 msgr2=0x55b294f3c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:27:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2cbe0 0x55b294f3c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 -- 172.21.15.161:0/3088018124 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b294cacde0 con 0x55b294f2c2d0 2023-07-14T11:27:39.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2cbe0 0x55b294f3c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c6979b700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2c2d0 0x55b294f3bb60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c6009a840 tx=0x7f6c6000b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb38f4b663b38133 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.487+0000 7f6c4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c4cff9700 1 -- 172.21.15.161:0/3088018124 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60097480 con 0x55b294f2c2d0 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c4cff9700 1 -- 172.21.15.161:0/3088018124 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f6c600975e0 con 0x55b294f2c2d0 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c60099110 con 0x55b294f2c2d0 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c4cff9700 1 -- 172.21.15.161:0/3088018124 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c600cb250 con 0x55b294f2c2d0 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 10 client.26773 fetch_fsmap learned FSMap version 25 2023-07-14T11:27:39.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f6c54021040 con 0x55b294f2c2d0 2023-07-14T11:27:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c4cff9700 1 -- 172.21.15.161:0/3088018124 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c60099110 con 0x55b294f2c2d0 2023-07-14T11:27:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c4cff9700 1 -- 172.21.15.161:0/3088018124 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c6009afd0 con 0x55b294f2c2d0 2023-07-14T11:27:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:27:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 10 client.26773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:27:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 20 client.26773 populate_metadata read hostname 'smithi161' 2023-07-14T11:27:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f9dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:27:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 4 client.26773 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:27:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.488+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b294e3a050 con 0x55b294f47260 2023-07-14T11:27:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f9dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:27:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c68f9a700 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f9dee0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f129bfb417f76 server_cookie=95d08c670623affd in_seq=0 out_seq=0 2023-07-14T11:27:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c4cff9700 10 client.26773 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c4cff9700 1 -- 172.21.15.161:0/3088018124 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b294e3a050 con 0x55b294f47260 2023-07-14T11:27:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c4cff9700 10 client.26773 handle_command_reply: tid=0 2023-07-14T11:27:39.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 client.26773 shutdown 2023-07-14T11:27:39.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2c2d0 msgr2=0x55b294f3bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2c2d0 0x55b294f3bb60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c6009a840 tx=0x7f6c6000b3e0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 shutdown_connections 2023-07-14T11:27:39.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b294f47260 0x55b294f9dee0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b294f37ad0 0x55b294f3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b294f2c2d0 0x55b294f3bb60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 --2- 172.21.15.161:0/3088018124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b294f2cbe0 0x55b294f3c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 >> 172.21.15.161:0/3088018124 conn(0x55b294f22da0 msgr2=0x55b294f43390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.489+0000 7f6c4cff9700 0 client.26773 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:27:39.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.490+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 shutdown_connections 2023-07-14T11:27:39.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.490+0000 7f6c7f243b80 1 -- 172.21.15.161:0/3088018124 wait complete. 2023-07-14T11:27:39.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.490+0000 7f6c7f243b80 20 client.26773 trim_cache size 0 max 16384 2023-07-14T11:27:39.503 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:27:39.503 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:27:39.503 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:27:39.503 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:27:39.504 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:27:39.504 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:27:39.504 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:27:39.504 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:27:39.504 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:27:39.504 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:27:39.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.490+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c500a12c0 msgr2=0x7f6c500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.490+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c500a12c0 0x7f6c500a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c64074920 tx=0x7f6c5c009920 comp rx=0 tx=0).stop 2023-07-14T11:27:39.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.490+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c640ffad0 msgr2=0x7f6c64072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:27:39.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.490+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c640ffad0 0x7f6c64072810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009810 tx=0x7f6c6000fa00 comp rx=0 tx=0).stop 2023-07-14T11:27:39.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 shutdown_connections 2023-07-14T11:27:39.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6c500a12c0 0x7f6c500a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c640ffad0 0x7f6c64072810 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c64105ae0 0x7f6c641a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 --2- 172.21.15.161:0/480103059 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6c64100480 0x7f6c64072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:27:39.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 >> 172.21.15.161:0/480103059 conn(0x7f6c640fb5f0 msgr2=0x7f6c640fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:27:39.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 shutdown_connections 2023-07-14T11:27:39.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:27:39.491+0000 7f6c6b9ff700 1 -- 172.21.15.161:0/480103059 wait complete. 2023-07-14T11:27:39.511 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:27:40.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 484778 9.84 MB/sec execute 142 sec latency 43.051 ms 2023-07-14T11:27:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:40 smithi182 ceph-mon[129366]: pgmap v165: 129 pgs: 129 active+clean; 290 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2023-07-14T11:27:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:40 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:27:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:40 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:40 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:27:40.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:40 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:27:40.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:40 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:40 smithi161 ceph-mon[95526]: pgmap v165: 129 pgs: 129 active+clean; 290 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2023-07-14T11:27:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:40 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:27:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:40 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:40.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:40 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:27:40.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:40 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:27:40.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:40 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:40.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:40 smithi175 ceph-mon[129081]: pgmap v165: 129 pgs: 129 active+clean; 290 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2023-07-14T11:27:40.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:40 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:27:40.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:40 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:40 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:27:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:40 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:27:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:40 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:27:41.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 487071 9.87 MB/sec execute 143 sec latency 30.176 ms 2023-07-14T11:27:42.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 488537 9.84 MB/sec execute 144 sec latency 16.870 ms 2023-07-14T11:27:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:42 smithi182 ceph-mon[129366]: pgmap v166: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2023-07-14T11:27:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:42 smithi161 ceph-mon[95526]: pgmap v166: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2023-07-14T11:27:42.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:42 smithi175 ceph-mon[129081]: pgmap v166: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2023-07-14T11:27:43.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 490698 9.87 MB/sec execute 145 sec latency 54.824 ms 2023-07-14T11:27:44.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:43 smithi175 ceph-mon[129081]: pgmap v167: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2023-07-14T11:27:44.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 492223 9.84 MB/sec execute 146 sec latency 30.717 ms 2023-07-14T11:27:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:43 smithi182 ceph-mon[129366]: pgmap v167: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2023-07-14T11:27:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:43 smithi161 ceph-mon[95526]: pgmap v167: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2023-07-14T11:27:45.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 494226 9.87 MB/sec execute 147 sec latency 21.386 ms 2023-07-14T11:27:46.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 495628 9.83 MB/sec execute 148 sec latency 32.313 ms 2023-07-14T11:27:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:46 smithi182 ceph-mon[129366]: pgmap v168: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2023-07-14T11:27:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:46 smithi161 ceph-mon[95526]: pgmap v168: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2023-07-14T11:27:46.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:46 smithi175 ceph-mon[129081]: pgmap v168: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2023-07-14T11:27:47.184 INFO:tasks.workunit.client.0.smithi161.stdout: 1 497798 9.86 MB/sec execute 149 sec latency 13.835 ms 2023-07-14T11:27:48.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 499275 9.83 MB/sec execute 150 sec latency 43.383 ms 2023-07-14T11:27:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:48 smithi182 ceph-mon[129366]: pgmap v169: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2023-07-14T11:27:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:48 smithi161 ceph-mon[95526]: pgmap v169: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2023-07-14T11:27:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:48 smithi175 ceph-mon[129081]: pgmap v169: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2023-07-14T11:27:49.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 501338 9.86 MB/sec execute 151 sec latency 26.831 ms 2023-07-14T11:27:50.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 502645 9.83 MB/sec execute 152 sec latency 50.713 ms 2023-07-14T11:27:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:50 smithi182 ceph-mon[129366]: pgmap v170: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-14T11:27:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:50 smithi161 ceph-mon[95526]: pgmap v170: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-14T11:27:50.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:50 smithi175 ceph-mon[129081]: pgmap v170: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-14T11:27:51.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 504815 9.85 MB/sec execute 153 sec latency 23.835 ms 2023-07-14T11:27:52.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 506267 9.82 MB/sec execute 154 sec latency 15.979 ms 2023-07-14T11:27:52.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:52 smithi182 ceph-mon[129366]: pgmap v171: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-07-14T11:27:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:52 smithi161 ceph-mon[95526]: pgmap v171: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-07-14T11:27:52.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:52 smithi175 ceph-mon[129081]: pgmap v171: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-07-14T11:27:53.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 508374 9.85 MB/sec execute 155 sec latency 50.212 ms 2023-07-14T11:27:54.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:53 smithi175 ceph-mon[129081]: pgmap v172: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 207 op/s 2023-07-14T11:27:54.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 509653 9.82 MB/sec execute 156 sec latency 27.046 ms 2023-07-14T11:27:54.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:53 smithi182 ceph-mon[129366]: pgmap v172: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 207 op/s 2023-07-14T11:27:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:53 smithi161 ceph-mon[95526]: pgmap v172: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 207 op/s 2023-07-14T11:27:55.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 511657 9.84 MB/sec execute 157 sec latency 156.853 ms 2023-07-14T11:27:56.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 512639 9.80 MB/sec execute 158 sec latency 31.373 ms 2023-07-14T11:27:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:56 smithi182 ceph-mon[129366]: pgmap v173: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 207 op/s 2023-07-14T11:27:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:56 smithi161 ceph-mon[95526]: pgmap v173: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 207 op/s 2023-07-14T11:27:56.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:56 smithi175 ceph-mon[129081]: pgmap v173: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 207 op/s 2023-07-14T11:27:57.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 515214 9.83 MB/sec execute 159 sec latency 14.680 ms 2023-07-14T11:27:58.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 516527 9.81 MB/sec execute 160 sec latency 16.565 ms 2023-07-14T11:27:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:27:58 smithi182 ceph-mon[129366]: pgmap v174: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 304 op/s 2023-07-14T11:27:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:27:58 smithi161 ceph-mon[95526]: pgmap v174: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 304 op/s 2023-07-14T11:27:58.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:27:58 smithi175 ceph-mon[129081]: pgmap v174: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 304 op/s 2023-07-14T11:27:59.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 518908 9.84 MB/sec execute 161 sec latency 17.138 ms 2023-07-14T11:28:00.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 519731 9.79 MB/sec execute 162 sec latency 84.845 ms 2023-07-14T11:28:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:00 smithi182 ceph-mon[129366]: pgmap v175: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2023-07-14T11:28:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:00 smithi161 ceph-mon[95526]: pgmap v175: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2023-07-14T11:28:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:00 smithi175 ceph-mon[129081]: pgmap v175: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2023-07-14T11:28:01.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 520947 9.76 MB/sec execute 163 sec latency 74.259 ms 2023-07-14T11:28:02.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 522223 9.77 MB/sec execute 164 sec latency 66.175 ms 2023-07-14T11:28:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:02 smithi182 ceph-mon[129366]: pgmap v176: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2023-07-14T11:28:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:02 smithi161 ceph-mon[95526]: pgmap v176: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2023-07-14T11:28:02.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:02 smithi175 ceph-mon[129081]: pgmap v176: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2023-07-14T11:28:03.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 523286 9.73 MB/sec execute 165 sec latency 16.730 ms 2023-07-14T11:28:04.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:03 smithi175 ceph-mon[129081]: pgmap v177: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-07-14T11:28:04.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 525758 9.76 MB/sec execute 166 sec latency 17.182 ms 2023-07-14T11:28:04.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:03 smithi182 ceph-mon[129366]: pgmap v177: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-07-14T11:28:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:03 smithi161 ceph-mon[95526]: pgmap v177: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-07-14T11:28:05.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 526738 9.72 MB/sec execute 167 sec latency 74.860 ms 2023-07-14T11:28:06.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 528695 9.73 MB/sec execute 168 sec latency 41.389 ms 2023-07-14T11:28:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:06 smithi182 ceph-mon[129366]: pgmap v178: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-07-14T11:28:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:06 smithi161 ceph-mon[95526]: pgmap v178: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-07-14T11:28:06.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:06 smithi175 ceph-mon[129081]: pgmap v178: 129 pgs: 129 active+clean; 340 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-07-14T11:28:07.185 INFO:tasks.workunit.client.0.smithi161.stdout: 1 530221 9.72 MB/sec execute 169 sec latency 20.632 ms 2023-07-14T11:28:08.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 532732 9.74 MB/sec execute 170 sec latency 16.556 ms 2023-07-14T11:28:08.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:08 smithi182 ceph-mon[129366]: pgmap v179: 129 pgs: 129 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 313 op/s 2023-07-14T11:28:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:08 smithi161 ceph-mon[95526]: pgmap v179: 129 pgs: 129 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 313 op/s 2023-07-14T11:28:08.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:08 smithi175 ceph-mon[129081]: pgmap v179: 129 pgs: 129 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 313 op/s 2023-07-14T11:28:09.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 534066 9.72 MB/sec execute 171 sec latency 16.995 ms 2023-07-14T11:28:09.512 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:28:09.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 -- 172.21.15.161:0/635632969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068101490 msgr2=0x7f8068109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 --2- 172.21.15.161:0/635632969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068101490 0x7f8068109b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8058009990 tx=0x7f8058009ca0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 -- 172.21.15.161:0/635632969 shutdown_connections 2023-07-14T11:28:09.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 --2- 172.21.15.161:0/635632969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8068100b80 0x7f8068100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 --2- 172.21.15.161:0/635632969 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f806810a0a0 0x7f806810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 --2- 172.21.15.161:0/635632969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068101490 0x7f8068109b60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 -- 172.21.15.161:0/635632969 >> 172.21.15.161:0/635632969 conn(0x7f80680fa4b0 msgr2=0x7f80680fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:09.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 -- 172.21.15.161:0/635632969 shutdown_connections 2023-07-14T11:28:09.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.728+0000 7f806f3ff700 1 -- 172.21.15.161:0/635632969 wait complete. 2023-07-14T11:28:09.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.729+0000 7f806f3ff700 1 Processor -- start 2023-07-14T11:28:09.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806f3ff700 1 -- start start 2023-07-14T11:28:09.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806f3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068100b80 0x7f8068197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806f3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8068101490 0x7f8068198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806f3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806810a0a0 0x7f806819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806f3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806819ceb0 con 0x7f806810a0a0 2023-07-14T11:28:09.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806f3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806819d020 con 0x7f8068100b80 2023-07-14T11:28:09.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806f3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806819d1e0 con 0x7f8068101490 2023-07-14T11:28:09.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068100b80 0x7f8068197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806c99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8068101490 0x7f8068198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068100b80 0x7f8068197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33520/0 (socket says 172.21.15.161:33520) 2023-07-14T11:28:09.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806d19b700 1 -- 172.21.15.161:0/3877064046 learned_addr learned my addr 172.21.15.161:0/3877064046 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.730+0000 7f806d99c700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806810a0a0 0x7f806819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806d19b700 1 -- 172.21.15.161:0/3877064046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8068101490 msgr2=0x7f8068198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806d19b700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8068101490 0x7f8068198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806d19b700 1 -- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806810a0a0 msgr2=0x7f806819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806d19b700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806810a0a0 0x7f806819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806d19b700 1 -- 172.21.15.161:0/3877064046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80580095f0 con 0x7f8068100b80 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806c99a700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8068101490 0x7f8068198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:28:09.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806d99c700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806810a0a0 0x7f806819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:28:09.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806d19b700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068100b80 0x7f8068197ed0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f806400e9a0 tx=0x7f806400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:09.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f805e7fc700 1 -- 172.21.15.161:0/3877064046 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064009790 con 0x7f8068100b80 2023-07-14T11:28:09.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f805e7fc700 1 -- 172.21.15.161:0/3877064046 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f806400cbd0 con 0x7f8068100b80 2023-07-14T11:28:09.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f805e7fc700 1 -- 172.21.15.161:0/3877064046 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064018680 con 0x7f8068100b80 2023-07-14T11:28:09.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806819d4c0 con 0x7f8068100b80 2023-07-14T11:28:09.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.731+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80681a4d00 con 0x7f8068100b80 2023-07-14T11:28:09.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.735+0000 7f805e7fc700 1 -- 172.21.15.161:0/3877064046 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f806400cd40 con 0x7f8068100b80 2023-07-14T11:28:09.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 Processor -- start 2023-07-14T11:28:09.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 -- start start 2023-07-14T11:28:09.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f805e7fc700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f80540a1340 0x7f80540a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af00942d0 0x559af00946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af0094be0 0x559af009f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af00a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeff9acb0 con 0x559af009fad0 2023-07-14T11:28:09.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeffeda40 con 0x559af0094be0 2023-07-14T11:28:09.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f8082d18b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeffe6ed0 con 0x559af00942d0 2023-07-14T11:28:09.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.736+0000 7f805e7fc700 1 -- 172.21.15.161:0/3877064046 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f8064014070 con 0x7f8068100b80 2023-07-14T11:28:09.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af0094be0 0x559af009f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af0094be0 0x559af009f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33528/0 (socket says 172.21.15.161:33528) 2023-07-14T11:28:09.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 -- 172.21.15.161:0/111635707 learned_addr learned my addr 172.21.15.161:0/111635707 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:09.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806d99c700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f80540a1340 0x7f80540a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806d19b700 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af00942d0 0x559af00946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806d99c700 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af00a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 -- 172.21.15.161:0/111635707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af00942d0 msgr2=0x559af00946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af00942d0 0x559af00946a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 -- 172.21.15.161:0/111635707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 msgr2=0x559af00a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af00a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 -- 172.21.15.161:0/111635707 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aefe14de0 con 0x559af0094be0 2023-07-14T11:28:09.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806d99c700 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af00a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:28:09.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806c99a700 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af0094be0 0x559af009f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8058000c00 tx=0x7f805800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=922dc6562dbbb858 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:09.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.737+0000 7f806d99c700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f80540a1340 0x7f80540a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8068069c40 tx=0x7f8060009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:09.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.738+0000 7f80537fe700 1 -- 172.21.15.161:0/111635707 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805801d050 con 0x559af0094be0 2023-07-14T11:28:09.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.738+0000 7f80537fe700 1 -- 172.21.15.161:0/111635707 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f805801e040 con 0x559af0094be0 2023-07-14T11:28:09.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.738+0000 7f80537fe700 1 -- 172.21.15.161:0/111635707 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580054f0 con 0x559af0094be0 2023-07-14T11:28:09.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.738+0000 7f8082d18b80 1 -- 172.21.15.161:0/111635707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af0094be0 msgr2=0x559af009f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.738+0000 7f8082d18b80 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af0094be0 0x559af009f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8058000c00 tx=0x7f805800b610 comp rx=0 tx=0).stop 2023-07-14T11:28:09.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 -- 172.21.15.161:0/111635707 shutdown_connections 2023-07-14T11:28:09.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af00a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af00942d0 0x559af00946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 --2- 172.21.15.161:0/111635707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af0094be0 0x559af009f590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 -- 172.21.15.161:0/111635707 >> 172.21.15.161:0/111635707 conn(0x559af008ada0 msgr2=0x559af00906a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 -- 172.21.15.161:0/111635707 shutdown_connections 2023-07-14T11:28:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 -- 172.21.15.161:0/111635707 wait complete. 2023-07-14T11:28:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 -- start start 2023-07-14T11:28:09.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.739+0000 7f8082d18b80 1 Processor -- start 2023-07-14T11:28:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af0094be0 0x559af009c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af009ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f8082d18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeff86ac0 con 0x559af009fad0 2023-07-14T11:28:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f8082d18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeff867c0 con 0x559af00942d0 2023-07-14T11:28:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f8082d18b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aefe6a920 con 0x559af0094be0 2023-07-14T11:28:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f8082d18b80 10 client.0 _ll_register_callbacks cb 0x559aeff84760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:28:09.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f806c99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af0094be0 0x559af009c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33536/0 (socket says 172.21.15.161:33536) 2023-07-14T11:28:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f806c99a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af0094be0 0x559af009c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52718/0 (socket says 172.21.15.161:52718) 2023-07-14T11:28:09.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f806d19b700 1 -- 172.21.15.161:0/1145330636 learned_addr learned my addr 172.21.15.161:0/1145330636 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.740+0000 7f806d99c700 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af009ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f806c99a700 1 -- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 msgr2=0x559af009f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f806c99a700 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f806c99a700 1 -- 172.21.15.161:0/1145330636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 msgr2=0x559af009ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f806c99a700 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af009ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f806c99a700 1 -- 172.21.15.161:0/1145330636 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806810d180 con 0x559af0094be0 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f806d19b700 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f806c99a700 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af0094be0 0x559af009c720 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8058009350 tx=0x7f8058005380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a355ecde2f0ec6e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f8050ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f8050ff9700 1 -- 172.21.15.161:0/1145330636 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580049e0 con 0x559af0094be0 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559aeff404f0 con 0x559af0094be0 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f8050ff9700 1 -- 172.21.15.161:0/1145330636 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f8058004b40 con 0x559af0094be0 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f8050ff9700 1 -- 172.21.15.161:0/1145330636 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8058004d00 con 0x559af0094be0 2023-07-14T11:28:09.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f8082d18b80 10 client.26785 fetch_fsmap learned FSMap version 25 2023-07-14T11:28:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.741+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559aefdfdc60 con 0x559af0094be0 2023-07-14T11:28:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8050ff9700 1 -- 172.21.15.161:0/1145330636 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559aeff404f0 con 0x559af0094be0 2023-07-14T11:28:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8050ff9700 1 -- 172.21.15.161:0/1145330636 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8058023460 con 0x559af0094be0 2023-07-14T11:28:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8082d18b80 10 client.26785 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:28:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8082d18b80 10 client.26785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:28:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8082d18b80 20 client.26785 populate_metadata read hostname 'smithi161' 2023-07-14T11:28:09.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8082d18b80 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af00aef40 0x559af00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8082d18b80 4 client.26785 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:28:09.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559aeffdfeb0 con 0x559af00aef40 2023-07-14T11:28:09.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.742+0000 7f806d19b700 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af00aef40 0x559af00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.743+0000 7f8050ff9700 10 client.26785 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:09.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.743+0000 7f806d19b700 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af00aef40 0x559af00b1350 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330fb014c9a45cbf server_cookie=9d73780919cffc4c in_seq=0 out_seq=0 2023-07-14T11:28:09.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.744+0000 7f8050ff9700 1 -- 172.21.15.161:0/1145330636 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559aeffdfeb0 con 0x559af00aef40 2023-07-14T11:28:09.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.744+0000 7f8050ff9700 10 client.26785 handle_command_reply: tid=0 2023-07-14T11:28:09.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.744+0000 7f8082d18b80 1 client.26785 shutdown 2023-07-14T11:28:09.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.744+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af0094be0 msgr2=0x559af009c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.744+0000 7f8082d18b80 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af0094be0 0x559af009c720 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8058009350 tx=0x7f8058005380 comp rx=0 tx=0).stop 2023-07-14T11:28:09.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 shutdown_connections 2023-07-14T11:28:09.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af00aef40 0x559af00b1350 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af009fad0 0x559af009ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8050ff9700 0 client.26785 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:09.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 --2- 172.21.15.161:0/1145330636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af0094be0 0x559af009c720 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 >> 172.21.15.161:0/1145330636 conn(0x559af008ada0 msgr2=0x559af0090230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:09.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 shutdown_connections 2023-07-14T11:28:09.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 1 -- 172.21.15.161:0/1145330636 wait complete. 2023-07-14T11:28:09.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.745+0000 7f8082d18b80 20 client.26785 trim_cache size 0 max 16384 2023-07-14T11:28:09.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.764+0000 7f8082d18b80 1 Processor -- start 2023-07-14T11:28:09.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.764+0000 7f8082d18b80 1 -- start start 2023-07-14T11:28:09.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.764+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.764+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af009b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af009f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f8082d18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeff404f0 con 0x559af0094be0 2023-07-14T11:28:09.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f8082d18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af0095540 con 0x559af00942d0 2023-07-14T11:28:09.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f8082d18b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af0095160 con 0x559af009fad0 2023-07-14T11:28:09.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33542/0 (socket says 172.21.15.161:33542) 2023-07-14T11:28:09.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 -- 172.21.15.161:0/3700998406 learned_addr learned my addr 172.21.15.161:0/3700998406 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:09.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d99c700 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af009f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806c99a700 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af009b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 -- 172.21.15.161:0/3700998406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 msgr2=0x559af009f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af009f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 -- 172.21.15.161:0/3700998406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 msgr2=0x559af009b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af009b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 -- 172.21.15.161:0/3700998406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af00ab950 con 0x559af00942d0 2023-07-14T11:28:09.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.765+0000 7f806d19b700 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009c6d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f806400e970 tx=0x7f8064086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d24a521cd183fb94 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:09.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8053fff700 1 -- 172.21.15.161:0/3700998406 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806409a040 con 0x559af00942d0 2023-07-14T11:28:09.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8053fff700 1 -- 172.21.15.161:0/3700998406 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8064099b00 con 0x559af00942d0 2023-07-14T11:28:09.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8053fff700 1 -- 172.21.15.161:0/3700998406 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064098190 con 0x559af00942d0 2023-07-14T11:28:09.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 -- 172.21.15.161:0/3700998406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 msgr2=0x559af009c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009c6d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f806400e970 tx=0x7f8064086040 comp rx=0 tx=0).stop 2023-07-14T11:28:09.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 -- 172.21.15.161:0/3700998406 shutdown_connections 2023-07-14T11:28:09.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af009f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af009c6d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3700998406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af009b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 -- 172.21.15.161:0/3700998406 >> 172.21.15.161:0/3700998406 conn(0x559af008ada0 msgr2=0x559af009bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:09.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 -- 172.21.15.161:0/3700998406 shutdown_connections 2023-07-14T11:28:09.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.766+0000 7f8082d18b80 1 -- 172.21.15.161:0/3700998406 wait complete. 2023-07-14T11:28:09.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.767+0000 7f8082d18b80 1 -- start start 2023-07-14T11:28:09.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.767+0000 7f8082d18b80 1 Processor -- start 2023-07-14T11:28:09.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af00a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f8082d18b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af00a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f8082d18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af0102e10 con 0x559af0094be0 2023-07-14T11:28:09.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f8082d18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af0102f50 con 0x559af00942d0 2023-07-14T11:28:09.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f8082d18b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af01030e0 con 0x559af009fad0 2023-07-14T11:28:09.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f8082d18b80 10 client.0 _ll_register_callbacks cb 0x7f8064010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:28:09.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af00a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af00a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af00a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52724/0 (socket says 172.21.15.161:52724) 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af00a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33548/0 (socket says 172.21.15.161:33548) 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 -- 172.21.15.161:0/3373744578 learned_addr learned my addr 172.21.15.161:0/3373744578 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806c99a700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 -- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 msgr2=0x559af00a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af00a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 -- 172.21.15.161:0/3373744578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 msgr2=0x559af00a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af00a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d99c700 1 -- 172.21.15.161:0/3373744578 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80681a4d00 con 0x559af009fad0 2023-07-14T11:28:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806d19b700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af00a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:28:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.768+0000 7f806c99a700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af00a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:28:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f806d99c700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af00a5630 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8060023e80 tx=0x7f8060004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd14f464e146d5b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f8050ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:28:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f8050ff9700 1 -- 172.21.15.161:0/3373744578 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060028040 con 0x559af009fad0 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f805801e040 con 0x559af009fad0 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f8050ff9700 1 -- 172.21.15.161:0/3373744578 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f8060003a00 con 0x559af009fad0 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f8050ff9700 1 -- 172.21.15.161:0/3373744578 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f806000e040 con 0x559af009fad0 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f8082d18b80 10 client.26791 fetch_fsmap learned FSMap version 25 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.769+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559aefe3b250 con 0x559af009fad0 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8050ff9700 1 -- 172.21.15.161:0/3373744578 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f805801e040 con 0x559af009fad0 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8050ff9700 1 -- 172.21.15.161:0/3373744578 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8060019460 con 0x559af009fad0 2023-07-14T11:28:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8082d18b80 10 client.26791 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T11:28:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8082d18b80 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:28:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8082d18b80 20 client.26791 populate_metadata read hostname 'smithi161' 2023-07-14T11:28:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af0104850 0x559af0106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8082d18b80 4 client.26791 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:28:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559aeffa2050 con 0x559af0104850 2023-07-14T11:28:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f806d19b700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af0104850 0x559af0106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f806d19b700 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af0104850 0x559af0106c60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d8a36d9642fe7c server_cookie=b3cd08bde4c0868b in_seq=0 out_seq=0 2023-07-14T11:28:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.770+0000 7f8050ff9700 10 client.26791 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8050ff9700 1 -- 172.21.15.161:0/3373744578 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559aeffa2050 con 0x559af0104850 2023-07-14T11:28:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8050ff9700 10 client.26791 handle_command_reply: tid=0 2023-07-14T11:28:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 client.26791 shutdown 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 msgr2=0x559af00a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af00a5630 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8060023e80 tx=0x7f8060004b40 comp rx=0 tx=0).stop 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 shutdown_connections 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559af0104850 0x559af0106c60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559af009fad0 0x559af00a5630 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8050ff9700 0 client.26791 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af00942d0 0x559af00a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 --2- 172.21.15.161:0/3373744578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559af0094be0 0x559af00a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 >> 172.21.15.161:0/3373744578 conn(0x559af008ada0 msgr2=0x559af00961a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:09.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 shutdown_connections 2023-07-14T11:28:09.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 1 -- 172.21.15.161:0/3373744578 wait complete. 2023-07-14T11:28:09.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.772+0000 7f8082d18b80 20 client.26791 trim_cache size 0 max 16384 2023-07-14T11:28:09.785 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:28:09.785 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:28:09.785 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:28:09.785 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:28:09.785 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:28:09.785 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:28:09.786 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:28:09.786 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:28:09.786 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:28:09.786 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:28:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f80540a1340 msgr2=0x7f80540a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f80540a1340 0x7f80540a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8068069c40 tx=0x7f8060009920 comp rx=0 tx=0).stop 2023-07-14T11:28:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068100b80 msgr2=0x7f8068197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068100b80 0x7f8068197ed0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f806400e9a0 tx=0x7f806400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 shutdown_connections 2023-07-14T11:28:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f80540a1340 0x7f80540a37f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806810a0a0 0x7f806819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8068100b80 0x7f8068197ed0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 --2- 172.21.15.161:0/3877064046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8068101490 0x7f8068198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 >> 172.21.15.161:0/3877064046 conn(0x7f80680fa4b0 msgr2=0x7f806810c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 shutdown_connections 2023-07-14T11:28:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:09.773+0000 7f806f3ff700 1 -- 172.21.15.161:0/3877064046 wait complete. 2023-07-14T11:28:09.794 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:28:10.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 536396 9.76 MB/sec execute 172 sec latency 39.851 ms 2023-07-14T11:28:10.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:10 smithi182 ceph-mon[129366]: pgmap v180: 129 pgs: 129 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 216 op/s 2023-07-14T11:28:10.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:10 smithi161 ceph-mon[95526]: pgmap v180: 129 pgs: 129 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 216 op/s 2023-07-14T11:28:10.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:10 smithi175 ceph-mon[129081]: pgmap v180: 129 pgs: 129 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 216 op/s 2023-07-14T11:28:11.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 537374 9.71 MB/sec execute 173 sec latency 40.260 ms 2023-07-14T11:28:12.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 539939 9.75 MB/sec execute 174 sec latency 15.270 ms 2023-07-14T11:28:12.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:12 smithi182 ceph-mon[129366]: pgmap v181: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 317 op/s 2023-07-14T11:28:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:12 smithi161 ceph-mon[95526]: pgmap v181: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 317 op/s 2023-07-14T11:28:12.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:12 smithi175 ceph-mon[129081]: pgmap v181: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 317 op/s 2023-07-14T11:28:13.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 540969 9.71 MB/sec execute 175 sec latency 75.107 ms 2023-07-14T11:28:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:13 smithi175 ceph-mon[129081]: pgmap v182: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-14T11:28:14.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 543505 9.75 MB/sec execute 176 sec latency 17.896 ms 2023-07-14T11:28:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:13 smithi182 ceph-mon[129366]: pgmap v182: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-14T11:28:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:13 smithi161 ceph-mon[95526]: pgmap v182: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-14T11:28:15.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 544593 9.72 MB/sec execute 177 sec latency 33.468 ms 2023-07-14T11:28:16.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 546494 9.72 MB/sec execute 178 sec latency 66.394 ms 2023-07-14T11:28:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:16 smithi182 ceph-mon[129366]: pgmap v183: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-14T11:28:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:16 smithi161 ceph-mon[95526]: pgmap v183: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-14T11:28:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:16 smithi175 ceph-mon[129081]: pgmap v183: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-14T11:28:17.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 547882 9.71 MB/sec execute 179 sec latency 20.138 ms 2023-07-14T11:28:18.186 INFO:tasks.workunit.client.0.smithi161.stdout: 1 550276 9.72 MB/sec execute 180 sec latency 14.800 ms 2023-07-14T11:28:18.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:18 smithi182 ceph-mon[129366]: pgmap v184: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 306 op/s 2023-07-14T11:28:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:18 smithi161 ceph-mon[95526]: pgmap v184: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 306 op/s 2023-07-14T11:28:18.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:18 smithi175 ceph-mon[129081]: pgmap v184: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 306 op/s 2023-07-14T11:28:19.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 551616 9.71 MB/sec execute 181 sec latency 16.383 ms 2023-07-14T11:28:20.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 554133 9.74 MB/sec execute 182 sec latency 16.630 ms 2023-07-14T11:28:20.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:20 smithi182 ceph-mon[129366]: pgmap v185: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-07-14T11:28:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:20 smithi161 ceph-mon[95526]: pgmap v185: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-07-14T11:28:20.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:20 smithi175 ceph-mon[129081]: pgmap v185: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-07-14T11:28:21.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 555047 9.70 MB/sec execute 183 sec latency 55.408 ms 2023-07-14T11:28:22.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 557084 9.72 MB/sec execute 184 sec latency 27.719 ms 2023-07-14T11:28:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:22 smithi182 ceph-mon[129366]: pgmap v186: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 294 op/s 2023-07-14T11:28:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:22 smithi161 ceph-mon[95526]: pgmap v186: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 294 op/s 2023-07-14T11:28:22.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:22 smithi175 ceph-mon[129081]: pgmap v186: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 294 op/s 2023-07-14T11:28:23.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 558445 9.70 MB/sec execute 185 sec latency 14.111 ms 2023-07-14T11:28:24.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:23 smithi175 ceph-mon[129081]: pgmap v187: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-14T11:28:24.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 560623 9.71 MB/sec execute 186 sec latency 15.410 ms 2023-07-14T11:28:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:23 smithi182 ceph-mon[129366]: pgmap v187: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-14T11:28:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:23 smithi161 ceph-mon[95526]: pgmap v187: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-14T11:28:25.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 562043 9.70 MB/sec execute 187 sec latency 20.289 ms 2023-07-14T11:28:26.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 563733 9.69 MB/sec execute 188 sec latency 32.810 ms 2023-07-14T11:28:26.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:26 smithi182 ceph-mon[129366]: pgmap v188: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-14T11:28:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:26 smithi161 ceph-mon[95526]: pgmap v188: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-14T11:28:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:26 smithi175 ceph-mon[129081]: pgmap v188: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-14T11:28:27.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 565187 9.70 MB/sec execute 189 sec latency 160.859 ms 2023-07-14T11:28:27.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:27 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:28:27.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:27 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:28:27.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:27 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:28:28.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 566287 9.67 MB/sec execute 190 sec latency 230.695 ms 2023-07-14T11:28:28.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:28 smithi182 ceph-mon[129366]: pgmap v189: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 296 op/s 2023-07-14T11:28:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:28 smithi161 ceph-mon[95526]: pgmap v189: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 296 op/s 2023-07-14T11:28:28.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:28 smithi175 ceph-mon[129081]: pgmap v189: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 296 op/s 2023-07-14T11:28:29.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 568609 9.69 MB/sec execute 191 sec latency 14.433 ms 2023-07-14T11:28:30.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 569779 9.67 MB/sec execute 192 sec latency 41.483 ms 2023-07-14T11:28:30.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:30 smithi182 ceph-mon[129366]: pgmap v190: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 196 op/s 2023-07-14T11:28:30.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:30 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:28:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:30 smithi161 ceph-mon[95526]: pgmap v190: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 196 op/s 2023-07-14T11:28:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:30 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:28:30.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:30 smithi175 ceph-mon[129081]: pgmap v190: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 196 op/s 2023-07-14T11:28:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:30 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:28:31.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 571893 9.69 MB/sec execute 193 sec latency 52.682 ms 2023-07-14T11:28:31.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:28:31.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:28:31.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:28:31.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:28:31.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:28:31.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:28:31.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:28:31.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:28:31.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:28:32.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 572947 9.65 MB/sec execute 194 sec latency 46.046 ms 2023-07-14T11:28:32.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:32 smithi182 ceph-mon[129366]: pgmap v191: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 287 op/s 2023-07-14T11:28:32.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:32 smithi161 ceph-mon[95526]: pgmap v191: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 287 op/s 2023-07-14T11:28:32.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:32 smithi175 ceph-mon[129081]: pgmap v191: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 287 op/s 2023-07-14T11:28:33.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 575399 9.68 MB/sec execute 195 sec latency 17.176 ms 2023-07-14T11:28:34.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:33 smithi175 ceph-mon[129081]: pgmap v192: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2023-07-14T11:28:34.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 576570 9.65 MB/sec execute 196 sec latency 17.284 ms 2023-07-14T11:28:34.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:33 smithi182 ceph-mon[129366]: pgmap v192: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2023-07-14T11:28:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:33 smithi161 ceph-mon[95526]: pgmap v192: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2023-07-14T11:28:35.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 578961 9.68 MB/sec execute 197 sec latency 39.657 ms 2023-07-14T11:28:36.187 INFO:tasks.workunit.client.0.smithi161.stdout: 1 579804 9.64 MB/sec execute 198 sec latency 62.796 ms 2023-07-14T11:28:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:36 smithi182 ceph-mon[129366]: pgmap v193: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2023-07-14T11:28:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:36 smithi161 ceph-mon[95526]: pgmap v193: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2023-07-14T11:28:36.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:36 smithi175 ceph-mon[129081]: pgmap v193: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2023-07-14T11:28:37.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 581949 9.66 MB/sec execute 199 sec latency 19.563 ms 2023-07-14T11:28:38.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 583237 9.64 MB/sec execute 200 sec latency 47.395 ms 2023-07-14T11:28:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:38 smithi182 ceph-mon[129366]: pgmap v194: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2023-07-14T11:28:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:38 smithi161 ceph-mon[95526]: pgmap v194: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2023-07-14T11:28:38.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:38 smithi175 ceph-mon[129081]: pgmap v194: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2023-07-14T11:28:39.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 584920 9.63 MB/sec execute 201 sec latency 107.507 ms 2023-07-14T11:28:39.795 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:28:40.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.003+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/2013237392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c08105a00 msgr2=0x7f3c08105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.003+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/2013237392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c08105a00 0x7f3c08105dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009990 tx=0x7f3bf8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.003+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/2013237392 shutdown_connections 2023-07-14T11:28:40.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.003+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/2013237392 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c081003a0 0x7f3c0810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.003+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/2013237392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c08105a00 0x7f3c08105dd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.003+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/2013237392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c080ff9f0 0x7f3c080ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.003+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/2013237392 >> 172.21.15.161:0/2013237392 conn(0x7f3c080fb510 msgr2=0x7f3c080fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:40.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.004+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/2013237392 shutdown_connections 2023-07-14T11:28:40.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.004+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/2013237392 wait complete. 2023-07-14T11:28:40.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.005+0000 7f3c0c9fe700 1 Processor -- start 2023-07-14T11:28:40.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.005+0000 7f3c0c9fe700 1 -- start start 2023-07-14T11:28:40.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0c9fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c080ff9f0 0x7f3c0819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0c9fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c081003a0 0x7f3c0819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0c9fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c08105a00 0x7f3c081957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0c9fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08195e30 con 0x7f3c08105a00 2023-07-14T11:28:40.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0c9fe700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08195fa0 con 0x7f3c080ff9f0 2023-07-14T11:28:40.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0c9fe700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08196160 con 0x7f3c081003a0 2023-07-14T11:28:40.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c080ff9f0 0x7f3c0819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c080ff9f0 0x7f3c0819b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48858/0 (socket says 172.21.15.161:48858) 2023-07-14T11:28:40.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 -- 172.21.15.161:0/4087023407 learned_addr learned my addr 172.21.15.161:0/4087023407 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:40.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0659c700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c081003a0 0x7f3c0819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0759e700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c08105a00 0x7f3c081957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 -- 172.21.15.161:0/4087023407 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c081003a0 msgr2=0x7f3c0819bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c081003a0 0x7f3c0819bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 -- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c08105a00 msgr2=0x7f3c081957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c08105a00 0x7f3c081957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c06d9d700 1 -- 172.21.15.161:0/4087023407 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf80095f0 con 0x7f3c080ff9f0 2023-07-14T11:28:40.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0759e700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c08105a00 0x7f3c081957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:28:40.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.006+0000 7f3c0659c700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c081003a0 0x7f3c0819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:28:40.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.007+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c080ff9f0 0x7f3c0819b700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8000c00 tx=0x7f3bf8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:40.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.007+0000 7f3beffff700 1 -- 172.21.15.161:0/4087023407 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf801d070 con 0x7f3c080ff9f0 2023-07-14T11:28:40.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.007+0000 7f3beffff700 1 -- 172.21.15.161:0/4087023407 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bf8021410 con 0x7f3c080ff9f0 2023-07-14T11:28:40.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.007+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c081963e0 con 0x7f3c080ff9f0 2023-07-14T11:28:40.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.007+0000 7f3beffff700 1 -- 172.21.15.161:0/4087023407 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8017650 con 0x7f3c080ff9f0 2023-07-14T11:28:40.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.007+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c081a26f0 con 0x7f3c080ff9f0 2023-07-14T11:28:40.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.010+0000 7f3beffff700 1 -- 172.21.15.161:0/4087023407 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f3bf8021580 con 0x7f3c080ff9f0 2023-07-14T11:28:40.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.012+0000 7f3c20c16b80 1 Processor -- start 2023-07-14T11:28:40.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.012+0000 7f3beffff700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3bf00a13e0 0x7f3bf00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.012+0000 7f3c20c16b80 1 -- start start 2023-07-14T11:28:40.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.012+0000 7f3beffff700 1 -- 172.21.15.161:0/4087023407 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f3bf80ce390 con 0x7f3c080ff9f0 2023-07-14T11:28:40.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.012+0000 7f3c0759e700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3bf00a13e0 0x7f3bf00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f027a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0288c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f028b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f017fcb0 con 0x5585f027a4f0 2023-07-14T11:28:40.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f01d2a40 con 0x5585f027ae00 2023-07-14T11:28:40.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f01cbed0 con 0x5585f0289140 2023-07-14T11:28:40.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0288c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c06d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f027a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0288c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48868/0 (socket says 172.21.15.161:48868) 2023-07-14T11:28:40.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 -- 172.21.15.161:0/1898220603 learned_addr learned my addr 172.21.15.161:0/1898220603 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:40.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0759e700 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f028b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0759e700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3bf00a13e0 0x7f3bf00a3890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c0819c400 tx=0x7f3c0000b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:40.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 -- 172.21.15.161:0/1898220603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 msgr2=0x5585f028b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f028b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 -- 172.21.15.161:0/1898220603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 msgr2=0x5585f027a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f027a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c0659c700 1 -- 172.21.15.161:0/1898220603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585efff9de0 con 0x5585f027ae00 2023-07-14T11:28:40.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.013+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f027a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.014+0000 7f3c0659c700 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0288c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0115f0 tx=0x7f3bfc011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a904463e0ecf05 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.014+0000 7f3bed7fa700 1 -- 172.21.15.161:0/1898220603 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004230 con 0x5585f027ae00 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.014+0000 7f3bed7fa700 1 -- 172.21.15.161:0/1898220603 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bfc004390 con 0x5585f027ae00 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.014+0000 7f3bed7fa700 1 -- 172.21.15.161:0/1898220603 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc019890 con 0x5585f027ae00 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.014+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1898220603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 msgr2=0x5585f0288c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.014+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0288c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0115f0 tx=0x7f3bfc011900 comp rx=0 tx=0).stop 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1898220603 shutdown_connections 2023-07-14T11:28:40.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f027a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f028b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1898220603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0288c00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1898220603 >> 172.21.15.161:0/1898220603 conn(0x5585f026fff0 msgr2=0x5585f0275930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:40.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1898220603 shutdown_connections 2023-07-14T11:28:40.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1898220603 wait complete. 2023-07-14T11:28:40.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 -- start start 2023-07-14T11:28:40.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.015+0000 7f3c20c16b80 1 Processor -- start 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f0288af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0285d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f02863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f016bac0 con 0x5585f027a4f0 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f016b7c0 con 0x5585f027ae00 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f004f920 con 0x5585f0289140 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c20c16b80 10 client.0 _ll_register_callbacks cb 0x5585f0169760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:28:40.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0285d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f02863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0285d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48878/0 (socket says 172.21.15.161:48878) 2023-07-14T11:28:40.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c0659c700 1 -- 172.21.15.161:0/430421647 learned_addr learned my addr 172.21.15.161:0/430421647 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:40.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.016+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f0288af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c0659c700 1 -- 172.21.15.161:0/430421647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 msgr2=0x5585f02863a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c0659c700 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f02863a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c0659c700 1 -- 172.21.15.161:0/430421647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 msgr2=0x5585f0288af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c0659c700 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f0288af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c0659c700 1 -- 172.21.15.161:0/430421647 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c081a26f0 con 0x5585f027ae00 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c0759e700 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f02863a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c0659c700 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0285d70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0115c0 tx=0x7f3bfc005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18b16a85ed03e0c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3beaff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3beaff5700 1 -- 172.21.15.161:0/430421647 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004950 con 0x5585f027ae00 2023-07-14T11:28:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585f01254f0 con 0x5585f027ae00 2023-07-14T11:28:40.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.017+0000 7f3beaff5700 1 -- 172.21.15.161:0/430421647 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f3bfc0056b0 con 0x5585f027ae00 2023-07-14T11:28:40.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3beaff5700 1 -- 172.21.15.161:0/430421647 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bfc004ab0 con 0x5585f027ae00 2023-07-14T11:28:40.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 10 client.26801 fetch_fsmap learned FSMap version 26 2023-07-14T11:28:40.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f3bfc009d50 con 0x5585f027ae00 2023-07-14T11:28:40.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3beaff5700 1 -- 172.21.15.161:0/430421647 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585f01254f0 con 0x5585f027ae00 2023-07-14T11:28:40.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3beaff5700 1 -- 172.21.15.161:0/430421647 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3bfc021040 con 0x5585f027ae00 2023-07-14T11:28:40.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 10 client.26801 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:28:40.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:28:40.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 20 client.26801 populate_metadata read hostname 'smithi161' 2023-07-14T11:28:40.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f0298a30 0x5585f029ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 4 client.26801 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:28:40.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585f01c4eb0 con 0x5585f0298a30 2023-07-14T11:28:40.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.018+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f0298a30 0x5585f029ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.019+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f0298a30 0x5585f029ae40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe1e58a0fc50b8d server_cookie=35388cf1d2794dd8 in_seq=0 out_seq=0 2023-07-14T11:28:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.019+0000 7f3beaff5700 10 client.26801 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.019+0000 7f3beaff5700 1 -- 172.21.15.161:0/430421647 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5585f01c4eb0 con 0x5585f0298a30 2023-07-14T11:28:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.019+0000 7f3beaff5700 10 client.26801 handle_command_reply: tid=0 2023-07-14T11:28:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.019+0000 7f3c20c16b80 1 client.26801 shutdown 2023-07-14T11:28:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 msgr2=0x5585f0285d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0285d70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0115c0 tx=0x7f3bfc005270 comp rx=0 tx=0).stop 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 shutdown_connections 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f0298a30 0x5585f029ae40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f0288af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f02863a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/430421647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0285d70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3beaff5700 0 client.26801 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 >> 172.21.15.161:0/430421647 conn(0x5585f026fff0 msgr2=0x5585f0275520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 shutdown_connections 2023-07-14T11:28:40.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 1 -- 172.21.15.161:0/430421647 wait complete. 2023-07-14T11:28:40.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.020+0000 7f3c20c16b80 20 client.26801 trim_cache size 0 max 16384 2023-07-14T11:28:40.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 Processor -- start 2023-07-14T11:28:40.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 -- start start 2023-07-14T11:28:40.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0294ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f0288690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0287350 con 0x5585f027a4f0 2023-07-14T11:28:40.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0275e70 con 0x5585f027ae00 2023-07-14T11:28:40.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f02766a0 con 0x5585f0289140 2023-07-14T11:28:40.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c06d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c06d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55578/0 (socket says 172.21.15.161:55578) 2023-07-14T11:28:40.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c06d9d700 1 -- 172.21.15.161:0/1514187802 learned_addr learned my addr 172.21.15.161:0/1514187802 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:40.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0294ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.038+0000 7f3c0759e700 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f0288690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3c06d9d700 1 -- 172.21.15.161:0/1514187802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 msgr2=0x5585f0288690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f0288690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3c06d9d700 1 -- 172.21.15.161:0/1514187802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 msgr2=0x5585f0294ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0294ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3c06d9d700 1 -- 172.21.15.161:0/1514187802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585effe2c60 con 0x5585f027a4f0 2023-07-14T11:28:40.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3c0659c700 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0294ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:28:40.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028fc50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80cb7b0 tx=0x7f3bf80a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cef653cbe028e80 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:40.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3bedffb700 1 -- 172.21.15.161:0/1514187802 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80cbea0 con 0x5585f027a4f0 2023-07-14T11:28:40.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3bedffb700 1 -- 172.21.15.161:0/1514187802 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bf809af30 con 0x5585f027a4f0 2023-07-14T11:28:40.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.039+0000 7f3bedffb700 1 -- 172.21.15.161:0/1514187802 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8099350 con 0x5585f027a4f0 2023-07-14T11:28:40.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1514187802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 msgr2=0x5585f028fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028fc50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80cb7b0 tx=0x7f3bf80a1040 comp rx=0 tx=0).stop 2023-07-14T11:28:40.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1514187802 shutdown_connections 2023-07-14T11:28:40.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028fc50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f0289140 0x5585f0288690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/1514187802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f027ae00 0x5585f0294ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1514187802 >> 172.21.15.161:0/1514187802 conn(0x5585f026fff0 msgr2=0x5585f0274060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:40.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1514187802 shutdown_connections 2023-07-14T11:28:40.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 -- 172.21.15.161:0/1514187802 wait complete. 2023-07-14T11:28:40.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 -- start start 2023-07-14T11:28:40.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.040+0000 7f3c20c16b80 1 Processor -- start 2023-07-14T11:28:40.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f027ae00 0x5585f028fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c20c16b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f0289140 0x5585f028dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f028e280 con 0x5585f027a4f0 2023-07-14T11:28:40.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f028e3f0 con 0x5585f0289140 2023-07-14T11:28:40.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c20c16b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f028e5b0 con 0x5585f027ae00 2023-07-14T11:28:40.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c20c16b80 10 client.0 _ll_register_callbacks cb 0x7f3bf8005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:28:40.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f027ae00 0x5585f028fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f027ae00 0x5585f028fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:39186/0 (socket says 172.21.15.161:39186) 2023-07-14T11:28:40.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0659c700 1 -- 172.21.15.161:0/892332429 learned_addr learned my addr 172.21.15.161:0/892332429 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:28:40.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0759e700 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f0289140 0x5585f028dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0659c700 1 -- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f0289140 msgr2=0x5585f028dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0659c700 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f0289140 0x5585f028dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0659c700 1 -- 172.21.15.161:0/892332429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 msgr2=0x5585f028f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.041+0000 7f3c0659c700 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3c0659c700 1 -- 172.21.15.161:0/892332429 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc009d50 con 0x5585f027ae00 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3c0659c700 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f027ae00 0x5585f028fb90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005950 tx=0x7f3bfc009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79773e76a170b83d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:28:40.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3beaff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:28:40.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3beaff5700 1 -- 172.21.15.161:0/892332429 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc023c50 con 0x5585f027ae00 2023-07-14T11:28:40.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3beaff5700 1 -- 172.21.15.161:0/892332429 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f3bfc0056b0 con 0x5585f027ae00 2023-07-14T11:28:40.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bfc004390 con 0x5585f027ae00 2023-07-14T11:28:40.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3beaff5700 1 -- 172.21.15.161:0/892332429 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bfc023810 con 0x5585f027ae00 2023-07-14T11:28:40.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3c20c16b80 10 client.26809 fetch_fsmap learned FSMap version 26 2023-07-14T11:28:40.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f3c081963e0 con 0x5585f027ae00 2023-07-14T11:28:40.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.042+0000 7f3beaff5700 1 -- 172.21.15.161:0/892332429 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bfc004390 con 0x5585f027ae00 2023-07-14T11:28:40.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3beaff5700 1 -- 172.21.15.161:0/892332429 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3bfc011970 con 0x5585f027ae00 2023-07-14T11:28:40.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c20c16b80 10 client.26809 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:28:40.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c20c16b80 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:28:40.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c20c16b80 20 client.26809 populate_metadata read hostname 'smithi161' 2023-07-14T11:28:40.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f02ed700 0x5585f02efb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:28:40.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c20c16b80 4 client.26809 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:28:40.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5585f0187050 con 0x5585f02ed700 2023-07-14T11:28:40.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f02ed700 0x5585f02efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:28:40.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3c06d9d700 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f02ed700 0x5585f02efb10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b673601cc78b67b9 server_cookie=d656ff3379c08bbc in_seq=0 out_seq=0 2023-07-14T11:28:40.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.043+0000 7f3beaff5700 10 client.26809 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:40.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3beaff5700 1 -- 172.21.15.161:0/892332429 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5585f0187050 con 0x5585f02ed700 2023-07-14T11:28:40.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3beaff5700 10 client.26809 handle_command_reply: tid=0 2023-07-14T11:28:40.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 client.26809 shutdown 2023-07-14T11:28:40.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f027ae00 msgr2=0x5585f028fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f027ae00 0x5585f028fb90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005950 tx=0x7f3bfc009750 comp rx=0 tx=0).stop 2023-07-14T11:28:40.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 shutdown_connections 2023-07-14T11:28:40.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5585f02ed700 0x5585f02efb10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f027a4f0 0x5585f028f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5585f0289140 0x5585f028dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 --2- 172.21.15.161:0/892332429 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5585f027ae00 0x5585f028fb90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3beaff5700 0 client.26809 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:28:40.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 >> 172.21.15.161:0/892332429 conn(0x5585f026fff0 msgr2=0x5585f0275460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:40.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 shutdown_connections 2023-07-14T11:28:40.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 1 -- 172.21.15.161:0/892332429 wait complete. 2023-07-14T11:28:40.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.044+0000 7f3c20c16b80 20 client.26809 trim_cache size 0 max 16384 2023-07-14T11:28:40.058 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:28:40.059 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:28:40.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:28:40.059 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:28:40.059 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:28:40.059 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:28:40.059 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:28:40.059 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:28:40.060 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:28:40.060 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:28:40.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3bf00a13e0 msgr2=0x7f3bf00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3bf00a13e0 0x7f3bf00a3890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c0819c400 tx=0x7f3c0000b040 comp rx=0 tx=0).stop 2023-07-14T11:28:40.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c080ff9f0 msgr2=0x7f3c0819b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:28:40.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c080ff9f0 0x7f3c0819b700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8000c00 tx=0x7f3bf8005740 comp rx=0 tx=0).stop 2023-07-14T11:28:40.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 shutdown_connections 2023-07-14T11:28:40.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f3bf00a13e0 0x7f3bf00a3890 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c08105a00 0x7f3c081957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c080ff9f0 0x7f3c0819b700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 --2- 172.21.15.161:0/4087023407 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f3c081003a0 0x7f3c0819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:28:40.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 >> 172.21.15.161:0/4087023407 conn(0x7f3c080fb510 msgr2=0x7f3c080fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:28:40.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 shutdown_connections 2023-07-14T11:28:40.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:28:40.045+0000 7f3c0c9fe700 1 -- 172.21.15.161:0/4087023407 wait complete. 2023-07-14T11:28:40.067 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:28:40.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 586529 9.64 MB/sec execute 202 sec latency 12.775 ms 2023-07-14T11:28:40.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:40 smithi182 ceph-mon[129366]: pgmap v195: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 187 op/s 2023-07-14T11:28:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:40 smithi161 ceph-mon[95526]: pgmap v195: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 187 op/s 2023-07-14T11:28:40.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:40 smithi175 ceph-mon[129081]: pgmap v195: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 187 op/s 2023-07-14T11:28:41.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 587932 9.62 MB/sec execute 203 sec latency 53.272 ms 2023-07-14T11:28:42.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 589939 9.64 MB/sec execute 204 sec latency 26.832 ms 2023-07-14T11:28:42.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:42 smithi182 ceph-mon[129366]: pgmap v196: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2023-07-14T11:28:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:42 smithi161 ceph-mon[95526]: pgmap v196: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2023-07-14T11:28:42.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:42 smithi175 ceph-mon[129081]: pgmap v196: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2023-07-14T11:28:43.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 591288 9.62 MB/sec execute 205 sec latency 20.283 ms 2023-07-14T11:28:44.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:43 smithi175 ceph-mon[129081]: pgmap v197: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2023-07-14T11:28:44.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 593457 9.64 MB/sec execute 206 sec latency 50.076 ms 2023-07-14T11:28:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:43 smithi182 ceph-mon[129366]: pgmap v197: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2023-07-14T11:28:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:43 smithi161 ceph-mon[95526]: pgmap v197: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2023-07-14T11:28:45.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 594813 9.62 MB/sec execute 207 sec latency 20.177 ms 2023-07-14T11:28:46.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 596989 9.64 MB/sec execute 208 sec latency 40.471 ms 2023-07-14T11:28:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:46 smithi182 ceph-mon[129366]: pgmap v198: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2023-07-14T11:28:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:46 smithi161 ceph-mon[95526]: pgmap v198: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2023-07-14T11:28:46.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:46 smithi175 ceph-mon[129081]: pgmap v198: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2023-07-14T11:28:47.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 598164 9.62 MB/sec execute 209 sec latency 24.349 ms 2023-07-14T11:28:48.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 600490 9.64 MB/sec execute 210 sec latency 14.084 ms 2023-07-14T11:28:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:48 smithi182 ceph-mon[129366]: pgmap v199: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 287 op/s 2023-07-14T11:28:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:48 smithi161 ceph-mon[95526]: pgmap v199: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 287 op/s 2023-07-14T11:28:48.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:48 smithi175 ceph-mon[129081]: pgmap v199: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 287 op/s 2023-07-14T11:28:49.188 INFO:tasks.workunit.client.0.smithi161.stdout: 1 601708 9.62 MB/sec execute 211 sec latency 36.680 ms 2023-07-14T11:28:50.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 604010 9.64 MB/sec execute 212 sec latency 40.736 ms 2023-07-14T11:28:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:50 smithi182 ceph-mon[129366]: pgmap v200: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-07-14T11:28:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:50 smithi161 ceph-mon[95526]: pgmap v200: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-07-14T11:28:50.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:50 smithi175 ceph-mon[129081]: pgmap v200: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-07-14T11:28:51.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 605042 9.60 MB/sec execute 213 sec latency 47.957 ms 2023-07-14T11:28:52.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 607347 9.63 MB/sec execute 214 sec latency 26.847 ms 2023-07-14T11:28:52.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:52 smithi182 ceph-mon[129366]: pgmap v201: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2023-07-14T11:28:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:52 smithi161 ceph-mon[95526]: pgmap v201: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2023-07-14T11:28:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:52 smithi175 ceph-mon[129081]: pgmap v201: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2023-07-14T11:28:53.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 608625 9.60 MB/sec execute 215 sec latency 12.415 ms 2023-07-14T11:28:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:53 smithi175 ceph-mon[129081]: pgmap v202: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-14T11:28:54.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 611024 9.63 MB/sec execute 216 sec latency 27.940 ms 2023-07-14T11:28:54.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:53 smithi182 ceph-mon[129366]: pgmap v202: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-14T11:28:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:53 smithi161 ceph-mon[95526]: pgmap v202: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-14T11:28:55.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 612126 9.60 MB/sec execute 217 sec latency 24.535 ms 2023-07-14T11:28:56.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 614267 9.61 MB/sec execute 218 sec latency 52.573 ms 2023-07-14T11:28:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:56 smithi182 ceph-mon[129366]: pgmap v203: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-14T11:28:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:56 smithi161 ceph-mon[95526]: pgmap v203: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-14T11:28:56.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:56 smithi175 ceph-mon[129081]: pgmap v203: 129 pgs: 129 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-14T11:28:57.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 615349 9.59 MB/sec execute 219 sec latency 32.392 ms 2023-07-14T11:28:58.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 617631 9.61 MB/sec execute 220 sec latency 25.989 ms 2023-07-14T11:28:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:28:58 smithi182 ceph-mon[129366]: pgmap v204: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 291 op/s 2023-07-14T11:28:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:28:58 smithi161 ceph-mon[95526]: pgmap v204: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 291 op/s 2023-07-14T11:28:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:28:58 smithi175 ceph-mon[129081]: pgmap v204: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 291 op/s 2023-07-14T11:28:59.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 618910 9.59 MB/sec execute 221 sec latency 13.614 ms 2023-07-14T11:29:00.189 INFO:tasks.workunit.client.0.smithi161.stdout: 1 621334 9.61 MB/sec execute 222 sec latency 18.071 ms 2023-07-14T11:29:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:00 smithi182 ceph-mon[129366]: pgmap v205: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 193 op/s 2023-07-14T11:29:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:00 smithi161 ceph-mon[95526]: pgmap v205: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 193 op/s 2023-07-14T11:29:00.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:00 smithi175 ceph-mon[129081]: pgmap v205: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 193 op/s 2023-07-14T11:29:01.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 621481 9.58 MB/sec execute 223 sec latency 843.662 ms 2023-07-14T11:29:02.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 621481 9.54 MB/sec execute 224 sec latency 1843.800 ms 2023-07-14T11:29:02.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:02 smithi182 ceph-mon[129366]: pgmap v206: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 283 op/s 2023-07-14T11:29:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:02 smithi161 ceph-mon[95526]: pgmap v206: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 283 op/s 2023-07-14T11:29:02.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:02 smithi175 ceph-mon[129081]: pgmap v206: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 283 op/s 2023-07-14T11:29:03.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 621481 9.50 MB/sec execute 225 sec latency 2843.885 ms 2023-07-14T11:29:04.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:03 smithi175 ceph-mon[129081]: pgmap v207: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-07-14T11:29:04.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 621481 9.46 MB/sec execute 226 sec latency 3844.063 ms 2023-07-14T11:29:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:03 smithi182 ceph-mon[129366]: pgmap v207: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-07-14T11:29:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:03 smithi161 ceph-mon[95526]: pgmap v207: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-07-14T11:29:05.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 621481 9.41 MB/sec execute 227 sec latency 4844.202 ms 2023-07-14T11:29:06.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 622435 9.38 MB/sec execute 228 sec latency 4925.544 ms 2023-07-14T11:29:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:06 smithi182 ceph-mon[129366]: pgmap v208: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-07-14T11:29:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:06 smithi161 ceph-mon[95526]: pgmap v208: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-07-14T11:29:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:06 smithi175 ceph-mon[129081]: pgmap v208: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-07-14T11:29:07.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 623981 9.37 MB/sec execute 229 sec latency 49.726 ms 2023-07-14T11:29:07.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:07 smithi182 ceph-mon[129366]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T11:29:07.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:07 smithi161 ceph-mon[95526]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T11:29:07.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:07 smithi175 ceph-mon[129081]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T11:29:08.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 625599 9.38 MB/sec execute 230 sec latency 14.917 ms 2023-07-14T11:29:08.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:08 smithi182 ceph-mon[129366]: pgmap v210: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-07-14T11:29:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:08 smithi161 ceph-mon[95526]: pgmap v210: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-07-14T11:29:08.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:08 smithi175 ceph-mon[129081]: pgmap v210: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-07-14T11:29:09.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 627512 9.37 MB/sec execute 231 sec latency 18.321 ms 2023-07-14T11:29:10.068 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:29:10.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 629083 9.38 MB/sec execute 232 sec latency 35.815 ms 2023-07-14T11:29:10.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 -- 172.21.15.161:0/3715400814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 msgr2=0x7fc8980fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 --2- 172.21.15.161:0/3715400814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc8980fff40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009990 tx=0x7fc88c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 -- 172.21.15.161:0/3715400814 shutdown_connections 2023-07-14T11:29:10.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 --2- 172.21.15.161:0/3715400814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898105ae0 0x7fc898105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 --2- 172.21.15.161:0/3715400814 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc898100480 0x7fc89810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 --2- 172.21.15.161:0/3715400814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc8980fff40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 -- 172.21.15.161:0/3715400814 >> 172.21.15.161:0/3715400814 conn(0x7fc8980fb5f0 msgr2=0x7fc8980fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:10.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 -- 172.21.15.161:0/3715400814 shutdown_connections 2023-07-14T11:29:10.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.286+0000 7fc89cfff700 1 -- 172.21.15.161:0/3715400814 wait complete. 2023-07-14T11:29:10.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.287+0000 7fc89cfff700 1 Processor -- start 2023-07-14T11:29:10.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.287+0000 7fc89cfff700 1 -- start start 2023-07-14T11:29:10.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89cfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc898197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89cfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898100480 0x7fc898198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89cfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc898105ae0 0x7fc89819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89cfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819ce30 con 0x7fc898100480 2023-07-14T11:29:10.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89cfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819cfa0 con 0x7fc8980ffad0 2023-07-14T11:29:10.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89cfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819d160 con 0x7fc898105ae0 2023-07-14T11:29:10.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc898197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc895d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898100480 0x7fc898198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc896d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc898105ae0 0x7fc89819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc898197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46260/0 (socket says 172.21.15.161:46260) 2023-07-14T11:29:10.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 -- 172.21.15.161:0/1743926130 learned_addr learned my addr 172.21.15.161:0/1743926130 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:10.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 -- 172.21.15.161:0/1743926130 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc898105ae0 msgr2=0x7fc89819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc898105ae0 0x7fc89819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 -- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898100480 msgr2=0x7fc898198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898100480 0x7fc898198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc89659c700 1 -- 172.21.15.161:0/1743926130 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0095f0 con 0x7fc8980ffad0 2023-07-14T11:29:10.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc896d9d700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc898105ae0 0x7fc89819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:29:10.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.288+0000 7fc895d9b700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898100480 0x7fc898198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:29:10.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.289+0000 7fc89659c700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc898197e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e9a0 tx=0x7fc88800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:10.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.289+0000 7fc8877fe700 1 -- 172.21.15.161:0/1743926130 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888009790 con 0x7fc8980ffad0 2023-07-14T11:29:10.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.289+0000 7fc8877fe700 1 -- 172.21.15.161:0/1743926130 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc888010430 con 0x7fc8980ffad0 2023-07-14T11:29:10.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.289+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89819d440 con 0x7fc8980ffad0 2023-07-14T11:29:10.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.289+0000 7fc8877fe700 1 -- 172.21.15.161:0/1743926130 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888018680 con 0x7fc8980ffad0 2023-07-14T11:29:10.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.289+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8981a4c80 con 0x7fc8980ffad0 2023-07-14T11:29:10.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.294+0000 7fc8877fe700 1 -- 172.21.15.161:0/1743926130 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc8880105a0 con 0x7fc8980ffad0 2023-07-14T11:29:10.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.295+0000 7fc8b09eeb80 1 Processor -- start 2023-07-14T11:29:10.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.295+0000 7fc8b09eeb80 1 -- start start 2023-07-14T11:29:10.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a99156a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9915be0 0x5617a9920590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a9922eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a981bcb0 con 0x5617a9920ad0 2023-07-14T11:29:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a986ea40 con 0x5617a9915be0 2023-07-14T11:29:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a9867ed0 con 0x5617a99152d0 2023-07-14T11:29:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8877fe700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8800a12c0 0x7fc8800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc89659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9915be0 0x5617a9920590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc895d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a9922eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc89659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9915be0 0x5617a9920590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46266/0 (socket says 172.21.15.161:46266) 2023-07-14T11:29:10.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc89659c700 1 -- 172.21.15.161:0/3836274211 learned_addr learned my addr 172.21.15.161:0/3836274211 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:10.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc896d9d700 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a99156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc8877fe700 1 -- 172.21.15.161:0/1743926130 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fc888014070 con 0x7fc8980ffad0 2023-07-14T11:29:10.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc896d9d700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8800a12c0 0x7fc8800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.296+0000 7fc89659c700 1 -- 172.21.15.161:0/3836274211 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 msgr2=0x5617a99156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc89659c700 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a99156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc89659c700 1 -- 172.21.15.161:0/3836274211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 msgr2=0x5617a9922eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc89659c700 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a9922eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc89659c700 1 -- 172.21.15.161:0/3836274211 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617a9695de0 con 0x5617a9915be0 2023-07-14T11:29:10.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc895d9b700 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a9922eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:29:10.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc89659c700 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9915be0 0x5617a9920590 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc8880ca4e0 tx=0x7fc888099ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb7c8e298ecfc33 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:10.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc884ff9700 1 -- 172.21.15.161:0/3836274211 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888099f20 con 0x5617a9915be0 2023-07-14T11:29:10.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc884ff9700 1 -- 172.21.15.161:0/3836274211 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc88809df50 con 0x5617a9915be0 2023-07-14T11:29:10.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.297+0000 7fc884ff9700 1 -- 172.21.15.161:0/3836274211 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8880cf320 con 0x5617a9915be0 2023-07-14T11:29:10.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3836274211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9915be0 msgr2=0x5617a9920590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9915be0 0x5617a9920590 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc8880ca4e0 tx=0x7fc888099ae0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc896d9d700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8800a12c0 0x7fc8800a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc890009fd0 tx=0x7fc89000b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3836274211 shutdown_connections 2023-07-14T11:29:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a9922eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a99156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3836274211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9915be0 0x5617a9920590 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3836274211 >> 172.21.15.161:0/3836274211 conn(0x5617a990bda0 msgr2=0x5617a99116a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3836274211 shutdown_connections 2023-07-14T11:29:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.298+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3836274211 wait complete. 2023-07-14T11:29:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 -- start start 2023-07-14T11:29:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 Processor -- start 2023-07-14T11:29:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a99152d0 0x5617a9920560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a9915be0 0x5617a991d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a991df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a9807ac0 con 0x5617a9920ad0 2023-07-14T11:29:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a98077c0 con 0x5617a99152d0 2023-07-14T11:29:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a96eb920 con 0x5617a9915be0 2023-07-14T11:29:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.299+0000 7fc8b09eeb80 10 client.0 _ll_register_callbacks cb 0x5617a9805760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:29:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a99152d0 0x5617a9920560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a99152d0 0x5617a9920560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46276/0 (socket says 172.21.15.161:46276) 2023-07-14T11:29:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 -- 172.21.15.161:0/2451281139 learned_addr learned my addr 172.21.15.161:0/2451281139 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc895d9b700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a9915be0 0x5617a991d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc896d9d700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a991df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 -- 172.21.15.161:0/2451281139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a9915be0 msgr2=0x5617a991d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a9915be0 0x5617a991d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 -- 172.21.15.161:0/2451281139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 msgr2=0x5617a991df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a991df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 -- 172.21.15.161:0/2451281139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89810d210 con 0x5617a99152d0 2023-07-14T11:29:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc896d9d700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a991df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:29:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc89659c700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a99152d0 0x5617a9920560 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc888096420 tx=0x7fc888010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a34a8e725a7437a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc87dffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:29:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc87dffb700 1 -- 172.21.15.161:0/2451281139 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88809a380 con 0x5617a99152d0 2023-07-14T11:29:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc87dffb700 1 -- 172.21.15.161:0/2451281139 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fc8880cf2d0 con 0x5617a99152d0 2023-07-14T11:29:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.300+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617a97c14f0 con 0x5617a99152d0 2023-07-14T11:29:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc87dffb700 1 -- 172.21.15.161:0/2451281139 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc888065420 con 0x5617a99152d0 2023-07-14T11:29:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 10 client.26819 fetch_fsmap learned FSMap version 26 2023-07-14T11:29:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc8981a4c80 con 0x5617a99152d0 2023-07-14T11:29:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc87dffb700 1 -- 172.21.15.161:0/2451281139 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617a97c14f0 con 0x5617a99152d0 2023-07-14T11:29:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc87dffb700 1 -- 172.21.15.161:0/2451281139 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc888095d10 con 0x5617a99152d0 2023-07-14T11:29:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 10 client.26819 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:29:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:29:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 20 client.26819 populate_metadata read hostname 'smithi161' 2023-07-14T11:29:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a99300d0 0x5617a99324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 4 client.26819 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:29:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617a9860eb0 con 0x5617a99300d0 2023-07-14T11:29:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.301+0000 7fc895d9b700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a99300d0 0x5617a99324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.302+0000 7fc895d9b700 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a99300d0 0x5617a99324b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f19f21ca4821ed server_cookie=96a0f65ae89a2d5b in_seq=0 out_seq=0 2023-07-14T11:29:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.302+0000 7fc87dffb700 10 client.26819 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:10.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc87dffb700 1 -- 172.21.15.161:0/2451281139 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5617a9860eb0 con 0x5617a99300d0 2023-07-14T11:29:10.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc87dffb700 10 client.26819 handle_command_reply: tid=0 2023-07-14T11:29:10.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 client.26819 shutdown 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a99152d0 msgr2=0x5617a9920560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a99152d0 0x5617a9920560 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc888096420 tx=0x7fc888010ab0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 shutdown_connections 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a99300d0 0x5617a99324b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9920ad0 0x5617a991df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a99152d0 0x5617a9920560 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/2451281139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a9915be0 0x5617a991d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc87dffb700 0 client.26819 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:10.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 >> 172.21.15.161:0/2451281139 conn(0x5617a990bda0 msgr2=0x5617a9911290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:10.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 shutdown_connections 2023-07-14T11:29:10.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/2451281139 wait complete. 2023-07-14T11:29:10.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.304+0000 7fc8b09eeb80 20 client.26819 trim_cache size 0 max 16384 2023-07-14T11:29:10.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 Processor -- start 2023-07-14T11:29:10.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 -- start start 2023-07-14T11:29:10.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9920100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a991aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a991afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a9912d20 con 0x5617a9915be0 2023-07-14T11:29:10.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a9916540 con 0x5617a9920ad0 2023-07-14T11:29:10.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a990b970 con 0x5617a99152d0 2023-07-14T11:29:10.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc89659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9920100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.322+0000 7fc896d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a991afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9920100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:39726/0 (socket says 172.21.15.161:39726) 2023-07-14T11:29:10.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc895d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a991aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 -- 172.21.15.161:0/576921073 learned_addr learned my addr 172.21.15.161:0/576921073 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:10.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 -- 172.21.15.161:0/576921073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 msgr2=0x5617a991afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 --2- 172.21.15.161:0/576921073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a991afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 -- 172.21.15.161:0/576921073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 msgr2=0x5617a991aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 --2- 172.21.15.161:0/576921073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a991aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 -- 172.21.15.161:0/576921073 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617a967ec60 con 0x5617a99152d0 2023-07-14T11:29:10.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc89659c700 1 --2- 172.21.15.161:0/576921073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9920100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e970 tx=0x7fc888010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f214d5eba3812306 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:10.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc8857fa700 1 -- 172.21.15.161:0/576921073 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888095f10 con 0x5617a99152d0 2023-07-14T11:29:10.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc8857fa700 1 -- 172.21.15.161:0/576921073 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc888065420 con 0x5617a99152d0 2023-07-14T11:29:10.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.323+0000 7fc8857fa700 1 -- 172.21.15.161:0/576921073 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888099670 con 0x5617a99152d0 2023-07-14T11:29:10.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/576921073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 msgr2=0x5617a9920100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/576921073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9920100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e970 tx=0x7fc888010c80 comp rx=0 tx=0).stop 2023-07-14T11:29:10.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/576921073 shutdown_connections 2023-07-14T11:29:10.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/576921073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a991afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/576921073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9920100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/576921073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a991aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/576921073 >> 172.21.15.161:0/576921073 conn(0x5617a990bda0 msgr2=0x5617a9918b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:10.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/576921073 shutdown_connections 2023-07-14T11:29:10.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/576921073 wait complete. 2023-07-14T11:29:10.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.324+0000 7fc8b09eeb80 1 -- start start 2023-07-14T11:29:10.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 1 Processor -- start 2023-07-14T11:29:10.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9925680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a9925bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a9982f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a99834a0 con 0x5617a9915be0 2023-07-14T11:29:10.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a99835e0 con 0x5617a9920ad0 2023-07-14T11:29:10.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a99837a0 con 0x5617a99152d0 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc8b09eeb80 10 client.0 _ll_register_callbacks cb 0x7fc888005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc895d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a9925bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc896d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a9982f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.325+0000 7fc89659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9925680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc896d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a9982f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46284/0 (socket says 172.21.15.161:46284) 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc896d9d700 1 -- 172.21.15.161:0/3442927881 learned_addr learned my addr 172.21.15.161:0/3442927881 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc895d9b700 1 -- 172.21.15.161:0/3442927881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 msgr2=0x5617a9925680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc895d9b700 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9925680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc895d9b700 1 -- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 msgr2=0x5617a9982f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:10.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc895d9b700 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a9982f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc895d9b700 1 -- 172.21.15.161:0/3442927881 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8981a4c80 con 0x5617a9915be0 2023-07-14T11:29:10.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc89659c700 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9925680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:29:10.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc896d9d700 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a9982f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:29:10.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc895d9b700 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a9925bc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b2e0 tx=0x7fc88c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae798c1ca1ed1ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:10.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.326+0000 7fc87dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:29:10.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc87dffb700 1 -- 172.21.15.161:0/3442927881 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c01d040 con 0x5617a9915be0 2023-07-14T11:29:10.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617a9823050 con 0x5617a9915be0 2023-07-14T11:29:10.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc87dffb700 1 -- 172.21.15.161:0/3442927881 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fc88c0290b0 con 0x5617a9915be0 2023-07-14T11:29:10.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc87dffb700 1 -- 172.21.15.161:0/3442927881 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc88c029270 con 0x5617a9915be0 2023-07-14T11:29:10.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 10 client.16776 fetch_fsmap learned FSMap version 26 2023-07-14T11:29:10.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5617a96bc250 con 0x5617a9915be0 2023-07-14T11:29:10.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc87dffb700 1 -- 172.21.15.161:0/3442927881 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617a9823050 con 0x5617a9915be0 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc87dffb700 1 -- 172.21.15.161:0/3442927881 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc88c004690 con 0x5617a9915be0 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 10 client.16776 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 20 client.16776 populate_metadata read hostname 'smithi161' 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a9984e00 0x5617a9987210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 4 client.16776 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.327+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5617a96b6f70 con 0x5617a9984e00 2023-07-14T11:29:10.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc89659c700 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a9984e00 0x5617a9987210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:10.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc89659c700 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a9984e00 0x5617a9987210 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331de3d39239cd5e server_cookie=a0312dec78b9bbb9 in_seq=0 out_seq=0 2023-07-14T11:29:10.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc87dffb700 10 client.16776 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:10.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc87dffb700 1 -- 172.21.15.161:0/3442927881 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5617a96b6f70 con 0x5617a9984e00 2023-07-14T11:29:10.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc87dffb700 10 client.16776 handle_command_reply: tid=0 2023-07-14T11:29:10.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc8b09eeb80 1 client.16776 shutdown 2023-07-14T11:29:10.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 msgr2=0x5617a9925bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.328+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a9925bc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b2e0 tx=0x7fc88c00bf50 comp rx=0 tx=0).stop 2023-07-14T11:29:10.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 shutdown_connections 2023-07-14T11:29:10.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617a9984e00 0x5617a9987210 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a9920ad0 0x5617a9982f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617a99152d0 0x5617a9925680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc87dffb700 0 client.16776 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:10.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 --2- 172.21.15.161:0/3442927881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617a9915be0 0x5617a9925bc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 >> 172.21.15.161:0/3442927881 conn(0x5617a990bda0 msgr2=0x5617a99171a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:10.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 shutdown_connections 2023-07-14T11:29:10.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 1 -- 172.21.15.161:0/3442927881 wait complete. 2023-07-14T11:29:10.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.329+0000 7fc8b09eeb80 20 client.16776 trim_cache size 0 max 16384 2023-07-14T11:29:10.341 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:29:10.341 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:29:10.341 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:29:10.342 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:29:10.342 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:29:10.342 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:29:10.342 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:29:10.342 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:29:10.342 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:29:10.342 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:29:10.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8800a12c0 msgr2=0x7fc8800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8800a12c0 0x7fc8800a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc890009fd0 tx=0x7fc89000b040 comp rx=0 tx=0).stop 2023-07-14T11:29:10.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 msgr2=0x7fc898197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:10.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc898197e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e9a0 tx=0x7fc88800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 shutdown_connections 2023-07-14T11:29:10.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc8800a12c0 0x7fc8800a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc898105ae0 0x7fc89819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8980ffad0 0x7fc898197e50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 --2- 172.21.15.161:0/1743926130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898100480 0x7fc898198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:10.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 >> 172.21.15.161:0/1743926130 conn(0x7fc8980fb5f0 msgr2=0x7fc89810aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:10.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 shutdown_connections 2023-07-14T11:29:10.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:10.330+0000 7fc89cfff700 1 -- 172.21.15.161:0/1743926130 wait complete. 2023-07-14T11:29:10.351 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:29:10.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:10 smithi182 ceph-mon[129366]: pgmap v211: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-07-14T11:29:10.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:10 smithi161 ceph-mon[95526]: pgmap v211: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-07-14T11:29:10.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:10 smithi175 ceph-mon[129081]: pgmap v211: 129 pgs: 129 active+clean; 424 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-07-14T11:29:11.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 630532 9.37 MB/sec execute 233 sec latency 38.600 ms 2023-07-14T11:29:12.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 632380 9.38 MB/sec execute 234 sec latency 63.905 ms 2023-07-14T11:29:12.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:12 smithi182 ceph-mon[129366]: pgmap v212: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2023-07-14T11:29:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:12 smithi161 ceph-mon[95526]: pgmap v212: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2023-07-14T11:29:12.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:12 smithi175 ceph-mon[129081]: pgmap v212: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2023-07-14T11:29:13.190 INFO:tasks.workunit.client.0.smithi161.stdout: 1 633551 9.37 MB/sec execute 235 sec latency 15.561 ms 2023-07-14T11:29:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:13 smithi175 ceph-mon[129081]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T11:29:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:13 smithi175 ceph-mon[129081]: pgmap v214: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 217 op/s 2023-07-14T11:29:14.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 635826 9.38 MB/sec execute 236 sec latency 55.763 ms 2023-07-14T11:29:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:13 smithi182 ceph-mon[129366]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T11:29:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:13 smithi182 ceph-mon[129366]: pgmap v214: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 217 op/s 2023-07-14T11:29:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:13 smithi161 ceph-mon[95526]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T11:29:14.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:13 smithi161 ceph-mon[95526]: pgmap v214: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 217 op/s 2023-07-14T11:29:15.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 637119 9.37 MB/sec execute 237 sec latency 14.852 ms 2023-07-14T11:29:16.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 639387 9.38 MB/sec execute 238 sec latency 18.087 ms 2023-07-14T11:29:16.199 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:16 smithi182 ceph-mon[129366]: pgmap v215: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 124 op/s 2023-07-14T11:29:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:16 smithi161 ceph-mon[95526]: pgmap v215: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 124 op/s 2023-07-14T11:29:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:16 smithi175 ceph-mon[129081]: pgmap v215: 129 pgs: 129 active+clean; 432 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 124 op/s 2023-07-14T11:29:17.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 640174 9.35 MB/sec execute 239 sec latency 58.264 ms 2023-07-14T11:29:18.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 642444 9.36 MB/sec execute 240 sec latency 24.827 ms 2023-07-14T11:29:18.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:18 smithi182 ceph-mon[129366]: pgmap v216: 129 pgs: 129 active+clean; 439 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-14T11:29:18.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:18 smithi161 ceph-mon[95526]: pgmap v216: 129 pgs: 129 active+clean; 439 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-14T11:29:18.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:18 smithi175 ceph-mon[129081]: pgmap v216: 129 pgs: 129 active+clean; 439 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-14T11:29:19.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 643686 9.35 MB/sec execute 241 sec latency 11.309 ms 2023-07-14T11:29:20.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 646009 9.37 MB/sec execute 242 sec latency 26.536 ms 2023-07-14T11:29:20.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:20 smithi182 ceph-mon[129366]: pgmap v217: 129 pgs: 129 active+clean; 439 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-14T11:29:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:20 smithi161 ceph-mon[95526]: pgmap v217: 129 pgs: 129 active+clean; 439 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-14T11:29:20.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:20 smithi175 ceph-mon[129081]: pgmap v217: 129 pgs: 129 active+clean; 439 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-14T11:29:21.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 647033 9.35 MB/sec execute 243 sec latency 73.333 ms 2023-07-14T11:29:22.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 648746 9.34 MB/sec execute 244 sec latency 52.468 ms 2023-07-14T11:29:22.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:22 smithi182 ceph-mon[129366]: pgmap v218: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-14T11:29:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:22 smithi161 ceph-mon[95526]: pgmap v218: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-14T11:29:22.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:22 smithi175 ceph-mon[129081]: pgmap v218: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-14T11:29:23.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 650357 9.35 MB/sec execute 245 sec latency 23.462 ms 2023-07-14T11:29:24.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:23 smithi175 ceph-mon[129081]: pgmap v219: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 229 op/s 2023-07-14T11:29:24.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 652260 9.34 MB/sec execute 246 sec latency 17.376 ms 2023-07-14T11:29:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:23 smithi182 ceph-mon[129366]: pgmap v219: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 229 op/s 2023-07-14T11:29:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:23 smithi161 ceph-mon[95526]: pgmap v219: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 229 op/s 2023-07-14T11:29:25.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 653946 9.35 MB/sec execute 247 sec latency 11.551 ms 2023-07-14T11:29:26.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 655768 9.34 MB/sec execute 248 sec latency 17.262 ms 2023-07-14T11:29:26.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:26 smithi182 ceph-mon[129366]: pgmap v220: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2023-07-14T11:29:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:26 smithi161 ceph-mon[95526]: pgmap v220: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2023-07-14T11:29:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:26 smithi175 ceph-mon[129081]: pgmap v220: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2023-07-14T11:29:27.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 657373 9.36 MB/sec execute 249 sec latency 75.424 ms 2023-07-14T11:29:28.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 658418 9.34 MB/sec execute 250 sec latency 66.365 ms 2023-07-14T11:29:28.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:28 smithi182 ceph-mon[129366]: pgmap v221: 129 pgs: 129 active+clean; 457 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 299 op/s 2023-07-14T11:29:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:28 smithi161 ceph-mon[95526]: pgmap v221: 129 pgs: 129 active+clean; 457 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 299 op/s 2023-07-14T11:29:28.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:28 smithi175 ceph-mon[129081]: pgmap v221: 129 pgs: 129 active+clean; 457 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 299 op/s 2023-07-14T11:29:29.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 660678 9.35 MB/sec execute 251 sec latency 28.235 ms 2023-07-14T11:29:30.191 INFO:tasks.workunit.client.0.smithi161.stdout: 1 661933 9.34 MB/sec execute 252 sec latency 12.645 ms 2023-07-14T11:29:30.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:30 smithi182 ceph-mon[129366]: pgmap v222: 129 pgs: 129 active+clean; 457 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2023-07-14T11:29:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:30 smithi161 ceph-mon[95526]: pgmap v222: 129 pgs: 129 active+clean; 457 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2023-07-14T11:29:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:30 smithi175 ceph-mon[129081]: pgmap v222: 129 pgs: 129 active+clean; 457 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2023-07-14T11:29:31.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 664082 9.35 MB/sec execute 253 sec latency 16.801 ms 2023-07-14T11:29:31.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:29:31.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:29:31.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:29:32.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 665042 9.33 MB/sec execute 254 sec latency 33.322 ms 2023-07-14T11:29:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:32 smithi161 ceph-mon[95526]: pgmap v223: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 297 op/s 2023-07-14T11:29:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:29:32.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:29:32.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:29:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:32 smithi175 ceph-mon[129081]: pgmap v223: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 297 op/s 2023-07-14T11:29:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:29:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:29:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:29:32.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:32 smithi182 ceph-mon[129366]: pgmap v223: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 297 op/s 2023-07-14T11:29:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:29:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:29:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:29:33.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 666948 9.33 MB/sec execute 255 sec latency 43.852 ms 2023-07-14T11:29:34.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:33 smithi175 ceph-mon[129081]: pgmap v224: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2023-07-14T11:29:34.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 668299 9.33 MB/sec execute 256 sec latency 20.638 ms 2023-07-14T11:29:34.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:33 smithi182 ceph-mon[129366]: pgmap v224: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2023-07-14T11:29:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:33 smithi161 ceph-mon[95526]: pgmap v224: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2023-07-14T11:29:35.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 670218 9.32 MB/sec execute 257 sec latency 17.302 ms 2023-07-14T11:29:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 671765 9.33 MB/sec execute 258 sec latency 15.394 ms 2023-07-14T11:29:36.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:36 smithi182 ceph-mon[129366]: pgmap v225: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2023-07-14T11:29:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:36 smithi161 ceph-mon[95526]: pgmap v225: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2023-07-14T11:29:36.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:36 smithi175 ceph-mon[129081]: pgmap v225: 129 pgs: 129 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2023-07-14T11:29:37.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 673636 9.32 MB/sec execute 259 sec latency 40.822 ms 2023-07-14T11:29:38.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 675206 9.33 MB/sec execute 260 sec latency 45.659 ms 2023-07-14T11:29:38.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:38 smithi182 ceph-mon[129366]: pgmap v226: 129 pgs: 129 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2023-07-14T11:29:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:38 smithi161 ceph-mon[95526]: pgmap v226: 129 pgs: 129 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2023-07-14T11:29:38.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:38 smithi175 ceph-mon[129081]: pgmap v226: 129 pgs: 129 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2023-07-14T11:29:39.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 676967 9.32 MB/sec execute 261 sec latency 20.805 ms 2023-07-14T11:29:40.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 678773 9.33 MB/sec execute 262 sec latency 10.758 ms 2023-07-14T11:29:40.353 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:29:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:40 smithi182 ceph-mon[129366]: pgmap v227: 129 pgs: 129 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2023-07-14T11:29:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:40 smithi161 ceph-mon[95526]: pgmap v227: 129 pgs: 129 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2023-07-14T11:29:40.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 -- 172.21.15.161:0/1993144916 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880731c0 msgr2=0x7f32881105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 --2- 172.21.15.161:0/1993144916 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880731c0 0x7f32881105b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f32780098f0 tx=0x7f3278009c00 comp rx=0 tx=0).stop 2023-07-14T11:29:40.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 -- 172.21.15.161:0/1993144916 shutdown_connections 2023-07-14T11:29:40.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 --2- 172.21.15.161:0/1993144916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3288110af0 0x7f3288112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 --2- 172.21.15.161:0/1993144916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32880728b0 0x7f3288072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 --2- 172.21.15.161:0/1993144916 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880731c0 0x7f32881105b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 -- 172.21.15.161:0/1993144916 >> 172.21.15.161:0/1993144916 conn(0x7f32880fb5e0 msgr2=0x7f32880fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:40.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 -- 172.21.15.161:0/1993144916 shutdown_connections 2023-07-14T11:29:40.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.581+0000 7f328ffff700 1 -- 172.21.15.161:0/1993144916 wait complete. 2023-07-14T11:29:40.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 Processor -- start 2023-07-14T11:29:40.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 -- start start 2023-07-14T11:29:40.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880728b0 0x7f328819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32880731c0 0x7f328819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3288110af0 0x7f32881a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32881a1270 con 0x7f3288110af0 2023-07-14T11:29:40.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32881a13e0 con 0x7f32880731c0 2023-07-14T11:29:40.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.582+0000 7f328ffff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32881a15a0 con 0x7f32880728b0 2023-07-14T11:29:40.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880728b0 0x7f328819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328d59a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32880731c0 0x7f328819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880728b0 0x7f328819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36376/0 (socket says 172.21.15.161:36376) 2023-07-14T11:29:40.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328dd9b700 1 -- 172.21.15.161:0/2365437366 learned_addr learned my addr 172.21.15.161:0/2365437366 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:40.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328e59c700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3288110af0 0x7f32881a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328d59a700 1 -- 172.21.15.161:0/2365437366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880728b0 msgr2=0x7f328819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328d59a700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880728b0 0x7f328819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328d59a700 1 -- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3288110af0 msgr2=0x7f32881a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328d59a700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3288110af0 0x7f32881a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328d59a700 1 -- 172.21.15.161:0/2365437366 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840095f0 con 0x7f32880731c0 2023-07-14T11:29:40.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.583+0000 7f328d59a700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32880731c0 0x7f328819c7d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3278009810 tx=0x7f327800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:40.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.584+0000 7f327effd700 1 -- 172.21.15.161:0/2365437366 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327801c070 con 0x7f32880731c0 2023-07-14T11:29:40.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.584+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3278009520 con 0x7f32880731c0 2023-07-14T11:29:40.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.584+0000 7f327effd700 1 -- 172.21.15.161:0/2365437366 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f327800bba0 con 0x7f32880731c0 2023-07-14T11:29:40.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.584+0000 7f327effd700 1 -- 172.21.15.161:0/2365437366 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278003970 con 0x7f32880731c0 2023-07-14T11:29:40.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.584+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32881a8ee0 con 0x7f32880731c0 2023-07-14T11:29:40.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.587+0000 7f327effd700 1 -- 172.21.15.161:0/2365437366 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f3278020030 con 0x7f32880731c0 2023-07-14T11:29:40.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 Processor -- start 2023-07-14T11:29:40.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f327effd700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32740a12c0 0x7f32740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f327effd700 1 -- 172.21.15.161:0/2365437366 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f32780cdea0 con 0x7f32880731c0 2023-07-14T11:29:40.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 -- start start 2023-07-14T11:29:40.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f328e59c700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32740a12c0 0x7f32740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8073b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80746590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80748eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c80641cb0 con 0x564c80746ad0 2023-07-14T11:29:40.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c80694a40 con 0x564c8073bbe0 2023-07-14T11:29:40.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.589+0000 7f32a3886b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8068ded0 con 0x564c8073b2d0 2023-07-14T11:29:40.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8073b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8073b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36386/0 (socket says 172.21.15.161:36386) 2023-07-14T11:29:40.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 -- 172.21.15.161:0/230873590 learned_addr learned my addr 172.21.15.161:0/230873590 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:40.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328d59a700 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80746590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328e59c700 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80748eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 -- 172.21.15.161:0/230873590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 msgr2=0x564c80746590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80746590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 -- 172.21.15.161:0/230873590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 msgr2=0x564c80748eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80748eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 -- 172.21.15.161:0/230873590 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c804bbde0 con 0x564c8073b2d0 2023-07-14T11:29:40.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328e59c700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32740a12c0 0x7f32740a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3288069a80 tx=0x7f3280009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:40.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f328dd9b700 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8073b6a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3284011510 tx=0x7f3284011820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=855182e3bf8c948e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:40.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.590+0000 7f3273fff700 1 -- 172.21.15.161:0/230873590 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284019630 con 0x564c8073b2d0 2023-07-14T11:29:40.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.591+0000 7f3273fff700 1 -- 172.21.15.161:0/230873590 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3284019c50 con 0x564c8073b2d0 2023-07-14T11:29:40.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.591+0000 7f3273fff700 1 -- 172.21.15.161:0/230873590 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400ed10 con 0x564c8073b2d0 2023-07-14T11:29:40.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.591+0000 7f32a3886b80 1 -- 172.21.15.161:0/230873590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 msgr2=0x564c8073b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.591+0000 7f32a3886b80 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8073b6a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3284011510 tx=0x7f3284011820 comp rx=0 tx=0).stop 2023-07-14T11:29:40.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 -- 172.21.15.161:0/230873590 shutdown_connections 2023-07-14T11:29:40.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80748eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8073b6a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 --2- 172.21.15.161:0/230873590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80746590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 -- 172.21.15.161:0/230873590 >> 172.21.15.161:0/230873590 conn(0x564c80731da0 msgr2=0x564c807376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:40.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 -- 172.21.15.161:0/230873590 shutdown_connections 2023-07-14T11:29:40.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 -- 172.21.15.161:0/230873590 wait complete. 2023-07-14T11:29:40.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.592+0000 7f32a3886b80 1 -- start start 2023-07-14T11:29:40.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 1 Processor -- start 2023-07-14T11:29:40.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80746580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80743790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80743ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8062dac0 con 0x564c80746ad0 2023-07-14T11:29:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8062d7c0 con 0x564c8073bbe0 2023-07-14T11:29:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c80511920 con 0x564c8073b2d0 2023-07-14T11:29:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f32a3886b80 10 client.0 _ll_register_callbacks cb 0x564c8062b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:29:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80746580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f328d59a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80743790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80746580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36402/0 (socket says 172.21.15.161:36402) 2023-07-14T11:29:40.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f328dd9b700 1 -- 172.21.15.161:0/3870239667 learned_addr learned my addr 172.21.15.161:0/3870239667 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:40.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.593+0000 7f328e59c700 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80743ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f328dd9b700 1 -- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 msgr2=0x564c80743790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f328dd9b700 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80743790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f328dd9b700 1 -- 172.21.15.161:0/3870239667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 msgr2=0x564c80743ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f328dd9b700 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80743ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f328dd9b700 1 -- 172.21.15.161:0/3870239667 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3284009cf0 con 0x564c8073b2d0 2023-07-14T11:29:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f328d59a700 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80743790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:29:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f328dd9b700 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80746580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f32840046e0 tx=0x7f328400ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c1f2c5d54e96e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f32717fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:29:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f32717fa700 1 -- 172.21.15.161:0/3870239667 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284019840 con 0x564c8073b2d0 2023-07-14T11:29:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c805e74f0 con 0x564c8073b2d0 2023-07-14T11:29:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.594+0000 7f32717fa700 1 -- 172.21.15.161:0/3870239667 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f3284011e40 con 0x564c8073b2d0 2023-07-14T11:29:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32717fa700 1 -- 172.21.15.161:0/3870239667 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f328400e3b0 con 0x564c8073b2d0 2023-07-14T11:29:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 10 client.26839 fetch_fsmap learned FSMap version 26 2023-07-14T11:29:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564c804a4c60 con 0x564c8073b2d0 2023-07-14T11:29:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32717fa700 1 -- 172.21.15.161:0/3870239667 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c805e74f0 con 0x564c8073b2d0 2023-07-14T11:29:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32717fa700 1 -- 172.21.15.161:0/3870239667 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f32840049b0 con 0x564c8073b2d0 2023-07-14T11:29:40.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 10 client.26839 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:29:40.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:29:40.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 20 client.26839 populate_metadata read hostname 'smithi161' 2023-07-14T11:29:40.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807562f0 0x564c80758700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 4 client.26839 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:29:40.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.595+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c80686eb0 con 0x564c807562f0 2023-07-14T11:29:40.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.596+0000 7f328d59a700 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807562f0 0x564c80758700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.596+0000 7f32717fa700 10 client.26839 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:40.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.596+0000 7f328d59a700 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807562f0 0x564c80758700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c00878c7d85cb24 server_cookie=e427f17f642387e0 in_seq=0 out_seq=0 2023-07-14T11:29:40.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32717fa700 1 -- 172.21.15.161:0/3870239667 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c80686eb0 con 0x564c807562f0 2023-07-14T11:29:40.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32717fa700 10 client.26839 handle_command_reply: tid=0 2023-07-14T11:29:40.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 client.26839 shutdown 2023-07-14T11:29:40.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 msgr2=0x564c80746580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80746580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f32840046e0 tx=0x7f328400ef70 comp rx=0 tx=0).stop 2023-07-14T11:29:40.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 shutdown_connections 2023-07-14T11:29:40.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807562f0 0x564c80758700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c80746ad0 0x564c80743ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80746580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32717fa700 0 client.26839 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:40.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 --2- 172.21.15.161:0/3870239667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c8073bbe0 0x564c80743790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 >> 172.21.15.161:0/3870239667 conn(0x564c80731da0 msgr2=0x564c80737230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:40.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 shutdown_connections 2023-07-14T11:29:40.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.597+0000 7f32a3886b80 1 -- 172.21.15.161:0/3870239667 wait complete. 2023-07-14T11:29:40.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.598+0000 7f32a3886b80 20 client.26839 trim_cache size 0 max 16384 2023-07-14T11:29:40.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 Processor -- start 2023-07-14T11:29:40.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 -- start start 2023-07-14T11:29:40.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80743740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c807460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c80740a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c80738d70 con 0x564c8073bbe0 2023-07-14T11:29:40.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8073bfe0 con 0x564c80746ad0 2023-07-14T11:29:40.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f32a3886b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c80738540 con 0x564c8073b2d0 2023-07-14T11:29:40.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80743740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80743740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36406/0 (socket says 172.21.15.161:36406) 2023-07-14T11:29:40.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f328dd9b700 1 -- 172.21.15.161:0/475567297 learned_addr learned my addr 172.21.15.161:0/475567297 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:40.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f328e59c700 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c80740a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.617+0000 7f328d59a700 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c807460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f328dd9b700 1 -- 172.21.15.161:0/475567297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 msgr2=0x564c80740a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f328dd9b700 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c80740a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f328dd9b700 1 -- 172.21.15.161:0/475567297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 msgr2=0x564c807460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f328dd9b700 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c807460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f328dd9b700 1 -- 172.21.15.161:0/475567297 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c804e2250 con 0x564c8073b2d0 2023-07-14T11:29:40.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f328dd9b700 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80743740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3284005b40 tx=0x7f3284021360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7011ad0170a78f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:40.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f327cff9700 1 -- 172.21.15.161:0/475567297 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400e3b0 con 0x564c8073b2d0 2023-07-14T11:29:40.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f327cff9700 1 -- 172.21.15.161:0/475567297 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3284004710 con 0x564c8073b2d0 2023-07-14T11:29:40.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.618+0000 7f327cff9700 1 -- 172.21.15.161:0/475567297 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284005240 con 0x564c8073b2d0 2023-07-14T11:29:40.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 -- 172.21.15.161:0/475567297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 msgr2=0x564c80743740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80743740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3284005b40 tx=0x7f3284021360 comp rx=0 tx=0).stop 2023-07-14T11:29:40.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 -- 172.21.15.161:0/475567297 shutdown_connections 2023-07-14T11:29:40.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c80740a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c80743740 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 --2- 172.21.15.161:0/475567297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c807460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 -- 172.21.15.161:0/475567297 >> 172.21.15.161:0/475567297 conn(0x564c80731da0 msgr2=0x564c80743030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:40.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 -- 172.21.15.161:0/475567297 shutdown_connections 2023-07-14T11:29:40.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.619+0000 7f32a3886b80 1 -- 172.21.15.161:0/475567297 wait complete. 2023-07-14T11:29:40.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.620+0000 7f32a3886b80 1 -- start start 2023-07-14T11:29:40.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.620+0000 7f32a3886b80 1 Processor -- start 2023-07-14T11:29:40.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8074d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c8074b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f32a3886b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c8074bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f32a3886b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8074c230 con 0x564c8073bbe0 2023-07-14T11:29:40.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f32a3886b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8074c370 con 0x564c80746ad0 2023-07-14T11:29:40.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f32a3886b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8074c530 con 0x564c8073b2d0 2023-07-14T11:29:40.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f32a3886b80 10 client.0 _ll_register_callbacks cb 0x564c8062b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:29:40.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8074d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f328dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8074d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36418/0 (socket says 172.21.15.161:36418) 2023-07-14T11:29:40.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f328dd9b700 1 -- 172.21.15.161:0/207514679 learned_addr learned my addr 172.21.15.161:0/207514679 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:29:40.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f328e59c700 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c8074bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.621+0000 7f328d59a700 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c8074b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.622+0000 7f328e59c700 1 -- 172.21.15.161:0/207514679 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 msgr2=0x564c8074d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.622+0000 7f328e59c700 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8074d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.622+0000 7f328e59c700 1 -- 172.21.15.161:0/207514679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 msgr2=0x564c8074b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:29:40.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.622+0000 7f328e59c700 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c8074b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.622+0000 7f328e59c700 1 -- 172.21.15.161:0/207514679 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c804a4c60 con 0x564c80746ad0 2023-07-14T11:29:40.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.622+0000 7f328e59c700 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c8074bca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f328000ffd0 tx=0x7f3280004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b152361653bf90 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:29:40.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.623+0000 7f32717fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:29:40.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.623+0000 7f32717fa700 1 -- 172.21.15.161:0/207514679 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000e040 con 0x564c80746ad0 2023-07-14T11:29:40.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.623+0000 7f32717fa700 1 -- 172.21.15.161:0/207514679 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f3280003c00 con 0x564c80746ad0 2023-07-14T11:29:40.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.623+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3284019c50 con 0x564c80746ad0 2023-07-14T11:29:40.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.623+0000 7f32717fa700 1 -- 172.21.15.161:0/207514679 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3280017e90 con 0x564c80746ad0 2023-07-14T11:29:40.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.623+0000 7f32a3886b80 10 client.26843 fetch_fsmap learned FSMap version 26 2023-07-14T11:29:40.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.623+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564c804ba2b0 con 0x564c80746ad0 2023-07-14T11:29:40.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32717fa700 1 -- 172.21.15.161:0/207514679 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3284019c50 con 0x564c80746ad0 2023-07-14T11:29:40.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32717fa700 1 -- 172.21.15.161:0/207514679 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f32800235d0 con 0x564c80746ad0 2023-07-14T11:29:40.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32a3886b80 10 client.26843 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:29:40.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32a3886b80 10 client.26843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:29:40.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32a3886b80 20 client.26843 populate_metadata read hostname 'smithi161' 2023-07-14T11:29:40.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32a3886b80 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807ab690 0x564c807adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:29:40.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32a3886b80 4 client.26843 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:29:40.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c80649050 con 0x564c807ab690 2023-07-14T11:29:40.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.624+0000 7f328dd9b700 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807ab690 0x564c807adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:29:40.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.625+0000 7f32717fa700 10 client.26843 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:40.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.625+0000 7f328dd9b700 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807ab690 0x564c807adaa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1e1dc2f5f1ccff server_cookie=be3d99a3836fe417 in_seq=0 out_seq=0 2023-07-14T11:29:40.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.625+0000 7f32717fa700 1 -- 172.21.15.161:0/207514679 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564c80649050 con 0x564c807ab690 2023-07-14T11:29:40.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.625+0000 7f32717fa700 10 client.26843 handle_command_reply: tid=0 2023-07-14T11:29:40.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.625+0000 7f32a3886b80 1 client.26843 shutdown 2023-07-14T11:29:40.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 msgr2=0x564c8074bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c8074bca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f328000ffd0 tx=0x7f3280004b40 comp rx=0 tx=0).stop 2023-07-14T11:29:40.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 shutdown_connections 2023-07-14T11:29:40.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x564c807ab690 0x564c807adaa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c80746ad0 0x564c8074bca0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x564c8073b2d0 0x564c8074d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32717fa700 0 client.26843 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:29:40.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 --2- 172.21.15.161:0/207514679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8073bbe0 0x564c8074b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 >> 172.21.15.161:0/207514679 conn(0x564c80731da0 msgr2=0x564c80737140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:40.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 shutdown_connections 2023-07-14T11:29:40.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 1 -- 172.21.15.161:0/207514679 wait complete. 2023-07-14T11:29:40.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.626+0000 7f32a3886b80 20 client.26843 trim_cache size 0 max 16384 2023-07-14T11:29:40.638 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:29:40.638 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:29:40.639 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:29:40.639 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:29:40.639 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:29:40.639 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:29:40.639 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:29:40.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:29:40.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:29:40.640 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:29:40.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32740a12c0 msgr2=0x7f32740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32740a12c0 0x7f32740a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3288069a80 tx=0x7f3280009920 comp rx=0 tx=0).stop 2023-07-14T11:29:40.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32880731c0 msgr2=0x7f328819c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:29:40.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32880731c0 0x7f328819c7d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3278009810 tx=0x7f327800fa00 comp rx=0 tx=0).stop 2023-07-14T11:29:40.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 shutdown_connections 2023-07-14T11:29:40.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f32740a12c0 0x7f32740a3770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3288110af0 0x7f32881a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f32880728b0 0x7f328819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 --2- 172.21.15.161:0/2365437366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32880731c0 0x7f328819c7d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:29:40.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 >> 172.21.15.161:0/2365437366 conn(0x7f32880fb5e0 msgr2=0x7f32880fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:29:40.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 shutdown_connections 2023-07-14T11:29:40.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:29:40.627+0000 7f328ffff700 1 -- 172.21.15.161:0/2365437366 wait complete. 2023-07-14T11:29:40.648 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:29:40.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:40 smithi175 ceph-mon[129081]: pgmap v227: 129 pgs: 129 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2023-07-14T11:29:41.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 680487 9.32 MB/sec execute 263 sec latency 20.589 ms 2023-07-14T11:29:42.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 682250 9.33 MB/sec execute 264 sec latency 14.222 ms 2023-07-14T11:29:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:42 smithi161 ceph-mon[95526]: pgmap v228: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2023-07-14T11:29:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:42 smithi175 ceph-mon[129081]: pgmap v228: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2023-07-14T11:29:42.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:42 smithi182 ceph-mon[129366]: pgmap v228: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2023-07-14T11:29:43.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 683310 9.32 MB/sec execute 265 sec latency 83.252 ms 2023-07-14T11:29:44.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:43 smithi175 ceph-mon[129081]: pgmap v229: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-14T11:29:44.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 685623 9.33 MB/sec execute 266 sec latency 16.096 ms 2023-07-14T11:29:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:43 smithi182 ceph-mon[129366]: pgmap v229: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-14T11:29:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:43 smithi161 ceph-mon[95526]: pgmap v229: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-14T11:29:45.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 686637 9.31 MB/sec execute 267 sec latency 44.567 ms 2023-07-14T11:29:46.192 INFO:tasks.workunit.client.0.smithi161.stdout: 1 689085 9.33 MB/sec execute 268 sec latency 18.390 ms 2023-07-14T11:29:46.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:46 smithi182 ceph-mon[129366]: pgmap v230: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-14T11:29:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:46 smithi161 ceph-mon[95526]: pgmap v230: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-14T11:29:46.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:46 smithi175 ceph-mon[129081]: pgmap v230: 129 pgs: 129 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-14T11:29:47.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 690206 9.31 MB/sec execute 269 sec latency 15.491 ms 2023-07-14T11:29:48.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 692457 9.33 MB/sec execute 270 sec latency 48.905 ms 2023-07-14T11:29:48.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:48 smithi182 ceph-mon[129366]: pgmap v231: 129 pgs: 129 active+clean; 490 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2023-07-14T11:29:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:48 smithi161 ceph-mon[95526]: pgmap v231: 129 pgs: 129 active+clean; 490 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2023-07-14T11:29:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:48 smithi175 ceph-mon[129081]: pgmap v231: 129 pgs: 129 active+clean; 490 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2023-07-14T11:29:49.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 693435 9.31 MB/sec execute 271 sec latency 37.024 ms 2023-07-14T11:29:50.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 695701 9.32 MB/sec execute 272 sec latency 30.044 ms 2023-07-14T11:29:50.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:50 smithi182 ceph-mon[129366]: pgmap v232: 129 pgs: 129 active+clean; 490 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 183 op/s 2023-07-14T11:29:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:50 smithi161 ceph-mon[95526]: pgmap v232: 129 pgs: 129 active+clean; 490 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 183 op/s 2023-07-14T11:29:50.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:50 smithi175 ceph-mon[129081]: pgmap v232: 129 pgs: 129 active+clean; 490 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 183 op/s 2023-07-14T11:29:51.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 696830 9.31 MB/sec execute 273 sec latency 20.573 ms 2023-07-14T11:29:52.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 698806 9.31 MB/sec execute 274 sec latency 73.828 ms 2023-07-14T11:29:52.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:52 smithi182 ceph-mon[129366]: pgmap v233: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2023-07-14T11:29:52.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:52 smithi161 ceph-mon[95526]: pgmap v233: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2023-07-14T11:29:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:52 smithi175 ceph-mon[129081]: pgmap v233: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2023-07-14T11:29:53.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 700031 9.31 MB/sec execute 275 sec latency 88.280 ms 2023-07-14T11:29:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:53 smithi175 ceph-mon[129081]: pgmap v234: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2023-07-14T11:29:54.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 701445 9.29 MB/sec execute 276 sec latency 47.219 ms 2023-07-14T11:29:54.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:53 smithi182 ceph-mon[129366]: pgmap v234: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2023-07-14T11:29:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:53 smithi161 ceph-mon[95526]: pgmap v234: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2023-07-14T11:29:55.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 703439 9.31 MB/sec execute 277 sec latency 30.975 ms 2023-07-14T11:29:56.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 704583 9.29 MB/sec execute 278 sec latency 76.843 ms 2023-07-14T11:29:56.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:56 smithi182 ceph-mon[129366]: pgmap v235: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2023-07-14T11:29:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:56 smithi161 ceph-mon[95526]: pgmap v235: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2023-07-14T11:29:56.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:56 smithi175 ceph-mon[129081]: pgmap v235: 129 pgs: 129 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2023-07-14T11:29:57.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 706898 9.31 MB/sec execute 279 sec latency 15.734 ms 2023-07-14T11:29:58.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 707861 9.28 MB/sec execute 280 sec latency 54.725 ms 2023-07-14T11:29:58.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:29:58 smithi182 ceph-mon[129366]: pgmap v236: 129 pgs: 129 active+clean; 504 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 290 op/s 2023-07-14T11:29:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:29:58 smithi161 ceph-mon[95526]: pgmap v236: 129 pgs: 129 active+clean; 504 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 290 op/s 2023-07-14T11:29:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:29:58 smithi175 ceph-mon[129081]: pgmap v236: 129 pgs: 129 active+clean; 504 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 290 op/s 2023-07-14T11:29:59.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 710170 9.31 MB/sec execute 281 sec latency 27.062 ms 2023-07-14T11:30:00.193 INFO:tasks.workunit.client.0.smithi161.stdout: 1 711168 9.28 MB/sec execute 282 sec latency 22.539 ms 2023-07-14T11:30:00.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:00 smithi182 ceph-mon[129366]: pgmap v237: 129 pgs: 129 active+clean; 504 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 191 op/s 2023-07-14T11:30:00.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:00 smithi182 ceph-mon[129366]: overall HEALTH_OK 2023-07-14T11:30:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:00 smithi161 ceph-mon[95526]: pgmap v237: 129 pgs: 129 active+clean; 504 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 191 op/s 2023-07-14T11:30:00.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:00 smithi161 ceph-mon[95526]: overall HEALTH_OK 2023-07-14T11:30:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:00 smithi175 ceph-mon[129081]: pgmap v237: 129 pgs: 129 active+clean; 504 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 191 op/s 2023-07-14T11:30:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:00 smithi175 ceph-mon[129081]: overall HEALTH_OK 2023-07-14T11:30:01.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 711411 9.25 MB/sec execute 283 sec latency 837.572 ms 2023-07-14T11:30:02.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 711411 9.22 MB/sec execute 284 sec latency 1837.653 ms 2023-07-14T11:30:02.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:02 smithi161 ceph-mon[95526]: pgmap v238: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-07-14T11:30:02.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:02 smithi175 ceph-mon[129081]: pgmap v238: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-07-14T11:30:02.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:02 smithi182 ceph-mon[129366]: pgmap v238: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-07-14T11:30:03.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 711411 9.19 MB/sec execute 285 sec latency 2837.788 ms 2023-07-14T11:30:04.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:03 smithi175 ceph-mon[129081]: pgmap v239: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:04.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 711411 9.16 MB/sec execute 286 sec latency 3837.893 ms 2023-07-14T11:30:04.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:03 smithi182 ceph-mon[129366]: pgmap v239: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:03 smithi161 ceph-mon[95526]: pgmap v239: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:05.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 711411 9.12 MB/sec execute 287 sec latency 4837.959 ms 2023-07-14T11:30:06.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 712508 9.11 MB/sec execute 288 sec latency 4921.696 ms 2023-07-14T11:30:06.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:06 smithi182 ceph-mon[129366]: pgmap v240: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:06 smithi161 ceph-mon[95526]: pgmap v240: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:06 smithi175 ceph-mon[129081]: pgmap v240: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:07.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 713596 9.10 MB/sec execute 289 sec latency 130.281 ms 2023-07-14T11:30:07.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:07 smithi161 ceph-mon[95526]: osdmap e96: 12 total, 12 up, 12 in 2023-07-14T11:30:07.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:07 smithi175 ceph-mon[129081]: osdmap e96: 12 total, 12 up, 12 in 2023-07-14T11:30:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:07 smithi182 ceph-mon[129366]: osdmap e96: 12 total, 12 up, 12 in 2023-07-14T11:30:08.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 714577 9.09 MB/sec execute 290 sec latency 75.301 ms 2023-07-14T11:30:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:08 smithi161 ceph-mon[95526]: pgmap v242: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2023-07-14T11:30:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:08 smithi175 ceph-mon[129081]: pgmap v242: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2023-07-14T11:30:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:08 smithi182 ceph-mon[129366]: pgmap v242: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2023-07-14T11:30:09.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 716539 9.10 MB/sec execute 291 sec latency 41.014 ms 2023-07-14T11:30:10.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 718046 9.09 MB/sec execute 292 sec latency 11.935 ms 2023-07-14T11:30:10.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:10 smithi161 ceph-mon[95526]: pgmap v243: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2023-07-14T11:30:10.649 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:30:10.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:10 smithi175 ceph-mon[129081]: pgmap v243: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2023-07-14T11:30:10.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:10 smithi182 ceph-mon[129366]: pgmap v243: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2023-07-14T11:30:10.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/804611920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8101f70 msgr2=0x7fd3c8102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:10.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/804611920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8101f70 0x7fd3c8102340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009990 tx=0x7fd3c4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/804611920 shutdown_connections 2023-07-14T11:30:10.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/804611920 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd3c8106fb0 0x7fd3c810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/804611920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8101f70 0x7fd3c8102340 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/804611920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c8102910 0x7fd3c81068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/804611920 >> 172.21.15.161:0/804611920 conn(0x7fd3c80fd7e0 msgr2=0x7fd3c80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:10.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/804611920 shutdown_connections 2023-07-14T11:30:10.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.871+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/804611920 wait complete. 2023-07-14T11:30:10.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 Processor -- start 2023-07-14T11:30:10.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 -- start start 2023-07-14T11:30:10.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c8101f70 0x7fd3c8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8102910 0x7fd3c8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd3c8106fb0 0x7fd3c819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c819ce30 con 0x7fd3c8101f70 2023-07-14T11:30:10.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c819cfa0 con 0x7fd3c8102910 2023-07-14T11:30:10.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.872+0000 7fd3cf9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c819d160 con 0x7fd3c8106fb0 2023-07-14T11:30:10.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3cd79b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c8101f70 0x7fd3c8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8102910 0x7fd3c8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8102910 0x7fd3c8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57706/0 (socket says 172.21.15.161:57706) 2023-07-14T11:30:10.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/3740714139 learned_addr learned my addr 172.21.15.161:0/3740714139 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:10.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3cdf9c700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd3c8106fb0 0x7fd3c819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/3740714139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd3c8106fb0 msgr2=0x7fd3c819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:10.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd3c8106fb0 0x7fd3c819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c8101f70 msgr2=0x7fd3c8197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:10.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c8101f70 0x7fd3c8197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/3740714139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c40095f0 con 0x7fd3c8102910 2023-07-14T11:30:10.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c8101f70 0x7fd3c8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:30:10.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8102910 0x7fd3c8198390 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800e940 tx=0x7fd3b800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:10.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.873+0000 7fd3be7fc700 1 -- 172.21.15.161:0/3740714139 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8009730 con 0x7fd3c8102910 2023-07-14T11:30:10.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.874+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c819d440 con 0x7fd3c8102910 2023-07-14T11:30:10.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.874+0000 7fd3be7fc700 1 -- 172.21.15.161:0/3740714139 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3b8010430 con 0x7fd3c8102910 2023-07-14T11:30:10.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.874+0000 7fd3be7fc700 1 -- 172.21.15.161:0/3740714139 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800cea0 con 0x7fd3c8102910 2023-07-14T11:30:10.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.874+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c81a4c80 con 0x7fd3c8102910 2023-07-14T11:30:10.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.877+0000 7fd3be7fc700 1 -- 172.21.15.161:0/3740714139 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd3b8010810 con 0x7fd3c8102910 2023-07-14T11:30:10.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.880+0000 7fd3e332eb80 1 Processor -- start 2023-07-14T11:30:10.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.880+0000 7fd3e332eb80 1 -- start start 2023-07-14T11:30:10.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.880+0000 7fd3be7fc700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd3b40a1340 0x7fd3b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df71eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df723f0 0x56013df7cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3be7fc700 1 -- 172.21.15.161:0/3740714139 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fd3b8014070 con 0x7fd3c8102910 2023-07-14T11:30:10.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013df7f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013de6fcb0 con 0x56013df7d2e0 2023-07-14T11:30:10.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013dec2a40 con 0x56013df723f0 2023-07-14T11:30:10.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013debbed0 con 0x56013df71ae0 2023-07-14T11:30:10.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df723f0 0x56013df7cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df723f0 0x56013df7cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57708/0 (socket says 172.21.15.161:57708) 2023-07-14T11:30:10.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/81611091 learned_addr learned my addr 172.21.15.161:0/81611091 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:10.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3cd79b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df71eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3cdf9c700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd3b40a1340 0x7fd3b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3cdf9c700 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013df7f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/81611091 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 msgr2=0x56013df71eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df71eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/81611091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 msgr2=0x56013df7f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013df7f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.881+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/81611091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56013dce9de0 con 0x56013df723f0 2023-07-14T11:30:10.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.882+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df723f0 0x56013df7cda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8096960 tx=0x7fd3b8099940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be77d75661eb8ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:10.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.882+0000 7fd3cdf9c700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd3b40a1340 0x7fd3b40a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009ec0 tx=0x7fd3c0009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:10.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.882+0000 7fd3b37fe700 1 -- 172.21.15.161:0/81611091 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80a2340 con 0x56013df723f0 2023-07-14T11:30:10.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.882+0000 7fd3b37fe700 1 -- 172.21.15.161:0/81611091 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3b8098290 con 0x56013df723f0 2023-07-14T11:30:10.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.882+0000 7fd3b37fe700 1 -- 172.21.15.161:0/81611091 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80a1eb0 con 0x56013df723f0 2023-07-14T11:30:10.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.883+0000 7fd3e332eb80 1 -- 172.21.15.161:0/81611091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df723f0 msgr2=0x56013df7cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:10.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.883+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df723f0 0x56013df7cda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8096960 tx=0x7fd3b8099940 comp rx=0 tx=0).stop 2023-07-14T11:30:10.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.884+0000 7fd3e332eb80 1 -- 172.21.15.161:0/81611091 shutdown_connections 2023-07-14T11:30:10.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.884+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013df7f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.884+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df71eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.884+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/81611091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df723f0 0x56013df7cda0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.884+0000 7fd3e332eb80 1 -- 172.21.15.161:0/81611091 >> 172.21.15.161:0/81611091 conn(0x56013df5fda0 msgr2=0x56013df656a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:10.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.884+0000 7fd3e332eb80 1 -- 172.21.15.161:0/81611091 shutdown_connections 2023-07-14T11:30:10.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.884+0000 7fd3e332eb80 1 -- 172.21.15.161:0/81611091 wait complete. 2023-07-14T11:30:10.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.885+0000 7fd3e332eb80 1 -- start start 2023-07-14T11:30:10.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.885+0000 7fd3e332eb80 1 Processor -- start 2023-07-14T11:30:10.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013df7a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df7a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df7d2e0 0x56013df7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013de5bac0 con 0x56013df723f0 2023-07-14T11:30:10.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013de5b7c0 con 0x56013df71ae0 2023-07-14T11:30:10.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013dd3f920 con 0x56013df7d2e0 2023-07-14T11:30:10.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3e332eb80 10 client.0 _ll_register_callbacks cb 0x56013de59760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:30:10.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3cdf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df7d2e0 0x56013df7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3cdf9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df7d2e0 0x56013df7ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34032/0 (socket says 172.21.15.161:34032) 2023-07-14T11:30:10.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3cdf9c700 1 -- 172.21.15.161:0/2000846094 learned_addr learned my addr 172.21.15.161:0/2000846094 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:10.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013df7a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.886+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df7a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.887+0000 7fd3cd79b700 1 -- 172.21.15.161:0/2000846094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df7d2e0 msgr2=0x56013df7ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.887+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df7d2e0 0x56013df7ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.887+0000 7fd3cd79b700 1 -- 172.21.15.161:0/2000846094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 msgr2=0x56013df7a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.887+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df7a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.887+0000 7fd3cd79b700 1 -- 172.21.15.161:0/2000846094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c81a4c80 con 0x56013df71ae0 2023-07-14T11:30:10.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.887+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df7a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:30:10.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013df7a080 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4000c00 tx=0x7fd3c4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46070b0658ae9029 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:10.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:30:10.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/2000846094 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c401d070 con 0x56013df71ae0 2023-07-14T11:30:10.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/2000846094 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fd3c4004910 con 0x56013df71ae0 2023-07-14T11:30:10.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56013de154f0 con 0x56013df71ae0 2023-07-14T11:30:10.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/2000846094 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3c4004ad0 con 0x56013df71ae0 2023-07-14T11:30:10.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3e332eb80 10 client.26855 fetch_fsmap learned FSMap version 26 2023-07-14T11:30:10.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.888+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56013dcd2c60 con 0x56013df71ae0 2023-07-14T11:30:10.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/2000846094 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56013de154f0 con 0x56013df71ae0 2023-07-14T11:30:10.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/2000846094 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd3c400fcf0 con 0x56013df71ae0 2023-07-14T11:30:10.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3e332eb80 10 client.26855 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:30:10.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3e332eb80 10 client.26855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:30:10.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3e332eb80 20 client.26855 populate_metadata read hostname 'smithi161' 2023-07-14T11:30:10.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013df8cc70 0x56013df8f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3e332eb80 4 client.26855 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:30:10.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.889+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56013deb4eb0 con 0x56013df8cc70 2023-07-14T11:30:10.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.890+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013df8cc70 0x56013df8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.890+0000 7fd3b0ff9700 10 client.26855 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:10.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.890+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013df8cc70 0x56013df8f080 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11d7c5ee92da564 server_cookie=5bef792e1e3d37d8 in_seq=0 out_seq=0 2023-07-14T11:30:10.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/2000846094 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56013deb4eb0 con 0x56013df8cc70 2023-07-14T11:30:10.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3b0ff9700 10 client.26855 handle_command_reply: tid=0 2023-07-14T11:30:10.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 client.26855 shutdown 2023-07-14T11:30:10.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 msgr2=0x56013df7a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:10.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013df7a080 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4000c00 tx=0x7fd3c4005270 comp rx=0 tx=0).stop 2023-07-14T11:30:10.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 shutdown_connections 2023-07-14T11:30:10.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013df8cc70 0x56013df8f080 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df7d2e0 0x56013df7ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3b0ff9700 0 client.26855 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:10.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013df7a080 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/2000846094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df7a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 >> 172.21.15.161:0/2000846094 conn(0x56013df5fda0 msgr2=0x56013df65290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:10.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 shutdown_connections 2023-07-14T11:30:10.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 1 -- 172.21.15.161:0/2000846094 wait complete. 2023-07-14T11:30:10.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.892+0000 7fd3e332eb80 20 client.26855 trim_cache size 0 max 16384 2023-07-14T11:30:10.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.910+0000 7fd3e332eb80 1 Processor -- start 2023-07-14T11:30:10.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3e332eb80 1 -- start start 2023-07-14T11:30:10.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df885a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df816e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df7d2e0 0x56013df77370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013df74f40 con 0x56013df723f0 2023-07-14T11:30:10.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013df72d50 con 0x56013df7d2e0 2023-07-14T11:30:10.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013df7b660 con 0x56013df71ae0 2023-07-14T11:30:10.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3cd79b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df885a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3cd79b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df885a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:34034/0 (socket says 172.21.15.161:34034) 2023-07-14T11:30:10.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3cd79b700 1 -- 172.21.15.161:0/4264713134 learned_addr learned my addr 172.21.15.161:0/4264713134 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:10.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3cdf9c700 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df7d2e0 0x56013df77370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.911+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df816e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3cd79b700 1 -- 172.21.15.161:0/4264713134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df7d2e0 msgr2=0x56013df77370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df7d2e0 0x56013df77370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3cd79b700 1 -- 172.21.15.161:0/4264713134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 msgr2=0x56013df816e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df816e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3cd79b700 1 -- 172.21.15.161:0/4264713134 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56013dd10250 con 0x56013df71ae0 2023-07-14T11:30:10.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df885a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009ce0 tx=0x7fd3c4005cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f81243638dc735db server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:10.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3b3fff700 1 -- 172.21.15.161:0/4264713134 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c401d040 con 0x56013df71ae0 2023-07-14T11:30:10.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3b3fff700 1 -- 172.21.15.161:0/4264713134 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3c401e040 con 0x56013df71ae0 2023-07-14T11:30:10.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.912+0000 7fd3b3fff700 1 -- 172.21.15.161:0/4264713134 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4024ae0 con 0x56013df71ae0 2023-07-14T11:30:10.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 -- 172.21.15.161:0/4264713134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 msgr2=0x56013df885a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:10.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df885a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009ce0 tx=0x7fd3c4005cb0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 -- 172.21.15.161:0/4264713134 shutdown_connections 2023-07-14T11:30:10.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df7d2e0 0x56013df77370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df71ae0 0x56013df885a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/4264713134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df723f0 0x56013df816e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 -- 172.21.15.161:0/4264713134 >> 172.21.15.161:0/4264713134 conn(0x56013df5fda0 msgr2=0x56013df87e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:10.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 -- 172.21.15.161:0/4264713134 shutdown_connections 2023-07-14T11:30:10.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.913+0000 7fd3e332eb80 1 -- 172.21.15.161:0/4264713134 wait complete. 2023-07-14T11:30:10.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 -- start start 2023-07-14T11:30:10.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 Processor -- start 2023-07-14T11:30:10.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013dfd4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df723f0 0x56013dfd51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013dfdfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:10.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013dfe0080 con 0x56013df7d2e0 2023-07-14T11:30:10.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013dfe01c0 con 0x56013df71ae0 2023-07-14T11:30:10.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.914+0000 7fd3e332eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013dfe0380 con 0x56013df723f0 2023-07-14T11:30:10.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3e332eb80 10 client.0 _ll_register_callbacks cb 0x7fd3c400bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:30:10.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3cd79b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013dfd4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3ccf9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df723f0 0x56013dfd51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3cd79b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013dfd4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57734/0 (socket says 172.21.15.161:57734) 2023-07-14T11:30:10.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3cd79b700 1 -- 172.21.15.161:0/850881680 learned_addr learned my addr 172.21.15.161:0/850881680 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:10.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3cdf9c700 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013dfdfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:10.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/850881680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 msgr2=0x56013dfd4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013dfd4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/850881680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 msgr2=0x56013dfdfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:10.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013dfdfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:10.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3ccf9a700 1 -- 172.21.15.161:0/850881680 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56013dcd2c60 con 0x56013df723f0 2023-07-14T11:30:10.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3ccf9a700 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df723f0 0x56013dfd51f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8096960 tx=0x7fd3b80a2430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72f076104618e21 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:10.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:30:10.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/850881680 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8010620 con 0x56013df723f0 2023-07-14T11:30:10.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3b8098290 con 0x56013df723f0 2023-07-14T11:30:10.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.915+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/850881680 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fd3b80b81b0 con 0x56013df723f0 2023-07-14T11:30:10.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/850881680 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b80a1eb0 con 0x56013df723f0 2023-07-14T11:30:11.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 10 client.26863 fetch_fsmap learned FSMap version 26 2023-07-14T11:30:11.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56013dce82b0 con 0x56013df723f0 2023-07-14T11:30:11.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/850881680 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3b8098290 con 0x56013df723f0 2023-07-14T11:30:11.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/850881680 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd3b8094b00 con 0x56013df723f0 2023-07-14T11:30:11.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 10 client.26863 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:30:11.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:30:11.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 20 client.26863 populate_metadata read hostname 'smithi161' 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013dfe1700 0x56013dfe3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 4 client.26863 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56013de77050 con 0x56013dfe1700 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.916+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013dfe1700 0x56013dfe3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3cd79b700 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013dfe1700 0x56013dfe3b10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2bc83dd620c08 server_cookie=ea8d577f20a50a70 in_seq=0 out_seq=0 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3b0ff9700 10 client.26863 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3b0ff9700 1 -- 172.21.15.161:0/850881680 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56013de77050 con 0x56013dfe1700 2023-07-14T11:30:11.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3b0ff9700 10 client.26863 handle_command_reply: tid=0 2023-07-14T11:30:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 client.26863 shutdown 2023-07-14T11:30:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df723f0 msgr2=0x56013dfd51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df723f0 0x56013dfd51f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8096960 tx=0x7fd3b80a2430 comp rx=0 tx=0).stop 2023-07-14T11:30:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 shutdown_connections 2023-07-14T11:30:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56013dfe1700 0x56013dfe3b10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56013df7d2e0 0x56013dfdfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013df71ae0 0x56013dfd4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 --2- 172.21.15.161:0/850881680 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56013df723f0 0x56013dfd51f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3b0ff9700 0 client.26863 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:11.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.917+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 >> 172.21.15.161:0/850881680 conn(0x56013df5fda0 msgr2=0x56013df739b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:11.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 shutdown_connections 2023-07-14T11:30:11.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3e332eb80 1 -- 172.21.15.161:0/850881680 wait complete. 2023-07-14T11:30:11.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3e332eb80 20 client.26863 trim_cache size 0 max 16384 2023-07-14T11:30:11.006 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:30:11.006 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:30:11.007 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:30:11.007 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:30:11.007 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:30:11.007 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:30:11.007 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:30:11.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:30:11.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:30:11.008 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:30:11.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd3b40a1340 msgr2=0x7fd3b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:11.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd3b40a1340 0x7fd3b40a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009ec0 tx=0x7fd3c0009920 comp rx=0 tx=0).stop 2023-07-14T11:30:11.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8102910 msgr2=0x7fd3c8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:11.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8102910 0x7fd3c8198390 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800e940 tx=0x7fd3b800ec50 comp rx=0 tx=0).stop 2023-07-14T11:30:11.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 shutdown_connections 2023-07-14T11:30:11.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd3b40a1340 0x7fd3b40a37f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c8101f70 0x7fd3c8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd3c8106fb0 0x7fd3c819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 --2- 172.21.15.161:0/3740714139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c8102910 0x7fd3c8198390 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:11.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.918+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 >> 172.21.15.161:0/3740714139 conn(0x7fd3c80fd7e0 msgr2=0x7fd3c81058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:11.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.919+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 shutdown_connections 2023-07-14T11:30:11.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:10.919+0000 7fd3cf9ff700 1 -- 172.21.15.161:0/3740714139 wait complete. 2023-07-14T11:30:11.011 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:30:11.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 720213 9.10 MB/sec execute 293 sec latency 25.206 ms 2023-07-14T11:30:12.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 721540 9.10 MB/sec execute 294 sec latency 30.766 ms 2023-07-14T11:30:12.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:12 smithi161 ceph-mon[95526]: pgmap v244: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.7 MiB/s wr, 187 op/s 2023-07-14T11:30:12.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:12 smithi175 ceph-mon[129081]: pgmap v244: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.7 MiB/s wr, 187 op/s 2023-07-14T11:30:12.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:12 smithi182 ceph-mon[129366]: pgmap v244: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.7 MiB/s wr, 187 op/s 2023-07-14T11:30:13.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 723225 9.09 MB/sec execute 295 sec latency 66.781 ms 2023-07-14T11:30:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:13 smithi175 ceph-mon[129081]: osdmap e97: 12 total, 12 up, 12 in 2023-07-14T11:30:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:13 smithi175 ceph-mon[129081]: pgmap v246: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 234 op/s 2023-07-14T11:30:14.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 724875 9.10 MB/sec execute 296 sec latency 28.381 ms 2023-07-14T11:30:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:13 smithi182 ceph-mon[129366]: osdmap e97: 12 total, 12 up, 12 in 2023-07-14T11:30:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:13 smithi182 ceph-mon[129366]: pgmap v246: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 234 op/s 2023-07-14T11:30:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:13 smithi161 ceph-mon[95526]: osdmap e97: 12 total, 12 up, 12 in 2023-07-14T11:30:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:13 smithi161 ceph-mon[95526]: pgmap v246: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 234 op/s 2023-07-14T11:30:15.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 726806 9.09 MB/sec execute 297 sec latency 16.028 ms 2023-07-14T11:30:16.194 INFO:tasks.workunit.client.0.smithi161.stdout: 1 728452 9.10 MB/sec execute 298 sec latency 14.525 ms 2023-07-14T11:30:16.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:16 smithi182 ceph-mon[129366]: pgmap v247: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.1 MiB/s wr, 138 op/s 2023-07-14T11:30:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:16 smithi161 ceph-mon[95526]: pgmap v247: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.1 MiB/s wr, 138 op/s 2023-07-14T11:30:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:16 smithi175 ceph-mon[129081]: pgmap v247: 129 pgs: 129 active+clean; 516 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.1 MiB/s wr, 138 op/s 2023-07-14T11:30:17.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 730306 9.09 MB/sec execute 299 sec latency 41.177 ms 2023-07-14T11:30:18.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 731855 9.10 MB/sec execute 300 sec latency 59.451 ms 2023-07-14T11:30:18.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:18 smithi161 ceph-mon[95526]: pgmap v248: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-07-14T11:30:18.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:18 smithi175 ceph-mon[129081]: pgmap v248: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-07-14T11:30:18.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:18 smithi182 ceph-mon[129366]: pgmap v248: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-07-14T11:30:19.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 733037 9.09 MB/sec execute 301 sec latency 44.501 ms 2023-07-14T11:30:20.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 735309 9.11 MB/sec execute 302 sec latency 15.161 ms 2023-07-14T11:30:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:20 smithi161 ceph-mon[95526]: pgmap v249: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-07-14T11:30:20.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:20 smithi175 ceph-mon[129081]: pgmap v249: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-07-14T11:30:20.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:20 smithi182 ceph-mon[129366]: pgmap v249: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-07-14T11:30:21.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 736490 9.09 MB/sec execute 303 sec latency 68.535 ms 2023-07-14T11:30:22.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 738731 9.11 MB/sec execute 304 sec latency 49.422 ms 2023-07-14T11:30:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:22 smithi161 ceph-mon[95526]: pgmap v250: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2023-07-14T11:30:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:22 smithi175 ceph-mon[129081]: pgmap v250: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2023-07-14T11:30:22.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:22 smithi182 ceph-mon[129366]: pgmap v250: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2023-07-14T11:30:23.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 739764 9.09 MB/sec execute 305 sec latency 25.313 ms 2023-07-14T11:30:24.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:23 smithi175 ceph-mon[129081]: pgmap v251: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2023-07-14T11:30:24.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 741911 9.10 MB/sec execute 306 sec latency 29.398 ms 2023-07-14T11:30:24.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:23 smithi182 ceph-mon[129366]: pgmap v251: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2023-07-14T11:30:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:23 smithi161 ceph-mon[95526]: pgmap v251: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2023-07-14T11:30:25.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 743045 9.09 MB/sec execute 307 sec latency 16.478 ms 2023-07-14T11:30:26.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 745455 9.10 MB/sec execute 308 sec latency 26.243 ms 2023-07-14T11:30:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:26 smithi161 ceph-mon[95526]: pgmap v252: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2023-07-14T11:30:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:26 smithi175 ceph-mon[129081]: pgmap v252: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2023-07-14T11:30:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:26 smithi182 ceph-mon[129366]: pgmap v252: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2023-07-14T11:30:27.195 INFO:tasks.workunit.client.0.smithi161.stdout: 1 746563 9.09 MB/sec execute 309 sec latency 36.463 ms 2023-07-14T11:30:28.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 749026 9.10 MB/sec execute 310 sec latency 15.531 ms 2023-07-14T11:30:28.498 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:28 smithi161 ceph-mon[95526]: pgmap v253: 129 pgs: 129 active+clean; 542 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2023-07-14T11:30:28.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:28 smithi175 ceph-mon[129081]: pgmap v253: 129 pgs: 129 active+clean; 542 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2023-07-14T11:30:28.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:28 smithi182 ceph-mon[129366]: pgmap v253: 129 pgs: 129 active+clean; 542 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2023-07-14T11:30:29.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 749821 9.09 MB/sec execute 311 sec latency 69.799 ms 2023-07-14T11:30:30.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 751614 9.08 MB/sec execute 312 sec latency 33.500 ms 2023-07-14T11:30:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:30 smithi161 ceph-mon[95526]: pgmap v254: 129 pgs: 129 active+clean; 542 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 197 op/s 2023-07-14T11:30:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:30 smithi175 ceph-mon[129081]: pgmap v254: 129 pgs: 129 active+clean; 542 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 197 op/s 2023-07-14T11:30:30.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:30 smithi182 ceph-mon[129366]: pgmap v254: 129 pgs: 129 active+clean; 542 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 197 op/s 2023-07-14T11:30:31.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 753221 9.09 MB/sec execute 313 sec latency 12.911 ms 2023-07-14T11:30:32.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 754660 9.08 MB/sec execute 314 sec latency 31.825 ms 2023-07-14T11:30:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:32 smithi161 ceph-mon[95526]: pgmap v255: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2023-07-14T11:30:32.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:30:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:32 smithi175 ceph-mon[129081]: pgmap v255: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2023-07-14T11:30:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:30:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:32 smithi182 ceph-mon[129366]: pgmap v255: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2023-07-14T11:30:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:30:33.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 756553 9.09 MB/sec execute 315 sec latency 111.369 ms 2023-07-14T11:30:33.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:30:33.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:30:33.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:30:33.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:30:33.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:30:33.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:30:33.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:30:33.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:30:33.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:30:34.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 757390 9.07 MB/sec execute 316 sec latency 89.387 ms 2023-07-14T11:30:34.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:34 smithi161 ceph-mon[95526]: pgmap v256: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2023-07-14T11:30:34.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:34 smithi175 ceph-mon[129081]: pgmap v256: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2023-07-14T11:30:34.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:34 smithi182 ceph-mon[129366]: pgmap v256: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2023-07-14T11:30:35.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 759657 9.08 MB/sec execute 317 sec latency 58.766 ms 2023-07-14T11:30:36.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 760779 9.07 MB/sec execute 318 sec latency 15.149 ms 2023-07-14T11:30:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:36 smithi161 ceph-mon[95526]: pgmap v257: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 191 op/s 2023-07-14T11:30:36.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:36 smithi175 ceph-mon[129081]: pgmap v257: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 191 op/s 2023-07-14T11:30:36.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:36 smithi182 ceph-mon[129366]: pgmap v257: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 191 op/s 2023-07-14T11:30:37.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 763039 9.08 MB/sec execute 319 sec latency 17.340 ms 2023-07-14T11:30:38.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 764219 9.07 MB/sec execute 320 sec latency 13.407 ms 2023-07-14T11:30:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:38 smithi161 ceph-mon[95526]: pgmap v258: 129 pgs: 129 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2023-07-14T11:30:38.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:38 smithi175 ceph-mon[129081]: pgmap v258: 129 pgs: 129 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2023-07-14T11:30:38.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:38 smithi182 ceph-mon[129366]: pgmap v258: 129 pgs: 129 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2023-07-14T11:30:39.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 765980 9.07 MB/sec execute 321 sec latency 51.224 ms 2023-07-14T11:30:40.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 767431 9.08 MB/sec execute 322 sec latency 29.569 ms 2023-07-14T11:30:40.499 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:40 smithi161 ceph-mon[95526]: pgmap v259: 129 pgs: 129 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-14T11:30:40.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:40 smithi175 ceph-mon[129081]: pgmap v259: 129 pgs: 129 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-14T11:30:40.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:40 smithi182 ceph-mon[129366]: pgmap v259: 129 pgs: 129 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-14T11:30:41.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:30:41.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 769359 9.07 MB/sec execute 323 sec latency 16.407 ms 2023-07-14T11:30:41.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 -- 172.21.15.161:0/1837270784 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f47080ffae0 msgr2=0x7f47080fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 --2- 172.21.15.161:0/1837270784 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f47080ffae0 0x7f47080fff50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009990 tx=0x7f46f8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 -- 172.21.15.161:0/1837270784 shutdown_connections 2023-07-14T11:30:41.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 --2- 172.21.15.161:0/1837270784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4708105af0 0x7f4708105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 --2- 172.21.15.161:0/1837270784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4708100490 0x7f470810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 --2- 172.21.15.161:0/1837270784 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f47080ffae0 0x7f47080fff50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 -- 172.21.15.161:0/1837270784 >> 172.21.15.161:0/1837270784 conn(0x7f47080fb5e0 msgr2=0x7f47080fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:41.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 -- 172.21.15.161:0/1837270784 shutdown_connections 2023-07-14T11:30:41.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.230+0000 7f470fdff700 1 -- 172.21.15.161:0/1837270784 wait complete. 2023-07-14T11:30:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.231+0000 7f470fdff700 1 Processor -- start 2023-07-14T11:30:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.231+0000 7f470fdff700 1 -- start start 2023-07-14T11:30:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470fdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47080ffae0 0x7f470819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470fdff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4708100490 0x7f470819bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470fdff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4708105af0 0x7f4708195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470fdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708196080 con 0x7f47080ffae0 2023-07-14T11:30:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470fdff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081961f0 con 0x7f4708100490 2023-07-14T11:30:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470fdff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081963b0 con 0x7f4708105af0 2023-07-14T11:30:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47080ffae0 0x7f470819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47080ffae0 0x7f470819b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:36104/0 (socket says 172.21.15.161:36104) 2023-07-14T11:30:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 -- 172.21.15.161:0/1660363182 learned_addr learned my addr 172.21.15.161:0/1660363182 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470d39a700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4708100490 0x7f470819bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470e39c700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4708105af0 0x7f4708195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 -- 172.21.15.161:0/1660363182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4708105af0 msgr2=0x7f4708195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4708105af0 0x7f4708195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 -- 172.21.15.161:0/1660363182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4708100490 msgr2=0x7f470819bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4708100490 0x7f470819bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470db9b700 1 -- 172.21.15.161:0/1660363182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704009520 con 0x7f47080ffae0 2023-07-14T11:30:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.232+0000 7f470d39a700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4708100490 0x7f470819bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:30:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.233+0000 7f470db9b700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47080ffae0 0x7f470819b9a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f470400eb30 tx=0x7f470400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.233+0000 7f46feffd700 1 -- 172.21.15.161:0/1660363182 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704009880 con 0x7f47080ffae0 2023-07-14T11:30:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.233+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f80095f0 con 0x7f47080ffae0 2023-07-14T11:30:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.233+0000 7f46feffd700 1 -- 172.21.15.161:0/1660363182 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47040044e0 con 0x7f47080ffae0 2023-07-14T11:30:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.233+0000 7f46feffd700 1 -- 172.21.15.161:0/1660363182 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704003d80 con 0x7f47080ffae0 2023-07-14T11:30:41.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.233+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47081a27b0 con 0x7f47080ffae0 2023-07-14T11:30:41.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.236+0000 7f46feffd700 1 -- 172.21.15.161:0/1660363182 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4704003f20 con 0x7f47080ffae0 2023-07-14T11:30:41.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.238+0000 7f46feffd700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46f40a12c0 0x7f46f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.238+0000 7f4723738b80 1 Processor -- start 2023-07-14T11:30:41.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.238+0000 7f46feffd700 1 -- 172.21.15.161:0/1660363182 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4704014070 con 0x7f47080ffae0 2023-07-14T11:30:41.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.238+0000 7f4723738b80 1 -- start start 2023-07-14T11:30:41.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.238+0000 7f470d39a700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46f40a12c0 0x7f46f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.238+0000 7f4723738b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6ca66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f4723738b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f4723738b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f4723738b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6baccb0 con 0x556ef6cb1ad0 2023-07-14T11:30:41.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f4723738b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6bffa40 con 0x556ef6ca6be0 2023-07-14T11:30:41.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f4723738b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6bf8ed0 con 0x556ef6ca62d0 2023-07-14T11:30:41.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470d39a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6ca66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470e39c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6ca66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59434/0 (socket says 172.21.15.161:59434) 2023-07-14T11:30:41.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 -- 172.21.15.161:0/1927772917 learned_addr learned my addr 172.21.15.161:0/1927772917 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:41.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470d39a700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46f40a12c0 0x7f46f40a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47081050d0 tx=0x7f46f8005fb0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:41.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 -- 172.21.15.161:0/1927772917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 msgr2=0x556ef6cb1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 -- 172.21.15.161:0/1927772917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6cb1ad0 msgr2=0x556ef6cb3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 -- 172.21.15.161:0/1927772917 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ef6a26de0 con 0x556ef6ca62d0 2023-07-14T11:30:41.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470e39c700 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:30:41.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.239+0000 7f470db9b700 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6ca66a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4704009230 tx=0x7f470409a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2552c4e4a4ae8122 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:41.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.240+0000 7f46f3fff700 1 -- 172.21.15.161:0/1927772917 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040cd050 con 0x556ef6ca62d0 2023-07-14T11:30:41.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.240+0000 7f46f3fff700 1 -- 172.21.15.161:0/1927772917 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470400cbd0 con 0x556ef6ca62d0 2023-07-14T11:30:41.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.240+0000 7f46f3fff700 1 -- 172.21.15.161:0/1927772917 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040cd1b0 con 0x556ef6ca62d0 2023-07-14T11:30:41.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.240+0000 7f4723738b80 1 -- 172.21.15.161:0/1927772917 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 msgr2=0x556ef6ca66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.240+0000 7f4723738b80 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6ca66a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4704009230 tx=0x7f470409a1c0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.240+0000 7f4723738b80 1 -- 172.21.15.161:0/1927772917 shutdown_connections 2023-07-14T11:30:41.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.240+0000 7f4723738b80 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.241+0000 7f4723738b80 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6ca66a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.241+0000 7f4723738b80 1 --2- 172.21.15.161:0/1927772917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.241+0000 7f4723738b80 1 -- 172.21.15.161:0/1927772917 >> 172.21.15.161:0/1927772917 conn(0x556ef6c9cda0 msgr2=0x556ef6ca26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.241+0000 7f4723738b80 1 -- 172.21.15.161:0/1927772917 shutdown_connections 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.241+0000 7f4723738b80 1 -- 172.21.15.161:0/1927772917 wait complete. 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.241+0000 7f4723738b80 1 -- start start 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.241+0000 7f4723738b80 1 Processor -- start 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f4723738b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca62d0 0x556ef6cb1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f4723738b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6cae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f4723738b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6caeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f4723738b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6b98ac0 con 0x556ef6ca6be0 2023-07-14T11:30:41.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f4723738b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6b987c0 con 0x556ef6ca62d0 2023-07-14T11:30:41.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f4723738b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6a7c920 con 0x556ef6cb1ad0 2023-07-14T11:30:41.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f4723738b80 10 client.0 _ll_register_callbacks cb 0x556ef6b96760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:30:41.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f470db9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca62d0 0x556ef6cb1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f470db9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca62d0 0x556ef6cb1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46982/0 (socket says 172.21.15.161:46982) 2023-07-14T11:30:41.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f470db9b700 1 -- 172.21.15.161:0/527797496 learned_addr learned my addr 172.21.15.161:0/527797496 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:41.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f470d39a700 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6cae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.242+0000 7f470e39c700 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6caeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f470db9b700 1 -- 172.21.15.161:0/527797496 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 msgr2=0x556ef6caeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f470db9b700 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6caeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f470db9b700 1 -- 172.21.15.161:0/527797496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 msgr2=0x556ef6cae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f470db9b700 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6cae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f470db9b700 1 -- 172.21.15.161:0/527797496 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4700011230 con 0x556ef6ca62d0 2023-07-14T11:30:41.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f470db9b700 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca62d0 0x556ef6cb1520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47040850f0 tx=0x7f470400eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e25d07b50b695e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:41.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f46f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:30:41.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f46f17fa700 1 -- 172.21.15.161:0/527797496 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040a2710 con 0x556ef6ca62d0 2023-07-14T11:30:41.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ef6b524f0 con 0x556ef6ca62d0 2023-07-14T11:30:41.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f46f17fa700 1 -- 172.21.15.161:0/527797496 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f470409a2c0 con 0x556ef6ca62d0 2023-07-14T11:30:41.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f46f17fa700 1 -- 172.21.15.161:0/527797496 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4704061070 con 0x556ef6ca62d0 2023-07-14T11:30:41.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f4723738b80 10 client.26873 fetch_fsmap learned FSMap version 26 2023-07-14T11:30:41.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f47081a27b0 con 0x556ef6ca62d0 2023-07-14T11:30:41.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.243+0000 7f46f17fa700 1 -- 172.21.15.161:0/527797496 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ef6b524f0 con 0x556ef6ca62d0 2023-07-14T11:30:41.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f46f17fa700 1 -- 172.21.15.161:0/527797496 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f47040998b0 con 0x556ef6ca62d0 2023-07-14T11:30:41.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f4723738b80 10 client.26873 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f4723738b80 10 client.26873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f4723738b80 20 client.26873 populate_metadata read hostname 'smithi161' 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f4723738b80 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6cc0fb0 0x556ef6cc33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f4723738b80 4 client.26873 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ef6bf1eb0 con 0x556ef6cc0fb0 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f470d39a700 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6cc0fb0 0x556ef6cc33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.244+0000 7f470d39a700 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6cc0fb0 0x556ef6cc33c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27531dc1e6418bb7 server_cookie=b98e49be72e09817 in_seq=0 out_seq=0 2023-07-14T11:30:41.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.245+0000 7f46f17fa700 10 client.26873 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:41.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.245+0000 7f46f17fa700 1 -- 172.21.15.161:0/527797496 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ef6bf1eb0 con 0x556ef6cc0fb0 2023-07-14T11:30:41.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.245+0000 7f46f17fa700 10 client.26873 handle_command_reply: tid=0 2023-07-14T11:30:41.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.245+0000 7f4723738b80 1 client.26873 shutdown 2023-07-14T11:30:41.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.245+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca62d0 msgr2=0x556ef6cb1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.245+0000 7f4723738b80 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca62d0 0x556ef6cb1520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47040850f0 tx=0x7f470400eac0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.245+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 shutdown_connections 2023-07-14T11:30:41.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6cc0fb0 0x556ef6cc33c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6caeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f46f17fa700 0 client.26873 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:41.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca62d0 0x556ef6cb1520 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 1 --2- 172.21.15.161:0/527797496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6cae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 >> 172.21.15.161:0/527797496 conn(0x556ef6c9cda0 msgr2=0x556ef6ca2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:41.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 shutdown_connections 2023-07-14T11:30:41.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 1 -- 172.21.15.161:0/527797496 wait complete. 2023-07-14T11:30:41.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.246+0000 7f4723738b80 20 client.26873 trim_cache size 0 max 16384 2023-07-14T11:30:41.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 Processor -- start 2023-07-14T11:30:41.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 -- start start 2023-07-14T11:30:41.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6cb10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6ca9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6cb1ad0 0x556ef6caba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6cbde70 con 0x556ef6ca6be0 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6ca7540 con 0x556ef6cb1ad0 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.265+0000 7f4723738b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6cafd80 con 0x556ef6ca62d0 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6cb1ad0 0x556ef6caba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470db9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6cb10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470db9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6cb10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59458/0 (socket says 172.21.15.161:59458) 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6cb1ad0 0x556ef6caba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46994/0 (socket says 172.21.15.161:46994) 2023-07-14T11:30:41.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470db9b700 1 -- 172.21.15.161:0/3189399404 learned_addr learned my addr 172.21.15.161:0/3189399404 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:41.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470d39a700 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6ca9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 -- 172.21.15.161:0/3189399404 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 msgr2=0x556ef6cb10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6cb10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 -- 172.21.15.161:0/3189399404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 msgr2=0x556ef6ca9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6ca9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 -- 172.21.15.161:0/3189399404 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ef6a0fc60 con 0x556ef6cb1ad0 2023-07-14T11:30:41.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.266+0000 7f470e39c700 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6cb1ad0 0x556ef6caba60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4700009550 tx=0x7f4700009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc96558c713f7557 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:41.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f46fcbff700 1 -- 172.21.15.161:0/3189399404 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000e040 con 0x556ef6cb1ad0 2023-07-14T11:30:41.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f46fcbff700 1 -- 172.21.15.161:0/3189399404 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470000f040 con 0x556ef6cb1ad0 2023-07-14T11:30:41.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f46fcbff700 1 -- 172.21.15.161:0/3189399404 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470001f430 con 0x556ef6cb1ad0 2023-07-14T11:30:41.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 -- 172.21.15.161:0/3189399404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6cb1ad0 msgr2=0x556ef6caba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6cb1ad0 0x556ef6caba60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4700009550 tx=0x7f4700009920 comp rx=0 tx=0).stop 2023-07-14T11:30:41.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 -- 172.21.15.161:0/3189399404 shutdown_connections 2023-07-14T11:30:41.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6cb1ad0 0x556ef6caba60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6ca62d0 0x556ef6cb10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 --2- 172.21.15.161:0/3189399404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca6be0 0x556ef6ca9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 -- 172.21.15.161:0/3189399404 >> 172.21.15.161:0/3189399404 conn(0x556ef6c9cda0 msgr2=0x556ef6cbd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.267+0000 7f4723738b80 1 -- 172.21.15.161:0/3189399404 shutdown_connections 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.268+0000 7f4723738b80 1 -- 172.21.15.161:0/3189399404 wait complete. 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.268+0000 7f4723738b80 1 -- start start 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.268+0000 7f4723738b80 1 Processor -- start 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f4723738b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca62d0 0x556ef6cb86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f4723738b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f4723738b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f4723738b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6cb77c0 con 0x556ef6ca62d0 2023-07-14T11:30:41.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f4723738b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6d14e40 con 0x556ef6ca6be0 2023-07-14T11:30:41.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f4723738b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef6d14f80 con 0x556ef6cb1ad0 2023-07-14T11:30:41.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f4723738b80 10 client.0 _ll_register_callbacks cb 0x7f4700009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:30:41.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f470d39a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f470e39c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f470d39a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47010/0 (socket says 172.21.15.161:47010) 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f470d39a700 1 -- 172.21.15.161:0/2053871415 learned_addr learned my addr 172.21.15.161:0/2053871415 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.269+0000 7f470db9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca62d0 0x556ef6cb86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f470e39c700 1 -- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 msgr2=0x556ef6cb6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f470e39c700 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f470e39c700 1 -- 172.21.15.161:0/2053871415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca62d0 msgr2=0x556ef6cb86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f470e39c700 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca62d0 0x556ef6cb86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f470e39c700 1 -- 172.21.15.161:0/2053871415 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4700009230 con 0x556ef6cb1ad0 2023-07-14T11:30:41.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f470d39a700 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:30:41.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f470e39c700 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb70b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f47000044b0 tx=0x7f4700004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0b97889fe807b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:30:41.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f46f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:30:41.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f46f17fa700 1 -- 172.21.15.161:0/2053871415 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000e040 con 0x556ef6cb1ad0 2023-07-14T11:30:41.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f470400cbd0 con 0x556ef6cb1ad0 2023-07-14T11:30:41.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f46f17fa700 1 -- 172.21.15.161:0/2053871415 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4700004510 con 0x556ef6cb1ad0 2023-07-14T11:30:41.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f46f17fa700 1 -- 172.21.15.161:0/2053871415 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47000042d0 con 0x556ef6cb1ad0 2023-07-14T11:30:41.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.270+0000 7f4723738b80 10 client.26881 fetch_fsmap learned FSMap version 26 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556ef6a4d250 con 0x556ef6cb1ad0 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f46f17fa700 1 -- 172.21.15.161:0/2053871415 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470400cbd0 con 0x556ef6cb1ad0 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f46f17fa700 1 -- 172.21.15.161:0/2053871415 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4700030020 con 0x556ef6cb1ad0 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f4723738b80 10 client.26881 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f4723738b80 10 client.26881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f4723738b80 20 client.26881 populate_metadata read hostname 'smithi161' 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f4723738b80 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6d16650 0x556ef6d18a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:30:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f4723738b80 4 client.26881 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:30:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ef6bb4050 con 0x556ef6d16650 2023-07-14T11:30:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.271+0000 7f470db9b700 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6d16650 0x556ef6d18a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:30:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f46f17fa700 10 client.26881 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f470db9b700 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6d16650 0x556ef6d18a60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4cf77830e23ef3c server_cookie=5969af2ded328f9c in_seq=0 out_seq=0 2023-07-14T11:30:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f46f17fa700 1 -- 172.21.15.161:0/2053871415 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556ef6bb4050 con 0x556ef6d16650 2023-07-14T11:30:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f46f17fa700 10 client.26881 handle_command_reply: tid=0 2023-07-14T11:30:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 client.26881 shutdown 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 msgr2=0x556ef6cb70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb70b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f47000044b0 tx=0x7f4700004230 comp rx=0 tx=0).stop 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 shutdown_connections 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556ef6d16650 0x556ef6d18a60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef6ca62d0 0x556ef6cb86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ef6cb1ad0 0x556ef6cb70b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 --2- 172.21.15.161:0/2053871415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ef6ca6be0 0x556ef6cb6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 >> 172.21.15.161:0/2053871415 conn(0x556ef6c9cda0 msgr2=0x556ef6ca9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:41.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f46f17fa700 0 client.26881 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:30:41.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 shutdown_connections 2023-07-14T11:30:41.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.272+0000 7f4723738b80 1 -- 172.21.15.161:0/2053871415 wait complete. 2023-07-14T11:30:41.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f4723738b80 20 client.26881 trim_cache size 0 max 16384 2023-07-14T11:30:41.284 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:30:41.285 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:30:41.285 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:30:41.285 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:30:41.285 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:30:41.285 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:30:41.285 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:30:41.285 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:30:41.286 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:30:41.286 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:30:41.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46f40a12c0 msgr2=0x7f46f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46f40a12c0 0x7f46f40a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47081050d0 tx=0x7f46f8005fb0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47080ffae0 msgr2=0x7f470819b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:30:41.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47080ffae0 0x7f470819b9a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f470400eb30 tx=0x7f470400ee40 comp rx=0 tx=0).stop 2023-07-14T11:30:41.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 shutdown_connections 2023-07-14T11:30:41.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f46f40a12c0 0x7f46f40a3770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47080ffae0 0x7f470819b9a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4708105af0 0x7f4708195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 --2- 172.21.15.161:0/1660363182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4708100490 0x7f470819bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:30:41.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 >> 172.21.15.161:0/1660363182 conn(0x7f47080fb5e0 msgr2=0x7f47081035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:30:41.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 shutdown_connections 2023-07-14T11:30:41.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:30:41.273+0000 7f470fdff700 1 -- 172.21.15.161:0/1660363182 wait complete. 2023-07-14T11:30:41.294 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:30:42.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 770971 9.08 MB/sec execute 324 sec latency 13.995 ms 2023-07-14T11:30:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:42 smithi161 ceph-mon[95526]: pgmap v260: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-07-14T11:30:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:42 smithi175 ceph-mon[129081]: pgmap v260: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-07-14T11:30:42.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:42 smithi182 ceph-mon[129366]: pgmap v260: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-07-14T11:30:43.196 INFO:tasks.workunit.client.0.smithi161.stdout: 1 772824 9.07 MB/sec execute 325 sec latency 17.991 ms 2023-07-14T11:30:44.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:43 smithi175 ceph-mon[129081]: pgmap v261: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-07-14T11:30:44.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 774362 9.08 MB/sec execute 326 sec latency 50.321 ms 2023-07-14T11:30:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:43 smithi182 ceph-mon[129366]: pgmap v261: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-07-14T11:30:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:43 smithi161 ceph-mon[95526]: pgmap v261: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-07-14T11:30:45.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 775520 9.07 MB/sec execute 327 sec latency 40.516 ms 2023-07-14T11:30:46.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 777807 9.08 MB/sec execute 328 sec latency 14.754 ms 2023-07-14T11:30:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:46 smithi161 ceph-mon[95526]: pgmap v262: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-07-14T11:30:46.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:46 smithi175 ceph-mon[129081]: pgmap v262: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-07-14T11:30:46.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:46 smithi182 ceph-mon[129366]: pgmap v262: 129 pgs: 129 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-07-14T11:30:47.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 779015 9.07 MB/sec execute 329 sec latency 16.931 ms 2023-07-14T11:30:48.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 781347 9.09 MB/sec execute 330 sec latency 17.918 ms 2023-07-14T11:30:48.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:48 smithi161 ceph-mon[95526]: pgmap v263: 129 pgs: 129 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2023-07-14T11:30:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:48 smithi175 ceph-mon[129081]: pgmap v263: 129 pgs: 129 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2023-07-14T11:30:48.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:48 smithi182 ceph-mon[129366]: pgmap v263: 129 pgs: 129 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2023-07-14T11:30:49.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 782231 9.07 MB/sec execute 331 sec latency 32.047 ms 2023-07-14T11:30:50.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 784406 9.08 MB/sec execute 332 sec latency 28.816 ms 2023-07-14T11:30:50.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:50 smithi161 ceph-mon[95526]: pgmap v264: 129 pgs: 129 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:50.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:50 smithi175 ceph-mon[129081]: pgmap v264: 129 pgs: 129 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:50.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:50 smithi182 ceph-mon[129366]: pgmap v264: 129 pgs: 129 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-14T11:30:51.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 785421 9.06 MB/sec execute 333 sec latency 34.448 ms 2023-07-14T11:30:52.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 787596 9.07 MB/sec execute 334 sec latency 19.656 ms 2023-07-14T11:30:52.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:52 smithi161 ceph-mon[95526]: pgmap v265: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 276 op/s 2023-07-14T11:30:52.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:52 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:30:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:52 smithi175 ceph-mon[129081]: pgmap v265: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 276 op/s 2023-07-14T11:30:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:52 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:30:52.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:52 smithi182 ceph-mon[129366]: pgmap v265: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 276 op/s 2023-07-14T11:30:52.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:52 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:30:53.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 788940 9.07 MB/sec execute 335 sec latency 14.106 ms 2023-07-14T11:30:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:53 smithi175 ceph-mon[129081]: pgmap v266: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:30:54.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 790684 9.06 MB/sec execute 336 sec latency 46.945 ms 2023-07-14T11:30:54.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:53 smithi182 ceph-mon[129366]: pgmap v266: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:30:54.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:53 smithi161 ceph-mon[95526]: pgmap v266: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:30:55.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 792230 9.07 MB/sec execute 337 sec latency 29.275 ms 2023-07-14T11:30:56.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 793671 9.06 MB/sec execute 338 sec latency 50.605 ms 2023-07-14T11:30:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:56 smithi161 ceph-mon[95526]: pgmap v267: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:30:56.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:56 smithi175 ceph-mon[129081]: pgmap v267: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:30:56.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:56 smithi182 ceph-mon[129366]: pgmap v267: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:30:57.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 795708 9.07 MB/sec execute 339 sec latency 26.741 ms 2023-07-14T11:30:58.197 INFO:tasks.workunit.client.0.smithi161.stdout: 1 797289 9.06 MB/sec execute 340 sec latency 16.515 ms 2023-07-14T11:30:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:30:58 smithi161 ceph-mon[95526]: pgmap v268: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2023-07-14T11:30:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:30:58 smithi175 ceph-mon[129081]: pgmap v268: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2023-07-14T11:30:58.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:30:58 smithi182 ceph-mon[129366]: pgmap v268: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2023-07-14T11:30:59.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 799186 9.07 MB/sec execute 341 sec latency 42.129 ms 2023-07-14T11:31:00.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 800381 9.06 MB/sec execute 342 sec latency 35.877 ms 2023-07-14T11:31:00.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:00 smithi161 ceph-mon[95526]: pgmap v269: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2023-07-14T11:31:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:00 smithi175 ceph-mon[129081]: pgmap v269: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2023-07-14T11:31:00.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:00 smithi182 ceph-mon[129366]: pgmap v269: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2023-07-14T11:31:01.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 801151 9.04 MB/sec execute 343 sec latency 69.731 ms 2023-07-14T11:31:02.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 802649 9.05 MB/sec execute 344 sec latency 120.663 ms 2023-07-14T11:31:02.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:02 smithi161 ceph-mon[95526]: pgmap v270: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 312 op/s 2023-07-14T11:31:02.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:02 smithi161 ceph-mon[95526]: osdmap e98: 12 total, 12 up, 12 in 2023-07-14T11:31:02.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:02 smithi175 ceph-mon[129081]: pgmap v270: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 312 op/s 2023-07-14T11:31:02.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:02 smithi175 ceph-mon[129081]: osdmap e98: 12 total, 12 up, 12 in 2023-07-14T11:31:02.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:02 smithi182 ceph-mon[129366]: pgmap v270: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 312 op/s 2023-07-14T11:31:02.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:02 smithi182 ceph-mon[129366]: osdmap e98: 12 total, 12 up, 12 in 2023-07-14T11:31:03.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 803864 9.04 MB/sec execute 345 sec latency 18.179 ms 2023-07-14T11:31:04.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:03 smithi175 ceph-mon[129081]: pgmap v272: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2023-07-14T11:31:04.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 806118 9.05 MB/sec execute 346 sec latency 49.542 ms 2023-07-14T11:31:04.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:03 smithi182 ceph-mon[129366]: pgmap v272: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2023-07-14T11:31:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:03 smithi161 ceph-mon[95526]: pgmap v272: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2023-07-14T11:31:05.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 807064 9.03 MB/sec execute 347 sec latency 32.258 ms 2023-07-14T11:31:06.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 809441 9.05 MB/sec execute 348 sec latency 26.059 ms 2023-07-14T11:31:06.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:06 smithi161 ceph-mon[95526]: pgmap v273: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2023-07-14T11:31:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:06 smithi175 ceph-mon[129081]: pgmap v273: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2023-07-14T11:31:06.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:06 smithi182 ceph-mon[129366]: pgmap v273: 129 pgs: 129 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2023-07-14T11:31:07.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 810328 9.03 MB/sec execute 349 sec latency 39.743 ms 2023-07-14T11:31:08.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 812093 9.03 MB/sec execute 350 sec latency 38.656 ms 2023-07-14T11:31:08.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:08 smithi161 ceph-mon[95526]: pgmap v274: 129 pgs: 129 active+clean; 609 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 257 op/s 2023-07-14T11:31:08.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:08 smithi161 ceph-mon[95526]: osdmap e99: 12 total, 12 up, 12 in 2023-07-14T11:31:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:08 smithi175 ceph-mon[129081]: pgmap v274: 129 pgs: 129 active+clean; 609 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 257 op/s 2023-07-14T11:31:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:08 smithi175 ceph-mon[129081]: osdmap e99: 12 total, 12 up, 12 in 2023-07-14T11:31:08.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:08 smithi182 ceph-mon[129366]: pgmap v274: 129 pgs: 129 active+clean; 609 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 257 op/s 2023-07-14T11:31:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:08 smithi182 ceph-mon[129366]: osdmap e99: 12 total, 12 up, 12 in 2023-07-14T11:31:09.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 813580 9.03 MB/sec execute 351 sec latency 12.574 ms 2023-07-14T11:31:10.198 INFO:tasks.workunit.client.0.smithi161.stdout: 1 814737 9.02 MB/sec execute 352 sec latency 68.919 ms 2023-07-14T11:31:10.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:10 smithi175 ceph-mon[129081]: pgmap v276: 129 pgs: 129 active+clean; 609 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2023-07-14T11:31:10.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:10 smithi182 ceph-mon[129366]: pgmap v276: 129 pgs: 129 active+clean; 609 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2023-07-14T11:31:10.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:10 smithi161 ceph-mon[95526]: pgmap v276: 129 pgs: 129 active+clean; 609 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2023-07-14T11:31:11.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 816861 9.04 MB/sec execute 353 sec latency 16.200 ms 2023-07-14T11:31:11.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:31:11.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 -- 172.21.15.161:0/1904744544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc34105ae0 msgr2=0x7fcc34105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 --2- 172.21.15.161:0/1904744544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc34105ae0 0x7fcc34105eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009990 tx=0x7fcc24009ca0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 -- 172.21.15.161:0/1904744544 shutdown_connections 2023-07-14T11:31:11.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 --2- 172.21.15.161:0/1904744544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc34100480 0x7fcc3410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 --2- 172.21.15.161:0/1904744544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc34105ae0 0x7fcc34105eb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 --2- 172.21.15.161:0/1904744544 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc340ffad0 0x7fcc340fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 -- 172.21.15.161:0/1904744544 >> 172.21.15.161:0/1904744544 conn(0x7fcc340fb5f0 msgr2=0x7fcc340fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:11.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 -- 172.21.15.161:0/1904744544 shutdown_connections 2023-07-14T11:31:11.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.501+0000 7fcc397ff700 1 -- 172.21.15.161:0/1904744544 wait complete. 2023-07-14T11:31:11.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.502+0000 7fcc397ff700 1 Processor -- start 2023-07-14T11:31:11.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.502+0000 7fcc397ff700 1 -- start start 2023-07-14T11:31:11.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc397ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc340ffad0 0x7fcc34197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc397ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc34100480 0x7fcc34198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc397ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc34105ae0 0x7fcc3419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc397ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3419ce10 con 0x7fcc34100480 2023-07-14T11:31:11.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc397ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3419cf80 con 0x7fcc340ffad0 2023-07-14T11:31:11.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc397ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3419d140 con 0x7fcc34105ae0 2023-07-14T11:31:11.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc34105ae0 0x7fcc3419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc34100480 0x7fcc34198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc32ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc340ffad0 0x7fcc34197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc34105ae0 0x7fcc3419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41886/0 (socket says 172.21.15.161:41886) 2023-07-14T11:31:11.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc32ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc340ffad0 0x7fcc34197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56512/0 (socket says 172.21.15.161:56512) 2023-07-14T11:31:11.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc337fe700 1 -- 172.21.15.161:0/622506404 learned_addr learned my addr 172.21.15.161:0/622506404 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:11.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc2bfff700 1 -- 172.21.15.161:0/622506404 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc34105ae0 msgr2=0x7fcc3419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc34105ae0 0x7fcc3419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc2bfff700 1 -- 172.21.15.161:0/622506404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc340ffad0 msgr2=0x7fcc34197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc340ffad0 0x7fcc34197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc2bfff700 1 -- 172.21.15.161:0/622506404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009520 con 0x7fcc34100480 2023-07-14T11:31:11.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.503+0000 7fcc337fe700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc34105ae0 0x7fcc3419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:31:11.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.504+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc34100480 0x7fcc34198370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e9f0 tx=0x7fcc1c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:11.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.504+0000 7fcc30ff9700 1 -- 172.21.15.161:0/622506404 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00cb70 con 0x7fcc34100480 2023-07-14T11:31:11.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.504+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc240095f0 con 0x7fcc34100480 2023-07-14T11:31:11.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.504+0000 7fcc30ff9700 1 -- 172.21.15.161:0/622506404 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c00ee90 con 0x7fcc34100480 2023-07-14T11:31:11.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.504+0000 7fcc30ff9700 1 -- 172.21.15.161:0/622506404 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003cf0 con 0x7fcc34100480 2023-07-14T11:31:11.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.504+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc341a4a80 con 0x7fcc34100480 2023-07-14T11:31:11.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.507+0000 7fcc30ff9700 1 -- 172.21.15.161:0/622506404 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcc1c00ccd0 con 0x7fcc34100480 2023-07-14T11:31:11.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.508+0000 7fcc4d1afb80 1 Processor -- start 2023-07-14T11:31:11.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.508+0000 7fcc4d1afb80 1 -- start start 2023-07-14T11:31:11.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf28eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745be21cb0 con 0x55745bf1bbe0 2023-07-14T11:31:11.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745be74a40 con 0x55745bf1b2d0 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745be6ded0 con 0x55745bf26ad0 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc30ff9700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcc200a12c0 0x7fcc200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf28eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc32ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf28eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41894/0 (socket says 172.21.15.161:41894) 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 -- 172.21.15.161:0/2692833085 learned_addr learned my addr 172.21.15.161:0/2692833085 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc32ffd700 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf26590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50406/0 (socket says 172.21.15.161:50406) 2023-07-14T11:31:11.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc337fe700 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc30ff9700 1 -- 172.21.15.161:0/622506404 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fcc1c014070 con 0x7fcc34100480 2023-07-14T11:31:11.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 -- 172.21.15.161:0/2692833085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 msgr2=0x55745bf1b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf1b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 -- 172.21.15.161:0/2692833085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 msgr2=0x55745bf26590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf26590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.509+0000 7fcc2bfff700 1 -- 172.21.15.161:0/2692833085 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55745bc9bde0 con 0x55745bf26ad0 2023-07-14T11:31:11.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.510+0000 7fcc337fe700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcc200a12c0 0x7fcc200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.510+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf28eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0a1fc0 tx=0x7fcc1c06c110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c67d2de76eb01782 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:11.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.510+0000 7fcc297fa700 1 -- 172.21.15.161:0/2692833085 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c068070 con 0x55745bf26ad0 2023-07-14T11:31:11.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.510+0000 7fcc297fa700 1 -- 172.21.15.161:0/2692833085 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c09a0a0 con 0x55745bf26ad0 2023-07-14T11:31:11.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.510+0000 7fcc297fa700 1 -- 172.21.15.161:0/2692833085 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c099790 con 0x55745bf26ad0 2023-07-14T11:31:11.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/2692833085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 msgr2=0x55745bf28eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc337fe700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcc200a12c0 0x7fcc200a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009d30 tx=0x7fcc2c00b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:11.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf28eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0a1fc0 tx=0x7fcc1c06c110 comp rx=0 tx=0).stop 2023-07-14T11:31:11.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/2692833085 shutdown_connections 2023-07-14T11:31:11.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf28eb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf1b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/2692833085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/2692833085 >> 172.21.15.161:0/2692833085 conn(0x55745bf11da0 msgr2=0x55745bf176a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:11.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/2692833085 shutdown_connections 2023-07-14T11:31:11.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/2692833085 wait complete. 2023-07-14T11:31:11.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.511+0000 7fcc4d1afb80 1 -- start start 2023-07-14T11:31:11.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 1 Processor -- start 2023-07-14T11:31:11.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf1b2d0 0x55745bf26560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf26ad0 0x55745bf23f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745be0dac0 con 0x55745bf1bbe0 2023-07-14T11:31:11.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745be0d7c0 con 0x55745bf26ad0 2023-07-14T11:31:11.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745bcf1920 con 0x55745bf1b2d0 2023-07-14T11:31:11.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc4d1afb80 10 client.0 _ll_register_callbacks cb 0x55745be0b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc32ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf1b2d0 0x55745bf26560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc32ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf1b2d0 0x55745bf26560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41896/0 (socket says 172.21.15.161:41896) 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc32ffd700 1 -- 172.21.15.161:0/1952572348 learned_addr learned my addr 172.21.15.161:0/1952572348 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc337fe700 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf26ad0 0x55745bf23f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.512+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc32ffd700 1 -- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf26ad0 msgr2=0x55745bf23f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc32ffd700 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf26ad0 0x55745bf23f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc32ffd700 1 -- 172.21.15.161:0/1952572348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 msgr2=0x55745bf237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc32ffd700 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc32ffd700 1 -- 172.21.15.161:0/1952572348 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc341a4a80 con 0x55745bf1b2d0 2023-07-14T11:31:11.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc32ffd700 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf1b2d0 0x55745bf26560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400c010 tx=0x7fcc24009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9183c2427f6d5d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:11.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc1a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:31:11.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/1952572348 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2401d030 con 0x55745bf1b2d0 2023-07-14T11:31:11.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55745bdc74f0 con 0x55745bf1b2d0 2023-07-14T11:31:11.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/1952572348 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fcc24004500 con 0x55745bf1b2d0 2023-07-14T11:31:11.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/1952572348 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc240046c0 con 0x55745bf1b2d0 2023-07-14T11:31:11.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc4d1afb80 10 client.26893 fetch_fsmap learned FSMap version 27 2023-07-14T11:31:11.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.513+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55745bc84c60 con 0x55745bf1b2d0 2023-07-14T11:31:11.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/1952572348 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55745bdc74f0 con 0x55745bf1b2d0 2023-07-14T11:31:11.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/1952572348 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcc2400f8a0 con 0x55745bf1b2d0 2023-07-14T11:31:11.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc4d1afb80 10 client.26893 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:31:11.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc4d1afb80 10 client.26893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc4d1afb80 20 client.26893 populate_metadata read hostname 'smithi161' 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf36480 0x55745bf38890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc4d1afb80 4 client.26893 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55745be66eb0 con 0x55745bf36480 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.514+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf36480 0x55745bf38890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.515+0000 7fcc1a7fc700 10 client.26893 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.515+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf36480 0x55745bf38890 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51ea770ea8dae94 server_cookie=8b2950e5de596ec6 in_seq=0 out_seq=0 2023-07-14T11:31:11.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.515+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/1952572348 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55745be66eb0 con 0x55745bf36480 2023-07-14T11:31:11.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc1a7fc700 10 client.26893 handle_command_reply: tid=0 2023-07-14T11:31:11.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 client.26893 shutdown 2023-07-14T11:31:11.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf1b2d0 msgr2=0x55745bf26560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf1b2d0 0x55745bf26560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400c010 tx=0x7fcc24009570 comp rx=0 tx=0).stop 2023-07-14T11:31:11.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 shutdown_connections 2023-07-14T11:31:11.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf36480 0x55745bf38890 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf26ad0 0x55745bf23f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf1b2d0 0x55745bf26560 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc1a7fc700 0 client.26893 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:11.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1952572348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 >> 172.21.15.161:0/1952572348 conn(0x55745bf11da0 msgr2=0x55745bf17230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:11.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 shutdown_connections 2023-07-14T11:31:11.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1952572348 wait complete. 2023-07-14T11:31:11.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.516+0000 7fcc4d1afb80 20 client.26893 trim_cache size 0 max 16384 2023-07-14T11:31:11.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.534+0000 7fcc4d1afb80 1 Processor -- start 2023-07-14T11:31:11.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc4d1afb80 1 -- start start 2023-07-14T11:31:11.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf26100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745bf32e60 con 0x55745bf1bbe0 2023-07-14T11:31:11.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745bf24da0 con 0x55745bf1b2d0 2023-07-14T11:31:11.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745bf1b8f0 con 0x55745bf26ad0 2023-07-14T11:31:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc32ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf22fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50430/0 (socket says 172.21.15.161:50430) 2023-07-14T11:31:11.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc2bfff700 1 -- 172.21.15.161:0/1502903843 learned_addr learned my addr 172.21.15.161:0/1502903843 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:11.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.535+0000 7fcc2bfff700 1 -- 172.21.15.161:0/1502903843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 msgr2=0x55745bf23500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/1502903843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf23500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc2bfff700 1 -- 172.21.15.161:0/1502903843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 msgr2=0x55745bf26100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/1502903843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf26100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc2bfff700 1 -- 172.21.15.161:0/1502903843 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55745bcc2250 con 0x55745bf1bbe0 2023-07-14T11:31:11.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/1502903843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf22fc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c09a030 tx=0x7fcc1c099d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869bcc512298d808 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:11.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc29ffb700 1 -- 172.21.15.161:0/1502903843 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0b1070 con 0x55745bf1bbe0 2023-07-14T11:31:11.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc29ffb700 1 -- 172.21.15.161:0/1502903843 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c06d050 con 0x55745bf1bbe0 2023-07-14T11:31:11.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.536+0000 7fcc29ffb700 1 -- 172.21.15.161:0/1502903843 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c068070 con 0x55745bf1bbe0 2023-07-14T11:31:11.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1502903843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 msgr2=0x55745bf22fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1502903843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf22fc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c09a030 tx=0x7fcc1c099d10 comp rx=0 tx=0).stop 2023-07-14T11:31:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1502903843 shutdown_connections 2023-07-14T11:31:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1502903843 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf23500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1502903843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1b2d0 0x55745bf26100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/1502903843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1bbe0 0x55745bf22fc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1502903843 >> 172.21.15.161:0/1502903843 conn(0x55745bf11da0 msgr2=0x55745bf1eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1502903843 shutdown_connections 2023-07-14T11:31:11.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/1502903843 wait complete. 2023-07-14T11:31:11.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 -- start start 2023-07-14T11:31:11.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.537+0000 7fcc4d1afb80 1 Processor -- start 2023-07-14T11:31:11.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1b2d0 0x55745bf2d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1bbe0 0x55745bf2b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc4d1afb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf2bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745bf2c090 con 0x55745bf1b2d0 2023-07-14T11:31:11.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745bf2c200 con 0x55745bf1bbe0 2023-07-14T11:31:11.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc4d1afb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745bf2c3c0 con 0x55745bf26ad0 2023-07-14T11:31:11.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc4d1afb80 10 client.0 _ll_register_callbacks cb 0x7fcc2400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:31:11.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc32ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1b2d0 0x55745bf2d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1bbe0 0x55745bf2b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf2bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1bbe0 0x55745bf2b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56544/0 (socket says 172.21.15.161:56544) 2023-07-14T11:31:11.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.538+0000 7fcc2bfff700 1 -- 172.21.15.161:0/14942175 learned_addr learned my addr 172.21.15.161:0/14942175 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:11.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc2bfff700 1 -- 172.21.15.161:0/14942175 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 msgr2=0x55745bf2bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf2bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc2bfff700 1 -- 172.21.15.161:0/14942175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1b2d0 msgr2=0x55745bf2d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:11.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1b2d0 0x55745bf2d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc2bfff700 1 -- 172.21.15.161:0/14942175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c0cae70 con 0x55745bf1bbe0 2023-07-14T11:31:11.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc337fe700 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf2bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc2bfff700 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1bbe0 0x55745bf2b610 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e9c0 tx=0x7fcc1c0cd410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2be1148ede5403 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc1a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/14942175 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c068070 con 0x55745bf1bbe0 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/14942175 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fcc1c098440 con 0x55745bf1bbe0 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc1c09a0a0 con 0x55745bf1bbe0 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/14942175 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc1c0b1070 con 0x55745bf1bbe0 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc4d1afb80 10 client.26897 fetch_fsmap learned FSMap version 27 2023-07-14T11:31:11.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.539+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55745bc9a2b0 con 0x55745bf1bbe0 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/14942175 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c0c8080 con 0x55745bf1bbe0 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/14942175 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcc1c078070 con 0x55745bf1bbe0 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc4d1afb80 10 client.26897 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc4d1afb80 10 client.26897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc4d1afb80 20 client.26897 populate_metadata read hostname 'smithi161' 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf8b310 0x55745bf8d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc4d1afb80 4 client.26897 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:31:11.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55745be29050 con 0x55745bf8b310 2023-07-14T11:31:11.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc32ffd700 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf8b310 0x55745bf8d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:11.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc32ffd700 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf8b310 0x55745bf8d720 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19785b3bf37a6f1f server_cookie=c873a5941f2bf6e2 in_seq=0 out_seq=0 2023-07-14T11:31:11.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.540+0000 7fcc1a7fc700 10 client.26897 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:11.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc1a7fc700 1 -- 172.21.15.161:0/14942175 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55745be29050 con 0x55745bf8b310 2023-07-14T11:31:11.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc1a7fc700 10 client.26897 handle_command_reply: tid=0 2023-07-14T11:31:11.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 client.26897 shutdown 2023-07-14T11:31:11.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1bbe0 msgr2=0x55745bf2b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1bbe0 0x55745bf2b610 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e9c0 tx=0x7fcc1c0cd410 comp rx=0 tx=0).stop 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 shutdown_connections 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55745bf8b310 0x55745bf8d720 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55745bf1b2d0 0x55745bf2d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55745bf26ad0 0x55745bf2bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc1a7fc700 0 client.26897 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 --2- 172.21.15.161:0/14942175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55745bf1bbe0 0x55745bf2b610 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 >> 172.21.15.161:0/14942175 conn(0x55745bf11da0 msgr2=0x55745bf1d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:11.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 shutdown_connections 2023-07-14T11:31:11.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 1 -- 172.21.15.161:0/14942175 wait complete. 2023-07-14T11:31:11.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.541+0000 7fcc4d1afb80 20 client.26897 trim_cache size 0 max 16384 2023-07-14T11:31:11.554 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:31:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:31:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:31:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:31:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:31:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:31:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:31:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:31:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:31:11.556 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:31:11.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcc200a12c0 msgr2=0x7fcc200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcc200a12c0 0x7fcc200a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009d30 tx=0x7fcc2c00b040 comp rx=0 tx=0).stop 2023-07-14T11:31:11.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc34100480 msgr2=0x7fcc34198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc34100480 0x7fcc34198370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e9f0 tx=0x7fcc1c00ed00 comp rx=0 tx=0).stop 2023-07-14T11:31:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 shutdown_connections 2023-07-14T11:31:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcc200a12c0 0x7fcc200a3770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcc34105ae0 0x7fcc3419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc340ffad0 0x7fcc34197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 --2- 172.21.15.161:0/622506404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc34100480 0x7fcc34198370 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:11.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 >> 172.21.15.161:0/622506404 conn(0x7fcc340fb5f0 msgr2=0x7fcc3410aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:11.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 shutdown_connections 2023-07-14T11:31:11.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:11.542+0000 7fcc397ff700 1 -- 172.21.15.161:0/622506404 wait complete. 2023-07-14T11:31:11.564 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:31:12.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 818065 9.03 MB/sec execute 354 sec latency 17.776 ms 2023-07-14T11:31:12.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:12 smithi175 ceph-mon[129081]: pgmap v277: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2023-07-14T11:31:12.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:12 smithi182 ceph-mon[129366]: pgmap v277: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2023-07-14T11:31:12.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:12 smithi161 ceph-mon[95526]: pgmap v277: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2023-07-14T11:31:13.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 820100 9.04 MB/sec execute 355 sec latency 69.780 ms 2023-07-14T11:31:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:13 smithi175 ceph-mon[129081]: pgmap v278: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-07-14T11:31:14.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 821079 9.02 MB/sec execute 356 sec latency 22.305 ms 2023-07-14T11:31:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:13 smithi182 ceph-mon[129366]: pgmap v278: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-07-14T11:31:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:13 smithi161 ceph-mon[95526]: pgmap v278: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-07-14T11:31:15.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 822626 9.01 MB/sec execute 357 sec latency 55.681 ms 2023-07-14T11:31:16.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 824240 9.02 MB/sec execute 358 sec latency 13.867 ms 2023-07-14T11:31:16.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:16 smithi161 ceph-mon[95526]: pgmap v279: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-07-14T11:31:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:16 smithi175 ceph-mon[129081]: pgmap v279: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-07-14T11:31:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:16 smithi182 ceph-mon[129366]: pgmap v279: 129 pgs: 129 active+clean; 619 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-07-14T11:31:17.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 826138 9.01 MB/sec execute 359 sec latency 17.016 ms 2023-07-14T11:31:18.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 827688 9.02 MB/sec execute 360 sec latency 41.297 ms 2023-07-14T11:31:18.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:18 smithi161 ceph-mon[95526]: pgmap v280: 129 pgs: 129 active+clean; 628 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 223 op/s 2023-07-14T11:31:18.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:18 smithi175 ceph-mon[129081]: pgmap v280: 129 pgs: 129 active+clean; 628 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 223 op/s 2023-07-14T11:31:18.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:18 smithi182 ceph-mon[129366]: pgmap v280: 129 pgs: 129 active+clean; 628 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 223 op/s 2023-07-14T11:31:19.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 829207 9.01 MB/sec execute 361 sec latency 18.415 ms 2023-07-14T11:31:20.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 831052 9.02 MB/sec execute 362 sec latency 39.337 ms 2023-07-14T11:31:20.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:20 smithi161 ceph-mon[95526]: pgmap v281: 129 pgs: 129 active+clean; 628 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 197 op/s 2023-07-14T11:31:20.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:20 smithi175 ceph-mon[129081]: pgmap v281: 129 pgs: 129 active+clean; 628 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 197 op/s 2023-07-14T11:31:20.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:20 smithi182 ceph-mon[129366]: pgmap v281: 129 pgs: 129 active+clean; 628 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 197 op/s 2023-07-14T11:31:21.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 832247 9.01 MB/sec execute 363 sec latency 23.561 ms 2023-07-14T11:31:22.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 834538 9.03 MB/sec execute 364 sec latency 14.090 ms 2023-07-14T11:31:22.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:22 smithi161 ceph-mon[95526]: pgmap v282: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 283 op/s 2023-07-14T11:31:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:22 smithi175 ceph-mon[129081]: pgmap v282: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 283 op/s 2023-07-14T11:31:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:22 smithi182 ceph-mon[129366]: pgmap v282: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 283 op/s 2023-07-14T11:31:23.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 835570 9.01 MB/sec execute 365 sec latency 21.322 ms 2023-07-14T11:31:24.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:23 smithi175 ceph-mon[129081]: pgmap v283: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2023-07-14T11:31:24.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 837910 9.03 MB/sec execute 366 sec latency 31.257 ms 2023-07-14T11:31:24.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:23 smithi182 ceph-mon[129366]: pgmap v283: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2023-07-14T11:31:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:23 smithi161 ceph-mon[95526]: pgmap v283: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2023-07-14T11:31:25.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 838729 9.01 MB/sec execute 367 sec latency 39.882 ms 2023-07-14T11:31:26.199 INFO:tasks.workunit.client.0.smithi161.stdout: 1 840821 9.01 MB/sec execute 368 sec latency 32.324 ms 2023-07-14T11:31:26.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:26 smithi161 ceph-mon[95526]: pgmap v284: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2023-07-14T11:31:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:26 smithi175 ceph-mon[129081]: pgmap v284: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2023-07-14T11:31:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:26 smithi182 ceph-mon[129366]: pgmap v284: 129 pgs: 129 active+clean; 638 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2023-07-14T11:31:27.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 842050 9.01 MB/sec execute 369 sec latency 33.053 ms 2023-07-14T11:31:28.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 844074 9.01 MB/sec execute 370 sec latency 19.809 ms 2023-07-14T11:31:28.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:28 smithi161 ceph-mon[95526]: pgmap v285: 129 pgs: 129 active+clean; 648 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 286 op/s 2023-07-14T11:31:28.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:28 smithi175 ceph-mon[129081]: pgmap v285: 129 pgs: 129 active+clean; 648 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 286 op/s 2023-07-14T11:31:28.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:28 smithi182 ceph-mon[129366]: pgmap v285: 129 pgs: 129 active+clean; 648 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 286 op/s 2023-07-14T11:31:29.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 845614 9.01 MB/sec execute 371 sec latency 13.531 ms 2023-07-14T11:31:30.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 846796 9.00 MB/sec execute 372 sec latency 44.823 ms 2023-07-14T11:31:30.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:30 smithi161 ceph-mon[95526]: pgmap v286: 129 pgs: 129 active+clean; 648 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2023-07-14T11:31:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:30 smithi175 ceph-mon[129081]: pgmap v286: 129 pgs: 129 active+clean; 648 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2023-07-14T11:31:30.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:30 smithi182 ceph-mon[129366]: pgmap v286: 129 pgs: 129 active+clean; 648 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2023-07-14T11:31:31.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 848870 9.01 MB/sec execute 373 sec latency 29.639 ms 2023-07-14T11:31:32.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 850222 9.00 MB/sec execute 374 sec latency 16.397 ms 2023-07-14T11:31:32.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:32 smithi161 ceph-mon[95526]: pgmap v287: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 297 op/s 2023-07-14T11:31:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:32 smithi175 ceph-mon[129081]: pgmap v287: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 297 op/s 2023-07-14T11:31:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:32 smithi182 ceph-mon[129366]: pgmap v287: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 297 op/s 2023-07-14T11:31:33.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 852363 9.02 MB/sec execute 375 sec latency 31.563 ms 2023-07-14T11:31:34.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:31:34.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:33 smithi175 ceph-mon[129081]: pgmap v288: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2023-07-14T11:31:34.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:31:34.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:31:34.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:31:34.234 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:31:34.234 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:33 smithi182 ceph-mon[129366]: pgmap v288: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2023-07-14T11:31:34.234 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:31:34.234 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:31:34.234 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:31:34.235 INFO:tasks.workunit.client.0.smithi161.stdout: 1 853521 9.01 MB/sec execute 376 sec latency 20.356 ms 2023-07-14T11:31:34.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:31:34.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:33 smithi161 ceph-mon[95526]: pgmap v288: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2023-07-14T11:31:34.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:31:34.250 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:31:34.251 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:31:35.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 855473 9.01 MB/sec execute 377 sec latency 39.342 ms 2023-07-14T11:31:36.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 856561 9.00 MB/sec execute 378 sec latency 26.863 ms 2023-07-14T11:31:36.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:36 smithi161 ceph-mon[95526]: pgmap v289: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2023-07-14T11:31:36.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:36 smithi175 ceph-mon[129081]: pgmap v289: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2023-07-14T11:31:36.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:36 smithi182 ceph-mon[129366]: pgmap v289: 129 pgs: 129 active+clean; 657 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2023-07-14T11:31:37.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 858940 9.01 MB/sec execute 379 sec latency 19.266 ms 2023-07-14T11:31:38.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 860126 9.00 MB/sec execute 380 sec latency 13.326 ms 2023-07-14T11:31:38.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:38 smithi161 ceph-mon[95526]: pgmap v290: 129 pgs: 129 active+clean; 661 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-07-14T11:31:38.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:38 smithi175 ceph-mon[129081]: pgmap v290: 129 pgs: 129 active+clean; 661 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-07-14T11:31:38.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:38 smithi182 ceph-mon[129366]: pgmap v290: 129 pgs: 129 active+clean; 661 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-07-14T11:31:39.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 862374 9.01 MB/sec execute 381 sec latency 17.414 ms 2023-07-14T11:31:40.200 INFO:tasks.workunit.client.0.smithi161.stdout: 1 863337 9.00 MB/sec execute 382 sec latency 63.054 ms 2023-07-14T11:31:40.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:40 smithi161 ceph-mon[95526]: pgmap v291: 129 pgs: 129 active+clean; 661 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2023-07-14T11:31:40.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:40 smithi175 ceph-mon[129081]: pgmap v291: 129 pgs: 129 active+clean; 661 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2023-07-14T11:31:40.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:40 smithi182 ceph-mon[129366]: pgmap v291: 129 pgs: 129 active+clean; 661 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2023-07-14T11:31:41.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 864605 8.99 MB/sec execute 383 sec latency 49.769 ms 2023-07-14T11:31:41.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:31:41.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 -- 172.21.15.161:0/2015144438 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec10c380 msgr2=0x7fc9ec10e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/2015144438 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec10c380 0x7fc9ec10e760 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009f90 tx=0x7fc9e400b330 comp rx=0 tx=0).stop 2023-07-14T11:31:41.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 -- 172.21.15.161:0/2015144438 shutdown_connections 2023-07-14T11:31:41.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/2015144438 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec10c380 0x7fc9ec10e760 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/2015144438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec072850 0x7fc9ec072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/2015144438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ec073160 0x7fc9ec10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 -- 172.21.15.161:0/2015144438 >> 172.21.15.161:0/2015144438 conn(0x7fc9ec0fb5e0 msgr2=0x7fc9ec0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:41.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 -- 172.21.15.161:0/2015144438 shutdown_connections 2023-07-14T11:31:41.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.780+0000 7fc9f1bff700 1 -- 172.21.15.161:0/2015144438 wait complete. 2023-07-14T11:31:41.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.781+0000 7fc9f1bff700 1 Processor -- start 2023-07-14T11:31:41.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.781+0000 7fc9f1bff700 1 -- start start 2023-07-14T11:31:41.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9f1bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ec072850 0x7fc9ec197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9f1bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec073160 0x7fc9ec1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9f1bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec10c380 0x7fc9ec19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9f1bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19cf50 con 0x7fc9ec072850 2023-07-14T11:31:41.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9f1bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19d0c0 con 0x7fc9ec10c380 2023-07-14T11:31:41.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9f1bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19d280 con 0x7fc9ec073160 2023-07-14T11:31:41.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec073160 0x7fc9ec1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec10c380 0x7fc9ec19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec073160 0x7fc9ec1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:40844/0 (socket says 172.21.15.161:40844) 2023-07-14T11:31:41.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ec072850 0x7fc9ec197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3840701356 learned_addr learned my addr 172.21.15.161:0/3840701356 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:41.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3840701356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec10c380 msgr2=0x7fc9ec19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec10c380 0x7fc9ec19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ec072850 msgr2=0x7fc9ec197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ec072850 0x7fc9ec197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3840701356 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4009c40 con 0x7fc9ec073160 2023-07-14T11:31:41.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9ebfff700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec10c380 0x7fc9ec19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:31:41.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.782+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ec072850 0x7fc9ec197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:31:41.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.783+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec073160 0x7fc9ec1984b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000e9c0 tx=0x7fc9e000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:41.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.783+0000 7fc9e8ff9700 1 -- 172.21.15.161:0/3840701356 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0004a30 con 0x7fc9ec073160 2023-07-14T11:31:41.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.783+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec19d560 con 0x7fc9ec073160 2023-07-14T11:31:41.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.783+0000 7fc9e8ff9700 1 -- 172.21.15.161:0/3840701356 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e000cc80 con 0x7fc9ec073160 2023-07-14T11:31:41.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.783+0000 7fc9e8ff9700 1 -- 172.21.15.161:0/3840701356 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00054e0 con 0x7fc9ec073160 2023-07-14T11:31:41.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.783+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec1a4da0 con 0x7fc9ec073160 2023-07-14T11:31:41.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.786+0000 7fc9e8ff9700 1 -- 172.21.15.161:0/3840701356 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc9e0004b90 con 0x7fc9ec073160 2023-07-14T11:31:41.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.787+0000 7fca0548db80 1 Processor -- start 2023-07-14T11:31:41.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.787+0000 7fca0548db80 1 -- start start 2023-07-14T11:31:41.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.787+0000 7fca0548db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f732ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.787+0000 7fca0548db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ffbe0 0x559f7330a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fca0548db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f7330ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fca0548db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73205cb0 con 0x559f732ffbe0 2023-07-14T11:31:41.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fca0548db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73258a40 con 0x559f732ff2d0 2023-07-14T11:31:41.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fca0548db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73251ed0 con 0x559f7330aad0 2023-07-14T11:31:41.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f732ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f732ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48354/0 (socket says 172.21.15.161:48354) 2023-07-14T11:31:41.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 -- 172.21.15.161:0/4241159402 learned_addr learned my addr 172.21.15.161:0/4241159402 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:41.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f7330ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ffbe0 0x559f7330a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 -- 172.21.15.161:0/4241159402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 msgr2=0x559f7330ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f7330ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 -- 172.21.15.161:0/4241159402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ffbe0 msgr2=0x559f7330a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ffbe0 0x559f7330a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 -- 172.21.15.161:0/4241159402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f7307fde0 con 0x559f732ff2d0 2023-07-14T11:31:41.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9e8ff9700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9d40a1340 0x7fc9d40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ffbe0 0x559f7330a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:31:41.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9e8ff9700 1 -- 172.21.15.161:0/3840701356 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fc9e0014070 con 0x7fc9ec073160 2023-07-14T11:31:41.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.788+0000 7fc9ebfff700 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f732ff6a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4015000 tx=0x7fc9e40078b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10dd76433388cb51 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:41.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.789+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9d40a1340 0x7fc9d40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.789+0000 7fc9d1ffb700 1 -- 172.21.15.161:0/4241159402 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e400e040 con 0x559f732ff2d0 2023-07-14T11:31:41.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.789+0000 7fc9d1ffb700 1 -- 172.21.15.161:0/4241159402 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e4007cd0 con 0x559f732ff2d0 2023-07-14T11:31:41.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.789+0000 7fc9d1ffb700 1 -- 172.21.15.161:0/4241159402 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e401c6a0 con 0x559f732ff2d0 2023-07-14T11:31:41.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.789+0000 7fca0548db80 1 -- 172.21.15.161:0/4241159402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 msgr2=0x559f732ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.789+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9d40a1340 0x7fc9d40a37f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc012660 tx=0x7fc9dc009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:41.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.789+0000 7fca0548db80 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f732ff6a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4015000 tx=0x7fc9e40078b0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 -- 172.21.15.161:0/4241159402 shutdown_connections 2023-07-14T11:31:41.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f7330ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f732ff6a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 --2- 172.21.15.161:0/4241159402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ffbe0 0x559f7330a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 -- 172.21.15.161:0/4241159402 >> 172.21.15.161:0/4241159402 conn(0x559f732f5da0 msgr2=0x559f732fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:41.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 -- 172.21.15.161:0/4241159402 shutdown_connections 2023-07-14T11:31:41.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 -- 172.21.15.161:0/4241159402 wait complete. 2023-07-14T11:31:41.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.790+0000 7fca0548db80 1 -- start start 2023-07-14T11:31:41.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 1 Processor -- start 2023-07-14T11:31:41.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f7330a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f73307710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f7330aad0 0x559f73307d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f731f1ac0 con 0x559f7330aad0 2023-07-14T11:31:41.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f731f17c0 con 0x559f732ff2d0 2023-07-14T11:31:41.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f730d5920 con 0x559f732ffbe0 2023-07-14T11:31:41.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fca0548db80 10 client.0 _ll_register_callbacks cb 0x559f731ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:31:41.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f73307710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f73307710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:40850/0 (socket says 172.21.15.161:40850) 2023-07-14T11:31:41.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1082142447 learned_addr learned my addr 172.21.15.161:0/1082142447 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:41.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fc9ebfff700 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f7330aad0 0x559f73307d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.791+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f7330a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 msgr2=0x559f7330a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f7330a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1082142447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f7330aad0 msgr2=0x559f73307d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f7330aad0 0x559f73307d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1082142447 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec10f3a0 con 0x559f732ffbe0 2023-07-14T11:31:41.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f73307710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00962a0 tx=0x7fc9e00960e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d791740ebe72a6f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:41.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9caffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:31:41.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9caffd700 1 -- 172.21.15.161:0/1082142447 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e009a200 con 0x559f732ffbe0 2023-07-14T11:31:41.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9caffd700 1 -- 172.21.15.161:0/1082142447 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fc9e000cdf0 con 0x559f732ffbe0 2023-07-14T11:31:41.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f731ab4f0 con 0x559f732ffbe0 2023-07-14T11:31:41.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fc9caffd700 1 -- 172.21.15.161:0/1082142447 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e0099700 con 0x559f732ffbe0 2023-07-14T11:31:41.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fca0548db80 10 client.26911 fetch_fsmap learned FSMap version 27 2023-07-14T11:31:41.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.792+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f73068c60 con 0x559f732ffbe0 2023-07-14T11:31:41.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fc9caffd700 1 -- 172.21.15.161:0/1082142447 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e00b6080 con 0x559f732ffbe0 2023-07-14T11:31:41.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fc9caffd700 1 -- 172.21.15.161:0/1082142447 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9e0099290 con 0x559f732ffbe0 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fca0548db80 10 client.26911 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fca0548db80 10 client.26911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fca0548db80 20 client.26911 populate_metadata read hostname 'smithi161' 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fca0548db80 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7331a410 0x559f7331c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fca0548db80 4 client.26911 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f7324aeb0 con 0x559f7331a410 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.793+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7331a410 0x559f7331c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.794+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7331a410 0x559f7331c820 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f1127a02247ac4 server_cookie=351f8ae97f345dc9 in_seq=0 out_seq=0 2023-07-14T11:31:41.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.794+0000 7fc9caffd700 10 client.26911 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:41.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.794+0000 7fc9caffd700 1 -- 172.21.15.161:0/1082142447 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559f7324aeb0 con 0x559f7331a410 2023-07-14T11:31:41.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.794+0000 7fc9caffd700 10 client.26911 handle_command_reply: tid=0 2023-07-14T11:31:41.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.794+0000 7fca0548db80 1 client.26911 shutdown 2023-07-14T11:31:41.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 msgr2=0x559f73307710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f73307710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00962a0 tx=0x7fc9e00960e0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 shutdown_connections 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7331a410 0x559f7331c820 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f7330aad0 0x559f73307d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ff2d0 0x559f7330a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 --2- 172.21.15.161:0/1082142447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f73307710 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fc9caffd700 0 client.26911 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 >> 172.21.15.161:0/1082142447 conn(0x559f732f5da0 msgr2=0x559f732fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 shutdown_connections 2023-07-14T11:31:41.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 1 -- 172.21.15.161:0/1082142447 wait complete. 2023-07-14T11:31:41.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.795+0000 7fca0548db80 20 client.26911 trim_cache size 0 max 16384 2023-07-14T11:31:41.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.813+0000 7fca0548db80 1 Processor -- start 2023-07-14T11:31:41.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fca0548db80 1 -- start start 2023-07-14T11:31:41.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fca0548db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73302a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fca0548db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ffbe0 0x559f73302f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fca0548db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f73306ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fca0548db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73300dd0 con 0x559f732ff2d0 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fca0548db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73316cd0 con 0x559f732ffbe0 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fca0548db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73307fa0 con 0x559f7330aad0 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ffbe0 0x559f73302f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fc9eb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73302a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ffbe0 0x559f73302f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:48372/0 (socket says 172.21.15.161:48372) 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3801697706 learned_addr learned my addr 172.21.15.161:0/3801697706 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:41.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.814+0000 7fc9ebfff700 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f73306ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3801697706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 msgr2=0x559f73306ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f73306ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3801697706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 msgr2=0x559f73302a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73302a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9eaffd700 1 -- 172.21.15.161:0/3801697706 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f730a6250 con 0x559f732ffbe0 2023-07-14T11:31:41.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73302a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:31:41.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ffbe0 0x559f73302f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c5b0 tx=0x7fc9e0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de271443b6ae1c05 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:41.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9d27fc700 1 -- 172.21.15.161:0/3801697706 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00b81c0 con 0x559f732ffbe0 2023-07-14T11:31:41.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9d27fc700 1 -- 172.21.15.161:0/3801697706 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f731ab4f0 con 0x559f732ffbe0 2023-07-14T11:31:41.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.815+0000 7fc9d27fc700 1 -- 172.21.15.161:0/3801697706 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e009ddd0 con 0x559f732ffbe0 2023-07-14T11:31:41.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 -- 172.21.15.161:0/3801697706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ffbe0 msgr2=0x559f73302f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ffbe0 0x559f73302f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c5b0 tx=0x7fc9e0005850 comp rx=0 tx=0).stop 2023-07-14T11:31:41.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 -- 172.21.15.161:0/3801697706 shutdown_connections 2023-07-14T11:31:41.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73302a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f7330aad0 0x559f73306ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 --2- 172.21.15.161:0/3801697706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f732ffbe0 0x559f73302f70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 -- 172.21.15.161:0/3801697706 >> 172.21.15.161:0/3801697706 conn(0x559f732f5da0 msgr2=0x559f733165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:41.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 -- 172.21.15.161:0/3801697706 shutdown_connections 2023-07-14T11:31:41.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 -- 172.21.15.161:0/3801697706 wait complete. 2023-07-14T11:31:41.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.816+0000 7fca0548db80 1 -- start start 2023-07-14T11:31:41.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.817+0000 7fca0548db80 1 Processor -- start 2023-07-14T11:31:41.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.817+0000 7fca0548db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73311480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.817+0000 7fca0548db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f7330f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fca0548db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f7330aad0 0x559f7330fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fca0548db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73310490 con 0x559f732ff2d0 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fca0548db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73310600 con 0x559f7330aad0 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fca0548db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7336e2e0 con 0x559f732ffbe0 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fca0548db80 10 client.0 _ll_register_callbacks cb 0x7fc9e0009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73311480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f7330f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f7330f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:40868/0 (socket says 172.21.15.161:40868) 2023-07-14T11:31:41.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1066476400 learned_addr learned my addr 172.21.15.161:0/1066476400 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:31:41.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9ebfff700 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f7330aad0 0x559f7330fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f7330aad0 msgr2=0x559f7330fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f7330aad0 0x559f7330fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1066476400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 msgr2=0x559f73311480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:31:41.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73311480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 -- 172.21.15.161:0/1066476400 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc02ca50 con 0x559f732ffbe0 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9eaffd700 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f7330f880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000c00 tx=0x7fc9e000cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c997651abe836a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.818+0000 7fc9caffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fc9caffd700 1 -- 172.21.15.161:0/1066476400 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00b81c0 con 0x559f732ffbe0 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fc9caffd700 1 -- 172.21.15.161:0/1066476400 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fc9e009ddd0 con 0x559f732ffbe0 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9e4007cd0 con 0x559f732ffbe0 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fc9caffd700 1 -- 172.21.15.161:0/1066476400 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e0099700 con 0x559f732ffbe0 2023-07-14T11:31:41.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 10 client.26917 fetch_fsmap learned FSMap version 27 2023-07-14T11:31:41.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f73068c60 con 0x559f732ffbe0 2023-07-14T11:31:41.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fc9caffd700 1 -- 172.21.15.161:0/1066476400 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e4007cd0 con 0x559f732ffbe0 2023-07-14T11:31:41.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fc9caffd700 1 -- 172.21.15.161:0/1066476400 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9e009a000 con 0x559f732ffbe0 2023-07-14T11:31:41.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 10 client.26917 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:31:41.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 10 client.26917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:31:41.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 20 client.26917 populate_metadata read hostname 'smithi161' 2023-07-14T11:31:41.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7336f380 0x559f73371760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:31:41.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 4 client.26917 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:31:41.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.819+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f7320d050 con 0x559f7336f380 2023-07-14T11:31:41.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.820+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7336f380 0x559f73371760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:31:41.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.820+0000 7fc9eb7fe700 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7336f380 0x559f73371760 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e672eddb6cf3d278 server_cookie=838210999219ab4f in_seq=0 out_seq=0 2023-07-14T11:31:41.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.820+0000 7fc9caffd700 10 client.26917 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:41.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.820+0000 7fc9caffd700 1 -- 172.21.15.161:0/1066476400 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559f7320d050 con 0x559f7336f380 2023-07-14T11:31:41.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.820+0000 7fc9caffd700 10 client.26917 handle_command_reply: tid=0 2023-07-14T11:31:41.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.820+0000 7fca0548db80 1 client.26917 shutdown 2023-07-14T11:31:41.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 msgr2=0x559f7330f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f7330f880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000c00 tx=0x7fc9e000cdf0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 shutdown_connections 2023-07-14T11:31:41.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559f7336f380 0x559f73371760 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559f732ff2d0 0x559f73311480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f7330aad0 0x559f7330fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fc9caffd700 0 client.26917 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:31:41.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 --2- 172.21.15.161:0/1066476400 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559f732ffbe0 0x559f7330f880 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 >> 172.21.15.161:0/1066476400 conn(0x559f732f5da0 msgr2=0x559f73302a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:41.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 shutdown_connections 2023-07-14T11:31:41.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 1 -- 172.21.15.161:0/1066476400 wait complete. 2023-07-14T11:31:41.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fca0548db80 20 client.26917 trim_cache size 0 max 16384 2023-07-14T11:31:41.833 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:31:41.833 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:31:41.833 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:31:41.833 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:31:41.833 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:31:41.833 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:31:41.833 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:31:41.834 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:31:41.834 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:31:41.834 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:31:41.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9d40a1340 msgr2=0x7fc9d40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9d40a1340 0x7fc9d40a37f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc012660 tx=0x7fc9dc009260 comp rx=0 tx=0).stop 2023-07-14T11:31:41.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec073160 msgr2=0x7fc9ec1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:31:41.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.821+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec073160 0x7fc9ec1984b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000e9c0 tx=0x7fc9e000ecd0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 shutdown_connections 2023-07-14T11:31:41.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9d40a1340 0x7fc9d40a37f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ec072850 0x7fc9ec197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec10c380 0x7fc9ec19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 --2- 172.21.15.161:0/3840701356 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9ec073160 0x7fc9ec1984b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:31:41.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 >> 172.21.15.161:0/3840701356 conn(0x7fc9ec0fb5e0 msgr2=0x7fc9ec10e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:31:41.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 shutdown_connections 2023-07-14T11:31:41.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:31:41.822+0000 7fc9f1bff700 1 -- 172.21.15.161:0/3840701356 wait complete. 2023-07-14T11:31:41.843 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:31:42.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 866585 9.00 MB/sec execute 384 sec latency 15.048 ms 2023-07-14T11:31:42.497 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:42 smithi161 ceph-mon[95526]: pgmap v292: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 285 op/s 2023-07-14T11:31:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:42 smithi175 ceph-mon[129081]: pgmap v292: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 285 op/s 2023-07-14T11:31:42.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:42 smithi182 ceph-mon[129366]: pgmap v292: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 285 op/s 2023-07-14T11:31:43.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 867719 8.99 MB/sec execute 385 sec latency 51.820 ms 2023-07-14T11:31:44.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:43 smithi175 ceph-mon[129081]: pgmap v293: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2023-07-14T11:31:44.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 869946 9.00 MB/sec execute 386 sec latency 93.919 ms 2023-07-14T11:31:44.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:43 smithi182 ceph-mon[129366]: pgmap v293: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2023-07-14T11:31:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:43 smithi161 ceph-mon[95526]: pgmap v293: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2023-07-14T11:31:45.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 870908 8.99 MB/sec execute 387 sec latency 72.449 ms 2023-07-14T11:31:46.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 872841 8.99 MB/sec execute 388 sec latency 44.699 ms 2023-07-14T11:31:46.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:46 smithi161 ceph-mon[95526]: pgmap v294: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2023-07-14T11:31:46.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:46 smithi175 ceph-mon[129081]: pgmap v294: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2023-07-14T11:31:46.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:46 smithi182 ceph-mon[129366]: pgmap v294: 129 pgs: 129 active+clean; 665 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2023-07-14T11:31:47.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 874211 8.99 MB/sec execute 389 sec latency 19.691 ms 2023-07-14T11:31:48.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 876406 9.00 MB/sec execute 390 sec latency 19.497 ms 2023-07-14T11:31:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:48 smithi175 ceph-mon[129081]: pgmap v295: 129 pgs: 129 active+clean; 672 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 271 op/s 2023-07-14T11:31:48.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:48 smithi182 ceph-mon[129366]: pgmap v295: 129 pgs: 129 active+clean; 672 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 271 op/s 2023-07-14T11:31:48.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:48 smithi161 ceph-mon[95526]: pgmap v295: 129 pgs: 129 active+clean; 672 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 271 op/s 2023-07-14T11:31:49.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 877750 8.99 MB/sec execute 391 sec latency 12.131 ms 2023-07-14T11:31:50.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 879906 9.00 MB/sec execute 392 sec latency 25.645 ms 2023-07-14T11:31:50.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:50 smithi175 ceph-mon[129081]: pgmap v296: 129 pgs: 129 active+clean; 672 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2023-07-14T11:31:50.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:50 smithi182 ceph-mon[129366]: pgmap v296: 129 pgs: 129 active+clean; 672 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2023-07-14T11:31:50.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:50 smithi161 ceph-mon[95526]: pgmap v296: 129 pgs: 129 active+clean; 672 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2023-07-14T11:31:51.201 INFO:tasks.workunit.client.0.smithi161.stdout: 1 880949 8.99 MB/sec execute 393 sec latency 45.693 ms 2023-07-14T11:31:52.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 882812 8.99 MB/sec execute 394 sec latency 26.922 ms 2023-07-14T11:31:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:52 smithi175 ceph-mon[129081]: pgmap v297: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2023-07-14T11:31:52.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:52 smithi182 ceph-mon[129366]: pgmap v297: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2023-07-14T11:31:52.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:52 smithi161 ceph-mon[95526]: pgmap v297: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2023-07-14T11:31:53.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 884574 9.00 MB/sec execute 395 sec latency 14.800 ms 2023-07-14T11:31:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:53 smithi175 ceph-mon[129081]: pgmap v298: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2023-07-14T11:31:54.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 886589 8.99 MB/sec execute 396 sec latency 15.732 ms 2023-07-14T11:31:54.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:53 smithi182 ceph-mon[129366]: pgmap v298: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2023-07-14T11:31:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:53 smithi161 ceph-mon[95526]: pgmap v298: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2023-07-14T11:31:55.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 888169 9.00 MB/sec execute 397 sec latency 11.947 ms 2023-07-14T11:31:56.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 889392 8.99 MB/sec execute 398 sec latency 51.166 ms 2023-07-14T11:31:56.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:56 smithi161 ceph-mon[95526]: pgmap v299: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2023-07-14T11:31:56.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:56 smithi175 ceph-mon[129081]: pgmap v299: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2023-07-14T11:31:56.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:56 smithi182 ceph-mon[129366]: pgmap v299: 129 pgs: 129 active+clean; 677 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2023-07-14T11:31:57.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 891372 9.00 MB/sec execute 399 sec latency 16.240 ms 2023-07-14T11:31:58.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 892493 8.99 MB/sec execute 400 sec latency 51.493 ms 2023-07-14T11:31:58.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:31:58 smithi161 ceph-mon[95526]: pgmap v300: 129 pgs: 129 active+clean; 682 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2023-07-14T11:31:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:31:58 smithi175 ceph-mon[129081]: pgmap v300: 129 pgs: 129 active+clean; 682 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2023-07-14T11:31:58.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:31:58 smithi182 ceph-mon[129366]: pgmap v300: 129 pgs: 129 active+clean; 682 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2023-07-14T11:31:59.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 894732 9.00 MB/sec execute 401 sec latency 16.563 ms 2023-07-14T11:32:00.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 895730 8.99 MB/sec execute 402 sec latency 15.455 ms 2023-07-14T11:32:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:00 smithi175 ceph-mon[129081]: pgmap v301: 129 pgs: 129 active+clean; 682 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:32:00.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:00 smithi182 ceph-mon[129366]: pgmap v301: 129 pgs: 129 active+clean; 682 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:32:00.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:00 smithi161 ceph-mon[95526]: pgmap v301: 129 pgs: 129 active+clean; 682 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2023-07-14T11:32:01.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 896928 8.97 MB/sec execute 403 sec latency 85.460 ms 2023-07-14T11:32:02.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 897551 8.97 MB/sec execute 404 sec latency 127.199 ms 2023-07-14T11:32:02.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:02 smithi175 ceph-mon[129081]: pgmap v302: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 299 op/s 2023-07-14T11:32:02.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:02 smithi175 ceph-mon[129081]: osdmap e100: 12 total, 12 up, 12 in 2023-07-14T11:32:02.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:02 smithi182 ceph-mon[129366]: pgmap v302: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 299 op/s 2023-07-14T11:32:02.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:02 smithi182 ceph-mon[129366]: osdmap e100: 12 total, 12 up, 12 in 2023-07-14T11:32:02.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:02 smithi161 ceph-mon[95526]: pgmap v302: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 299 op/s 2023-07-14T11:32:02.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:02 smithi161 ceph-mon[95526]: osdmap e100: 12 total, 12 up, 12 in 2023-07-14T11:32:03.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 898748 8.96 MB/sec execute 405 sec latency 26.062 ms 2023-07-14T11:32:04.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:03 smithi175 ceph-mon[129081]: pgmap v304: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2023-07-14T11:32:04.202 INFO:tasks.workunit.client.0.smithi161.stdout: 1 900341 8.96 MB/sec execute 406 sec latency 56.700 ms 2023-07-14T11:32:04.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:03 smithi182 ceph-mon[129366]: pgmap v304: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2023-07-14T11:32:04.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:03 smithi161 ceph-mon[95526]: pgmap v304: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2023-07-14T11:32:05.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 902136 8.97 MB/sec execute 407 sec latency 16.645 ms 2023-07-14T11:32:06.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 903257 8.96 MB/sec execute 408 sec latency 61.954 ms 2023-07-14T11:32:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:06 smithi175 ceph-mon[129081]: pgmap v305: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2023-07-14T11:32:06.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:06 smithi182 ceph-mon[129366]: pgmap v305: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2023-07-14T11:32:06.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:06 smithi161 ceph-mon[95526]: pgmap v305: 129 pgs: 129 active+clean; 688 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2023-07-14T11:32:07.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 905430 8.97 MB/sec execute 409 sec latency 19.236 ms 2023-07-14T11:32:08.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 906426 8.95 MB/sec execute 410 sec latency 33.324 ms 2023-07-14T11:32:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:08 smithi175 ceph-mon[129081]: pgmap v306: 129 pgs: 129 active+clean; 690 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2023-07-14T11:32:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:08 smithi175 ceph-mon[129081]: osdmap e101: 12 total, 12 up, 12 in 2023-07-14T11:32:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:08 smithi182 ceph-mon[129366]: pgmap v306: 129 pgs: 129 active+clean; 690 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2023-07-14T11:32:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:08 smithi182 ceph-mon[129366]: osdmap e101: 12 total, 12 up, 12 in 2023-07-14T11:32:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:08 smithi161 ceph-mon[95526]: pgmap v306: 129 pgs: 129 active+clean; 690 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2023-07-14T11:32:08.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:08 smithi161 ceph-mon[95526]: osdmap e101: 12 total, 12 up, 12 in 2023-07-14T11:32:09.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 908898 8.97 MB/sec execute 411 sec latency 13.766 ms 2023-07-14T11:32:10.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 909947 8.95 MB/sec execute 412 sec latency 12.525 ms 2023-07-14T11:32:10.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:10 smithi175 ceph-mon[129081]: pgmap v308: 129 pgs: 129 active+clean; 690 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.2 MiB/s wr, 146 op/s 2023-07-14T11:32:10.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:10 smithi182 ceph-mon[129366]: pgmap v308: 129 pgs: 129 active+clean; 690 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.2 MiB/s wr, 146 op/s 2023-07-14T11:32:10.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:10 smithi161 ceph-mon[95526]: pgmap v308: 129 pgs: 129 active+clean; 690 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.2 MiB/s wr, 146 op/s 2023-07-14T11:32:11.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 911811 8.96 MB/sec execute 413 sec latency 76.542 ms 2023-07-14T11:32:11.844 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:32:12.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 -- 172.21.15.161:0/680222408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 msgr2=0x7faba8072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 --2- 172.21.15.161:0/680222408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 0x7faba8072c20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fab98009990 tx=0x7fab98009ca0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 -- 172.21.15.161:0/680222408 shutdown_connections 2023-07-14T11:32:12.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 --2- 172.21.15.161:0/680222408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faba810c380 0x7faba810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 --2- 172.21.15.161:0/680222408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 0x7faba8072c20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 --2- 172.21.15.161:0/680222408 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 0x7faba810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 -- 172.21.15.161:0/680222408 >> 172.21.15.161:0/680222408 conn(0x7faba80fb5e0 msgr2=0x7faba80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:12.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 -- 172.21.15.161:0/680222408 shutdown_connections 2023-07-14T11:32:12.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.067+0000 7fabac9ff700 1 -- 172.21.15.161:0/680222408 wait complete. 2023-07-14T11:32:12.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.068+0000 7fabac9ff700 1 Processor -- start 2023-07-14T11:32:12.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.068+0000 7fabac9ff700 1 -- start start 2023-07-14T11:32:12.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7fabac9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 0x7faba8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7fabac9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 0x7faba8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7fabac9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faba810c380 0x7faba819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7fabac9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba819ce20 con 0x7faba810c380 2023-07-14T11:32:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7fabac9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba819cf90 con 0x7faba8072850 2023-07-14T11:32:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7fabac9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba819d150 con 0x7faba8073160 2023-07-14T11:32:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 0x7faba8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 0x7faba8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:55592/0 (socket says 172.21.15.161:55592) 2023-07-14T11:32:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 0x7faba8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 -- 172.21.15.161:0/2111179876 learned_addr learned my addr 172.21.15.161:0/2111179876 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba6d9d700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faba810c380 0x7faba819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 -- 172.21.15.161:0/2111179876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 msgr2=0x7faba8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 0x7faba8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 -- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faba810c380 msgr2=0x7faba819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faba810c380 0x7faba819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.069+0000 7faba5d9b700 1 -- 172.21.15.161:0/2111179876 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab980095f0 con 0x7faba8073160 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.070+0000 7faba5d9b700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 0x7faba8198380 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9a0 tx=0x7fab9c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.070+0000 7fab977fe700 1 -- 172.21.15.161:0/2111179876 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c009790 con 0x7faba8073160 2023-07-14T11:32:12.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.070+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba819d430 con 0x7faba8073160 2023-07-14T11:32:12.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.070+0000 7fab977fe700 1 -- 172.21.15.161:0/2111179876 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab9c010430 con 0x7faba8073160 2023-07-14T11:32:12.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.070+0000 7fab977fe700 1 -- 172.21.15.161:0/2111179876 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c018680 con 0x7faba8073160 2023-07-14T11:32:12.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.070+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba81a4c70 con 0x7faba8073160 2023-07-14T11:32:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.073+0000 7fab977fe700 1 -- 172.21.15.161:0/2111179876 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fab9c0105a0 con 0x7faba8073160 2023-07-14T11:32:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.074+0000 7fabc0358b80 1 Processor -- start 2023-07-14T11:32:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.074+0000 7fabc0358b80 1 -- start start 2023-07-14T11:32:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fab977fe700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fab900a12c0 0x7fab900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fab977fe700 1 -- 172.21.15.161:0/2111179876 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fab9c014070 con 0x7faba8073160 2023-07-14T11:32:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e82d0 0x5653d49e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e8be0 0x5653d49f3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49f3ad0 0x5653d49f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fabc0358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d48eecb0 con 0x5653d49f3ad0 2023-07-14T11:32:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fabc0358b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d4941a40 con 0x5653d49e82d0 2023-07-14T11:32:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7fabc0358b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d493aed0 con 0x5653d49e8be0 2023-07-14T11:32:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e82d0 0x5653d49e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba6d9d700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fab900a12c0 0x7fab900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e82d0 0x5653d49e86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51548/0 (socket says 172.21.15.161:51548) 2023-07-14T11:32:12.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 -- 172.21.15.161:0/1973532024 learned_addr learned my addr 172.21.15.161:0/1973532024 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:12.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba6d9d700 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49f3ad0 0x5653d49f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba5d9b700 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e8be0 0x5653d49f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 -- 172.21.15.161:0/1973532024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e8be0 msgr2=0x5653d49f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e8be0 0x5653d49f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 -- 172.21.15.161:0/1973532024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49f3ad0 msgr2=0x5653d49f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49f3ad0 0x5653d49f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba659c700 1 -- 172.21.15.161:0/1973532024 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d4768de0 con 0x5653d49e82d0 2023-07-14T11:32:12.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.075+0000 7faba5d9b700 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e8be0 0x5653d49f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:32:12.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.076+0000 7faba659c700 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e82d0 0x5653d49e86a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab98005ea0 tx=0x7fab98005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed70449795911ffb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:12.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.076+0000 7faba6d9d700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fab900a12c0 0x7fab900a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faba0009ec0 tx=0x7faba0009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:12.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.076+0000 7fab94ff9700 1 -- 172.21.15.161:0/1973532024 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9801d070 con 0x5653d49e82d0 2023-07-14T11:32:12.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.076+0000 7fab94ff9700 1 -- 172.21.15.161:0/1973532024 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab9801e040 con 0x5653d49e82d0 2023-07-14T11:32:12.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.076+0000 7fab94ff9700 1 -- 172.21.15.161:0/1973532024 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98004af0 con 0x5653d49e82d0 2023-07-14T11:32:12.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 -- 172.21.15.161:0/1973532024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e82d0 msgr2=0x5653d49e86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e82d0 0x5653d49e86a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab98005ea0 tx=0x7fab98005ed0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 -- 172.21.15.161:0/1973532024 shutdown_connections 2023-07-14T11:32:12.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49f3ad0 0x5653d49f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e82d0 0x5653d49e86a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1973532024 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e8be0 0x5653d49f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 -- 172.21.15.161:0/1973532024 >> 172.21.15.161:0/1973532024 conn(0x5653d49deda0 msgr2=0x5653d49e46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:12.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 -- 172.21.15.161:0/1973532024 shutdown_connections 2023-07-14T11:32:12.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 -- 172.21.15.161:0/1973532024 wait complete. 2023-07-14T11:32:12.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.077+0000 7fabc0358b80 1 -- start start 2023-07-14T11:32:12.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 1 Processor -- start 2023-07-14T11:32:12.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e82d0 0x5653d49f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e8be0 0x5653d49f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49f3ad0 0x5653d49f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d48daac0 con 0x5653d49e8be0 2023-07-14T11:32:12.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d48da7c0 con 0x5653d49f3ad0 2023-07-14T11:32:12.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d47be920 con 0x5653d49e82d0 2023-07-14T11:32:12.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7fabc0358b80 10 client.0 _ll_register_callbacks cb 0x5653d48d8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:32:12.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7faba659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e82d0 0x5653d49f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7faba659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e82d0 0x5653d49f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:55606/0 (socket says 172.21.15.161:55606) 2023-07-14T11:32:12.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7faba659c700 1 -- 172.21.15.161:0/3318028682 learned_addr learned my addr 172.21.15.161:0/3318028682 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:12.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7faba6d9d700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49f3ad0 0x5653d49f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.078+0000 7faba5d9b700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e8be0 0x5653d49f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7faba659c700 1 -- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49f3ad0 msgr2=0x5653d49f0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7faba659c700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49f3ad0 0x5653d49f0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7faba659c700 1 -- 172.21.15.161:0/3318028682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e8be0 msgr2=0x5653d49f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7faba659c700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e8be0 0x5653d49f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7faba659c700 1 -- 172.21.15.161:0/3318028682 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba810f3a0 con 0x5653d49e82d0 2023-07-14T11:32:12.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7faba5d9b700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e8be0 0x5653d49f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:32:12.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7faba659c700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e82d0 0x5653d49f33f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab9800b2e0 tx=0x7fab980226b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ce92bd82d0cf14 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:12.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7fab8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:32:12.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7fab8dffb700 1 -- 172.21.15.161:0/3318028682 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9801d030 con 0x5653d49e82d0 2023-07-14T11:32:12.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653d48944f0 con 0x5653d49e82d0 2023-07-14T11:32:12.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7fab8dffb700 1 -- 172.21.15.161:0/3318028682 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fab98022da0 con 0x5653d49e82d0 2023-07-14T11:32:12.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7fab8dffb700 1 -- 172.21.15.161:0/3318028682 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab98005bc0 con 0x5653d49e82d0 2023-07-14T11:32:12.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.079+0000 7fabc0358b80 10 client.26926 fetch_fsmap learned FSMap version 27 2023-07-14T11:32:12.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5653d4751c60 con 0x5653d49e82d0 2023-07-14T11:32:12.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fab8dffb700 1 -- 172.21.15.161:0/3318028682 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653d48944f0 con 0x5653d49e82d0 2023-07-14T11:32:12.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fab8dffb700 1 -- 172.21.15.161:0/3318028682 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab9802c490 con 0x5653d49e82d0 2023-07-14T11:32:12.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fabc0358b80 10 client.26926 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:32:12.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fabc0358b80 10 client.26926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:32:12.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fabc0358b80 20 client.26926 populate_metadata read hostname 'smithi161' 2023-07-14T11:32:12.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fabc0358b80 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a02d10 0x5653d4a050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fabc0358b80 4 client.26926 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:32:12.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d4933eb0 con 0x5653d4a02d10 2023-07-14T11:32:12.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.080+0000 7faba5d9b700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a02d10 0x5653d4a050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.081+0000 7faba5d9b700 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a02d10 0x5653d4a050f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a007122be070009 server_cookie=2d314f2d4c316 in_seq=0 out_seq=0 2023-07-14T11:32:12.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.081+0000 7fab8dffb700 10 client.26926 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:12.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.081+0000 7fab8dffb700 1 -- 172.21.15.161:0/3318028682 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5653d4933eb0 con 0x5653d4a02d10 2023-07-14T11:32:12.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.081+0000 7fab8dffb700 10 client.26926 handle_command_reply: tid=0 2023-07-14T11:32:12.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 client.26926 shutdown 2023-07-14T11:32:12.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e82d0 msgr2=0x5653d49f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e82d0 0x5653d49f33f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab9800b2e0 tx=0x7fab980226b0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 shutdown_connections 2023-07-14T11:32:12.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a02d10 0x5653d4a050f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49f3ad0 0x5653d49f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49e82d0 0x5653d49f33f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 --2- 172.21.15.161:0/3318028682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e8be0 0x5653d49f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fab8dffb700 0 client.26926 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:12.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 >> 172.21.15.161:0/3318028682 conn(0x5653d49deda0 msgr2=0x5653d49e4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:12.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 shutdown_connections 2023-07-14T11:32:12.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 1 -- 172.21.15.161:0/3318028682 wait complete. 2023-07-14T11:32:12.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.082+0000 7fabc0358b80 20 client.26926 trim_cache size 0 max 16384 2023-07-14T11:32:12.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 Processor -- start 2023-07-14T11:32:12.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 -- start start 2023-07-14T11:32:12.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d4a050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49eba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d49e4ab0 con 0x5653d49e82d0 2023-07-14T11:32:12.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d49e4c20 con 0x5653d49e8be0 2023-07-14T11:32:12.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.100+0000 7fabc0358b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d49f34d0 con 0x5653d49f3ad0 2023-07-14T11:32:12.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba6d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba6d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:55620/0 (socket says 172.21.15.161:55620) 2023-07-14T11:32:12.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51570/0 (socket says 172.21.15.161:51570) 2023-07-14T11:32:12.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba6d9d700 1 -- 172.21.15.161:0/1629764480 learned_addr learned my addr 172.21.15.161:0/1629764480 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:12.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba659c700 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d4a050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 -- 172.21.15.161:0/1629764480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 msgr2=0x5653d49eba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49eba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 -- 172.21.15.161:0/1629764480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 msgr2=0x5653d4a050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d4a050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 -- 172.21.15.161:0/1629764480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d478f250 con 0x5653d49e8be0 2023-07-14T11:32:12.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7faba5d9b700 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f2f90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0ca630 tx=0x7fab9c005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f744d17b311a8ff9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:12.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.101+0000 7fab957fa700 1 -- 172.21.15.161:0/1629764480 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0b1070 con 0x5653d49e8be0 2023-07-14T11:32:12.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fab957fa700 1 -- 172.21.15.161:0/1629764480 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab9c0a2400 con 0x5653d49e8be0 2023-07-14T11:32:12.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fab957fa700 1 -- 172.21.15.161:0/1629764480 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0994a0 con 0x5653d49e8be0 2023-07-14T11:32:12.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 -- 172.21.15.161:0/1629764480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 msgr2=0x5653d49f2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f2f90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0ca630 tx=0x7fab9c005b40 comp rx=0 tx=0).stop 2023-07-14T11:32:12.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 -- 172.21.15.161:0/1629764480 shutdown_connections 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d4a050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49eba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 --2- 172.21.15.161:0/1629764480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f2f90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 -- 172.21.15.161:0/1629764480 >> 172.21.15.161:0/1629764480 conn(0x5653d49deda0 msgr2=0x5653d49ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 -- 172.21.15.161:0/1629764480 shutdown_connections 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.102+0000 7fabc0358b80 1 -- 172.21.15.161:0/1629764480 wait complete. 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 -- start start 2023-07-14T11:32:12.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 Processor -- start 2023-07-14T11:32:12.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d49fa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d49f7510 con 0x5653d49e82d0 2023-07-14T11:32:12.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d49f7680 con 0x5653d49e8be0 2023-07-14T11:32:12.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.103+0000 7fabc0358b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d4a56b60 con 0x5653d49f3ad0 2023-07-14T11:32:12.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7fabc0358b80 10 client.0 _ll_register_callbacks cb 0x7fab98009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:32:12.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d49fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d49fa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49568/0 (socket says 172.21.15.161:49568) 2023-07-14T11:32:12.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51578/0 (socket says 172.21.15.161:51578) 2023-07-14T11:32:12.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 -- 172.21.15.161:0/503357672 learned_addr learned my addr 172.21.15.161:0/503357672 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:12.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba6d9d700 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 -- 172.21.15.161:0/503357672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 msgr2=0x5653d49f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 -- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 msgr2=0x5653d49f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:12.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 -- 172.21.15.161:0/503357672 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba81a4c70 con 0x5653d49e82d0 2023-07-14T11:32:12.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba5d9b700 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:32:12.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7faba659c700 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d49fa460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab9800bb30 tx=0x7fab9802c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58de9e1d1b43ed23 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:12.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.104+0000 7fab8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:32:12.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fab8dffb700 1 -- 172.21.15.161:0/503357672 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9801d070 con 0x5653d49e82d0 2023-07-14T11:32:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fab8dffb700 1 -- 172.21.15.161:0/503357672 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fab98000e40 con 0x5653d49e82d0 2023-07-14T11:32:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab9801e040 con 0x5653d49e82d0 2023-07-14T11:32:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fab8dffb700 1 -- 172.21.15.161:0/503357672 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab980227d0 con 0x5653d49e82d0 2023-07-14T11:32:12.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 10 client.16884 fetch_fsmap learned FSMap version 27 2023-07-14T11:32:12.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5653d47672b0 con 0x5653d49e82d0 2023-07-14T11:32:12.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fab8dffb700 1 -- 172.21.15.161:0/503357672 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab9801e040 con 0x5653d49e82d0 2023-07-14T11:32:12.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fab8dffb700 1 -- 172.21.15.161:0/503357672 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab980049c0 con 0x5653d49e82d0 2023-07-14T11:32:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 10 client.16884 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:32:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:32:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 20 client.16884 populate_metadata read hostname 'smithi161' 2023-07-14T11:32:12.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a57d20 0x5653d4a5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:12.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 4 client.16884 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:32:12.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.105+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5653d48f6050 con 0x5653d4a57d20 2023-07-14T11:32:12.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.106+0000 7faba5d9b700 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a57d20 0x5653d4a5a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:12.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.106+0000 7faba5d9b700 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a57d20 0x5653d4a5a130 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8355d275e18348b9 server_cookie=ec3c2b23137b2592 in_seq=0 out_seq=0 2023-07-14T11:32:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.106+0000 7fab8dffb700 10 client.16884 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.106+0000 7fab8dffb700 1 -- 172.21.15.161:0/503357672 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5653d48f6050 con 0x5653d4a57d20 2023-07-14T11:32:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.106+0000 7fab8dffb700 10 client.16884 handle_command_reply: tid=0 2023-07-14T11:32:12.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.106+0000 7fabc0358b80 1 client.16884 shutdown 2023-07-14T11:32:12.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 msgr2=0x5653d49fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d49fa460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab9800bb30 tx=0x7fab9802c630 comp rx=0 tx=0).stop 2023-07-14T11:32:12.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 shutdown_connections 2023-07-14T11:32:12.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5653d4a57d20 0x5653d4a5a130 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d49e82d0 0x5653d49fa460 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5653d49f3ad0 0x5653d49f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fab8dffb700 0 client.16884 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:12.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 --2- 172.21.15.161:0/503357672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653d49e8be0 0x5653d49f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 >> 172.21.15.161:0/503357672 conn(0x5653d49deda0 msgr2=0x5653d49e38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 shutdown_connections 2023-07-14T11:32:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 1 -- 172.21.15.161:0/503357672 wait complete. 2023-07-14T11:32:12.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabc0358b80 20 client.16884 trim_cache size 0 max 16384 2023-07-14T11:32:12.132 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:32:12.133 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:32:12.133 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:32:12.133 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:32:12.133 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:32:12.133 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:32:12.133 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:32:12.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:32:12.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:32:12.134 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:32:12.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.107+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fab900a12c0 msgr2=0x7fab900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fab900a12c0 0x7fab900a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faba0009ec0 tx=0x7faba0009920 comp rx=0 tx=0).stop 2023-07-14T11:32:12.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 msgr2=0x7faba8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:12.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 0x7faba8198380 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9a0 tx=0x7fab9c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 shutdown_connections 2023-07-14T11:32:12.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fab900a12c0 0x7fab900a3770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faba810c380 0x7faba819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba8072850 0x7faba8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 --2- 172.21.15.161:0/2111179876 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faba8073160 0x7faba8198380 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:12.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 >> 172.21.15.161:0/2111179876 conn(0x7faba80fb5e0 msgr2=0x7faba810e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:12.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 shutdown_connections 2023-07-14T11:32:12.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:12.108+0000 7fabac9ff700 1 -- 172.21.15.161:0/2111179876 wait complete. 2023-07-14T11:32:12.137 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:32:12.203 INFO:tasks.workunit.client.0.smithi161.stdout: 1 912842 8.95 MB/sec execute 414 sec latency 33.417 ms 2023-07-14T11:32:12.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:12 smithi175 ceph-mon[129081]: pgmap v309: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2023-07-14T11:32:12.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:12 smithi182 ceph-mon[129366]: pgmap v309: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2023-07-14T11:32:12.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:12 smithi161 ceph-mon[95526]: pgmap v309: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2023-07-14T11:32:13.204 INFO:tasks.workunit.client.0.smithi161.stdout: 1 914339 8.94 MB/sec execute 415 sec latency 16.391 ms 2023-07-14T11:32:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:13 smithi175 ceph-mon[129081]: pgmap v310: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-07-14T11:32:14.204 INFO:tasks.workunit.client.0.smithi161.stdout: 1 916213 8.96 MB/sec execute 416 sec latency 18.705 ms 2023-07-14T11:32:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:13 smithi182 ceph-mon[129366]: pgmap v310: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-07-14T11:32:14.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:13 smithi161 ceph-mon[95526]: pgmap v310: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-07-14T11:32:15.204 INFO:tasks.workunit.client.0.smithi161.stdout: 1 916967 8.96 MB/sec execute 417 sec latency 79.991 ms 2023-07-14T11:32:16.204 INFO:tasks.workunit.client.0.smithi161.stdout: 1 918091 8.98 MB/sec execute 418 sec latency 57.822 ms 2023-07-14T11:32:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:16 smithi175 ceph-mon[129081]: pgmap v311: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-07-14T11:32:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:16 smithi182 ceph-mon[129366]: pgmap v311: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-07-14T11:32:16.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:16 smithi161 ceph-mon[95526]: pgmap v311: 129 pgs: 129 active+clean; 697 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-07-14T11:32:17.204 INFO:tasks.workunit.client.0.smithi161.stdout: 1 920060 8.98 MB/sec execute 419 sec latency 105.516 ms 2023-07-14T11:32:18.205 INFO:tasks.workunit.client.0.smithi161.stdout: 1 921070 8.97 MB/sec execute 420 sec latency 21.837 ms 2023-07-14T11:32:18.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:18 smithi175 ceph-mon[129081]: pgmap v312: 129 pgs: 129 active+clean; 701 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 217 op/s 2023-07-14T11:32:18.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:18 smithi182 ceph-mon[129366]: pgmap v312: 129 pgs: 129 active+clean; 701 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 217 op/s 2023-07-14T11:32:18.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:18 smithi161 ceph-mon[95526]: pgmap v312: 129 pgs: 129 active+clean; 701 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 217 op/s 2023-07-14T11:32:19.204 INFO:tasks.workunit.client.0.smithi161.stdout: 1 922431 8.96 MB/sec execute 421 sec latency 122.527 ms 2023-07-14T11:32:19.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:19 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:32:19.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:19 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:32:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:19 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:32:20.204 INFO:tasks.workunit.client.0.smithi161.stdout: 1 923608 8.96 MB/sec execute 422 sec latency 562.263 ms 2023-07-14T11:32:20.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:20 smithi175 ceph-mon[129081]: pgmap v313: 129 pgs: 129 active+clean; 701 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 191 op/s 2023-07-14T11:32:20.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:20 smithi182 ceph-mon[129366]: pgmap v313: 129 pgs: 129 active+clean; 701 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 191 op/s 2023-07-14T11:32:20.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:20 smithi161 ceph-mon[95526]: pgmap v313: 129 pgs: 129 active+clean; 701 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 191 op/s 2023-07-14T11:32:21.205 INFO:tasks.workunit.client.0.smithi161.stdout: 1 923621 8.94 MB/sec execute 423 sec latency 1075.060 ms 2023-07-14T11:32:22.205 INFO:tasks.workunit.client.0.smithi161.stdout: 1 923621 8.92 MB/sec execute 424 sec latency 1472.645 ms 2023-07-14T11:32:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:22 smithi175 ceph-mon[129081]: pgmap v314: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2023-07-14T11:32:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:22 smithi182 ceph-mon[129366]: pgmap v314: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2023-07-14T11:32:22.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:22 smithi161 ceph-mon[95526]: pgmap v314: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2023-07-14T11:32:23.207 INFO:tasks.workunit.client.0.smithi161.stdout: 1 924238 8.91 MB/sec execute 425 sec latency 1726.151 ms 2023-07-14T11:32:24.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:23 smithi175 ceph-mon[129081]: pgmap v315: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2023-07-14T11:32:24.207 INFO:tasks.workunit.client.0.smithi161.stdout: 1 925616 8.90 MB/sec execute 426 sec latency 23.111 ms 2023-07-14T11:32:24.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:23 smithi182 ceph-mon[129366]: pgmap v315: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2023-07-14T11:32:24.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:23 smithi161 ceph-mon[95526]: pgmap v315: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2023-07-14T11:32:25.207 INFO:tasks.workunit.client.0.smithi161.stdout: 1 927566 8.91 MB/sec execute 427 sec latency 67.771 ms 2023-07-14T11:32:26.207 INFO:tasks.workunit.client.0.smithi161.stdout: 1 928460 8.89 MB/sec execute 428 sec latency 64.031 ms 2023-07-14T11:32:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:26 smithi175 ceph-mon[129081]: pgmap v316: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2023-07-14T11:32:26.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:26 smithi182 ceph-mon[129366]: pgmap v316: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2023-07-14T11:32:26.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:26 smithi161 ceph-mon[95526]: pgmap v316: 129 pgs: 129 active+clean; 711 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2023-07-14T11:32:27.210 INFO:tasks.workunit.client.0.smithi161.stdout: 1 930239 8.90 MB/sec execute 429 sec latency 35.739 ms 2023-07-14T11:32:28.210 INFO:tasks.workunit.client.0.smithi161.stdout: 1 931436 8.89 MB/sec execute 430 sec latency 19.613 ms 2023-07-14T11:32:28.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:28 smithi175 ceph-mon[129081]: pgmap v317: 129 pgs: 129 active+clean; 714 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 234 op/s 2023-07-14T11:32:28.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:28 smithi182 ceph-mon[129366]: pgmap v317: 129 pgs: 129 active+clean; 714 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 234 op/s 2023-07-14T11:32:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:28 smithi161 ceph-mon[95526]: pgmap v317: 129 pgs: 129 active+clean; 714 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 234 op/s 2023-07-14T11:32:29.210 INFO:tasks.workunit.client.0.smithi161.stdout: 1 933083 8.88 MB/sec execute 431 sec latency 60.220 ms 2023-07-14T11:32:30.210 INFO:tasks.workunit.client.0.smithi161.stdout: 1 934815 8.89 MB/sec execute 432 sec latency 76.243 ms 2023-07-14T11:32:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:30 smithi175 ceph-mon[129081]: pgmap v318: 129 pgs: 129 active+clean; 714 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 145 op/s 2023-07-14T11:32:30.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:30 smithi182 ceph-mon[129366]: pgmap v318: 129 pgs: 129 active+clean; 714 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 145 op/s 2023-07-14T11:32:30.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:30 smithi161 ceph-mon[95526]: pgmap v318: 129 pgs: 129 active+clean; 714 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 145 op/s 2023-07-14T11:32:31.210 INFO:tasks.workunit.client.0.smithi161.stdout: 1 936091 8.89 MB/sec execute 433 sec latency 61.569 ms 2023-07-14T11:32:32.212 INFO:tasks.workunit.client.0.smithi161.stdout: 1 938237 8.90 MB/sec execute 434 sec latency 41.948 ms 2023-07-14T11:32:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:32 smithi175 ceph-mon[129081]: pgmap v319: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 224 op/s 2023-07-14T11:32:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:32 smithi182 ceph-mon[129366]: pgmap v319: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 224 op/s 2023-07-14T11:32:32.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:32 smithi161 ceph-mon[95526]: pgmap v319: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 224 op/s 2023-07-14T11:32:33.212 INFO:tasks.workunit.client.0.smithi161.stdout: 1 939233 8.88 MB/sec execute 435 sec latency 31.807 ms 2023-07-14T11:32:34.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:33 smithi175 ceph-mon[129081]: pgmap v320: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-14T11:32:34.212 INFO:tasks.workunit.client.0.smithi161.stdout: 1 941542 8.90 MB/sec execute 436 sec latency 17.620 ms 2023-07-14T11:32:34.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:33 smithi182 ceph-mon[129366]: pgmap v320: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-14T11:32:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:33 smithi161 ceph-mon[95526]: pgmap v320: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-14T11:32:35.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:34 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:32:35.212 INFO:tasks.workunit.client.0.smithi161.stdout: 1 942365 8.88 MB/sec execute 437 sec latency 22.732 ms 2023-07-14T11:32:35.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:34 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:32:35.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:34 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:32:36.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:35 smithi175 ceph-mon[129081]: pgmap v321: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-14T11:32:36.212 INFO:tasks.workunit.client.0.smithi161.stdout: 1 943970 8.88 MB/sec execute 438 sec latency 36.732 ms 2023-07-14T11:32:36.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:35 smithi182 ceph-mon[129366]: pgmap v321: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-14T11:32:36.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:35 smithi161 ceph-mon[95526]: pgmap v321: 129 pgs: 129 active+clean; 712 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-14T11:32:37.212 INFO:tasks.workunit.client.0.smithi161.stdout: 1 945415 8.88 MB/sec execute 439 sec latency 58.771 ms 2023-07-14T11:32:38.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: pgmap v322: 129 pgs: 129 active+clean; 715 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2023-07-14T11:32:38.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:32:38.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:32:38.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:37 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.213 INFO:tasks.workunit.client.0.smithi161.stdout: 1 946515 8.87 MB/sec execute 440 sec latency 56.016 ms 2023-07-14T11:32:38.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: pgmap v322: 129 pgs: 129 active+clean; 715 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2023-07-14T11:32:38.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:32:38.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:32:38.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:37 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: pgmap v322: 129 pgs: 129 active+clean; 715 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2023-07-14T11:32:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:38.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:32:38.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:32:38.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:38.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:37 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.213 INFO:tasks.workunit.client.0.smithi161.stdout: 1 948615 8.88 MB/sec execute 441 sec latency 62.386 ms 2023-07-14T11:32:39.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:32:39.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:32:39.250 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:38 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:32:39.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:32:39.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:38 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:39.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:32:39.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:32:39.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:32:39.467 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:38 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:32:40.213 INFO:tasks.workunit.client.0.smithi161.stdout: 1 949778 8.87 MB/sec execute 442 sec latency 18.298 ms 2023-07-14T11:32:40.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:39 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:32:40.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:39 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:32:40.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:39 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:32:40.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:39 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:32:40.248 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:39 smithi161 ceph-mon[95526]: pgmap v323: 129 pgs: 129 active+clean; 715 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2023-07-14T11:32:40.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:39 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:32:40.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:39 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:32:40.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:39 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:32:40.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:39 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:32:40.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:39 smithi175 ceph-mon[129081]: pgmap v323: 129 pgs: 129 active+clean; 715 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2023-07-14T11:32:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:39 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:32:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:39 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:32:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:39 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:32:40.465 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:39 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:32:40.466 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:39 smithi182 ceph-mon[129366]: pgmap v323: 129 pgs: 129 active+clean; 715 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2023-07-14T11:32:41.213 INFO:tasks.workunit.client.0.smithi161.stdout: 1 951665 8.87 MB/sec execute 443 sec latency 93.967 ms 2023-07-14T11:32:42.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:32:42.213 INFO:tasks.workunit.client.0.smithi161.stdout: 1 952706 8.87 MB/sec execute 444 sec latency 41.035 ms 2023-07-14T11:32:42.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 -- 172.21.15.161:0/582689244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae5010c310 msgr2=0x7fae5010e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 --2- 172.21.15.161:0/582689244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae5010c310 0x7fae5010e6f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b0a0 tx=0x7fae4800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 -- 172.21.15.161:0/582689244 shutdown_connections 2023-07-14T11:32:42.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 --2- 172.21.15.161:0/582689244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae5010c310 0x7fae5010e6f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 --2- 172.21.15.161:0/582689244 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae500728b0 0x7fae50072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 --2- 172.21.15.161:0/582689244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae500731c0 0x7fae5010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 -- 172.21.15.161:0/582689244 >> 172.21.15.161:0/582689244 conn(0x7fae500fb5f0 msgr2=0x7fae500fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:42.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 -- 172.21.15.161:0/582689244 shutdown_connections 2023-07-14T11:32:42.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.355+0000 7fae559ff700 1 -- 172.21.15.161:0/582689244 wait complete. 2023-07-14T11:32:42.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 Processor -- start 2023-07-14T11:32:42.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 -- start start 2023-07-14T11:32:42.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae500728b0 0x7fae50197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae500731c0 0x7fae501983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae5010c310 0x7fae5019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5019ce60 con 0x7fae500728b0 2023-07-14T11:32:42.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5019cfd0 con 0x7fae500731c0 2023-07-14T11:32:42.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.356+0000 7fae559ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5019d190 con 0x7fae5010c310 2023-07-14T11:32:42.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae5010c310 0x7fae5019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae500731c0 0x7fae501983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae5010c310 0x7fae5019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57116/0 (socket says 172.21.15.161:57116) 2023-07-14T11:32:42.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae500731c0 0x7fae501983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49926/0 (socket says 172.21.15.161:49926) 2023-07-14T11:32:42.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 -- 172.21.15.161:0/723538949 learned_addr learned my addr 172.21.15.161:0/723538949 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:42.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4effd700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae500728b0 0x7fae50197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 -- 172.21.15.161:0/723538949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae500731c0 msgr2=0x7fae501983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae500731c0 0x7fae501983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 -- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae500728b0 msgr2=0x7fae50197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae500728b0 0x7fae50197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 -- 172.21.15.161:0/723538949 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae48009d00 con 0x7fae5010c310 2023-07-14T11:32:42.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.357+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae5010c310 0x7fae5019c820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b780 tx=0x7fae48007990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:42.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.358+0000 7fae549fd700 1 -- 172.21.15.161:0/723538949 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4800e040 con 0x7fae5010c310 2023-07-14T11:32:42.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.358+0000 7fae549fd700 1 -- 172.21.15.161:0/723538949 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae4801d470 con 0x7fae5010c310 2023-07-14T11:32:42.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.358+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5019d410 con 0x7fae5010c310 2023-07-14T11:32:42.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.358+0000 7fae549fd700 1 -- 172.21.15.161:0/723538949 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae480052a0 con 0x7fae5010c310 2023-07-14T11:32:42.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.358+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae501a4cb0 con 0x7fae5010c310 2023-07-14T11:32:42.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.361+0000 7fae549fd700 1 -- 172.21.15.161:0/723538949 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fae4801d5e0 con 0x7fae5010c310 2023-07-14T11:32:42.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.363+0000 7fae549fd700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fae380a1340 0x7fae380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.363+0000 7fae6923eb80 1 Processor -- start 2023-07-14T11:32:42.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.363+0000 7fae549fd700 1 -- 172.21.15.161:0/723538949 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fae480ce4b0 con 0x7fae5010c310 2023-07-14T11:32:42.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.363+0000 7fae4effd700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fae380a1340 0x7fae380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.363+0000 7fae6923eb80 1 -- start start 2023-07-14T11:32:42.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf42d0 0x55987eaf46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb01eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae6923eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987e9facb0 con 0x55987eaffad0 2023-07-14T11:32:42.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae6923eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987ea4da40 con 0x55987eaf4be0 2023-07-14T11:32:42.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae6923eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987ea46ed0 con 0x55987eaf42d0 2023-07-14T11:32:42.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fae380a1340 0x7fae380a37f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae40009c60 tx=0x7fae40009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:42.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf42d0 0x55987eaf46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf42d0 0x55987eaf46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57126/0 (socket says 172.21.15.161:57126) 2023-07-14T11:32:42.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 -- 172.21.15.161:0/2728190869 learned_addr learned my addr 172.21.15.161:0/2728190869 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:42.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaff590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49938/0 (socket says 172.21.15.161:49938) 2023-07-14T11:32:42.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb01eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 -- 172.21.15.161:0/2728190869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 msgr2=0x55987eaff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 -- 172.21.15.161:0/2728190869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 msgr2=0x55987eb01eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb01eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.364+0000 7fae4effd700 1 -- 172.21.15.161:0/2728190869 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55987e874de0 con 0x55987eaf42d0 2023-07-14T11:32:42.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.365+0000 7fae4effd700 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf42d0 0x55987eaf46a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae4000e4e0 tx=0x7fae40014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2ad3097862a2a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:42.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.365+0000 7fae35ffb700 1 -- 172.21.15.161:0/2728190869 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4000ee80 con 0x55987eaf42d0 2023-07-14T11:32:42.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.365+0000 7fae35ffb700 1 -- 172.21.15.161:0/2728190869 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae40021070 con 0x55987eaf42d0 2023-07-14T11:32:42.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.365+0000 7fae35ffb700 1 -- 172.21.15.161:0/2728190869 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40025790 con 0x55987eaf42d0 2023-07-14T11:32:42.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 -- 172.21.15.161:0/2728190869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf42d0 msgr2=0x55987eaf46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf42d0 0x55987eaf46a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae4000e4e0 tx=0x7fae40014000 comp rx=0 tx=0).stop 2023-07-14T11:32:42.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 -- 172.21.15.161:0/2728190869 shutdown_connections 2023-07-14T11:32:42.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb01eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf42d0 0x55987eaf46a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2728190869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 -- 172.21.15.161:0/2728190869 >> 172.21.15.161:0/2728190869 conn(0x55987eaeada0 msgr2=0x55987eaf06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:42.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 -- 172.21.15.161:0/2728190869 shutdown_connections 2023-07-14T11:32:42.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.366+0000 7fae6923eb80 1 -- 172.21.15.161:0/2728190869 wait complete. 2023-07-14T11:32:42.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.367+0000 7fae6923eb80 1 -- start start 2023-07-14T11:32:42.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.367+0000 7fae6923eb80 1 Processor -- start 2023-07-14T11:32:42.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eaff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eafc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaffad0 0x55987eafce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae6923eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987e9e6ac0 con 0x55987eaf42d0 2023-07-14T11:32:42.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae6923eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987e9e67c0 con 0x55987eaffad0 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae6923eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987e8ca920 con 0x55987eaf4be0 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae6923eb80 10 client.0 _ll_register_callbacks cb 0x55987e9e4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaffad0 0x55987eafce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaffad0 0x55987eafce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49940/0 (socket says 172.21.15.161:49940) 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eaff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 -- 172.21.15.161:0/673186138 learned_addr learned my addr 172.21.15.161:0/673186138 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4e7fc700 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eafc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 -- 172.21.15.161:0/673186138 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 msgr2=0x55987eafc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eafc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 -- 172.21.15.161:0/673186138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 msgr2=0x55987eaff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eaff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.368+0000 7fae4f7fe700 1 -- 172.21.15.161:0/673186138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae40017070 con 0x55987eaffad0 2023-07-14T11:32:42.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaffad0 0x55987eafce30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae4801dc40 tx=0x7fae4801dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dcb62d1105de067 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:42.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae2effd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:32:42.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae2effd700 1 -- 172.21.15.161:0/673186138 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4809f120 con 0x55987eaffad0 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae2effd700 1 -- 172.21.15.161:0/673186138 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fae4809f300 con 0x55987eaffad0 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55987e9a04f0 con 0x55987eaffad0 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae2effd700 1 -- 172.21.15.161:0/673186138 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae480a3690 con 0x55987eaffad0 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae6923eb80 10 client.26945 fetch_fsmap learned FSMap version 28 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55987e85dc60 con 0x55987eaffad0 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.369+0000 7fae2effd700 1 -- 172.21.15.161:0/673186138 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55987e9a04f0 con 0x55987eaffad0 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae2effd700 1 -- 172.21.15.161:0/673186138 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fae480a3200 con 0x55987eaffad0 2023-07-14T11:32:42.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae6923eb80 10 client.26945 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T11:32:42.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae6923eb80 10 client.26945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:32:42.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae6923eb80 20 client.26945 populate_metadata read hostname 'smithi161' 2023-07-14T11:32:42.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae6923eb80 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae6923eb80 4 client.26945 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:32:42.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55987ea3feb0 con 0x55987eb0f1d0 2023-07-14T11:32:42.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.370+0000 7fae4effd700 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.371+0000 7fae2effd700 10 client.26945 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.371+0000 7fae4effd700 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb115e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd776e8c821b3d5c server_cookie=287368ba3ff4208e in_seq=0 out_seq=0 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.371+0000 7fae2effd700 1 -- 172.21.15.161:0/673186138 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55987ea3feb0 con 0x55987eb0f1d0 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.371+0000 7fae2effd700 10 client.26945 handle_command_reply: tid=0 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.371+0000 7fae6923eb80 1 client.26945 shutdown 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.371+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaffad0 msgr2=0x55987eafce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.371+0000 7fae6923eb80 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaffad0 0x55987eafce30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae4801dc40 tx=0x7fae4801dc90 comp rx=0 tx=0).stop 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 shutdown_connections 2023-07-14T11:32:42.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb115e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eaff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae2effd700 0 client.26945 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:42.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaffad0 0x55987eafce30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 --2- 172.21.15.161:0/673186138 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eafc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 >> 172.21.15.161:0/673186138 conn(0x55987eaeada0 msgr2=0x55987eaf0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:42.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 shutdown_connections 2023-07-14T11:32:42.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 1 -- 172.21.15.161:0/673186138 wait complete. 2023-07-14T11:32:42.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.372+0000 7fae6923eb80 20 client.26945 trim_cache size 0 max 16384 2023-07-14T11:32:42.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 Processor -- start 2023-07-14T11:32:42.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 -- start start 2023-07-14T11:32:42.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eb064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaf6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaffad0 0x55987eaf9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987eafdd90 con 0x55987eaf42d0 2023-07-14T11:32:42.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987eafd1f0 con 0x55987eaf4be0 2023-07-14T11:32:42.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae6923eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987eae8020 con 0x55987eaffad0 2023-07-14T11:32:42.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaffad0 0x55987eaf9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaf6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaffad0 0x55987eaf9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57158/0 (socket says 172.21.15.161:57158) 2023-07-14T11:32:42.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae4f7fe700 1 -- 172.21.15.161:0/2506302766 learned_addr learned my addr 172.21.15.161:0/2506302766 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:42.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.391+0000 7fae4effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eb064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae4f7fe700 1 -- 172.21.15.161:0/2506302766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 msgr2=0x55987eaf6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/2506302766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaf6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae4f7fe700 1 -- 172.21.15.161:0/2506302766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 msgr2=0x55987eb064f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/2506302766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eb064f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae4f7fe700 1 -- 172.21.15.161:0/2506302766 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55987e89b250 con 0x55987eaffad0 2023-07-14T11:32:42.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/2506302766 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaffad0 0x55987eaf9a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae4809afd0 tx=0x7fae4809b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671540842c29e423 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:42.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae367fc700 1 -- 172.21.15.161:0/2506302766 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4809f120 con 0x55987eaffad0 2023-07-14T11:32:42.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae367fc700 1 -- 172.21.15.161:0/2506302766 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae4809ac30 con 0x55987eaffad0 2023-07-14T11:32:42.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.392+0000 7fae367fc700 1 -- 172.21.15.161:0/2506302766 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4809b530 con 0x55987eaffad0 2023-07-14T11:32:42.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 -- 172.21.15.161:0/2506302766 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaffad0 msgr2=0x55987eaf9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2506302766 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaffad0 0x55987eaf9a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae4809afd0 tx=0x7fae4809b450 comp rx=0 tx=0).stop 2023-07-14T11:32:42.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 -- 172.21.15.161:0/2506302766 shutdown_connections 2023-07-14T11:32:42.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2506302766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaf42d0 0x55987eb064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2506302766 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaffad0 0x55987eaf9a20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 --2- 172.21.15.161:0/2506302766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf4be0 0x55987eaf6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 -- 172.21.15.161:0/2506302766 >> 172.21.15.161:0/2506302766 conn(0x55987eaeada0 msgr2=0x55987eaf63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:42.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 -- 172.21.15.161:0/2506302766 shutdown_connections 2023-07-14T11:32:42.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.393+0000 7fae6923eb80 1 -- 172.21.15.161:0/2506302766 wait complete. 2023-07-14T11:32:42.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 -- start start 2023-07-14T11:32:42.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 Processor -- start 2023-07-14T11:32:42.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf42d0 0x55987eb57210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eb577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb5b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987eb5b950 con 0x55987eaffad0 2023-07-14T11:32:42.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987eb5bac0 con 0x55987eaf42d0 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987eb5bc80 con 0x55987eaf4be0 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae6923eb80 10 client.0 _ll_register_callbacks cb 0x55987e9e4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eb577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.394+0000 7fae4effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf42d0 0x55987eb57210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf42d0 0x55987eb57210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49960/0 (socket says 172.21.15.161:49960) 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eb577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:57168/0 (socket says 172.21.15.161:57168) 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 -- 172.21.15.161:0/1524873221 learned_addr learned my addr 172.21.15.161:0/1524873221 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:32:42.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4f7fe700 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb5b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 -- 172.21.15.161:0/1524873221 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 msgr2=0x55987eb577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eb577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 -- 172.21.15.161:0/1524873221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 msgr2=0x55987eb5b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:32:42.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb5b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 -- 172.21.15.161:0/1524873221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55987e85dc60 con 0x55987eaf42d0 2023-07-14T11:32:42.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae4effd700 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf42d0 0x55987eb57210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fae4001df80 tx=0x7fae40005cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ae80a8a44b1994 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:32:42.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae2effd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae2effd700 1 -- 172.21.15.161:0/1524873221 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40039070 con 0x55987eaf42d0 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae2effd700 1 -- 172.21.15.161:0/1524873221 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fae40009990 con 0x55987eaf42d0 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.395+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae40021070 con 0x55987eaf42d0 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae2effd700 1 -- 172.21.15.161:0/1524873221 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae40035b50 con 0x55987eaf42d0 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 10 client.26951 fetch_fsmap learned FSMap version 28 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55987e8732b0 con 0x55987eaf42d0 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae2effd700 1 -- 172.21.15.161:0/1524873221 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae40021070 con 0x55987eaf42d0 2023-07-14T11:32:42.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae2effd700 1 -- 172.21.15.161:0/1524873221 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fae4003c030 con 0x55987eaf42d0 2023-07-14T11:32:42.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 10 client.26951 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T11:32:42.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 10 client.26951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:32:42.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 20 client.26951 populate_metadata read hostname 'smithi161' 2023-07-14T11:32:42.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:32:42.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 4 client.26951 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:32:42.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.396+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55987ea02050 con 0x55987eb0f1d0 2023-07-14T11:32:42.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.397+0000 7fae4e7fc700 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:32:42.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.397+0000 7fae4e7fc700 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb664f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2949d6aab86cb3f server_cookie=17d0e1c4c5e96c07 in_seq=0 out_seq=0 2023-07-14T11:32:42.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.397+0000 7fae2effd700 10 client.26951 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:42.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.397+0000 7fae2effd700 1 -- 172.21.15.161:0/1524873221 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55987ea02050 con 0x55987eb0f1d0 2023-07-14T11:32:42.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.397+0000 7fae2effd700 10 client.26951 handle_command_reply: tid=0 2023-07-14T11:32:42.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.397+0000 7fae6923eb80 1 client.26951 shutdown 2023-07-14T11:32:42.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf42d0 msgr2=0x55987eb57210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf42d0 0x55987eb57210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fae4001df80 tx=0x7fae40005cd0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 shutdown_connections 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55987eb0f1d0 0x55987eb664f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987eaffad0 0x55987eb5b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55987eaf42d0 0x55987eb57210 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 --2- 172.21.15.161:0/1524873221 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55987eaf4be0 0x55987eb577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 >> 172.21.15.161:0/1524873221 conn(0x55987eaeada0 msgr2=0x55987eaf6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae2effd700 0 client.26951 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:32:42.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 shutdown_connections 2023-07-14T11:32:42.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 1 -- 172.21.15.161:0/1524873221 wait complete. 2023-07-14T11:32:42.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.398+0000 7fae6923eb80 20 client.26951 trim_cache size 0 max 16384 2023-07-14T11:32:42.411 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:32:42.411 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:32:42.411 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:32:42.412 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:32:42.412 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:32:42.412 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:32:42.412 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:32:42.412 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:32:42.412 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:32:42.412 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:32:42.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fae380a1340 msgr2=0x7fae380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fae380a1340 0x7fae380a37f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae40009c60 tx=0x7fae40009260 comp rx=0 tx=0).stop 2023-07-14T11:32:42.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae5010c310 msgr2=0x7fae5019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:32:42.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae5010c310 0x7fae5019c820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b780 tx=0x7fae48007990 comp rx=0 tx=0).stop 2023-07-14T11:32:42.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 shutdown_connections 2023-07-14T11:32:42.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fae380a1340 0x7fae380a37f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae500728b0 0x7fae50197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fae5010c310 0x7fae5019c820 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 --2- 172.21.15.161:0/723538949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae500731c0 0x7fae501983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:32:42.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 >> 172.21.15.161:0/723538949 conn(0x7fae500fb5f0 msgr2=0x7fae50103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:32:42.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.399+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 shutdown_connections 2023-07-14T11:32:42.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:32:42.400+0000 7fae559ff700 1 -- 172.21.15.161:0/723538949 wait complete. 2023-07-14T11:32:42.425 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:32:42.496 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:42 smithi161 ceph-mon[95526]: pgmap v324: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-07-14T11:32:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:42 smithi175 ceph-mon[129081]: pgmap v324: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-07-14T11:32:42.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:42 smithi182 ceph-mon[129366]: pgmap v324: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-07-14T11:32:43.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 954534 8.86 MB/sec execute 445 sec latency 19.755 ms 2023-07-14T11:32:44.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:43 smithi175 ceph-mon[129081]: pgmap v325: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-07-14T11:32:44.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 956189 8.87 MB/sec execute 446 sec latency 51.431 ms 2023-07-14T11:32:44.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:43 smithi182 ceph-mon[129366]: pgmap v325: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-07-14T11:32:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:43 smithi161 ceph-mon[95526]: pgmap v325: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-07-14T11:32:45.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 958114 8.87 MB/sec execute 447 sec latency 19.381 ms 2023-07-14T11:32:46.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 959685 8.88 MB/sec execute 448 sec latency 133.917 ms 2023-07-14T11:32:46.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:46 smithi175 ceph-mon[129081]: pgmap v326: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-07-14T11:32:46.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:46 smithi182 ceph-mon[129366]: pgmap v326: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-07-14T11:32:46.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:46 smithi161 ceph-mon[95526]: pgmap v326: 129 pgs: 129 active+clean; 724 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-07-14T11:32:47.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 960804 8.87 MB/sec execute 449 sec latency 102.881 ms 2023-07-14T11:32:48.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 963077 8.88 MB/sec execute 450 sec latency 33.711 ms 2023-07-14T11:32:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:48 smithi175 ceph-mon[129081]: pgmap v327: 129 pgs: 129 active+clean; 730 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2023-07-14T11:32:48.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:48 smithi182 ceph-mon[129366]: pgmap v327: 129 pgs: 129 active+clean; 730 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2023-07-14T11:32:48.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:48 smithi161 ceph-mon[95526]: pgmap v327: 129 pgs: 129 active+clean; 730 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2023-07-14T11:32:49.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 964354 8.87 MB/sec execute 451 sec latency 14.788 ms 2023-07-14T11:32:50.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 966620 8.88 MB/sec execute 452 sec latency 25.663 ms 2023-07-14T11:32:50.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:50 smithi175 ceph-mon[129081]: pgmap v328: 129 pgs: 129 active+clean; 730 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 178 op/s 2023-07-14T11:32:50.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:50 smithi182 ceph-mon[129366]: pgmap v328: 129 pgs: 129 active+clean; 730 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 178 op/s 2023-07-14T11:32:50.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:50 smithi161 ceph-mon[95526]: pgmap v328: 129 pgs: 129 active+clean; 730 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 178 op/s 2023-07-14T11:32:51.215 INFO:tasks.workunit.client.0.smithi161.stdout: 1 967897 8.87 MB/sec execute 453 sec latency 16.683 ms 2023-07-14T11:32:52.214 INFO:tasks.workunit.client.0.smithi161.stdout: 1 970062 8.88 MB/sec execute 454 sec latency 46.194 ms 2023-07-14T11:32:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:52 smithi175 ceph-mon[129081]: pgmap v329: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 271 op/s 2023-07-14T11:32:52.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:52 smithi182 ceph-mon[129366]: pgmap v329: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 271 op/s 2023-07-14T11:32:52.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:52 smithi161 ceph-mon[95526]: pgmap v329: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 271 op/s 2023-07-14T11:32:53.215 INFO:tasks.workunit.client.0.smithi161.stdout: 1 971146 8.87 MB/sec execute 455 sec latency 14.823 ms 2023-07-14T11:32:54.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:53 smithi175 ceph-mon[129081]: pgmap v330: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 188 op/s 2023-07-14T11:32:54.215 INFO:tasks.workunit.client.0.smithi161.stdout: 1 973610 8.88 MB/sec execute 456 sec latency 17.601 ms 2023-07-14T11:32:54.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:53 smithi182 ceph-mon[129366]: pgmap v330: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 188 op/s 2023-07-14T11:32:54.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:53 smithi161 ceph-mon[95526]: pgmap v330: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 188 op/s 2023-07-14T11:32:55.215 INFO:tasks.workunit.client.0.smithi161.stdout: 1 974878 8.88 MB/sec execute 457 sec latency 15.692 ms 2023-07-14T11:32:56.215 INFO:tasks.workunit.client.0.smithi161.stdout: 1 977228 8.89 MB/sec execute 458 sec latency 18.697 ms 2023-07-14T11:32:56.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:56 smithi175 ceph-mon[129081]: pgmap v331: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 188 op/s 2023-07-14T11:32:56.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:56 smithi182 ceph-mon[129366]: pgmap v331: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 188 op/s 2023-07-14T11:32:56.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:56 smithi161 ceph-mon[95526]: pgmap v331: 129 pgs: 129 active+clean; 755 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 188 op/s 2023-07-14T11:32:57.215 INFO:tasks.workunit.client.0.smithi161.stdout: 1 978195 8.88 MB/sec execute 459 sec latency 27.587 ms 2023-07-14T11:32:58.216 INFO:tasks.workunit.client.0.smithi161.stdout: 1 980392 8.88 MB/sec execute 460 sec latency 31.344 ms 2023-07-14T11:32:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:32:58 smithi175 ceph-mon[129081]: pgmap v332: 129 pgs: 129 active+clean; 760 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 277 op/s 2023-07-14T11:32:58.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:32:58 smithi182 ceph-mon[129366]: pgmap v332: 129 pgs: 129 active+clean; 760 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 277 op/s 2023-07-14T11:32:58.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:32:58 smithi161 ceph-mon[95526]: pgmap v332: 129 pgs: 129 active+clean; 760 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 277 op/s 2023-07-14T11:32:59.216 INFO:tasks.workunit.client.0.smithi161.stdout: 1 981536 8.88 MB/sec execute 461 sec latency 25.698 ms 2023-07-14T11:33:00.216 INFO:tasks.workunit.client.0.smithi161.stdout: 1 983928 8.89 MB/sec execute 462 sec latency 15.593 ms 2023-07-14T11:33:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:00 smithi175 ceph-mon[129081]: pgmap v333: 129 pgs: 129 active+clean; 760 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 181 op/s 2023-07-14T11:33:00.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:00 smithi182 ceph-mon[129366]: pgmap v333: 129 pgs: 129 active+clean; 760 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 181 op/s 2023-07-14T11:33:00.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:00 smithi161 ceph-mon[95526]: pgmap v333: 129 pgs: 129 active+clean; 760 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 181 op/s 2023-07-14T11:33:01.216 INFO:tasks.workunit.client.0.smithi161.stdout: 1 985021 8.88 MB/sec execute 463 sec latency 16.955 ms 2023-07-14T11:33:02.216 INFO:tasks.workunit.client.0.smithi161.stdout: 1 985142 8.86 MB/sec execute 464 sec latency 134.772 ms 2023-07-14T11:33:02.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:02 smithi175 ceph-mon[129081]: pgmap v334: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 284 op/s 2023-07-14T11:33:02.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:02 smithi175 ceph-mon[129081]: osdmap e102: 12 total, 12 up, 12 in 2023-07-14T11:33:02.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:02 smithi182 ceph-mon[129366]: pgmap v334: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 284 op/s 2023-07-14T11:33:02.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:02 smithi182 ceph-mon[129366]: osdmap e102: 12 total, 12 up, 12 in 2023-07-14T11:33:02.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:02 smithi161 ceph-mon[95526]: pgmap v334: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 284 op/s 2023-07-14T11:33:02.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:02 smithi161 ceph-mon[95526]: osdmap e102: 12 total, 12 up, 12 in 2023-07-14T11:33:03.216 INFO:tasks.workunit.client.0.smithi161.stdout: 1 986249 8.85 MB/sec execute 465 sec latency 82.649 ms 2023-07-14T11:33:04.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:03 smithi175 ceph-mon[129081]: pgmap v336: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2023-07-14T11:33:04.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:03 smithi182 ceph-mon[129366]: pgmap v336: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2023-07-14T11:33:04.217 INFO:tasks.workunit.client.0.smithi161.stdout: 1 988008 8.86 MB/sec execute 466 sec latency 42.735 ms 2023-07-14T11:33:04.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:03 smithi161 ceph-mon[95526]: pgmap v336: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2023-07-14T11:33:05.216 INFO:tasks.workunit.client.0.smithi161.stdout: 1 989343 8.85 MB/sec execute 467 sec latency 18.431 ms 2023-07-14T11:33:06.217 INFO:tasks.workunit.client.0.smithi161.stdout: 1 991604 8.86 MB/sec execute 468 sec latency 15.234 ms 2023-07-14T11:33:06.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:06 smithi175 ceph-mon[129081]: pgmap v337: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2023-07-14T11:33:06.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:06 smithi182 ceph-mon[129366]: pgmap v337: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2023-07-14T11:33:06.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:06 smithi161 ceph-mon[95526]: pgmap v337: 129 pgs: 129 active+clean; 769 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2023-07-14T11:33:07.217 INFO:tasks.workunit.client.0.smithi161.stdout: 1 992737 8.86 MB/sec execute 469 sec latency 55.549 ms 2023-07-14T11:33:08.217 INFO:tasks.workunit.client.0.smithi161.stdout: 1 994908 8.86 MB/sec execute 470 sec latency 36.106 ms 2023-07-14T11:33:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:08 smithi175 ceph-mon[129081]: pgmap v338: 129 pgs: 129 active+clean; 751 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2023-07-14T11:33:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:08 smithi175 ceph-mon[129081]: osdmap e103: 12 total, 12 up, 12 in 2023-07-14T11:33:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:08 smithi182 ceph-mon[129366]: pgmap v338: 129 pgs: 129 active+clean; 751 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2023-07-14T11:33:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:08 smithi182 ceph-mon[129366]: osdmap e103: 12 total, 12 up, 12 in 2023-07-14T11:33:08.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:08 smithi161 ceph-mon[95526]: pgmap v338: 129 pgs: 129 active+clean; 751 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2023-07-14T11:33:08.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:08 smithi161 ceph-mon[95526]: osdmap e103: 12 total, 12 up, 12 in 2023-07-14T11:33:09.217 INFO:tasks.workunit.client.0.smithi161.stdout: 1 995995 8.85 MB/sec execute 471 sec latency 31.481 ms 2023-07-14T11:33:10.217 INFO:tasks.workunit.client.0.smithi161.stdout: 1 998166 8.86 MB/sec execute 472 sec latency 79.078 ms 2023-07-14T11:33:10.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:10 smithi175 ceph-mon[129081]: pgmap v340: 129 pgs: 129 active+clean; 751 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 149 op/s 2023-07-14T11:33:10.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:10 smithi182 ceph-mon[129366]: pgmap v340: 129 pgs: 129 active+clean; 751 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 149 op/s 2023-07-14T11:33:10.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:10 smithi161 ceph-mon[95526]: pgmap v340: 129 pgs: 129 active+clean; 751 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 149 op/s 2023-07-14T11:33:11.217 INFO:tasks.workunit.client.0.smithi161.stdout: 1 999313 8.85 MB/sec execute 473 sec latency 20.423 ms 2023-07-14T11:33:12.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1001295 8.86 MB/sec execute 474 sec latency 19.161 ms 2023-07-14T11:33:12.427 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:33:12.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.630+0000 7f55b25ff700 1 -- 172.21.15.161:0/3477208986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 msgr2=0x7f55ac10e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.630+0000 7f55b25ff700 1 --2- 172.21.15.161:0/3477208986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 0x7f55ac10e760 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f55a400b0a0 tx=0x7f55a400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.630+0000 7f55b25ff700 1 -- 172.21.15.161:0/3477208986 shutdown_connections 2023-07-14T11:33:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.630+0000 7f55b25ff700 1 --2- 172.21.15.161:0/3477208986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55ac072850 0x7f55ac072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.630+0000 7f55b25ff700 1 --2- 172.21.15.161:0/3477208986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 0x7f55ac10e760 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.630+0000 7f55b25ff700 1 --2- 172.21.15.161:0/3477208986 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 0x7f55ac10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.630+0000 7f55b25ff700 1 -- 172.21.15.161:0/3477208986 >> 172.21.15.161:0/3477208986 conn(0x7f55ac0fb5e0 msgr2=0x7f55ac0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.631+0000 7f55b25ff700 1 -- 172.21.15.161:0/3477208986 shutdown_connections 2023-07-14T11:33:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.631+0000 7f55b25ff700 1 -- 172.21.15.161:0/3477208986 wait complete. 2023-07-14T11:33:12.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.631+0000 7f55b25ff700 1 Processor -- start 2023-07-14T11:33:12.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b25ff700 1 -- start start 2023-07-14T11:33:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b25ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55ac072850 0x7f55ac197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b25ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 0x7f55ac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b25ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 0x7f55ac19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b25ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac19cef0 con 0x7f55ac072850 2023-07-14T11:33:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b25ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac19d060 con 0x7f55ac10c380 2023-07-14T11:33:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b25ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac19d220 con 0x7f55ac073160 2023-07-14T11:33:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55abfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55ac072850 0x7f55ac197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55ab7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 0x7f55ac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55b0b9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 0x7f55ac19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55ab7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 0x7f55ac198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:44408/0 (socket says 172.21.15.161:44408) 2023-07-14T11:33:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.632+0000 7f55ab7fe700 1 -- 172.21.15.161:0/2493555373 learned_addr learned my addr 172.21.15.161:0/2493555373 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b0b9c700 1 -- 172.21.15.161:0/2493555373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 msgr2=0x7f55ac198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 0x7f55ac198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b0b9c700 1 -- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55ac072850 msgr2=0x7f55ac197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55ac072850 0x7f55ac197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b0b9c700 1 -- 172.21.15.161:0/2493555373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4009d00 con 0x7f55ac10c380 2023-07-14T11:33:12.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 0x7f55ac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:33:12.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 0x7f55ac19c8b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55a400b070 tx=0x7f55a4007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:12.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55a97fa700 1 -- 172.21.15.161:0/2493555373 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400e040 con 0x7f55ac10c380 2023-07-14T11:33:12.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55a97fa700 1 -- 172.21.15.161:0/2493555373 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a401d470 con 0x7f55ac10c380 2023-07-14T11:33:12.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55a97fa700 1 -- 172.21.15.161:0/2493555373 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a40052a0 con 0x7f55ac10c380 2023-07-14T11:33:12.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac19d4a0 con 0x7f55ac10c380 2023-07-14T11:33:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.633+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac1a4d40 con 0x7f55ac10c380 2023-07-14T11:33:12.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.636+0000 7f55a97fa700 1 -- 172.21.15.161:0/2493555373 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f55a401d5e0 con 0x7f55ac10c380 2023-07-14T11:33:12.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.638+0000 7f55c5f2cb80 1 Processor -- start 2023-07-14T11:33:12.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.638+0000 7f55c5f2cb80 1 -- start start 2023-07-14T11:33:12.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55a97fa700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f55940a12c0 0x7f55940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad896f2d0 0x55aad896f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad897a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad897aad0 0x55aad897ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55a97fa700 1 -- 172.21.15.161:0/2493555373 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f55a40ce4f0 con 0x7f55ac10c380 2023-07-14T11:33:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad8875cb0 con 0x55aad896fbe0 2023-07-14T11:33:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad88c8a40 con 0x55aad897aad0 2023-07-14T11:33:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad88c1ed0 con 0x55aad896f2d0 2023-07-14T11:33:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f55940a12c0 0x7f55940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55abfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad896f2d0 0x55aad896f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55abfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad896f2d0 0x55aad896f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:44424/0 (socket says 172.21.15.161:44424) 2023-07-14T11:33:12.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55abfff700 1 -- 172.21.15.161:0/3185496262 learned_addr learned my addr 172.21.15.161:0/3185496262 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:12.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad897a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.639+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad897aad0 0x55aad897ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55abfff700 1 -- 172.21.15.161:0/3185496262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad897aad0 msgr2=0x55aad897ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:12.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55abfff700 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad897aad0 0x55aad897ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55abfff700 1 -- 172.21.15.161:0/3185496262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 msgr2=0x55aad897a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:12.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55abfff700 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad897a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55abfff700 1 -- 172.21.15.161:0/3185496262 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aad86efde0 con 0x55aad896f2d0 2023-07-14T11:33:12.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f55940a12c0 0x7f55940a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f55a00096b0 tx=0x7f55a00134e0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:12.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55abfff700 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad896f2d0 0x55aad896f6a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f559c009d80 tx=0x7f559c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1aabb36f92aaa74 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:12.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55927fc700 1 -- 172.21.15.161:0/3185496262 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c004230 con 0x55aad896f2d0 2023-07-14T11:33:12.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55927fc700 1 -- 172.21.15.161:0/3185496262 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f559c004390 con 0x55aad896f2d0 2023-07-14T11:33:12.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.640+0000 7f55927fc700 1 -- 172.21.15.161:0/3185496262 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c019860 con 0x55aad896f2d0 2023-07-14T11:33:12.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/3185496262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad896f2d0 msgr2=0x55aad896f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad896f2d0 0x55aad896f6a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f559c009d80 tx=0x7f559c011720 comp rx=0 tx=0).stop 2023-07-14T11:33:12.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/3185496262 shutdown_connections 2023-07-14T11:33:12.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad897aad0 0x55aad897ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad896f2d0 0x55aad896f6a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/3185496262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad897a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/3185496262 >> 172.21.15.161:0/3185496262 conn(0x55aad8965da0 msgr2=0x55aad896b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:12.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/3185496262 shutdown_connections 2023-07-14T11:33:12.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.641+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/3185496262 wait complete. 2023-07-14T11:33:12.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 -- start start 2023-07-14T11:33:12.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 Processor -- start 2023-07-14T11:33:12.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896f2d0 0x55aad897a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896fbe0 0x55aad8977720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8977d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad8861ac0 con 0x55aad896f2d0 2023-07-14T11:33:12.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad88617c0 con 0x55aad896fbe0 2023-07-14T11:33:12.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad8745920 con 0x55aad897aad0 2023-07-14T11:33:12.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.642+0000 7f55c5f2cb80 10 client.0 _ll_register_callbacks cb 0x55aad885f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:33:12.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896fbe0 0x55aad8977720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896fbe0 0x55aad8977720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47062/0 (socket says 172.21.15.161:47062) 2023-07-14T11:33:12.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 -- 172.21.15.161:0/4055250873 learned_addr learned my addr 172.21.15.161:0/4055250873 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:12.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55abfff700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896f2d0 0x55aad897a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8977d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 -- 172.21.15.161:0/4055250873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 msgr2=0x55aad8977d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:12.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8977d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 -- 172.21.15.161:0/4055250873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896f2d0 msgr2=0x55aad897a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:12.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896f2d0 0x55aad897a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 -- 172.21.15.161:0/4055250873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c0099e0 con 0x55aad896fbe0 2023-07-14T11:33:12.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8977d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:33:12.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896fbe0 0x55aad8977720 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f55a0011fc0 tx=0x7f55a001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84878b042b538ea4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:12.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.643+0000 7f558b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:33:12.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f558b7fe700 1 -- 172.21.15.161:0/4055250873 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a001de90 con 0x55aad896fbe0 2023-07-14T11:33:12.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f558b7fe700 1 -- 172.21.15.161:0/4055250873 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f55a0004020 con 0x55aad896fbe0 2023-07-14T11:33:12.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aad881b4f0 con 0x55aad896fbe0 2023-07-14T11:33:12.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f558b7fe700 1 -- 172.21.15.161:0/4055250873 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a00041e0 con 0x55aad896fbe0 2023-07-14T11:33:12.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f55c5f2cb80 10 client.26963 fetch_fsmap learned FSMap version 29 2023-07-14T11:33:12.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55aad86d8c60 con 0x55aad896fbe0 2023-07-14T11:33:12.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f558b7fe700 1 -- 172.21.15.161:0/4055250873 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aad881b4f0 con 0x55aad896fbe0 2023-07-14T11:33:12.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.644+0000 7f558b7fe700 1 -- 172.21.15.161:0/4055250873 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f55a0020460 con 0x55aad896fbe0 2023-07-14T11:33:12.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55c5f2cb80 10 client.26963 fetch_fsmap finished waiting for FSMap version 29 2023-07-14T11:33:12.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55c5f2cb80 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:33:12.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55c5f2cb80 20 client.26963 populate_metadata read hostname 'smithi161' 2023-07-14T11:33:12.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad898a410 0x55aad898c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55c5f2cb80 4 client.26963 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:33:12.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aad88baeb0 con 0x55aad898a410 2023-07-14T11:33:12.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55abfff700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad898a410 0x55aad898c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f558b7fe700 10 client.26963 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:12.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.645+0000 7f55abfff700 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad898a410 0x55aad898c820 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b0a4e3a973c197 server_cookie=4ded6887031ea28e in_seq=0 out_seq=0 2023-07-14T11:33:12.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.646+0000 7f558b7fe700 1 -- 172.21.15.161:0/4055250873 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aad88baeb0 con 0x55aad898a410 2023-07-14T11:33:12.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.646+0000 7f558b7fe700 10 client.26963 handle_command_reply: tid=0 2023-07-14T11:33:12.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.646+0000 7f55c5f2cb80 1 client.26963 shutdown 2023-07-14T11:33:12.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896fbe0 msgr2=0x55aad8977720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896fbe0 0x55aad8977720 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f55a0011fc0 tx=0x7f55a001b000 comp rx=0 tx=0).stop 2023-07-14T11:33:12.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 shutdown_connections 2023-07-14T11:33:12.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad898a410 0x55aad898c820 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896f2d0 0x55aad897a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f558b7fe700 0 client.26963 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:12.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8977d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/4055250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896fbe0 0x55aad8977720 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 >> 172.21.15.161:0/4055250873 conn(0x55aad8965da0 msgr2=0x55aad896b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:12.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 shutdown_connections 2023-07-14T11:33:12.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/4055250873 wait complete. 2023-07-14T11:33:12.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.647+0000 7f55c5f2cb80 20 client.26963 trim_cache size 0 max 16384 2023-07-14T11:33:12.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:12 smithi175 ceph-mon[129081]: pgmap v341: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-07-14T11:33:12.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:12 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:33:12.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 Processor -- start 2023-07-14T11:33:12.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 -- start start 2023-07-14T11:33:12.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad89814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad8985c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8972af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad89711a0 con 0x55aad896fbe0 2023-07-14T11:33:12.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad897a770 con 0x55aad896f2d0 2023-07-14T11:33:12.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.682+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad8975cc0 con 0x55aad897aad0 2023-07-14T11:33:12.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.683+0000 7f55ab7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad8985c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.683+0000 7f55ab7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad8985c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42412/0 (socket says 172.21.15.161:42412) 2023-07-14T11:33:12.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.683+0000 7f55ab7fe700 1 -- 172.21.15.161:0/302311886 learned_addr learned my addr 172.21.15.161:0/302311886 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:12.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8972af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55abfff700 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad89814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55ab7fe700 1 -- 172.21.15.161:0/302311886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 msgr2=0x55aad8972af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:12.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8972af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55ab7fe700 1 -- 172.21.15.161:0/302311886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 msgr2=0x55aad89814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:12.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad89814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55ab7fe700 1 -- 172.21.15.161:0/302311886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aad8716250 con 0x55aad896fbe0 2023-07-14T11:33:12.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55abfff700 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad89814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:33:12.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.684+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8972af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:33:12.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.685+0000 7f55ab7fe700 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad8985c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f55a00207c0 tx=0x7f55a00136f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0ceb2a5616918a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:12.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.686+0000 7f5592ffd700 1 -- 172.21.15.161:0/302311886 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a001de90 con 0x55aad896fbe0 2023-07-14T11:33:12.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.686+0000 7f5592ffd700 1 -- 172.21.15.161:0/302311886 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a001da10 con 0x55aad896fbe0 2023-07-14T11:33:12.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.686+0000 7f5592ffd700 1 -- 172.21.15.161:0/302311886 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0003680 con 0x55aad896fbe0 2023-07-14T11:33:12.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.687+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/302311886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 msgr2=0x55aad8985c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.687+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad8985c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f55a00207c0 tx=0x7f55a00136f0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.687+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/302311886 shutdown_connections 2023-07-14T11:33:12.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.687+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8972af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.687+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad89814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.687+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/302311886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad8985c90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.687+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/302311886 >> 172.21.15.161:0/302311886 conn(0x55aad8965da0 msgr2=0x55aad897a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:12.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.688+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/302311886 shutdown_connections 2023-07-14T11:33:12.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.689+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/302311886 wait complete. 2023-07-14T11:33:12.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.690+0000 7f55c5f2cb80 1 -- start start 2023-07-14T11:33:12.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.690+0000 7f55c5f2cb80 1 Processor -- start 2023-07-14T11:33:12.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.691+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad897fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.691+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad89802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.691+0000 7f55c5f2cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8980800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.691+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad89de3d0 con 0x55aad896fbe0 2023-07-14T11:33:12.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.691+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad89de540 con 0x55aad896f2d0 2023-07-14T11:33:12.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.691+0000 7f55c5f2cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad89ef710 con 0x55aad897aad0 2023-07-14T11:33:12.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.691+0000 7f55c5f2cb80 10 client.0 _ll_register_callbacks cb 0x7f55a0011b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:33:12.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.692+0000 7f55abfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad897fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.692+0000 7f55abfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad897fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47078/0 (socket says 172.21.15.161:47078) 2023-07-14T11:33:12.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.692+0000 7f55abfff700 1 -- 172.21.15.161:0/1172907251 learned_addr learned my addr 172.21.15.161:0/1172907251 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:12.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.692+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8980800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.693+0000 7f55b0b9c700 1 -- 172.21.15.161:0/1172907251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 msgr2=0x55aad897fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:12.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.693+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad897fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.693+0000 7f55b0b9c700 1 -- 172.21.15.161:0/1172907251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 msgr2=0x55aad89802c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:33:12.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.693+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad89802c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.693+0000 7f55b0b9c700 1 -- 172.21.15.161:0/1172907251 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aad86d8c60 con 0x55aad897aad0 2023-07-14T11:33:12.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.693+0000 7f55b0b9c700 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8980800 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f55a40cb8c0 tx=0x7f55a401df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fa893f6ca78621 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:12.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.694+0000 7f558b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:33:12.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.694+0000 7f558b7fe700 1 -- 172.21.15.161:0/1172907251 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a401dae0 con 0x55aad897aad0 2023-07-14T11:33:12.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.694+0000 7f558b7fe700 1 -- 172.21.15.161:0/1172907251 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f55a409b2d0 con 0x55aad897aad0 2023-07-14T11:33:12.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.694+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f559c004390 con 0x55aad897aad0 2023-07-14T11:33:12.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.694+0000 7f558b7fe700 1 -- 172.21.15.161:0/1172907251 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a409f300 con 0x55aad897aad0 2023-07-14T11:33:12.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.694+0000 7f55c5f2cb80 10 client.26968 fetch_fsmap learned FSMap version 29 2023-07-14T11:33:12.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.694+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55aad86ee2b0 con 0x55aad897aad0 2023-07-14T11:33:12.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f558b7fe700 1 -- 172.21.15.161:0/1172907251 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a40b7080 con 0x55aad897aad0 2023-07-14T11:33:12.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f558b7fe700 1 -- 172.21.15.161:0/1172907251 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 29) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f55a409a7c0 con 0x55aad897aad0 2023-07-14T11:33:12.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f55c5f2cb80 10 client.26968 fetch_fsmap finished waiting for FSMap version 29 2023-07-14T11:33:12.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f55c5f2cb80 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:33:12.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f55c5f2cb80 20 client.26968 populate_metadata read hostname 'smithi161' 2023-07-14T11:33:12.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad89f0c10 0x55aad89f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:12.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f55c5f2cb80 4 client.26968 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:33:12.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.695+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aad887d050 con 0x55aad89f0c10 2023-07-14T11:33:12.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.696+0000 7f55abfff700 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad89f0c10 0x55aad89f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:12.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.699+0000 7f55abfff700 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad89f0c10 0x55aad89f3020 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168dddb09d73bf4e server_cookie=a54a89296bbe993c in_seq=0 out_seq=0 2023-07-14T11:33:12.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.699+0000 7f558b7fe700 10 client.26968 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:12.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.700+0000 7f558b7fe700 1 -- 172.21.15.161:0/1172907251 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55aad887d050 con 0x55aad89f0c10 2023-07-14T11:33:12.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.700+0000 7f558b7fe700 10 client.26968 handle_command_reply: tid=0 2023-07-14T11:33:12.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.700+0000 7f55c5f2cb80 1 client.26968 shutdown 2023-07-14T11:33:12.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 msgr2=0x55aad8980800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8980800 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f55a40cb8c0 tx=0x7f55a401df90 comp rx=0 tx=0).stop 2023-07-14T11:33:12.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 shutdown_connections 2023-07-14T11:33:12.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55aad89f0c10 0x55aad89f3020 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55aad897aad0 0x55aad8980800 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aad896f2d0 0x55aad897fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 --2- 172.21.15.161:0/1172907251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad896fbe0 0x55aad89802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 >> 172.21.15.161:0/1172907251 conn(0x55aad8965da0 msgr2=0x55aad8972af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:12.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f558b7fe700 0 client.26968 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:12.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 shutdown_connections 2023-07-14T11:33:12.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.701+0000 7f55c5f2cb80 1 -- 172.21.15.161:0/1172907251 wait complete. 2023-07-14T11:33:12.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.702+0000 7f55c5f2cb80 20 client.26968 trim_cache size 0 max 16384 2023-07-14T11:33:12.707 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:33:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:33:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:33:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:33:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:33:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:33:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:33:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:33:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:33:12.709 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:33:12.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f55940a12c0 msgr2=0x7f55940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f55940a12c0 0x7f55940a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f55a00096b0 tx=0x7f55a00134e0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 msgr2=0x7f55ac19c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:12.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 0x7f55ac19c8b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55a400b070 tx=0x7f55a4007930 comp rx=0 tx=0).stop 2023-07-14T11:33:12.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 shutdown_connections 2023-07-14T11:33:12.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f55940a12c0 0x7f55940a3770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55ac072850 0x7f55ac197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55ac10c380 0x7f55ac19c8b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 --2- 172.21.15.161:0/2493555373 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f55ac073160 0x7f55ac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:12.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.704+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 >> 172.21.15.161:0/2493555373 conn(0x7f55ac0fb5e0 msgr2=0x7f55ac103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:12.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.705+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 shutdown_connections 2023-07-14T11:33:12.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:12.705+0000 7f55b25ff700 1 -- 172.21.15.161:0/2493555373 wait complete. 2023-07-14T11:33:12.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:12 smithi182 ceph-mon[129366]: pgmap v341: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-07-14T11:33:12.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:12 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:33:12.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:12 smithi161 ceph-mon[95526]: pgmap v341: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-07-14T11:33:12.729 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:12 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:33:12.730 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:33:13.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1001595 8.84 MB/sec execute 475 sec latency 914.494 ms 2023-07-14T11:33:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:13 smithi175 ceph-mon[129081]: pgmap v342: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-07-14T11:33:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:13 smithi182 ceph-mon[129366]: pgmap v342: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-07-14T11:33:14.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1001595 8.83 MB/sec execute 476 sec latency 1914.646 ms 2023-07-14T11:33:14.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:13 smithi161 ceph-mon[95526]: pgmap v342: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-07-14T11:33:15.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1001595 8.81 MB/sec execute 477 sec latency 2914.744 ms 2023-07-14T11:33:16.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1001595 8.79 MB/sec execute 478 sec latency 3914.816 ms 2023-07-14T11:33:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:16 smithi175 ceph-mon[129081]: pgmap v343: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-07-14T11:33:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:16 smithi182 ceph-mon[129366]: pgmap v343: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-07-14T11:33:16.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:16 smithi161 ceph-mon[95526]: pgmap v343: 129 pgs: 129 active+clean; 757 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-07-14T11:33:17.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1001595 8.77 MB/sec execute 479 sec latency 4914.902 ms 2023-07-14T11:33:18.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1002308 8.76 MB/sec execute 480 sec latency 5155.128 ms 2023-07-14T11:33:18.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:18 smithi175 ceph-mon[129081]: pgmap v344: 129 pgs: 129 active+clean; 762 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 161 op/s 2023-07-14T11:33:18.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:18 smithi182 ceph-mon[129366]: pgmap v344: 129 pgs: 129 active+clean; 762 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 161 op/s 2023-07-14T11:33:18.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:18 smithi161 ceph-mon[95526]: pgmap v344: 129 pgs: 129 active+clean; 762 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 161 op/s 2023-07-14T11:33:19.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1003690 8.75 MB/sec execute 481 sec latency 26.579 ms 2023-07-14T11:33:20.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1005722 8.76 MB/sec execute 482 sec latency 15.320 ms 2023-07-14T11:33:20.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:20 smithi175 ceph-mon[129081]: pgmap v345: 129 pgs: 129 active+clean; 762 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 142 op/s 2023-07-14T11:33:20.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:20 smithi182 ceph-mon[129366]: pgmap v345: 129 pgs: 129 active+clean; 762 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 142 op/s 2023-07-14T11:33:20.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:20 smithi161 ceph-mon[95526]: pgmap v345: 129 pgs: 129 active+clean; 762 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 142 op/s 2023-07-14T11:33:21.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1006907 8.76 MB/sec execute 483 sec latency 60.498 ms 2023-07-14T11:33:22.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1009179 8.77 MB/sec execute 484 sec latency 15.491 ms 2023-07-14T11:33:22.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:22 smithi175 ceph-mon[129081]: pgmap v346: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 211 op/s 2023-07-14T11:33:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:22 smithi182 ceph-mon[129366]: pgmap v346: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 211 op/s 2023-07-14T11:33:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:22 smithi161 ceph-mon[95526]: pgmap v346: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 211 op/s 2023-07-14T11:33:23.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1009993 8.75 MB/sec execute 485 sec latency 80.178 ms 2023-07-14T11:33:24.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:23 smithi175 ceph-mon[129081]: pgmap v347: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.3 MiB/s wr, 119 op/s 2023-07-14T11:33:24.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:23 smithi182 ceph-mon[129366]: pgmap v347: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.3 MiB/s wr, 119 op/s 2023-07-14T11:33:24.218 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1012152 8.76 MB/sec execute 486 sec latency 20.613 ms 2023-07-14T11:33:24.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:23 smithi161 ceph-mon[95526]: pgmap v347: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.3 MiB/s wr, 119 op/s 2023-07-14T11:33:25.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1013226 8.75 MB/sec execute 487 sec latency 37.452 ms 2023-07-14T11:33:26.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1015223 8.76 MB/sec execute 488 sec latency 40.600 ms 2023-07-14T11:33:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:26 smithi175 ceph-mon[129081]: pgmap v348: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.3 MiB/s wr, 119 op/s 2023-07-14T11:33:26.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:26 smithi182 ceph-mon[129366]: pgmap v348: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.3 MiB/s wr, 119 op/s 2023-07-14T11:33:26.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:26 smithi161 ceph-mon[95526]: pgmap v348: 129 pgs: 129 active+clean; 766 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.3 MiB/s wr, 119 op/s 2023-07-14T11:33:27.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1016731 8.76 MB/sec execute 489 sec latency 17.652 ms 2023-07-14T11:33:28.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1018190 8.75 MB/sec execute 490 sec latency 97.816 ms 2023-07-14T11:33:28.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:28 smithi175 ceph-mon[129081]: pgmap v349: 129 pgs: 129 active+clean; 771 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 214 op/s 2023-07-14T11:33:28.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:28 smithi182 ceph-mon[129366]: pgmap v349: 129 pgs: 129 active+clean; 771 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 214 op/s 2023-07-14T11:33:28.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:28 smithi161 ceph-mon[95526]: pgmap v349: 129 pgs: 129 active+clean; 771 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 214 op/s 2023-07-14T11:33:29.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1020027 8.76 MB/sec execute 491 sec latency 34.227 ms 2023-07-14T11:33:30.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1021441 8.75 MB/sec execute 492 sec latency 66.951 ms 2023-07-14T11:33:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:30 smithi175 ceph-mon[129081]: pgmap v350: 129 pgs: 129 active+clean; 771 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 171 op/s 2023-07-14T11:33:30.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:30 smithi182 ceph-mon[129366]: pgmap v350: 129 pgs: 129 active+clean; 771 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 171 op/s 2023-07-14T11:33:30.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:30 smithi161 ceph-mon[95526]: pgmap v350: 129 pgs: 129 active+clean; 771 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 171 op/s 2023-07-14T11:33:31.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1022959 8.75 MB/sec execute 493 sec latency 440.374 ms 2023-07-14T11:33:32.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1022959 8.74 MB/sec execute 494 sec latency 1440.527 ms 2023-07-14T11:33:32.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:32 smithi175 ceph-mon[129081]: pgmap v351: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 263 op/s 2023-07-14T11:33:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:32 smithi182 ceph-mon[129366]: pgmap v351: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 263 op/s 2023-07-14T11:33:32.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:32 smithi161 ceph-mon[95526]: pgmap v351: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 263 op/s 2023-07-14T11:33:33.219 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1022959 8.72 MB/sec execute 495 sec latency 2440.660 ms 2023-07-14T11:33:34.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:33 smithi175 ceph-mon[129081]: pgmap v352: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:33:34.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:33 smithi182 ceph-mon[129366]: pgmap v352: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:33:34.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1022959 8.70 MB/sec execute 496 sec latency 3440.755 ms 2023-07-14T11:33:34.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:33 smithi161 ceph-mon[95526]: pgmap v352: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:33:35.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1022959 8.68 MB/sec execute 497 sec latency 4440.891 ms 2023-07-14T11:33:36.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1023929 8.68 MB/sec execute 498 sec latency 4501.938 ms 2023-07-14T11:33:36.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:36 smithi175 ceph-mon[129081]: pgmap v353: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:33:36.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:36 smithi182 ceph-mon[129366]: pgmap v353: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:33:36.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:36 smithi161 ceph-mon[95526]: pgmap v353: 129 pgs: 129 active+clean; 776 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:33:37.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1026009 8.68 MB/sec execute 499 sec latency 22.714 ms 2023-07-14T11:33:37.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:37 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:33:37.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:37 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:33:37.755 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:37 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:33:38.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1027096 8.68 MB/sec execute 500 sec latency 82.664 ms 2023-07-14T11:33:38.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:38 smithi175 ceph-mon[129081]: pgmap v354: 129 pgs: 129 active+clean; 779 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 212 op/s 2023-07-14T11:33:38.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:38 smithi182 ceph-mon[129366]: pgmap v354: 129 pgs: 129 active+clean; 779 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 212 op/s 2023-07-14T11:33:38.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:38 smithi161 ceph-mon[95526]: pgmap v354: 129 pgs: 129 active+clean; 779 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 212 op/s 2023-07-14T11:33:39.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1028326 8.67 MB/sec execute 501 sec latency 23.864 ms 2023-07-14T11:33:39.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:39 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:33:39.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:39 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:33:39.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:39 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:33:40.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1030318 8.68 MB/sec execute 502 sec latency 47.943 ms 2023-07-14T11:33:40.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:40 smithi175 ceph-mon[129081]: pgmap v355: 129 pgs: 129 active+clean; 779 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2023-07-14T11:33:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:40 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:40 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:40 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:33:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:40 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:33:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:40 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:40 smithi182 ceph-mon[129366]: pgmap v355: 129 pgs: 129 active+clean; 779 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2023-07-14T11:33:40.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:40 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:40 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:40 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:33:40.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:40 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:33:40.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:40 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:40 smithi161 ceph-mon[95526]: pgmap v355: 129 pgs: 129 active+clean; 779 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2023-07-14T11:33:40.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:40 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:40 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:40.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:40 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:33:40.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:40 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:33:40.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:40 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:33:41.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1031269 8.67 MB/sec execute 503 sec latency 19.768 ms 2023-07-14T11:33:42.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1032839 8.66 MB/sec execute 504 sec latency 101.624 ms 2023-07-14T11:33:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:42 smithi175 ceph-mon[129081]: pgmap v356: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2023-07-14T11:33:42.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:42 smithi182 ceph-mon[129366]: pgmap v356: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2023-07-14T11:33:42.731 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:33:42.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:42 smithi161 ceph-mon[95526]: pgmap v356: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2023-07-14T11:33:42.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.979+0000 7f5d35d9d700 1 -- 172.21.15.161:0/2583043067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d3010c2b0 msgr2=0x7f5d3010e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:42.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.979+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/2583043067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d3010c2b0 0x7f5d3010e690 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009f90 tx=0x7f5d2800b330 comp rx=0 tx=0).stop 2023-07-14T11:33:42.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.980+0000 7f5d35d9d700 1 -- 172.21.15.161:0/2583043067 shutdown_connections 2023-07-14T11:33:42.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.980+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/2583043067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d3010c2b0 0x7f5d3010e690 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.980+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/2583043067 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 0x7f5d30072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.980+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/2583043067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d30073140 0x7f5d3010bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.980+0000 7f5d35d9d700 1 -- 172.21.15.161:0/2583043067 >> 172.21.15.161:0/2583043067 conn(0x7f5d300fb510 msgr2=0x7f5d300fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:42.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.980+0000 7f5d35d9d700 1 -- 172.21.15.161:0/2583043067 shutdown_connections 2023-07-14T11:33:42.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.980+0000 7f5d35d9d700 1 -- 172.21.15.161:0/2583043067 wait complete. 2023-07-14T11:33:42.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 Processor -- start 2023-07-14T11:33:42.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 -- start start 2023-07-14T11:33:42.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 0x7f5d30197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d30073140 0x7f5d30198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d3010c2b0 0x7f5d3019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3019cd00 con 0x7f5d3010c2b0 2023-07-14T11:33:42.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3019ce70 con 0x7f5d30073140 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d35d9d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3019d030 con 0x7f5d30072830 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d34d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 0x7f5d30197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.981+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d30073140 0x7f5d30198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d30073140 0x7f5d30198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:53426/0 (socket says 172.21.15.161:53426) 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d34d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 0x7f5d30197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38022/0 (socket says 172.21.15.161:38022) 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 -- 172.21.15.161:0/704650768 learned_addr learned my addr 172.21.15.161:0/704650768 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d3559c700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d3010c2b0 0x7f5d3019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 -- 172.21.15.161:0/704650768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 msgr2=0x7f5d30197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:42.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 0x7f5d30197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 -- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d3010c2b0 msgr2=0x7f5d3019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:42.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d3010c2b0 0x7f5d3019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 -- 172.21.15.161:0/704650768 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c009520 con 0x7f5d30073140 2023-07-14T11:33:42.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 0x7f5d30197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:33:42.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d30073140 0x7f5d30198260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00eb30 tx=0x7f5d1c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:42.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2dffb700 1 -- 172.21.15.161:0/704650768 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c009880 con 0x7f5d30073140 2023-07-14T11:33:42.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d2dffb700 1 -- 172.21.15.161:0/704650768 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1c0044e0 con 0x7f5d30073140 2023-07-14T11:33:42.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.982+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d28009c40 con 0x7f5d30073140 2023-07-14T11:33:42.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.983+0000 7f5d2dffb700 1 -- 172.21.15.161:0/704650768 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c003d80 con 0x7f5d30073140 2023-07-14T11:33:42.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.983+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3019d500 con 0x7f5d30073140 2023-07-14T11:33:42.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.986+0000 7f5d2dffb700 1 -- 172.21.15.161:0/704650768 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5d1c003ee0 con 0x7f5d30073140 2023-07-14T11:33:42.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.987+0000 7f5d4a894b80 1 Processor -- start 2023-07-14T11:33:42.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.987+0000 7f5d4a894b80 1 -- start start 2023-07-14T11:33:42.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374e98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e9e00 0x55dd374f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd374f6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd373eecb0 con 0x55dd374e94f0 2023-07-14T11:33:42.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37441a40 con 0x55dd374e9e00 2023-07-14T11:33:42.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd3743aed0 con 0x55dd374f3d30 2023-07-14T11:33:42.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd374f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d34d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e9e00 0x55dd374f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d34d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e9e00 0x55dd374f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:53438/0 (socket says 172.21.15.161:53438) 2023-07-14T11:33:42.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d34d9b700 1 -- 172.21.15.161:0/359915176 learned_addr learned my addr 172.21.15.161:0/359915176 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:42.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d3559c700 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374e98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.988+0000 7f5d2dffb700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d200a13e0 0x7f5d200a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d34d9b700 1 -- 172.21.15.161:0/359915176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 msgr2=0x55dd374f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:42.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd374f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d2dffb700 1 -- 172.21.15.161:0/704650768 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d1c014070 con 0x7f5d30073140 2023-07-14T11:33:42.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d34d9b700 1 -- 172.21.15.161:0/359915176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 msgr2=0x55dd374e98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:42.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374e98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d34d9b700 1 -- 172.21.15.161:0/359915176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd37268de0 con 0x55dd374e9e00 2023-07-14T11:33:42.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d3559c700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d200a13e0 0x7f5d200a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e9e00 0x55dd374f37f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d240115a0 tx=0x7f5d240118b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbebd735015028dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:42.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d1affd700 1 -- 172.21.15.161:0/359915176 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24004230 con 0x55dd374e9e00 2023-07-14T11:33:42.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d3559c700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d200a13e0 0x7f5d200a3890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d280060d0 tx=0x7f5d28009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:42.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d1affd700 1 -- 172.21.15.161:0/359915176 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d24004390 con 0x55dd374e9e00 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.989+0000 7f5d1affd700 1 -- 172.21.15.161:0/359915176 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24019900 con 0x55dd374e9e00 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 -- 172.21.15.161:0/359915176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e9e00 msgr2=0x55dd374f37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e9e00 0x55dd374f37f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d240115a0 tx=0x7f5d240118b0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 -- 172.21.15.161:0/359915176 shutdown_connections 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374e98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd374f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/359915176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e9e00 0x55dd374f37f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:42.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 -- 172.21.15.161:0/359915176 >> 172.21.15.161:0/359915176 conn(0x55dd374deff0 msgr2=0x55dd374e4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 -- 172.21.15.161:0/359915176 shutdown_connections 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.990+0000 7f5d4a894b80 1 -- 172.21.15.161:0/359915176 wait complete. 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 -- start start 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 Processor -- start 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374f0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374f3d30 0x55dd374f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:42.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd373daac0 con 0x55dd374e94f0 2023-07-14T11:33:42.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd373da7c0 con 0x55dd374f3d30 2023-07-14T11:33:42.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd372be920 con 0x55dd374e9e00 2023-07-14T11:33:42.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.991+0000 7f5d4a894b80 10 client.0 _ll_register_callbacks cb 0x55dd373d8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:33:42.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d34d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d3559c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374f3d30 0x55dd374f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:42.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374f0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38050/0 (socket says 172.21.15.161:38050) 2023-07-14T11:33:43.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 -- 172.21.15.161:0/3702077545 learned_addr learned my addr 172.21.15.161:0/3702077545 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:43.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 -- 172.21.15.161:0/3702077545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374f3d30 msgr2=0x55dd374f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:43.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374f3d30 0x55dd374f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 -- 172.21.15.161:0/3702077545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 msgr2=0x55dd374f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:43.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 -- 172.21.15.161:0/3702077545 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d24009d90 con 0x55dd374e9e00 2023-07-14T11:33:43.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:33:43.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374f0950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00eb00 tx=0x7f5d1c099ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc5cd0b9f1c82bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:43.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d0ffff700 1 -- 172.21.15.161:0/3702077545 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c016330 con 0x55dd374e9e00 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.992+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd373944f0 con 0x55dd374e9e00 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d0ffff700 1 -- 172.21.15.161:0/3702077545 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d1c00cbd0 con 0x55dd374e9e00 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d0ffff700 1 -- 172.21.15.161:0/3702077545 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d1c0cac00 con 0x55dd374e9e00 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 10 client.26980 fetch_fsmap learned FSMap version 30 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dd37251c60 con 0x55dd374e9e00 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d0ffff700 1 -- 172.21.15.161:0/3702077545 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd373944f0 con 0x55dd374e9e00 2023-07-14T11:33:43.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d0ffff700 1 -- 172.21.15.161:0/3702077545 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5d1c09a350 con 0x55dd374e9e00 2023-07-14T11:33:43.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 10 client.26980 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T11:33:43.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 10 client.26980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:33:43.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 20 client.26980 populate_metadata read hostname 'smithi161' 2023-07-14T11:33:43.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd375035c0 0x55dd375059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 4 client.26980 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:33:43.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd37433eb0 con 0x55dd375035c0 2023-07-14T11:33:43.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.993+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd375035c0 0x55dd375059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.994+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd375035c0 0x55dd375059d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e100b6e467ce7df server_cookie=4cc47036b635a898 in_seq=0 out_seq=0 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.994+0000 7f5d0ffff700 10 client.26980 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d0ffff700 1 -- 172.21.15.161:0/3702077545 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd37433eb0 con 0x55dd375035c0 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d0ffff700 10 client.26980 handle_command_reply: tid=0 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 client.26980 shutdown 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 msgr2=0x55dd374f0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374f0950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00eb00 tx=0x7f5d1c099ce0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 shutdown_connections 2023-07-14T11:33:43.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd375035c0 0x55dd375059d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e94f0 0x55dd374f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374f3d30 0x55dd374f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d0ffff700 0 client.26980 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:43.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/3702077545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374f0950 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 >> 172.21.15.161:0/3702077545 conn(0x55dd374deff0 msgr2=0x55dd374e4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:43.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 shutdown_connections 2023-07-14T11:33:43.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 1 -- 172.21.15.161:0/3702077545 wait complete. 2023-07-14T11:33:43.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:42.995+0000 7f5d4a894b80 20 client.26980 trim_cache size 0 max 16384 2023-07-14T11:33:43.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.012+0000 7f5d4a894b80 1 Processor -- start 2023-07-14T11:33:43.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.012+0000 7f5d4a894b80 1 -- start start 2023-07-14T11:33:43.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374f3d30 0x55dd374f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd374ea1d0 con 0x55dd374f3d30 2023-07-14T11:33:43.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd374ea980 con 0x55dd374e94f0 2023-07-14T11:33:43.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd374eeef0 con 0x55dd374e9e00 2023-07-14T11:33:43.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d3559c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374f3d30 0x55dd374f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d34d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374fefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38052/0 (socket says 172.21.15.161:38052) 2023-07-14T11:33:43.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 -- 172.21.15.161:0/2391134477 learned_addr learned my addr 172.21.15.161:0/2391134477 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:43.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 -- 172.21.15.161:0/2391134477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 msgr2=0x55dd374fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:43.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/2391134477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374fa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 -- 172.21.15.161:0/2391134477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374f3d30 msgr2=0x55dd374f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:43.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/2391134477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374f3d30 0x55dd374f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.013+0000 7f5d2ffff700 1 -- 172.21.15.161:0/2391134477 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd3728f250 con 0x55dd374e9e00 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.014+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/2391134477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374fefb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004800 tx=0x7f5d1c0961c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b41bbabac09470fe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.014+0000 7f5d1b7fe700 1 -- 172.21.15.161:0/2391134477 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0cac00 con 0x55dd374e9e00 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.014+0000 7f5d1b7fe700 1 -- 172.21.15.161:0/2391134477 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1c016330 con 0x55dd374e9e00 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.014+0000 7f5d1b7fe700 1 -- 172.21.15.161:0/2391134477 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c099700 con 0x55dd374e9e00 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.014+0000 7f5d4a894b80 1 -- 172.21.15.161:0/2391134477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 msgr2=0x55dd374fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.014+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/2391134477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374fefb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004800 tx=0x7f5d1c0961c0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 -- 172.21.15.161:0/2391134477 shutdown_connections 2023-07-14T11:33:43.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/2391134477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374f3d30 0x55dd374f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/2391134477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/2391134477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374e9e00 0x55dd374fefb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 -- 172.21.15.161:0/2391134477 >> 172.21.15.161:0/2391134477 conn(0x55dd374deff0 msgr2=0x55dd374e3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:43.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 -- 172.21.15.161:0/2391134477 shutdown_connections 2023-07-14T11:33:43.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 -- 172.21.15.161:0/2391134477 wait complete. 2023-07-14T11:33:43.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 -- start start 2023-07-14T11:33:43.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.015+0000 7f5d4a894b80 1 Processor -- start 2023-07-14T11:33:43.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e9e00 0x55dd374f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d4a894b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd37556480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd375569c0 con 0x55dd374e9e00 2023-07-14T11:33:43.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37556b00 con 0x55dd374e94f0 2023-07-14T11:33:43.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d4a894b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37556cc0 con 0x55dd374f3d30 2023-07-14T11:33:43.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d4a894b80 10 client.0 _ll_register_callbacks cb 0x55dd373d8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd37556480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d34d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd37556480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38062/0 (socket says 172.21.15.161:38062) 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 -- 172.21.15.161:0/749867034 learned_addr learned my addr 172.21.15.161:0/749867034 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d2ffff700 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e9e00 0x55dd374f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 -- 172.21.15.161:0/749867034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 msgr2=0x55dd374f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 -- 172.21.15.161:0/749867034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e9e00 msgr2=0x55dd374f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:33:43.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e9e00 0x55dd374f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.016+0000 7f5d3559c700 1 -- 172.21.15.161:0/749867034 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd37251c60 con 0x55dd374f3d30 2023-07-14T11:33:43.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d3559c700 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd37556480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5d28003c80 tx=0x7f5d280096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d7743fe0c4542f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:33:43.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:33:43.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d0ffff700 1 -- 172.21.15.161:0/749867034 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800e040 con 0x55dd374f3d30 2023-07-14T11:33:43.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d0ffff700 1 -- 172.21.15.161:0/749867034 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d2800bbf0 con 0x55dd374f3d30 2023-07-14T11:33:43.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d24004390 con 0x55dd374f3d30 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d0ffff700 1 -- 172.21.15.161:0/749867034 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d2800bdb0 con 0x55dd374f3d30 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 10 client.26986 fetch_fsmap learned FSMap version 30 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dd372672b0 con 0x55dd374f3d30 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d0ffff700 1 -- 172.21.15.161:0/749867034 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d24004390 con 0x55dd374f3d30 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d0ffff700 1 -- 172.21.15.161:0/749867034 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5d2801e430 con 0x55dd374f3d30 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 10 client.26986 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:33:43.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 20 client.26986 populate_metadata read hostname 'smithi161' 2023-07-14T11:33:43.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd37558510 0x55dd3755a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:33:43.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 4 client.26986 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:33:43.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.017+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd373f6050 con 0x55dd37558510 2023-07-14T11:33:43.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.018+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd37558510 0x55dd3755a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:33:43.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.018+0000 7f5d0ffff700 10 client.26986 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:43.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.018+0000 7f5d34d9b700 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd37558510 0x55dd3755a920 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2bd46b9e5eb18d5 server_cookie=f04e128a6a12042a in_seq=0 out_seq=0 2023-07-14T11:33:43.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.018+0000 7f5d0ffff700 1 -- 172.21.15.161:0/749867034 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55dd373f6050 con 0x55dd37558510 2023-07-14T11:33:43.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.018+0000 7f5d0ffff700 10 client.26986 handle_command_reply: tid=0 2023-07-14T11:33:43.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.018+0000 7f5d4a894b80 1 client.26986 shutdown 2023-07-14T11:33:43.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 msgr2=0x55dd37556480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:43.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd37556480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5d28003c80 tx=0x7f5d280096c0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 shutdown_connections 2023-07-14T11:33:43.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd37558510 0x55dd3755a920 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd374f3d30 0x55dd37556480 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd374e94f0 0x55dd374f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 --2- 172.21.15.161:0/749867034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd374e9e00 0x55dd374f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d0ffff700 0 client.26986 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 >> 172.21.15.161:0/749867034 conn(0x55dd374deff0 msgr2=0x55dd374fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 shutdown_connections 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 1 -- 172.21.15.161:0/749867034 wait complete. 2023-07-14T11:33:43.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d4a894b80 20 client.26986 trim_cache size 0 max 16384 2023-07-14T11:33:43.032 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:33:43.032 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:33:43.032 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:33:43.032 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:33:43.032 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:33:43.033 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:33:43.033 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:33:43.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:33:43.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:33:43.033 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:33:43.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d200a13e0 msgr2=0x7f5d200a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:43.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d200a13e0 0x7f5d200a3890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d280060d0 tx=0x7f5d28009920 comp rx=0 tx=0).stop 2023-07-14T11:33:43.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d30073140 msgr2=0x7f5d30198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:33:43.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.019+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d30073140 0x7f5d30198260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00eb30 tx=0x7f5d1c00ee40 comp rx=0 tx=0).stop 2023-07-14T11:33:43.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 shutdown_connections 2023-07-14T11:33:43.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d200a13e0 0x7f5d200a3890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d3010c2b0 0x7f5d3019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d30072830 0x7f5d30197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 --2- 172.21.15.161:0/704650768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d30073140 0x7f5d30198260 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:33:43.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 >> 172.21.15.161:0/704650768 conn(0x7f5d300fb510 msgr2=0x7f5d30103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:33:43.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 shutdown_connections 2023-07-14T11:33:43.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:33:43.020+0000 7f5d35d9d700 1 -- 172.21.15.161:0/704650768 wait complete. 2023-07-14T11:33:43.041 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:33:43.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1034163 8.67 MB/sec execute 505 sec latency 75.607 ms 2023-07-14T11:33:44.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:43 smithi175 ceph-mon[129081]: pgmap v357: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2023-07-14T11:33:44.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:43 smithi182 ceph-mon[129366]: pgmap v357: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2023-07-14T11:33:44.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1035400 8.66 MB/sec execute 506 sec latency 22.763 ms 2023-07-14T11:33:44.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:43 smithi161 ceph-mon[95526]: pgmap v357: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2023-07-14T11:33:45.220 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1037545 8.67 MB/sec execute 507 sec latency 28.477 ms 2023-07-14T11:33:46.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1038500 8.66 MB/sec execute 508 sec latency 17.504 ms 2023-07-14T11:33:46.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:46 smithi175 ceph-mon[129081]: pgmap v358: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2023-07-14T11:33:46.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:46 smithi182 ceph-mon[129366]: pgmap v358: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2023-07-14T11:33:46.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:46 smithi161 ceph-mon[95526]: pgmap v358: 129 pgs: 129 active+clean; 782 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2023-07-14T11:33:47.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1040507 8.66 MB/sec execute 509 sec latency 54.751 ms 2023-07-14T11:33:48.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1041427 8.66 MB/sec execute 510 sec latency 28.613 ms 2023-07-14T11:33:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:48 smithi175 ceph-mon[129081]: pgmap v359: 129 pgs: 129 active+clean; 791 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 205 op/s 2023-07-14T11:33:48.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:48 smithi182 ceph-mon[129366]: pgmap v359: 129 pgs: 129 active+clean; 791 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 205 op/s 2023-07-14T11:33:48.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:48 smithi161 ceph-mon[95526]: pgmap v359: 129 pgs: 129 active+clean; 791 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 205 op/s 2023-07-14T11:33:49.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1042853 8.65 MB/sec execute 511 sec latency 49.860 ms 2023-07-14T11:33:50.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1044780 8.66 MB/sec execute 512 sec latency 15.670 ms 2023-07-14T11:33:50.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:50 smithi175 ceph-mon[129081]: pgmap v360: 129 pgs: 129 active+clean; 791 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-14T11:33:50.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:50 smithi182 ceph-mon[129366]: pgmap v360: 129 pgs: 129 active+clean; 791 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-14T11:33:50.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:50 smithi161 ceph-mon[95526]: pgmap v360: 129 pgs: 129 active+clean; 791 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-14T11:33:51.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1046024 8.66 MB/sec execute 513 sec latency 65.793 ms 2023-07-14T11:33:52.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1048202 8.67 MB/sec execute 514 sec latency 27.357 ms 2023-07-14T11:33:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:52 smithi175 ceph-mon[129081]: pgmap v361: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 251 op/s 2023-07-14T11:33:52.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:52 smithi182 ceph-mon[129366]: pgmap v361: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 251 op/s 2023-07-14T11:33:52.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:52 smithi161 ceph-mon[95526]: pgmap v361: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 251 op/s 2023-07-14T11:33:53.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1049197 8.65 MB/sec execute 515 sec latency 55.879 ms 2023-07-14T11:33:54.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:53 smithi175 ceph-mon[129081]: pgmap v362: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-07-14T11:33:54.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:53 smithi182 ceph-mon[129366]: pgmap v362: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-07-14T11:33:54.222 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1051320 8.66 MB/sec execute 516 sec latency 34.214 ms 2023-07-14T11:33:54.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:53 smithi161 ceph-mon[95526]: pgmap v362: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-07-14T11:33:55.222 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1052321 8.65 MB/sec execute 517 sec latency 31.254 ms 2023-07-14T11:33:56.222 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1054401 8.66 MB/sec execute 518 sec latency 19.907 ms 2023-07-14T11:33:56.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:56 smithi175 ceph-mon[129081]: pgmap v363: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-07-14T11:33:56.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:56 smithi182 ceph-mon[129366]: pgmap v363: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-07-14T11:33:56.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:56 smithi161 ceph-mon[95526]: pgmap v363: 129 pgs: 129 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-07-14T11:33:57.221 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1055754 8.66 MB/sec execute 519 sec latency 21.047 ms 2023-07-14T11:33:58.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1057546 8.65 MB/sec execute 520 sec latency 37.650 ms 2023-07-14T11:33:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:33:58 smithi175 ceph-mon[129081]: pgmap v364: 129 pgs: 129 active+clean; 803 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 247 op/s 2023-07-14T11:33:58.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:33:58 smithi182 ceph-mon[129366]: pgmap v364: 129 pgs: 129 active+clean; 803 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 247 op/s 2023-07-14T11:33:58.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:33:58 smithi161 ceph-mon[95526]: pgmap v364: 129 pgs: 129 active+clean; 803 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 247 op/s 2023-07-14T11:33:59.223 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1058977 8.66 MB/sec execute 521 sec latency 86.670 ms 2023-07-14T11:34:00.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1060285 8.65 MB/sec execute 522 sec latency 17.332 ms 2023-07-14T11:34:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:00 smithi175 ceph-mon[129081]: pgmap v365: 129 pgs: 129 active+clean; 803 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2023-07-14T11:34:00.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:00 smithi182 ceph-mon[129366]: pgmap v365: 129 pgs: 129 active+clean; 803 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2023-07-14T11:34:00.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:00 smithi161 ceph-mon[95526]: pgmap v365: 129 pgs: 129 active+clean; 803 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2023-07-14T11:34:01.223 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1061850 8.65 MB/sec execute 523 sec latency 68.711 ms 2023-07-14T11:34:02.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1062383 8.65 MB/sec execute 524 sec latency 82.123 ms 2023-07-14T11:34:02.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:02 smithi175 ceph-mon[129081]: pgmap v366: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2023-07-14T11:34:02.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:02 smithi182 ceph-mon[129366]: pgmap v366: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2023-07-14T11:34:02.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:02 smithi161 ceph-mon[95526]: pgmap v366: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2023-07-14T11:34:03.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1063427 8.63 MB/sec execute 525 sec latency 15.154 ms 2023-07-14T11:34:04.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:03 smithi175 ceph-mon[129081]: pgmap v367: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 201 op/s 2023-07-14T11:34:04.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:03 smithi182 ceph-mon[129366]: pgmap v367: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 201 op/s 2023-07-14T11:34:04.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1065138 8.64 MB/sec execute 526 sec latency 77.998 ms 2023-07-14T11:34:04.250 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:03 smithi161 ceph-mon[95526]: pgmap v367: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 201 op/s 2023-07-14T11:34:05.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1066242 8.63 MB/sec execute 527 sec latency 34.025 ms 2023-07-14T11:34:06.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1067888 8.63 MB/sec execute 528 sec latency 20.529 ms 2023-07-14T11:34:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:06 smithi175 ceph-mon[129081]: pgmap v368: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 201 op/s 2023-07-14T11:34:06.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:06 smithi182 ceph-mon[129366]: pgmap v368: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 201 op/s 2023-07-14T11:34:06.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:06 smithi161 ceph-mon[95526]: pgmap v368: 129 pgs: 129 active+clean; 808 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 201 op/s 2023-07-14T11:34:07.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1069543 8.64 MB/sec execute 529 sec latency 23.362 ms 2023-07-14T11:34:07.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:07 smithi175 ceph-mon[129081]: osdmap e104: 12 total, 12 up, 12 in 2023-07-14T11:34:07.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:07 smithi182 ceph-mon[129366]: osdmap e104: 12 total, 12 up, 12 in 2023-07-14T11:34:07.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:07 smithi161 ceph-mon[95526]: osdmap e104: 12 total, 12 up, 12 in 2023-07-14T11:34:08.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1070691 8.63 MB/sec execute 530 sec latency 16.591 ms 2023-07-14T11:34:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:08 smithi175 ceph-mon[129081]: pgmap v370: 129 pgs: 129 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2023-07-14T11:34:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:08 smithi182 ceph-mon[129366]: pgmap v370: 129 pgs: 129 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2023-07-14T11:34:08.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:08 smithi161 ceph-mon[95526]: pgmap v370: 129 pgs: 129 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2023-07-14T11:34:09.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1072623 8.63 MB/sec execute 531 sec latency 41.447 ms 2023-07-14T11:34:10.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1073520 8.62 MB/sec execute 532 sec latency 29.608 ms 2023-07-14T11:34:10.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:10 smithi175 ceph-mon[129081]: pgmap v371: 129 pgs: 129 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2023-07-14T11:34:10.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:10 smithi182 ceph-mon[129366]: pgmap v371: 129 pgs: 129 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2023-07-14T11:34:10.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:10 smithi161 ceph-mon[95526]: pgmap v371: 129 pgs: 129 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2023-07-14T11:34:11.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1075377 8.62 MB/sec execute 533 sec latency 16.666 ms 2023-07-14T11:34:12.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1076930 8.63 MB/sec execute 534 sec latency 20.063 ms 2023-07-14T11:34:12.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:12 smithi175 ceph-mon[129081]: pgmap v372: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2023-07-14T11:34:12.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:12 smithi182 ceph-mon[129366]: pgmap v372: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2023-07-14T11:34:12.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:12 smithi161 ceph-mon[95526]: pgmap v372: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2023-07-14T11:34:13.042 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:34:13.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1078717 8.62 MB/sec execute 535 sec latency 18.098 ms 2023-07-14T11:34:13.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 -- 172.21.15.161:0/374274025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20200ffad0 msgr2=0x7f20200fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 --2- 172.21.15.161:0/374274025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20200ffad0 0x7f20200fff40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f200c009990 tx=0x7f200c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 -- 172.21.15.161:0/374274025 shutdown_connections 2023-07-14T11:34:13.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 --2- 172.21.15.161:0/374274025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2020105ae0 0x7f2020105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 --2- 172.21.15.161:0/374274025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2020100480 0x7f202010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 --2- 172.21.15.161:0/374274025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20200ffad0 0x7f20200fff40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 -- 172.21.15.161:0/374274025 >> 172.21.15.161:0/374274025 conn(0x7f20200fb5f0 msgr2=0x7f20200fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:13.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 -- 172.21.15.161:0/374274025 shutdown_connections 2023-07-14T11:34:13.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.257+0000 7f2026fff700 1 -- 172.21.15.161:0/374274025 wait complete. 2023-07-14T11:34:13.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.258+0000 7f2026fff700 1 Processor -- start 2023-07-14T11:34:13.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.258+0000 7f2026fff700 1 -- start start 2023-07-14T11:34:13.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.258+0000 7f2026fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20200ffad0 0x7f2020197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.258+0000 7f2026fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2020100480 0x7f2020198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.258+0000 7f2026fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2020105ae0 0x7f202019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.258+0000 7f2026fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202019ce10 con 0x7f20200ffad0 2023-07-14T11:34:13.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f2026fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202019cf80 con 0x7f2020105ae0 2023-07-14T11:34:13.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f2026fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202019d140 con 0x7f2020100480 2023-07-14T11:34:13.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2020105ae0 0x7f202019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f201ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2020100480 0x7f2020198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2020105ae0 0x7f202019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:40788/0 (socket says 172.21.15.161:40788) 2023-07-14T11:34:13.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 -- 172.21.15.161:0/3207807134 learned_addr learned my addr 172.21.15.161:0/3207807134 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:13.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f2024d9b700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20200ffad0 0x7f2020197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 -- 172.21.15.161:0/3207807134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2020100480 msgr2=0x7f2020198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2020100480 0x7f2020198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 -- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20200ffad0 msgr2=0x7f2020197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20200ffad0 0x7f2020197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 -- 172.21.15.161:0/3207807134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018009c40 con 0x7f2020105ae0 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.259+0000 7f202559c700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2020105ae0 0x7f202019c7d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f201800c370 tx=0x7f201800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.260+0000 7f201dffb700 1 -- 172.21.15.161:0/3207807134 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018017070 con 0x7f2020105ae0 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.260+0000 7f201dffb700 1 -- 172.21.15.161:0/3207807134 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201800f040 con 0x7f2020105ae0 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.260+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c0095f0 con 0x7f2020105ae0 2023-07-14T11:34:13.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.260+0000 7f201dffb700 1 -- 172.21.15.161:0/3207807134 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800e050 con 0x7f2020105ae0 2023-07-14T11:34:13.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.260+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20201a4a80 con 0x7f2020105ae0 2023-07-14T11:34:13.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.263+0000 7f201dffb700 1 -- 172.21.15.161:0/3207807134 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f201800cb60 con 0x7f2020105ae0 2023-07-14T11:34:13.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.264+0000 7f203a89ab80 1 Processor -- start 2023-07-14T11:34:13.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f203a89ab80 1 -- start start 2023-07-14T11:34:13.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f201dffb700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f20100a12c0 0x7f20100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1afc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b07590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f203a89ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1a02cb0 con 0x5589b1afcbe0 2023-07-14T11:34:13.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f203a89ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1a55a40 con 0x5589b1afc2d0 2023-07-14T11:34:13.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f203a89ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1a4eed0 con 0x5589b1b07ad0 2023-07-14T11:34:13.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f201dffb700 1 -- 172.21.15.161:0/3207807134 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f20180cca60 con 0x7f2020105ae0 2023-07-14T11:34:13.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f202559c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f2024d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1afc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f202559c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b09eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59868/0 (socket says 172.21.15.161:59868) 2023-07-14T11:34:13.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f202559c700 1 -- 172.21.15.161:0/4167492845 learned_addr learned my addr 172.21.15.161:0/4167492845 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:13.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.265+0000 7f201ffff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f20100a12c0 0x7f20100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f201ffff700 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b07590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f202559c700 1 -- 172.21.15.161:0/4167492845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 msgr2=0x5589b1afc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f202559c700 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1afc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f202559c700 1 -- 172.21.15.161:0/4167492845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 msgr2=0x5589b1b07590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f202559c700 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b07590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f202559c700 1 -- 172.21.15.161:0/4167492845 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589b187cde0 con 0x5589b1b07ad0 2023-07-14T11:34:13.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f201ffff700 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:34:13.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f202559c700 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b09eb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f20180a1690 tx=0x7f2018095bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f27a96c6d2b3920 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:13.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f200affd700 1 -- 172.21.15.161:0/4167492845 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180092e0 con 0x5589b1b07ad0 2023-07-14T11:34:13.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f200affd700 1 -- 172.21.15.161:0/4167492845 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201806dc50 con 0x5589b1b07ad0 2023-07-14T11:34:13.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.266+0000 7f200affd700 1 -- 172.21.15.161:0/4167492845 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180a1bc0 con 0x5589b1b07ad0 2023-07-14T11:34:13.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 -- 172.21.15.161:0/4167492845 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 msgr2=0x5589b1b09eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f201ffff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f20100a12c0 0x7f20100a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2020199440 tx=0x7f200c01a040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:13.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b09eb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f20180a1690 tx=0x7f2018095bc0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 -- 172.21.15.161:0/4167492845 shutdown_connections 2023-07-14T11:34:13.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b09eb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1afc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 --2- 172.21.15.161:0/4167492845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 -- 172.21.15.161:0/4167492845 >> 172.21.15.161:0/4167492845 conn(0x5589b1af2da0 msgr2=0x5589b1af86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:13.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 -- 172.21.15.161:0/4167492845 shutdown_connections 2023-07-14T11:34:13.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.267+0000 7f203a89ab80 1 -- 172.21.15.161:0/4167492845 wait complete. 2023-07-14T11:34:13.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 -- start start 2023-07-14T11:34:13.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 Processor -- start 2023-07-14T11:34:13.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b07570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1afcbe0 0x5589b1b04860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1b07ad0 0x5589b1b04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b19eeac0 con 0x5589b1b07ad0 2023-07-14T11:34:13.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b19ee7c0 con 0x5589b1afc2d0 2023-07-14T11:34:13.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.268+0000 7f203a89ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b18d2920 con 0x5589b1afcbe0 2023-07-14T11:34:13.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f203a89ab80 10 client.0 _ll_register_callbacks cb 0x5589b19ec760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:34:13.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1afcbe0 0x5589b1b04860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f2024d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b07570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1afcbe0 0x5589b1b04860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59872/0 (socket says 172.21.15.161:59872) 2023-07-14T11:34:13.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 -- 172.21.15.161:0/1818656866 learned_addr learned my addr 172.21.15.161:0/1818656866 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:13.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f2024d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b07570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:40812/0 (socket says 172.21.15.161:40812) 2023-07-14T11:34:13.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f202559c700 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1b07ad0 0x5589b1b04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 -- 172.21.15.161:0/1818656866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 msgr2=0x5589b1b07570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b07570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 -- 172.21.15.161:0/1818656866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1b07ad0 msgr2=0x5589b1b04fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1b07ad0 0x5589b1b04fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 -- 172.21.15.161:0/1818656866 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20201a4a80 con 0x5589b1afcbe0 2023-07-14T11:34:13.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f201ffff700 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1afcbe0 0x5589b1b04860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f200c00c010 tx=0x7f200c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ddfe80c96b44f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:13.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.269+0000 7f1ffffff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:34:13.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f1ffffff700 1 -- 172.21.15.161:0/1818656866 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c02c070 con 0x5589b1afcbe0 2023-07-14T11:34:13.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589b19a84f0 con 0x5589b1afcbe0 2023-07-14T11:34:13.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f1ffffff700 1 -- 172.21.15.161:0/1818656866 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f200c004b20 con 0x5589b1afcbe0 2023-07-14T11:34:13.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f1ffffff700 1 -- 172.21.15.161:0/1818656866 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f200c0055d0 con 0x5589b1afcbe0 2023-07-14T11:34:13.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 10 client.26998 fetch_fsmap learned FSMap version 30 2023-07-14T11:34:13.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5589b1865c60 con 0x5589b1afcbe0 2023-07-14T11:34:13.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f1ffffff700 1 -- 172.21.15.161:0/1818656866 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589b19a84f0 con 0x5589b1afcbe0 2023-07-14T11:34:13.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f1ffffff700 1 -- 172.21.15.161:0/1818656866 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f200c030400 con 0x5589b1afcbe0 2023-07-14T11:34:13.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 10 client.26998 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T11:34:13.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 10 client.26998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:34:13.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 20 client.26998 populate_metadata read hostname 'smithi161' 2023-07-14T11:34:13.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b174b0 0x5589b1b198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 4 client.26998 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:34:13.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.270+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589b1a47eb0 con 0x5589b1b174b0 2023-07-14T11:34:13.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.271+0000 7f2024d9b700 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b174b0 0x5589b1b198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.271+0000 7f1ffffff700 10 client.26998 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:13.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.271+0000 7f2024d9b700 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b174b0 0x5589b1b198c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41b3e46c0c1bf95 server_cookie=6129de1902c2aa23 in_seq=0 out_seq=0 2023-07-14T11:34:13.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f1ffffff700 1 -- 172.21.15.161:0/1818656866 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5589b1a47eb0 con 0x5589b1b174b0 2023-07-14T11:34:13.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f1ffffff700 10 client.26998 handle_command_reply: tid=0 2023-07-14T11:34:13.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 client.26998 shutdown 2023-07-14T11:34:13.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1afcbe0 msgr2=0x5589b1b04860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1afcbe0 0x5589b1b04860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f200c00c010 tx=0x7f200c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 shutdown_connections 2023-07-14T11:34:13.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b174b0 0x5589b1b198c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1b07ad0 0x5589b1b04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b07570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1818656866 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1afcbe0 0x5589b1b04860 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f1ffffff700 0 client.26998 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:13.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 >> 172.21.15.161:0/1818656866 conn(0x5589b1af2da0 msgr2=0x5589b1af8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:13.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 shutdown_connections 2023-07-14T11:34:13.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.273+0000 7f203a89ab80 1 -- 172.21.15.161:0/1818656866 wait complete. 2023-07-14T11:34:13.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.274+0000 7f203a89ab80 20 client.26998 trim_cache size 0 max 16384 2023-07-14T11:34:13.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 Processor -- start 2023-07-14T11:34:13.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 -- start start 2023-07-14T11:34:13.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afc2d0 0x5589b1affb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afcbe0 0x5589b1b000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b04020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1aeffb0 con 0x5589b1afc2d0 2023-07-14T11:34:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1af2970 con 0x5589b1afcbe0 2023-07-14T11:34:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.293+0000 7f203a89ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1b19f20 con 0x5589b1b07ad0 2023-07-14T11:34:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afcbe0 0x5589b1b000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afcbe0 0x5589b1b000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:40826/0 (socket says 172.21.15.161:40826) 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 -- 172.21.15.161:0/1805501825 learned_addr learned my addr 172.21.15.161:0/1805501825 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f202559c700 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b04020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f2024d9b700 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afc2d0 0x5589b1affb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 -- 172.21.15.161:0/1805501825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 msgr2=0x5589b1b04020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b04020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 -- 172.21.15.161:0/1805501825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afc2d0 msgr2=0x5589b1affb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afc2d0 0x5589b1affb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 -- 172.21.15.161:0/1805501825 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589b18a3250 con 0x5589b1afcbe0 2023-07-14T11:34:13.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f201ffff700 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afcbe0 0x5589b1b000b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f200c0307b0 tx=0x7f200c0098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=754d08e9d3d61235 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:13.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f200b7fe700 1 -- 172.21.15.161:0/1805501825 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c02c070 con 0x5589b1afcbe0 2023-07-14T11:34:13.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f200b7fe700 1 -- 172.21.15.161:0/1805501825 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f200c030400 con 0x5589b1afcbe0 2023-07-14T11:34:13.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.294+0000 7f200b7fe700 1 -- 172.21.15.161:0/1805501825 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c00f6c0 con 0x5589b1afcbe0 2023-07-14T11:34:13.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 -- 172.21.15.161:0/1805501825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afcbe0 msgr2=0x5589b1b000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afcbe0 0x5589b1b000b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f200c0307b0 tx=0x7f200c0098c0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 -- 172.21.15.161:0/1805501825 shutdown_connections 2023-07-14T11:34:13.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afc2d0 0x5589b1affb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b04020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 --2- 172.21.15.161:0/1805501825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afcbe0 0x5589b1b000b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 -- 172.21.15.161:0/1805501825 >> 172.21.15.161:0/1805501825 conn(0x5589b1af2da0 msgr2=0x5589b1b071a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:13.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 -- 172.21.15.161:0/1805501825 shutdown_connections 2023-07-14T11:34:13.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.295+0000 7f203a89ab80 1 -- 172.21.15.161:0/1805501825 wait complete. 2023-07-14T11:34:13.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.296+0000 7f203a89ab80 1 -- start start 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.296+0000 7f203a89ab80 1 Processor -- start 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b0e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f203a89ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b0c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f203a89ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1b0caa0 con 0x5589b1afcbe0 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f203a89ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1b0cc10 con 0x5589b1afc2d0 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f203a89ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1b0cdd0 con 0x5589b1b07ad0 2023-07-14T11:34:13.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f203a89ab80 10 client.0 _ll_register_callbacks cb 0x7f200c00bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:34:13.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f2024d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f2024d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b0df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:40838/0 (socket says 172.21.15.161:40838) 2023-07-14T11:34:13.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f2024d9b700 1 -- 172.21.15.161:0/2367315070 learned_addr learned my addr 172.21.15.161:0/2367315070 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:13.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f202559c700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b0c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f201ffff700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b0e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f2024d9b700 1 -- 172.21.15.161:0/2367315070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 msgr2=0x5589b1b0c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f2024d9b700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b0c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f2024d9b700 1 -- 172.21.15.161:0/2367315070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 msgr2=0x5589b1b0e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:13.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.297+0000 7f2024d9b700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b0e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f2024d9b700 1 -- 172.21.15.161:0/2367315070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589b1865c60 con 0x5589b1afc2d0 2023-07-14T11:34:13.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f202559c700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b0c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:34:13.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f201ffff700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b0e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:34:13.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f2024d9b700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b0df80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2014004790 tx=0x7f201401fe10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332433adcee9b20e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:13.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f1ffffff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:34:13.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f1ffffff700 1 -- 172.21.15.161:0/2367315070 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014017040 con 0x5589b1afc2d0 2023-07-14T11:34:13.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f1ffffff700 1 -- 172.21.15.161:0/2367315070 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f2014004830 con 0x5589b1afc2d0 2023-07-14T11:34:13.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f201806dc50 con 0x5589b1afc2d0 2023-07-14T11:34:13.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f1ffffff700 1 -- 172.21.15.161:0/2367315070 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f201401f600 con 0x5589b1afc2d0 2023-07-14T11:34:13.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f203a89ab80 10 client.27002 fetch_fsmap learned FSMap version 30 2023-07-14T11:34:13.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.298+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5589b187b2b0 con 0x5589b1afc2d0 2023-07-14T11:34:13.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f1ffffff700 1 -- 172.21.15.161:0/2367315070 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201806dc50 con 0x5589b1afc2d0 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f1ffffff700 1 -- 172.21.15.161:0/2367315070 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2014010790 con 0x5589b1afc2d0 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f203a89ab80 10 client.27002 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f203a89ab80 10 client.27002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f203a89ab80 20 client.27002 populate_metadata read hostname 'smithi161' 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f203a89ab80 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b6c540 0x5589b1b6e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f203a89ab80 4 client.27002 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5589b1a0a050 con 0x5589b1b6c540 2023-07-14T11:34:13.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.299+0000 7f201ffff700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b6c540 0x5589b1b6e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:13.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f1ffffff700 10 client.27002 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:13.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f201ffff700 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b6c540 0x5589b1b6e950 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fd0fad022f7f19 server_cookie=9ccb12f03d66f765 in_seq=0 out_seq=0 2023-07-14T11:34:13.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f1ffffff700 1 -- 172.21.15.161:0/2367315070 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5589b1a0a050 con 0x5589b1b6c540 2023-07-14T11:34:13.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f1ffffff700 10 client.27002 handle_command_reply: tid=0 2023-07-14T11:34:13.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 client.27002 shutdown 2023-07-14T11:34:13.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 msgr2=0x5589b1b0df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b0df80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2014004790 tx=0x7f201401fe10 comp rx=0 tx=0).stop 2023-07-14T11:34:13.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 shutdown_connections 2023-07-14T11:34:13.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5589b1b6c540 0x5589b1b6e950 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5589b1b07ad0 0x5589b1b0c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589b1afc2d0 0x5589b1b0df80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f1ffffff700 0 client.27002 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:13.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 --2- 172.21.15.161:0/2367315070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589b1afcbe0 0x5589b1b0e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 >> 172.21.15.161:0/2367315070 conn(0x5589b1af2da0 msgr2=0x5589b1afe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:13.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 shutdown_connections 2023-07-14T11:34:13.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.300+0000 7f203a89ab80 1 -- 172.21.15.161:0/2367315070 wait complete. 2023-07-14T11:34:13.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f203a89ab80 20 client.27002 trim_cache size 0 max 16384 2023-07-14T11:34:13.314 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:34:13.314 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:34:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:34:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:34:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:34:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:34:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:34:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:34:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:34:13.316 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:34:13.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f20100a12c0 msgr2=0x7f20100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f20100a12c0 0x7f20100a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2020199440 tx=0x7f200c01a040 comp rx=0 tx=0).stop 2023-07-14T11:34:13.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2020105ae0 msgr2=0x7f202019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:13.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2020105ae0 0x7f202019c7d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f201800c370 tx=0x7f201800c680 comp rx=0 tx=0).stop 2023-07-14T11:34:13.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 shutdown_connections 2023-07-14T11:34:13.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f20100a12c0 0x7f20100a3770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20200ffad0 0x7f2020197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2020105ae0 0x7f202019c7d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 --2- 172.21.15.161:0/3207807134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2020100480 0x7f2020198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 >> 172.21.15.161:0/3207807134 conn(0x7f20200fb5f0 msgr2=0x7f202010af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 shutdown_connections 2023-07-14T11:34:13.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:13.301+0000 7f2026fff700 1 -- 172.21.15.161:0/3207807134 wait complete. 2023-07-14T11:34:13.325 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:34:14.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:13 smithi175 ceph-mon[129081]: osdmap e105: 12 total, 12 up, 12 in 2023-07-14T11:34:14.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:13 smithi175 ceph-mon[129081]: pgmap v374: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2023-07-14T11:34:14.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:13 smithi182 ceph-mon[129366]: osdmap e105: 12 total, 12 up, 12 in 2023-07-14T11:34:14.216 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:13 smithi182 ceph-mon[129366]: pgmap v374: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2023-07-14T11:34:14.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1080139 8.63 MB/sec execute 536 sec latency 51.069 ms 2023-07-14T11:34:14.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:13 smithi161 ceph-mon[95526]: osdmap e105: 12 total, 12 up, 12 in 2023-07-14T11:34:14.250 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:13 smithi161 ceph-mon[95526]: pgmap v374: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2023-07-14T11:34:15.224 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1081324 8.62 MB/sec execute 537 sec latency 22.641 ms 2023-07-14T11:34:16.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1083537 8.63 MB/sec execute 538 sec latency 38.502 ms 2023-07-14T11:34:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:16 smithi175 ceph-mon[129081]: pgmap v375: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 102 op/s 2023-07-14T11:34:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:16 smithi182 ceph-mon[129366]: pgmap v375: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 102 op/s 2023-07-14T11:34:16.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:16 smithi161 ceph-mon[95526]: pgmap v375: 129 pgs: 129 active+clean; 822 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 102 op/s 2023-07-14T11:34:17.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1084641 8.62 MB/sec execute 539 sec latency 18.105 ms 2023-07-14T11:34:18.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1086704 8.63 MB/sec execute 540 sec latency 78.753 ms 2023-07-14T11:34:18.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:18 smithi175 ceph-mon[129081]: pgmap v376: 129 pgs: 129 active+clean; 829 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2023-07-14T11:34:18.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:18 smithi182 ceph-mon[129366]: pgmap v376: 129 pgs: 129 active+clean; 829 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2023-07-14T11:34:18.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:18 smithi161 ceph-mon[95526]: pgmap v376: 129 pgs: 129 active+clean; 829 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2023-07-14T11:34:19.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1087488 8.62 MB/sec execute 541 sec latency 85.527 ms 2023-07-14T11:34:20.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1088940 8.62 MB/sec execute 542 sec latency 40.558 ms 2023-07-14T11:34:20.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:20 smithi175 ceph-mon[129081]: pgmap v377: 129 pgs: 129 active+clean; 829 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2023-07-14T11:34:20.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:20 smithi182 ceph-mon[129366]: pgmap v377: 129 pgs: 129 active+clean; 829 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2023-07-14T11:34:20.749 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:20 smithi161 ceph-mon[95526]: pgmap v377: 129 pgs: 129 active+clean; 829 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2023-07-14T11:34:21.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1090825 8.63 MB/sec execute 543 sec latency 34.198 ms 2023-07-14T11:34:22.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1092119 8.62 MB/sec execute 544 sec latency 17.967 ms 2023-07-14T11:34:22.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:22 smithi175 ceph-mon[129081]: pgmap v378: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 216 op/s 2023-07-14T11:34:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:22 smithi182 ceph-mon[129366]: pgmap v378: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 216 op/s 2023-07-14T11:34:22.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:22 smithi161 ceph-mon[95526]: pgmap v378: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 216 op/s 2023-07-14T11:34:23.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1094280 8.63 MB/sec execute 545 sec latency 17.147 ms 2023-07-14T11:34:24.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:23 smithi175 ceph-mon[129081]: pgmap v379: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 209 op/s 2023-07-14T11:34:24.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:23 smithi182 ceph-mon[129366]: pgmap v379: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 209 op/s 2023-07-14T11:34:24.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1095208 8.62 MB/sec execute 546 sec latency 26.680 ms 2023-07-14T11:34:24.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:23 smithi161 ceph-mon[95526]: pgmap v379: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 209 op/s 2023-07-14T11:34:25.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1097391 8.62 MB/sec execute 547 sec latency 35.078 ms 2023-07-14T11:34:26.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1098138 8.62 MB/sec execute 548 sec latency 108.880 ms 2023-07-14T11:34:26.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:26 smithi175 ceph-mon[129081]: pgmap v380: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 180 op/s 2023-07-14T11:34:26.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:26 smithi182 ceph-mon[129366]: pgmap v380: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 180 op/s 2023-07-14T11:34:26.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:26 smithi161 ceph-mon[95526]: pgmap v380: 129 pgs: 129 active+clean; 831 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 180 op/s 2023-07-14T11:34:27.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1099978 8.61 MB/sec execute 549 sec latency 26.875 ms 2023-07-14T11:34:28.225 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1101593 8.62 MB/sec execute 550 sec latency 12.537 ms 2023-07-14T11:34:28.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:28 smithi175 ceph-mon[129081]: pgmap v381: 129 pgs: 129 active+clean; 838 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2023-07-14T11:34:28.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:28 smithi182 ceph-mon[129366]: pgmap v381: 129 pgs: 129 active+clean; 838 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2023-07-14T11:34:28.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:28 smithi161 ceph-mon[95526]: pgmap v381: 129 pgs: 129 active+clean; 838 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2023-07-14T11:34:29.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1102848 8.61 MB/sec execute 551 sec latency 77.476 ms 2023-07-14T11:34:30.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1104953 8.62 MB/sec execute 552 sec latency 41.156 ms 2023-07-14T11:34:30.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:30 smithi175 ceph-mon[129081]: pgmap v382: 129 pgs: 129 active+clean; 838 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 179 op/s 2023-07-14T11:34:30.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:30 smithi182 ceph-mon[129366]: pgmap v382: 129 pgs: 129 active+clean; 838 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 179 op/s 2023-07-14T11:34:30.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:30 smithi161 ceph-mon[95526]: pgmap v382: 129 pgs: 129 active+clean; 838 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 179 op/s 2023-07-14T11:34:31.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1106103 8.62 MB/sec execute 553 sec latency 56.755 ms 2023-07-14T11:34:32.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1108376 8.62 MB/sec execute 554 sec latency 19.606 ms 2023-07-14T11:34:32.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:32 smithi175 ceph-mon[129081]: pgmap v383: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2023-07-14T11:34:32.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:32 smithi182 ceph-mon[129366]: pgmap v383: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2023-07-14T11:34:32.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:32 smithi161 ceph-mon[95526]: pgmap v383: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2023-07-14T11:34:33.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1109620 8.62 MB/sec execute 555 sec latency 16.154 ms 2023-07-14T11:34:34.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:33 smithi175 ceph-mon[129081]: pgmap v384: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 187 op/s 2023-07-14T11:34:34.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:33 smithi182 ceph-mon[129366]: pgmap v384: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 187 op/s 2023-07-14T11:34:34.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1111727 8.63 MB/sec execute 556 sec latency 54.261 ms 2023-07-14T11:34:34.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:33 smithi161 ceph-mon[95526]: pgmap v384: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 187 op/s 2023-07-14T11:34:35.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1112728 8.62 MB/sec execute 557 sec latency 26.071 ms 2023-07-14T11:34:36.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1114805 8.62 MB/sec execute 558 sec latency 30.147 ms 2023-07-14T11:34:36.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:36 smithi175 ceph-mon[129081]: pgmap v385: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 187 op/s 2023-07-14T11:34:36.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:36 smithi182 ceph-mon[129366]: pgmap v385: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 187 op/s 2023-07-14T11:34:36.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:36 smithi161 ceph-mon[95526]: pgmap v385: 129 pgs: 129 active+clean; 841 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 187 op/s 2023-07-14T11:34:37.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1116086 8.62 MB/sec execute 559 sec latency 27.333 ms 2023-07-14T11:34:38.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1118220 8.62 MB/sec execute 560 sec latency 15.718 ms 2023-07-14T11:34:38.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:38 smithi175 ceph-mon[129081]: pgmap v386: 129 pgs: 129 active+clean; 847 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 276 op/s 2023-07-14T11:34:38.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:38 smithi182 ceph-mon[129366]: pgmap v386: 129 pgs: 129 active+clean; 847 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 276 op/s 2023-07-14T11:34:38.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:38 smithi161 ceph-mon[95526]: pgmap v386: 129 pgs: 129 active+clean; 847 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 276 op/s 2023-07-14T11:34:39.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1119381 8.62 MB/sec execute 561 sec latency 76.058 ms 2023-07-14T11:34:40.227 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1120578 8.62 MB/sec execute 562 sec latency 38.602 ms 2023-07-14T11:34:40.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:40 smithi175 ceph-mon[129081]: pgmap v387: 129 pgs: 129 active+clean; 847 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:34:40.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:40 smithi182 ceph-mon[129366]: pgmap v387: 129 pgs: 129 active+clean; 847 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:34:40.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:40 smithi161 ceph-mon[95526]: pgmap v387: 129 pgs: 129 active+clean; 847 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 186 op/s 2023-07-14T11:34:41.226 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1122646 8.62 MB/sec execute 563 sec latency 40.367 ms 2023-07-14T11:34:41.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:34:41.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:34:41.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:34:41.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:41.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:42.227 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1123573 8.61 MB/sec execute 564 sec latency 60.745 ms 2023-07-14T11:34:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:42 smithi175 ceph-mon[129081]: pgmap v388: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2023-07-14T11:34:42.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:42 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:34:42.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:42 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:34:42.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:42 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:42.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:42 smithi182 ceph-mon[129366]: pgmap v388: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2023-07-14T11:34:42.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:42 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:34:42.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:42 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:34:42.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:42 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:42.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:42 smithi161 ceph-mon[95526]: pgmap v388: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2023-07-14T11:34:42.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:42 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:34:42.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:42 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:34:42.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:42 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:34:43.227 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1125625 8.62 MB/sec execute 565 sec latency 91.181 ms 2023-07-14T11:34:43.328 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:34:43.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.546+0000 7fbedd59d700 1 -- 172.21.15.161:0/15370221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 msgr2=0x7fbed8105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.546+0000 7fbedd59d700 1 --2- 172.21.15.161:0/15370221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8105dd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009990 tx=0x7fbec8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.547+0000 7fbedd59d700 1 -- 172.21.15.161:0/15370221 shutdown_connections 2023-07-14T11:34:43.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.547+0000 7fbedd59d700 1 --2- 172.21.15.161:0/15370221 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbed81003a0 0x7fbed810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.547+0000 7fbedd59d700 1 --2- 172.21.15.161:0/15370221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8105dd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.547+0000 7fbedd59d700 1 --2- 172.21.15.161:0/15370221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbed80ff9f0 0x7fbed80ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.547+0000 7fbedd59d700 1 -- 172.21.15.161:0/15370221 >> 172.21.15.161:0/15370221 conn(0x7fbed80fb510 msgr2=0x7fbed80fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:43.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.547+0000 7fbedd59d700 1 -- 172.21.15.161:0/15370221 shutdown_connections 2023-07-14T11:34:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.547+0000 7fbedd59d700 1 -- 172.21.15.161:0/15370221 wait complete. 2023-07-14T11:34:43.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 Processor -- start 2023-07-14T11:34:43.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 -- start start 2023-07-14T11:34:43.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbed80ff9f0 0x7fbed819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbed81003a0 0x7fbed819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8197fe0 con 0x7fbed81003a0 2023-07-14T11:34:43.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8198150 con 0x7fbed8105a00 2023-07-14T11:34:43.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedd59d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8198310 con 0x7fbed80ff9f0 2023-07-14T11:34:43.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.548+0000 7fbedcd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43448/0 (socket says 172.21.15.161:43448) 2023-07-14T11:34:43.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 -- 172.21.15.161:0/2603693700 learned_addr learned my addr 172.21.15.161:0/2603693700 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:43.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbed77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbed81003a0 0x7fbed819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbed7fff700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbed80ff9f0 0x7fbed819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 -- 172.21.15.161:0/2603693700 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbed80ff9f0 msgr2=0x7fbed819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbed80ff9f0 0x7fbed819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 -- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbed81003a0 msgr2=0x7fbed819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbed81003a0 0x7fbed819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 -- 172.21.15.161:0/2603693700 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80095f0 con 0x7fbed8105a00 2023-07-14T11:34:43.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbed7fff700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbed80ff9f0 0x7fbed819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:34:43.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8197970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbed000ef30 tx=0x7fbed000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:43.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbed57fa700 1 -- 172.21.15.161:0/2603693700 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0018070 con 0x7fbed8105a00 2023-07-14T11:34:43.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbed57fa700 1 -- 172.21.15.161:0/2603693700 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed000f040 con 0x7fbed8105a00 2023-07-14T11:34:43.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.549+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed8198590 con 0x7fbed8105a00 2023-07-14T11:34:43.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.550+0000 7fbed57fa700 1 -- 172.21.15.161:0/2603693700 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000cd90 con 0x7fbed8105a00 2023-07-14T11:34:43.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.550+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed81a48f0 con 0x7fbed8105a00 2023-07-14T11:34:43.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.553+0000 7fbed57fa700 1 -- 172.21.15.161:0/2603693700 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fbed0004ad0 con 0x7fbed8105a00 2023-07-14T11:34:43.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.554+0000 7fbef2048b80 1 Processor -- start 2023-07-14T11:34:43.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.554+0000 7fbef2048b80 1 -- start start 2023-07-14T11:34:43.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.554+0000 7fbed57fa700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbec00a13e0 0x7fbec00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b40b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed57fa700 1 -- 172.21.15.161:0/2603693700 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fbed00ccb80 con 0x7fbed8105a00 2023-07-14T11:34:43.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40be00 0x55ba0b4157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b415d30 0x55ba0b418110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbef2048b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b310cb0 con 0x55ba0b415d30 2023-07-14T11:34:43.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbef2048b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b363a40 con 0x55ba0b40be00 2023-07-14T11:34:43.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbef2048b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b35ced0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbec00a13e0 0x7fbec00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40be00 0x55ba0b4157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40be00 0x55ba0b4157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43452/0 (socket says 172.21.15.161:43452) 2023-07-14T11:34:43.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 -- 172.21.15.161:0/4040147275 learned_addr learned my addr 172.21.15.161:0/4040147275 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:43.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b415d30 0x55ba0b418110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed7fff700 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b40b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 -- 172.21.15.161:0/4040147275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 msgr2=0x55ba0b40b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b40b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 -- 172.21.15.161:0/4040147275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b415d30 msgr2=0x55ba0b418110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b415d30 0x55ba0b418110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.555+0000 7fbed77fe700 1 -- 172.21.15.161:0/4040147275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba0b18ade0 con 0x55ba0b40be00 2023-07-14T11:34:43.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.556+0000 7fbed77fe700 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40be00 0x55ba0b4157f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbecc011b70 tx=0x7fbecc011e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=528ab7bd5820708b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:43.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.556+0000 7fbed77fe700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbec00a13e0 0x7fbec00a3890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00f6a0 tx=0x7fbecc018000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:43.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.556+0000 7fbebe7fc700 1 -- 172.21.15.161:0/4040147275 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc009830 con 0x55ba0b40be00 2023-07-14T11:34:43.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.556+0000 7fbebe7fc700 1 -- 172.21.15.161:0/4040147275 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbecc004830 con 0x55ba0b40be00 2023-07-14T11:34:43.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.556+0000 7fbebe7fc700 1 -- 172.21.15.161:0/4040147275 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc005350 con 0x55ba0b40be00 2023-07-14T11:34:43.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 -- 172.21.15.161:0/4040147275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40be00 msgr2=0x55ba0b4157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40be00 0x55ba0b4157f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbecc011b70 tx=0x7fbecc011e80 comp rx=0 tx=0).stop 2023-07-14T11:34:43.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 -- 172.21.15.161:0/4040147275 shutdown_connections 2023-07-14T11:34:43.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b415d30 0x55ba0b418110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b40b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 --2- 172.21.15.161:0/4040147275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40be00 0x55ba0b4157f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 -- 172.21.15.161:0/4040147275 >> 172.21.15.161:0/4040147275 conn(0x55ba0b400ff0 msgr2=0x55ba0b406930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:43.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 -- 172.21.15.161:0/4040147275 shutdown_connections 2023-07-14T11:34:43.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.557+0000 7fbef2048b80 1 -- 172.21.15.161:0/4040147275 wait complete. 2023-07-14T11:34:43.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 -- start start 2023-07-14T11:34:43.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 Processor -- start 2023-07-14T11:34:43.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b4157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b412a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b413140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b2fcac0 con 0x55ba0b40be00 2023-07-14T11:34:43.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b2fc7c0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.558+0000 7fbef2048b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b1e0920 con 0x55ba0b415d30 2023-07-14T11:34:43.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbef2048b80 10 client.0 _ll_register_callbacks cb 0x55ba0b2fa760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:34:43.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b4157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b4157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43464/0 (socket says 172.21.15.161:43464) 2023-07-14T11:34:43.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 -- 172.21.15.161:0/1998340636 learned_addr learned my addr 172.21.15.161:0/1998340636 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:43.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b413140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed77fe700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b412a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 -- 172.21.15.161:0/1998340636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 msgr2=0x55ba0b413140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b413140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 -- 172.21.15.161:0/1998340636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 msgr2=0x55ba0b412a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b412a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 -- 172.21.15.161:0/1998340636 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc019040 con 0x55ba0b40b4f0 2023-07-14T11:34:43.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b413140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:34:43.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbed7fff700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b4157f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbed8100b30 tx=0x7fbec800bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47c074a0398b5e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:43.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.559+0000 7fbeb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:34:43.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbeb77fe700 1 -- 172.21.15.161:0/1998340636 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800f8f0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbeb77fe700 1 -- 172.21.15.161:0/1998340636 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fbec800fa50 con 0x55ba0b40b4f0 2023-07-14T11:34:43.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba0b2b64f0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbeb77fe700 1 -- 172.21.15.161:0/1998340636 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec800fc10 con 0x55ba0b40b4f0 2023-07-14T11:34:43.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbef2048b80 10 client.27014 fetch_fsmap learned FSMap version 30 2023-07-14T11:34:43.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba0b173c60 con 0x55ba0b40b4f0 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbeb77fe700 1 -- 172.21.15.161:0/1998340636 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba0b2b64f0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbeb77fe700 1 -- 172.21.15.161:0/1998340636 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbec80173f0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbef2048b80 10 client.27014 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbef2048b80 10 client.27014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbef2048b80 20 client.27014 populate_metadata read hostname 'smithi161' 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.560+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b425770 0x55ba0b427b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.561+0000 7fbef2048b80 4 client.27014 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:34:43.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.561+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba0b355eb0 con 0x55ba0b425770 2023-07-14T11:34:43.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.561+0000 7fbed77fe700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b425770 0x55ba0b427b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.561+0000 7fbeb77fe700 10 client.27014 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:43.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.561+0000 7fbed77fe700 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b425770 0x55ba0b427b80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee96d8b228818052 server_cookie=1bc44d1afa3378f0 in_seq=0 out_seq=0 2023-07-14T11:34:43.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbeb77fe700 1 -- 172.21.15.161:0/1998340636 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba0b355eb0 con 0x55ba0b425770 2023-07-14T11:34:43.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbeb77fe700 10 client.27014 handle_command_reply: tid=0 2023-07-14T11:34:43.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 client.27014 shutdown 2023-07-14T11:34:43.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 msgr2=0x55ba0b4157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b4157f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbed8100b30 tx=0x7fbec800bba0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 shutdown_connections 2023-07-14T11:34:43.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b425770 0x55ba0b427b80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b413140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b4157f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1998340636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b412a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbeb77fe700 0 client.27014 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:43.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 >> 172.21.15.161:0/1998340636 conn(0x55ba0b400ff0 msgr2=0x55ba0b406520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:43.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 shutdown_connections 2023-07-14T11:34:43.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 1 -- 172.21.15.161:0/1998340636 wait complete. 2023-07-14T11:34:43.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.563+0000 7fbef2048b80 20 client.27014 trim_cache size 0 max 16384 2023-07-14T11:34:43.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.583+0000 7fbef2048b80 1 Processor -- start 2023-07-14T11:34:43.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.583+0000 7fbef2048b80 1 -- start start 2023-07-14T11:34:43.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.583+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b415320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.583+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b4122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b415d30 0x55ba0b4127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbef2048b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b400e00 con 0x55ba0b40be00 2023-07-14T11:34:43.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbef2048b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b407510 con 0x55ba0b415d30 2023-07-14T11:34:43.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbef2048b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b413740 con 0x55ba0b40b4f0 2023-07-14T11:34:43.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b415320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbedcd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b415d30 0x55ba0b4127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b415320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60996/0 (socket says 172.21.15.161:60996) 2023-07-14T11:34:43.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 -- 172.21.15.161:0/1270141905 learned_addr learned my addr 172.21.15.161:0/1270141905 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:43.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed77fe700 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b4122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 -- 172.21.15.161:0/1270141905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b415d30 msgr2=0x55ba0b4127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b415d30 0x55ba0b4127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 -- 172.21.15.161:0/1270141905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 msgr2=0x55ba0b4122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b4122a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 -- 172.21.15.161:0/1270141905 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba0b1b1250 con 0x55ba0b40b4f0 2023-07-14T11:34:43.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.584+0000 7fbed7fff700 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b415320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000c00 tx=0x7fbec8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef03c70705f3e454 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:43.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.585+0000 7fbebeffd700 1 -- 172.21.15.161:0/1270141905 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80175f0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.585+0000 7fbebeffd700 1 -- 172.21.15.161:0/1270141905 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec801c070 con 0x55ba0b40b4f0 2023-07-14T11:34:43.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.585+0000 7fbebeffd700 1 -- 172.21.15.161:0/1270141905 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800f6f0 con 0x55ba0b40b4f0 2023-07-14T11:34:43.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.585+0000 7fbef2048b80 1 -- 172.21.15.161:0/1270141905 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 msgr2=0x55ba0b415320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.585+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b415320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000c00 tx=0x7fbec8009310 comp rx=0 tx=0).stop 2023-07-14T11:34:43.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 -- 172.21.15.161:0/1270141905 shutdown_connections 2023-07-14T11:34:43.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b415d30 0x55ba0b4127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b40b4f0 0x55ba0b415320 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 --2- 172.21.15.161:0/1270141905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b4122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 -- 172.21.15.161:0/1270141905 >> 172.21.15.161:0/1270141905 conn(0x55ba0b400ff0 msgr2=0x55ba0b40dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:43.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 -- 172.21.15.161:0/1270141905 shutdown_connections 2023-07-14T11:34:43.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 -- 172.21.15.161:0/1270141905 wait complete. 2023-07-14T11:34:43.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 -- start start 2023-07-14T11:34:43.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.586+0000 7fbef2048b80 1 Processor -- start 2023-07-14T11:34:43.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b41ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b41bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbef2048b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b478630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbef2048b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b41c830 con 0x55ba0b40be00 2023-07-14T11:34:43.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbef2048b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b478b70 con 0x55ba0b40b4f0 2023-07-14T11:34:43.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbef2048b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b478cb0 con 0x55ba0b415d30 2023-07-14T11:34:43.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbef2048b80 10 client.0 _ll_register_callbacks cb 0x7fbecc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:34:43.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbed7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b41ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbedcd9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b478630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbed7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b41ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43492/0 (socket says 172.21.15.161:43492) 2023-07-14T11:34:43.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbed7fff700 1 -- 172.21.15.161:0/114621406 learned_addr learned my addr 172.21.15.161:0/114621406 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:34:43.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbed77fe700 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b41bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbedcd9c700 1 -- 172.21.15.161:0/114621406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 msgr2=0x55ba0b41ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b41ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbedcd9c700 1 -- 172.21.15.161:0/114621406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 msgr2=0x55ba0b41bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:34:43.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b41bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.587+0000 7fbedcd9c700 1 -- 172.21.15.161:0/114621406 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba0b173c60 con 0x55ba0b415d30 2023-07-14T11:34:43.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbedcd9c700 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b478630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbed00c9f10 tx=0x7fbed0095c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f6bacabb0378 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:34:43.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbeb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:34:43.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbeb77fe700 1 -- 172.21.15.161:0/114621406 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00a1a50 con 0x55ba0b415d30 2023-07-14T11:34:43.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbecc004830 con 0x55ba0b415d30 2023-07-14T11:34:43.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbeb77fe700 1 -- 172.21.15.161:0/114621406 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fbed00a1ce0 con 0x55ba0b415d30 2023-07-14T11:34:43.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbeb77fe700 1 -- 172.21.15.161:0/114621406 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed009dbb0 con 0x55ba0b415d30 2023-07-14T11:34:43.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbef2048b80 10 client.27022 fetch_fsmap learned FSMap version 30 2023-07-14T11:34:43.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba0b1892b0 con 0x55ba0b415d30 2023-07-14T11:34:43.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbeb77fe700 1 -- 172.21.15.161:0/114621406 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbecc004830 con 0x55ba0b415d30 2023-07-14T11:34:43.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.588+0000 7fbeb77fe700 1 -- 172.21.15.161:0/114621406 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbed001b9d0 con 0x55ba0b415d30 2023-07-14T11:34:43.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbef2048b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T11:34:43.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbef2048b80 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:34:43.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbef2048b80 20 client.27022 populate_metadata read hostname 'smithi161' 2023-07-14T11:34:43.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbef2048b80 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b47a260 0x55ba0b47c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:34:43.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbef2048b80 4 client.27022 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:34:43.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba0b318050 con 0x55ba0b47a260 2023-07-14T11:34:43.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbed7fff700 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b47a260 0x55ba0b47c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:34:43.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbed7fff700 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b47a260 0x55ba0b47c670 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2796f057aed5079 server_cookie=70d76845d8a1b23a in_seq=0 out_seq=0 2023-07-14T11:34:43.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.589+0000 7fbeb77fe700 10 client.27022 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbeb77fe700 1 -- 172.21.15.161:0/114621406 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ba0b318050 con 0x55ba0b47a260 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbeb77fe700 10 client.27022 handle_command_reply: tid=0 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 client.27022 shutdown 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 msgr2=0x55ba0b478630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b478630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbed00c9f10 tx=0x7fbed0095c60 comp rx=0 tx=0).stop 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 shutdown_connections 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba0b47a260 0x55ba0b47c670 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba0b415d30 0x55ba0b478630 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba0b40b4f0 0x55ba0b41ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 --2- 172.21.15.161:0/114621406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba0b40be00 0x55ba0b41bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 >> 172.21.15.161:0/114621406 conn(0x55ba0b400ff0 msgr2=0x55ba0b40a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:43.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbeb77fe700 0 client.27022 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:34:43.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 shutdown_connections 2023-07-14T11:34:43.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 1 -- 172.21.15.161:0/114621406 wait complete. 2023-07-14T11:34:43.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.590+0000 7fbef2048b80 20 client.27022 trim_cache size 0 max 16384 2023-07-14T11:34:43.609 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:34:43.609 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:34:43.609 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:34:43.609 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:34:43.610 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:34:43.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:34:43.610 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:34:43.610 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:34:43.610 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:34:43.610 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:34:43.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbec00a13e0 msgr2=0x7fbec00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbec00a13e0 0x7fbec00a3890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00f6a0 tx=0x7fbecc018000 comp rx=0 tx=0).stop 2023-07-14T11:34:43.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 msgr2=0x7fbed8197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:34:43.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8197970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbed000ef30 tx=0x7fbed000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 shutdown_connections 2023-07-14T11:34:43.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbec00a13e0 0x7fbec00a3890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed8105a00 0x7fbed8197970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbed80ff9f0 0x7fbed819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 --2- 172.21.15.161:0/2603693700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbed81003a0 0x7fbed819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:34:43.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 >> 172.21.15.161:0/2603693700 conn(0x7fbed80fb510 msgr2=0x7fbed8072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:34:43.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 shutdown_connections 2023-07-14T11:34:43.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:34:43.591+0000 7fbedd59d700 1 -- 172.21.15.161:0/2603693700 wait complete. 2023-07-14T11:34:43.615 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:34:44.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:43 smithi175 ceph-mon[129081]: pgmap v389: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2023-07-14T11:34:44.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:43 smithi182 ceph-mon[129366]: pgmap v389: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2023-07-14T11:34:44.227 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1126747 8.61 MB/sec execute 566 sec latency 26.480 ms 2023-07-14T11:34:44.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:43 smithi161 ceph-mon[95526]: pgmap v389: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2023-07-14T11:34:45.227 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1128408 8.61 MB/sec execute 567 sec latency 58.353 ms 2023-07-14T11:34:46.227 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1129985 8.62 MB/sec execute 568 sec latency 20.160 ms 2023-07-14T11:34:46.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:46 smithi175 ceph-mon[129081]: pgmap v390: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2023-07-14T11:34:46.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:46 smithi182 ceph-mon[129366]: pgmap v390: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2023-07-14T11:34:46.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:46 smithi161 ceph-mon[95526]: pgmap v390: 129 pgs: 129 active+clean; 849 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2023-07-14T11:34:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1131897 8.61 MB/sec execute 569 sec latency 19.580 ms 2023-07-14T11:34:48.228 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1133403 8.62 MB/sec execute 570 sec latency 157.803 ms 2023-07-14T11:34:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:48 smithi175 ceph-mon[129081]: pgmap v391: 129 pgs: 129 active+clean; 854 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 269 op/s 2023-07-14T11:34:48.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:48 smithi182 ceph-mon[129366]: pgmap v391: 129 pgs: 129 active+clean; 854 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 269 op/s 2023-07-14T11:34:48.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:48 smithi161 ceph-mon[95526]: pgmap v391: 129 pgs: 129 active+clean; 854 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 269 op/s 2023-07-14T11:34:49.228 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1134535 8.61 MB/sec execute 571 sec latency 65.137 ms 2023-07-14T11:34:49.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:49 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:34:49.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:49 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:34:49.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:49 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:34:50.230 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1136618 8.62 MB/sec execute 572 sec latency 53.060 ms 2023-07-14T11:34:50.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:50 smithi175 ceph-mon[129081]: pgmap v392: 129 pgs: 129 active+clean; 854 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 180 op/s 2023-07-14T11:34:50.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:50 smithi182 ceph-mon[129366]: pgmap v392: 129 pgs: 129 active+clean; 854 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 180 op/s 2023-07-14T11:34:50.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:50 smithi161 ceph-mon[95526]: pgmap v392: 129 pgs: 129 active+clean; 854 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 180 op/s 2023-07-14T11:34:51.230 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1137466 8.61 MB/sec execute 573 sec latency 36.059 ms 2023-07-14T11:34:52.230 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1139675 8.61 MB/sec execute 574 sec latency 16.933 ms 2023-07-14T11:34:52.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:52 smithi175 ceph-mon[129081]: pgmap v393: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2023-07-14T11:34:52.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:52 smithi182 ceph-mon[129366]: pgmap v393: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2023-07-14T11:34:52.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:52 smithi161 ceph-mon[95526]: pgmap v393: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2023-07-14T11:34:53.230 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1141009 8.61 MB/sec execute 575 sec latency 27.299 ms 2023-07-14T11:34:54.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:53 smithi182 ceph-mon[129366]: pgmap v394: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 179 op/s 2023-07-14T11:34:54.230 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1142961 8.62 MB/sec execute 576 sec latency 27.331 ms 2023-07-14T11:34:54.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:53 smithi161 ceph-mon[95526]: pgmap v394: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 179 op/s 2023-07-14T11:34:54.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:53 smithi175 ceph-mon[129081]: pgmap v394: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 179 op/s 2023-07-14T11:34:55.231 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1144178 8.61 MB/sec execute 577 sec latency 41.737 ms 2023-07-14T11:34:56.231 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1145448 8.61 MB/sec execute 578 sec latency 31.348 ms 2023-07-14T11:34:56.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:56 smithi175 ceph-mon[129081]: pgmap v395: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 179 op/s 2023-07-14T11:34:56.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:56 smithi182 ceph-mon[129366]: pgmap v395: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 179 op/s 2023-07-14T11:34:56.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:56 smithi161 ceph-mon[95526]: pgmap v395: 129 pgs: 129 active+clean; 858 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 179 op/s 2023-07-14T11:34:57.231 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1147552 8.62 MB/sec execute 579 sec latency 43.775 ms 2023-07-14T11:34:58.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1148744 8.61 MB/sec execute 580 sec latency 27.422 ms 2023-07-14T11:34:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:34:58 smithi175 ceph-mon[129081]: pgmap v396: 129 pgs: 129 active+clean; 861 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2023-07-14T11:34:58.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:34:58 smithi182 ceph-mon[129366]: pgmap v396: 129 pgs: 129 active+clean; 861 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2023-07-14T11:34:58.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:34:58 smithi161 ceph-mon[95526]: pgmap v396: 129 pgs: 129 active+clean; 861 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2023-07-14T11:34:59.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1150969 8.62 MB/sec execute 581 sec latency 24.874 ms 2023-07-14T11:35:00.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1151904 8.61 MB/sec execute 582 sec latency 38.511 ms 2023-07-14T11:35:00.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:00 smithi175 ceph-mon[129081]: pgmap v397: 129 pgs: 129 active+clean; 861 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2023-07-14T11:35:00.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:00 smithi182 ceph-mon[129366]: pgmap v397: 129 pgs: 129 active+clean; 861 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2023-07-14T11:35:00.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:00 smithi161 ceph-mon[95526]: pgmap v397: 129 pgs: 129 active+clean; 861 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2023-07-14T11:35:01.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1153108 8.60 MB/sec execute 583 sec latency 59.720 ms 2023-07-14T11:35:02.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1154192 8.60 MB/sec execute 584 sec latency 66.422 ms 2023-07-14T11:35:02.314 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:02 smithi175 ceph-mon[129081]: pgmap v398: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2023-07-14T11:35:02.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:02 smithi182 ceph-mon[129366]: pgmap v398: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2023-07-14T11:35:02.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:02 smithi161 ceph-mon[95526]: pgmap v398: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2023-07-14T11:35:03.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1155174 8.60 MB/sec execute 585 sec latency 42.145 ms 2023-07-14T11:35:04.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:03 smithi182 ceph-mon[129366]: pgmap v399: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2023-07-14T11:35:04.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1157102 8.60 MB/sec execute 586 sec latency 17.527 ms 2023-07-14T11:35:04.247 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:03 smithi161 ceph-mon[95526]: pgmap v399: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2023-07-14T11:35:04.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:03 smithi175 ceph-mon[129081]: pgmap v399: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2023-07-14T11:35:05.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1158364 8.60 MB/sec execute 587 sec latency 64.004 ms 2023-07-14T11:35:06.232 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1159586 8.59 MB/sec execute 588 sec latency 56.184 ms 2023-07-14T11:35:06.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:06 smithi175 ceph-mon[129081]: pgmap v400: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2023-07-14T11:35:06.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:06 smithi182 ceph-mon[129366]: pgmap v400: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2023-07-14T11:35:06.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:06 smithi161 ceph-mon[95526]: pgmap v400: 129 pgs: 129 active+clean; 862 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2023-07-14T11:35:07.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1161611 8.60 MB/sec execute 589 sec latency 66.469 ms 2023-07-14T11:35:07.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:07 smithi175 ceph-mon[129081]: osdmap e106: 12 total, 12 up, 12 in 2023-07-14T11:35:07.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:07 smithi182 ceph-mon[129366]: osdmap e106: 12 total, 12 up, 12 in 2023-07-14T11:35:07.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:07 smithi161 ceph-mon[95526]: osdmap e106: 12 total, 12 up, 12 in 2023-07-14T11:35:08.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1162533 8.59 MB/sec execute 590 sec latency 15.442 ms 2023-07-14T11:35:08.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:08 smithi175 ceph-mon[129081]: pgmap v402: 129 pgs: 129 active+clean; 864 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-07-14T11:35:08.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:08 smithi182 ceph-mon[129366]: pgmap v402: 129 pgs: 129 active+clean; 864 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-07-14T11:35:08.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:08 smithi161 ceph-mon[95526]: pgmap v402: 129 pgs: 129 active+clean; 864 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-07-14T11:35:09.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1164488 8.59 MB/sec execute 591 sec latency 18.606 ms 2023-07-14T11:35:10.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1165460 8.59 MB/sec execute 592 sec latency 69.927 ms 2023-07-14T11:35:10.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:10 smithi175 ceph-mon[129081]: pgmap v403: 129 pgs: 129 active+clean; 864 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-07-14T11:35:10.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:10 smithi182 ceph-mon[129366]: pgmap v403: 129 pgs: 129 active+clean; 864 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-07-14T11:35:10.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:10 smithi161 ceph-mon[95526]: pgmap v403: 129 pgs: 129 active+clean; 864 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-07-14T11:35:11.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1166547 8.58 MB/sec execute 593 sec latency 53.761 ms 2023-07-14T11:35:12.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1168695 8.59 MB/sec execute 594 sec latency 44.218 ms 2023-07-14T11:35:12.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:12 smithi175 ceph-mon[129081]: pgmap v404: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 213 op/s 2023-07-14T11:35:12.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:12 smithi182 ceph-mon[129366]: pgmap v404: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 213 op/s 2023-07-14T11:35:12.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:12 smithi161 ceph-mon[95526]: pgmap v404: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 213 op/s 2023-07-14T11:35:13.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1169755 8.58 MB/sec execute 595 sec latency 21.680 ms 2023-07-14T11:35:13.616 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:35:13.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 -- 172.21.15.161:0/3155263278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe718105ae0 msgr2=0x7fe718105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3155263278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe718105ae0 0x7fe718105eb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe714009990 tx=0x7fe714009ca0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 -- 172.21.15.161:0/3155263278 shutdown_connections 2023-07-14T11:35:13.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3155263278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe718105ae0 0x7fe718105eb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3155263278 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe718100480 0x7fe71810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3155263278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7180ffad0 0x7fe7180fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 -- 172.21.15.161:0/3155263278 >> 172.21.15.161:0/3155263278 conn(0x7fe7180fb5f0 msgr2=0x7fe7180fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:13.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 -- 172.21.15.161:0/3155263278 shutdown_connections 2023-07-14T11:35:13.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.837+0000 7fe71ffff700 1 -- 172.21.15.161:0/3155263278 wait complete. 2023-07-14T11:35:13.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.838+0000 7fe71ffff700 1 Processor -- start 2023-07-14T11:35:13.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.839+0000 7fe71ffff700 1 -- start start 2023-07-14T11:35:13.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.839+0000 7fe71ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7180ffad0 0x7fe718072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.839+0000 7fe71ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe718100480 0x7fe718072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.839+0000 7fe71ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe718105ae0 0x7fe7181a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.839+0000 7fe71ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe718073580 con 0x7fe7180ffad0 2023-07-14T11:35:13.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.839+0000 7fe71ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7181a42d0 con 0x7fe718105ae0 2023-07-14T11:35:13.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.839+0000 7fe71ffff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7181a4410 con 0x7fe718100480 2023-07-14T11:35:13.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71d59a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe718100480 0x7fe718072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe718105ae0 0x7fe7181a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71dd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7180ffad0 0x7fe718072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe718105ae0 0x7fe7181a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33628/0 (socket says 172.21.15.161:33628) 2023-07-14T11:35:13.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 -- 172.21.15.161:0/3685259816 learned_addr learned my addr 172.21.15.161:0/3685259816 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:13.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 -- 172.21.15.161:0/3685259816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe718100480 msgr2=0x7fe718072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe718100480 0x7fe718072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 -- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7180ffad0 msgr2=0x7fe718072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7180ffad0 0x7fe718072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 -- 172.21.15.161:0/3685259816 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710009c40 con 0x7fe718105ae0 2023-07-14T11:35:13.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71d59a700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe718100480 0x7fe718072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:35:13.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.840+0000 7fe71e59c700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe718105ae0 0x7fe7181a3d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c370 tx=0x7fe71000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:13.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.841+0000 7fe70effd700 1 -- 172.21.15.161:0/3685259816 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710017070 con 0x7fe718105ae0 2023-07-14T11:35:13.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.841+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7140095f0 con 0x7fe718105ae0 2023-07-14T11:35:13.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.841+0000 7fe70effd700 1 -- 172.21.15.161:0/3685259816 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe71000f040 con 0x7fe718105ae0 2023-07-14T11:35:13.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.841+0000 7fe70effd700 1 -- 172.21.15.161:0/3685259816 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000e050 con 0x7fe718105ae0 2023-07-14T11:35:13.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.841+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7181a4960 con 0x7fe718105ae0 2023-07-14T11:35:13.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.845+0000 7fe70effd700 1 -- 172.21.15.161:0/3685259816 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe71000cb60 con 0x7fe718105ae0 2023-07-14T11:35:13.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.846+0000 7fe733989b80 1 Processor -- start 2023-07-14T11:35:13.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe733989b80 1 -- start start 2023-07-14T11:35:13.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe733989b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac82d0 0x562080ac86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe733989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac8be0 0x562080ad3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe733989b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe733989b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620809cecb0 con 0x562080ad3ad0 2023-07-14T11:35:13.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe733989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080a21a40 con 0x562080ac8be0 2023-07-14T11:35:13.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe733989b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080a1aed0 con 0x562080ac82d0 2023-07-14T11:35:13.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe70effd700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe7040a1340 0x7fe7040a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe71dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac8be0 0x562080ad3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe71dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac8be0 0x562080ad3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33630/0 (socket says 172.21.15.161:33630) 2023-07-14T11:35:13.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe71dd9b700 1 -- 172.21.15.161:0/2329740113 learned_addr learned my addr 172.21.15.161:0/2329740113 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:13.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe71d59a700 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac82d0 0x562080ac86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe71e59c700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe7040a1340 0x7fe7040a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.847+0000 7fe71d59a700 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe70effd700 1 -- 172.21.15.161:0/3685259816 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fe7100cc920 con 0x7fe718105ae0 2023-07-14T11:35:13.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71dd9b700 1 -- 172.21.15.161:0/2329740113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac82d0 msgr2=0x562080ac86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac82d0 0x562080ac86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71dd9b700 1 -- 172.21.15.161:0/2329740113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 msgr2=0x562080ad5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71dd9b700 1 -- 172.21.15.161:0/2329740113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562080848de0 con 0x562080ac8be0 2023-07-14T11:35:13.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71d59a700 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:35:13.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac8be0 0x562080ad3590 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe714005ea0 tx=0x7fe714005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f116eeee6d0cccb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:13.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe71e59c700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe7040a1340 0x7fe7040a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c9d0 tx=0x7fe710009630 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:13.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe703fff700 1 -- 172.21.15.161:0/2329740113 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71401d070 con 0x562080ac8be0 2023-07-14T11:35:13.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe703fff700 1 -- 172.21.15.161:0/2329740113 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe71401e040 con 0x562080ac8be0 2023-07-14T11:35:13.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.848+0000 7fe703fff700 1 -- 172.21.15.161:0/2329740113 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714004880 con 0x562080ac8be0 2023-07-14T11:35:13.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 -- 172.21.15.161:0/2329740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac8be0 msgr2=0x562080ad3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac8be0 0x562080ad3590 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe714005ea0 tx=0x7fe714005ed0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 -- 172.21.15.161:0/2329740113 shutdown_connections 2023-07-14T11:35:13.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac82d0 0x562080ac86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 --2- 172.21.15.161:0/2329740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac8be0 0x562080ad3590 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 -- 172.21.15.161:0/2329740113 >> 172.21.15.161:0/2329740113 conn(0x562080abeda0 msgr2=0x562080ac46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:13.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 -- 172.21.15.161:0/2329740113 shutdown_connections 2023-07-14T11:35:13.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.849+0000 7fe733989b80 1 -- 172.21.15.161:0/2329740113 wait complete. 2023-07-14T11:35:13.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.850+0000 7fe733989b80 1 -- start start 2023-07-14T11:35:13.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.850+0000 7fe733989b80 1 Processor -- start 2023-07-14T11:35:13.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.850+0000 7fe733989b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac82d0 0x562080ad3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe733989b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe733989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ad3ad0 0x562080ad0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe733989b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620809baac0 con 0x562080ac82d0 2023-07-14T11:35:13.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe733989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620809ba7c0 con 0x562080ad3ad0 2023-07-14T11:35:13.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe733989b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56208089e920 con 0x562080ac8be0 2023-07-14T11:35:13.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe733989b80 10 client.0 _ll_register_callbacks cb 0x5620809b8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:35:13.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33472/0 (socket says 172.21.15.161:33472) 2023-07-14T11:35:13.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 -- 172.21.15.161:0/1932340985 learned_addr learned my addr 172.21.15.161:0/1932340985 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:13.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71d59a700 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac82d0 0x562080ad3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71d59a700 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ad3ad0 0x562080ad0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 -- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ad3ad0 msgr2=0x562080ad0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ad3ad0 0x562080ad0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 -- 172.21.15.161:0/1932340985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac82d0 msgr2=0x562080ad3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac82d0 0x562080ad3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.851+0000 7fe71dd9b700 1 -- 172.21.15.161:0/1932340985 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71810d210 con 0x562080ac8be0 2023-07-14T11:35:13.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad06e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe714005820 tx=0x7fe71400fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121b734147ea29a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:13.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe7017fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:35:13.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe7017fa700 1 -- 172.21.15.161:0/1932340985 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71401d040 con 0x562080ac8be0 2023-07-14T11:35:13.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620809744f0 con 0x562080ac8be0 2023-07-14T11:35:13.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe7017fa700 1 -- 172.21.15.161:0/1932340985 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fe714005980 con 0x562080ac8be0 2023-07-14T11:35:13.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe7017fa700 1 -- 172.21.15.161:0/1932340985 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7140055a0 con 0x562080ac8be0 2023-07-14T11:35:13.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe733989b80 10 client.27034 fetch_fsmap learned FSMap version 31 2023-07-14T11:35:13.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562080831c60 con 0x562080ac8be0 2023-07-14T11:35:13.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.852+0000 7fe7017fa700 1 -- 172.21.15.161:0/1932340985 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620809744f0 con 0x562080ac8be0 2023-07-14T11:35:13.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe7017fa700 1 -- 172.21.15.161:0/1932340985 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe714022400 con 0x562080ac8be0 2023-07-14T11:35:13.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe733989b80 10 client.27034 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T11:35:13.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe733989b80 10 client.27034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:35:13.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe733989b80 20 client.27034 populate_metadata read hostname 'smithi161' 2023-07-14T11:35:13.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe733989b80 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080ae2dd0 0x562080ae51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe733989b80 4 client.27034 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:35:13.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562080a13eb0 con 0x562080ae2dd0 2023-07-14T11:35:13.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe71d59a700 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080ae2dd0 0x562080ae51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe71d59a700 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080ae2dd0 0x562080ae51e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=304f959eb09251e server_cookie=31bf1602298018e7 in_seq=0 out_seq=0 2023-07-14T11:35:13.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.853+0000 7fe7017fa700 10 client.27034 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:13.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe7017fa700 1 -- 172.21.15.161:0/1932340985 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562080a13eb0 con 0x562080ae2dd0 2023-07-14T11:35:13.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe7017fa700 10 client.27034 handle_command_reply: tid=0 2023-07-14T11:35:13.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 client.27034 shutdown 2023-07-14T11:35:13.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 msgr2=0x562080ad06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad06e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe714005820 tx=0x7fe71400fc00 comp rx=0 tx=0).stop 2023-07-14T11:35:13.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 shutdown_connections 2023-07-14T11:35:13.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080ae2dd0 0x562080ae51e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac82d0 0x562080ad3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ad3ad0 0x562080ad0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 --2- 172.21.15.161:0/1932340985 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad06e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe7017fa700 0 client.27034 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:13.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 >> 172.21.15.161:0/1932340985 conn(0x562080abeda0 msgr2=0x562080ac4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:13.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 shutdown_connections 2023-07-14T11:35:13.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.856+0000 7fe733989b80 1 -- 172.21.15.161:0/1932340985 wait complete. 2023-07-14T11:35:13.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.857+0000 7fe733989b80 20 client.27034 trim_cache size 0 max 16384 2023-07-14T11:35:13.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.875+0000 7fe733989b80 1 Processor -- start 2023-07-14T11:35:13.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe733989b80 1 -- start start 2023-07-14T11:35:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe733989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ad3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe733989b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac8be0 0x562080acfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe733989b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ad3ad0 0x562080ad03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe733989b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080ac9540 con 0x562080ac8be0 2023-07-14T11:35:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe733989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080ac9160 con 0x562080ac82d0 2023-07-14T11:35:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe733989b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080ac9dd0 con 0x562080ad3ad0 2023-07-14T11:35:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe71d59a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ad3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe71d59a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ad3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:33654/0 (socket says 172.21.15.161:33654) 2023-07-14T11:35:13.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe71d59a700 1 -- 172.21.15.161:0/2159379961 learned_addr learned my addr 172.21.15.161:0/2159379961 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac8be0 0x562080acfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.876+0000 7fe71d59a700 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ad3ad0 0x562080ad03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe71d59a700 1 -- 172.21.15.161:0/2159379961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ad3ad0 msgr2=0x562080ad03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe71d59a700 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ad3ad0 0x562080ad03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe71d59a700 1 -- 172.21.15.161:0/2159379961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac8be0 msgr2=0x562080acfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe71d59a700 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac8be0 0x562080acfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe71d59a700 1 -- 172.21.15.161:0/2159379961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56208086f250 con 0x562080ac82d0 2023-07-14T11:35:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe71d59a700 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ad3000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x562080ae5af0 tx=0x7fe708013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67e5ed55b0eb0e08 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe70cff9700 1 -- 172.21.15.161:0/2159379961 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708003880 con 0x562080ac82d0 2023-07-14T11:35:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe70cff9700 1 -- 172.21.15.161:0/2159379961 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe708003ea0 con 0x562080ac82d0 2023-07-14T11:35:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.877+0000 7fe70cff9700 1 -- 172.21.15.161:0/2159379961 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708005020 con 0x562080ac82d0 2023-07-14T11:35:13.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 -- 172.21.15.161:0/2159379961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 msgr2=0x562080ad3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ad3000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x562080ae5af0 tx=0x7fe708013040 comp rx=0 tx=0).stop 2023-07-14T11:35:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 -- 172.21.15.161:0/2159379961 shutdown_connections 2023-07-14T11:35:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ad3ad0 0x562080ad03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ad3000 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 --2- 172.21.15.161:0/2159379961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ac8be0 0x562080acfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 -- 172.21.15.161:0/2159379961 >> 172.21.15.161:0/2159379961 conn(0x562080abeda0 msgr2=0x562080adf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 -- 172.21.15.161:0/2159379961 shutdown_connections 2023-07-14T11:35:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.878+0000 7fe733989b80 1 -- 172.21.15.161:0/2159379961 wait complete. 2023-07-14T11:35:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 -- start start 2023-07-14T11:35:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 Processor -- start 2023-07-14T11:35:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ada2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080ad9040 con 0x562080ad3ad0 2023-07-14T11:35:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080ad91b0 con 0x562080ac82d0 2023-07-14T11:35:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080ad9370 con 0x562080ac8be0 2023-07-14T11:35:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.879+0000 7fe733989b80 10 client.0 _ll_register_callbacks cb 0x7fe708009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:35:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33482/0 (socket says 172.21.15.161:33482) 2023-07-14T11:35:13.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 -- 172.21.15.161:0/3424009721 learned_addr learned my addr 172.21.15.161:0/3424009721 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:13.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71d59a700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ada2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71d59a700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 -- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 msgr2=0x562080ada2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ada2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 -- 172.21.15.161:0/3424009721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 msgr2=0x562080ad8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:13.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 -- 172.21.15.161:0/3424009721 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70802c040 con 0x562080ac8be0 2023-07-14T11:35:13.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71d59a700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe71dd9b700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad85c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe714000c00 tx=0x7fe71400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be0a10b9957ad3e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.880+0000 7fe7017fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe7017fa700 1 -- 172.21.15.161:0/3424009721 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71401d030 con 0x562080ac8be0 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe71401e040 con 0x562080ac8be0 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe7017fa700 1 -- 172.21.15.161:0/3424009721 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fe714022a80 con 0x562080ac8be0 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe7017fa700 1 -- 172.21.15.161:0/3424009721 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe714022400 con 0x562080ac8be0 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 10 client.27040 fetch_fsmap learned FSMap version 31 2023-07-14T11:35:13.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5620808472b0 con 0x562080ac8be0 2023-07-14T11:35:13.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe7017fa700 1 -- 172.21.15.161:0/3424009721 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe71401e040 con 0x562080ac8be0 2023-07-14T11:35:13.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe7017fa700 1 -- 172.21.15.161:0/3424009721 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe71400b850 con 0x562080ac8be0 2023-07-14T11:35:13.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 10 client.27040 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T11:35:13.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 10 client.27040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:35:13.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 20 client.27040 populate_metadata read hostname 'smithi161' 2023-07-14T11:35:13.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080b37c60 0x562080b3a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:13.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 4 client.27040 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:35:13.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.881+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620809d6050 con 0x562080b37c60 2023-07-14T11:35:13.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.882+0000 7fe71d59a700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080b37c60 0x562080b3a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:13.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.882+0000 7fe7017fa700 10 client.27040 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:13.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.882+0000 7fe71d59a700 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080b37c60 0x562080b3a040 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c6d89c6c550781 server_cookie=e2b135ae35b548f6 in_seq=0 out_seq=0 2023-07-14T11:35:13.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.883+0000 7fe7017fa700 1 -- 172.21.15.161:0/3424009721 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5620809d6050 con 0x562080b37c60 2023-07-14T11:35:13.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.883+0000 7fe7017fa700 10 client.27040 handle_command_reply: tid=0 2023-07-14T11:35:13.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.883+0000 7fe733989b80 1 client.27040 shutdown 2023-07-14T11:35:13.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.883+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 msgr2=0x562080ad85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.883+0000 7fe733989b80 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad85c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe714000c00 tx=0x7fe71400b610 comp rx=0 tx=0).stop 2023-07-14T11:35:13.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 shutdown_connections 2023-07-14T11:35:13.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562080b37c60 0x562080b3a040 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562080ad3ad0 0x562080ad8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562080ac82d0 0x562080ada2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 --2- 172.21.15.161:0/3424009721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562080ac8be0 0x562080ad85c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe7017fa700 0 client.27040 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:13.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 >> 172.21.15.161:0/3424009721 conn(0x562080abeda0 msgr2=0x562080ac38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:13.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 shutdown_connections 2023-07-14T11:35:13.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 1 -- 172.21.15.161:0/3424009721 wait complete. 2023-07-14T11:35:13.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.884+0000 7fe733989b80 20 client.27040 trim_cache size 0 max 16384 2023-07-14T11:35:13.899 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:35:13.899 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:35:13.899 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:35:13.899 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:35:13.900 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:35:13.900 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:35:13.900 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:35:13.900 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:35:13.900 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:35:13.900 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:35:13.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe7040a1340 msgr2=0x7fe7040a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe7040a1340 0x7fe7040a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c9d0 tx=0x7fe710009630 comp rx=0 tx=0).stop 2023-07-14T11:35:13.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe718105ae0 msgr2=0x7fe7181a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:13.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe718105ae0 0x7fe7181a3d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c370 tx=0x7fe71000c680 comp rx=0 tx=0).stop 2023-07-14T11:35:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 shutdown_connections 2023-07-14T11:35:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe7040a1340 0x7fe7040a37f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7180ffad0 0x7fe718072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe718105ae0 0x7fe7181a3d90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 --2- 172.21.15.161:0/3685259816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe718100480 0x7fe718072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 >> 172.21.15.161:0/3685259816 conn(0x7fe7180fb5f0 msgr2=0x7fe7180fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 shutdown_connections 2023-07-14T11:35:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:13.885+0000 7fe71ffff700 1 -- 172.21.15.161:0/3685259816 wait complete. 2023-07-14T11:35:13.909 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:35:13.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:13 smithi161 ceph-mon[95526]: osdmap e107: 12 total, 12 up, 12 in 2023-07-14T11:35:13.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:13 smithi161 ceph-mon[95526]: pgmap v406: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 267 op/s 2023-07-14T11:35:14.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:13 smithi182 ceph-mon[129366]: osdmap e107: 12 total, 12 up, 12 in 2023-07-14T11:35:14.218 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:13 smithi182 ceph-mon[129366]: pgmap v406: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 267 op/s 2023-07-14T11:35:14.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1171953 8.59 MB/sec execute 596 sec latency 28.943 ms 2023-07-14T11:35:14.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:13 smithi175 ceph-mon[129081]: osdmap e107: 12 total, 12 up, 12 in 2023-07-14T11:35:14.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:13 smithi175 ceph-mon[129081]: pgmap v406: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 267 op/s 2023-07-14T11:35:15.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1172807 8.58 MB/sec execute 597 sec latency 66.941 ms 2023-07-14T11:35:16.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1174088 8.58 MB/sec execute 598 sec latency 42.415 ms 2023-07-14T11:35:16.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:16 smithi175 ceph-mon[129081]: pgmap v407: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2023-07-14T11:35:16.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:16 smithi182 ceph-mon[129366]: pgmap v407: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2023-07-14T11:35:16.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:16 smithi161 ceph-mon[95526]: pgmap v407: 129 pgs: 129 active+clean; 867 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2023-07-14T11:35:17.233 INFO:tasks.workunit.client.0.smithi161.stdout: 1 1176037 8.59 MB/sec execute 599 sec latency 19.285 ms 2023-07-14T11:35:18.234 INFO:tasks.workunit.client.0.smithi161.stdout: 1 cleanup 600 sec 2023-07-14T11:35:18.424 INFO:tasks.workunit.client.0.smithi161.stdout: 0 cleanup 600 sec 2023-07-14T11:35:18.424 INFO:tasks.workunit.client.0.smithi161.stdout: 2023-07-14T11:35:18.424 INFO:tasks.workunit.client.0.smithi161.stdout: Operation Count AvgLat MaxLat 2023-07-14T11:35:18.424 INFO:tasks.workunit.client.0.smithi161.stdout: ---------------------------------------- 2023-07-14T11:35:18.424 INFO:tasks.workunit.client.0.smithi161.stdout: NTCreateX 164059 1.460 5155.119 2023-07-14T11:35:18.425 INFO:tasks.workunit.client.0.smithi161.stdout: Close 120525 0.004 0.199 2023-07-14T11:35:18.425 INFO:tasks.workunit.client.0.smithi161.stdout: Rename 6945 4.885 93.943 2023-07-14T11:35:18.425 INFO:tasks.workunit.client.0.smithi161.stdout: Unlink 33123 1.887 4925.531 2023-07-14T11:35:18.425 INFO:tasks.workunit.client.0.smithi161.stdout: Deltree 4 84.392 104.124 2023-07-14T11:35:18.425 INFO:tasks.workunit.client.0.smithi161.stdout: Mkdir 2 0.015 0.015 2023-07-14T11:35:18.425 INFO:tasks.workunit.client.0.smithi161.stdout: Qpathinfo 148710 1.152 4921.683 2023-07-14T11:35:18.425 INFO:tasks.workunit.client.0.smithi161.stdout: Qfileinfo 26057 0.002 0.057 2023-07-14T11:35:18.426 INFO:tasks.workunit.client.0.smithi161.stdout: Qfsinfo 27254 0.253 11.410 2023-07-14T11:35:18.426 INFO:tasks.workunit.client.0.smithi161.stdout: Sfileinfo 13358 0.024 4.408 2023-07-14T11:35:18.426 INFO:tasks.workunit.client.0.smithi161.stdout: Find 57480 0.236 84.809 2023-07-14T11:35:18.426 INFO:tasks.workunit.client.0.smithi161.stdout: WriteX 81760 0.132 134.762 2023-07-14T11:35:18.426 INFO:tasks.workunit.client.0.smithi161.stdout: ReadX 257102 0.009 39.324 2023-07-14T11:35:18.426 INFO:tasks.workunit.client.0.smithi161.stdout: LockX 534 0.673 7.553 2023-07-14T11:35:18.426 INFO:tasks.workunit.client.0.smithi161.stdout: UnlockX 534 0.601 9.882 2023-07-14T11:35:18.427 INFO:tasks.workunit.client.0.smithi161.stdout: Flush 11496 4.681 230.682 2023-07-14T11:35:18.427 INFO:tasks.workunit.client.0.smithi161.stdout: 2023-07-14T11:35:18.427 INFO:tasks.workunit.client.0.smithi161.stdout:Throughput 8.58534 MB/sec 1 clients 1 procs max_latency=5155.128 ms 2023-07-14T11:35:18.427 INFO:tasks.workunit.client.0.smithi161.stderr:+ dbench 10 2023-07-14T11:35:18.428 INFO:tasks.workunit.client.0.smithi161.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-14T11:35:18.428 INFO:tasks.workunit.client.0.smithi161.stdout: 2023-07-14T11:35:18.428 INFO:tasks.workunit.client.0.smithi161.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-14T11:35:18.428 INFO:tasks.workunit.client.0.smithi161.stdout:9 of 10 processes prepared for launch 0 sec 2023-07-14T11:35:18.529 INFO:tasks.workunit.client.0.smithi161.stdout:10 of 10 processes prepared for launch 0 sec 2023-07-14T11:35:18.529 INFO:tasks.workunit.client.0.smithi161.stdout:releasing clients 2023-07-14T11:35:18.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:18 smithi175 ceph-mon[129081]: pgmap v408: 129 pgs: 129 active+clean; 870 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2023-07-14T11:35:18.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:18 smithi182 ceph-mon[129366]: pgmap v408: 129 pgs: 129 active+clean; 870 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2023-07-14T11:35:18.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:18 smithi161 ceph-mon[95526]: pgmap v408: 129 pgs: 129 active+clean; 870 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2023-07-14T11:35:19.529 INFO:tasks.workunit.client.0.smithi161.stdout: 10 388 137.99 MB/sec warmup 1 sec latency 48.251 ms 2023-07-14T11:35:20.535 INFO:tasks.workunit.client.0.smithi161.stdout: 10 695 103.10 MB/sec warmup 2 sec latency 430.670 ms 2023-07-14T11:35:20.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:20 smithi175 ceph-mon[129081]: pgmap v409: 129 pgs: 129 active+clean; 870 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2023-07-14T11:35:20.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:20 smithi182 ceph-mon[129366]: pgmap v409: 129 pgs: 129 active+clean; 870 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2023-07-14T11:35:20.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:20 smithi161 ceph-mon[95526]: pgmap v409: 129 pgs: 129 active+clean; 870 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2023-07-14T11:35:21.535 INFO:tasks.workunit.client.0.smithi161.stdout: 10 844 70.31 MB/sec warmup 3 sec latency 760.619 ms 2023-07-14T11:35:21.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:21 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:21.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:21 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:21.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:21 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:22.535 INFO:tasks.workunit.client.0.smithi161.stdout: 10 1564 64.31 MB/sec warmup 4 sec latency 107.136 ms 2023-07-14T11:35:22.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:22 smithi175 ceph-mon[129081]: pgmap v410: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 315 op/s 2023-07-14T11:35:22.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:22 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:35:22.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:22 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:35:22.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:22 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:22.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:22 smithi182 ceph-mon[129366]: pgmap v410: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 315 op/s 2023-07-14T11:35:22.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:22 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:35:22.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:22 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:35:22.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:22 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:22.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:22 smithi161 ceph-mon[95526]: pgmap v410: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 315 op/s 2023-07-14T11:35:22.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:22 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:35:22.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:22 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:35:22.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:22 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:23.535 INFO:tasks.workunit.client.0.smithi161.stdout: 10 2621 60.38 MB/sec warmup 5 sec latency 144.385 ms 2023-07-14T11:35:23.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:23 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:23.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:23 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:23.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:23 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:24.535 INFO:tasks.workunit.client.0.smithi161.stdout: 10 3372 58.89 MB/sec warmup 6 sec latency 119.852 ms 2023-07-14T11:35:24.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:24 smithi175 ceph-mon[129081]: pgmap v411: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 305 op/s 2023-07-14T11:35:24.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:24 smithi182 ceph-mon[129366]: pgmap v411: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 305 op/s 2023-07-14T11:35:24.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:24 smithi161 ceph-mon[95526]: pgmap v411: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 305 op/s 2023-07-14T11:35:25.536 INFO:tasks.workunit.client.0.smithi161.stdout: 10 4186 58.74 MB/sec warmup 7 sec latency 133.243 ms 2023-07-14T11:35:26.536 INFO:tasks.workunit.client.0.smithi161.stdout: 10 4407 52.88 MB/sec warmup 8 sec latency 230.164 ms 2023-07-14T11:35:26.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:26 smithi175 ceph-mon[129081]: pgmap v412: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 262 op/s 2023-07-14T11:35:26.921 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:26 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:26.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:26 smithi182 ceph-mon[129366]: pgmap v412: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 262 op/s 2023-07-14T11:35:26.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:26 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:26.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:26 smithi161 ceph-mon[95526]: pgmap v412: 129 pgs: 129 active+clean; 990 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 262 op/s 2023-07-14T11:35:26.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:26 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:27.536 INFO:tasks.workunit.client.0.smithi161.stdout: 10 4921 49.26 MB/sec warmup 9 sec latency 49.550 ms 2023-07-14T11:35:27.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:27 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:27.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:27 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:27.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:27 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:28.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 5511 46.78 MB/sec warmup 10 sec latency 71.148 ms 2023-07-14T11:35:28.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:28 smithi175 ceph-mon[129081]: pgmap v413: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 24 MiB/s wr, 504 op/s 2023-07-14T11:35:28.922 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:28 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:28.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:28 smithi182 ceph-mon[129366]: pgmap v413: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 24 MiB/s wr, 504 op/s 2023-07-14T11:35:28.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:28 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:28.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:28 smithi161 ceph-mon[95526]: pgmap v413: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 24 MiB/s wr, 504 op/s 2023-07-14T11:35:28.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:28 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:35:29.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 6193 46.14 MB/sec warmup 11 sec latency 74.392 ms 2023-07-14T11:35:30.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 7247 45.75 MB/sec warmup 12 sec latency 103.036 ms 2023-07-14T11:35:30.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:30 smithi175 ceph-mon[129081]: pgmap v414: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 23 MiB/s wr, 411 op/s 2023-07-14T11:35:30.965 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:30 smithi182 ceph-mon[129366]: pgmap v414: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 23 MiB/s wr, 411 op/s 2023-07-14T11:35:30.997 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:30 smithi161 ceph-mon[95526]: pgmap v414: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 23 MiB/s wr, 411 op/s 2023-07-14T11:35:31.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 7914 45.82 MB/sec warmup 13 sec latency 254.970 ms 2023-07-14T11:35:31.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:31 smithi175 ceph-mon[129081]: pgmap v415: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 26 MiB/s wr, 592 op/s 2023-07-14T11:35:31.966 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:31 smithi182 ceph-mon[129366]: pgmap v415: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 26 MiB/s wr, 592 op/s 2023-07-14T11:35:31.998 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:31 smithi161 ceph-mon[95526]: pgmap v415: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 26 MiB/s wr, 592 op/s 2023-07-14T11:35:32.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 8744 46.76 MB/sec warmup 14 sec latency 100.706 ms 2023-07-14T11:35:33.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 9353 45.73 MB/sec warmup 15 sec latency 40.843 ms 2023-07-14T11:35:34.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:33 smithi182 ceph-mon[129366]: pgmap v416: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 421 op/s 2023-07-14T11:35:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:33 smithi161 ceph-mon[95526]: pgmap v416: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 421 op/s 2023-07-14T11:35:34.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:33 smithi175 ceph-mon[129081]: pgmap v416: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 421 op/s 2023-07-14T11:35:34.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 10077 44.77 MB/sec warmup 16 sec latency 59.151 ms 2023-07-14T11:35:35.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 10851 44.30 MB/sec warmup 17 sec latency 75.281 ms 2023-07-14T11:35:36.539 INFO:tasks.workunit.client.0.smithi161.stdout: 10 11893 44.88 MB/sec warmup 18 sec latency 79.111 ms 2023-07-14T11:35:36.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:36 smithi175 ceph-mon[129081]: pgmap v417: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 421 op/s 2023-07-14T11:35:36.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:36 smithi182 ceph-mon[129366]: pgmap v417: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 421 op/s 2023-07-14T11:35:36.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:36 smithi161 ceph-mon[95526]: pgmap v417: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 421 op/s 2023-07-14T11:35:37.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 12568 44.71 MB/sec warmup 19 sec latency 221.217 ms 2023-07-14T11:35:38.538 INFO:tasks.workunit.client.0.smithi161.stdout: 10 13573 45.62 MB/sec warmup 20 sec latency 94.389 ms 2023-07-14T11:35:38.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:38 smithi175 ceph-mon[129081]: pgmap v418: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 628 op/s 2023-07-14T11:35:38.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:38 smithi182 ceph-mon[129366]: pgmap v418: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 628 op/s 2023-07-14T11:35:38.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:38 smithi161 ceph-mon[95526]: pgmap v418: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 628 op/s 2023-07-14T11:35:39.540 INFO:tasks.workunit.client.0.smithi161.stdout: 10 14183 44.79 MB/sec warmup 21 sec latency 76.218 ms 2023-07-14T11:35:40.541 INFO:tasks.workunit.client.0.smithi161.stdout: 10 14987 44.51 MB/sec warmup 22 sec latency 49.217 ms 2023-07-14T11:35:40.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:40 smithi175 ceph-mon[129081]: pgmap v419: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 386 op/s 2023-07-14T11:35:40.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:40 smithi182 ceph-mon[129366]: pgmap v419: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 386 op/s 2023-07-14T11:35:40.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:40 smithi161 ceph-mon[95526]: pgmap v419: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 386 op/s 2023-07-14T11:35:41.540 INFO:tasks.workunit.client.0.smithi161.stdout: 10 15790 44.70 MB/sec warmup 23 sec latency 68.063 ms 2023-07-14T11:35:42.546 INFO:tasks.workunit.client.0.smithi161.stdout: 10 16575 43.98 MB/sec warmup 24 sec latency 63.738 ms 2023-07-14T11:35:42.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:42 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:35:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:42 smithi175 ceph-mon[129081]: pgmap v420: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.7 MiB/s wr, 604 op/s 2023-07-14T11:35:42.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:42 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:35:42.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:42 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:35:42.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:42 smithi182 ceph-mon[129366]: pgmap v420: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.7 MiB/s wr, 604 op/s 2023-07-14T11:35:42.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:42 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:35:42.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:42 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:35:42.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:42 smithi161 ceph-mon[95526]: pgmap v420: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.7 MiB/s wr, 604 op/s 2023-07-14T11:35:42.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:42 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:35:43.541 INFO:tasks.workunit.client.0.smithi161.stdout: 10 17521 44.56 MB/sec warmup 25 sec latency 74.656 ms 2023-07-14T11:35:43.910 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:35:44.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 -- 172.21.15.161:0/2881146984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8102910 msgr2=0x7f8fa81068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 --2- 172.21.15.161:0/2881146984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8102910 0x7f8fa81068d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0099c0 tx=0x7f8f9c009cd0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 -- 172.21.15.161:0/2881146984 shutdown_connections 2023-07-14T11:35:44.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 --2- 172.21.15.161:0/2881146984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fa8101f70 0x7f8fa8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 --2- 172.21.15.161:0/2881146984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fa8106fb0 0x7f8fa810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 --2- 172.21.15.161:0/2881146984 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8102910 0x7f8fa81068d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 -- 172.21.15.161:0/2881146984 >> 172.21.15.161:0/2881146984 conn(0x7f8fa80fd7e0 msgr2=0x7f8fa80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:44.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 -- 172.21.15.161:0/2881146984 shutdown_connections 2023-07-14T11:35:44.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.172+0000 7f8facbff700 1 -- 172.21.15.161:0/2881146984 wait complete. 2023-07-14T11:35:44.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.173+0000 7f8facbff700 1 Processor -- start 2023-07-14T11:35:44.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.173+0000 7f8facbff700 1 -- start start 2023-07-14T11:35:44.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8facbff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8101f70 0x7f8fa8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8facbff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fa8102910 0x7f8fa8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8facbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fa8106fb0 0x7f8fa819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8facbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa819cdf0 con 0x7f8fa8106fb0 2023-07-14T11:35:44.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8facbff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa819cf60 con 0x7f8fa8102910 2023-07-14T11:35:44.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8facbff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa819d120 con 0x7f8fa8101f70 2023-07-14T11:35:44.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8fa659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8101f70 0x7f8fa8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8fa659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8101f70 0x7f8fa8197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60390/0 (socket says 172.21.15.161:60390) 2023-07-14T11:35:44.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8fa659c700 1 -- 172.21.15.161:0/1819629441 learned_addr learned my addr 172.21.15.161:0/1819629441 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:44.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fa8106fb0 0x7f8fa819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.174+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fa8102910 0x7f8fa8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8fa659c700 1 -- 172.21.15.161:0/1819629441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fa8102910 msgr2=0x7f8fa8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8fa659c700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fa8102910 0x7f8fa8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8fa659c700 1 -- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fa8106fb0 msgr2=0x7f8fa819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8fa659c700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fa8106fb0 0x7f8fa819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8fa659c700 1 -- 172.21.15.161:0/1819629441 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c0095f0 con 0x7f8fa8101f70 2023-07-14T11:35:44.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8fa659c700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8101f70 0x7f8fa8197e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800e9a0 tx=0x7f8f9800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8f977fe700 1 -- 172.21.15.161:0/1819629441 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98009790 con 0x7f8fa8101f70 2023-07-14T11:35:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8f977fe700 1 -- 172.21.15.161:0/1819629441 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f9800cbd0 con 0x7f8fa8101f70 2023-07-14T11:35:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8f977fe700 1 -- 172.21.15.161:0/1819629441 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98018680 con 0x7f8fa8101f70 2023-07-14T11:35:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa819d400 con 0x7f8fa8101f70 2023-07-14T11:35:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.175+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa81a4c40 con 0x7f8fa8101f70 2023-07-14T11:35:44.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.180+0000 7f8f977fe700 1 -- 172.21.15.161:0/1819629441 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8f9800cd40 con 0x7f8fa8101f70 2023-07-14T11:35:44.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.181+0000 7f8fc052db80 1 Processor -- start 2023-07-14T11:35:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.182+0000 7f8fc052db80 1 -- start start 2023-07-14T11:35:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8f977fe700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8f900a13c0 0x7f8f900a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c691be0 0x55cc4c69c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c69eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fc052db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c597cb0 con 0x55cc4c691be0 2023-07-14T11:35:44.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fc052db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c5eaa40 con 0x55cc4c69cad0 2023-07-14T11:35:44.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fc052db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c5e3ed0 con 0x55cc4c6912d0 2023-07-14T11:35:44.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8f977fe700 1 -- 172.21.15.161:0/1819629441 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8f98014070 con 0x7f8fa8101f70 2023-07-14T11:35:44.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8f900a13c0 0x7f8f900a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fa5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c691be0 0x55cc4c69c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fa5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c691be0 0x55cc4c69c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55456/0 (socket says 172.21.15.161:55456) 2023-07-14T11:35:44.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fa6d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c69eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/2508165936 learned_addr learned my addr 172.21.15.161:0/2508165936 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.183+0000 7f8fa659c700 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/2508165936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 msgr2=0x55cc4c6916a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6916a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/2508165936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 msgr2=0x55cc4c69eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c69eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/2508165936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc4c411de0 con 0x55cc4c691be0 2023-07-14T11:35:44.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c69eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:35:44.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8f900a13c0 0x7f8f900a3870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007750 tx=0x7f8fa0007b10 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:44.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c691be0 0x55cc4c69c590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009990 tx=0x7f8f9c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b8ceb2a733d98f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:44.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8f94ff9700 1 -- 172.21.15.161:0/2508165936 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00b990 con 0x55cc4c691be0 2023-07-14T11:35:44.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8f94ff9700 1 -- 172.21.15.161:0/2508165936 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f9c01e040 con 0x55cc4c691be0 2023-07-14T11:35:44.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.184+0000 7f8f94ff9700 1 -- 172.21.15.161:0/2508165936 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00fd80 con 0x55cc4c691be0 2023-07-14T11:35:44.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.186+0000 7f8fc052db80 1 -- 172.21.15.161:0/2508165936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c691be0 msgr2=0x55cc4c69c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.186+0000 7f8fc052db80 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c691be0 0x55cc4c69c590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009990 tx=0x7f8f9c005f00 comp rx=0 tx=0).stop 2023-07-14T11:35:44.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.186+0000 7f8fc052db80 1 -- 172.21.15.161:0/2508165936 shutdown_connections 2023-07-14T11:35:44.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.186+0000 7f8fc052db80 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c69eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.186+0000 7f8fc052db80 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.186+0000 7f8fc052db80 1 --2- 172.21.15.161:0/2508165936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c691be0 0x55cc4c69c590 secure :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009990 tx=0x7f8f9c005f00 comp rx=0 tx=0).stop 2023-07-14T11:35:44.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.186+0000 7f8fc052db80 1 -- 172.21.15.161:0/2508165936 >> 172.21.15.161:0/2508165936 conn(0x55cc4c687da0 msgr2=0x55cc4c68d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:44.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.187+0000 7f8fc052db80 1 -- 172.21.15.161:0/2508165936 shutdown_connections 2023-07-14T11:35:44.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.187+0000 7f8fc052db80 1 -- 172.21.15.161:0/2508165936 wait complete. 2023-07-14T11:35:44.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.188+0000 7f8fc052db80 1 -- start start 2023-07-14T11:35:44.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.188+0000 7f8fc052db80 1 Processor -- start 2023-07-14T11:35:44.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.188+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c6912d0 0x55cc4c699980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c69cad0 0x55cc4c699ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c6a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fc052db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c583ac0 con 0x55cc4c6912d0 2023-07-14T11:35:44.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fc052db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c5837c0 con 0x55cc4c69a400 2023-07-14T11:35:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fc052db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c467920 con 0x55cc4c69cad0 2023-07-14T11:35:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fc052db80 10 client.0 _ll_register_callbacks cb 0x55cc4c581760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:35:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c69cad0 0x55cc4c699ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c6912d0 0x55cc4c699980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c69cad0 0x55cc4c699ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60402/0 (socket says 172.21.15.161:60402) 2023-07-14T11:35:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa6d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c6a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/1457446573 learned_addr learned my addr 172.21.15.161:0/1457446573 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c6912d0 0x55cc4c699980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55472/0 (socket says 172.21.15.161:55472) 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/1457446573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 msgr2=0x55cc4c6a85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c6a85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/1457446573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c6912d0 msgr2=0x55cc4c699980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c6912d0 0x55cc4c699980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.189+0000 7f8fa5d9b700 1 -- 172.21.15.161:0/1457446573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa810f3c0 con 0x55cc4c69cad0 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c69cad0 0x55cc4c699ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005b20 tx=0x7f8f9c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2561cb83d969c827 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:44.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8f8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:35:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8f8dffb700 1 -- 172.21.15.161:0/1457446573 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00b8b0 con 0x55cc4c69cad0 2023-07-14T11:35:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc4c53d4f0 con 0x55cc4c69cad0 2023-07-14T11:35:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8f8dffb700 1 -- 172.21.15.161:0/1457446573 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8f9c00ba10 con 0x55cc4c69cad0 2023-07-14T11:35:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8f8dffb700 1 -- 172.21.15.161:0/1457446573 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f9c00bbd0 con 0x55cc4c69cad0 2023-07-14T11:35:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8fc052db80 10 client.27052 fetch_fsmap learned FSMap version 37 2023-07-14T11:35:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.190+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cc4c3fac60 con 0x55cc4c69cad0 2023-07-14T11:35:44.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8f8dffb700 1 -- 172.21.15.161:0/1457446573 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc4c53d4f0 con 0x55cc4c69cad0 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8f8dffb700 1 -- 172.21.15.161:0/1457446573 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8f9c021070 con 0x55cc4c69cad0 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8fc052db80 10 client.27052 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8fc052db80 10 client.27052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8fc052db80 20 client.27052 populate_metadata read hostname 'smithi161' 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8fc052db80 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c6ac500 0x55cc4c6ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8fc052db80 4 client.27052 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc4c5dceb0 con 0x55cc4c6ac500 2023-07-14T11:35:44.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.191+0000 7f8fa659c700 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c6ac500 0x55cc4c6ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.192+0000 7f8fa659c700 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c6ac500 0x55cc4c6ae8e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11c84ab2a248ce1 server_cookie=16e5d2e7b95c0834 in_seq=0 out_seq=0 2023-07-14T11:35:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.192+0000 7f8f8dffb700 10 client.27052 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8f8dffb700 1 -- 172.21.15.161:0/1457446573 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cc4c5dceb0 con 0x55cc4c6ac500 2023-07-14T11:35:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8f8dffb700 10 client.27052 handle_command_reply: tid=0 2023-07-14T11:35:44.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 client.27052 shutdown 2023-07-14T11:35:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c69cad0 msgr2=0x55cc4c699ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c69cad0 0x55cc4c699ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005b20 tx=0x7f8f9c009cd0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 shutdown_connections 2023-07-14T11:35:44.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c6ac500 0x55cc4c6ae8e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c6912d0 0x55cc4c699980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c6a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8f8dffb700 0 client.27052 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 --2- 172.21.15.161:0/1457446573 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c69cad0 0x55cc4c699ec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 >> 172.21.15.161:0/1457446573 conn(0x55cc4c687da0 msgr2=0x55cc4c68d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 shutdown_connections 2023-07-14T11:35:44.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 1 -- 172.21.15.161:0/1457446573 wait complete. 2023-07-14T11:35:44.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.194+0000 7f8fc052db80 20 client.27052 trim_cache size 0 max 16384 2023-07-14T11:35:44.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:43 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:35:44.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:43 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:35:44.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:43 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:35:44.215 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:43 smithi182 ceph-mon[129366]: pgmap v421: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-07-14T11:35:44.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 Processor -- start 2023-07-14T11:35:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 -- start start 2023-07-14T11:35:44.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69a400 0x55cc4c6a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c696ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c692540 con 0x55cc4c69a400 2023-07-14T11:35:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c69af60 con 0x55cc4c69cad0 2023-07-14T11:35:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.218+0000 7f8fc052db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c68dbf0 con 0x55cc4c6912d0 2023-07-14T11:35:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c696ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c696ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:50986/0 (socket says 172.21.15.161:50986) 2023-07-14T11:35:44.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 -- 172.21.15.161:0/181408177 learned_addr learned my addr 172.21.15.161:0/181408177 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:44.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa659c700 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69a400 0x55cc4c6a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 -- 172.21.15.161:0/181408177 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 msgr2=0x55cc4c6a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 -- 172.21.15.161:0/181408177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69a400 msgr2=0x55cc4c6a34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69a400 0x55cc4c6a34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 -- 172.21.15.161:0/181408177 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc4c438250 con 0x55cc4c69cad0 2023-07-14T11:35:44.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69a400 0x55cc4c6a34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:35:44.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa659c700 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:35:44.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.219+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c696ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009fb0 tx=0x7f8fa001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915637a8a169b0d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:44.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.220+0000 7f8f957fa700 1 -- 172.21.15.161:0/181408177 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000e050 con 0x55cc4c69cad0 2023-07-14T11:35:44.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.220+0000 7f8f957fa700 1 -- 172.21.15.161:0/181408177 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa000f040 con 0x55cc4c69cad0 2023-07-14T11:35:44.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.220+0000 7f8f957fa700 1 -- 172.21.15.161:0/181408177 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0017690 con 0x55cc4c69cad0 2023-07-14T11:35:44.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.221+0000 7f8fc052db80 1 -- 172.21.15.161:0/181408177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 msgr2=0x55cc4c696ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.221+0000 7f8fc052db80 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c696ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009fb0 tx=0x7f8fa001d000 comp rx=0 tx=0).stop 2023-07-14T11:35:44.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.221+0000 7f8fc052db80 1 -- 172.21.15.161:0/181408177 shutdown_connections 2023-07-14T11:35:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.221+0000 7f8fc052db80 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69cad0 0x55cc4c696ca0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.221+0000 7f8fc052db80 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c6a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.221+0000 7f8fc052db80 1 --2- 172.21.15.161:0/181408177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69a400 0x55cc4c6a34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.221+0000 7f8fc052db80 1 -- 172.21.15.161:0/181408177 >> 172.21.15.161:0/181408177 conn(0x55cc4c687da0 msgr2=0x55cc4c6a8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.222+0000 7f8fc052db80 1 -- 172.21.15.161:0/181408177 shutdown_connections 2023-07-14T11:35:44.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.222+0000 7f8fc052db80 1 -- 172.21.15.161:0/181408177 wait complete. 2023-07-14T11:35:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 -- start start 2023-07-14T11:35:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 Processor -- start 2023-07-14T11:35:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c702450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c702990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69cad0 0x55cc4c6fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c6fd1e0 con 0x55cc4c69cad0 2023-07-14T11:35:44.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c6fd350 con 0x55cc4c69a400 2023-07-14T11:35:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.223+0000 7f8fc052db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4c6fd510 con 0x55cc4c6912d0 2023-07-14T11:35:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fc052db80 10 client.0 _ll_register_callbacks cb 0x7f8f98010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:35:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c702450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa6d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69cad0 0x55cc4c6fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c702450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60426/0 (socket says 172.21.15.161:60426) 2023-07-14T11:35:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa659c700 1 -- 172.21.15.161:0/446739799 learned_addr learned my addr 172.21.15.161:0/446739799 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:35:44.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c702990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa6d9d700 1 -- 172.21.15.161:0/446739799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 msgr2=0x55cc4c702450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c702450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa6d9d700 1 -- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 msgr2=0x55cc4c702990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:35:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c702990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.224+0000 7f8fa6d9d700 1 -- 172.21.15.161:0/446739799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa0021040 con 0x55cc4c69cad0 2023-07-14T11:35:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8fa5d9b700 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c702990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:35:44.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8fa6d9d700 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69cad0 0x55cc4c6fcb00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009690 tx=0x7f8fa00259f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17a50d1a4b9f86b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8f8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8f8dffb700 1 -- 172.21.15.161:0/446739799 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0025a90 con 0x55cc4c69cad0 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f9c01e040 con 0x55cc4c69cad0 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8f8dffb700 1 -- 172.21.15.161:0/446739799 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8fa0025630 con 0x55cc4c69cad0 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8f8dffb700 1 -- 172.21.15.161:0/446739799 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fa00257f0 con 0x55cc4c69cad0 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.225+0000 7f8fc052db80 10 client.16998 fetch_fsmap learned FSMap version 37 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cc4c4102b0 con 0x55cc4c69cad0 2023-07-14T11:35:44.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8f8dffb700 1 -- 172.21.15.161:0/446739799 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f9c01e040 con 0x55cc4c69cad0 2023-07-14T11:35:44.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8f8dffb700 1 -- 172.21.15.161:0/446739799 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8fa0017430 con 0x55cc4c69cad0 2023-07-14T11:35:44.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fc052db80 10 client.16998 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:35:44.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fc052db80 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:35:44.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fc052db80 20 client.16998 populate_metadata read hostname 'smithi161' 2023-07-14T11:35:44.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fc052db80 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c70a080 0x55cc4c70c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:35:44.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fc052db80 4 client.16998 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:35:44.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc4c59f050 con 0x55cc4c70a080 2023-07-14T11:35:44.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.226+0000 7f8fa659c700 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c70a080 0x55cc4c70c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:35:44.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.227+0000 7f8f8dffb700 10 client.16998 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:44.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.227+0000 7f8fa659c700 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c70a080 0x55cc4c70c490 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa58c6ecaba83cb server_cookie=972cca669e875bbd in_seq=0 out_seq=0 2023-07-14T11:35:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.230+0000 7f8f8dffb700 1 -- 172.21.15.161:0/446739799 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cc4c59f050 con 0x55cc4c70a080 2023-07-14T11:35:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.230+0000 7f8f8dffb700 10 client.16998 handle_command_reply: tid=0 2023-07-14T11:35:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.230+0000 7f8fc052db80 1 client.16998 shutdown 2023-07-14T11:35:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69cad0 msgr2=0x55cc4c6fcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69cad0 0x55cc4c6fcb00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009690 tx=0x7f8fa00259f0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 shutdown_connections 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cc4c70a080 0x55cc4c70c490 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc4c69cad0 0x55cc4c6fcb00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cc4c6912d0 0x55cc4c702450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 --2- 172.21.15.161:0/446739799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4c69a400 0x55cc4c702990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 >> 172.21.15.161:0/446739799 conn(0x55cc4c687da0 msgr2=0x55cc4c68c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8f8dffb700 0 client.16998 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 shutdown_connections 2023-07-14T11:35:44.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 1 -- 172.21.15.161:0/446739799 wait complete. 2023-07-14T11:35:44.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.231+0000 7f8fc052db80 20 client.16998 trim_cache size 0 max 16384 2023-07-14T11:35:44.237 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:35:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:35:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:35:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bac55f28-3d3a-46c0-b971-0c29560579dc": { 2023-07-14T11:35:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:35:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "bac55f28-3d3a-46c0-b971-0c29560579dc", 2023-07-14T11:35:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:35:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:35:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:35:44.238 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:35:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8f900a13c0 msgr2=0x7f8f900a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8f900a13c0 0x7f8f900a3870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007750 tx=0x7f8fa0007b10 comp rx=0 tx=0).stop 2023-07-14T11:35:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8101f70 msgr2=0x7f8fa8197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:35:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8101f70 0x7f8fa8197e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800e9a0 tx=0x7f8f9800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 shutdown_connections 2023-07-14T11:35:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8f900a13c0 0x7f8f900a3870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fa8106fb0 0x7f8fa819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8fa8101f70 0x7f8fa8197e10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 --2- 172.21.15.161:0/1819629441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fa8102910 0x7f8fa8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:35:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 >> 172.21.15.161:0/1819629441 conn(0x7f8fa80fd7e0 msgr2=0x7f8fa80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:35:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 shutdown_connections 2023-07-14T11:35:44.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:35:44.232+0000 7f8facbff700 1 -- 172.21.15.161:0/1819629441 wait complete. 2023-07-14T11:35:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:43 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:35:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:43 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:35:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:43 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:35:44.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:43 smithi161 ceph-mon[95526]: pgmap v421: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-07-14T11:35:44.258 INFO:tasks.cephfs.filesystem:scrub status for tag:bac55f28-3d3a-46c0-b971-0c29560579dc - {'path': '/', 'tag': 'bac55f28-3d3a-46c0-b971-0c29560579dc', 'options': 'recursive,force'} 2023-07-14T11:35:44.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:43 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:35:44.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:43 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:35:44.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:43 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:35:44.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:43 smithi175 ceph-mon[129081]: pgmap v421: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-07-14T11:35:44.541 INFO:tasks.workunit.client.0.smithi161.stdout: 10 18404 44.91 MB/sec warmup 26 sec latency 63.067 ms 2023-07-14T11:35:45.541 INFO:tasks.workunit.client.0.smithi161.stdout: 10 19167 44.89 MB/sec warmup 27 sec latency 107.500 ms 2023-07-14T11:35:46.542 INFO:tasks.workunit.client.0.smithi161.stdout: 10 19779 44.39 MB/sec warmup 28 sec latency 152.684 ms 2023-07-14T11:35:46.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:46 smithi175 ceph-mon[129081]: pgmap v422: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-07-14T11:35:46.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:46 smithi182 ceph-mon[129366]: pgmap v422: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-07-14T11:35:46.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:46 smithi161 ceph-mon[95526]: pgmap v422: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-07-14T11:35:47.542 INFO:tasks.workunit.client.0.smithi161.stdout: 10 20593 43.88 MB/sec warmup 29 sec latency 58.636 ms 2023-07-14T11:35:48.542 INFO:tasks.workunit.client.0.smithi161.stdout: 10 21505 44.23 MB/sec warmup 30 sec latency 74.791 ms 2023-07-14T11:35:48.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:48 smithi175 ceph-mon[129081]: pgmap v423: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 634 op/s 2023-07-14T11:35:48.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:48 smithi182 ceph-mon[129366]: pgmap v423: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 634 op/s 2023-07-14T11:35:48.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:48 smithi161 ceph-mon[95526]: pgmap v423: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 634 op/s 2023-07-14T11:35:49.542 INFO:tasks.workunit.client.0.smithi161.stdout: 10 22539 44.91 MB/sec warmup 31 sec latency 222.739 ms 2023-07-14T11:35:50.543 INFO:tasks.workunit.client.0.smithi161.stdout: 10 23131 44.64 MB/sec warmup 32 sec latency 48.155 ms 2023-07-14T11:35:50.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:50 smithi175 ceph-mon[129081]: pgmap v424: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.8 MiB/s wr, 427 op/s 2023-07-14T11:35:50.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:50 smithi182 ceph-mon[129366]: pgmap v424: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.8 MiB/s wr, 427 op/s 2023-07-14T11:35:50.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:50 smithi161 ceph-mon[95526]: pgmap v424: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.8 MiB/s wr, 427 op/s 2023-07-14T11:35:51.544 INFO:tasks.workunit.client.0.smithi161.stdout: 10 23701 43.92 MB/sec warmup 33 sec latency 104.766 ms 2023-07-14T11:35:52.546 INFO:tasks.workunit.client.0.smithi161.stdout: 10 23907 42.96 MB/sec warmup 34 sec latency 683.420 ms 2023-07-14T11:35:52.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:52 smithi175 ceph-mon[129081]: pgmap v425: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.5 MiB/s wr, 631 op/s 2023-07-14T11:35:52.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:52 smithi182 ceph-mon[129366]: pgmap v425: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.5 MiB/s wr, 631 op/s 2023-07-14T11:35:52.758 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:52 smithi161 ceph-mon[95526]: pgmap v425: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.5 MiB/s wr, 631 op/s 2023-07-14T11:35:53.545 INFO:tasks.workunit.client.0.smithi161.stdout: 10 24045 41.96 MB/sec warmup 35 sec latency 1273.564 ms 2023-07-14T11:35:54.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:53 smithi161 ceph-mon[95526]: pgmap v426: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 413 op/s 2023-07-14T11:35:54.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:53 smithi175 ceph-mon[129081]: pgmap v426: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 413 op/s 2023-07-14T11:35:54.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:53 smithi182 ceph-mon[129366]: pgmap v426: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 413 op/s 2023-07-14T11:35:54.555 INFO:tasks.workunit.client.0.smithi161.stdout: 10 24524 41.52 MB/sec warmup 36 sec latency 243.277 ms 2023-07-14T11:35:55.556 INFO:tasks.workunit.client.0.smithi161.stdout: 10 25131 41.11 MB/sec warmup 37 sec latency 79.140 ms 2023-07-14T11:35:56.556 INFO:tasks.workunit.client.0.smithi161.stdout: 10 25785 41.02 MB/sec warmup 38 sec latency 96.654 ms 2023-07-14T11:35:56.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:56 smithi175 ceph-mon[129081]: pgmap v427: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 413 op/s 2023-07-14T11:35:56.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:56 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:35:56.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:56 smithi182 ceph-mon[129366]: pgmap v427: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 413 op/s 2023-07-14T11:35:56.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:56 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:35:56.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:56 smithi161 ceph-mon[95526]: pgmap v427: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 413 op/s 2023-07-14T11:35:56.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:56 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:35:57.557 INFO:tasks.workunit.client.0.smithi161.stdout: 10 26572 41.23 MB/sec warmup 39 sec latency 75.176 ms 2023-07-14T11:35:57.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:57 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:35:57.717 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:57 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:35:57.749 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:57 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:35:58.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 26637 40.24 MB/sec warmup 40 sec latency 581.113 ms 2023-07-14T11:35:58.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:35:58 smithi175 ceph-mon[129081]: pgmap v428: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 865 op/s 2023-07-14T11:35:58.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:35:58 smithi182 ceph-mon[129366]: pgmap v428: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 865 op/s 2023-07-14T11:35:58.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:35:58 smithi161 ceph-mon[95526]: pgmap v428: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 865 op/s 2023-07-14T11:35:59.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 27001 39.73 MB/sec warmup 41 sec latency 220.146 ms 2023-07-14T11:36:00.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 27338 39.27 MB/sec warmup 42 sec latency 83.059 ms 2023-07-14T11:36:00.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:00 smithi175 ceph-mon[129081]: pgmap v429: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 15 MiB/s wr, 655 op/s 2023-07-14T11:36:00.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:00 smithi182 ceph-mon[129366]: pgmap v429: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 15 MiB/s wr, 655 op/s 2023-07-14T11:36:00.750 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:00 smithi161 ceph-mon[95526]: pgmap v429: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 15 MiB/s wr, 655 op/s 2023-07-14T11:36:01.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 27778 38.70 MB/sec warmup 43 sec latency 199.223 ms 2023-07-14T11:36:02.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 27922 37.94 MB/sec warmup 44 sec latency 284.101 ms 2023-07-14T11:36:02.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:02 smithi175 ceph-mon[129081]: pgmap v430: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 24 MiB/s wr, 1.04k op/s 2023-07-14T11:36:02.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:02 smithi182 ceph-mon[129366]: pgmap v430: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 24 MiB/s wr, 1.04k op/s 2023-07-14T11:36:02.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:02 smithi161 ceph-mon[95526]: pgmap v430: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 24 MiB/s wr, 1.04k op/s 2023-07-14T11:36:03.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 27931 37.17 MB/sec warmup 45 sec latency 376.316 ms 2023-07-14T11:36:04.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:03 smithi161 ceph-mon[95526]: pgmap v431: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 834 op/s 2023-07-14T11:36:04.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:03 smithi175 ceph-mon[129081]: pgmap v431: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 834 op/s 2023-07-14T11:36:04.464 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:03 smithi182 ceph-mon[129366]: pgmap v431: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 834 op/s 2023-07-14T11:36:04.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 28219 36.82 MB/sec warmup 46 sec latency 252.316 ms 2023-07-14T11:36:05.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 28900 36.79 MB/sec warmup 47 sec latency 273.754 ms 2023-07-14T11:36:06.558 INFO:tasks.workunit.client.0.smithi161.stdout: 10 29755 36.90 MB/sec warmup 48 sec latency 78.748 ms 2023-07-14T11:36:06.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:06 smithi175 ceph-mon[129081]: pgmap v432: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 834 op/s 2023-07-14T11:36:06.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:06 smithi182 ceph-mon[129366]: pgmap v432: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 834 op/s 2023-07-14T11:36:06.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:06 smithi161 ceph-mon[95526]: pgmap v432: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 834 op/s 2023-07-14T11:36:07.561 INFO:tasks.workunit.client.0.smithi161.stdout: 10 30532 37.24 MB/sec warmup 49 sec latency 79.407 ms 2023-07-14T11:36:07.674 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:07 smithi175 ceph-mon[129081]: osdmap e108: 12 total, 12 up, 12 in 2023-07-14T11:36:07.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:07 smithi182 ceph-mon[129366]: osdmap e108: 12 total, 12 up, 12 in 2023-07-14T11:36:07.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:07 smithi161 ceph-mon[95526]: osdmap e108: 12 total, 12 up, 12 in 2023-07-14T11:36:08.559 INFO:tasks.workunit.client.0.smithi161.stdout: 10 31285 37.16 MB/sec warmup 50 sec latency 37.389 ms 2023-07-14T11:36:08.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:08 smithi175 ceph-mon[129081]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 830 op/s 2023-07-14T11:36:08.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:08 smithi182 ceph-mon[129366]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 830 op/s 2023-07-14T11:36:08.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:08 smithi161 ceph-mon[95526]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 830 op/s 2023-07-14T11:36:09.559 INFO:tasks.workunit.client.0.smithi161.stdout: 10 32127 37.26 MB/sec warmup 51 sec latency 42.834 ms 2023-07-14T11:36:10.559 INFO:tasks.workunit.client.0.smithi161.stdout: 10 32982 37.55 MB/sec warmup 52 sec latency 48.320 ms 2023-07-14T11:36:10.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:10 smithi175 ceph-mon[129081]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 830 op/s 2023-07-14T11:36:10.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:10 smithi182 ceph-mon[129366]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 830 op/s 2023-07-14T11:36:10.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:10 smithi161 ceph-mon[95526]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 830 op/s 2023-07-14T11:36:11.559 INFO:tasks.workunit.client.0.smithi161.stdout: 10 33605 37.44 MB/sec warmup 53 sec latency 66.502 ms 2023-07-14T11:36:12.559 INFO:tasks.workunit.client.0.smithi161.stdout: 10 34434 37.49 MB/sec warmup 54 sec latency 63.424 ms 2023-07-14T11:36:12.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:12 smithi175 ceph-mon[129081]: pgmap v436: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 622 op/s 2023-07-14T11:36:12.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:12 smithi182 ceph-mon[129366]: pgmap v436: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 622 op/s 2023-07-14T11:36:12.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:12 smithi161 ceph-mon[95526]: pgmap v436: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 622 op/s 2023-07-14T11:36:13.559 INFO:tasks.workunit.client.0.smithi161.stdout: 10 35094 37.47 MB/sec warmup 55 sec latency 243.773 ms 2023-07-14T11:36:14.213 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:13 smithi182 ceph-mon[129366]: osdmap e109: 12 total, 12 up, 12 in 2023-07-14T11:36:14.214 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:13 smithi182 ceph-mon[129366]: pgmap v438: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.8 MiB/s wr, 778 op/s 2023-07-14T11:36:14.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:13 smithi161 ceph-mon[95526]: osdmap e109: 12 total, 12 up, 12 in 2023-07-14T11:36:14.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:13 smithi161 ceph-mon[95526]: pgmap v438: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.8 MiB/s wr, 778 op/s 2023-07-14T11:36:14.259 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:36:14.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:13 smithi175 ceph-mon[129081]: osdmap e109: 12 total, 12 up, 12 in 2023-07-14T11:36:14.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:13 smithi175 ceph-mon[129081]: pgmap v438: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.8 MiB/s wr, 778 op/s 2023-07-14T11:36:14.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 -- 172.21.15.161:0/1110263402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 msgr2=0x7f402810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 --2- 172.21.15.161:0/1110263402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 0x7f402810c910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f402000b0a0 tx=0x7f402000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 -- 172.21.15.161:0/1110263402 shutdown_connections 2023-07-14T11:36:14.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 --2- 172.21.15.161:0/1110263402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 0x7f402810c910 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 --2- 172.21.15.161:0/1110263402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4028105ae0 0x7f4028105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 --2- 172.21.15.161:0/1110263402 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f40280ffad0 0x7f40280fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 -- 172.21.15.161:0/1110263402 >> 172.21.15.161:0/1110263402 conn(0x7f40280fb5f0 msgr2=0x7f40280fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:14.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 -- 172.21.15.161:0/1110263402 shutdown_connections 2023-07-14T11:36:14.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.516+0000 7f402ffff700 1 -- 172.21.15.161:0/1110263402 wait complete. 2023-07-14T11:36:14.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.517+0000 7f402ffff700 1 Processor -- start 2023-07-14T11:36:14.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.517+0000 7f402ffff700 1 -- start start 2023-07-14T11:36:14.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffad0 0x7f402819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 0x7f402819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028105ae0 0x7f4028195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028196120 con 0x7f4028100480 2023-07-14T11:36:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028196290 con 0x7f40280ffad0 2023-07-14T11:36:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402ffff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028196450 con 0x7f4028105ae0 2023-07-14T11:36:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffad0 0x7f402819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffad0 0x7f402819ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44976/0 (socket says 172.21.15.161:44976) 2023-07-14T11:36:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402dd9b700 1 -- 172.21.15.161:0/1282123140 learned_addr learned my addr 172.21.15.161:0/1282123140 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402e59c700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028105ae0 0x7f4028195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402d59a700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 0x7f402819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402e59c700 1 -- 172.21.15.161:0/1282123140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffad0 msgr2=0x7f402819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402e59c700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffad0 0x7f402819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402e59c700 1 -- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 msgr2=0x7f402819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402e59c700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 0x7f402819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.518+0000 7f402e59c700 1 -- 172.21.15.161:0/1282123140 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020009d00 con 0x7f4028105ae0 2023-07-14T11:36:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.519+0000 7f402e59c700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028105ae0 0x7f4028195ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f402000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.519+0000 7f401effd700 1 -- 172.21.15.161:0/1282123140 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000e040 con 0x7f4028105ae0 2023-07-14T11:36:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.519+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40281966d0 con 0x7f4028105ae0 2023-07-14T11:36:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.519+0000 7f401effd700 1 -- 172.21.15.161:0/1282123140 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40200094c0 con 0x7f4028105ae0 2023-07-14T11:36:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.519+0000 7f401effd700 1 -- 172.21.15.161:0/1282123140 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020005040 con 0x7f4028105ae0 2023-07-14T11:36:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.519+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40281a2a30 con 0x7f4028105ae0 2023-07-14T11:36:14.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.523+0000 7f401effd700 1 -- 172.21.15.161:0/1282123140 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f402001d470 con 0x7f4028105ae0 2023-07-14T11:36:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.525+0000 7f40439c2b80 1 Processor -- start 2023-07-14T11:36:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.525+0000 7f401effd700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40140a1340 0x7f40140a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.525+0000 7f40439c2b80 1 -- start start 2023-07-14T11:36:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.525+0000 7f401effd700 1 -- 172.21.15.161:0/1282123140 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f40200ce680 con 0x7f4028105ae0 2023-07-14T11:36:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.525+0000 7f402d59a700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40140a1340 0x7f40140a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978b5be0 0x55ef978c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978c4ee0 0x55ef978c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f40439c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef977bbcb0 con 0x55ef978b5be0 2023-07-14T11:36:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f40439c2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9780ea40 con 0x55ef978c4ee0 2023-07-14T11:36:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f40439c2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef97807ed0 con 0x55ef978b52d0 2023-07-14T11:36:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41428/0 (socket says 172.21.15.161:41428) 2023-07-14T11:36:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 -- 172.21.15.161:0/1091723678 learned_addr learned my addr 172.21.15.161:0/1091723678 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402d59a700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40140a1340 0x7f40140a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4028105110 tx=0x7f4018009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402d59a700 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978b5be0 0x55ef978c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402e59c700 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978c4ee0 0x55ef978c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 -- 172.21.15.161:0/1091723678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978c4ee0 msgr2=0x55ef978c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978c4ee0 0x55ef978c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 -- 172.21.15.161:0/1091723678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978b5be0 msgr2=0x55ef978c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978b5be0 0x55ef978c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.526+0000 7f402dd9b700 1 -- 172.21.15.161:0/1091723678 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef97635de0 con 0x55ef978b52d0 2023-07-14T11:36:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.527+0000 7f402dd9b700 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978b56a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f402400add0 tx=0x7f402400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acaa18c473da749a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.527+0000 7f4013fff700 1 -- 172.21.15.161:0/1091723678 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400a8a0 con 0x55ef978b52d0 2023-07-14T11:36:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.527+0000 7f4013fff700 1 -- 172.21.15.161:0/1091723678 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f402400fe60 con 0x55ef978b52d0 2023-07-14T11:36:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.527+0000 7f4013fff700 1 -- 172.21.15.161:0/1091723678 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024010a30 con 0x55ef978b52d0 2023-07-14T11:36:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 -- 172.21.15.161:0/1091723678 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 msgr2=0x55ef978b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978b56a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f402400add0 tx=0x7f402400f6a0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 -- 172.21.15.161:0/1091723678 shutdown_connections 2023-07-14T11:36:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978c4ee0 0x55ef978c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978b56a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1091723678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978b5be0 0x55ef978c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 -- 172.21.15.161:0/1091723678 >> 172.21.15.161:0/1091723678 conn(0x55ef978abda0 msgr2=0x55ef978b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 -- 172.21.15.161:0/1091723678 shutdown_connections 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.528+0000 7f40439c2b80 1 -- 172.21.15.161:0/1091723678 wait complete. 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.529+0000 7f40439c2b80 1 -- start start 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.529+0000 7f40439c2b80 1 Processor -- start 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f40439c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef977a7ac0 con 0x55ef978c4ee0 2023-07-14T11:36:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f40439c2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef977a77c0 con 0x55ef978b5be0 2023-07-14T11:36:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f40439c2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9768b920 con 0x55ef978b52d0 2023-07-14T11:36:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978bfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41440/0 (socket says 172.21.15.161:41440) 2023-07-14T11:36:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f402dd9b700 1 -- 172.21.15.161:0/1471262732 learned_addr learned my addr 172.21.15.161:0/1471262732 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f40439c2b80 10 client.0 _ll_register_callbacks cb 0x55ef977a5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.530+0000 7f402d59a700 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f402e59c700 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f402dd9b700 1 -- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 msgr2=0x55ef978c0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f402dd9b700 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f402dd9b700 1 -- 172.21.15.161:0/1471262732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 msgr2=0x55ef978c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f402dd9b700 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f402dd9b700 1 -- 172.21.15.161:0/1471262732 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402400aa00 con 0x55ef978b52d0 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f402dd9b700 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978bfe10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f402400a320 tx=0x7f4024010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67f8e23669df8bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.531+0000 7f40117fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40117fa700 1 -- 172.21.15.161:0/1471262732 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400a8a0 con 0x55ef978b52d0 2023-07-14T11:36:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef977614f0 con 0x55ef978b52d0 2023-07-14T11:36:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40117fa700 1 -- 172.21.15.161:0/1471262732 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4024005460 con 0x55ef978b52d0 2023-07-14T11:36:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40117fa700 1 -- 172.21.15.161:0/1471262732 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4024005620 con 0x55ef978b52d0 2023-07-14T11:36:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 10 client.27070 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:14.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ef9761ec60 con 0x55ef978b52d0 2023-07-14T11:36:14.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40117fa700 1 -- 172.21.15.161:0/1471262732 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef977614f0 con 0x55ef978b52d0 2023-07-14T11:36:14.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40117fa700 1 -- 172.21.15.161:0/1471262732 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4024018560 con 0x55ef978b52d0 2023-07-14T11:36:14.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 10 client.27070 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:14.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 10 client.27070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:14.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 20 client.27070 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:14.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef978e1350 0x55ef978e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 4 client.27070 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.532+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef97800eb0 con 0x55ef978e1350 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.533+0000 7f402d59a700 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef978e1350 0x55ef978e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.533+0000 7f402d59a700 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef978e1350 0x55ef978e3760 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beff8fb0fc69ed45 server_cookie=a0f751d3165f524b in_seq=0 out_seq=0 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.533+0000 7f40117fa700 10 client.27070 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.536+0000 7f40117fa700 1 -- 172.21.15.161:0/1471262732 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef97800eb0 con 0x55ef978e1350 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.536+0000 7f40117fa700 10 client.27070 handle_command_reply: tid=0 2023-07-14T11:36:14.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.536+0000 7f40439c2b80 1 client.27070 shutdown 2023-07-14T11:36:14.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 msgr2=0x55ef978bfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978bfe10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f402400a320 tx=0x7f4024010cb0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 shutdown_connections 2023-07-14T11:36:14.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef978e1350 0x55ef978e3760 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978bfe10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 --2- 172.21.15.161:0/1471262732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 >> 172.21.15.161:0/1471262732 conn(0x55ef978abda0 msgr2=0x55ef978b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:14.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40117fa700 0 client.27070 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:14.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 shutdown_connections 2023-07-14T11:36:14.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 1 -- 172.21.15.161:0/1471262732 wait complete. 2023-07-14T11:36:14.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.537+0000 7f40439c2b80 20 client.27070 trim_cache size 0 max 16384 2023-07-14T11:36:14.559 INFO:tasks.workunit.client.0.smithi161.stdout: 10 35820 37.64 MB/sec warmup 56 sec latency 52.666 ms 2023-07-14T11:36:14.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.558+0000 7f40439c2b80 1 Processor -- start 2023-07-14T11:36:14.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.558+0000 7f40439c2b80 1 -- start start 2023-07-14T11:36:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.558+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f40439c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef978ab970 con 0x55ef978c4ee0 2023-07-14T11:36:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f40439c2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef978b5fb0 con 0x55ef978b5be0 2023-07-14T11:36:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f40439c2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef978e3dc0 con 0x55ef978b52d0 2023-07-14T11:36:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41442/0 (socket says 172.21.15.161:41442) 2023-07-14T11:36:14.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 -- 172.21.15.161:0/2437810941 learned_addr learned my addr 172.21.15.161:0/2437810941 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:14.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402d59a700 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402e59c700 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 -- 172.21.15.161:0/2437810941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 msgr2=0x55ef978b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 -- 172.21.15.161:0/2437810941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 msgr2=0x55ef978bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 -- 172.21.15.161:0/2437810941 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef978ad5e0 con 0x55ef978b52d0 2023-07-14T11:36:14.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402e59c700 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.559+0000 7f402dd9b700 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c06c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f402400faa0 tx=0x7f402400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e16838377cf117f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f401cff9700 1 -- 172.21.15.161:0/2437810941 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400a8a0 con 0x55ef978b52d0 2023-07-14T11:36:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f401cff9700 1 -- 172.21.15.161:0/2437810941 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4024010430 con 0x55ef978b52d0 2023-07-14T11:36:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f401cff9700 1 -- 172.21.15.161:0/2437810941 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024004950 con 0x55ef978b52d0 2023-07-14T11:36:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f40439c2b80 1 -- 172.21.15.161:0/2437810941 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 msgr2=0x55ef978c06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f40439c2b80 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c06c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f402400faa0 tx=0x7f402400f6a0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f40439c2b80 1 -- 172.21.15.161:0/2437810941 shutdown_connections 2023-07-14T11:36:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f40439c2b80 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f40439c2b80 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c06c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f40439c2b80 1 --2- 172.21.15.161:0/2437810941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.560+0000 7f40439c2b80 1 -- 172.21.15.161:0/2437810941 >> 172.21.15.161:0/2437810941 conn(0x55ef978abda0 msgr2=0x55ef978b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:14.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.561+0000 7f40439c2b80 1 -- 172.21.15.161:0/2437810941 shutdown_connections 2023-07-14T11:36:14.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.561+0000 7f40439c2b80 1 -- 172.21.15.161:0/2437810941 wait complete. 2023-07-14T11:36:14.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.561+0000 7f40439c2b80 1 -- start start 2023-07-14T11:36:14.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 1 Processor -- start 2023-07-14T11:36:14.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef978c7f40 con 0x55ef978c4ee0 2023-07-14T11:36:14.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef978c8080 con 0x55ef978b5be0 2023-07-14T11:36:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef978c8240 con 0x55ef978b52d0 2023-07-14T11:36:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f40439c2b80 10 client.0 _ll_register_callbacks cb 0x7f402400ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f402dd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41456/0 (socket says 172.21.15.161:41456) 2023-07-14T11:36:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f402dd9b700 1 -- 172.21.15.161:0/3522646266 learned_addr learned my addr 172.21.15.161:0/3522646266 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.562+0000 7f402d59a700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402e59c700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402dd9b700 1 -- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 msgr2=0x55ef978c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402dd9b700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402dd9b700 1 -- 172.21.15.161:0/3522646266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 msgr2=0x55ef978c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402dd9b700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402dd9b700 1 -- 172.21.15.161:0/3522646266 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef9761ec60 con 0x55ef978b52d0 2023-07-14T11:36:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402d59a700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f402dd9b700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c91c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4024010010 tx=0x7f402400a8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e48d471e5c61c72 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.563+0000 7f40117fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40117fa700 1 -- 172.21.15.161:0/3522646266 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024018550 con 0x55ef978b52d0 2023-07-14T11:36:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40117fa700 1 -- 172.21.15.161:0/3522646266 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4024000e40 con 0x55ef978b52d0 2023-07-14T11:36:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef977c3050 con 0x55ef978b52d0 2023-07-14T11:36:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40117fa700 1 -- 172.21.15.161:0/3522646266 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40240186b0 con 0x55ef978b52d0 2023-07-14T11:36:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40439c2b80 10 client.27076 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:14.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ef9765c250 con 0x55ef978b52d0 2023-07-14T11:36:14.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40117fa700 1 -- 172.21.15.161:0/3522646266 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef977c3050 con 0x55ef978b52d0 2023-07-14T11:36:14.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40117fa700 1 -- 172.21.15.161:0/3522646266 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4024004a30 con 0x55ef978b52d0 2023-07-14T11:36:14.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40439c2b80 10 client.27076 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:14.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40439c2b80 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.564+0000 7f40439c2b80 20 client.27076 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.565+0000 7f40439c2b80 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef97936270 0x55ef97938650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.565+0000 7f40439c2b80 4 client.27076 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:36:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.565+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef97656f70 con 0x55ef97936270 2023-07-14T11:36:14.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.565+0000 7f402d59a700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef97936270 0x55ef97938650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.565+0000 7f402d59a700 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef97936270 0x55ef97938650 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ade854642860781 server_cookie=36f24b93deb75071 in_seq=0 out_seq=0 2023-07-14T11:36:14.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.565+0000 7f40117fa700 10 client.27076 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:14.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.568+0000 7f40117fa700 1 -- 172.21.15.161:0/3522646266 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ef97656f70 con 0x55ef97936270 2023-07-14T11:36:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.568+0000 7f40117fa700 10 client.27076 handle_command_reply: tid=0 2023-07-14T11:36:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.568+0000 7f40439c2b80 1 client.27076 shutdown 2023-07-14T11:36:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 msgr2=0x55ef978c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c91c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4024010010 tx=0x7f402400a8a0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 shutdown_connections 2023-07-14T11:36:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ef97936270 0x55ef97938650 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef978c4ee0 0x55ef978c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ef978b52d0 0x55ef978c91c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 --2- 172.21.15.161:0/3522646266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef978b5be0 0x55ef978c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40117fa700 0 client.27076 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:14.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 >> 172.21.15.161:0/3522646266 conn(0x55ef978abda0 msgr2=0x55ef978b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:14.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 shutdown_connections 2023-07-14T11:36:14.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 1 -- 172.21.15.161:0/3522646266 wait complete. 2023-07-14T11:36:14.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.569+0000 7f40439c2b80 20 client.27076 trim_cache size 0 max 16384 2023-07-14T11:36:14.583 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:36:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:36:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:36:14.584 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:36:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40140a1340 msgr2=0x7f40140a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40140a1340 0x7f40140a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4028105110 tx=0x7f4018009260 comp rx=0 tx=0).stop 2023-07-14T11:36:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028105ae0 msgr2=0x7f4028195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028105ae0 0x7f4028195ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f402000bce0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 shutdown_connections 2023-07-14T11:36:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40140a1340 0x7f40140a37f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4028105ae0 0x7f4028195ab0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40280ffad0 0x7f402819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 --2- 172.21.15.161:0/1282123140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4028100480 0x7f402819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 >> 172.21.15.161:0/1282123140 conn(0x7f40280fb5f0 msgr2=0x7f40281035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:14.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 shutdown_connections 2023-07-14T11:36:14.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.570+0000 7f402ffff700 1 -- 172.21.15.161:0/1282123140 wait complete. 2023-07-14T11:36:14.593 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:36:14.593 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:36:14.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.848+0000 7f57647ff700 1 -- 172.21.15.161:0/700211287 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c0ffb50 msgr2=0x7f575c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.848+0000 7f57647ff700 1 --2- 172.21.15.161:0/700211287 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c0ffb50 0x7f575c0fffc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f57500098f0 tx=0x7f5750009c00 comp rx=0 tx=0).stop 2023-07-14T11:36:14.852 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.849+0000 7f57647ff700 1 -- 172.21.15.161:0/700211287 shutdown_connections 2023-07-14T11:36:14.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.849+0000 7f57647ff700 1 --2- 172.21.15.161:0/700211287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f575c105b80 0x7f575c105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.849+0000 7f57647ff700 1 --2- 172.21.15.161:0/700211287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575c100500 0x7f575c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.849+0000 7f57647ff700 1 --2- 172.21.15.161:0/700211287 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c0ffb50 0x7f575c0fffc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.849+0000 7f57647ff700 1 -- 172.21.15.161:0/700211287 >> 172.21.15.161:0/700211287 conn(0x7f575c0fb650 msgr2=0x7f575c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:14.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.849+0000 7f57647ff700 1 -- 172.21.15.161:0/700211287 shutdown_connections 2023-07-14T11:36:14.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.849+0000 7f57647ff700 1 -- 172.21.15.161:0/700211287 wait complete. 2023-07-14T11:36:14.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 Processor -- start 2023-07-14T11:36:14.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 -- start start 2023-07-14T11:36:14.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575c0ffb50 0x7f575c197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c100500 0x7f575c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f575c105b80 0x7f575c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c19cd70 con 0x7f575c105b80 2023-07-14T11:36:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c19cee0 con 0x7f575c0ffb50 2023-07-14T11:36:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.850+0000 7f57647ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c19d0a0 con 0x7f575c100500 2023-07-14T11:36:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f576259b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575c0ffb50 0x7f575c197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5761d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c100500 0x7f575c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f576259b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575c0ffb50 0x7f575c197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45028/0 (socket says 172.21.15.161:45028) 2023-07-14T11:36:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f576259b700 1 -- 172.21.15.161:0/1303572485 learned_addr learned my addr 172.21.15.161:0/1303572485 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5762d9c700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f575c105b80 0x7f575c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5761d9a700 1 -- 172.21.15.161:0/1303572485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575c0ffb50 msgr2=0x7f575c197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5761d9a700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575c0ffb50 0x7f575c197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5761d9a700 1 -- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f575c105b80 msgr2=0x7f575c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5761d9a700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f575c105b80 0x7f575c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5761d9a700 1 -- 172.21.15.161:0/1303572485 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5750009520 con 0x7f575c100500 2023-07-14T11:36:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5762d9c700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f575c105b80 0x7f575c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.851+0000 7f5761d9a700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c100500 0x7f575c1982d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5750009810 tx=0x7f575000fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.852+0000 7f574f7fe700 1 -- 172.21.15.161:0/1303572485 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575001c070 con 0x7f575c100500 2023-07-14T11:36:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.852+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575c19d320 con 0x7f575c100500 2023-07-14T11:36:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.852+0000 7f574f7fe700 1 -- 172.21.15.161:0/1303572485 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5750005040 con 0x7f575c100500 2023-07-14T11:36:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.852+0000 7f574f7fe700 1 -- 172.21.15.161:0/1303572485 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57500039c0 con 0x7f575c100500 2023-07-14T11:36:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.852+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575c1a4bc0 con 0x7f575c100500 2023-07-14T11:36:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.853+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575c067300 con 0x7f575c100500 2023-07-14T11:36:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.855+0000 7f574f7fe700 1 -- 172.21.15.161:0/1303572485 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5750020030 con 0x7f575c100500 2023-07-14T11:36:14.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.857+0000 7f574f7fe700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f57480a1300 0x7f57480a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:14.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.858+0000 7f574f7fe700 1 -- 172.21.15.161:0/1303572485 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f57500cdfe0 con 0x7f575c100500 2023-07-14T11:36:14.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.858+0000 7f576259b700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f57480a1300 0x7f57480a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:14.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.860+0000 7f576259b700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f57480a1300 0x7f57480a37b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57540095b0 tx=0x7f5754009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:14.867+0000 7f574f7fe700 1 -- 172.21.15.161:0/1303572485 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5750097100 con 0x7f575c100500 2023-07-14T11:36:15.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.249+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f575c199180 con 0x7f575c100500 2023-07-14T11:36:15.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.258+0000 7f574f7fe700 1 -- 172.21.15.161:0/1303572485 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f575009a720 con 0x7f575c100500 2023-07-14T11:36:15.264 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:36:15.265 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:35:27.518481+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:36:15.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f57480a1300 msgr2=0x7f57480a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f57480a1300 0x7f57480a37b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57540095b0 tx=0x7f5754009040 comp rx=0 tx=0).stop 2023-07-14T11:36:15.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c100500 msgr2=0x7f575c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c100500 0x7f575c1982d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5750009810 tx=0x7f575000fa50 comp rx=0 tx=0).stop 2023-07-14T11:36:15.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 shutdown_connections 2023-07-14T11:36:15.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f57480a1300 0x7f57480a37b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f575c105b80 0x7f575c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f575c0ffb50 0x7f575c197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 --2- 172.21.15.161:0/1303572485 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f575c100500 0x7f575c1982d0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5750009810 tx=0x7f575000fa50 comp rx=0 tx=0).stop 2023-07-14T11:36:15.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.263+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 >> 172.21.15.161:0/1303572485 conn(0x7f575c0fb650 msgr2=0x7f575c072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:15.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.264+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 shutdown_connections 2023-07-14T11:36:15.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.264+0000 7f57647ff700 1 -- 172.21.15.161:0/1303572485 wait complete. 2023-07-14T11:36:15.277 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 37 2023-07-14T11:36:15.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:36:15.561 INFO:tasks.workunit.client.0.smithi161.stdout: 10 36462 37.39 MB/sec warmup 57 sec latency 58.888 ms 2023-07-14T11:36:15.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.647+0000 7fa185fff700 1 -- 172.21.15.161:0/3612700872 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa180100490 msgr2=0x7fa18010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.647+0000 7fa185fff700 1 --2- 172.21.15.161:0/3612700872 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa180100490 0x7fa18010c920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa17800f4c0 tx=0x7fa17800f7d0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.647+0000 7fa185fff700 1 -- 172.21.15.161:0/3612700872 shutdown_connections 2023-07-14T11:36:15.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.647+0000 7fa185fff700 1 --2- 172.21.15.161:0/3612700872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa180105af0 0x7fa180105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.647+0000 7fa185fff700 1 --2- 172.21.15.161:0/3612700872 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa180100490 0x7fa18010c920 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.647+0000 7fa185fff700 1 --2- 172.21.15.161:0/3612700872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1800ffae0 0x7fa1800fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.647+0000 7fa185fff700 1 -- 172.21.15.161:0/3612700872 >> 172.21.15.161:0/3612700872 conn(0x7fa1800fb5e0 msgr2=0x7fa1800fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:15.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.648+0000 7fa185fff700 1 -- 172.21.15.161:0/3612700872 shutdown_connections 2023-07-14T11:36:15.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.652+0000 7fa185fff700 1 -- 172.21.15.161:0/3612700872 wait complete. 2023-07-14T11:36:15.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.654+0000 7fa185fff700 1 Processor -- start 2023-07-14T11:36:15.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.654+0000 7fa185fff700 1 -- start start 2023-07-14T11:36:15.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.655+0000 7fa185fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa1800ffae0 0x7fa18010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.655+0000 7fa185fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa180100490 0x7fa18010e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.655+0000 7fa185fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa180105af0 0x7fa1801b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.655+0000 7fa185fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801b0d70 con 0x7fa180105af0 2023-07-14T11:36:15.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.655+0000 7fa185fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801b0ee0 con 0x7fa180100490 2023-07-14T11:36:15.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.655+0000 7fa185fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801b10a0 con 0x7fa1800ffae0 2023-07-14T11:36:15.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.656+0000 7fa17effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa180100490 0x7fa18010e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.656+0000 7fa17effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa180100490 0x7fa18010e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45042/0 (socket says 172.21.15.161:45042) 2023-07-14T11:36:15.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.656+0000 7fa17effd700 1 -- 172.21.15.161:0/1813125595 learned_addr learned my addr 172.21.15.161:0/1813125595 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:15.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.658+0000 7fa17effd700 1 -- 172.21.15.161:0/1813125595 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa1800ffae0 msgr2=0x7fa18010df50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:36:15.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.658+0000 7fa17effd700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa1800ffae0 0x7fa18010df50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.658+0000 7fa17effd700 1 -- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa180105af0 msgr2=0x7fa1801b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.658+0000 7fa17effd700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa180105af0 0x7fa1801b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.658+0000 7fa17effd700 1 -- 172.21.15.161:0/1813125595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17800f170 con 0x7fa180100490 2023-07-14T11:36:15.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.659+0000 7fa17effd700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa180100490 0x7fa18010e490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa17400b530 tx=0x7fa17400b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:15.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.659+0000 7fa17cff9700 1 -- 172.21.15.161:0/1813125595 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011830 con 0x7fa180100490 2023-07-14T11:36:15.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.659+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1801b1380 con 0x7fa180100490 2023-07-14T11:36:15.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.659+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801b18e0 con 0x7fa180100490 2023-07-14T11:36:15.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.661+0000 7fa17cff9700 1 -- 172.21.15.161:0/1813125595 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa174011e50 con 0x7fa180100490 2023-07-14T11:36:15.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.661+0000 7fa17cff9700 1 -- 172.21.15.161:0/1813125595 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400f2e0 con 0x7fa180100490 2023-07-14T11:36:15.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:15 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1303572485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:36:15.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.669+0000 7fa199999b80 1 Processor -- start 2023-07-14T11:36:15.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.669+0000 7fa17cff9700 1 -- 172.21.15.161:0/1813125595 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa17400f440 con 0x7fa180100490 2023-07-14T11:36:15.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.670+0000 7fa199999b80 1 -- start start 2023-07-14T11:36:15.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa199999b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549d2d0 0x55706549d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa199999b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa199999b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa199999b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570653a3cb0 con 0x55706549dbe0 2023-07-14T11:36:15.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa199999b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570653f6a40 con 0x5570654a8ad0 2023-07-14T11:36:15.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa199999b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570653efed0 con 0x55706549d2d0 2023-07-14T11:36:15.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa17effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa17f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa17f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38690/0 (socket says 172.21.15.161:38690) 2023-07-14T11:36:15.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.671+0000 7fa17f7fe700 1 -- 172.21.15.161:0/3614222271 learned_addr learned my addr 172.21.15.161:0/3614222271 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:15.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.672+0000 7fa17f7fe700 1 -- 172.21.15.161:0/3614222271 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549d2d0 msgr2=0x55706549d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:15.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.672+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549d2d0 0x55706549d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.672+0000 7fa17f7fe700 1 -- 172.21.15.161:0/3614222271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 msgr2=0x5570654aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:15.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.672+0000 7fa17cff9700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1680a12c0 0x7fa1680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.672+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.673+0000 7fa17f7fe700 1 -- 172.21.15.161:0/3614222271 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706521dde0 con 0x55706549dbe0 2023-07-14T11:36:15.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.673+0000 7fa17cff9700 1 -- 172.21.15.161:0/1813125595 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa1740ccef0 con 0x7fa180100490 2023-07-14T11:36:15.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.673+0000 7fa17ffff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1680a12c0 0x7fa1680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.673+0000 7fa17effd700 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:15.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.673+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a8590 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa17000ba90 tx=0x7fa17000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417d4e948d6ffe33 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:15.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.674+0000 7fa165ffb700 1 -- 172.21.15.161:0/3614222271 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170004a30 con 0x55706549dbe0 2023-07-14T11:36:15.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.674+0000 7fa165ffb700 1 -- 172.21.15.161:0/3614222271 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1700054e0 con 0x55706549dbe0 2023-07-14T11:36:15.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.674+0000 7fa165ffb700 1 -- 172.21.15.161:0/3614222271 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000dca0 con 0x55706549dbe0 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.674+0000 7fa199999b80 1 -- 172.21.15.161:0/3614222271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 msgr2=0x5570654a8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.674+0000 7fa199999b80 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a8590 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa17000ba90 tx=0x7fa17000bda0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.674+0000 7fa17ffff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1680a12c0 0x7fa1680a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa178011010 tx=0x7fa17800fc10 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.675+0000 7fa199999b80 1 -- 172.21.15.161:0/3614222271 shutdown_connections 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.675+0000 7fa199999b80 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.675+0000 7fa199999b80 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549d2d0 0x55706549d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.675+0000 7fa199999b80 1 --2- 172.21.15.161:0/3614222271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a8590 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.675+0000 7fa199999b80 1 -- 172.21.15.161:0/3614222271 >> 172.21.15.161:0/3614222271 conn(0x557065493da0 msgr2=0x5570654996a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:15.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.676+0000 7fa199999b80 1 -- 172.21.15.161:0/3614222271 shutdown_connections 2023-07-14T11:36:15.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.676+0000 7fa199999b80 1 -- 172.21.15.161:0/3614222271 wait complete. 2023-07-14T11:36:15.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.676+0000 7fa199999b80 1 -- start start 2023-07-14T11:36:15.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 1 Processor -- start 2023-07-14T11:36:15.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x5570654a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5570654a8ad0 0x5570654a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706538fac0 con 0x55706549dbe0 2023-07-14T11:36:15.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706538f7c0 con 0x55706549d2d0 2023-07-14T11:36:15.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557065273920 con 0x5570654a8ad0 2023-07-14T11:36:15.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.677+0000 7fa199999b80 10 client.0 _ll_register_callbacks cb 0x55706538d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:15.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.678+0000 7fa17ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5570654a8ad0 0x5570654a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.678+0000 7fa17ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5570654a8ad0 0x5570654a5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41520/0 (socket says 172.21.15.161:41520) 2023-07-14T11:36:15.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.678+0000 7fa17ffff700 1 -- 172.21.15.161:0/3699465456 learned_addr learned my addr 172.21.15.161:0/3699465456 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:15.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.678+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x5570654a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.678+0000 7fa17effd700 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.679+0000 7fa17ffff700 1 -- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 msgr2=0x5570654a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:15.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.679+0000 7fa17ffff700 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x5570654a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.679+0000 7fa17ffff700 1 -- 172.21.15.161:0/3699465456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 msgr2=0x5570654a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:15.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.679+0000 7fa17ffff700 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.679+0000 7fa17ffff700 1 -- 172.21.15.161:0/3699465456 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17000b740 con 0x5570654a8ad0 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.682+0000 7fa17ffff700 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5570654a8ad0 0x5570654a5e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa178017d60 tx=0x7fa178016410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aeadd5b772c3966 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.682+0000 7fa15effd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.682+0000 7fa15effd700 1 -- 172.21.15.161:0/3699465456 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178012040 con 0x5570654a8ad0 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.682+0000 7fa15effd700 1 -- 172.21.15.161:0/3699465456 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa17800bd50 con 0x5570654a8ad0 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.682+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570653494f0 con 0x5570654a8ad0 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.683+0000 7fa15effd700 1 -- 172.21.15.161:0/3699465456 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17800b530 con 0x5570654a8ad0 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.683+0000 7fa199999b80 10 client.27088 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:15.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.683+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x557065206c60 con 0x5570654a8ad0 2023-07-14T11:36:15.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.683+0000 7fa15effd700 1 -- 172.21.15.161:0/3699465456 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570653494f0 con 0x5570654a8ad0 2023-07-14T11:36:15.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.684+0000 7fa15effd700 1 -- 172.21.15.161:0/3699465456 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa1780074a0 con 0x5570654a8ad0 2023-07-14T11:36:15.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.684+0000 7fa199999b80 10 client.27088 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:15.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.684+0000 7fa199999b80 10 client.27088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:36:15.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.684+0000 7fa199999b80 20 client.27088 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:15.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.684+0000 7fa199999b80 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5570654c9450 0x5570654cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.684+0000 7fa199999b80 4 client.27088 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:15.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.684+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570653e8eb0 con 0x5570654c9450 2023-07-14T11:36:15.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.685+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5570654c9450 0x5570654cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.685+0000 7fa15effd700 10 client.27088 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:15.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.685+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5570654c9450 0x5570654cb860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c31630133f06f717 server_cookie=ca77a84341b44c89 in_seq=0 out_seq=0 2023-07-14T11:36:15.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.686+0000 7fa15effd700 1 -- 172.21.15.161:0/3699465456 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5570653e8eb0 con 0x5570654c9450 2023-07-14T11:36:15.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.686+0000 7fa15effd700 10 client.27088 handle_command_reply: tid=0 2023-07-14T11:36:15.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.686+0000 7fa199999b80 1 client.27088 shutdown 2023-07-14T11:36:15.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5570654a8ad0 msgr2=0x5570654a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5570654a8ad0 0x5570654a5e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa178017d60 tx=0x7fa178016410 comp rx=0 tx=0).stop 2023-07-14T11:36:15.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 shutdown_connections 2023-07-14T11:36:15.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5570654c9450 0x5570654cb860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5570654a8ad0 0x5570654a5e70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x5570654a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 --2- 172.21.15.161:0/3699465456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549dbe0 0x5570654a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 >> 172.21.15.161:0/3699465456 conn(0x557065493da0 msgr2=0x557065499230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:15.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.687+0000 7fa15effd700 0 client.27088 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:15.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.689+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 shutdown_connections 2023-07-14T11:36:15.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.689+0000 7fa199999b80 1 -- 172.21.15.161:0/3699465456 wait complete. 2023-07-14T11:36:15.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.689+0000 7fa199999b80 20 client.27088 trim_cache size 0 max 16384 2023-07-14T11:36:15.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:15 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1303572485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:36:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.713+0000 7fa199999b80 1 Processor -- start 2023-07-14T11:36:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.714+0000 7fa199999b80 1 -- start start 2023-07-14T11:36:15.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.714+0000 7fa199999b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549d2d0 0x5570654a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.714+0000 7fa199999b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065497e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.714+0000 7fa199999b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.714+0000 7fa199999b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706549f1a0 con 0x55706549d2d0 2023-07-14T11:36:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.714+0000 7fa199999b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557065498350 con 0x5570654a8ad0 2023-07-14T11:36:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.714+0000 7fa199999b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570654a5940 con 0x55706549dbe0 2023-07-14T11:36:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.715+0000 7fa17f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549d2d0 0x5570654a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.715+0000 7fa17f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549d2d0 0x5570654a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38704/0 (socket says 172.21.15.161:38704) 2023-07-14T11:36:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.715+0000 7fa17f7fe700 1 -- 172.21.15.161:0/1113721486 learned_addr learned my addr 172.21.15.161:0/1113721486 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.716+0000 7fa17f7fe700 1 -- 172.21.15.161:0/1113721486 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 msgr2=0x557065497e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:36:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.716+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/1113721486 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065497e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.716+0000 7fa17f7fe700 1 -- 172.21.15.161:0/1113721486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 msgr2=0x5570654a4ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:36:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.716+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/1113721486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654a4ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.716+0000 7fa17f7fe700 1 -- 172.21.15.161:0/1113721486 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557065244250 con 0x55706549d2d0 2023-07-14T11:36:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.716+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/1113721486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549d2d0 0x5570654a5dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa17000ba30 tx=0x7fa17000b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19c8d06736ee159 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.717+0000 7fa1667fc700 1 -- 172.21.15.161:0/1113721486 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170020040 con 0x55706549d2d0 2023-07-14T11:36:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.717+0000 7fa1667fc700 1 -- 172.21.15.161:0/1113721486 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa170014070 con 0x55706549d2d0 2023-07-14T11:36:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.717+0000 7fa1667fc700 1 -- 172.21.15.161:0/1113721486 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000dbd0 con 0x55706549d2d0 2023-07-14T11:36:15.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 -- 172.21.15.161:0/1113721486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549d2d0 msgr2=0x5570654a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 --2- 172.21.15.161:0/1113721486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549d2d0 0x5570654a5dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa17000ba30 tx=0x7fa17000b3a0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 -- 172.21.15.161:0/1113721486 shutdown_connections 2023-07-14T11:36:15.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 --2- 172.21.15.161:0/1113721486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706549d2d0 0x5570654a5dd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 --2- 172.21.15.161:0/1113721486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570654a8ad0 0x5570654a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 --2- 172.21.15.161:0/1113721486 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065497e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 -- 172.21.15.161:0/1113721486 >> 172.21.15.161:0/1113721486 conn(0x557065493da0 msgr2=0x5570654d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 -- 172.21.15.161:0/1113721486 shutdown_connections 2023-07-14T11:36:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.720+0000 7fa199999b80 1 -- 172.21.15.161:0/1113721486 wait complete. 2023-07-14T11:36:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.721+0000 7fa199999b80 1 -- start start 2023-07-14T11:36:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.721+0000 7fa199999b80 1 Processor -- start 2023-07-14T11:36:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa199999b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x557065516610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa199999b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065516b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa199999b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5570654a8ad0 0x5570654ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa199999b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570654adb20 con 0x5570654a8ad0 2023-07-14T11:36:15.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa199999b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570654adc90 con 0x55706549d2d0 2023-07-14T11:36:15.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa199999b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570654ade50 con 0x55706549dbe0 2023-07-14T11:36:15.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa199999b80 10 client.0 _ll_register_callbacks cb 0x7fa178019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa17effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065516b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa17ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5570654a8ad0 0x5570654ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa17ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5570654a8ad0 0x5570654ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38720/0 (socket says 172.21.15.161:38720) 2023-07-14T11:36:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa17effd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065516b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41536/0 (socket says 172.21.15.161:41536) 2023-07-14T11:36:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.722+0000 7fa17ffff700 1 -- 172.21.15.161:0/1473359930 learned_addr learned my addr 172.21.15.161:0/1473359930 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x557065516610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17ffff700 1 -- 172.21.15.161:0/1473359930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 msgr2=0x557065516b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17ffff700 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065516b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17ffff700 1 -- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 msgr2=0x557065516610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17ffff700 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x557065516610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17ffff700 1 -- 172.21.15.161:0/1473359930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55706521dde0 con 0x5570654a8ad0 2023-07-14T11:36:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x557065516610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa17ffff700 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5570654a8ad0 0x5570654ad560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa178007800 tx=0x7fa17800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff5fc43fd331046 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.723+0000 7fa15effd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:36:15.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.724+0000 7fa15effd700 1 -- 172.21.15.161:0/1473359930 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178012070 con 0x5570654a8ad0 2023-07-14T11:36:15.741 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.724+0000 7fa15effd700 1 -- 172.21.15.161:0/1473359930 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa17800b530 con 0x5570654a8ad0 2023-07-14T11:36:15.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.724+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1700054e0 con 0x5570654a8ad0 2023-07-14T11:36:15.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.724+0000 7fa15effd700 1 -- 172.21.15.161:0/1473359930 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17800b6f0 con 0x5570654a8ad0 2023-07-14T11:36:15.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 10 client.17037 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:15.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55706521c2b0 con 0x5570654a8ad0 2023-07-14T11:36:15.742 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa15effd700 1 -- 172.21.15.161:0/1473359930 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1700054e0 con 0x5570654a8ad0 2023-07-14T11:36:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa15effd700 1 -- 172.21.15.161:0/1473359930 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa178037740 con 0x5570654a8ad0 2023-07-14T11:36:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 10 client.17037 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 10 client.17037 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:36:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 20 client.17037 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55706551e260 0x557065520670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 4 client.17037 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:36:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.726+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5570653ab050 con 0x55706551e260 2023-07-14T11:36:15.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.727+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55706551e260 0x557065520670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:15.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.728+0000 7fa17f7fe700 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55706551e260 0x557065520670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a58375421d253bb server_cookie=773c533ed60b0fc3 in_seq=0 out_seq=0 2023-07-14T11:36:15.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.728+0000 7fa15effd700 10 client.17037 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:15.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.728+0000 7fa15effd700 1 -- 172.21.15.161:0/1473359930 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5570653ab050 con 0x55706551e260 2023-07-14T11:36:15.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.728+0000 7fa15effd700 10 client.17037 handle_command_reply: tid=0 2023-07-14T11:36:15.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.728+0000 7fa199999b80 1 client.17037 shutdown 2023-07-14T11:36:15.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5570654a8ad0 msgr2=0x5570654ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5570654a8ad0 0x5570654ad560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa178007800 tx=0x7fa17800bd70 comp rx=0 tx=0).stop 2023-07-14T11:36:15.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 shutdown_connections 2023-07-14T11:36:15.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55706551e260 0x557065520670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5570654a8ad0 0x5570654ad560 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55706549d2d0 0x557065516610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 --2- 172.21.15.161:0/1473359930 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55706549dbe0 0x557065516b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 >> 172.21.15.161:0/1473359930 conn(0x557065493da0 msgr2=0x5570654a0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:15.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa15effd700 0 client.17037 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:15.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.729+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 shutdown_connections 2023-07-14T11:36:15.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.730+0000 7fa199999b80 1 -- 172.21.15.161:0/1473359930 wait complete. 2023-07-14T11:36:15.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.730+0000 7fa199999b80 20 client.17037 trim_cache size 0 max 16384 2023-07-14T11:36:15.746 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:36:15.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.730+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1680a12c0 msgr2=0x7fa1680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.730+0000 7fa185fff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1680a12c0 0x7fa1680a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa178011010 tx=0x7fa17800fc10 comp rx=0 tx=0).stop 2023-07-14T11:36:15.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.730+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa180100490 msgr2=0x7fa18010e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:15.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.730+0000 7fa185fff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa180100490 0x7fa18010e490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa17400b530 tx=0x7fa17400b8f0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 shutdown_connections 2023-07-14T11:36:15.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa1680a12c0 0x7fa1680a3770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa180105af0 0x7fa1801b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa1800ffae0 0x7fa18010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 --2- 172.21.15.161:0/1813125595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa180100490 0x7fa18010e490 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:15.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 >> 172.21.15.161:0/1813125595 conn(0x7fa1800fb5e0 msgr2=0x7fa1800fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:15.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 shutdown_connections 2023-07-14T11:36:15.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:15.731+0000 7fa185fff700 1 -- 172.21.15.161:0/1813125595 wait complete. 2023-07-14T11:36:15.749 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:15 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1303572485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:36:15.781 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:36:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 -- 172.21.15.161:0/4202200534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 msgr2=0x7f5d2c10bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 --2- 172.21.15.161:0/4202200534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c10bd60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009990 tx=0x7f5d1c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 -- 172.21.15.161:0/4202200534 shutdown_connections 2023-07-14T11:36:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 --2- 172.21.15.161:0/4202200534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c072850 0x7f5d2c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 --2- 172.21.15.161:0/4202200534 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d2c10c2a0 0x7f5d2c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 --2- 172.21.15.161:0/4202200534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c10bd60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 -- 172.21.15.161:0/4202200534 >> 172.21.15.161:0/4202200534 conn(0x7f5d2c0fb500 msgr2=0x7f5d2c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 -- 172.21.15.161:0/4202200534 shutdown_connections 2023-07-14T11:36:16.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.030+0000 7f5d3279d700 1 -- 172.21.15.161:0/4202200534 wait complete. 2023-07-14T11:36:16.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 Processor -- start 2023-07-14T11:36:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 -- start start 2023-07-14T11:36:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c072850 0x7f5d2c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d2c10c2a0 0x7f5d2c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c1980f0 con 0x7f5d2c072850 2023-07-14T11:36:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c198260 con 0x7f5d2c073160 2023-07-14T11:36:16.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d3279d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c198420 con 0x7f5d2c10c2a0 2023-07-14T11:36:16.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d30f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d30f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c19df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45082/0 (socket says 172.21.15.161:45082) 2023-07-14T11:36:16.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.032+0000 7f5d30f9a700 1 -- 172.21.15.161:0/283410780 learned_addr learned my addr 172.21.15.161:0/283410780 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d3179b700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c072850 0x7f5d2c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d31f9c700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d2c10c2a0 0x7f5d2c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d30f9a700 1 -- 172.21.15.161:0/283410780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d2c10c2a0 msgr2=0x7f5d2c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d2c10c2a0 0x7f5d2c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d30f9a700 1 -- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c072850 msgr2=0x7f5d2c19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c072850 0x7f5d2c19da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d30f9a700 1 -- 172.21.15.161:0/283410780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d28009520 con 0x7f5d2c073160 2023-07-14T11:36:16.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c19df50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005910 tx=0x7f5d1c004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.033+0000 7f5d227fc700 1 -- 172.21.15.161:0/283410780 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c01d070 con 0x7f5d2c073160 2023-07-14T11:36:16.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.034+0000 7f5d227fc700 1 -- 172.21.15.161:0/283410780 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1c022470 con 0x7f5d2c073160 2023-07-14T11:36:16.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.034+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c0095f0 con 0x7f5d2c073160 2023-07-14T11:36:16.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.034+0000 7f5d227fc700 1 -- 172.21.15.161:0/283410780 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c017650 con 0x7f5d2c073160 2023-07-14T11:36:16.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.034+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c1a4820 con 0x7f5d2c073160 2023-07-14T11:36:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.037+0000 7f5d227fc700 1 -- 172.21.15.161:0/283410780 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5d1c0225e0 con 0x7f5d2c073160 2023-07-14T11:36:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.038+0000 7f5d47227b80 1 Processor -- start 2023-07-14T11:36:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.038+0000 7f5d47227b80 1 -- start start 2023-07-14T11:36:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.038+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b85124f0 0x55d0b85128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.038+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b8512e00 0x55d0b851c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.038+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b851f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d47227b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b8417cb0 con 0x55d0b851cd30 2023-07-14T11:36:16.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d47227b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b846aa40 con 0x55d0b85124f0 2023-07-14T11:36:16.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d47227b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b8463ed0 con 0x55d0b8512e00 2023-07-14T11:36:16.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d227fc700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d180a1360 0x7f5d180a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b8512e00 0x55d0b851c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b8512e00 0x55d0b851c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41568/0 (socket says 172.21.15.161:41568) 2023-07-14T11:36:16.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 -- 172.21.15.161:0/1377688789 learned_addr learned my addr 172.21.15.161:0/1377688789 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d31f9c700 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b85124f0 0x55d0b85128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d227fc700 1 -- 172.21.15.161:0/283410780 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d1c0ce2c0 con 0x7f5d2c073160 2023-07-14T11:36:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b851f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d31f9c700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d180a1360 0x7f5d180a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 -- 172.21.15.161:0/1377688789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b85124f0 msgr2=0x55d0b85128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b85124f0 0x55d0b85128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 -- 172.21.15.161:0/1377688789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 msgr2=0x55d0b851f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b851f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.039+0000 7f5d3179b700 1 -- 172.21.15.161:0/1377688789 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0b8291de0 con 0x55d0b8512e00 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.040+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b8512e00 0x55d0b851c7f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009f10 tx=0x7f5d28011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509c90336b0d62ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.040+0000 7f5d177fe700 1 -- 172.21.15.161:0/1377688789 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28019630 con 0x55d0b8512e00 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.040+0000 7f5d31f9c700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d180a1360 0x7f5d180a3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009ec0 tx=0x7f5d2400b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.040+0000 7f5d177fe700 1 -- 172.21.15.161:0/1377688789 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d28019c50 con 0x55d0b8512e00 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.040+0000 7f5d177fe700 1 -- 172.21.15.161:0/1377688789 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800ece0 con 0x55d0b8512e00 2023-07-14T11:36:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.040+0000 7f5d47227b80 1 -- 172.21.15.161:0/1377688789 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b8512e00 msgr2=0x55d0b851c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.040+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b8512e00 0x55d0b851c7f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009f10 tx=0x7f5d28011720 comp rx=0 tx=0).stop 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 -- 172.21.15.161:0/1377688789 shutdown_connections 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b851f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b85124f0 0x55d0b85128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1377688789 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b8512e00 0x55d0b851c7f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 -- 172.21.15.161:0/1377688789 >> 172.21.15.161:0/1377688789 conn(0x55d0b8507ff0 msgr2=0x55d0b850d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 -- 172.21.15.161:0/1377688789 shutdown_connections 2023-07-14T11:36:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 -- 172.21.15.161:0/1377688789 wait complete. 2023-07-14T11:36:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.041+0000 7f5d47227b80 1 -- start start 2023-07-14T11:36:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 1 Processor -- start 2023-07-14T11:36:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b85124f0 0x55d0b851c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8519930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b8519f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b8403ac0 con 0x55d0b851cd30 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b84037c0 con 0x55d0b8512e00 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b82e7920 con 0x55d0b85124f0 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d47227b80 10 client.0 _ll_register_callbacks cb 0x55d0b8401760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d3179b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b85124f0 0x55d0b851c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d3179b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b85124f0 0x55d0b851c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41572/0 (socket says 172.21.15.161:41572) 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d3179b700 1 -- 172.21.15.161:0/1065972763 learned_addr learned my addr 172.21.15.161:0/1065972763 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.042+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8519930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d31f9c700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b8519f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d3179b700 1 -- 172.21.15.161:0/1065972763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 msgr2=0x55d0b8519930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8519930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d3179b700 1 -- 172.21.15.161:0/1065972763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 msgr2=0x55d0b8519f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b8519f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d3179b700 1 -- 172.21.15.161:0/1065972763 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d28009b70 con 0x55d0b85124f0 2023-07-14T11:36:16.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d31f9c700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b8519f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:16.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b85124f0 0x55d0b851c720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000c00 tx=0x7f5d2800ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a05c9747bc43c1d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.043+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1065972763 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28019630 con 0x55d0b85124f0 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1065972763 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d28019790 con 0x55d0b85124f0 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0b83bd4f0 con 0x55d0b85124f0 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1065972763 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d2800e680 con 0x55d0b85124f0 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 10 client.27103 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d0b827ac60 con 0x55d0b85124f0 2023-07-14T11:36:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1065972763 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0b83bd4f0 con 0x55d0b85124f0 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1065972763 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5d28004830 con 0x55d0b85124f0 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 10 client.27103 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 10 client.27103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 20 client.27103 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b852e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 4 client.27103 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.044+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0b845ceb0 con 0x55d0b852c4d0 2023-07-14T11:36:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.045+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b852e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.045+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b852e8e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56726f0b8be85b18 server_cookie=b62556e44244d394 in_seq=0 out_seq=0 2023-07-14T11:36:16.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.045+0000 7f5d14ff9700 10 client.27103 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:16.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.046+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1065972763 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d0b845ceb0 con 0x55d0b852c4d0 2023-07-14T11:36:16.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.046+0000 7f5d14ff9700 10 client.27103 handle_command_reply: tid=0 2023-07-14T11:36:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.046+0000 7f5d47227b80 1 client.27103 shutdown 2023-07-14T11:36:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.046+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b85124f0 msgr2=0x55d0b851c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.046+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b85124f0 0x55d0b851c720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000c00 tx=0x7f5d2800ed10 comp rx=0 tx=0).stop 2023-07-14T11:36:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 shutdown_connections 2023-07-14T11:36:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b852e8e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b851cd30 0x55d0b8519f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b85124f0 0x55d0b851c720 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d14ff9700 0 client.27103 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1065972763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8519930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 >> 172.21.15.161:0/1065972763 conn(0x55d0b8507ff0 msgr2=0x55d0b850d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 shutdown_connections 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 1 -- 172.21.15.161:0/1065972763 wait complete. 2023-07-14T11:36:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.047+0000 7f5d47227b80 20 client.27103 trim_cache size 0 max 16384 2023-07-14T11:36:16.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 Processor -- start 2023-07-14T11:36:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 -- start start 2023-07-14T11:36:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b851c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b85191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b85141a0 con 0x55d0b85124f0 2023-07-14T11:36:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b8517e90 con 0x55d0b8512e00 2023-07-14T11:36:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d47227b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b850eef0 con 0x55d0b851cd30 2023-07-14T11:36:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d30f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b851c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d3179b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d30f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b851c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45104/0 (socket says 172.21.15.161:45104) 2023-07-14T11:36:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d30f9a700 1 -- 172.21.15.161:0/2753373559 learned_addr learned my addr 172.21.15.161:0/2753373559 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.065+0000 7f5d3179b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85289e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38760/0 (socket says 172.21.15.161:38760) 2023-07-14T11:36:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d31f9c700 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b85191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d3179b700 1 -- 172.21.15.161:0/2753373559 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 msgr2=0x55d0b85191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d3179b700 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b85191d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d3179b700 1 -- 172.21.15.161:0/2753373559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 msgr2=0x55d0b851c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d3179b700 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b851c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d3179b700 1 -- 172.21.15.161:0/2753373559 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0b82b8250 con 0x55d0b85124f0 2023-07-14T11:36:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.066+0000 7f5d3179b700 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85289e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5d28004bc0 tx=0x7f5d28005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449163c7b8173b19 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.067+0000 7f5d17fff700 1 -- 172.21.15.161:0/2753373559 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28011bd0 con 0x55d0b85124f0 2023-07-14T11:36:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.067+0000 7f5d17fff700 1 -- 172.21.15.161:0/2753373559 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d2800e5c0 con 0x55d0b85124f0 2023-07-14T11:36:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.067+0000 7f5d17fff700 1 -- 172.21.15.161:0/2753373559 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28003d40 con 0x55d0b85124f0 2023-07-14T11:36:16.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.068+0000 7f5d47227b80 1 -- 172.21.15.161:0/2753373559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 msgr2=0x55d0b85289e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.068+0000 7f5d47227b80 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85289e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5d28004bc0 tx=0x7f5d28005e00 comp rx=0 tx=0).stop 2023-07-14T11:36:16.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.068+0000 7f5d47227b80 1 -- 172.21.15.161:0/2753373559 shutdown_connections 2023-07-14T11:36:16.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.068+0000 7f5d47227b80 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85289e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.068+0000 7f5d47227b80 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b85191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.068+0000 7f5d47227b80 1 --2- 172.21.15.161:0/2753373559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b851c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.068+0000 7f5d47227b80 1 -- 172.21.15.161:0/2753373559 >> 172.21.15.161:0/2753373559 conn(0x55d0b8507ff0 msgr2=0x55d0b850c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.069+0000 7f5d47227b80 1 -- 172.21.15.161:0/2753373559 shutdown_connections 2023-07-14T11:36:16.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.069+0000 7f5d47227b80 1 -- 172.21.15.161:0/2753373559 wait complete. 2023-07-14T11:36:16.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.070+0000 7f5d47227b80 1 -- start start 2023-07-14T11:36:16.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.070+0000 7f5d47227b80 1 Processor -- start 2023-07-14T11:36:16.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.070+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.070+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8521f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d47227b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b8583b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d47227b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b85840c0 con 0x55d0b85124f0 2023-07-14T11:36:16.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d47227b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b8584230 con 0x55d0b8512e00 2023-07-14T11:36:16.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d47227b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b85843f0 con 0x55d0b851cd30 2023-07-14T11:36:16.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d47227b80 10 client.0 _ll_register_callbacks cb 0x7f5d28009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:16.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d30f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8521f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d31f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b8583b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d31f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b8583b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41588/0 (socket says 172.21.15.161:41588) 2023-07-14T11:36:16.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d30f9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8521f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45114/0 (socket says 172.21.15.161:45114) 2023-07-14T11:36:16.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d31f9c700 1 -- 172.21.15.161:0/1321619658 learned_addr learned my addr 172.21.15.161:0/1321619658 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d30f9a700 1 -- 172.21.15.161:0/1321619658 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 msgr2=0x55d0b8583b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b8583b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d30f9a700 1 -- 172.21.15.161:0/1321619658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 msgr2=0x55d0b85219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d30f9a700 1 -- 172.21.15.161:0/1321619658 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0b827ac60 con 0x55d0b8512e00 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.071+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.072+0000 7f5d30f9a700 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8521f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c097ad0 tx=0x7f5d1c05bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b20540d225cfb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.072+0000 7f5d14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.072+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1321619658 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c05f070 con 0x55d0b8512e00 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.072+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1321619658 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d1c09b060 con 0x55d0b8512e00 2023-07-14T11:36:16.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.072+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d28019c50 con 0x55d0b8512e00 2023-07-14T11:36:16.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.072+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1321619658 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d1c09a9c0 con 0x55d0b8512e00 2023-07-14T11:36:16.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.073+0000 7f5d47227b80 10 client.27101 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:16.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.073+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d0b82902b0 con 0x55d0b8512e00 2023-07-14T11:36:16.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.073+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1321619658 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1c0c9080 con 0x55d0b8512e00 2023-07-14T11:36:16.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1321619658 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5d1c0b9200 con 0x55d0b8512e00 2023-07-14T11:36:16.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d47227b80 10 client.27101 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:16.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d47227b80 10 client.27101 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:36:16.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d47227b80 20 client.27101 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:16.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b859c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d47227b80 4 client.27101 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:36:16.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d0b841f050 con 0x55d0b852c4d0 2023-07-14T11:36:16.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.074+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b859c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.075+0000 7f5d3179b700 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b859c750 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2722e7182c58c061 server_cookie=75a16b279a7ff0d2 in_seq=0 out_seq=0 2023-07-14T11:36:16.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.075+0000 7f5d14ff9700 10 client.27101 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:16.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.079+0000 7f5d14ff9700 1 -- 172.21.15.161:0/1321619658 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d0b841f050 con 0x55d0b852c4d0 2023-07-14T11:36:16.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.079+0000 7f5d14ff9700 10 client.27101 handle_command_reply: tid=0 2023-07-14T11:36:16.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.079+0000 7f5d47227b80 1 client.27101 shutdown 2023-07-14T11:36:16.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 msgr2=0x55d0b8521f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8521f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c097ad0 tx=0x7f5d1c05bf00 comp rx=0 tx=0).stop 2023-07-14T11:36:16.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 shutdown_connections 2023-07-14T11:36:16.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d0b852c4d0 0x55d0b859c750 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0b85124f0 0x55d0b85219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0b851cd30 0x55d0b8583b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 --2- 172.21.15.161:0/1321619658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b8512e00 0x55d0b8521f20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d14ff9700 0 client.27101 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:16.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 >> 172.21.15.161:0/1321619658 conn(0x55d0b8507ff0 msgr2=0x55d0b8514cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 shutdown_connections 2023-07-14T11:36:16.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 1 -- 172.21.15.161:0/1321619658 wait complete. 2023-07-14T11:36:16.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.080+0000 7f5d47227b80 20 client.27101 trim_cache size 0 max 16384 2023-07-14T11:36:16.085 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:36:16.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d180a1360 msgr2=0x7f5d180a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d180a1360 0x7f5d180a3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009ec0 tx=0x7f5d2400b040 comp rx=0 tx=0).stop 2023-07-14T11:36:16.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 msgr2=0x7f5d2c19df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c19df50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005910 tx=0x7f5d1c004710 comp rx=0 tx=0).stop 2023-07-14T11:36:16.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 shutdown_connections 2023-07-14T11:36:16.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d180a1360 0x7f5d180a3810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c072850 0x7f5d2c19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d2c10c2a0 0x7f5d2c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 --2- 172.21.15.161:0/283410780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d2c073160 0x7f5d2c19df50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 >> 172.21.15.161:0/283410780 conn(0x7f5d2c0fb500 msgr2=0x7f5d2c103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 shutdown_connections 2023-07-14T11:36:16.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.081+0000 7f5d3279d700 1 -- 172.21.15.161:0/283410780 wait complete. 2023-07-14T11:36:16.117 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:36:16.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 -- 172.21.15.161:0/2255165590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec80ffad0 msgr2=0x7feec80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 --2- 172.21.15.161:0/2255165590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec80ffad0 0x7feec80fff40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009990 tx=0x7feeb8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 -- 172.21.15.161:0/2255165590 shutdown_connections 2023-07-14T11:36:16.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 --2- 172.21.15.161:0/2255165590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec8105ae0 0x7feec8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 --2- 172.21.15.161:0/2255165590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec8100480 0x7feec810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 --2- 172.21.15.161:0/2255165590 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec80ffad0 0x7feec80fff40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 -- 172.21.15.161:0/2255165590 >> 172.21.15.161:0/2255165590 conn(0x7feec80fb5f0 msgr2=0x7feec80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 -- 172.21.15.161:0/2255165590 shutdown_connections 2023-07-14T11:36:16.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.408+0000 7feed07ff700 1 -- 172.21.15.161:0/2255165590 wait complete. 2023-07-14T11:36:16.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.409+0000 7feed07ff700 1 Processor -- start 2023-07-14T11:36:16.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.409+0000 7feed07ff700 1 -- start start 2023-07-14T11:36:16.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feed07ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec80ffad0 0x7feec8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feed07ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec8100480 0x7feec81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feed07ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec8105ae0 0x7feec819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feed07ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec819ce50 con 0x7feec8105ae0 2023-07-14T11:36:16.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feed07ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec819cfc0 con 0x7feec80ffad0 2023-07-14T11:36:16.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feed07ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec819d180 con 0x7feec8100480 2023-07-14T11:36:16.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec80ffad0 0x7feec8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feecdd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec8100480 0x7feec81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec80ffad0 0x7feec8197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45124/0 (socket says 172.21.15.161:45124) 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feece59b700 1 -- 172.21.15.161:0/3702028947 learned_addr learned my addr 172.21.15.161:0/3702028947 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.410+0000 7feeced9c700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec8105ae0 0x7feec819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feecdd9a700 1 -- 172.21.15.161:0/3702028947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec80ffad0 msgr2=0x7feec8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feecdd9a700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec80ffad0 0x7feec8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feecdd9a700 1 -- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec8105ae0 msgr2=0x7feec819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feecdd9a700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec8105ae0 0x7feec819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feecdd9a700 1 -- 172.21.15.161:0/3702028947 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80095f0 con 0x7feec8100480 2023-07-14T11:36:16.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feeced9c700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec8105ae0 0x7feec819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:16.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feecdd9a700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec8100480 0x7feec81983b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feebf7fe700 1 -- 172.21.15.161:0/3702028947 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb801d070 con 0x7feec8100480 2023-07-14T11:36:16.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feebf7fe700 1 -- 172.21.15.161:0/3702028947 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feeb8005050 con 0x7feec8100480 2023-07-14T11:36:16.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feebf7fe700 1 -- 172.21.15.161:0/3702028947 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8004a20 con 0x7feec8100480 2023-07-14T11:36:16.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec819d400 con 0x7feec8100480 2023-07-14T11:36:16.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.411+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec819d680 con 0x7feec8100480 2023-07-14T11:36:16.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 Processor -- start 2023-07-14T11:36:16.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 -- start start 2023-07-14T11:36:16.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d9532c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d9532eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95227cb0 con 0x558d95321be0 2023-07-14T11:36:16.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9527aa40 con 0x558d953212d0 2023-07-14T11:36:16.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.417+0000 7feee415db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95273ed0 con 0x558d9532cad0 2023-07-14T11:36:16.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d9532c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d9532c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38782/0 (socket says 172.21.15.161:38782) 2023-07-14T11:36:16.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 -- 172.21.15.161:0/3513958210 learned_addr learned my addr 172.21.15.161:0/3513958210 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feebf7fe700 1 -- 172.21.15.161:0/3702028947 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7feeb80051c0 con 0x7feec8100480 2023-07-14T11:36:16.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feeced9c700 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feecdd9a700 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d9532eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 -- 172.21.15.161:0/3513958210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 msgr2=0x558d9532eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d9532eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 -- 172.21.15.161:0/3513958210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 msgr2=0x558d953216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feece59b700 1 -- 172.21.15.161:0/3513958210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d950a1de0 con 0x558d95321be0 2023-07-14T11:36:16.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.418+0000 7feecdd9a700 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d9532eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:16.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.419+0000 7feece59b700 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d9532c590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7feec400be20 tx=0x7feec40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf947aa65bdc4ca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.419+0000 7feebf7fe700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7feeb40a12c0 0x7feeb40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.420+0000 7feebcff9700 1 -- 172.21.15.161:0/3513958210 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011440 con 0x558d95321be0 2023-07-14T11:36:16.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.420+0000 7feebf7fe700 1 -- 172.21.15.161:0/3702028947 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7feeb80cdab0 con 0x7feec8100480 2023-07-14T11:36:16.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.420+0000 7feebcff9700 1 -- 172.21.15.161:0/3513958210 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec4015070 con 0x558d95321be0 2023-07-14T11:36:16.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.420+0000 7feebcff9700 1 -- 172.21.15.161:0/3513958210 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40197f0 con 0x558d95321be0 2023-07-14T11:36:16.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.420+0000 7feeced9c700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7feeb40a12c0 0x7feeb40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 -- 172.21.15.161:0/3513958210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 msgr2=0x558d9532c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feeced9c700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7feeb40a12c0 0x7feeb40a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feec0009df0 tx=0x7feec0009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d9532c590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7feec400be20 tx=0x7feec40106b0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 -- 172.21.15.161:0/3513958210 shutdown_connections 2023-07-14T11:36:16.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d9532eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 --2- 172.21.15.161:0/3513958210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d9532c590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 -- 172.21.15.161:0/3513958210 >> 172.21.15.161:0/3513958210 conn(0x558d95317da0 msgr2=0x558d9531d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 -- 172.21.15.161:0/3513958210 shutdown_connections 2023-07-14T11:36:16.426 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.421+0000 7feee415db80 1 -- 172.21.15.161:0/3513958210 wait complete. 2023-07-14T11:36:16.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.422+0000 7feee415db80 1 -- start start 2023-07-14T11:36:16.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.422+0000 7feee415db80 1 Processor -- start 2023-07-14T11:36:16.427 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feee415db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feee415db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d953297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feee415db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d95329ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feee415db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95213ac0 con 0x558d95321be0 2023-07-14T11:36:16.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feee415db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d952137c0 con 0x558d953212d0 2023-07-14T11:36:16.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feee415db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d950f7920 con 0x558d9532cad0 2023-07-14T11:36:16.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feee415db80 10 client.0 _ll_register_callbacks cb 0x558d95211760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:16.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.425+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45154/0 (socket says 172.21.15.161:45154) 2023-07-14T11:36:16.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feece59b700 1 -- 172.21.15.161:0/2534280391 learned_addr learned my addr 172.21.15.161:0/2534280391 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feeced9c700 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d95329ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feece59b700 1 -- 172.21.15.161:0/2534280391 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 msgr2=0x558d95329ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feece59b700 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d95329ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feece59b700 1 -- 172.21.15.161:0/2534280391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 msgr2=0x558d953297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feece59b700 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d953297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.426+0000 7feece59b700 1 -- 172.21.15.161:0/2534280391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec400ba50 con 0x558d953212d0 2023-07-14T11:36:16.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feece59b700 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532c550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feec400b910 tx=0x7feec4010d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0a31f79086764f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feeb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:36:16.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2534280391 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4005040 con 0x558d953212d0 2023-07-14T11:36:16.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2534280391 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7feec40051a0 con 0x558d953212d0 2023-07-14T11:36:16.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d951cd4f0 con 0x558d953212d0 2023-07-14T11:36:16.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2534280391 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec4005360 con 0x558d953212d0 2023-07-14T11:36:16.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feee415db80 10 client.27113 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:16.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.427+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558d9508ac60 con 0x558d953212d0 2023-07-14T11:36:16.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2534280391 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d951cd4f0 con 0x558d953212d0 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2534280391 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feec4004910 con 0x558d953212d0 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feee415db80 10 client.27113 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feee415db80 10 client.27113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feee415db80 20 client.27113 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feee415db80 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d9533c3b0 0x558d9533e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feee415db80 4 client.27113 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.428+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d9526ceb0 con 0x558d9533c3b0 2023-07-14T11:36:16.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.429+0000 7feecdd9a700 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d9533c3b0 0x558d9533e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.429+0000 7feeb1ffb700 10 client.27113 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:16.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.429+0000 7feecdd9a700 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d9533c3b0 0x558d9533e7c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a8eb3ebf3a82d server_cookie=af31be9298e8cebe in_seq=0 out_seq=0 2023-07-14T11:36:16.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.430+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2534280391 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558d9526ceb0 con 0x558d9533c3b0 2023-07-14T11:36:16.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.430+0000 7feeb1ffb700 10 client.27113 handle_command_reply: tid=0 2023-07-14T11:36:16.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.431+0000 7feee415db80 1 client.27113 shutdown 2023-07-14T11:36:16.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.431+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 msgr2=0x558d9532c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.431+0000 7feee415db80 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532c550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feec400b910 tx=0x7feec4010d20 comp rx=0 tx=0).stop 2023-07-14T11:36:16.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 shutdown_connections 2023-07-14T11:36:16.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feee415db80 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d9533c3b0 0x558d9533e7c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feee415db80 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d95329ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feee415db80 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532c550 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feee415db80 1 --2- 172.21.15.161:0/2534280391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d953297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 >> 172.21.15.161:0/2534280391 conn(0x558d95317da0 msgr2=0x558d9531d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feeb1ffb700 0 client.27113 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.432+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 shutdown_connections 2023-07-14T11:36:16.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.433+0000 7feee415db80 1 -- 172.21.15.161:0/2534280391 wait complete. 2023-07-14T11:36:16.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.433+0000 7feee415db80 20 client.27113 trim_cache size 0 max 16384 2023-07-14T11:36:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.454+0000 7feee415db80 1 Processor -- start 2023-07-14T11:36:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.454+0000 7feee415db80 1 -- start start 2023-07-14T11:36:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feee415db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feee415db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d95321be0 0x558d9532c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feee415db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d9532cad0 0x558d95324af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feee415db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95317970 con 0x558d9532cad0 2023-07-14T11:36:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feee415db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95322dd0 con 0x558d953212d0 2023-07-14T11:36:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feee415db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95327d50 con 0x558d95321be0 2023-07-14T11:36:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feecdd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d95321be0 0x558d9532c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45170/0 (socket says 172.21.15.161:45170) 2023-07-14T11:36:16.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.455+0000 7feece59b700 1 -- 172.21.15.161:0/1708613743 learned_addr learned my addr 172.21.15.161:0/1708613743 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.456+0000 7feece59b700 1 -- 172.21.15.161:0/1708613743 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d95321be0 msgr2=0x558d9532c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.456+0000 7feece59b700 1 --2- 172.21.15.161:0/1708613743 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d95321be0 0x558d9532c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.456+0000 7feece59b700 1 -- 172.21.15.161:0/1708613743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d9532cad0 msgr2=0x558d95324af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.456+0000 7feece59b700 1 --2- 172.21.15.161:0/1708613743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d9532cad0 0x558d95324af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.456+0000 7feece59b700 1 -- 172.21.15.161:0/1708613743 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d950c8250 con 0x558d953212d0 2023-07-14T11:36:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.457+0000 7feece59b700 1 --2- 172.21.15.161:0/1708613743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532eeb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17f983689c5ecff3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.459+0000 7feebd7fa700 1 -- 172.21.15.161:0/1708613743 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4010ab0 con 0x558d953212d0 2023-07-14T11:36:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.459+0000 7feebd7fa700 1 -- 172.21.15.161:0/1708613743 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec400b500 con 0x558d953212d0 2023-07-14T11:36:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.459+0000 7feebd7fa700 1 -- 172.21.15.161:0/1708613743 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40116a0 con 0x558d953212d0 2023-07-14T11:36:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.460+0000 7feee415db80 1 -- 172.21.15.161:0/1708613743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 msgr2=0x558d9532eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.460+0000 7feee415db80 1 --2- 172.21.15.161:0/1708613743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532eeb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec400bfd0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.460+0000 7feee415db80 1 -- 172.21.15.161:0/1708613743 shutdown_connections 2023-07-14T11:36:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.460+0000 7feee415db80 1 --2- 172.21.15.161:0/1708613743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d9532cad0 0x558d95324af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.460+0000 7feee415db80 1 --2- 172.21.15.161:0/1708613743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d9532eeb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.460+0000 7feee415db80 1 --2- 172.21.15.161:0/1708613743 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d95321be0 0x558d9532c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.460+0000 7feee415db80 1 -- 172.21.15.161:0/1708613743 >> 172.21.15.161:0/1708613743 conn(0x558d95317da0 msgr2=0x558d9531be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.461+0000 7feee415db80 1 -- 172.21.15.161:0/1708613743 shutdown_connections 2023-07-14T11:36:16.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.461+0000 7feee415db80 1 -- 172.21.15.161:0/1708613743 wait complete. 2023-07-14T11:36:16.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.461+0000 7feee415db80 1 -- start start 2023-07-14T11:36:16.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.461+0000 7feee415db80 1 Processor -- start 2023-07-14T11:36:16.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feee415db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feee415db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d95331c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feee415db80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d95391a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feee415db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95391f90 con 0x558d95321be0 2023-07-14T11:36:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feee415db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d953920d0 con 0x558d953212d0 2023-07-14T11:36:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feee415db80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d95392240 con 0x558d9532cad0 2023-07-14T11:36:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feee415db80 10 client.0 _ll_register_callbacks cb 0x7feec4005910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feece59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953316c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45178/0 (socket says 172.21.15.161:45178) 2023-07-14T11:36:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.462+0000 7feece59b700 1 -- 172.21.15.161:0/2738398251 learned_addr learned my addr 172.21.15.161:0/2738398251 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feece59b700 1 -- 172.21.15.161:0/2738398251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 msgr2=0x558d95391a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:36:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feecdd9a700 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d95331c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feece59b700 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d95391a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feece59b700 1 -- 172.21.15.161:0/2738398251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 msgr2=0x558d95331c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:16.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feece59b700 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d95331c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feece59b700 1 -- 172.21.15.161:0/2738398251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d9508ac60 con 0x558d953212d0 2023-07-14T11:36:16.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feecdd9a700 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d95331c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:16.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.463+0000 7feece59b700 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953316c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feec4011e30 tx=0x7feec402b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a8bf16f2901f0be server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:16.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feeb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:36:16.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2738398251 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011b60 con 0x558d953212d0 2023-07-14T11:36:16.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec4015070 con 0x558d953212d0 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2738398251 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7feec4000e40 con 0x558d953212d0 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2738398251 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec40193d0 con 0x558d953212d0 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feee415db80 10 client.27119 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558d950a02b0 con 0x558d953212d0 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.464+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2738398251 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec4015070 con 0x558d953212d0 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2738398251 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feec4003af0 con 0x558d953212d0 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feee415db80 10 client.27119 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:16.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feee415db80 10 client.27119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:16.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feee415db80 20 client.27119 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:16.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feee415db80 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d953aa5d0 0x558d953ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:16.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feee415db80 4 client.27119 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:36:16.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d9522f050 con 0x558d953aa5d0 2023-07-14T11:36:16.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.465+0000 7feecdd9a700 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d953aa5d0 0x558d953ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:16.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.466+0000 7feeb1ffb700 10 client.27119 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:16.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.466+0000 7feecdd9a700 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d953aa5d0 0x558d953ac9b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccec39001d7866a server_cookie=1f2de035fcfed33e in_seq=0 out_seq=0 2023-07-14T11:36:16.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feeb1ffb700 1 -- 172.21.15.161:0/2738398251 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d9522f050 con 0x558d953aa5d0 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feeb1ffb700 10 client.27119 handle_command_reply: tid=0 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 client.27119 shutdown 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 msgr2=0x558d953316c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953316c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feec4011e30 tx=0x7feec402b410 comp rx=0 tx=0).stop 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 shutdown_connections 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d953aa5d0 0x558d953ac9b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d9532cad0 0x558d95391a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d953212d0 0x558d953316c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 --2- 172.21.15.161:0/2738398251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d95321be0 0x558d95331c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.468+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 >> 172.21.15.161:0/2738398251 conn(0x558d95317da0 msgr2=0x558d95324af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.469+0000 7feeb1ffb700 0 client.27119 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:16.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.469+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 shutdown_connections 2023-07-14T11:36:16.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.469+0000 7feee415db80 1 -- 172.21.15.161:0/2738398251 wait complete. 2023-07-14T11:36:16.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.469+0000 7feee415db80 20 client.27119 trim_cache size 0 max 16384 2023-07-14T11:36:16.475 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:36:16.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7feeb40a12c0 msgr2=0x7feeb40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7feeb40a12c0 0x7feeb40a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feec0009df0 tx=0x7feec0009040 comp rx=0 tx=0).stop 2023-07-14T11:36:16.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec8100480 msgr2=0x7feec81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:16.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec8100480 0x7feec81983b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb800f690 comp rx=0 tx=0).stop 2023-07-14T11:36:16.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 shutdown_connections 2023-07-14T11:36:16.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7feeb40a12c0 0x7feeb40a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec8105ae0 0x7feec819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec80ffad0 0x7feec8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 --2- 172.21.15.161:0/3702028947 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7feec8100480 0x7feec81983b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:16.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 >> 172.21.15.161:0/3702028947 conn(0x7feec80fb5f0 msgr2=0x7feec810ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:16.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 shutdown_connections 2023-07-14T11:36:16.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:16.470+0000 7feed07ff700 1 -- 172.21.15.161:0/3702028947 wait complete. 2023-07-14T11:36:16.492 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:16 smithi161 ceph-mon[95526]: pgmap v439: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 280 op/s 2023-07-14T11:36:16.561 INFO:tasks.workunit.client.0.smithi161.stdout: 10 37046 37.32 MB/sec warmup 58 sec latency 193.754 ms 2023-07-14T11:36:16.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:16 smithi175 ceph-mon[129081]: pgmap v439: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 280 op/s 2023-07-14T11:36:16.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:16 smithi182 ceph-mon[129366]: pgmap v439: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 280 op/s 2023-07-14T11:36:17.495 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:36:17.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:36:17.561 INFO:tasks.workunit.client.0.smithi161.stdout: 10 37732 37.35 MB/sec warmup 59 sec latency 225.827 ms 2023-07-14T11:36:17.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.769+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2946367209 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40fe820 msgr2=0x7f13b40febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.769+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2946367209 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40fe820 0x7f13b40febf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009990 tx=0x7f13b0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.770+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2946367209 shutdown_connections 2023-07-14T11:36:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.770+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2946367209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b410c3d0 0x7f13b410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.770+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2946367209 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40fe820 0x7f13b40febf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.770+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2946367209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b40ff1c0 0x7f13b410be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.770+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2946367209 >> 172.21.15.161:0/2946367209 conn(0x7f13b40fa510 msgr2=0x7f13b40fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:17.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.770+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2946367209 shutdown_connections 2023-07-14T11:36:17.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.770+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2946367209 wait complete. 2023-07-14T11:36:17.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.771+0000 7f13bc3ff700 1 Processor -- start 2023-07-14T11:36:17.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.771+0000 7f13bc3ff700 1 -- start start 2023-07-14T11:36:17.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13bc3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b40fe820 0x7f13b4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13bc3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40ff1c0 0x7f13b4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13bc3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b410c3d0 0x7f13b419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13bc3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419cf30 con 0x7f13b410c3d0 2023-07-14T11:36:17.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13bc3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d0a0 con 0x7f13b40fe820 2023-07-14T11:36:17.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13bc3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d260 con 0x7f13b40ff1c0 2023-07-14T11:36:17.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13b999a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40ff1c0 0x7f13b4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13ba19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b40fe820 0x7f13b4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13ba99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b410c3d0 0x7f13b419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13b999a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40ff1c0 0x7f13b4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41642/0 (socket says 172.21.15.161:41642) 2023-07-14T11:36:17.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.772+0000 7f13b999a700 1 -- 172.21.15.161:0/2087544271 learned_addr learned my addr 172.21.15.161:0/2087544271 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:17.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13b999a700 1 -- 172.21.15.161:0/2087544271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b40fe820 msgr2=0x7f13b4197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13b999a700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b40fe820 0x7f13b4197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13b999a700 1 -- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b410c3d0 msgr2=0x7f13b419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13b999a700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b410c3d0 0x7f13b419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13b999a700 1 -- 172.21.15.161:0/2087544271 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b00095f0 con 0x7f13b40ff1c0 2023-07-14T11:36:17.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13b999a700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40ff1c0 0x7f13b4198490 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f13a800b530 tx=0x7f13a800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:17.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13a77fe700 1 -- 172.21.15.161:0/2087544271 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8011830 con 0x7f13b40ff1c0 2023-07-14T11:36:17.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b419d540 con 0x7f13b40ff1c0 2023-07-14T11:36:17.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13a77fe700 1 -- 172.21.15.161:0/2087544271 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a8011e50 con 0x7f13b40ff1c0 2023-07-14T11:36:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.773+0000 7f13a77fe700 1 -- 172.21.15.161:0/2087544271 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8010e70 con 0x7f13b40ff1c0 2023-07-14T11:36:17.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.774+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b41a4d80 con 0x7f13b40ff1c0 2023-07-14T11:36:17.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.777+0000 7f13a77fe700 1 -- 172.21.15.161:0/2087544271 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f13a8011990 con 0x7f13b40ff1c0 2023-07-14T11:36:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.778+0000 7f13cfd78b80 1 Processor -- start 2023-07-14T11:36:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.778+0000 7f13cfd78b80 1 -- start start 2023-07-14T11:36:17.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 0x558425f4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f4cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425e45d30 con 0x558425f4ab50 2023-07-14T11:36:17.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425e98ac0 con 0x558425f3fc60 2023-07-14T11:36:17.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425e91f50 con 0x558425f3f350 2023-07-14T11:36:17.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f3f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41648/0 (socket says 172.21.15.161:41648) 2023-07-14T11:36:17.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 -- 172.21.15.161:0/3563726976 learned_addr learned my addr 172.21.15.161:0/3563726976 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:17.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13b999a700 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f4cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 -- 172.21.15.161:0/3563726976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 msgr2=0x558425f4a610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:36:17.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 0x558425f4a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 -- 172.21.15.161:0/3563726976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 msgr2=0x558425f4cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:17.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f4cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.779+0000 7f13ba99c700 1 -- 172.21.15.161:0/3563726976 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558425cbfe60 con 0x558425f3f350 2023-07-14T11:36:17.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.780+0000 7f13ba99c700 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f3f720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0115c0 tx=0x7f13ac009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e468f5ebbc9031 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:17.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.780+0000 7f13a4bfe700 1 -- 172.21.15.161:0/3563726976 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00e040 con 0x558425f3f350 2023-07-14T11:36:17.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.780+0000 7f13a77fe700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13a00a1340 0x7f13a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.780+0000 7f13a4bfe700 1 -- 172.21.15.161:0/3563726976 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac00f040 con 0x558425f3f350 2023-07-14T11:36:17.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.780+0000 7f13a4bfe700 1 -- 172.21.15.161:0/3563726976 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0048d0 con 0x558425f3f350 2023-07-14T11:36:17.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.780+0000 7f13a77fe700 1 -- 172.21.15.161:0/2087544271 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f13a80cc840 con 0x7f13b40ff1c0 2023-07-14T11:36:17.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.780+0000 7f13ba19b700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13a00a1340 0x7f13a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13ba19b700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13a00a1340 0x7f13a00a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005f50 tx=0x7f13b0005e00 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:17.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 -- 172.21.15.161:0/3563726976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 msgr2=0x558425f3f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f3f720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0115c0 tx=0x7f13ac009920 comp rx=0 tx=0).stop 2023-07-14T11:36:17.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 -- 172.21.15.161:0/3563726976 shutdown_connections 2023-07-14T11:36:17.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f4cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f3f720 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/3563726976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 0x558425f4a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 -- 172.21.15.161:0/3563726976 >> 172.21.15.161:0/3563726976 conn(0x558425f35e20 msgr2=0x558425f3b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:17.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.781+0000 7f13cfd78b80 1 -- 172.21.15.161:0/3563726976 shutdown_connections 2023-07-14T11:36:17.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.782+0000 7f13cfd78b80 1 -- 172.21.15.161:0/3563726976 wait complete. 2023-07-14T11:36:17.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.782+0000 7f13cfd78b80 1 -- start start 2023-07-14T11:36:17.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.782+0000 7f13cfd78b80 1 Processor -- start 2023-07-14T11:36:17.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f4a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 0x558425f477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f47e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425e31b40 con 0x558425f4ab50 2023-07-14T11:36:17.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425e31840 con 0x558425f3fc60 2023-07-14T11:36:17.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425d159a0 con 0x558425f3f350 2023-07-14T11:36:17.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13cfd78b80 10 client.0 _ll_register_callbacks cb 0x558425e2f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:17.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f4a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f4a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41652/0 (socket says 172.21.15.161:41652) 2023-07-14T11:36:17.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba19b700 1 -- 172.21.15.161:0/1639979435 learned_addr learned my addr 172.21.15.161:0/1639979435 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:17.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba99c700 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f47e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba19b700 1 -- 172.21.15.161:0/1639979435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 msgr2=0x558425f477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:17.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba19b700 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 0x558425f477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba19b700 1 -- 172.21.15.161:0/1639979435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 msgr2=0x558425f47e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:17.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.783+0000 7f13ba19b700 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f47e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.784+0000 7f13ba19b700 1 -- 172.21.15.161:0/1639979435 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac011270 con 0x558425f3f350 2023-07-14T11:36:17.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.784+0000 7f13ba19b700 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f4a570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13b0015000 tx=0x7f13b0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e47db0685f0953 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:17.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.784+0000 7f139dffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:17.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.784+0000 7f139dffb700 1 -- 172.21.15.161:0/1639979435 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000f460 con 0x558425f3f350 2023-07-14T11:36:17.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.784+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558425deb570 con 0x558425f3f350 2023-07-14T11:36:17.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.784+0000 7f139dffb700 1 -- 172.21.15.161:0/1639979435 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f13b000f5c0 con 0x558425f3f350 2023-07-14T11:36:17.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f139dffb700 1 -- 172.21.15.161:0/1639979435 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b000f780 con 0x558425f3f350 2023-07-14T11:36:17.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 10 client.27136 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:17.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558425ca8ce0 con 0x558425f3f350 2023-07-14T11:36:17.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f139dffb700 1 -- 172.21.15.161:0/1639979435 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558425deb570 con 0x558425f3f350 2023-07-14T11:36:17.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f139dffb700 1 -- 172.21.15.161:0/1639979435 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f13b0037070 con 0x558425f3f350 2023-07-14T11:36:17.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 10 client.27136 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:17.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 10 client.27136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:17.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 20 client.27136 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425f62cf0 0x558425f65100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 4 client.27136 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558425e8af30 con 0x558425f62cf0 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.785+0000 7f13b999a700 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425f62cf0 0x558425f65100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.786+0000 7f13b999a700 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425f62cf0 0x558425f65100 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8271dae9f85726 server_cookie=4f0c520a278c2c92 in_seq=0 out_seq=0 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.786+0000 7f139dffb700 10 client.27136 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.790+0000 7f139dffb700 1 -- 172.21.15.161:0/1639979435 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558425e8af30 con 0x558425f62cf0 2023-07-14T11:36:17.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.790+0000 7f139dffb700 10 client.27136 handle_command_reply: tid=0 2023-07-14T11:36:17.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.790+0000 7f13cfd78b80 1 client.27136 shutdown 2023-07-14T11:36:17.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.790+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 msgr2=0x558425f4a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.790+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f4a570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13b0015000 tx=0x7f13b0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 shutdown_connections 2023-07-14T11:36:17.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425f62cf0 0x558425f65100 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f47e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f4a570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1639979435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3fc60 0x558425f477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f139dffb700 0 client.27136 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:17.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 >> 172.21.15.161:0/1639979435 conn(0x558425f35e20 msgr2=0x558425f3b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:17.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 shutdown_connections 2023-07-14T11:36:17.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1639979435 wait complete. 2023-07-14T11:36:17.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.791+0000 7f13cfd78b80 20 client.27136 trim_cache size 0 max 16384 2023-07-14T11:36:17.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 Processor -- start 2023-07-14T11:36:17.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 -- start start 2023-07-14T11:36:17.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3f350 0x558425f4a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3fc60 0x558425f470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425f40240 con 0x558425f4ab50 2023-07-14T11:36:17.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425f603a0 con 0x558425f3f350 2023-07-14T11:36:17.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.817+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425f45db0 con 0x558425f3fc60 2023-07-14T11:36:17.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3fc60 0x558425f470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3fc60 0x558425f470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46494/0 (socket says 172.21.15.161:46494) 2023-07-14T11:36:17.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 -- 172.21.15.161:0/1809838279 learned_addr learned my addr 172.21.15.161:0/1809838279 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:17.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13ba99c700 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13ba19b700 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3f350 0x558425f4a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 -- 172.21.15.161:0/1809838279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3f350 msgr2=0x558425f4a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:17.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3f350 0x558425f4a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 -- 172.21.15.161:0/1809838279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 msgr2=0x558425f475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:17.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.818+0000 7f13b999a700 1 -- 172.21.15.161:0/1809838279 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558425ce62d0 con 0x558425f3fc60 2023-07-14T11:36:17.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.819+0000 7f13b999a700 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3fc60 0x558425f470b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13a800bb90 tx=0x7f13a80b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdd0d2cb3a3de138 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:17.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.819+0000 7f13a53ff700 1 -- 172.21.15.161:0/1809838279 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a809d690 con 0x558425f3fc60 2023-07-14T11:36:17.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.819+0000 7f13a53ff700 1 -- 172.21.15.161:0/1809838279 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a80a1ab0 con 0x558425f3fc60 2023-07-14T11:36:17.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.819+0000 7f13a53ff700 1 -- 172.21.15.161:0/1809838279 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a80cc6a0 con 0x558425f3fc60 2023-07-14T11:36:17.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1809838279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3fc60 msgr2=0x558425f470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3fc60 0x558425f470b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13a800bb90 tx=0x7f13a80b9040 comp rx=0 tx=0).stop 2023-07-14T11:36:17.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1809838279 shutdown_connections 2023-07-14T11:36:17.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f4ab50 0x558425f475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f3f350 0x558425f4a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/1809838279 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3fc60 0x558425f470b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1809838279 >> 172.21.15.161:0/1809838279 conn(0x558425f35e20 msgr2=0x558425f5f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:17.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1809838279 shutdown_connections 2023-07-14T11:36:17.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.820+0000 7f13cfd78b80 1 -- 172.21.15.161:0/1809838279 wait complete. 2023-07-14T11:36:17.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.821+0000 7f13cfd78b80 1 -- start start 2023-07-14T11:36:17.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.821+0000 7f13cfd78b80 1 Processor -- start 2023-07-14T11:36:17.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f58720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f3fc60 0x558425f58c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13cfd78b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f4ab50 0x558425f591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425fb6d10 con 0x558425f3fc60 2023-07-14T11:36:17.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425fb6e50 con 0x558425f4ab50 2023-07-14T11:36:17.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13cfd78b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558425fb7010 con 0x558425f3f350 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13cfd78b80 10 client.0 _ll_register_callbacks cb 0x7f13b000baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13ba19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f58720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13ba19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f58720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46508/0 (socket says 172.21.15.161:46508) 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.822+0000 7f13ba19b700 1 -- 172.21.15.161:0/791661731 learned_addr learned my addr 172.21.15.161:0/791661731 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba99c700 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f4ab50 0x558425f591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba19b700 1 -- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f4ab50 msgr2=0x558425f591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba19b700 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f4ab50 0x558425f591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba19b700 1 -- 172.21.15.161:0/791661731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f3fc60 msgr2=0x558425f58c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:17.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba19b700 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f3fc60 0x558425f58c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba19b700 1 -- 172.21.15.161:0/791661731 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b41a4d80 con 0x558425f3f350 2023-07-14T11:36:17.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba99c700 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f4ab50 0x558425f591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:17.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13ba19b700 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f58720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f13b0004bf0 tx=0x7f13b0024920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be60789de59e8824 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:17.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f139dffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:17.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f139dffb700 1 -- 172.21.15.161:0/791661731 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000f870 con 0x558425f3f350 2023-07-14T11:36:17.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13ac00f040 con 0x558425f3f350 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.823+0000 7f139dffb700 1 -- 172.21.15.161:0/791661731 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f13b0024cf0 con 0x558425f3f350 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f139dffb700 1 -- 172.21.15.161:0/791661731 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b000f9d0 con 0x558425f3f350 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 10 client.27142 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558425cbe330 con 0x558425f3f350 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f139dffb700 1 -- 172.21.15.161:0/791661731 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac00f040 con 0x558425f3f350 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f139dffb700 1 -- 172.21.15.161:0/791661731 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f13b0004500 con 0x558425f3f350 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 10 client.27142 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:17.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 10 client.27142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:17.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 20 client.27142 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:17.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425fb8370 0x558425fba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:17.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 4 client.27142 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:36:17.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.824+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558425e4d0d0 con 0x558425fb8370 2023-07-14T11:36:17.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.825+0000 7f13b999a700 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425fb8370 0x558425fba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:17.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.825+0000 7f13b999a700 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425fb8370 0x558425fba750 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5679adf94336a4d server_cookie=6ae0480e29b5b3eb in_seq=0 out_seq=0 2023-07-14T11:36:17.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.825+0000 7f139dffb700 10 client.27142 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:17.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f139dffb700 1 -- 172.21.15.161:0/791661731 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558425e4d0d0 con 0x558425fb8370 2023-07-14T11:36:17.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f139dffb700 10 client.27142 handle_command_reply: tid=0 2023-07-14T11:36:17.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 client.27142 shutdown 2023-07-14T11:36:17.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 msgr2=0x558425f58720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f58720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f13b0004bf0 tx=0x7f13b0024920 comp rx=0 tx=0).stop 2023-07-14T11:36:17.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 shutdown_connections 2023-07-14T11:36:17.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558425fb8370 0x558425fba750 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558425f4ab50 0x558425f591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558425f3f350 0x558425f58720 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 --2- 172.21.15.161:0/791661731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558425f3fc60 0x558425f58c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 >> 172.21.15.161:0/791661731 conn(0x558425f35e20 msgr2=0x558425f470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:17.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f139dffb700 0 client.27142 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:17.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 shutdown_connections 2023-07-14T11:36:17.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.831+0000 7f13cfd78b80 1 -- 172.21.15.161:0/791661731 wait complete. 2023-07-14T11:36:17.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13cfd78b80 20 client.27142 trim_cache size 0 max 16384 2023-07-14T11:36:17.836 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:36:17.836 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:36:17.836 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "304268cc-3b8b-4b03-be80-e16718671176", 2023-07-14T11:36:17.837 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:36:17.837 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:36:17.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13a00a1340 msgr2=0x7f13a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13a00a1340 0x7f13a00a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005f50 tx=0x7f13b0005e00 comp rx=0 tx=0).stop 2023-07-14T11:36:17.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40ff1c0 msgr2=0x7f13b4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:17.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40ff1c0 0x7f13b4198490 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f13a800b530 tx=0x7f13a800b840 comp rx=0 tx=0).stop 2023-07-14T11:36:17.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 shutdown_connections 2023-07-14T11:36:17.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13a00a1340 0x7f13a00a37f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b410c3d0 0x7f13b419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b40fe820 0x7f13b4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 --2- 172.21.15.161:0/2087544271 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13b40ff1c0 0x7f13b4198490 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:17.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 >> 172.21.15.161:0/2087544271 conn(0x7f13b40fa510 msgr2=0x7f13b410ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:17.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.832+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 shutdown_connections 2023-07-14T11:36:17.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:17.833+0000 7f13bc3ff700 1 -- 172.21.15.161:0/2087544271 wait complete. 2023-07-14T11:36:17.856 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:36:18.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.115+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/934324950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b640ff1c0 msgr2=0x7f4b64110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.115+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/934324950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b640ff1c0 0x7f4b64110710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009990 tx=0x7f4b54009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.118+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/934324950 shutdown_connections 2023-07-14T11:36:18.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.118+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/934324950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b640fe8b0 0x7f4b640fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.118+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/934324950 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b64104a80 0x7f4b64112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.118+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/934324950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b640ff1c0 0x7f4b64110710 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.118+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/934324950 >> 172.21.15.161:0/934324950 conn(0x7f4b640fa4e0 msgr2=0x7f4b640fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:18.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.118+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/934324950 shutdown_connections 2023-07-14T11:36:18.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.118+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/934324950 wait complete. 2023-07-14T11:36:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.120+0000 7f4b6c1ff700 1 Processor -- start 2023-07-14T11:36:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.120+0000 7f4b6c1ff700 1 -- start start 2023-07-14T11:36:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.121+0000 7f4b6c1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b640fe8b0 0x7f4b6419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.121+0000 7f4b6c1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b640ff1c0 0x7f4b6419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.121+0000 7f4b6c1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b64104a80 0x7f4b641a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.121+0000 7f4b6c1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641a1260 con 0x7f4b640fe8b0 2023-07-14T11:36:18.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.121+0000 7f4b6c1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641a13d0 con 0x7f4b64104a80 2023-07-14T11:36:18.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.121+0000 7f4b6c1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641a1590 con 0x7f4b640ff1c0 2023-07-14T11:36:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b6979a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b640ff1c0 0x7f4b6419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b6979a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b640ff1c0 0x7f4b6419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46532/0 (socket says 172.21.15.161:46532) 2023-07-14T11:36:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b6979a700 1 -- 172.21.15.161:0/694829267 learned_addr learned my addr 172.21.15.161:0/694829267 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b640fe8b0 0x7f4b6419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b69f9b700 1 -- 172.21.15.161:0/694829267 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b640ff1c0 msgr2=0x7f4b6419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b640ff1c0 0x7f4b6419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b69f9b700 1 -- 172.21.15.161:0/694829267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b64104a80 msgr2=0x7f4b641a0c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:36:18.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.122+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b64104a80 0x7f4b641a0c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.123+0000 7f4b69f9b700 1 -- 172.21.15.161:0/694829267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540095f0 con 0x7f4b640fe8b0 2023-07-14T11:36:18.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.123+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b640fe8b0 0x7f4b6419c280 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000d950 tx=0x7f4b6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:18.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.124+0000 7f4b5affd700 1 -- 172.21.15.161:0/694829267 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60009930 con 0x7f4b640fe8b0 2023-07-14T11:36:18.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.124+0000 7f4b5affd700 1 -- 172.21.15.161:0/694829267 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b60010460 con 0x7f4b640fe8b0 2023-07-14T11:36:18.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.125+0000 7f4b5affd700 1 -- 172.21.15.161:0/694829267 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000b5d0 con 0x7f4b640fe8b0 2023-07-14T11:36:18.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.125+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b641a1870 con 0x7f4b640fe8b0 2023-07-14T11:36:18.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.125+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b641a90b0 con 0x7f4b640fe8b0 2023-07-14T11:36:18.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b5affd700 1 -- 172.21.15.161:0/694829267 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4b60010820 con 0x7f4b640fe8b0 2023-07-14T11:36:18.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 Processor -- start 2023-07-14T11:36:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 -- start start 2023-07-14T11:36:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19b2d0 0x55920f19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f1a6ad0 0x55920f1a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f0a1cb0 con 0x55920f1a6ad0 2023-07-14T11:36:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f0f4a40 con 0x55920f19b2d0 2023-07-14T11:36:18.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.132+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f0eded0 con 0x55920f19bbe0 2023-07-14T11:36:18.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46546/0 (socket says 172.21.15.161:46546) 2023-07-14T11:36:18.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 -- 172.21.15.161:0/3779123780 learned_addr learned my addr 172.21.15.161:0/3779123780 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:18.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19b2d0 0x55920f19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 -- 172.21.15.161:0/3779123780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19b2d0 msgr2=0x55920f19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19b2d0 0x55920f19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 -- 172.21.15.161:0/3779123780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f1a6ad0 msgr2=0x55920f1a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f1a6ad0 0x55920f1a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.133+0000 7f4b69f9b700 1 -- 172.21.15.161:0/3779123780 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55920ef1bde0 con 0x55920f19bbe0 2023-07-14T11:36:18.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.134+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a6590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b600c9f50 tx=0x7f4b600d0ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed1d73649c938de8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:18.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.134+0000 7f4b4ffff700 1 -- 172.21.15.161:0/3779123780 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600d59e0 con 0x55920f19bbe0 2023-07-14T11:36:18.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.135+0000 7f4b4ffff700 1 -- 172.21.15.161:0/3779123780 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b600d8430 con 0x55920f19bbe0 2023-07-14T11:36:18.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.135+0000 7f4b4ffff700 1 -- 172.21.15.161:0/3779123780 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600d75f0 con 0x55920f19bbe0 2023-07-14T11:36:18.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.135+0000 7f4b5affd700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4b500a13c0 0x7f4b500a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b5affd700 1 -- 172.21.15.161:0/694829267 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4b600ccc80 con 0x7f4b640fe8b0 2023-07-14T11:36:18.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3779123780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 msgr2=0x55920f1a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a6590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b600c9f50 tx=0x7f4b600d0ba0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3779123780 shutdown_connections 2023-07-14T11:36:18.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f1a6ad0 0x55920f1a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19b2d0 0x55920f19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3779123780 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a6590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3779123780 >> 172.21.15.161:0/3779123780 conn(0x55920f191da0 msgr2=0x55920f1976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:18.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4b500a13c0 0x7f4b500a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.136+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3779123780 shutdown_connections 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.137+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3779123780 wait complete. 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.137+0000 7f4b7fa85b80 1 -- start start 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.137+0000 7f4b7fa85b80 1 Processor -- start 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.138+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.138+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.138+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f1a6ad0 0x55920f1b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.138+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f08dac0 con 0x55920f19b2d0 2023-07-14T11:36:18.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.138+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f08d7c0 con 0x55920f1a6ad0 2023-07-14T11:36:18.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.138+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920ef71920 con 0x55920f19bbe0 2023-07-14T11:36:18.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.138+0000 7f4b7fa85b80 10 client.0 _ll_register_callbacks cb 0x55920f08b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:18.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f1a6ad0 0x55920f1b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f1a6ad0 0x55920f1b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59032/0 (socket says 172.21.15.161:59032) 2023-07-14T11:36:18.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 -- 172.21.15.161:0/145453437 learned_addr learned my addr 172.21.15.161:0/145453437 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:18.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4b500a13c0 0x7f4b500a3870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b640673e0 tx=0x7f4b5c009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:18.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6979a700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 -- 172.21.15.161:0/145453437 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 msgr2=0x55920f1a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 -- 172.21.15.161:0/145453437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 msgr2=0x55920f1a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6a79c700 1 -- 172.21.15.161:0/145453437 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b641a90b0 con 0x55920f1a6ad0 2023-07-14T11:36:18.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.139+0000 7f4b6979a700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:18.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.140+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:18.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.140+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f1a6ad0 0x55920f1b1f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0199e0 tx=0x7f4b5c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485e69983b549ec4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:18.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.140+0000 7f4b4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:36:18.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.140+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/145453437 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c01e070 con 0x55920f1a6ad0 2023-07-14T11:36:18.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.140+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55920f0474f0 con 0x55920f1a6ad0 2023-07-14T11:36:18.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.140+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/145453437 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4b5c017410 con 0x55920f1a6ad0 2023-07-14T11:36:18.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.141+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/145453437 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5c028690 con 0x55920f1a6ad0 2023-07-14T11:36:18.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.141+0000 7f4b7fa85b80 10 client.27137 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:18.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.141+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55920ef04c60 con 0x55920f1a6ad0 2023-07-14T11:36:18.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.144+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/145453437 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55920f0474f0 con 0x55920f1a6ad0 2023-07-14T11:36:18.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.144+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/145453437 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4b5c017650 con 0x55920f1a6ad0 2023-07-14T11:36:18.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.145+0000 7f4b7fa85b80 10 client.27137 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:18.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.145+0000 7f4b7fa85b80 10 client.27137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:18.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.145+0000 7f4b7fa85b80 20 client.27137 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:18.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.145+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f1b6490 0x55920f1b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.145+0000 7f4b7fa85b80 4 client.27137 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:18.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.145+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55920f0e6eb0 con 0x55920f1b6490 2023-07-14T11:36:18.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.147+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f1b6490 0x55920f1b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.148+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f1b6490 0x55920f1b88a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7263b1b817b4eb server_cookie=d153f02a762cd4b in_seq=0 out_seq=0 2023-07-14T11:36:18.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.148+0000 7f4b4d7fa700 10 client.27137 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:18.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/145453437 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55920f0e6eb0 con 0x55920f1b6490 2023-07-14T11:36:18.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b4d7fa700 10 client.27137 handle_command_reply: tid=0 2023-07-14T11:36:18.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 client.27137 shutdown 2023-07-14T11:36:18.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f1a6ad0 msgr2=0x55920f1b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f1a6ad0 0x55920f1b1f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0199e0 tx=0x7f4b5c004b40 comp rx=0 tx=0).stop 2023-07-14T11:36:18.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 shutdown_connections 2023-07-14T11:36:18.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f1b6490 0x55920f1b88a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f1a6ad0 0x55920f1b1f20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/145453437 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f19bbe0 0x55920f1a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 >> 172.21.15.161:0/145453437 conn(0x55920f191da0 msgr2=0x55920f197230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:18.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.150+0000 7f4b4d7fa700 0 client.27137 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:18.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.151+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 shutdown_connections 2023-07-14T11:36:18.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.151+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/145453437 wait complete. 2023-07-14T11:36:18.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.151+0000 7f4b7fa85b80 20 client.27137 trim_cache size 0 max 16384 2023-07-14T11:36:18.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.172+0000 7f4b7fa85b80 1 Processor -- start 2023-07-14T11:36:18.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.172+0000 7f4b7fa85b80 1 -- start start 2023-07-14T11:36:18.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.172+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f195e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.172+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f1a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f1a5c90 con 0x55920f19b2d0 2023-07-14T11:36:18.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f1b3ac0 con 0x55920f19bbe0 2023-07-14T11:36:18.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f1a63f0 con 0x55920f1a6ad0 2023-07-14T11:36:18.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6979a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6a79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f1a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b69f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6979a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1a6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59034/0 (socket says 172.21.15.161:59034) 2023-07-14T11:36:18.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6a79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f1a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46554/0 (socket says 172.21.15.161:46554) 2023-07-14T11:36:18.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6979a700 1 -- 172.21.15.161:0/3473660734 learned_addr learned my addr 172.21.15.161:0/3473660734 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:18.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6a79c700 1 -- 172.21.15.161:0/3473660734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 msgr2=0x55920f1a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6a79c700 1 -- 172.21.15.161:0/3473660734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 msgr2=0x55920f195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6a79c700 1 -- 172.21.15.161:0/3473660734 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55920f1935e0 con 0x55920f1a6ad0 2023-07-14T11:36:18.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.173+0000 7f4b6979a700 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:18.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.174+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f1a73e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009500 tx=0x7f4b5c019e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe58acdde61b6934 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:18.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.174+0000 7f4b58ff9700 1 -- 172.21.15.161:0/3473660734 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c01e040 con 0x55920f1a6ad0 2023-07-14T11:36:18.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.174+0000 7f4b58ff9700 1 -- 172.21.15.161:0/3473660734 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5c011040 con 0x55920f1a6ad0 2023-07-14T11:36:18.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.174+0000 7f4b58ff9700 1 -- 172.21.15.161:0/3473660734 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c022df0 con 0x55920f1a6ad0 2023-07-14T11:36:18.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3473660734 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 msgr2=0x55920f1a73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f1a73e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009500 tx=0x7f4b5c019e50 comp rx=0 tx=0).stop 2023-07-14T11:36:18.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3473660734 shutdown_connections 2023-07-14T11:36:18.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f1a73e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/3473660734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3473660734 >> 172.21.15.161:0/3473660734 conn(0x55920f191da0 msgr2=0x55920f1a0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:18.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3473660734 shutdown_connections 2023-07-14T11:36:18.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.175+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/3473660734 wait complete. 2023-07-14T11:36:18.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.176+0000 7f4b7fa85b80 1 -- start start 2023-07-14T11:36:18.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.176+0000 7f4b7fa85b80 1 Processor -- start 2023-07-14T11:36:18.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1aa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b7fa85b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f203370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f2038b0 con 0x55920f19b2d0 2023-07-14T11:36:18.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f203a20 con 0x55920f19bbe0 2023-07-14T11:36:18.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b7fa85b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920f203be0 con 0x55920f1a6ad0 2023-07-14T11:36:18.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b7fa85b80 10 client.0 _ll_register_callbacks cb 0x7f4b5c007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:18.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b69f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b69f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1aa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52656/0 (socket says 172.21.15.161:52656) 2023-07-14T11:36:18.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.177+0000 7f4b69f9b700 1 -- 172.21.15.161:0/1666932166 learned_addr learned my addr 172.21.15.161:0/1666932166 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:18.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b6979a700 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b6a79c700 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f203370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b69f9b700 1 -- 172.21.15.161:0/1666932166 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 msgr2=0x55920f203370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f203370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b69f9b700 1 -- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 msgr2=0x55920f1aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:18.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b69f9b700 1 -- 172.21.15.161:0/1666932166 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b641137e0 con 0x55920f19b2d0 2023-07-14T11:36:18.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b69f9b700 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1aa010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b600a18b0 tx=0x7f4b600d8d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a62558b124c69e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:18.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.178+0000 7f4b4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:36:18.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.179+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/1666932166 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600d01a0 con 0x55920f19b2d0 2023-07-14T11:36:18.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.179+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/1666932166 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4b60084040 con 0x55920f19b2d0 2023-07-14T11:36:18.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.179+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b600d8430 con 0x55920f19b2d0 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.179+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/1666932166 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b600c9f80 con 0x55920f19b2d0 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.179+0000 7f4b7fa85b80 10 client.17088 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.179+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55920ef04c60 con 0x55920f19b2d0 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/1666932166 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b600b1080 con 0x55920f19b2d0 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/1666932166 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4b600960a0 con 0x55920f19b2d0 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b7fa85b80 10 client.17088 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b7fa85b80 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:18.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b7fa85b80 20 client.17088 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:18.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f20ae00 0x55920f20d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:18.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b7fa85b80 4 client.17088 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:36:18.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.180+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55920f0a9050 con 0x55920f20ae00 2023-07-14T11:36:18.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.181+0000 7f4b6979a700 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f20ae00 0x55920f20d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:18.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.182+0000 7f4b4d7fa700 10 client.17088 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:18.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.182+0000 7f4b6979a700 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f20ae00 0x55920f20d1e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ae136ad642cfe7 server_cookie=b76082dde4e03776 in_seq=0 out_seq=0 2023-07-14T11:36:18.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.185+0000 7f4b4d7fa700 1 -- 172.21.15.161:0/1666932166 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55920f0a9050 con 0x55920f20ae00 2023-07-14T11:36:18.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.185+0000 7f4b4d7fa700 10 client.17088 handle_command_reply: tid=0 2023-07-14T11:36:18.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.185+0000 7f4b7fa85b80 1 client.17088 shutdown 2023-07-14T11:36:18.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.185+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 msgr2=0x55920f1aa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.185+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1aa010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b600a18b0 tx=0x7f4b600d8d20 comp rx=0 tx=0).stop 2023-07-14T11:36:18.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 shutdown_connections 2023-07-14T11:36:18.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55920f20ae00 0x55920f20d1e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55920f19b2d0 0x55920f1aa010 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55920f1a6ad0 0x55920f203370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b7fa85b80 1 --2- 172.21.15.161:0/1666932166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55920f19bbe0 0x55920f1aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 >> 172.21.15.161:0/1666932166 conn(0x55920f191da0 msgr2=0x55920f1a1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:18.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b4d7fa700 0 client.17088 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:18.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.186+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 shutdown_connections 2023-07-14T11:36:18.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.187+0000 7f4b7fa85b80 1 -- 172.21.15.161:0/1666932166 wait complete. 2023-07-14T11:36:18.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.187+0000 7f4b7fa85b80 20 client.17088 trim_cache size 0 max 16384 2023-07-14T11:36:18.192 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:36:18.192 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (126 inodes in the stack)", 2023-07-14T11:36:18.192 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:36:18.192 INFO:teuthology.orchestra.run.smithi161.stdout: "304268cc-3b8b-4b03-be80-e16718671176": { 2023-07-14T11:36:18.192 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:36:18.193 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "304268cc-3b8b-4b03-be80-e16718671176", 2023-07-14T11:36:18.193 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:36:18.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:36:18.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:36:18.193 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:36:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4b500a13c0 msgr2=0x7f4b500a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4b500a13c0 0x7f4b500a3870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b640673e0 tx=0x7f4b5c009040 comp rx=0 tx=0).stop 2023-07-14T11:36:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b640fe8b0 msgr2=0x7f4b6419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b640fe8b0 0x7f4b6419c280 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000d950 tx=0x7f4b6000dc60 comp rx=0 tx=0).stop 2023-07-14T11:36:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 shutdown_connections 2023-07-14T11:36:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4b500a13c0 0x7f4b500a3870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b640fe8b0 0x7f4b6419c280 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b64104a80 0x7f4b641a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 --2- 172.21.15.161:0/694829267 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4b640ff1c0 0x7f4b6419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:18.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.188+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 >> 172.21.15.161:0/694829267 conn(0x7f4b640fa4e0 msgr2=0x7f4b640fafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:18.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.189+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 shutdown_connections 2023-07-14T11:36:18.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:18.189+0000 7f4b6c1ff700 1 -- 172.21.15.161:0/694829267 wait complete. 2023-07-14T11:36:18.213 INFO:tasks.cephfs.filesystem:scrub status for tag:304268cc-3b8b-4b03-be80-e16718671176 - {'path': '/', 'tag': '304268cc-3b8b-4b03-be80-e16718671176', 'options': 'recursive,force'} 2023-07-14T11:36:18.500 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:18 smithi161 ceph-mon[95526]: pgmap v440: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 489 op/s 2023-07-14T11:36:18.500 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:18 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:36:18.500 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:18 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:18.500 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:18 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:18.564 INFO:tasks.workunit.client.0.smithi161.stdout: 10 38441 37.43 MB/sec warmup 60 sec latency 112.073 ms 2023-07-14T11:36:18.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:18 smithi175 ceph-mon[129081]: pgmap v440: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 489 op/s 2023-07-14T11:36:18.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:18 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:36:18.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:18 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:18.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:18 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:18.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:18 smithi182 ceph-mon[129366]: pgmap v440: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 489 op/s 2023-07-14T11:36:18.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:18 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:36:18.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:18 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:18.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:18 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:19.562 INFO:tasks.workunit.client.0.smithi161.stdout: 10 38519 36.89 MB/sec warmup 61 sec latency 836.947 ms 2023-07-14T11:36:19.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:19 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:19.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:19 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:19.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:19 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:19 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:19.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:19 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:19 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:19 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:19 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:19.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:19 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:19 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:19 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:19.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:19 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:19 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:19 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:19 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:19 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:19 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:19 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:19 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:19 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:19.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:19 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:20.563 INFO:tasks.workunit.client.0.smithi161.stdout: 10 38884 36.63 MB/sec warmup 62 sec latency 129.282 ms 2023-07-14T11:36:20.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:20 smithi175 ceph-mon[129081]: pgmap v441: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 489 op/s 2023-07-14T11:36:20.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:20 smithi182 ceph-mon[129366]: pgmap v441: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 489 op/s 2023-07-14T11:36:20.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:20 smithi161 ceph-mon[95526]: pgmap v441: 129 pgs: 129 active+clean; 1.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 489 op/s 2023-07-14T11:36:21.563 INFO:tasks.workunit.client.0.smithi161.stdout: 10 39406 36.42 MB/sec warmup 63 sec latency 114.945 ms 2023-07-14T11:36:22.563 INFO:tasks.workunit.client.0.smithi161.stdout: 10 40120 36.57 MB/sec warmup 64 sec latency 78.896 ms 2023-07-14T11:36:22.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:22 smithi175 ceph-mon[129081]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 637 op/s 2023-07-14T11:36:22.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:22 smithi182 ceph-mon[129366]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 637 op/s 2023-07-14T11:36:22.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:22 smithi161 ceph-mon[95526]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 637 op/s 2023-07-14T11:36:23.563 INFO:tasks.workunit.client.0.smithi161.stdout: 10 40834 36.35 MB/sec warmup 65 sec latency 46.764 ms 2023-07-14T11:36:24.217 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:23 smithi182 ceph-mon[129366]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 617 op/s 2023-07-14T11:36:24.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:23 smithi161 ceph-mon[95526]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 617 op/s 2023-07-14T11:36:24.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:23 smithi175 ceph-mon[129081]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 617 op/s 2023-07-14T11:36:24.564 INFO:tasks.workunit.client.0.smithi161.stdout: 10 41365 36.38 MB/sec warmup 66 sec latency 450.742 ms 2023-07-14T11:36:25.564 INFO:tasks.workunit.client.0.smithi161.stdout: 10 41834 36.11 MB/sec warmup 67 sec latency 614.044 ms 2023-07-14T11:36:26.564 INFO:tasks.workunit.client.0.smithi161.stdout: 10 42236 35.96 MB/sec warmup 68 sec latency 109.282 ms 2023-07-14T11:36:26.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:26 smithi175 ceph-mon[129081]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 530 op/s 2023-07-14T11:36:26.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:26 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:36:26.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:26 smithi182 ceph-mon[129366]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 530 op/s 2023-07-14T11:36:26.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:26 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:36:26.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:26 smithi161 ceph-mon[95526]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 530 op/s 2023-07-14T11:36:26.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:26 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:36:27.564 INFO:tasks.workunit.client.0.smithi161.stdout: 10 43010 36.23 MB/sec warmup 69 sec latency 81.286 ms 2023-07-14T11:36:27.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:27 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:36:27.670 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:27 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:36:27.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:27 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:36:27.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:27 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:27 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:27 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:36:28.568 INFO:tasks.workunit.client.0.smithi161.stdout: 10 43550 36.00 MB/sec warmup 70 sec latency 82.319 ms 2023-07-14T11:36:28.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:28 smithi175 ceph-mon[129081]: pgmap v445: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 836 op/s 2023-07-14T11:36:28.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:28 smithi182 ceph-mon[129366]: pgmap v445: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 836 op/s 2023-07-14T11:36:28.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:28 smithi161 ceph-mon[95526]: pgmap v445: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 836 op/s 2023-07-14T11:36:29.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 44412 36.03 MB/sec warmup 71 sec latency 79.072 ms 2023-07-14T11:36:30.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 45213 36.26 MB/sec warmup 72 sec latency 109.226 ms 2023-07-14T11:36:30.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:30 smithi175 ceph-mon[129081]: pgmap v446: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 637 op/s 2023-07-14T11:36:30.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:30 smithi182 ceph-mon[129366]: pgmap v446: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 637 op/s 2023-07-14T11:36:30.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:30 smithi161 ceph-mon[95526]: pgmap v446: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 637 op/s 2023-07-14T11:36:31.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 45889 36.20 MB/sec warmup 73 sec latency 101.658 ms 2023-07-14T11:36:32.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 46561 36.24 MB/sec warmup 74 sec latency 56.079 ms 2023-07-14T11:36:32.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:32 smithi175 ceph-mon[129081]: pgmap v447: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 868 op/s 2023-07-14T11:36:32.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:32 smithi182 ceph-mon[129366]: pgmap v447: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 868 op/s 2023-07-14T11:36:32.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:32 smithi161 ceph-mon[95526]: pgmap v447: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 868 op/s 2023-07-14T11:36:33.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 47325 36.27 MB/sec warmup 75 sec latency 101.802 ms 2023-07-14T11:36:34.246 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:33 smithi161 ceph-mon[95526]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 536 op/s 2023-07-14T11:36:34.417 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:33 smithi175 ceph-mon[129081]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 536 op/s 2023-07-14T11:36:34.462 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:33 smithi182 ceph-mon[129366]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 536 op/s 2023-07-14T11:36:34.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 47992 36.41 MB/sec warmup 76 sec latency 48.865 ms 2023-07-14T11:36:35.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 48742 36.24 MB/sec warmup 77 sec latency 75.292 ms 2023-07-14T11:36:36.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 49583 36.46 MB/sec warmup 78 sec latency 121.524 ms 2023-07-14T11:36:36.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:36 smithi175 ceph-mon[129081]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 536 op/s 2023-07-14T11:36:36.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:36 smithi182 ceph-mon[129366]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 536 op/s 2023-07-14T11:36:36.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:36 smithi161 ceph-mon[95526]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 536 op/s 2023-07-14T11:36:37.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 50332 36.56 MB/sec warmup 79 sec latency 66.172 ms 2023-07-14T11:36:38.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 50910 36.51 MB/sec warmup 80 sec latency 50.466 ms 2023-07-14T11:36:38.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:38 smithi175 ceph-mon[129081]: pgmap v450: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 737 op/s 2023-07-14T11:36:38.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:38 smithi182 ceph-mon[129366]: pgmap v450: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 737 op/s 2023-07-14T11:36:38.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:38 smithi161 ceph-mon[95526]: pgmap v450: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 737 op/s 2023-07-14T11:36:39.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 51501 36.36 MB/sec warmup 81 sec latency 151.581 ms 2023-07-14T11:36:40.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 52414 36.50 MB/sec warmup 82 sec latency 73.357 ms 2023-07-14T11:36:40.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:40 smithi175 ceph-mon[129081]: pgmap v451: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 432 op/s 2023-07-14T11:36:40.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:40 smithi182 ceph-mon[129366]: pgmap v451: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 432 op/s 2023-07-14T11:36:40.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:40 smithi161 ceph-mon[95526]: pgmap v451: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 432 op/s 2023-07-14T11:36:41.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 53404 36.77 MB/sec warmup 83 sec latency 63.262 ms 2023-07-14T11:36:42.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 54092 36.78 MB/sec warmup 84 sec latency 76.145 ms 2023-07-14T11:36:42.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:42 smithi175 ceph-mon[129081]: pgmap v452: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2023-07-14T11:36:42.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:42 smithi182 ceph-mon[129366]: pgmap v452: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2023-07-14T11:36:42.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:42 smithi161 ceph-mon[95526]: pgmap v452: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2023-07-14T11:36:43.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 54711 36.75 MB/sec warmup 85 sec latency 133.565 ms 2023-07-14T11:36:44.244 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:43 smithi161 ceph-mon[95526]: pgmap v453: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2023-07-14T11:36:44.244 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:43 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:36:44.417 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:43 smithi175 ceph-mon[129081]: pgmap v453: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2023-07-14T11:36:44.417 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:43 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:36:44.506 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:43 smithi182 ceph-mon[129366]: pgmap v453: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2023-07-14T11:36:44.507 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:43 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:36:44.566 INFO:tasks.workunit.client.0.smithi161.stdout: 10 55430 36.75 MB/sec warmup 86 sec latency 164.543 ms 2023-07-14T11:36:45.567 INFO:tasks.workunit.client.0.smithi161.stdout: 10 55848 36.57 MB/sec warmup 87 sec latency 423.898 ms 2023-07-14T11:36:46.571 INFO:tasks.workunit.client.0.smithi161.stdout: 10 56167 36.31 MB/sec warmup 88 sec latency 637.704 ms 2023-07-14T11:36:46.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:46 smithi175 ceph-mon[129081]: pgmap v454: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2023-07-14T11:36:46.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:46 smithi161 ceph-mon[95526]: pgmap v454: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2023-07-14T11:36:46.893 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:46 smithi182 ceph-mon[129366]: pgmap v454: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2023-07-14T11:36:47.574 INFO:tasks.workunit.client.0.smithi161.stdout: 10 56528 36.12 MB/sec warmup 89 sec latency 158.432 ms 2023-07-14T11:36:48.215 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:36:48.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/3980145763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 msgr2=0x7f8b841010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/3980145763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b841010d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009990 tx=0x7f8b80009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/3980145763 shutdown_connections 2023-07-14T11:36:48.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/3980145763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8410a040 0x7f8b8410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/3980145763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b841010d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/3980145763 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b84101610 0x7f8b84109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/3980145763 >> 172.21.15.161:0/3980145763 conn(0x7f8b840fa4b0 msgr2=0x7f8b840fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:48.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.488+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/3980145763 shutdown_connections 2023-07-14T11:36:48.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.489+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/3980145763 wait complete. 2023-07-14T11:36:48.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.490+0000 7f8b8b9ff700 1 Processor -- start 2023-07-14T11:36:48.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.490+0000 7f8b8b9ff700 1 -- start start 2023-07-14T11:36:48.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8b9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b84197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8b9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b84101610 0x7f8b84198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8b9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8410a040 0x7f8b8419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8b9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8419ce10 con 0x7f8b8410a040 2023-07-14T11:36:48.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8b9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8419cf80 con 0x7f8b84100d00 2023-07-14T11:36:48.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8b9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8419d140 con 0x7f8b84101610 2023-07-14T11:36:48.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b84197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b88f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b84101610 0x7f8b84198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b84197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46838/0 (socket says 172.21.15.161:46838) 2023-07-14T11:36:48.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 -- 172.21.15.161:0/790264093 learned_addr learned my addr 172.21.15.161:0/790264093 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:48.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8410a040 0x7f8b8419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 -- 172.21.15.161:0/790264093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b84101610 msgr2=0x7f8b84198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b84101610 0x7f8b84198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 -- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8410a040 msgr2=0x7f8b8419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8410a040 0x7f8b8419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b8979b700 1 -- 172.21.15.161:0/790264093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800095f0 con 0x7f8b84100d00 2023-07-14T11:36:48.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.491+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b84101610 0x7f8b84198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:48.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.492+0000 7f8b8979b700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b84197e30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000c00 tx=0x7f8b8000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.492+0000 7f8b7a7fc700 1 -- 172.21.15.161:0/790264093 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8001d070 con 0x7f8b84100d00 2023-07-14T11:36:48.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.492+0000 7f8b7a7fc700 1 -- 172.21.15.161:0/790264093 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8000be40 con 0x7f8b84100d00 2023-07-14T11:36:48.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.492+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8419d3c0 con 0x7f8b84100d00 2023-07-14T11:36:48.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.492+0000 7f8b7a7fc700 1 -- 172.21.15.161:0/790264093 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800173f0 con 0x7f8b84100d00 2023-07-14T11:36:48.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.492+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b841a4c60 con 0x7f8b84100d00 2023-07-14T11:36:48.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.496+0000 7f8b7a7fc700 1 -- 172.21.15.161:0/790264093 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8b80022470 con 0x7f8b84100d00 2023-07-14T11:36:48.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.497+0000 7f8b9f409b80 1 Processor -- start 2023-07-14T11:36:48.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.497+0000 7f8b9f409b80 1 -- start start 2023-07-14T11:36:48.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89502d0 0x5617d89506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d8963da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89642e0 0x5617d89666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d8856cb0 con 0x5617d8950be0 2023-07-14T11:36:48.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d88a9a40 con 0x5617d89502d0 2023-07-14T11:36:48.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d88a2ed0 con 0x5617d89642e0 2023-07-14T11:36:48.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b89f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89502d0 0x5617d89506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b89f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89502d0 0x5617d89506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46840/0 (socket says 172.21.15.161:46840) 2023-07-14T11:36:48.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.498+0000 7f8b89f9c700 1 -- 172.21.15.161:0/883754592 learned_addr learned my addr 172.21.15.161:0/883754592 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:48.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89642e0 0x5617d89666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b8979b700 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d8963da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b89f9c700 1 -- 172.21.15.161:0/883754592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89642e0 msgr2=0x5617d89666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89642e0 0x5617d89666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b89f9c700 1 -- 172.21.15.161:0/883754592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 msgr2=0x5617d8963da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d8963da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b7a7fc700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8b700a12c0 0x7f8b700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b89f9c700 1 -- 172.21.15.161:0/883754592 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617d86d0de0 con 0x5617d89502d0 2023-07-14T11:36:48.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.499+0000 7f8b7a7fc700 1 -- 172.21.15.161:0/790264093 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8b800ce310 con 0x7f8b84100d00 2023-07-14T11:36:48.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.500+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89502d0 0x5617d89506a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c003fa0 tx=0x7f8b7c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7514f0438f01be5a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.500+0000 7f8b8979b700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8b700a12c0 0x7f8b700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.501+0000 7f8b6f7fe700 1 -- 172.21.15.161:0/883754592 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c019070 con 0x5617d89502d0 2023-07-14T11:36:48.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.501+0000 7f8b6f7fe700 1 -- 172.21.15.161:0/883754592 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7c00f040 con 0x5617d89502d0 2023-07-14T11:36:48.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.501+0000 7f8b6f7fe700 1 -- 172.21.15.161:0/883754592 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c007af0 con 0x5617d89502d0 2023-07-14T11:36:48.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.501+0000 7f8b9f409b80 1 -- 172.21.15.161:0/883754592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89502d0 msgr2=0x5617d89506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.501+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89502d0 0x5617d89506a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c003fa0 tx=0x7f8b7c009ba0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.501+0000 7f8b8979b700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8b700a12c0 0x7f8b700a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b800975f0 tx=0x7f8b80005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.502+0000 7f8b9f409b80 1 -- 172.21.15.161:0/883754592 shutdown_connections 2023-07-14T11:36:48.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.502+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89642e0 0x5617d89666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.502+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89502d0 0x5617d89506a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.502+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/883754592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d8963da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.502+0000 7f8b9f409b80 1 -- 172.21.15.161:0/883754592 >> 172.21.15.161:0/883754592 conn(0x5617d8946da0 msgr2=0x5617d894c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:48.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.502+0000 7f8b9f409b80 1 -- 172.21.15.161:0/883754592 shutdown_connections 2023-07-14T11:36:48.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.502+0000 7f8b9f409b80 1 -- 172.21.15.161:0/883754592 wait complete. 2023-07-14T11:36:48.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.503+0000 7f8b9f409b80 1 -- start start 2023-07-14T11:36:48.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.503+0000 7f8b9f409b80 1 Processor -- start 2023-07-14T11:36:48.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d8961080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d8842ac0 con 0x5617d8950be0 2023-07-14T11:36:48.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d88427c0 con 0x5617d89642e0 2023-07-14T11:36:48.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d8726920 con 0x5617d89502d0 2023-07-14T11:36:48.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b9f409b80 10 client.0 _ll_register_callbacks cb 0x5617d8840760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:48.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b88f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b89f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d8961080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b88f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:36942/0 (socket says 172.21.15.161:36942) 2023-07-14T11:36:48.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b88f9a700 1 -- 172.21.15.161:0/1494365592 learned_addr learned my addr 172.21.15.161:0/1494365592 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:48.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.504+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.505+0000 7f8b89f9c700 1 -- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 msgr2=0x5617d8961b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.505+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.505+0000 7f8b89f9c700 1 -- 172.21.15.161:0/1494365592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 msgr2=0x5617d89615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.505+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.505+0000 7f8b89f9c700 1 -- 172.21.15.161:0/1494365592 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c003bd0 con 0x5617d89502d0 2023-07-14T11:36:48.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.505+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d8961080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c003f70 tx=0x7f8b7c007630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2923eb5d460bb81 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.505+0000 7f8b6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:48.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.506+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1494365592 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c019070 con 0x5617d89502d0 2023-07-14T11:36:48.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.506+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617d87fc4f0 con 0x5617d89502d0 2023-07-14T11:36:48.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.506+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1494365592 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8b7c0076d0 con 0x5617d89502d0 2023-07-14T11:36:48.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.506+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1494365592 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b7c007890 con 0x5617d89502d0 2023-07-14T11:36:48.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.506+0000 7f8b9f409b80 10 client.27169 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:48.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.506+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5617d86b9c60 con 0x5617d89502d0 2023-07-14T11:36:48.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.506+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1494365592 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7c02d020 con 0x5617d89502d0 2023-07-14T11:36:48.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.507+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1494365592 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8b7c015cd0 con 0x5617d89502d0 2023-07-14T11:36:48.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.507+0000 7f8b9f409b80 10 client.27169 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:48.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.507+0000 7f8b9f409b80 10 client.27169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:48.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.507+0000 7f8b9f409b80 20 client.27169 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:48.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.507+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d8973b50 0x5617d8975f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.507+0000 7f8b9f409b80 4 client.27169 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:48.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.507+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617d889beb0 con 0x5617d8973b50 2023-07-14T11:36:48.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.508+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d8973b50 0x5617d8975f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.508+0000 7f8b6cff9700 10 client.27169 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:48.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.508+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d8973b50 0x5617d8975f60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b97f957be862754f server_cookie=a2dc4f5350491b3c in_seq=0 out_seq=0 2023-07-14T11:36:48.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1494365592 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5617d889beb0 con 0x5617d8973b50 2023-07-14T11:36:48.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b6cff9700 10 client.27169 handle_command_reply: tid=0 2023-07-14T11:36:48.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b9f409b80 1 client.27169 shutdown 2023-07-14T11:36:48.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 msgr2=0x5617d8961080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d8961080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c003f70 tx=0x7f8b7c007630 comp rx=0 tx=0).stop 2023-07-14T11:36:48.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 shutdown_connections 2023-07-14T11:36:48.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d8973b50 0x5617d8975f60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.511+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.512+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d8961080 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.512+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1494365592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.512+0000 7f8b6cff9700 0 client.27169 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:48.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.512+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 >> 172.21.15.161:0/1494365592 conn(0x5617d8946da0 msgr2=0x5617d894c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:48.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.512+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 shutdown_connections 2023-07-14T11:36:48.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.512+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1494365592 wait complete. 2023-07-14T11:36:48.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.512+0000 7f8b9f409b80 20 client.27169 trim_cache size 0 max 16384 2023-07-14T11:36:48.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.536+0000 7f8b9f409b80 1 Processor -- start 2023-07-14T11:36:48.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.536+0000 7f8b9f409b80 1 -- start start 2023-07-14T11:36:48.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.536+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d89502d0 0x5617d896acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.536+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d8950be0 0x5617d8952270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.536+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d8962690 con 0x5617d89502d0 2023-07-14T11:36:48.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d8951d60 con 0x5617d89642e0 2023-07-14T11:36:48.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d89765f0 con 0x5617d8950be0 2023-07-14T11:36:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b89f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d89502d0 0x5617d896acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d8950be0 0x5617d8952270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d8950be0 0x5617d8952270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45790/0 (socket says 172.21.15.161:45790) 2023-07-14T11:36:48.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b89f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d89502d0 0x5617d896acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:36948/0 (socket says 172.21.15.161:36948) 2023-07-14T11:36:48.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 -- 172.21.15.161:0/2285520347 learned_addr learned my addr 172.21.15.161:0/2285520347 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:48.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 -- 172.21.15.161:0/2285520347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 msgr2=0x5617d8961840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 -- 172.21.15.161:0/2285520347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d89502d0 msgr2=0x5617d896acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d89502d0 0x5617d896acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.537+0000 7f8b88f9a700 1 -- 172.21.15.161:0/2285520347 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617d89485e0 con 0x5617d8950be0 2023-07-14T11:36:48.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.538+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d8950be0 0x5617d8952270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5617d8962630 tx=0x7f8b74009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa99a3b70f33cb16 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.538+0000 7f8b6ffff700 1 -- 172.21.15.161:0/2285520347 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74016030 con 0x5617d8950be0 2023-07-14T11:36:48.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.538+0000 7f8b6ffff700 1 -- 172.21.15.161:0/2285520347 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b74017040 con 0x5617d8950be0 2023-07-14T11:36:48.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.538+0000 7f8b6ffff700 1 -- 172.21.15.161:0/2285520347 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74004a90 con 0x5617d8950be0 2023-07-14T11:36:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 -- 172.21.15.161:0/2285520347 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d8950be0 msgr2=0x5617d8952270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d8950be0 0x5617d8952270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5617d8962630 tx=0x7f8b74009260 comp rx=0 tx=0).stop 2023-07-14T11:36:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 -- 172.21.15.161:0/2285520347 shutdown_connections 2023-07-14T11:36:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d89502d0 0x5617d896acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d8961840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/2285520347 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d8950be0 0x5617d8952270 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 -- 172.21.15.161:0/2285520347 >> 172.21.15.161:0/2285520347 conn(0x5617d8946da0 msgr2=0x5617d896ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:48.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 -- 172.21.15.161:0/2285520347 shutdown_connections 2023-07-14T11:36:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.539+0000 7f8b9f409b80 1 -- 172.21.15.161:0/2285520347 wait complete. 2023-07-14T11:36:48.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.540+0000 7f8b9f409b80 1 -- start start 2023-07-14T11:36:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.540+0000 7f8b9f409b80 1 Processor -- start 2023-07-14T11:36:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d89c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b9f409b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d89bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d89bbf30 con 0x5617d8950be0 2023-07-14T11:36:48.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d89bc0a0 con 0x5617d89642e0 2023-07-14T11:36:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b9f409b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617d89bc260 con 0x5617d89502d0 2023-07-14T11:36:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b9f409b80 10 client.0 _ll_register_callbacks cb 0x7f8b7c007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b89f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d89c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b89f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d89c1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45796/0 (socket says 172.21.15.161:45796) 2023-07-14T11:36:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b89f9c700 1 -- 172.21.15.161:0/1957266378 learned_addr learned my addr 172.21.15.161:0/1957266378 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.541+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d89bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b89f9c700 1 -- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 msgr2=0x5617d89bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d89bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b89f9c700 1 -- 172.21.15.161:0/1957266378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 msgr2=0x5617d89c1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89c1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b89f9c700 1 -- 172.21.15.161:0/1957266378 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b74004420 con 0x5617d89502d0 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89c1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b89f9c700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d89c1050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c01f500 tx=0x7f8b7c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a3718e905e8abd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1957266378 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c019040 con 0x5617d89502d0 2023-07-14T11:36:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1957266378 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8b7c02e080 con 0x5617d89502d0 2023-07-14T11:36:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.542+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b7c00f040 con 0x5617d89502d0 2023-07-14T11:36:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1957266378 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b7c0076a0 con 0x5617d89502d0 2023-07-14T11:36:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 10 client.27175 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:48.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5617d86f7250 con 0x5617d89502d0 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1957266378 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7c00f040 con 0x5617d89502d0 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1957266378 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8b7c0050c0 con 0x5617d89502d0 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 10 client.27175 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 10 client.27175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 20 client.27175 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d89c89b0 0x5617d89cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 4 client.27175 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:36:48.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.543+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5617d885e050 con 0x5617d89c89b0 2023-07-14T11:36:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.544+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d89c89b0 0x5617d89cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.544+0000 7f8b88f9a700 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d89c89b0 0x5617d89cadc0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f58c5a5fccd75ca server_cookie=644da323b8ee01e4 in_seq=0 out_seq=0 2023-07-14T11:36:48.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.544+0000 7f8b6cff9700 10 client.27175 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:48.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b6cff9700 1 -- 172.21.15.161:0/1957266378 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5617d885e050 con 0x5617d89c89b0 2023-07-14T11:36:48.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b6cff9700 10 client.27175 handle_command_reply: tid=0 2023-07-14T11:36:48.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 client.27175 shutdown 2023-07-14T11:36:48.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 msgr2=0x5617d89c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d89c1050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c01f500 tx=0x7f8b7c009ba0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 shutdown_connections 2023-07-14T11:36:48.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5617d89c89b0 0x5617d89cadc0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d89642e0 0x5617d89bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5617d89502d0 0x5617d89c1050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 --2- 172.21.15.161:0/1957266378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617d8950be0 0x5617d89c1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b6cff9700 0 client.27175 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:48.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 >> 172.21.15.161:0/1957266378 conn(0x5617d8946da0 msgr2=0x5617d896ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:48.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 shutdown_connections 2023-07-14T11:36:48.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 1 -- 172.21.15.161:0/1957266378 wait complete. 2023-07-14T11:36:48.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.551+0000 7f8b9f409b80 20 client.27175 trim_cache size 0 max 16384 2023-07-14T11:36:48.554 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:36:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:36:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:36:48.554 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:36:48.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8b700a12c0 msgr2=0x7f8b700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8b700a12c0 0x7f8b700a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b800975f0 tx=0x7f8b80005fd0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 msgr2=0x7f8b84197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b84197e30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000c00 tx=0x7f8b8000f690 comp rx=0 tx=0).stop 2023-07-14T11:36:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 shutdown_connections 2023-07-14T11:36:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8b700a12c0 0x7f8b700a3770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8410a040 0x7f8b8419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b84100d00 0x7f8b84197e30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 --2- 172.21.15.161:0/790264093 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8b84101610 0x7f8b84198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 >> 172.21.15.161:0/790264093 conn(0x7f8b840fa4b0 msgr2=0x7f8b8410c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 shutdown_connections 2023-07-14T11:36:48.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.552+0000 7f8b8b9ff700 1 -- 172.21.15.161:0/790264093 wait complete. 2023-07-14T11:36:48.573 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:48 smithi161 ceph-mon[95526]: pgmap v455: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 654 op/s 2023-07-14T11:36:48.574 INFO:tasks.workunit.client.0.smithi161.stdout: 10 57277 36.22 MB/sec warmup 90 sec latency 61.058 ms 2023-07-14T11:36:48.575 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:36:48.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:36:48.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:48 smithi175 ceph-mon[129081]: pgmap v455: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 654 op/s 2023-07-14T11:36:48.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:48 smithi182 ceph-mon[129366]: pgmap v455: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 654 op/s 2023-07-14T11:36:48.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.860+0000 7fe81e3ff700 1 -- 172.21.15.161:0/1561401073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 msgr2=0x7fe81810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.860+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/1561401073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 0x7fe81810c910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b0a0 tx=0x7fe81000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.861+0000 7fe81e3ff700 1 -- 172.21.15.161:0/1561401073 shutdown_connections 2023-07-14T11:36:48.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.861+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/1561401073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe818105ae0 0x7fe818105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.861+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/1561401073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 0x7fe81810c910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.861+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/1561401073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 0x7fe8180fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.861+0000 7fe81e3ff700 1 -- 172.21.15.161:0/1561401073 >> 172.21.15.161:0/1561401073 conn(0x7fe8180fb590 msgr2=0x7fe8180fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:48.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.861+0000 7fe81e3ff700 1 -- 172.21.15.161:0/1561401073 shutdown_connections 2023-07-14T11:36:48.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.861+0000 7fe81e3ff700 1 -- 172.21.15.161:0/1561401073 wait complete. 2023-07-14T11:36:48.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.862+0000 7fe81e3ff700 1 Processor -- start 2023-07-14T11:36:48.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.862+0000 7fe81e3ff700 1 -- start start 2023-07-14T11:36:48.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe81e3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 0x7fe81819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe81e3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 0x7fe81819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe81e3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe818105ae0 0x7fe818195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe81e3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe818196140 con 0x7fe818105ae0 2023-07-14T11:36:48.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe81e3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8181962b0 con 0x7fe818100480 2023-07-14T11:36:48.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe81e3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe818196470 con 0x7fe8180ffad0 2023-07-14T11:36:48.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe817fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 0x7fe81819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe817fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 0x7fe81819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45816/0 (socket says 172.21.15.161:45816) 2023-07-14T11:36:48.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe817fff700 1 -- 172.21.15.161:0/2853200372 learned_addr learned my addr 172.21.15.161:0/2853200372 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:48.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.863+0000 7fe8177fe700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 0x7fe81819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe817fff700 1 -- 172.21.15.161:0/2853200372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 msgr2=0x7fe81819bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe817fff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 0x7fe81819bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe817fff700 1 -- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe818105ae0 msgr2=0x7fe818195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:48.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe817fff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe818105ae0 0x7fe818195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:48.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe817fff700 1 -- 172.21.15.161:0/2853200372 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810009d00 con 0x7fe8180ffad0 2023-07-14T11:36:48.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe8177fe700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 0x7fe81819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:48.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe817fff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 0x7fe81819ba60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe80800e9a0 tx=0x7fe80800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe8157fa700 1 -- 172.21.15.161:0/2853200372 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808009790 con 0x7fe8180ffad0 2023-07-14T11:36:48.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe8157fa700 1 -- 172.21.15.161:0/2853200372 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe80800cbd0 con 0x7fe8180ffad0 2023-07-14T11:36:48.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe818196750 con 0x7fe8180ffad0 2023-07-14T11:36:48.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe8157fa700 1 -- 172.21.15.161:0/2853200372 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808018680 con 0x7fe8180ffad0 2023-07-14T11:36:48.871 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.864+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8181a2a50 con 0x7fe8180ffad0 2023-07-14T11:36:48.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.866+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe818067300 con 0x7fe8180ffad0 2023-07-14T11:36:48.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.868+0000 7fe8157fa700 1 -- 172.21.15.161:0/2853200372 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe80800cd40 con 0x7fe8180ffad0 2023-07-14T11:36:48.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.869+0000 7fe8157fa700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8000a1380 0x7fe8000a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:48.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.869+0000 7fe8157fa700 1 -- 172.21.15.161:0/2853200372 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fe808014070 con 0x7fe8180ffad0 2023-07-14T11:36:48.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.869+0000 7fe8177fe700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8000a1380 0x7fe8000a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:48.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.870+0000 7fe8177fe700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8000a1380 0x7fe8000a3830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe818105110 tx=0x7fe80c009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:48.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:48.875+0000 7fe8157fa700 1 -- 172.21.15.161:0/2853200372 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe808051790 con 0x7fe8180ffad0 2023-07-14T11:36:49.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.254+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8181968e0 con 0x7fe8180ffad0 2023-07-14T11:36:49.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.255+0000 7fe8157fa700 1 -- 172.21.15.161:0/2853200372 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+10873 (secure 0 0 0) 0x7fe8181968e0 con 0x7fe8180ffad0 2023-07-14T11:36:49.256 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:36:49.256 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:35:27.518481+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:36:49.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8000a1380 msgr2=0x7fe8000a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8000a1380 0x7fe8000a3830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe818105110 tx=0x7fe80c009260 comp rx=0 tx=0).stop 2023-07-14T11:36:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 msgr2=0x7fe81819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 0x7fe81819ba60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe80800e9a0 tx=0x7fe80800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 shutdown_connections 2023-07-14T11:36:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe8000a1380 0x7fe8000a3830 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe818105ae0 0x7fe818195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe8180ffad0 0x7fe81819ba60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 --2- 172.21.15.161:0/2853200372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe818100480 0x7fe81819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 >> 172.21.15.161:0/2853200372 conn(0x7fe8180fb590 msgr2=0x7fe8181035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 shutdown_connections 2023-07-14T11:36:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.260+0000 7fe81e3ff700 1 -- 172.21.15.161:0/2853200372 wait complete. 2023-07-14T11:36:49.263 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 37 2023-07-14T11:36:49.292 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:36:49.573 INFO:tasks.workunit.client.0.smithi161.stdout: 10 58062 36.29 MB/sec warmup 91 sec latency 85.880 ms 2023-07-14T11:36:49.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 -- 172.21.15.161:0/202116649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac072850 msgr2=0x7f21ac072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 --2- 172.21.15.161:0/202116649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac072850 0x7f21ac072c20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f219c009990 tx=0x7f219c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 -- 172.21.15.161:0/202116649 shutdown_connections 2023-07-14T11:36:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 --2- 172.21.15.161:0/202116649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac10c3a0 0x7f21ac10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 --2- 172.21.15.161:0/202116649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac072850 0x7f21ac072c20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 --2- 172.21.15.161:0/202116649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 0x7f21ac10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 -- 172.21.15.161:0/202116649 >> 172.21.15.161:0/202116649 conn(0x7f21ac0fb600 msgr2=0x7f21ac0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 -- 172.21.15.161:0/202116649 shutdown_connections 2023-07-14T11:36:49.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.587+0000 7f21b09ff700 1 -- 172.21.15.161:0/202116649 wait complete. 2023-07-14T11:36:49.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.588+0000 7f21b09ff700 1 Processor -- start 2023-07-14T11:36:49.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.588+0000 7f21b09ff700 1 -- start start 2023-07-14T11:36:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21b09ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac072850 0x7f21ac197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21b09ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 0x7f21ac1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21b09ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac10c3a0 0x7f21ac19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21b09ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac19ce70 con 0x7f21ac072850 2023-07-14T11:36:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21b09ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac19cfe0 con 0x7f21ac10c3a0 2023-07-14T11:36:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21b09ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac19d1a0 con 0x7f21ac073160 2023-07-14T11:36:49.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21aa59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac072850 0x7f21ac197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 0x7f21ac1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 0x7f21ac1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45842/0 (socket says 172.21.15.161:45842) 2023-07-14T11:36:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 -- 172.21.15.161:0/2003645453 learned_addr learned my addr 172.21.15.161:0/2003645453 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21aad9d700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac10c3a0 0x7f21ac19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 -- 172.21.15.161:0/2003645453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac10c3a0 msgr2=0x7f21ac19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac10c3a0 0x7f21ac19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 -- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac072850 msgr2=0x7f21ac197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac072850 0x7f21ac197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21a9d9b700 1 -- 172.21.15.161:0/2003645453 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0095f0 con 0x7f21ac073160 2023-07-14T11:36:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.589+0000 7f21aa59c700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac072850 0x7f21ac197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.590+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 0x7f21ac1983d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f21a000e9a0 tx=0x7f21a000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.590+0000 7f219b7fe700 1 -- 172.21.15.161:0/2003645453 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0009790 con 0x7f21ac073160 2023-07-14T11:36:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.590+0000 7f219b7fe700 1 -- 172.21.15.161:0/2003645453 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21a0010430 con 0x7f21ac073160 2023-07-14T11:36:49.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.590+0000 7f219b7fe700 1 -- 172.21.15.161:0/2003645453 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0018680 con 0x7f21ac073160 2023-07-14T11:36:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.590+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac19d480 con 0x7f21ac073160 2023-07-14T11:36:49.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.590+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac1a4cc0 con 0x7f21ac073160 2023-07-14T11:36:49.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.595+0000 7f219b7fe700 1 -- 172.21.15.161:0/2003645453 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f21a00105a0 con 0x7f21ac073160 2023-07-14T11:36:49.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.596+0000 7f21c4282b80 1 Processor -- start 2023-07-14T11:36:49.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.596+0000 7f21c4282b80 1 -- start start 2023-07-14T11:36:49.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.596+0000 7f219b7fe700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f21940a1340 0x7f21940a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.596+0000 7f219b7fe700 1 -- 172.21.15.161:0/2003645453 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f21a0014070 con 0x7f21ac073160 2023-07-14T11:36:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.596+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db102d0 0x55574db106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db10be0 0x55574db1b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21c4282b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574da16cb0 con 0x55574db1bad0 2023-07-14T11:36:49.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21c4282b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574da69a40 con 0x55574db10be0 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21c4282b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574da62ed0 con 0x55574db102d0 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aad9d700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f21940a1340 0x7f21940a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aa59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db102d0 0x55574db106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aa59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db102d0 0x55574db106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45852/0 (socket says 172.21.15.161:45852) 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aa59c700 1 -- 172.21.15.161:0/3271503043 learned_addr learned my addr 172.21.15.161:0/3271503043 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db10be0 0x55574db1b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aad9d700 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aa59c700 1 -- 172.21.15.161:0/3271503043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db10be0 msgr2=0x55574db1b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aa59c700 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db10be0 0x55574db1b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.597+0000 7f21aa59c700 1 -- 172.21.15.161:0/3271503043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 msgr2=0x55574db1deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f21aa59c700 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f21aa59c700 1 -- 172.21.15.161:0/3271503043 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55574d890de0 con 0x55574db102d0 2023-07-14T11:36:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f21aad9d700 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f21aa59c700 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db102d0 0x55574db106a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f219c005ea0 tx=0x7f219c005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19d77904bb5f297 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f21aad9d700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f21940a1340 0x7f21940a37f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009ec0 tx=0x7f21a4009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f2198ff9700 1 -- 172.21.15.161:0/3271503043 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c026070 con 0x55574db102d0 2023-07-14T11:36:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f2198ff9700 1 -- 172.21.15.161:0/3271503043 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f219c01e040 con 0x55574db102d0 2023-07-14T11:36:49.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.598+0000 7f2198ff9700 1 -- 172.21.15.161:0/3271503043 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c01d070 con 0x55574db102d0 2023-07-14T11:36:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 -- 172.21.15.161:0/3271503043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db102d0 msgr2=0x55574db106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db102d0 0x55574db106a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f219c005ea0 tx=0x7f219c005ed0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 -- 172.21.15.161:0/3271503043 shutdown_connections 2023-07-14T11:36:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db102d0 0x55574db106a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3271503043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db10be0 0x55574db1b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 -- 172.21.15.161:0/3271503043 >> 172.21.15.161:0/3271503043 conn(0x55574db06da0 msgr2=0x55574db0c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.599+0000 7f21c4282b80 1 -- 172.21.15.161:0/3271503043 shutdown_connections 2023-07-14T11:36:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.600+0000 7f21c4282b80 1 -- 172.21.15.161:0/3271503043 wait complete. 2023-07-14T11:36:49.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.600+0000 7f21c4282b80 1 -- start start 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 1 Processor -- start 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db18770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db18e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574da02ac0 con 0x55574db102d0 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574da027c0 con 0x55574db1bad0 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574d8e6920 con 0x55574db10be0 2023-07-14T11:36:49.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21c4282b80 10 client.0 _ll_register_callbacks cb 0x55574da00760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21a9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db18770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21aad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db18e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21a9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db18770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45866/0 (socket says 172.21.15.161:45866) 2023-07-14T11:36:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21a9d9b700 1 -- 172.21.15.161:0/3438616706 learned_addr learned my addr 172.21.15.161:0/3438616706 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.601+0000 7f21aa59c700 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21a9d9b700 1 -- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 msgr2=0x55574db18e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db18e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21a9d9b700 1 -- 172.21.15.161:0/3438616706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 msgr2=0x55574db1b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db1b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21a9d9b700 1 -- 172.21.15.161:0/3438616706 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac10f3c0 con 0x55574db10be0 2023-07-14T11:36:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21aad9d700 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db18e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db18770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f21a00962a0 tx=0x7f21a0099e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=198aef47fa9b5eaa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f2191ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:49.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f2191ffb700 1 -- 172.21.15.161:0/3438616706 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00994e0 con 0x55574db10be0 2023-07-14T11:36:49.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f2191ffb700 1 -- 172.21.15.161:0/3438616706 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f21a00a2240 con 0x55574db10be0 2023-07-14T11:36:49.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.602+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55574d9bc4f0 con 0x55574db10be0 2023-07-14T11:36:49.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.603+0000 7f2191ffb700 1 -- 172.21.15.161:0/3438616706 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21a00b8080 con 0x55574db10be0 2023-07-14T11:36:49.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.603+0000 7f21c4282b80 10 client.27193 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:49.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.603+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55574d879c60 con 0x55574db10be0 2023-07-14T11:36:49.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.603+0000 7f2191ffb700 1 -- 172.21.15.161:0/3438616706 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21a00c8020 con 0x55574db10be0 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.603+0000 7f2191ffb700 1 -- 172.21.15.161:0/3438616706 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f21a00cabc0 con 0x55574db10be0 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.604+0000 7f21c4282b80 10 client.27193 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.604+0000 7f21c4282b80 10 client.27193 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.604+0000 7f21c4282b80 20 client.27193 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.604+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db33c60 0x55574db36070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.604+0000 7f21c4282b80 4 client.27193 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.604+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55574da5beb0 con 0x55574db33c60 2023-07-14T11:36:49.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.604+0000 7f21aa59c700 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db33c60 0x55574db36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.605+0000 7f21aa59c700 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db33c60 0x55574db36070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fbb48d5f268c906e server_cookie=c7fd6c32fda2bd5f in_seq=0 out_seq=0 2023-07-14T11:36:49.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.605+0000 7f2191ffb700 10 client.27193 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:49.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f2191ffb700 1 -- 172.21.15.161:0/3438616706 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55574da5beb0 con 0x55574db33c60 2023-07-14T11:36:49.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f2191ffb700 10 client.27193 handle_command_reply: tid=0 2023-07-14T11:36:49.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 client.27193 shutdown 2023-07-14T11:36:49.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 msgr2=0x55574db18770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db18770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f21a00962a0 tx=0x7f21a0099e70 comp rx=0 tx=0).stop 2023-07-14T11:36:49.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 shutdown_connections 2023-07-14T11:36:49.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db33c60 0x55574db36070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db1b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db18e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 --2- 172.21.15.161:0/3438616706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db18770 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 >> 172.21.15.161:0/3438616706 conn(0x55574db06da0 msgr2=0x55574db0c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f2191ffb700 0 client.27193 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:49.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.607+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 shutdown_connections 2023-07-14T11:36:49.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.608+0000 7f21c4282b80 1 -- 172.21.15.161:0/3438616706 wait complete. 2023-07-14T11:36:49.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.608+0000 7f21c4282b80 20 client.27193 trim_cache size 0 max 16384 2023-07-14T11:36:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 Processor -- start 2023-07-14T11:36:49.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 -- start start 2023-07-14T11:36:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db102d0 0x55574db17f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574db06970 con 0x55574db1bad0 2023-07-14T11:36:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574db121a0 con 0x55574db102d0 2023-07-14T11:36:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.627+0000 7f21c4282b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574db16cf0 con 0x55574db10be0 2023-07-14T11:36:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21aa59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db102d0 0x55574db17f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21aa59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db102d0 0x55574db17f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46932/0 (socket says 172.21.15.161:46932) 2023-07-14T11:36:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21a9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21aa59c700 1 -- 172.21.15.161:0/1521006870 learned_addr learned my addr 172.21.15.161:0/1521006870 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:49.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21aad9d700 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21a9d9b700 1 -- 172.21.15.161:0/1521006870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db102d0 msgr2=0x55574db17f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db102d0 0x55574db17f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21a9d9b700 1 -- 172.21.15.161:0/1521006870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 msgr2=0x55574db1b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.628+0000 7f21a9d9b700 1 -- 172.21.15.161:0/1521006870 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55574d8b7250 con 0x55574db10be0 2023-07-14T11:36:49.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.630+0000 7f21a9d9b700 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db35c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21a0099af0 tx=0x7f21a0010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=939970232ebeb60f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.633+0000 7f21997fa700 1 -- 172.21.15.161:0/1521006870 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0095be0 con 0x55574db10be0 2023-07-14T11:36:49.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.633+0000 7f21997fa700 1 -- 172.21.15.161:0/1521006870 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21a00c8070 con 0x55574db10be0 2023-07-14T11:36:49.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.633+0000 7f21997fa700 1 -- 172.21.15.161:0/1521006870 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0095f10 con 0x55574db10be0 2023-07-14T11:36:49.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 -- 172.21.15.161:0/1521006870 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 msgr2=0x55574db35c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db35c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21a0099af0 tx=0x7f21a0010a90 comp rx=0 tx=0).stop 2023-07-14T11:36:49.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 -- 172.21.15.161:0/1521006870 shutdown_connections 2023-07-14T11:36:49.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db1bad0 0x55574db1b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db102d0 0x55574db17f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1521006870 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db35c70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 -- 172.21.15.161:0/1521006870 >> 172.21.15.161:0/1521006870 conn(0x55574db06da0 msgr2=0x55574db0ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 -- 172.21.15.161:0/1521006870 shutdown_connections 2023-07-14T11:36:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.634+0000 7f21c4282b80 1 -- 172.21.15.161:0/1521006870 wait complete. 2023-07-14T11:36:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.635+0000 7f21c4282b80 1 -- start start 2023-07-14T11:36:49.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.635+0000 7f21c4282b80 1 Processor -- start 2023-07-14T11:36:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db2af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db28d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21c4282b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21c4282b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574db29b60 con 0x55574db102d0 2023-07-14T11:36:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21c4282b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574db86b20 con 0x55574db1bad0 2023-07-14T11:36:49.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21c4282b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574db86c60 con 0x55574db10be0 2023-07-14T11:36:49.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21c4282b80 10 client.0 _ll_register_callbacks cb 0x7f21a0099a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:49.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21a9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db28d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21aa59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db2af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21a9d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db28d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45892/0 (socket says 172.21.15.161:45892) 2023-07-14T11:36:49.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21a9d9b700 1 -- 172.21.15.161:0/1622532302 learned_addr learned my addr 172.21.15.161:0/1622532302 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:49.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.636+0000 7f21aad9d700 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.637+0000 7f21aad9d700 1 -- 172.21.15.161:0/1622532302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 msgr2=0x55574db28d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.637+0000 7f21aad9d700 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db28d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.637+0000 7f21aad9d700 1 -- 172.21.15.161:0/1622532302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 msgr2=0x55574db2af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.637+0000 7f21aad9d700 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db2af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.637+0000 7f21aad9d700 1 -- 172.21.15.161:0/1622532302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55574d879c60 con 0x55574db1bad0 2023-07-14T11:36:49.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.637+0000 7f21aad9d700 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db29450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009550 tx=0x7f21a4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3112acb6bdc84789 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.637+0000 7f2191ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:36:49.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.639+0000 7f2191ffb700 1 -- 172.21.15.161:0/1622532302 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a400e070 con 0x55574db1bad0 2023-07-14T11:36:49.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.639+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f219c01e040 con 0x55574db1bad0 2023-07-14T11:36:49.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.639+0000 7f2191ffb700 1 -- 172.21.15.161:0/1622532302 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f21a4021040 con 0x55574db1bad0 2023-07-14T11:36:49.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.639+0000 7f2191ffb700 1 -- 172.21.15.161:0/1622532302 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21a4018ae0 con 0x55574db1bad0 2023-07-14T11:36:49.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.639+0000 7f21c4282b80 10 client.27185 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:49.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.639+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55574d88f2b0 con 0x55574db1bad0 2023-07-14T11:36:49.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.640+0000 7f2191ffb700 1 -- 172.21.15.161:0/1622532302 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f219c01e040 con 0x55574db1bad0 2023-07-14T11:36:49.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f2191ffb700 1 -- 172.21.15.161:0/1622532302 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f21a40193f0 con 0x55574db1bad0 2023-07-14T11:36:49.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f21c4282b80 10 client.27185 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f21c4282b80 10 client.27185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:36:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f21c4282b80 20 client.27185 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db883e0 0x55574db8a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f21c4282b80 4 client.27185 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:36:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55574da1e050 con 0x55574db883e0 2023-07-14T11:36:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.641+0000 7f21aa59c700 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db883e0 0x55574db8a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.642+0000 7f21aa59c700 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db883e0 0x55574db8a7f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d611111e1cb4b8bd server_cookie=3dab670209428f22 in_seq=0 out_seq=0 2023-07-14T11:36:49.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.643+0000 7f2191ffb700 10 client.27185 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.646+0000 7f2191ffb700 1 -- 172.21.15.161:0/1622532302 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55574da1e050 con 0x55574db883e0 2023-07-14T11:36:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.646+0000 7f2191ffb700 10 client.27185 handle_command_reply: tid=0 2023-07-14T11:36:49.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 client.27185 shutdown 2023-07-14T11:36:49.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 msgr2=0x55574db29450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db29450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009550 tx=0x7f21a4004ad0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 shutdown_connections 2023-07-14T11:36:49.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55574db883e0 0x55574db8a7f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55574db102d0 0x55574db2af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55574db1bad0 0x55574db29450 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 --2- 172.21.15.161:0/1622532302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55574db10be0 0x55574db28d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.647+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 >> 172.21.15.161:0/1622532302 conn(0x55574db06da0 msgr2=0x55574db35c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.648+0000 7f2191ffb700 0 client.27185 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:36:49.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.648+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 shutdown_connections 2023-07-14T11:36:49.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.648+0000 7f21c4282b80 1 -- 172.21.15.161:0/1622532302 wait complete. 2023-07-14T11:36:49.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.648+0000 7f21c4282b80 20 client.27185 trim_cache size 0 max 16384 2023-07-14T11:36:49.650 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:36:49.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f21940a1340 msgr2=0x7f21940a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f21940a1340 0x7f21940a37f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009ec0 tx=0x7f21a4009920 comp rx=0 tx=0).stop 2023-07-14T11:36:49.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 msgr2=0x7f21ac1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 0x7f21ac1983d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f21a000e9a0 tx=0x7f21a000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 shutdown_connections 2023-07-14T11:36:49.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f21940a1340 0x7f21940a37f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac072850 0x7f21ac197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21ac10c3a0 0x7f21ac19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 --2- 172.21.15.161:0/2003645453 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f21ac073160 0x7f21ac1983d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 >> 172.21.15.161:0/2003645453 conn(0x7f21ac0fb600 msgr2=0x7f21ac10e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 shutdown_connections 2023-07-14T11:36:49.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.649+0000 7f21b09ff700 1 -- 172.21.15.161:0/2003645453 wait complete. 2023-07-14T11:36:49.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:49 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:36:49.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:49 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:36:49.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:49 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:36:49.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:49 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2853200372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:36:49.670 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:49 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:36:49.670 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:49 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:36:49.671 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:49 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:36:49.671 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2853200372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:36:49.673 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:36:49.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:49 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:36:49.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:49 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:36:49.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:49 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:36:49.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:49 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2853200372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:36:49.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.974+0000 7f5d491ff700 1 -- 172.21.15.161:0/3146039348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440728b0 msgr2=0x7f5d44072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.974+0000 7f5d491ff700 1 --2- 172.21.15.161:0/3146039348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440728b0 0x7f5d44072c80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009990 tx=0x7f5d34009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.974+0000 7f5d491ff700 1 -- 172.21.15.161:0/3146039348 shutdown_connections 2023-07-14T11:36:49.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.974+0000 7f5d491ff700 1 --2- 172.21.15.161:0/3146039348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d44110af0 0x7f5d44112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.974+0000 7f5d491ff700 1 --2- 172.21.15.161:0/3146039348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440728b0 0x7f5d44072c80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.974+0000 7f5d491ff700 1 --2- 172.21.15.161:0/3146039348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440731c0 0x7f5d441105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.974+0000 7f5d491ff700 1 -- 172.21.15.161:0/3146039348 >> 172.21.15.161:0/3146039348 conn(0x7f5d440fb5e0 msgr2=0x7f5d440fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.975+0000 7f5d491ff700 1 -- 172.21.15.161:0/3146039348 shutdown_connections 2023-07-14T11:36:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.975+0000 7f5d491ff700 1 -- 172.21.15.161:0/3146039348 wait complete. 2023-07-14T11:36:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.976+0000 7f5d491ff700 1 Processor -- start 2023-07-14T11:36:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.977+0000 7f5d491ff700 1 -- start start 2023-07-14T11:36:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d491ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440728b0 0x7f5d4419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d491ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d440731c0 0x7f5d4419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d491ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d44110af0 0x7f5d441a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d491ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d441a1200 con 0x7f5d440731c0 2023-07-14T11:36:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d491ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d441a1370 con 0x7f5d44110af0 2023-07-14T11:36:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d491ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d441a1530 con 0x7f5d440728b0 2023-07-14T11:36:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440728b0 0x7f5d4419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440728b0 0x7f5d4419c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45908/0 (socket says 172.21.15.161:45908) 2023-07-14T11:36:49.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d42d9d700 1 -- 172.21.15.161:0/346541533 learned_addr learned my addr 172.21.15.161:0/346541533 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d4359e700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d44110af0 0x7f5d441a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.978+0000 7f5d4259c700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d440731c0 0x7f5d4419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.979+0000 7f5d42d9d700 1 -- 172.21.15.161:0/346541533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d44110af0 msgr2=0x7f5d441a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.979+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d44110af0 0x7f5d441a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.979+0000 7f5d42d9d700 1 -- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d440731c0 msgr2=0x7f5d4419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.979+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d440731c0 0x7f5d4419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.979+0000 7f5d42d9d700 1 -- 172.21.15.161:0/346541533 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d340095f0 con 0x7f5d440728b0 2023-07-14T11:36:49.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.979+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440728b0 0x7f5d4419c220 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5d34005910 tx=0x7f5d34005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.979+0000 7f5d2bfff700 1 -- 172.21.15.161:0/346541533 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401d070 con 0x7f5d440728b0 2023-07-14T11:36:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.980+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d441a17b0 con 0x7f5d440728b0 2023-07-14T11:36:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.980+0000 7f5d2bfff700 1 -- 172.21.15.161:0/346541533 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d34022470 con 0x7f5d440728b0 2023-07-14T11:36:49.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.980+0000 7f5d2bfff700 1 -- 172.21.15.161:0/346541533 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34017650 con 0x7f5d440728b0 2023-07-14T11:36:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.980+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d441a9050 con 0x7f5d440728b0 2023-07-14T11:36:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.984+0000 7f5d2bfff700 1 -- 172.21.15.161:0/346541533 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5d3400fad0 con 0x7f5d440728b0 2023-07-14T11:36:49.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 Processor -- start 2023-07-14T11:36:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 -- start start 2023-07-14T11:36:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc012d0 0x562d7bc016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc0cad0 0x562d7bc0eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bb07cb0 con 0x562d7bc012d0 2023-07-14T11:36:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bb5aa40 con 0x562d7bc01be0 2023-07-14T11:36:49.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.986+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bb53ed0 con 0x562d7bc0cad0 2023-07-14T11:36:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d2bfff700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d2c0a1340 0x7f5d2c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc012d0 0x562d7bc016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:46976/0 (socket says 172.21.15.161:46976) 2023-07-14T11:36:49.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d42d9d700 1 -- 172.21.15.161:0/391096617 learned_addr learned my addr 172.21.15.161:0/391096617 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:49.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d2bfff700 1 -- 172.21.15.161:0/346541533 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d340ce4f0 con 0x7f5d440728b0 2023-07-14T11:36:49.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4359e700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d2c0a1340 0x7f5d2c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4259c700 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc0cad0 0x562d7bc0eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:49.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4359e700 1 -- 172.21.15.161:0/391096617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc0cad0 msgr2=0x562d7bc0eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4359e700 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc0cad0 0x562d7bc0eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4359e700 1 -- 172.21.15.161:0/391096617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 msgr2=0x562d7bc0c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:49.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4359e700 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.987+0000 7f5d4359e700 1 -- 172.21.15.161:0/391096617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d7b981de0 con 0x562d7bc012d0 2023-07-14T11:36:49.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.988+0000 7f5d4359e700 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc012d0 0x562d7bc016a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d44069a80 tx=0x7f5d3c007a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf978cd2915bff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.988+0000 7f5d4359e700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d2c0a1340 0x7f5d2c0a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007fd0 tx=0x7f5d3c01c000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:49.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.988+0000 7f5d297fa700 1 -- 172.21.15.161:0/391096617 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c009e80 con 0x562d7bc012d0 2023-07-14T11:36:49.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.988+0000 7f5d297fa700 1 -- 172.21.15.161:0/391096617 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d3c00f040 con 0x562d7bc012d0 2023-07-14T11:36:49.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.988+0000 7f5d297fa700 1 -- 172.21.15.161:0/391096617 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004730 con 0x562d7bc012d0 2023-07-14T11:36:49.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/391096617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc012d0 msgr2=0x562d7bc016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:49.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc012d0 0x562d7bc016a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d44069a80 tx=0x7f5d3c007a70 comp rx=0 tx=0).stop 2023-07-14T11:36:49.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/391096617 shutdown_connections 2023-07-14T11:36:49.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc012d0 0x562d7bc016a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc0cad0 0x562d7bc0eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/391096617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:49.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/391096617 >> 172.21.15.161:0/391096617 conn(0x562d7bbf7da0 msgr2=0x562d7bbfd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:49.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/391096617 shutdown_connections 2023-07-14T11:36:49.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.989+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/391096617 wait complete. 2023-07-14T11:36:49.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.990+0000 7f5d5ca93b80 1 -- start start 2023-07-14T11:36:49.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.990+0000 7f5d5ca93b80 1 Processor -- start 2023-07-14T11:36:50.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc0c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc01be0 0x562d7bc096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc0cad0 0x562d7bc09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7baf3ac0 con 0x562d7bc01be0 2023-07-14T11:36:50.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7baf37c0 con 0x562d7bc0cad0 2023-07-14T11:36:50.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7b9d7920 con 0x562d7bc012d0 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d5ca93b80 10 client.0 _ll_register_callbacks cb 0x562d7baf1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc0c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc0c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45920/0 (socket says 172.21.15.161:45920) 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d42d9d700 1 -- 172.21.15.161:0/3263414732 learned_addr learned my addr 172.21.15.161:0/3263414732 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.991+0000 7f5d4359e700 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc0cad0 0x562d7bc09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d4259c700 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc01be0 0x562d7bc096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d42d9d700 1 -- 172.21.15.161:0/3263414732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc0cad0 msgr2=0x562d7bc09d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc0cad0 0x562d7bc09d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d42d9d700 1 -- 172.21.15.161:0/3263414732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc01be0 msgr2=0x562d7bc096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc01be0 0x562d7bc096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d42d9d700 1 -- 172.21.15.161:0/3263414732 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c011040 con 0x562d7bc012d0 2023-07-14T11:36:50.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc0c470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5d340a3160 tx=0x7f5d34009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea1aaea95320a01 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:50.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d26ff5700 1 -- 172.21.15.161:0/3263414732 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3409f120 con 0x562d7bc012d0 2023-07-14T11:36:50.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d26ff5700 1 -- 172.21.15.161:0/3263414732 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d3409b2d0 con 0x562d7bc012d0 2023-07-14T11:36:50.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d7baad4f0 con 0x562d7bc012d0 2023-07-14T11:36:50.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d26ff5700 1 -- 172.21.15.161:0/3263414732 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d3409f300 con 0x562d7bc012d0 2023-07-14T11:36:50.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d5ca93b80 10 client.27211 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:50.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.992+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562d7b96ac60 con 0x562d7bc012d0 2023-07-14T11:36:50.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d26ff5700 1 -- 172.21.15.161:0/3263414732 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d7baad4f0 con 0x562d7bc012d0 2023-07-14T11:36:50.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d26ff5700 1 -- 172.21.15.161:0/3263414732 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5d3409a8e0 con 0x562d7bc012d0 2023-07-14T11:36:50.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d5ca93b80 10 client.27211 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:50.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d5ca93b80 10 client.27211 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:36:50.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d5ca93b80 20 client.27211 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:50.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc1c160 0x562d7bc1e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d5ca93b80 4 client.27211 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:50.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.993+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d7bb4ceb0 con 0x562d7bc1c160 2023-07-14T11:36:50.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.994+0000 7f5d4259c700 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc1c160 0x562d7bc1e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.994+0000 7f5d4259c700 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc1c160 0x562d7bc1e570 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d63d89c96b7615f server_cookie=5dbbea96c15e7b65 in_seq=0 out_seq=0 2023-07-14T11:36:50.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.994+0000 7f5d26ff5700 10 client.27211 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:50.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d26ff5700 1 -- 172.21.15.161:0/3263414732 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d7bb4ceb0 con 0x562d7bc1c160 2023-07-14T11:36:50.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d26ff5700 10 client.27211 handle_command_reply: tid=0 2023-07-14T11:36:50.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 client.27211 shutdown 2023-07-14T11:36:50.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 msgr2=0x562d7bc0c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc0c470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5d340a3160 tx=0x7f5d34009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 shutdown_connections 2023-07-14T11:36:50.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc1c160 0x562d7bc1e570 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc0cad0 0x562d7bc09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc0c470 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/3263414732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc01be0 0x562d7bc096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 >> 172.21.15.161:0/3263414732 conn(0x562d7bbf7da0 msgr2=0x562d7bbfd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.995+0000 7f5d26ff5700 0 client.27211 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:50.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.996+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 shutdown_connections 2023-07-14T11:36:50.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.996+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/3263414732 wait complete. 2023-07-14T11:36:50.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:49.996+0000 7f5d5ca93b80 20 client.27211 trim_cache size 0 max 16384 2023-07-14T11:36:50.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 Processor -- start 2023-07-14T11:36:50.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 -- start start 2023-07-14T11:36:50.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc13510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bc0a320 con 0x562d7bc0cad0 2023-07-14T11:36:50.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bbf7970 con 0x562d7bc01be0 2023-07-14T11:36:50.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.012+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bc031a0 con 0x562d7bc012d0 2023-07-14T11:36:50.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc13510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc13510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45924/0 (socket says 172.21.15.161:45924) 2023-07-14T11:36:50.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 -- 172.21.15.161:0/2016901302 learned_addr learned my addr 172.21.15.161:0/2016901302 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d4259c700 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d4359e700 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 -- 172.21.15.161:0/2016901302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 msgr2=0x562d7bc0c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 -- 172.21.15.161:0/2016901302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 msgr2=0x562d7bc08eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc08eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 -- 172.21.15.161:0/2016901302 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d7b9a8250 con 0x562d7bc012d0 2023-07-14T11:36:50.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc13510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5d34005f50 tx=0x7f5d3400ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9decb9a1c9963f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d29ffb700 1 -- 172.21.15.161:0/2016901302 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3409f120 con 0x562d7bc012d0 2023-07-14T11:36:50.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d29ffb700 1 -- 172.21.15.161:0/2016901302 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d34066900 con 0x562d7bc012d0 2023-07-14T11:36:50.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.013+0000 7f5d29ffb700 1 -- 172.21.15.161:0/2016901302 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340ce330 con 0x562d7bc012d0 2023-07-14T11:36:50.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/2016901302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 msgr2=0x562d7bc13510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc13510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5d34005f50 tx=0x7f5d3400ff90 comp rx=0 tx=0).stop 2023-07-14T11:36:50.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/2016901302 shutdown_connections 2023-07-14T11:36:50.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc08eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc13510 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/2016901302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc0c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/2016901302 >> 172.21.15.161:0/2016901302 conn(0x562d7bbf7da0 msgr2=0x562d7bc04a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/2016901302 shutdown_connections 2023-07-14T11:36:50.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.014+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/2016901302 wait complete. 2023-07-14T11:36:50.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.015+0000 7f5d5ca93b80 1 -- start start 2023-07-14T11:36:50.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.015+0000 7f5d5ca93b80 1 Processor -- start 2023-07-14T11:36:50.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc68300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc68840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d5ca93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc0f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bc0fc00 con 0x562d7bc0cad0 2023-07-14T11:36:50.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bc0fd70 con 0x562d7bc01be0 2023-07-14T11:36:50.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d5ca93b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7bc0ff30 con 0x562d7bc012d0 2023-07-14T11:36:50.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d5ca93b80 10 client.0 _ll_register_callbacks cb 0x562d7baf1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:50.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc68300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d4259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc68840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc68300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45940/0 (socket says 172.21.15.161:45940) 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 -- 172.21.15.161:0/1860743153 learned_addr learned my addr 172.21.15.161:0/1860743153 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d4359e700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc0f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 -- 172.21.15.161:0/1860743153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 msgr2=0x562d7bc68840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc68840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 -- 172.21.15.161:0/1860743153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 msgr2=0x562d7bc0f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc0f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d42d9d700 1 -- 172.21.15.161:0/1860743153 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d7b96ac60 con 0x562d7bc012d0 2023-07-14T11:36:50.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d4359e700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc0f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:50.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.016+0000 7f5d4259c700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc68840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:50.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d42d9d700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc68300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d340977d0 tx=0x7f5d34086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=625974c2349cb334 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:50.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d26ff5700 1 -- 172.21.15.161:0/1860743153 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34097650 con 0x562d7bc012d0 2023-07-14T11:36:50.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d26ff5700 1 -- 172.21.15.161:0/1860743153 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f5d3409b2d0 con 0x562d7bc012d0 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d3c00f040 con 0x562d7bc012d0 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d26ff5700 1 -- 172.21.15.161:0/1860743153 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d3409f120 con 0x562d7bc012d0 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d5ca93b80 10 client.27217 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.017+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562d7b9802b0 con 0x562d7bc012d0 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d26ff5700 1 -- 172.21.15.161:0/1860743153 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d3c00f040 con 0x562d7bc012d0 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d26ff5700 1 -- 172.21.15.161:0/1860743153 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5d34011d30 con 0x562d7bc012d0 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d5ca93b80 10 client.27217 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:50.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d5ca93b80 10 client.27217 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:36:50.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d5ca93b80 20 client.27217 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:50.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc70d30 0x562d7bc73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d5ca93b80 4 client.27217 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:36:50.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.018+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d7bb0f050 con 0x562d7bc70d30 2023-07-14T11:36:50.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.019+0000 7f5d4259c700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc70d30 0x562d7bc73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.019+0000 7f5d4259c700 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc70d30 0x562d7bc73110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=366af39ef5958414 server_cookie=101b18e085906491 in_seq=0 out_seq=0 2023-07-14T11:36:50.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.019+0000 7f5d26ff5700 10 client.27217 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:50.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.019+0000 7f5d26ff5700 1 -- 172.21.15.161:0/1860743153 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d7bb0f050 con 0x562d7bc70d30 2023-07-14T11:36:50.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.019+0000 7f5d26ff5700 10 client.27217 handle_command_reply: tid=0 2023-07-14T11:36:50.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.019+0000 7f5d5ca93b80 1 client.27217 shutdown 2023-07-14T11:36:50.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 msgr2=0x562d7bc68300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc68300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d340977d0 tx=0x7f5d34086040 comp rx=0 tx=0).stop 2023-07-14T11:36:50.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 shutdown_connections 2023-07-14T11:36:50.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x562d7bc70d30 0x562d7bc73110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d7bc0cad0 0x562d7bc0f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562d7bc012d0 0x562d7bc68300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 --2- 172.21.15.161:0/1860743153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d7bc01be0 0x562d7bc68840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 >> 172.21.15.161:0/1860743153 conn(0x562d7bbf7da0 msgr2=0x562d7bc033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d26ff5700 0 client.27217 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 shutdown_connections 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 1 -- 172.21.15.161:0/1860743153 wait complete. 2023-07-14T11:36:50.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.020+0000 7f5d5ca93b80 20 client.27217 trim_cache size 0 max 16384 2023-07-14T11:36:50.031 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:36:50.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d2c0a1340 msgr2=0x7f5d2c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d2c0a1340 0x7f5d2c0a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007fd0 tx=0x7f5d3c01c000 comp rx=0 tx=0).stop 2023-07-14T11:36:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440728b0 msgr2=0x7f5d4419c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440728b0 0x7f5d4419c220 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5d34005910 tx=0x7f5d34005740 comp rx=0 tx=0).stop 2023-07-14T11:36:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 shutdown_connections 2023-07-14T11:36:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f5d2c0a1340 0x7f5d2c0a37f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d44110af0 0x7f5d441a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5d440728b0 0x7f5d4419c220 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 --2- 172.21.15.161:0/346541533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d440731c0 0x7f5d4419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 >> 172.21.15.161:0/346541533 conn(0x7f5d440fb5e0 msgr2=0x7f5d440fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 shutdown_connections 2023-07-14T11:36:50.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.021+0000 7f5d491ff700 1 -- 172.21.15.161:0/346541533 wait complete. 2023-07-14T11:36:50.041 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:36:50.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 -- 172.21.15.161:0/347925398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0101f70 msgr2=0x7f8de0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 --2- 172.21.15.161:0/347925398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0101f70 0x7f8de0102340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009920 tx=0x7f8ddc009c30 comp rx=0 tx=0).stop 2023-07-14T11:36:50.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 -- 172.21.15.161:0/347925398 shutdown_connections 2023-07-14T11:36:50.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 --2- 172.21.15.161:0/347925398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0101f70 0x7f8de0102340 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 --2- 172.21.15.161:0/347925398 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8de0106fb0 0x7f8de010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 --2- 172.21.15.161:0/347925398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 0x7f8de01068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 -- 172.21.15.161:0/347925398 >> 172.21.15.161:0/347925398 conn(0x7f8de00fd7e0 msgr2=0x7f8de00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 -- 172.21.15.161:0/347925398 shutdown_connections 2023-07-14T11:36:50.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.293+0000 7f8de77ff700 1 -- 172.21.15.161:0/347925398 wait complete. 2023-07-14T11:36:50.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.294+0000 7f8de77ff700 1 Processor -- start 2023-07-14T11:36:50.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.294+0000 7f8de77ff700 1 -- start start 2023-07-14T11:36:50.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de77ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8de0101f70 0x7f8de0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de77ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 0x7f8de0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de77ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0106fb0 0x7f8de019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de77ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de019cee0 con 0x7f8de0106fb0 2023-07-14T11:36:50.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de77ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de019d050 con 0x7f8de0102910 2023-07-14T11:36:50.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de77ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de019d210 con 0x7f8de0101f70 2023-07-14T11:36:50.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de5d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0106fb0 0x7f8de019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 0x7f8de0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 0x7f8de0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47016/0 (socket says 172.21.15.161:47016) 2023-07-14T11:36:50.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de4d9a700 1 -- 172.21.15.161:0/1883666698 learned_addr learned my addr 172.21.15.161:0/1883666698 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de559b700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8de0101f70 0x7f8de0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de5d9c700 1 -- 172.21.15.161:0/1883666698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8de0101f70 msgr2=0x7f8de0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8de0101f70 0x7f8de0197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de5d9c700 1 -- 172.21.15.161:0/1883666698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 msgr2=0x7f8de0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 0x7f8de0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de5d9c700 1 -- 172.21.15.161:0/1883666698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc009520 con 0x7f8de0106fb0 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.295+0000 7f8de4d9a700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 0x7f8de0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.296+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0106fb0 0x7f8de019c8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800efc0 tx=0x7f8dd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.296+0000 7f8dd67fc700 1 -- 172.21.15.161:0/1883666698 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8018070 con 0x7f8de0106fb0 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.296+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de019d490 con 0x7f8de0106fb0 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.296+0000 7f8dd67fc700 1 -- 172.21.15.161:0/1883666698 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd800f040 con 0x7f8de0106fb0 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.296+0000 7f8dd67fc700 1 -- 172.21.15.161:0/1883666698 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800cd30 con 0x7f8de0106fb0 2023-07-14T11:36:50.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.296+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de01a4d30 con 0x7f8de0106fb0 2023-07-14T11:36:50.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.300+0000 7f8dd67fc700 1 -- 172.21.15.161:0/1883666698 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8dd8003a00 con 0x7f8de0106fb0 2023-07-14T11:36:50.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.302+0000 7f8dd67fc700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8dcc0a1340 0x7f8dcc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.302+0000 7f8dfb1f5b80 1 Processor -- start 2023-07-14T11:36:50.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dd67fc700 1 -- 172.21.15.161:0/1883666698 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8dd80ccbd0 con 0x7f8de0106fb0 2023-07-14T11:36:50.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dfb1f5b80 1 -- start start 2023-07-14T11:36:50.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8de559b700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8dcc0a1340 0x7f8dcc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f32d0 0x5582613f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613fead0 0x558261400eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.305 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582612f9cb0 con 0x5582613f32d0 2023-07-14T11:36:50.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55826134ca40 con 0x5582613fead0 2023-07-14T11:36:50.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.303+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558261345ed0 con 0x5582613f3be0 2023-07-14T11:36:50.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45978/0 (socket says 172.21.15.161:45978) 2023-07-14T11:36:50.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de559b700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8dcc0a1340 0x7f8dcc0a37f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00c010 tx=0x7f8ddc019040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 -- 172.21.15.161:0/3167177319 learned_addr learned my addr 172.21.15.161:0/3167177319 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de559b700 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f32d0 0x5582613f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613fead0 0x558261400eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 -- 172.21.15.161:0/3167177319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613fead0 msgr2=0x558261400eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613fead0 0x558261400eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 -- 172.21.15.161:0/3167177319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f32d0 msgr2=0x5582613f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f32d0 0x5582613f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.304+0000 7f8de4d9a700 1 -- 172.21.15.161:0/3167177319 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558261173de0 con 0x5582613f3be0 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.305+0000 7f8de4d9a700 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fe590 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00115f0 tx=0x7f8dd0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f96c769c9dbbc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.305+0000 7f8dcb7fe700 1 -- 172.21.15.161:0/3167177319 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0004230 con 0x5582613f3be0 2023-07-14T11:36:50.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.305+0000 7f8dcb7fe700 1 -- 172.21.15.161:0/3167177319 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd0004390 con 0x5582613f3be0 2023-07-14T11:36:50.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.305+0000 7f8dcb7fe700 1 -- 172.21.15.161:0/3167177319 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0019920 con 0x5582613f3be0 2023-07-14T11:36:50.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/3167177319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 msgr2=0x5582613fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fe590 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00115f0 tx=0x7f8dd0011900 comp rx=0 tx=0).stop 2023-07-14T11:36:50.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/3167177319 shutdown_connections 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f32d0 0x5582613f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613fead0 0x558261400eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/3167177319 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fe590 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/3167177319 >> 172.21.15.161:0/3167177319 conn(0x5582613e9da0 msgr2=0x5582613ef6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/3167177319 shutdown_connections 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.306+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/3167177319 wait complete. 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.307+0000 7f8dfb1f5b80 1 -- start start 2023-07-14T11:36:50.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.307+0000 7f8dfb1f5b80 1 Processor -- start 2023-07-14T11:36:50.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x5582613fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x5582613fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582612e5ac0 con 0x5582613f3be0 2023-07-14T11:36:50.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582612e57c0 con 0x5582613f32d0 2023-07-14T11:36:50.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582611c9920 con 0x5582613fead0 2023-07-14T11:36:50.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8dfb1f5b80 10 client.0 _ll_register_callbacks cb 0x5582612e3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:50.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.308+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x5582613fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x5582613fb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37118/0 (socket says 172.21.15.161:37118) 2023-07-14T11:36:50.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de4d9a700 1 -- 172.21.15.161:0/510492724 learned_addr learned my addr 172.21.15.161:0/510492724 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de559b700 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x5582613fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de5d9c700 1 -- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 msgr2=0x5582613fe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de5d9c700 1 -- 172.21.15.161:0/510492724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 msgr2=0x5582613fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x5582613fb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de5d9c700 1 -- 172.21.15.161:0/510492724 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd0009dc0 con 0x5582613fead0 2023-07-14T11:36:50.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x5582613fbf20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80c9f60 tx=0x7f8dd8099530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c25090116505afb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.309+0000 7f8dc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:50.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.310+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/510492724 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8099dd0 con 0x5582613fead0 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.310+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55826129f4f0 con 0x5582613fead0 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.310+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/510492724 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8dd8099970 con 0x5582613fead0 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.310+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/510492724 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dd80ca3f0 con 0x5582613fead0 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.310+0000 7f8dfb1f5b80 10 client.27229 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.310+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55826115cc60 con 0x5582613fead0 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.311+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/510492724 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55826129f4f0 con 0x5582613fead0 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.311+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/510492724 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8dd80a18a0 con 0x5582613fead0 2023-07-14T11:36:50.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.312+0000 7f8dfb1f5b80 10 client.27229 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:50.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.312+0000 7f8dfb1f5b80 10 client.27229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:50.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.312+0000 7f8dfb1f5b80 20 client.27229 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:50.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.312+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558261416ba0 0x558261418fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.312+0000 7f8dfb1f5b80 4 client.27229 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:50.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.312+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55826133eeb0 con 0x558261416ba0 2023-07-14T11:36:50.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.312+0000 7f8de559b700 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558261416ba0 0x558261418fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.313+0000 7f8de559b700 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558261416ba0 0x558261418fb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78424a4daf312826 server_cookie=5b60fc020aff42e8 in_seq=0 out_seq=0 2023-07-14T11:36:50.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.313+0000 7f8dc8ff9700 10 client.27229 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:50.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.316+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/510492724 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55826133eeb0 con 0x558261416ba0 2023-07-14T11:36:50.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.316+0000 7f8dc8ff9700 10 client.27229 handle_command_reply: tid=0 2023-07-14T11:36:50.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.316+0000 7f8dfb1f5b80 1 client.27229 shutdown 2023-07-14T11:36:50.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.316+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 msgr2=0x5582613fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.316+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x5582613fbf20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80c9f60 tx=0x7f8dd8099530 comp rx=0 tx=0).stop 2023-07-14T11:36:50.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.317+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 shutdown_connections 2023-07-14T11:36:50.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.317+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558261416ba0 0x558261418fb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.317+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x5582613fbf20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.317+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.317+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/510492724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x5582613fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.317+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 >> 172.21.15.161:0/510492724 conn(0x5582613e9da0 msgr2=0x5582613ef290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.317+0000 7f8dc8ff9700 0 client.27229 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:50.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.319+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 shutdown_connections 2023-07-14T11:36:50.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.319+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/510492724 wait complete. 2023-07-14T11:36:50.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.320+0000 7f8dfb1f5b80 20 client.27229 trim_cache size 0 max 16384 2023-07-14T11:36:50.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 Processor -- start 2023-07-14T11:36:50.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 -- start start 2023-07-14T11:36:50.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613fead0 0x558261412f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582613fb5e0 con 0x5582613fead0 2023-07-14T11:36:50.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582614191d0 con 0x5582613f32d0 2023-07-14T11:36:50.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.339+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558261419390 con 0x5582613f3be0 2023-07-14T11:36:50.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.340+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.340+0000 7f8de4d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45996/0 (socket says 172.21.15.161:45996) 2023-07-14T11:36:50.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.340+0000 7f8de4d9a700 1 -- 172.21.15.161:0/2649614664 learned_addr learned my addr 172.21.15.161:0/2649614664 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.340+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613fead0 0x558261412f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.340+0000 7f8de559b700 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.340+0000 7f8de4d9a700 1 -- 172.21.15.161:0/2649614664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 msgr2=0x5582613fb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8de4d9a700 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8de4d9a700 1 -- 172.21.15.161:0/2649614664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613fead0 msgr2=0x558261412f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8de4d9a700 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613fead0 0x558261412f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8de4d9a700 1 -- 172.21.15.161:0/2649614664 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55826119a250 con 0x5582613f3be0 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8de559b700 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613fead0 0x558261412f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8de4d9a700 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fb030 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00115f0 tx=0x7f8dd0009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391ea9c4b3bd9fd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8dcbfff700 1 -- 172.21.15.161:0/2649614664 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0005530 con 0x5582613f3be0 2023-07-14T11:36:50.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8dcbfff700 1 -- 172.21.15.161:0/2649614664 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd0005690 con 0x5582613f3be0 2023-07-14T11:36:50.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.341+0000 7f8dcbfff700 1 -- 172.21.15.161:0/2649614664 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0019810 con 0x5582613f3be0 2023-07-14T11:36:50.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/2649614664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 msgr2=0x5582613fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fb030 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00115f0 tx=0x7f8dd0009380 comp rx=0 tx=0).stop 2023-07-14T11:36:50.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/2649614664 shutdown_connections 2023-07-14T11:36:50.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613fead0 0x558261412f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x5582613fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/2649614664 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613f3be0 0x5582613fb030 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/2649614664 >> 172.21.15.161:0/2649614664 conn(0x5582613e9da0 msgr2=0x55826141d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/2649614664 shutdown_connections 2023-07-14T11:36:50.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.342+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/2649614664 wait complete. 2023-07-14T11:36:50.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.343+0000 7f8dfb1f5b80 1 -- start start 2023-07-14T11:36:50.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.343+0000 7f8dfb1f5b80 1 Processor -- start 2023-07-14T11:36:50.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x55826140d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x55826140dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8dfb1f5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x55826140bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55826140c3f0 con 0x5582613f3be0 2023-07-14T11:36:50.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55826140c560 con 0x5582613f32d0 2023-07-14T11:36:50.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8dfb1f5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55826140c6a0 con 0x5582613fead0 2023-07-14T11:36:50.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8dfb1f5b80 10 client.0 _ll_register_callbacks cb 0x7f8dd8095cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:50.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8de5d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x55826140bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8de5d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x55826140bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46012/0 (socket says 172.21.15.161:46012) 2023-07-14T11:36:50.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.344+0000 7f8de5d9c700 1 -- 172.21.15.161:0/1326261731 learned_addr learned my addr 172.21.15.161:0/1326261731 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:50.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.345+0000 7f8de5d9c700 1 -- 172.21.15.161:0/1326261731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 msgr2=0x55826140d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.345+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x55826140d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.345+0000 7f8de5d9c700 1 -- 172.21.15.161:0/1326261731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 msgr2=0x55826140dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:50.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.345+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x55826140dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.345+0000 7f8de5d9c700 1 -- 172.21.15.161:0/1326261731 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558261173de0 con 0x5582613fead0 2023-07-14T11:36:50.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.345+0000 7f8de5d9c700 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x55826140bce0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80963e0 tx=0x7f8dd80093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cb354999022a834 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:50.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.345+0000 7f8dc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:50.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/1326261731 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80ca3f0 con 0x5582613fead0 2023-07-14T11:36:50.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/1326261731 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8dd8099970 con 0x5582613fead0 2023-07-14T11:36:50.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dd0004390 con 0x5582613fead0 2023-07-14T11:36:50.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/1326261731 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dd80992d0 con 0x5582613fead0 2023-07-14T11:36:50.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dfb1f5b80 10 client.27235 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:50.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5582611722b0 con 0x5582613fead0 2023-07-14T11:36:50.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/1326261731 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd0004390 con 0x5582613fead0 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/1326261731 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8dd8098b40 con 0x5582613fead0 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dfb1f5b80 10 client.27235 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dfb1f5b80 10 client.27235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.346+0000 7f8dfb1f5b80 20 client.27235 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.347+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55826147c7c0 0x55826147ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.347+0000 7f8dfb1f5b80 4 client.27235 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.347+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558261301050 con 0x55826147c7c0 2023-07-14T11:36:50.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.347+0000 7f8de559b700 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55826147c7c0 0x55826147ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:50.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.347+0000 7f8de559b700 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55826147c7c0 0x55826147ec30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e488dd1d98b9bf95 server_cookie=38d9da0ae98d65cb in_seq=0 out_seq=0 2023-07-14T11:36:50.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.347+0000 7f8dc8ff9700 10 client.27235 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:50.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dc8ff9700 1 -- 172.21.15.161:0/1326261731 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558261301050 con 0x55826147c7c0 2023-07-14T11:36:50.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dc8ff9700 10 client.27235 handle_command_reply: tid=0 2023-07-14T11:36:50.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 client.27235 shutdown 2023-07-14T11:36:50.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 msgr2=0x55826140bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x55826140bce0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80963e0 tx=0x7f8dd80093f0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 shutdown_connections 2023-07-14T11:36:50.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55826147c7c0 0x55826147ec30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5582613fead0 0x55826140bce0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582613f32d0 0x55826140d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 --2- 172.21.15.161:0/1326261731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582613f3be0 0x55826140dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 >> 172.21.15.161:0/1326261731 conn(0x5582613e9da0 msgr2=0x5582614124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dc8ff9700 0 client.27235 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:50.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 shutdown_connections 2023-07-14T11:36:50.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.352+0000 7f8dfb1f5b80 1 -- 172.21.15.161:0/1326261731 wait complete. 2023-07-14T11:36:50.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.353+0000 7f8dfb1f5b80 20 client.27235 trim_cache size 0 max 16384 2023-07-14T11:36:50.355 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:36:50.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8dcc0a1340 msgr2=0x7f8dcc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8dcc0a1340 0x7f8dcc0a37f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00c010 tx=0x7f8ddc019040 comp rx=0 tx=0).stop 2023-07-14T11:36:50.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0106fb0 msgr2=0x7f8de019c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:50.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0106fb0 0x7f8de019c8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800efc0 tx=0x7f8dd800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 shutdown_connections 2023-07-14T11:36:50.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8dcc0a1340 0x7f8dcc0a37f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8de0106fb0 0x7f8de019c8a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8de0101f70 0x7f8de0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 --2- 172.21.15.161:0/1883666698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de0102910 0x7f8de0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:50.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.354+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 >> 172.21.15.161:0/1883666698 conn(0x7f8de00fd7e0 msgr2=0x7f8de00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:50.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.355+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 shutdown_connections 2023-07-14T11:36:50.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:50.355+0000 7f8de77ff700 1 -- 172.21.15.161:0/1883666698 wait complete. 2023-07-14T11:36:50.573 INFO:tasks.workunit.client.0.smithi161.stdout: 10 58765 36.33 MB/sec warmup 92 sec latency 100.705 ms 2023-07-14T11:36:50.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:50 smithi175 ceph-mon[129081]: pgmap v456: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 453 op/s 2023-07-14T11:36:50.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:50 smithi182 ceph-mon[129366]: pgmap v456: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 453 op/s 2023-07-14T11:36:50.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:50 smithi161 ceph-mon[95526]: pgmap v456: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 453 op/s 2023-07-14T11:36:51.378 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:36:51.378 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:36:51.574 INFO:tasks.workunit.client.0.smithi161.stdout: 10 58933 36.03 MB/sec warmup 93 sec latency 576.089 ms 2023-07-14T11:36:51.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.864+0000 7f033539a700 1 --2- 172.21.15.161:0/1178597360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330107a50 0x7f0330107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:51.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.865+0000 7f0337dff700 1 -- 172.21.15.161:0/1178597360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330112920 msgr2=0x7f0330114d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:51.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.865+0000 7f0337dff700 1 --2- 172.21.15.161:0/1178597360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330112920 0x7f0330114d00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0320009990 tx=0x7f0320009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.869+0000 7f0337dff700 1 -- 172.21.15.161:0/1178597360 shutdown_connections 2023-07-14T11:36:51.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.869+0000 7f0337dff700 1 --2- 172.21.15.161:0/1178597360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f033006d8e0 0x7f0330107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.869+0000 7f0337dff700 1 --2- 172.21.15.161:0/1178597360 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330112920 0x7f0330114d00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.869+0000 7f0337dff700 1 --2- 172.21.15.161:0/1178597360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330107a50 0x7f0330107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.869+0000 7f0337dff700 1 -- 172.21.15.161:0/1178597360 >> 172.21.15.161:0/1178597360 conn(0x7f033006bf70 msgr2=0x7f0330070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:51.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.881+0000 7f0337dff700 1 -- 172.21.15.161:0/1178597360 shutdown_connections 2023-07-14T11:36:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.881+0000 7f0337dff700 1 -- 172.21.15.161:0/1178597360 wait complete. 2023-07-14T11:36:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.882+0000 7f0337dff700 1 Processor -- start 2023-07-14T11:36:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.883+0000 7f0337dff700 1 -- start start 2023-07-14T11:36:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.883+0000 7f0337dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f033006d8e0 0x7f033011c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.883+0000 7f0337dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330107a50 0x7f033011c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.883+0000 7f0337dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330112920 0x7f0330117250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.883+0000 7f0337dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330117a20 con 0x7f033006d8e0 2023-07-14T11:36:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.883+0000 7f0337dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330117b90 con 0x7f0330112920 2023-07-14T11:36:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.883+0000 7f0337dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330117d50 con 0x7f0330107a50 2023-07-14T11:36:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.884+0000 7f033539a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330107a50 0x7f033011c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.884+0000 7f033539a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330107a50 0x7f033011c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46040/0 (socket says 172.21.15.161:46040) 2023-07-14T11:36:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.884+0000 7f033539a700 1 -- 172.21.15.161:0/669740998 learned_addr learned my addr 172.21.15.161:0/669740998 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.885+0000 7f033539a700 1 -- 172.21.15.161:0/669740998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330112920 msgr2=0x7f0330117250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.885+0000 7f033539a700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330112920 0x7f0330117250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.885+0000 7f033539a700 1 -- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f033006d8e0 msgr2=0x7f033011c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.885+0000 7f033539a700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f033006d8e0 0x7f033011c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.885+0000 7f033539a700 1 -- 172.21.15.161:0/669740998 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200095f0 con 0x7f0330107a50 2023-07-14T11:36:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.885+0000 7f033539a700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330107a50 0x7f033011c6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0328007e20 tx=0x7f032800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:51.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.890+0000 7f0326ffd700 1 -- 172.21.15.161:0/669740998 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032800ae00 con 0x7f0330107a50 2023-07-14T11:36:51.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.890+0000 7f0326ffd700 1 -- 172.21.15.161:0/669740998 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03280128e0 con 0x7f0330107a50 2023-07-14T11:36:51.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.890+0000 7f0326ffd700 1 -- 172.21.15.161:0/669740998 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328011840 con 0x7f0330107a50 2023-07-14T11:36:51.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.890+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0330118030 con 0x7f0330107a50 2023-07-14T11:36:51.894 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.890+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03301241f0 con 0x7f0330107a50 2023-07-14T11:36:51.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.897+0000 7f034b684b80 1 Processor -- start 2023-07-14T11:36:51.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.898+0000 7f034b684b80 1 -- start start 2023-07-14T11:36:51.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.898+0000 7f034b684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.898+0000 7f034b684b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbc4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.898+0000 7f034b684b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc4cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.898+0000 7f034b684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbb45d30 con 0x559bfbc3f350 2023-07-14T11:36:51.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.898+0000 7f034b684b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbb98ac0 con 0x559bfbc4ab50 2023-07-14T11:36:51.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.898+0000 7f034b684b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbb91f50 con 0x559bfbc3fc60 2023-07-14T11:36:51.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.901+0000 7f033639c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.901+0000 7f033639c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc3f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37154/0 (socket says 172.21.15.161:37154) 2023-07-14T11:36:51.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.901+0000 7f033639c700 1 -- 172.21.15.161:0/966691857 learned_addr learned my addr 172.21.15.161:0/966691857 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:51.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.903+0000 7f0335b9b700 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbc4a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.903+0000 7f033639c700 1 -- 172.21.15.161:0/966691857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 msgr2=0x559bfbc4a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:51.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.903+0000 7f033639c700 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbc4a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.903+0000 7f033639c700 1 -- 172.21.15.161:0/966691857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 msgr2=0x559bfbc4cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:36:51.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.903+0000 7f033639c700 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc4cf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.903+0000 7f033639c700 1 -- 172.21.15.161:0/966691857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bfb9bfe60 con 0x559bfbc3f350 2023-07-14T11:36:51.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.904+0000 7f0335b9b700 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbc4a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:36:51.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.904+0000 7f033639c700 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc3f720 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f032000b5c0 tx=0x7f0320003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8343a5418f209b9f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:51.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.905+0000 7f0326ffd700 1 -- 172.21.15.161:0/669740998 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0328012420 con 0x7f0330107a50 2023-07-14T11:36:51.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.907+0000 7f0326ffd700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f031c0a14d0 0x7f031c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.907+0000 7f0326ffd700 1 -- 172.21.15.161:0/669740998 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f03280ccc80 con 0x7f0330107a50 2023-07-14T11:36:51.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.909+0000 7f031bfff700 1 -- 172.21.15.161:0/966691857 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032001d070 con 0x559bfbc3f350 2023-07-14T11:36:51.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.910+0000 7f031bfff700 1 -- 172.21.15.161:0/966691857 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f032000be00 con 0x559bfbc3f350 2023-07-14T11:36:51.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.910+0000 7f031bfff700 1 -- 172.21.15.161:0/966691857 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200054f0 con 0x559bfbc3f350 2023-07-14T11:36:51.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.913+0000 7f0335b9b700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f031c0a14d0 0x7f031c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.915+0000 7f034b684b80 1 -- 172.21.15.161:0/966691857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 msgr2=0x559bfbc3f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:51.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.915+0000 7f034b684b80 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc3f720 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f032000b5c0 tx=0x7f0320003680 comp rx=0 tx=0).stop 2023-07-14T11:36:51.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.915+0000 7f0335b9b700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f031c0a14d0 0x7f031c0a3980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f032c00aca0 tx=0x7f032c00a680 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:51.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.916+0000 7f034b684b80 1 -- 172.21.15.161:0/966691857 shutdown_connections 2023-07-14T11:36:51.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.916+0000 7f034b684b80 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc3f720 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.916+0000 7f034b684b80 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc4cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.916+0000 7f034b684b80 1 --2- 172.21.15.161:0/966691857 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbc4a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.916+0000 7f034b684b80 1 -- 172.21.15.161:0/966691857 >> 172.21.15.161:0/966691857 conn(0x559bfbc35e20 msgr2=0x559bfbc3b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:51.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.917+0000 7f034b684b80 1 -- 172.21.15.161:0/966691857 shutdown_connections 2023-07-14T11:36:51.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.919+0000 7f034b684b80 1 -- 172.21.15.161:0/966691857 wait complete. 2023-07-14T11:36:51.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.919+0000 7f034b684b80 1 -- start start 2023-07-14T11:36:51.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.920+0000 7f034b684b80 1 Processor -- start 2023-07-14T11:36:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f034b684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc47930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f034b684b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 0x559bfbc47e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f034b684b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 0x559bfbc483b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f034b684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbb31b40 con 0x559bfbc3f350 2023-07-14T11:36:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f034b684b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbb31840 con 0x559bfbc3fc60 2023-07-14T11:36:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f034b684b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfba159a0 con 0x559bfbc4ab50 2023-07-14T11:36:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f034b684b80 10 client.0 _ll_register_callbacks cb 0x559bfbb2f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:51.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f0335b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc47930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f0335b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc47930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37160/0 (socket says 172.21.15.161:37160) 2023-07-14T11:36:51.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.921+0000 7f0335b9b700 1 -- 172.21.15.161:0/3303190745 learned_addr learned my addr 172.21.15.161:0/3303190745 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:51.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.922+0000 7f0335b9b700 1 -- 172.21.15.161:0/3303190745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 msgr2=0x559bfbc483b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:36:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.922+0000 7f0335b9b700 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 0x559bfbc483b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.922+0000 7f0335b9b700 1 -- 172.21.15.161:0/3303190745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 msgr2=0x559bfbc47e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:36:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.922+0000 7f0335b9b700 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 0x559bfbc47e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.922+0000 7f0335b9b700 1 -- 172.21.15.161:0/3303190745 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0330108b10 con 0x559bfbc3f350 2023-07-14T11:36:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.923+0000 7f0335b9b700 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc47930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f032c0109c0 tx=0x7f032c005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1298b81586588b71 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.923+0000 7f03197fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:36:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.923+0000 7f03197fa700 1 -- 172.21.15.161:0/3303190745 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00f9d0 con 0x559bfbc3f350 2023-07-14T11:36:51.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.923+0000 7f03197fa700 1 -- 172.21.15.161:0/3303190745 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f032c00fb30 con 0x559bfbc3f350 2023-07-14T11:36:51.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.924+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bfbaeb570 con 0x559bfbc3f350 2023-07-14T11:36:51.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.924+0000 7f03197fa700 1 -- 172.21.15.161:0/3303190745 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f032c00fcf0 con 0x559bfbc3f350 2023-07-14T11:36:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.926+0000 7f034b684b80 10 client.17160 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.926+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x559bfb9a8ce0 con 0x559bfbc3f350 2023-07-14T11:36:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.927+0000 7f03197fa700 1 -- 172.21.15.161:0/3303190745 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559bfbaeb570 con 0x559bfbc3f350 2023-07-14T11:36:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.927+0000 7f03197fa700 1 -- 172.21.15.161:0/3303190745 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f032c025590 con 0x559bfbc3f350 2023-07-14T11:36:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.928+0000 7f034b684b80 10 client.17160 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.928+0000 7f034b684b80 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:51.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.928+0000 7f034b684b80 20 client.17160 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:51.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.928+0000 7f034b684b80 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbc5a560 0x559bfbc5c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.928+0000 7f034b684b80 4 client.17160 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:51.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.928+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bfbb8af30 con 0x559bfbc5a560 2023-07-14T11:36:51.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.930+0000 7f033539a700 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbc5a560 0x559bfbc5c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.933+0000 7f033539a700 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbc5a560 0x559bfbc5c970 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9781c08e310cfba5 server_cookie=d8b2dbe92a3f7cc4 in_seq=0 out_seq=0 2023-07-14T11:36:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.933+0000 7f03197fa700 10 client.17160 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:51.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.938+0000 7f03197fa700 1 -- 172.21.15.161:0/3303190745 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559bfbb8af30 con 0x559bfbc5a560 2023-07-14T11:36:51.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.938+0000 7f03197fa700 10 client.17160 handle_command_reply: tid=0 2023-07-14T11:36:51.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.938+0000 7f034b684b80 1 client.17160 shutdown 2023-07-14T11:36:51.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 msgr2=0x559bfbc47930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:51.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc47930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f032c0109c0 tx=0x7f032c005ba0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 shutdown_connections 2023-07-14T11:36:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbc5a560 0x559bfbc5c970 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc47930 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 0x559bfbc483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 --2- 172.21.15.161:0/3303190745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 0x559bfbc47e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.941+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 >> 172.21.15.161:0/3303190745 conn(0x559bfbc35e20 msgr2=0x559bfbc3b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.942+0000 7f03197fa700 0 client.17160 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.942+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 shutdown_connections 2023-07-14T11:36:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.942+0000 7f034b684b80 1 -- 172.21.15.161:0/3303190745 wait complete. 2023-07-14T11:36:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.943+0000 7f034b684b80 20 client.17160 trim_cache size 0 max 16384 2023-07-14T11:36:51.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.976+0000 7f034b684b80 1 Processor -- start 2023-07-14T11:36:51.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.976+0000 7f034b684b80 1 -- start start 2023-07-14T11:36:51.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.977+0000 7f034b684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.977+0000 7f034b684b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 0x559bfbc44be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.977+0000 7f034b684b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 0x559bfbc45120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.977+0000 7f034b684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbc48ec0 con 0x559bfbc3f350 2023-07-14T11:36:51.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.977+0000 7f034b684b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbc3cda0 con 0x559bfbc3fc60 2023-07-14T11:36:51.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.977+0000 7f034b684b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbc427b0 con 0x559bfbc4ab50 2023-07-14T11:36:51.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.978+0000 7f0335b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.978+0000 7f0335b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37170/0 (socket says 172.21.15.161:37170) 2023-07-14T11:36:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.978+0000 7f0335b9b700 1 -- 172.21.15.161:0/2415823666 learned_addr learned my addr 172.21.15.161:0/2415823666 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.978+0000 7f033639c700 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 0x559bfbc45120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.979+0000 7f0335b9b700 1 -- 172.21.15.161:0/2415823666 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 msgr2=0x559bfbc45120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:51.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.979+0000 7f0335b9b700 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 0x559bfbc45120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.979+0000 7f0335b9b700 1 -- 172.21.15.161:0/2415823666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 msgr2=0x559bfbc44be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:51.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.979+0000 7f0335b9b700 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 0x559bfbc44be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.979+0000 7f0335b9b700 1 -- 172.21.15.161:0/2415823666 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bfb9e62d0 con 0x559bfbc3f350 2023-07-14T11:36:51.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.980+0000 7f0335b9b700 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc481e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f032c010990 tx=0x7f032c025ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d4e2e715b17bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:51.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.981+0000 7f0324bff700 1 -- 172.21.15.161:0/2415823666 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c0255b0 con 0x559bfbc3f350 2023-07-14T11:36:51.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.981+0000 7f0324bff700 1 -- 172.21.15.161:0/2415823666 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f032c00aed0 con 0x559bfbc3f350 2023-07-14T11:36:51.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.981+0000 7f0324bff700 1 -- 172.21.15.161:0/2415823666 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00f9b0 con 0x559bfbc3f350 2023-07-14T11:36:51.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.982+0000 7f034b684b80 1 -- 172.21.15.161:0/2415823666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 msgr2=0x559bfbc481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:51.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.982+0000 7f034b684b80 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc481e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f032c010990 tx=0x7f032c025ac0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.983+0000 7f034b684b80 1 -- 172.21.15.161:0/2415823666 shutdown_connections 2023-07-14T11:36:51.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.983+0000 7f034b684b80 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbc481e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.983+0000 7f034b684b80 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc4ab50 0x559bfbc45120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.983+0000 7f034b684b80 1 --2- 172.21.15.161:0/2415823666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc3fc60 0x559bfbc44be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.983+0000 7f034b684b80 1 -- 172.21.15.161:0/2415823666 >> 172.21.15.161:0/2415823666 conn(0x559bfbc35e20 msgr2=0x559bfbc41cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:51.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.985+0000 7f034b684b80 1 -- 172.21.15.161:0/2415823666 shutdown_connections 2023-07-14T11:36:51.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.985+0000 7f034b684b80 1 -- 172.21.15.161:0/2415823666 wait complete. 2023-07-14T11:36:51.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.985+0000 7f034b684b80 1 -- start start 2023-07-14T11:36:51.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.986+0000 7f034b684b80 1 Processor -- start 2023-07-14T11:36:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.988+0000 7f034b684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbca7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.988+0000 7f034b684b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbca8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.988+0000 7f034b684b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc50a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.988+0000 7f034b684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbc51110 con 0x559bfbc3f350 2023-07-14T11:36:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.988+0000 7f034b684b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbc51280 con 0x559bfbc4ab50 2023-07-14T11:36:51.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.988+0000 7f034b684b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfbc51440 con 0x559bfbc3fc60 2023-07-14T11:36:51.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.988+0000 7f034b684b80 10 client.0 _ll_register_callbacks cb 0x7f032c00ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:51.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033639c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc50a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033639c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc50a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47134/0 (socket says 172.21.15.161:47134) 2023-07-14T11:36:51.990 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033639c700 1 -- 172.21.15.161:0/2009705026 learned_addr learned my addr 172.21.15.161:0/2009705026 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f0335b9b700 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbca7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033539a700 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbca8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033539a700 1 -- 172.21.15.161:0/2009705026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 msgr2=0x559bfbc50a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033539a700 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc50a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033539a700 1 -- 172.21.15.161:0/2009705026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 msgr2=0x559bfbca7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033539a700 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbca7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.990+0000 7f033539a700 1 -- 172.21.15.161:0/2009705026 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bfb9a8ce0 con 0x559bfbc3fc60 2023-07-14T11:36:51.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.991+0000 7f033539a700 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbca8170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f03280ca120 tx=0x7f0328099aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f67a348dc88ae8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:51.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.991+0000 7f03197fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:51.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.991+0000 7f03197fa700 1 -- 172.21.15.161:0/2009705026 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03280b2260 con 0x559bfbc3fc60 2023-07-14T11:36:51.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.991+0000 7f03197fa700 1 -- 172.21.15.161:0/2009705026 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f03280a1ef0 con 0x559bfbc3fc60 2023-07-14T11:36:51.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.992+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f032000be00 con 0x559bfbc3fc60 2023-07-14T11:36:51.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.992+0000 7f03197fa700 1 -- 172.21.15.161:0/2009705026 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0328099d00 con 0x559bfbc3fc60 2023-07-14T11:36:51.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.992+0000 7f034b684b80 10 client.27250 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:51.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.992+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x559bfb9be330 con 0x559bfbc3fc60 2023-07-14T11:36:51.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.992+0000 7f03197fa700 1 -- 172.21.15.161:0/2009705026 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f032000be00 con 0x559bfbc3fc60 2023-07-14T11:36:51.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.993+0000 7f03197fa700 1 -- 172.21.15.161:0/2009705026 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f03280a1a60 con 0x559bfbc3fc60 2023-07-14T11:36:51.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.993+0000 7f034b684b80 10 client.27250 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:51.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.993+0000 7f034b684b80 10 client.27250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:51.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.993+0000 7f034b684b80 20 client.27250 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:51.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.993+0000 7f034b684b80 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbcb0cd0 0x559bfbcb30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.993+0000 7f034b684b80 4 client.27250 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:36:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.993+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559bfbb4d0d0 con 0x559bfbcb0cd0 2023-07-14T11:36:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:51.996+0000 7f0335b9b700 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbcb0cd0 0x559bfbcb30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.006+0000 7f0335b9b700 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbcb0cd0 0x559bfbcb30e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e8361e54d2ff74 server_cookie=a69b446f9ce98ed1 in_seq=0 out_seq=0 2023-07-14T11:36:52.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.006+0000 7f03197fa700 10 client.27250 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:52.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.055+0000 7f03197fa700 1 -- 172.21.15.161:0/2009705026 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559bfbb4d0d0 con 0x559bfbcb0cd0 2023-07-14T11:36:52.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.055+0000 7f03197fa700 10 client.27250 handle_command_reply: tid=0 2023-07-14T11:36:52.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.056+0000 7f034b684b80 1 client.27250 shutdown 2023-07-14T11:36:52.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.056+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 msgr2=0x559bfbca8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.056+0000 7f034b684b80 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbca8170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f03280ca120 tx=0x7f0328099aa0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.057+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 shutdown_connections 2023-07-14T11:36:52.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.057+0000 7f034b684b80 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559bfbcb0cd0 0x559bfbcb30e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.057+0000 7f034b684b80 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bfbc3f350 0x559bfbca7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.057+0000 7f034b684b80 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bfbc4ab50 0x559bfbc50a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.057+0000 7f034b684b80 1 --2- 172.21.15.161:0/2009705026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559bfbc3fc60 0x559bfbca8170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.057+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 >> 172.21.15.161:0/2009705026 conn(0x559bfbc35e20 msgr2=0x559bfbc47ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.058+0000 7f03197fa700 0 client.27250 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:52.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.058+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 shutdown_connections 2023-07-14T11:36:52.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.059+0000 7f034b684b80 1 -- 172.21.15.161:0/2009705026 wait complete. 2023-07-14T11:36:52.061 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:36:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:36:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "27377ff5-0f55-4cb2-84f9-8543692119d6", 2023-07-14T11:36:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:36:52.061 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:36:52.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.060+0000 7f034b684b80 20 client.27250 trim_cache size 0 max 16384 2023-07-14T11:36:52.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f031c0a14d0 msgr2=0x7f031c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f031c0a14d0 0x7f031c0a3980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f032c00aca0 tx=0x7f032c00a680 comp rx=0 tx=0).stop 2023-07-14T11:36:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330107a50 msgr2=0x7f033011c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330107a50 0x7f033011c6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0328007e20 tx=0x7f032800a340 comp rx=0 tx=0).stop 2023-07-14T11:36:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 shutdown_connections 2023-07-14T11:36:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f031c0a14d0 0x7f031c0a3980 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f033006d8e0 0x7f033011c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330112920 0x7f0330117250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 --2- 172.21.15.161:0/669740998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0330107a50 0x7f033011c6f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.062+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 >> 172.21.15.161:0/669740998 conn(0x7f033006bf70 msgr2=0x7f033006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.063+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 shutdown_connections 2023-07-14T11:36:52.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.064+0000 7f0337dff700 1 -- 172.21.15.161:0/669740998 wait complete. 2023-07-14T11:36:52.102 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:36:52.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:52 smithi161 ceph-mon[95526]: pgmap v457: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 542 op/s 2023-07-14T11:36:52.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.469+0000 7fd9827ff700 1 -- 172.21.15.161:0/3911555862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c107a50 msgr2=0x7fd97c107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.469+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3911555862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c107a50 0x7fd97c107ec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009990 tx=0x7fd96c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.470+0000 7fd9827ff700 1 -- 172.21.15.161:0/3911555862 shutdown_connections 2023-07-14T11:36:52.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.470+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3911555862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97c112920 0x7fd97c114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.470+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3911555862 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd97c06d8e0 0x7fd97c107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.470+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3911555862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c107a50 0x7fd97c107ec0 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009990 tx=0x7fd96c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.470+0000 7fd9827ff700 1 -- 172.21.15.161:0/3911555862 >> 172.21.15.161:0/3911555862 conn(0x7fd97c06bf70 msgr2=0x7fd97c070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.483+0000 7fd9827ff700 1 -- 172.21.15.161:0/3911555862 shutdown_connections 2023-07-14T11:36:52.484 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.484+0000 7fd9827ff700 1 -- 172.21.15.161:0/3911555862 wait complete. 2023-07-14T11:36:52.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.485+0000 7fd9827ff700 1 Processor -- start 2023-07-14T11:36:52.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.485+0000 7fd9827ff700 1 -- start start 2023-07-14T11:36:52.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.486+0000 7fd9827ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97c06d8e0 0x7fd97c11daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.486+0000 7fd9827ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c112920 0x7fd97c116a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.486+0000 7fd9827ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd97c116fd0 0x7fd97c117440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.486+0000 7fd9827ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c11b610 con 0x7fd97c112920 2023-07-14T11:36:52.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.486+0000 7fd9827ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c11b780 con 0x7fd97c06d8e0 2023-07-14T11:36:52.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.486+0000 7fd9827ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c11b940 con 0x7fd97c116fd0 2023-07-14T11:36:52.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.487+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c112920 0x7fd97c116a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c112920 0x7fd97c116a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37204/0 (socket says 172.21.15.161:37204) 2023-07-14T11:36:52.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3413656469 learned_addr learned my addr 172.21.15.161:0/3413656469 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:52.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd980d9c700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd97c116fd0 0x7fd97c117440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3413656469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd97c116fd0 msgr2=0x7fd97c117440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd97c116fd0 0x7fd97c117440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3413656469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97c06d8e0 msgr2=0x7fd97c11daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97c06d8e0 0x7fd97c11daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3413656469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0095f0 con 0x7fd97c112920 2023-07-14T11:36:52.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.488+0000 7fd980d9c700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd97c116fd0 0x7fd97c117440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:36:52.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.489+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c112920 0x7fd97c116a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0098e0 tx=0x7fd96c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:52.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.495+0000 7fd9797fa700 1 -- 172.21.15.161:0/3413656469 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c010670 con 0x7fd97c112920 2023-07-14T11:36:52.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.495+0000 7fd9797fa700 1 -- 172.21.15.161:0/3413656469 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96c005610 con 0x7fd97c112920 2023-07-14T11:36:52.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.495+0000 7fd9797fa700 1 -- 172.21.15.161:0/3413656469 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c018430 con 0x7fd97c112920 2023-07-14T11:36:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.497+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c11bbc0 con 0x7fd97c112920 2023-07-14T11:36:52.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.497+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c11c050 con 0x7fd97c112920 2023-07-14T11:36:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.504+0000 7fd99619ab80 1 Processor -- start 2023-07-14T11:36:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.505+0000 7fd99619ab80 1 -- start start 2023-07-14T11:36:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22e2d0 0x55d79c22e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c239590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c239ad0 0x55d79c23beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd99619ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c134cb0 con 0x55d79c22e2d0 2023-07-14T11:36:52.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd99619ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c187a40 con 0x55d79c239ad0 2023-07-14T11:36:52.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd99619ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c180ed0 con 0x55d79c22ebe0 2023-07-14T11:36:52.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd9797fa700 1 -- 172.21.15.161:0/3413656469 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd96c005150 con 0x7fd97c112920 2023-07-14T11:36:52.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd980d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22e2d0 0x55d79c22e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd980d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22e2d0 0x55d79c22e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37208/0 (socket says 172.21.15.161:37208) 2023-07-14T11:36:52.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.506+0000 7fd980d9c700 1 -- 172.21.15.161:0/1344883072 learned_addr learned my addr 172.21.15.161:0/1344883072 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:52.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.507+0000 7fd980d9c700 1 -- 172.21.15.161:0/1344883072 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 msgr2=0x55d79c239590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:36:52.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.507+0000 7fd980d9c700 1 --2- 172.21.15.161:0/1344883072 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c239590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.507+0000 7fd980d9c700 1 -- 172.21.15.161:0/1344883072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c239ad0 msgr2=0x55d79c23beb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:36:52.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.507+0000 7fd980d9c700 1 --2- 172.21.15.161:0/1344883072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c239ad0 0x55d79c23beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.507+0000 7fd980d9c700 1 -- 172.21.15.161:0/1344883072 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d79bfaede0 con 0x55d79c22e2d0 2023-07-14T11:36:52.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.508+0000 7fd980d9c700 1 --2- 172.21.15.161:0/1344883072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22e2d0 0x55d79c22e6a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd974007f00 tx=0x7fd97400c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97da7b193228092e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:52.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.510+0000 7fd9627fc700 1 -- 172.21.15.161:0/1344883072 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400f640 con 0x55d79c22e2d0 2023-07-14T11:36:52.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.510+0000 7fd9627fc700 1 -- 172.21.15.161:0/1344883072 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd97400ce60 con 0x55d79c22e2d0 2023-07-14T11:36:52.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.510+0000 7fd9627fc700 1 -- 172.21.15.161:0/1344883072 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400eb10 con 0x55d79c22e2d0 2023-07-14T11:36:52.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.512+0000 7fd99619ab80 1 -- 172.21.15.161:0/1344883072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22e2d0 msgr2=0x55d79c22e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.512+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1344883072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22e2d0 0x55d79c22e6a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd974007f00 tx=0x7fd97400c5a0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.514+0000 7fd99619ab80 1 -- 172.21.15.161:0/1344883072 shutdown_connections 2023-07-14T11:36:52.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.514+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1344883072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22e2d0 0x55d79c22e6a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.514+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1344883072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c239ad0 0x55d79c23beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.514+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1344883072 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c239590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.514+0000 7fd99619ab80 1 -- 172.21.15.161:0/1344883072 >> 172.21.15.161:0/1344883072 conn(0x55d79c224da0 msgr2=0x55d79c22a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.514+0000 7fd9797fa700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd9640a14d0 0x7fd9640a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.515+0000 7fd9797fa700 1 -- 172.21.15.161:0/3413656469 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd96c0ce6a0 con 0x7fd97c112920 2023-07-14T11:36:52.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.515+0000 7fd99619ab80 1 -- 172.21.15.161:0/1344883072 shutdown_connections 2023-07-14T11:36:52.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.515+0000 7fd99619ab80 1 -- 172.21.15.161:0/1344883072 wait complete. 2023-07-14T11:36:52.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.516+0000 7fd99619ab80 1 -- start start 2023-07-14T11:36:52.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.516+0000 7fd99619ab80 1 Processor -- start 2023-07-14T11:36:52.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.517+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c236890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.517+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22ebe0 0x55d79c236dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.517+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c239ad0 0x55d79c237310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.517+0000 7fd99619ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c120ac0 con 0x55d79c22ebe0 2023-07-14T11:36:52.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.517+0000 7fd99619ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c1207c0 con 0x55d79c22e2d0 2023-07-14T11:36:52.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.517+0000 7fd99619ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c004920 con 0x55d79c239ad0 2023-07-14T11:36:52.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.517+0000 7fd99619ab80 10 client.0 _ll_register_callbacks cb 0x55d79c11e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:52.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.518+0000 7fd97bfff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd9640a14d0 0x7fd9640a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.519+0000 7fd97bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c236890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.519+0000 7fd97bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c236890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47166/0 (socket says 172.21.15.161:47166) 2023-07-14T11:36:52.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.519+0000 7fd97bfff700 1 -- 172.21.15.161:0/1334763149 learned_addr learned my addr 172.21.15.161:0/1334763149 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:52.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.519+0000 7fd97bfff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd9640a14d0 0x7fd9640a3980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd97000cee0 tx=0x7fd970010710 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:52.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.520+0000 7fd980d9c700 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c239ad0 0x55d79c237310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.521+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22ebe0 0x55d79c236dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.522+0000 7fd97b7fe700 1 -- 172.21.15.161:0/1334763149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c239ad0 msgr2=0x55d79c237310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:52.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.522+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c239ad0 0x55d79c237310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.522+0000 7fd97b7fe700 1 -- 172.21.15.161:0/1334763149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 msgr2=0x55d79c236890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:52.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.522+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c236890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.522+0000 7fd97b7fe700 1 -- 172.21.15.161:0/1334763149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd974007b30 con 0x55d79c22ebe0 2023-07-14T11:36:52.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.533+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22ebe0 0x55d79c236dd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0cbb40 tx=0x7fd96c09b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80682d4a6d4d8cf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:52.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.533+0000 7fd95b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:36:52.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.534+0000 7fd95b7fe700 1 -- 172.21.15.161:0/1334763149 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0a3a30 con 0x55d79c22ebe0 2023-07-14T11:36:52.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.534+0000 7fd95b7fe700 1 -- 172.21.15.161:0/1334763149 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd96c0ce2a0 con 0x55d79c22ebe0 2023-07-14T11:36:52.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.535+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d79c0da4f0 con 0x55d79c22ebe0 2023-07-14T11:36:52.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.535+0000 7fd95b7fe700 1 -- 172.21.15.161:0/1334763149 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd96c0ce460 con 0x55d79c22ebe0 2023-07-14T11:36:52.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.537+0000 7fd99619ab80 10 client.17178 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:52.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.537+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d79bf97c60 con 0x55d79c22ebe0 2023-07-14T11:36:52.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd95b7fe700 1 -- 172.21.15.161:0/1334763149 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d79c0da4f0 con 0x55d79c22ebe0 2023-07-14T11:36:52.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd95b7fe700 1 -- 172.21.15.161:0/1334763149 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd96c0a35a0 con 0x55d79c22ebe0 2023-07-14T11:36:52.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd99619ab80 10 client.17178 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:52.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd99619ab80 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:52.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd99619ab80 20 client.17178 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:52.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c249200 0x55d79c24b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd99619ab80 4 client.17178 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:36:52.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.538+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d79c179eb0 con 0x55d79c249200 2023-07-14T11:36:52.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.545+0000 7fd97bfff700 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c249200 0x55d79c24b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.550+0000 7fd97bfff700 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c249200 0x55d79c24b610 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcffe27992e5e28 server_cookie=80ea635d349b5e1 in_seq=0 out_seq=0 2023-07-14T11:36:52.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.550+0000 7fd95b7fe700 10 client.17178 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:52.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.561+0000 7fd95b7fe700 1 -- 172.21.15.161:0/1334763149 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d79c179eb0 con 0x55d79c249200 2023-07-14T11:36:52.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.561+0000 7fd95b7fe700 10 client.17178 handle_command_reply: tid=0 2023-07-14T11:36:52.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.561+0000 7fd99619ab80 1 client.17178 shutdown 2023-07-14T11:36:52.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.574+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22ebe0 msgr2=0x55d79c236dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.574+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22ebe0 0x55d79c236dd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0cbb40 tx=0x7fd96c09b0a0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 59191 35.78 MB/sec warmup 94 sec latency 601.173 ms 2023-07-14T11:36:52.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.582+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 shutdown_connections 2023-07-14T11:36:52.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.582+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c249200 0x55d79c24b610 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.582+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c239ad0 0x55d79c237310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.582+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c236890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.583 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.582+0000 7fd99619ab80 1 --2- 172.21.15.161:0/1334763149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c22ebe0 0x55d79c236dd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.584 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.582+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 >> 172.21.15.161:0/1334763149 conn(0x55d79c224da0 msgr2=0x55d79c22a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.585+0000 7fd95b7fe700 0 client.17178 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.594+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 shutdown_connections 2023-07-14T11:36:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.595+0000 7fd99619ab80 1 -- 172.21.15.161:0/1334763149 wait complete. 2023-07-14T11:36:52.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.603+0000 7fd99619ab80 20 client.17178 trim_cache size 0 max 16384 2023-07-14T11:36:52.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.635+0000 7fd99619ab80 1 Processor -- start 2023-07-14T11:36:52.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.635+0000 7fd99619ab80 1 -- start start 2023-07-14T11:36:52.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.636+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c240540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.636+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c233b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.636+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 0x55d79c234040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.636+0000 7fd99619ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c22fd60 con 0x55d79c239ad0 2023-07-14T11:36:52.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.636+0000 7fd99619ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c179eb0 con 0x55d79c22e2d0 2023-07-14T11:36:52.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.636+0000 7fd99619ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c0da4f0 con 0x55d79c22ebe0 2023-07-14T11:36:52.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c233b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c233b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46094/0 (socket says 172.21.15.161:46094) 2023-07-14T11:36:52.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3743045746 learned_addr learned my addr 172.21.15.161:0/3743045746 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:52.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3743045746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 msgr2=0x55d79c240540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:36:52.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/3743045746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c240540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3743045746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 msgr2=0x55d79c234040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:52.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/3743045746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 0x55d79c234040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 -- 172.21.15.161:0/3743045746 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d79c2265e0 con 0x55d79c22ebe0 2023-07-14T11:36:52.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.638+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/3743045746 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c233b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd96c097b70 tx=0x7fd96c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0b087cea8bcdf2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:52.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.639+0000 7fd962ffd700 1 -- 172.21.15.161:0/3743045746 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c097820 con 0x55d79c22ebe0 2023-07-14T11:36:52.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.640+0000 7fd962ffd700 1 -- 172.21.15.161:0/3743045746 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96c09adf0 con 0x55d79c22ebe0 2023-07-14T11:36:52.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.640+0000 7fd962ffd700 1 -- 172.21.15.161:0/3743045746 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0974b0 con 0x55d79c22ebe0 2023-07-14T11:36:52.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.641+0000 7fd99619ab80 1 -- 172.21.15.161:0/3743045746 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 msgr2=0x55d79c233b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.641+0000 7fd99619ab80 1 --2- 172.21.15.161:0/3743045746 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c233b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd96c097b70 tx=0x7fd96c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.645+0000 7fd99619ab80 1 -- 172.21.15.161:0/3743045746 shutdown_connections 2023-07-14T11:36:52.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.645+0000 7fd99619ab80 1 --2- 172.21.15.161:0/3743045746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 0x55d79c234040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.645+0000 7fd99619ab80 1 --2- 172.21.15.161:0/3743045746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c240540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.645+0000 7fd99619ab80 1 --2- 172.21.15.161:0/3743045746 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c233b00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.645+0000 7fd99619ab80 1 -- 172.21.15.161:0/3743045746 >> 172.21.15.161:0/3743045746 conn(0x55d79c224da0 msgr2=0x55d79c231b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.651+0000 7fd99619ab80 1 -- 172.21.15.161:0/3743045746 shutdown_connections 2023-07-14T11:36:52.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.656+0000 7fd99619ab80 1 -- 172.21.15.161:0/3743045746 wait complete. 2023-07-14T11:36:52.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.657+0000 7fd99619ab80 1 -- start start 2023-07-14T11:36:52.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.657+0000 7fd99619ab80 1 Processor -- start 2023-07-14T11:36:52.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.658+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c240390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.658+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c23e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.658+0000 7fd99619ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 0x55d79c23ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.658+0000 7fd99619ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c23f140 con 0x55d79c239ad0 2023-07-14T11:36:52.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.658+0000 7fd99619ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c23f280 con 0x55d79c22e2d0 2023-07-14T11:36:52.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.658+0000 7fd99619ab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79c23f440 con 0x55d79c22ebe0 2023-07-14T11:36:52.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.658+0000 7fd99619ab80 10 client.0 _ll_register_callbacks cb 0x55d79c11e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:36:52.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.659+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c23e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.659+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c23e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46096/0 (socket says 172.21.15.161:46096) 2023-07-14T11:36:52.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.659+0000 7fd97b7fe700 1 -- 172.21.15.161:0/2715541619 learned_addr learned my addr 172.21.15.161:0/2715541619 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:36:52.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.660+0000 7fd97bfff700 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c240390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.660+0000 7fd97b7fe700 1 -- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 msgr2=0x55d79c240390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:52.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.660+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c240390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.660+0000 7fd97b7fe700 1 -- 172.21.15.161:0/2715541619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 msgr2=0x55d79c23ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:36:52.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.660+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 0x55d79c23ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.660+0000 7fd97b7fe700 1 -- 172.21.15.161:0/2715541619 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d79bf97c60 con 0x55d79c22ebe0 2023-07-14T11:36:52.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.667+0000 7fd97b7fe700 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c23e6c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0999c0 tx=0x7fd96c011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dcebe6509327113 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:36:52.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.668+0000 7fd95b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:36:52.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.669+0000 7fd95b7fe700 1 -- 172.21.15.161:0/2715541619 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c09f900 con 0x55d79c22ebe0 2023-07-14T11:36:52.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.669+0000 7fd95b7fe700 1 -- 172.21.15.161:0/2715541619 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd96c09fa60 con 0x55d79c22ebe0 2023-07-14T11:36:52.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.670+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd97400ce60 con 0x55d79c22ebe0 2023-07-14T11:36:52.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.671+0000 7fd95b7fe700 1 -- 172.21.15.161:0/2715541619 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd96c0ce2a0 con 0x55d79c22ebe0 2023-07-14T11:36:52.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.674+0000 7fd99619ab80 10 client.27259 fetch_fsmap learned FSMap version 37 2023-07-14T11:36:52.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.674+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d79bfd5250 con 0x55d79c22ebe0 2023-07-14T11:36:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd95b7fe700 1 -- 172.21.15.161:0/2715541619 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd97400ce60 con 0x55d79c22ebe0 2023-07-14T11:36:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd95b7fe700 1 -- 172.21.15.161:0/2715541619 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd96c0aa050 con 0x55d79c22ebe0 2023-07-14T11:36:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd99619ab80 10 client.27259 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T11:36:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd99619ab80 10 client.27259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:36:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd99619ab80 20 client.27259 populate_metadata read hostname 'smithi161' 2023-07-14T11:36:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd99619ab80 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c29e0a0 0x55d79c2a0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:36:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd99619ab80 4 client.27259 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:36:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.676+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d79c13c050 con 0x55d79c29e0a0 2023-07-14T11:36:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.677+0000 7fd97bfff700 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c29e0a0 0x55d79c2a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:36:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.679+0000 7fd97bfff700 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c29e0a0 0x55d79c2a0480 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22003753f08924f server_cookie=3ce1eadfb47f4680 in_seq=0 out_seq=0 2023-07-14T11:36:52.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.679+0000 7fd95b7fe700 10 client.27259 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:52.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.685+0000 7fd95b7fe700 1 -- 172.21.15.161:0/2715541619 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55d79c13c050 con 0x55d79c29e0a0 2023-07-14T11:36:52.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.685+0000 7fd95b7fe700 10 client.27259 handle_command_reply: tid=0 2023-07-14T11:36:52.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.685+0000 7fd99619ab80 1 client.27259 shutdown 2023-07-14T11:36:52.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.698+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 msgr2=0x55d79c23e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.698+0000 7fd99619ab80 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c23e6c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0999c0 tx=0x7fd96c011f90 comp rx=0 tx=0).stop 2023-07-14T11:36:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 shutdown_connections 2023-07-14T11:36:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d79c29e0a0 0x55d79c2a0480 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d79c239ad0 0x55d79c23ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d79c22e2d0 0x55d79c240390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 --2- 172.21.15.161:0/2715541619 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d79c22ebe0 0x55d79c23e6c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 >> 172.21.15.161:0/2715541619 conn(0x55d79c224da0 msgr2=0x55d79c236830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd95b7fe700 0 client.27259 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:36:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 shutdown_connections 2023-07-14T11:36:52.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.699+0000 7fd99619ab80 1 -- 172.21.15.161:0/2715541619 wait complete. 2023-07-14T11:36:52.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.708+0000 7fd99619ab80 20 client.27259 trim_cache size 0 max 16384 2023-07-14T11:36:52.709 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:36:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (198 inodes in the stack)", 2023-07-14T11:36:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:36:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "27377ff5-0f55-4cb2-84f9-8543692119d6": { 2023-07-14T11:36:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:36:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "27377ff5-0f55-4cb2-84f9-8543692119d6", 2023-07-14T11:36:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:36:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:36:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:36:52.711 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:36:52.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:52 smithi182 ceph-mon[129366]: pgmap v457: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 542 op/s 2023-07-14T11:36:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.712+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd9640a14d0 msgr2=0x7fd9640a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.712+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd9640a14d0 0x7fd9640a3980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd97000cee0 tx=0x7fd970010710 comp rx=0 tx=0).stop 2023-07-14T11:36:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.712+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c112920 msgr2=0x7fd97c116a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:36:52.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.712+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c112920 0x7fd97c116a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0098e0 tx=0x7fd96c011df0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.714+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 shutdown_connections 2023-07-14T11:36:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.714+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd9640a14d0 0x7fd9640a3980 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.714+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd97c116fd0 0x7fd97c117440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.714+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd97c06d8e0 0x7fd97c11daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.714+0000 7fd9827ff700 1 --2- 172.21.15.161:0/3413656469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd97c112920 0x7fd97c116a90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:36:52.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.714+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 >> 172.21.15.161:0/3413656469 conn(0x7fd97c06bf70 msgr2=0x7fd97c10a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:36:52.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.717+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 shutdown_connections 2023-07-14T11:36:52.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:36:52.718+0000 7fd9827ff700 1 -- 172.21.15.161:0/3413656469 wait complete. 2023-07-14T11:36:52.752 INFO:tasks.cephfs.filesystem:scrub status for tag:27377ff5-0f55-4cb2-84f9-8543692119d6 - {'path': '/', 'tag': '27377ff5-0f55-4cb2-84f9-8543692119d6', 'options': 'recursive,force'} 2023-07-14T11:36:52.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:52 smithi175 ceph-mon[129081]: pgmap v457: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 542 op/s 2023-07-14T11:36:53.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 59466 35.54 MB/sec warmup 95 sec latency 163.072 ms 2023-07-14T11:36:53.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:36:53.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:53.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:53.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:53.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:36:53.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:53 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:36:53.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:53.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:53.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:53.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.746 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:36:53.747 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:53 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:36:53.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:53.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:53.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:53.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:53.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:36:53.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:53 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:54.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 60071 35.55 MB/sec warmup 96 sec latency 92.096 ms 2023-07-14T11:36:54.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:54 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:54.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:54 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:54.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:54 smithi182 ceph-mon[129366]: pgmap v458: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 15 MiB/s wr, 582 op/s 2023-07-14T11:36:54.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:54 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:54.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:54 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:54.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:54 smithi161 ceph-mon[95526]: pgmap v458: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 15 MiB/s wr, 582 op/s 2023-07-14T11:36:54.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:54 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:54.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:54 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:36:54.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:54 smithi175 ceph-mon[129081]: pgmap v458: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 15 MiB/s wr, 582 op/s 2023-07-14T11:36:55.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 60650 35.44 MB/sec warmup 97 sec latency 147.929 ms 2023-07-14T11:36:56.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 61279 35.47 MB/sec warmup 98 sec latency 93.111 ms 2023-07-14T11:36:56.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:56 smithi182 ceph-mon[129366]: pgmap v459: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 15 MiB/s wr, 582 op/s 2023-07-14T11:36:56.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:56 smithi161 ceph-mon[95526]: pgmap v459: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 15 MiB/s wr, 582 op/s 2023-07-14T11:36:56.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:56 smithi175 ceph-mon[129081]: pgmap v459: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 15 MiB/s wr, 582 op/s 2023-07-14T11:36:57.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 62279 35.55 MB/sec warmup 99 sec latency 63.682 ms 2023-07-14T11:36:58.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 63040 35.71 MB/sec warmup 100 sec latency 69.689 ms 2023-07-14T11:36:58.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:36:58 smithi161 ceph-mon[95526]: pgmap v460: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 884 op/s 2023-07-14T11:36:58.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:36:58 smithi175 ceph-mon[129081]: pgmap v460: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 884 op/s 2023-07-14T11:36:58.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:36:58 smithi182 ceph-mon[129366]: pgmap v460: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 884 op/s 2023-07-14T11:36:59.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 63648 35.73 MB/sec warmup 101 sec latency 127.079 ms 2023-07-14T11:37:00.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 64120 35.57 MB/sec warmup 102 sec latency 163.930 ms 2023-07-14T11:37:00.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:00 smithi161 ceph-mon[95526]: pgmap v461: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 627 op/s 2023-07-14T11:37:00.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:00 smithi175 ceph-mon[129081]: pgmap v461: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 627 op/s 2023-07-14T11:37:00.962 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:00 smithi182 ceph-mon[129366]: pgmap v461: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 627 op/s 2023-07-14T11:37:01.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 64340 35.29 MB/sec warmup 103 sec latency 630.580 ms 2023-07-14T11:37:01.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:01 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:37:01.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:01 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:37:01.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:01 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:37:01.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:01 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:37:01.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:01 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:37:01.962 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:01 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:37:02.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 64399 34.96 MB/sec warmup 104 sec latency 1534.012 ms 2023-07-14T11:37:02.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:02 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:37:02.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:02 smithi161 ceph-mon[95526]: pgmap v462: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 718 op/s 2023-07-14T11:37:02.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:02 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:37:02.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:02 smithi175 ceph-mon[129081]: pgmap v462: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 718 op/s 2023-07-14T11:37:02.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:02 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:37:02.964 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:02 smithi182 ceph-mon[129366]: pgmap v462: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 718 op/s 2023-07-14T11:37:03.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 64795 34.81 MB/sec warmup 105 sec latency 239.204 ms 2023-07-14T11:37:04.245 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:03 smithi161 ceph-mon[95526]: pgmap v463: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 716 op/s 2023-07-14T11:37:04.416 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:03 smithi175 ceph-mon[129081]: pgmap v463: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 716 op/s 2023-07-14T11:37:04.462 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:03 smithi182 ceph-mon[129366]: pgmap v463: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 716 op/s 2023-07-14T11:37:04.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 64806 34.51 MB/sec warmup 106 sec latency 368.049 ms 2023-07-14T11:37:05.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 64815 34.22 MB/sec warmup 107 sec latency 655.135 ms 2023-07-14T11:37:06.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 65047 34.01 MB/sec warmup 108 sec latency 359.810 ms 2023-07-14T11:37:06.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:06 smithi175 ceph-mon[129081]: pgmap v464: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 479 op/s 2023-07-14T11:37:06.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:06 smithi182 ceph-mon[129366]: pgmap v464: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 479 op/s 2023-07-14T11:37:06.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:06 smithi161 ceph-mon[95526]: pgmap v464: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 479 op/s 2023-07-14T11:37:07.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 65780 34.12 MB/sec warmup 109 sec latency 230.911 ms 2023-07-14T11:37:07.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:07 smithi175 ceph-mon[129081]: osdmap e110: 12 total, 12 up, 12 in 2023-07-14T11:37:07.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:07 smithi182 ceph-mon[129366]: osdmap e110: 12 total, 12 up, 12 in 2023-07-14T11:37:07.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:07 smithi161 ceph-mon[95526]: osdmap e110: 12 total, 12 up, 12 in 2023-07-14T11:37:08.579 INFO:tasks.workunit.client.0.smithi161.stdout: 10 66471 34.15 MB/sec warmup 110 sec latency 65.855 ms 2023-07-14T11:37:08.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:08 smithi175 ceph-mon[129081]: pgmap v466: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-07-14T11:37:08.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:08 smithi182 ceph-mon[129366]: pgmap v466: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-07-14T11:37:08.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:08 smithi161 ceph-mon[95526]: pgmap v466: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-07-14T11:37:09.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 66990 34.09 MB/sec warmup 111 sec latency 170.465 ms 2023-07-14T11:37:10.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 67721 34.11 MB/sec warmup 112 sec latency 77.664 ms 2023-07-14T11:37:10.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:10 smithi175 ceph-mon[129081]: pgmap v467: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-07-14T11:37:10.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:10 smithi182 ceph-mon[129366]: pgmap v467: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-07-14T11:37:10.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:10 smithi161 ceph-mon[95526]: pgmap v467: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-07-14T11:37:11.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 68463 34.15 MB/sec warmup 113 sec latency 73.211 ms 2023-07-14T11:37:12.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 68739 33.98 MB/sec warmup 114 sec latency 375.173 ms 2023-07-14T11:37:12.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:12 smithi175 ceph-mon[129081]: pgmap v468: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 500 op/s 2023-07-14T11:37:12.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:12 smithi182 ceph-mon[129366]: pgmap v468: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 500 op/s 2023-07-14T11:37:12.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:12 smithi161 ceph-mon[95526]: pgmap v468: 129 pgs: 2 active+clean+snaptrim, 119 active+clean, 8 active+clean+snaptrim_wait; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 500 op/s 2023-07-14T11:37:13.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 69204 33.93 MB/sec warmup 115 sec latency 86.750 ms 2023-07-14T11:37:14.417 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:14 smithi175 ceph-mon[129081]: pgmap v469: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 523 op/s 2023-07-14T11:37:14.462 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:14 smithi182 ceph-mon[129366]: pgmap v469: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 523 op/s 2023-07-14T11:37:14.492 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:14 smithi161 ceph-mon[95526]: pgmap v469: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 523 op/s 2023-07-14T11:37:14.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 69924 33.98 MB/sec warmup 116 sec latency 108.674 ms 2023-07-14T11:37:15.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 70781 34.09 MB/sec warmup 117 sec latency 100.646 ms 2023-07-14T11:37:16.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 71361 34.10 MB/sec warmup 118 sec latency 120.678 ms 2023-07-14T11:37:16.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:16 smithi175 ceph-mon[129081]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 523 op/s 2023-07-14T11:37:16.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:16 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:16.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:16 smithi182 ceph-mon[129366]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 523 op/s 2023-07-14T11:37:16.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:16 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:16.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:16 smithi161 ceph-mon[95526]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 523 op/s 2023-07-14T11:37:16.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:16 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:17.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 71577 33.91 MB/sec warmup 119 sec latency 743.662 ms 2023-07-14T11:37:18.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:18 smithi182 ceph-mon[129366]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.8 MiB/s wr, 570 op/s 2023-07-14T11:37:18.714 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:18 smithi182 ceph-mon[129366]: osdmap e111: 12 total, 12 up, 12 in 2023-07-14T11:37:18.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:18 smithi161 ceph-mon[95526]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.8 MiB/s wr, 570 op/s 2023-07-14T11:37:18.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:18 smithi161 ceph-mon[95526]: osdmap e111: 12 total, 12 up, 12 in 2023-07-14T11:37:18.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:18 smithi175 ceph-mon[129081]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.8 MiB/s wr, 570 op/s 2023-07-14T11:37:18.917 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:18 smithi175 ceph-mon[129081]: osdmap e111: 12 total, 12 up, 12 in 2023-07-14T11:37:19.581 INFO:tasks.workunit.client.0.smithi161.stdout: 10 72095 14.57 MB/sec execute 1 sec latency 1122.111 ms 2023-07-14T11:37:20.580 INFO:tasks.workunit.client.0.smithi161.stdout: 10 72571 21.33 MB/sec execute 2 sec latency 1939.987 ms 2023-07-14T11:37:20.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:20 smithi182 ceph-mon[129366]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 547 op/s 2023-07-14T11:37:20.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:20 smithi161 ceph-mon[95526]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 547 op/s 2023-07-14T11:37:20.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:20 smithi175 ceph-mon[129081]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 547 op/s 2023-07-14T11:37:21.581 INFO:tasks.workunit.client.0.smithi161.stdout: 10 73276 29.91 MB/sec execute 3 sec latency 2940.036 ms 2023-07-14T11:37:22.581 INFO:tasks.workunit.client.0.smithi161.stdout: 10 73782 27.28 MB/sec execute 4 sec latency 3940.088 ms 2023-07-14T11:37:22.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:22 smithi182 ceph-mon[129366]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 614 op/s 2023-07-14T11:37:22.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:22 smithi161 ceph-mon[95526]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 614 op/s 2023-07-14T11:37:22.753 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:37:22.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:22 smithi175 ceph-mon[129081]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 614 op/s 2023-07-14T11:37:23.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.115+0000 7f068b3ff700 1 -- 172.21.15.161:0/439480606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406c9a0 msgr2=0x7f068406ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.115+0000 7f068b3ff700 1 --2- 172.21.15.161:0/439480606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406c9a0 0x7f068406ce10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b0a0 tx=0x7f067c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.116+0000 7f068b3ff700 1 -- 172.21.15.161:0/439480606 shutdown_connections 2023-07-14T11:37:23.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.116+0000 7f068b3ff700 1 --2- 172.21.15.161:0/439480606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f068406d3a0 0x7f068411b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.116+0000 7f068b3ff700 1 --2- 172.21.15.161:0/439480606 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f068410e280 0x7f068410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.116+0000 7f068b3ff700 1 --2- 172.21.15.161:0/439480606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406c9a0 0x7f068406ce10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.116+0000 7f068b3ff700 1 -- 172.21.15.161:0/439480606 >> 172.21.15.161:0/439480606 conn(0x7f068406bf70 msgr2=0x7f0684070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.118+0000 7f068b3ff700 1 -- 172.21.15.161:0/439480606 shutdown_connections 2023-07-14T11:37:23.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.120+0000 7f068b3ff700 1 -- 172.21.15.161:0/439480606 wait complete. 2023-07-14T11:37:23.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.122+0000 7f068b3ff700 1 Processor -- start 2023-07-14T11:37:23.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.122+0000 7f068b3ff700 1 -- start start 2023-07-14T11:37:23.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.122+0000 7f068b3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f068406c9a0 0x7f06841aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.122+0000 7f068b3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406d3a0 0x7f06841aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.122+0000 7f068b3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f068410e280 0x7f06841a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.123+0000 7f068b3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a50b0 con 0x7f068406d3a0 2023-07-14T11:37:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.123+0000 7f068b3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a5220 con 0x7f068406c9a0 2023-07-14T11:37:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.123+0000 7f068b3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a53e0 con 0x7f068410e280 2023-07-14T11:37:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.123+0000 7f068919b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f068406c9a0 0x7f06841aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.123+0000 7f068919b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f068406c9a0 0x7f06841aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51354/0 (socket says 172.21.15.161:51354) 2023-07-14T11:37:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.123+0000 7f068919b700 1 -- 172.21.15.161:0/203260520 learned_addr learned my addr 172.21.15.161:0/203260520 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.123+0000 7f068899a700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406d3a0 0x7f06841aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.124+0000 7f068899a700 1 -- 172.21.15.161:0/203260520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f068410e280 msgr2=0x7f06841a4a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:37:23.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.124+0000 7f068899a700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f068410e280 0x7f06841a4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.124+0000 7f068899a700 1 -- 172.21.15.161:0/203260520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f068406c9a0 msgr2=0x7f06841aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.124+0000 7f068899a700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f068406c9a0 0x7f06841aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.124+0000 7f068899a700 1 -- 172.21.15.161:0/203260520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009d00 con 0x7f068406d3a0 2023-07-14T11:37:23.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.125+0000 7f068899a700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406d3a0 0x7f06841aaf10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b070 tx=0x7f067c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.125+0000 7f067a7fc700 1 -- 172.21.15.161:0/203260520 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c021070 con 0x7f068406d3a0 2023-07-14T11:37:23.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.125+0000 7f067a7fc700 1 -- 172.21.15.161:0/203260520 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067c004e20 con 0x7f068406d3a0 2023-07-14T11:37:23.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.125+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06841a5660 con 0x7f068406d3a0 2023-07-14T11:37:23.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.126+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068410ee20 con 0x7f068406d3a0 2023-07-14T11:37:23.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.126+0000 7f067a7fc700 1 -- 172.21.15.161:0/203260520 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c00e040 con 0x7f068406d3a0 2023-07-14T11:37:23.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.132+0000 7f069edb9b80 1 Processor -- start 2023-07-14T11:37:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.133+0000 7f069edb9b80 1 -- start start 2023-07-14T11:37:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.133+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca02d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.133+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 0x55bfca038590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.133+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 0x55bfca03aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.133+0000 7f069edb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9f33cb0 con 0x55bfca02d2d0 2023-07-14T11:37:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.133+0000 7f069edb9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9f86a40 con 0x55bfca02dbe0 2023-07-14T11:37:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.133+0000 7f069edb9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9f7fed0 con 0x55bfca038ad0 2023-07-14T11:37:23.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.134+0000 7f068999c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca02d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.134+0000 7f068999c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca02d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51050/0 (socket says 172.21.15.161:51050) 2023-07-14T11:37:23.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.134+0000 7f068999c700 1 -- 172.21.15.161:0/2517293165 learned_addr learned my addr 172.21.15.161:0/2517293165 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:23.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.134+0000 7f067a7fc700 1 -- 172.21.15.161:0/203260520 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f067c0040b0 con 0x7f068406d3a0 2023-07-14T11:37:23.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.135+0000 7f068919b700 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 0x55bfca038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.135+0000 7f068999c700 1 -- 172.21.15.161:0/2517293165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 msgr2=0x55bfca03aeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:37:23.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.135+0000 7f068999c700 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 0x55bfca03aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.135+0000 7f068999c700 1 -- 172.21.15.161:0/2517293165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 msgr2=0x55bfca038590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:23.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.135+0000 7f068999c700 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 0x55bfca038590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.135+0000 7f068999c700 1 -- 172.21.15.161:0/2517293165 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc9dadde0 con 0x55bfca02d2d0 2023-07-14T11:37:23.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.135+0000 7f068999c700 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca02d6a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0680009780 tx=0x7f0680009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26865d7464aa0a58 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.136+0000 7f066f7fe700 1 -- 172.21.15.161:0/2517293165 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680011830 con 0x55bfca02d2d0 2023-07-14T11:37:23.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.136+0000 7f066f7fe700 1 -- 172.21.15.161:0/2517293165 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0680011e50 con 0x55bfca02d2d0 2023-07-14T11:37:23.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.136+0000 7f066f7fe700 1 -- 172.21.15.161:0/2517293165 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680010e80 con 0x55bfca02d2d0 2023-07-14T11:37:23.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.137+0000 7f067a7fc700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06700a1380 0x7f06700a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.137+0000 7f067a7fc700 1 -- 172.21.15.161:0/203260520 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f067c0ce180 con 0x7f068406d3a0 2023-07-14T11:37:23.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.137+0000 7f069edb9b80 1 -- 172.21.15.161:0/2517293165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 msgr2=0x55bfca02d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.137+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca02d6a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0680009780 tx=0x7f0680009a90 comp rx=0 tx=0).stop 2023-07-14T11:37:23.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.137+0000 7f068919b700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06700a1380 0x7f06700a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.138+0000 7f069edb9b80 1 -- 172.21.15.161:0/2517293165 shutdown_connections 2023-07-14T11:37:23.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.138+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca02d6a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.138+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 0x55bfca03aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.138+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2517293165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 0x55bfca038590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.138+0000 7f069edb9b80 1 -- 172.21.15.161:0/2517293165 >> 172.21.15.161:0/2517293165 conn(0x55bfca023da0 msgr2=0x55bfca0296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.138+0000 7f069edb9b80 1 -- 172.21.15.161:0/2517293165 shutdown_connections 2023-07-14T11:37:23.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.138+0000 7f069edb9b80 1 -- 172.21.15.161:0/2517293165 wait complete. 2023-07-14T11:37:23.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.139+0000 7f069edb9b80 1 -- start start 2023-07-14T11:37:23.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.139+0000 7f069edb9b80 1 Processor -- start 2023-07-14T11:37:23.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.140+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca038590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.140+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 0x55bfca0357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.140+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 0x55bfca035ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.140+0000 7f069edb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9f1fac0 con 0x55bfca02d2d0 2023-07-14T11:37:23.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.140+0000 7f069edb9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9f1f7c0 con 0x55bfca02dbe0 2023-07-14T11:37:23.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.140+0000 7f069edb9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9e03920 con 0x55bfca038ad0 2023-07-14T11:37:23.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.140+0000 7f069edb9b80 10 client.0 _ll_register_callbacks cb 0x55bfc9f1d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:23.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.143+0000 7f068919b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.143+0000 7f068919b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca038590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51052/0 (socket says 172.21.15.161:51052) 2023-07-14T11:37:23.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.143+0000 7f068919b700 1 -- 172.21.15.161:0/1428291604 learned_addr learned my addr 172.21.15.161:0/1428291604 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:23.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.143+0000 7f068919b700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06700a1380 0x7f06700a3830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0674005950 tx=0x7f067400b500 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.144+0000 7f068919b700 1 -- 172.21.15.161:0/1428291604 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 msgr2=0x55bfca035ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.144+0000 7f068919b700 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 0x55bfca035ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.144+0000 7f068919b700 1 -- 172.21.15.161:0/1428291604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 msgr2=0x55bfca0357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.144+0000 7f068919b700 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 0x55bfca0357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.144+0000 7f068919b700 1 -- 172.21.15.161:0/1428291604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06800093b0 con 0x55bfca02d2d0 2023-07-14T11:37:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.144+0000 7f068919b700 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca038590 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0674010a40 tx=0x7f0674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf64fa0fff4c2a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.147+0000 7f066cff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:37:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.147+0000 7f066cff9700 1 -- 172.21.15.161:0/1428291604 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674004460 con 0x55bfca02d2d0 2023-07-14T11:37:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.147+0000 7f066cff9700 1 -- 172.21.15.161:0/1428291604 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f067401d890 con 0x55bfca02d2d0 2023-07-14T11:37:23.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.148+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfc9ed94f0 con 0x55bfca02d2d0 2023-07-14T11:37:23.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.148+0000 7f066cff9700 1 -- 172.21.15.161:0/1428291604 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0674010eb0 con 0x55bfca02d2d0 2023-07-14T11:37:23.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.148+0000 7f069edb9b80 10 client.17190 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:23.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.148+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55bfc9d96c60 con 0x55bfca02d2d0 2023-07-14T11:37:23.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.149+0000 7f066cff9700 1 -- 172.21.15.161:0/1428291604 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfc9ed94f0 con 0x55bfca02d2d0 2023-07-14T11:37:23.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.150+0000 7f066cff9700 1 -- 172.21.15.161:0/1428291604 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0674005020 con 0x55bfca02d2d0 2023-07-14T11:37:23.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.150+0000 7f069edb9b80 10 client.17190 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:23.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.150+0000 7f069edb9b80 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:23.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.150+0000 7f069edb9b80 20 client.17190 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:23.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.150+0000 7f069edb9b80 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca048430 0x55bfca04a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.150+0000 7f069edb9b80 4 client.17190 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:23.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.150+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfc9f78eb0 con 0x55bfca048430 2023-07-14T11:37:23.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.154+0000 7f068899a700 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca048430 0x55bfca04a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.156+0000 7f068899a700 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca048430 0x55bfca04a840 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae813b6892c465e5 server_cookie=da2712100d8cd8e2 in_seq=0 out_seq=0 2023-07-14T11:37:23.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.157+0000 7f066cff9700 10 client.17190 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:23.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.158+0000 7f066cff9700 1 -- 172.21.15.161:0/1428291604 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bfc9f78eb0 con 0x55bfca048430 2023-07-14T11:37:23.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.158+0000 7f066cff9700 10 client.17190 handle_command_reply: tid=0 2023-07-14T11:37:23.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.158+0000 7f069edb9b80 1 client.17190 shutdown 2023-07-14T11:37:23.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.160+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 msgr2=0x55bfca038590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.160+0000 7f069edb9b80 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca038590 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0674010a40 tx=0x7f0674004750 comp rx=0 tx=0).stop 2023-07-14T11:37:23.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.160+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 shutdown_connections 2023-07-14T11:37:23.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.160+0000 7f069edb9b80 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca048430 0x55bfca04a840 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.160+0000 7f069edb9b80 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca038590 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.160+0000 7f069edb9b80 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca038ad0 0x55bfca035ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.160+0000 7f069edb9b80 1 --2- 172.21.15.161:0/1428291604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02dbe0 0x55bfca0357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.161+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 >> 172.21.15.161:0/1428291604 conn(0x55bfca023da0 msgr2=0x55bfca029230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.161+0000 7f066cff9700 0 client.17190 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:23.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.161+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 shutdown_connections 2023-07-14T11:37:23.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.161+0000 7f069edb9b80 1 -- 172.21.15.161:0/1428291604 wait complete. 2023-07-14T11:37:23.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.162+0000 7f069edb9b80 20 client.17190 trim_cache size 0 max 16384 2023-07-14T11:37:23.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.184+0000 7f069edb9b80 1 Processor -- start 2023-07-14T11:37:23.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.184+0000 7f069edb9b80 1 -- start start 2023-07-14T11:37:23.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.184+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca0380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.184+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca035040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca038ad0 0x55bfca035580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f069edb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfca02e540 con 0x55bfca02d2d0 2023-07-14T11:37:23.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f069edb9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfca044e40 con 0x55bfca038ad0 2023-07-14T11:37:23.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f069edb9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfca02da20 con 0x55bfca02dbe0 2023-07-14T11:37:23.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068899a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca035040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068899a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca035040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36206/0 (socket says 172.21.15.161:36206) 2023-07-14T11:37:23.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068899a700 1 -- 172.21.15.161:0/4022828477 learned_addr learned my addr 172.21.15.161:0/4022828477 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:23.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068999c700 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca038ad0 0x55bfca035580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068919b700 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca0380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068999c700 1 -- 172.21.15.161:0/4022828477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 msgr2=0x55bfca035040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:23.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068999c700 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca035040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068999c700 1 -- 172.21.15.161:0/4022828477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 msgr2=0x55bfca0380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:23.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068999c700 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca0380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.185+0000 7f068999c700 1 -- 172.21.15.161:0/4022828477 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfca0255e0 con 0x55bfca038ad0 2023-07-14T11:37:23.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.186+0000 7f068919b700 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca0380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:37:23.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.186+0000 7f068899a700 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca035040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:23.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.186+0000 7f068999c700 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca038ad0 0x55bfca035580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0680009750 tx=0x7f0680009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bc19f891ad0ac0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.186+0000 7f066ffff700 1 -- 172.21.15.161:0/4022828477 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680011830 con 0x55bfca038ad0 2023-07-14T11:37:23.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.186+0000 7f066ffff700 1 -- 172.21.15.161:0/4022828477 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f068001a420 con 0x55bfca038ad0 2023-07-14T11:37:23.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.186+0000 7f066ffff700 1 -- 172.21.15.161:0/4022828477 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680021630 con 0x55bfca038ad0 2023-07-14T11:37:23.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.187+0000 7f069edb9b80 1 -- 172.21.15.161:0/4022828477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca038ad0 msgr2=0x55bfca035580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.187+0000 7f069edb9b80 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca038ad0 0x55bfca035580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0680009750 tx=0x7f0680009290 comp rx=0 tx=0).stop 2023-07-14T11:37:23.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 -- 172.21.15.161:0/4022828477 shutdown_connections 2023-07-14T11:37:23.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca02d2d0 0x55bfca0380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca038ad0 0x55bfca035580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 --2- 172.21.15.161:0/4022828477 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca035040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 -- 172.21.15.161:0/4022828477 >> 172.21.15.161:0/4022828477 conn(0x55bfca023da0 msgr2=0x55bfca030ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 -- 172.21.15.161:0/4022828477 shutdown_connections 2023-07-14T11:37:23.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 -- 172.21.15.161:0/4022828477 wait complete. 2023-07-14T11:37:23.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.188+0000 7f069edb9b80 1 -- start start 2023-07-14T11:37:23.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.189+0000 7f069edb9b80 1 Processor -- start 2023-07-14T11:37:23.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.189+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02d2d0 0x55bfca03f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.189+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca03d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f069edb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca038ad0 0x55bfca03dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f069edb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfca03e270 con 0x55bfca038ad0 2023-07-14T11:37:23.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f069edb9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfca03e3b0 con 0x55bfca02d2d0 2023-07-14T11:37:23.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f069edb9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfca03e570 con 0x55bfca02dbe0 2023-07-14T11:37:23.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f069edb9b80 10 client.0 _ll_register_callbacks cb 0x7f06740105f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:23.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f068919b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02d2d0 0x55bfca03f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f068999c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca038ad0 0x55bfca03dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f068919b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02d2d0 0x55bfca03f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51384/0 (socket says 172.21.15.161:51384) 2023-07-14T11:37:23.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f068919b700 1 -- 172.21.15.161:0/2645965817 learned_addr learned my addr 172.21.15.161:0/2645965817 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:23.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f068899a700 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca03d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.190+0000 7f068899a700 1 -- 172.21.15.161:0/2645965817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02d2d0 msgr2=0x55bfca03f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:23.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f068899a700 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02d2d0 0x55bfca03f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f068899a700 1 -- 172.21.15.161:0/2645965817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca038ad0 msgr2=0x55bfca03dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:23.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f068899a700 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca038ad0 0x55bfca03dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f068899a700 1 -- 172.21.15.161:0/2645965817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068410ee20 con 0x55bfca02dbe0 2023-07-14T11:37:23.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f068899a700 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca03d760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f067c097a60 tx=0x7f067c00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374356976ec74a47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f066cff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:23.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f066cff9700 1 -- 172.21.15.161:0/2645965817 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0cb9d0 con 0x55bfca02dbe0 2023-07-14T11:37:23.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f066cff9700 1 -- 172.21.15.161:0/2645965817 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f067c09aff0 con 0x55bfca02dbe0 2023-07-14T11:37:23.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.191+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0680011e50 con 0x55bfca02dbe0 2023-07-14T11:37:23.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.192+0000 7f066cff9700 1 -- 172.21.15.161:0/2645965817 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f067c09f020 con 0x55bfca02dbe0 2023-07-14T11:37:23.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.192+0000 7f069edb9b80 10 client.27268 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:23.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.192+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55bfc9dadde0 con 0x55bfca02dbe0 2023-07-14T11:37:23.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.192+0000 7f066cff9700 1 -- 172.21.15.161:0/2645965817 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067c0c9080 con 0x55bfca02dbe0 2023-07-14T11:37:23.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.193+0000 7f066cff9700 1 -- 172.21.15.161:0/2645965817 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f067c09b250 con 0x55bfca02dbe0 2023-07-14T11:37:23.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.193+0000 7f069edb9b80 10 client.27268 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:23.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.193+0000 7f069edb9b80 10 client.27268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:23.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.193+0000 7f069edb9b80 20 client.27268 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:23.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.193+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca09d4c0 0x55bfca09f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.193+0000 7f069edb9b80 4 client.27268 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:37:23.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.193+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bfc9f3b050 con 0x55bfca09d4c0 2023-07-14T11:37:23.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.194+0000 7f068919b700 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca09d4c0 0x55bfca09f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.195+0000 7f068919b700 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca09d4c0 0x55bfca09f8d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75a3cca8d34f17b server_cookie=1d1dc7cf27db5000 in_seq=0 out_seq=0 2023-07-14T11:37:23.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.195+0000 7f066cff9700 10 client.27268 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:23.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.196+0000 7f066cff9700 1 -- 172.21.15.161:0/2645965817 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bfc9f3b050 con 0x55bfca09d4c0 2023-07-14T11:37:23.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.196+0000 7f066cff9700 10 client.27268 handle_command_reply: tid=0 2023-07-14T11:37:23.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.196+0000 7f069edb9b80 1 client.27268 shutdown 2023-07-14T11:37:23.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.196+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 msgr2=0x55bfca03d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.196+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca03d760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f067c097a60 tx=0x7f067c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 shutdown_connections 2023-07-14T11:37:23.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bfca09d4c0 0x55bfca09f8d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfca038ad0 0x55bfca03dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bfca02d2d0 0x55bfca03f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 --2- 172.21.15.161:0/2645965817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bfca02dbe0 0x55bfca03d760 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 >> 172.21.15.161:0/2645965817 conn(0x55bfca023da0 msgr2=0x55bfca035040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f066cff9700 0 client.27268 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:23.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 shutdown_connections 2023-07-14T11:37:23.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 1 -- 172.21.15.161:0/2645965817 wait complete. 2023-07-14T11:37:23.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.198+0000 7f069edb9b80 20 client.27268 trim_cache size 0 max 16384 2023-07-14T11:37:23.212 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:37:23.212 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:37:23.212 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:37:23.212 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:37:23.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06700a1380 msgr2=0x7f06700a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06700a1380 0x7f06700a3830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0674005950 tx=0x7f067400b500 comp rx=0 tx=0).stop 2023-07-14T11:37:23.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406d3a0 msgr2=0x7f06841aaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406d3a0 0x7f06841aaf10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b070 tx=0x7f067c00ce50 comp rx=0 tx=0).stop 2023-07-14T11:37:23.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 shutdown_connections 2023-07-14T11:37:23.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06700a1380 0x7f06700a3830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f068410e280 0x7f06841a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f068406c9a0 0x7f06841aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 --2- 172.21.15.161:0/203260520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f068406d3a0 0x7f06841aaf10 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b070 tx=0x7f067c00ce50 comp rx=0 tx=0).stop 2023-07-14T11:37:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.199+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 >> 172.21.15.161:0/203260520 conn(0x7f068406bf70 msgr2=0x7f068406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.202+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 shutdown_connections 2023-07-14T11:37:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.202+0000 7f068b3ff700 1 -- 172.21.15.161:0/203260520 wait complete. 2023-07-14T11:37:23.230 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:37:23.230 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:37:23.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 -- 172.21.15.161:0/1611659764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d901004a0 msgr2=0x7f8d9010c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.486 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1611659764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d901004a0 0x7f8d9010c930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009f90 tx=0x7f8d8800b330 comp rx=0 tx=0).stop 2023-07-14T11:37:23.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 -- 172.21.15.161:0/1611659764 shutdown_connections 2023-07-14T11:37:23.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1611659764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d901004a0 0x7f8d9010c930 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1611659764 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d90105b00 0x7f8d90105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1611659764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 0x7f8d900fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 -- 172.21.15.161:0/1611659764 >> 172.21.15.161:0/1611659764 conn(0x7f8d900fb630 msgr2=0x7f8d900fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 -- 172.21.15.161:0/1611659764 shutdown_connections 2023-07-14T11:37:23.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.487+0000 7f8d955ff700 1 -- 172.21.15.161:0/1611659764 wait complete. 2023-07-14T11:37:23.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.488+0000 7f8d955ff700 1 Processor -- start 2023-07-14T11:37:23.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.488+0000 7f8d955ff700 1 -- start start 2023-07-14T11:37:23.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d955ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 0x7f8d90197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d955ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d901004a0 0x7f8d901984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d955ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d90105b00 0x7f8d9019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d955ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019cf80 con 0x7f8d90105b00 2023-07-14T11:37:23.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d955ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019d0f0 con 0x7f8d900ffaf0 2023-07-14T11:37:23.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d955ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019d2b0 con 0x7f8d901004a0 2023-07-14T11:37:23.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 0x7f8d90197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 0x7f8d90197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51400/0 (socket says 172.21.15.161:51400) 2023-07-14T11:37:23.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8effd700 1 -- 172.21.15.161:0/1481630560 learned_addr learned my addr 172.21.15.161:0/1481630560 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:23.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8e7fc700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d901004a0 0x7f8d901984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8f7fe700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d90105b00 0x7f8d9019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8effd700 1 -- 172.21.15.161:0/1481630560 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d901004a0 msgr2=0x7f8d901984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8effd700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d901004a0 0x7f8d901984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8effd700 1 -- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d90105b00 msgr2=0x7f8d9019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.489+0000 7f8d8effd700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d90105b00 0x7f8d9019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d8effd700 1 -- 172.21.15.161:0/1481630560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800095f0 con 0x7f8d900ffaf0 2023-07-14T11:37:23.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d8f7fe700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d90105b00 0x7f8d9019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:37:23.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d8effd700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 0x7f8d90197fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000ec10 tx=0x7f8d8000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d77fff700 1 -- 172.21.15.161:0/1481630560 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000cd40 con 0x7f8d900ffaf0 2023-07-14T11:37:23.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d88009c40 con 0x7f8d900ffaf0 2023-07-14T11:37:23.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9019d780 con 0x7f8d900ffaf0 2023-07-14T11:37:23.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d77fff700 1 -- 172.21.15.161:0/1481630560 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d800044e0 con 0x7f8d900ffaf0 2023-07-14T11:37:23.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.490+0000 7f8d77fff700 1 -- 172.21.15.161:0/1481630560 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80010690 con 0x7f8d900ffaf0 2023-07-14T11:37:23.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.491+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7c005320 con 0x7f8d900ffaf0 2023-07-14T11:37:23.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.494+0000 7f8d77fff700 1 -- 172.21.15.161:0/1481630560 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8d80003680 con 0x7f8d900ffaf0 2023-07-14T11:37:23.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.495+0000 7f8d77fff700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8d780a1380 0x7f8d780a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:23.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.495+0000 7f8d77fff700 1 -- 172.21.15.161:0/1481630560 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8d80014070 con 0x7f8d900ffaf0 2023-07-14T11:37:23.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.496+0000 7f8d8e7fc700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8d780a1380 0x7f8d780a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:23.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.496+0000 7f8d8e7fc700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8d780a1380 0x7f8d780a3830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d901995b0 tx=0x7f8d84009190 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:23.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.499+0000 7f8d77fff700 1 -- 172.21.15.161:0/1481630560 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8d80096540 con 0x7f8d900ffaf0 2023-07-14T11:37:23.581 INFO:tasks.workunit.client.0.smithi161.stdout: 10 74455 27.85 MB/sec execute 5 sec latency 4940.128 ms 2023-07-14T11:37:23.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.841+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d7c005190 con 0x7f8d900ffaf0 2023-07-14T11:37:23.841 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:37:23.841 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:37:15.325158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:37:23.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.842+0000 7f8d77fff700 1 -- 172.21.15.161:0/1481630560 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10871 (secure 0 0 0) 0x7f8d80099b60 con 0x7f8d900ffaf0 2023-07-14T11:37:23.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8d780a1380 msgr2=0x7f8d780a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8d780a1380 0x7f8d780a3830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d901995b0 tx=0x7f8d84009190 comp rx=0 tx=0).stop 2023-07-14T11:37:23.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 msgr2=0x7f8d90197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:23.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 0x7f8d90197fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000ec10 tx=0x7f8d8000ef20 comp rx=0 tx=0).stop 2023-07-14T11:37:23.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 shutdown_connections 2023-07-14T11:37:23.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8d780a1380 0x7f8d780a3830 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d90105b00 0x7f8d9019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d900ffaf0 0x7f8d90197fa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 --2- 172.21.15.161:0/1481630560 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8d901004a0 0x7f8d901984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:23.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 >> 172.21.15.161:0/1481630560 conn(0x7f8d900fb630 msgr2=0x7f8d9010ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:23.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 shutdown_connections 2023-07-14T11:37:23.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:23.845+0000 7f8d955ff700 1 -- 172.21.15.161:0/1481630560 wait complete. 2023-07-14T11:37:23.846 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 38 2023-07-14T11:37:23.867 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:37:24.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.126+0000 7ff551fff700 1 -- 172.21.15.161:0/570808670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 msgr2=0x7ff54c1068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.126+0000 7ff551fff700 1 --2- 172.21.15.161:0/570808670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c1068d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff534009f90 tx=0x7ff53400b330 comp rx=0 tx=0).stop 2023-07-14T11:37:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.127+0000 7ff551fff700 1 -- 172.21.15.161:0/570808670 shutdown_connections 2023-07-14T11:37:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.127+0000 7ff551fff700 1 --2- 172.21.15.161:0/570808670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 0x7ff54c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.127+0000 7ff551fff700 1 --2- 172.21.15.161:0/570808670 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff54c106fb0 0x7ff54c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.127+0000 7ff551fff700 1 --2- 172.21.15.161:0/570808670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c1068d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.127+0000 7ff551fff700 1 -- 172.21.15.161:0/570808670 >> 172.21.15.161:0/570808670 conn(0x7ff54c0fd7e0 msgr2=0x7ff54c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.127+0000 7ff551fff700 1 -- 172.21.15.161:0/570808670 shutdown_connections 2023-07-14T11:37:24.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.127+0000 7ff551fff700 1 -- 172.21.15.161:0/570808670 wait complete. 2023-07-14T11:37:24.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.128+0000 7ff551fff700 1 Processor -- start 2023-07-14T11:37:24.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.129+0000 7ff551fff700 1 -- start start 2023-07-14T11:37:24.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.129+0000 7ff551fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 0x7ff54c10d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.129+0000 7ff551fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c10d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.129+0000 7ff551fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff54c106fb0 0x7ff54c1a8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.129+0000 7ff551fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a86b0 con 0x7ff54c101f70 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.129+0000 7ff551fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a87f0 con 0x7ff54c102910 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.129+0000 7ff551fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a8930 con 0x7ff54c106fb0 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 0x7ff54c10d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c10d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c10d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51406/0 (socket says 172.21.15.161:51406) 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 -- 172.21.15.161:0/3313520828 learned_addr learned my addr 172.21.15.161:0/3313520828 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff54c106fb0 0x7ff54c1a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 0x7ff54c10d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51086/0 (socket says 172.21.15.161:51086) 2023-07-14T11:37:24.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 -- 172.21.15.161:0/3313520828 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff54c106fb0 msgr2=0x7ff54c1a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff54c106fb0 0x7ff54c1a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 -- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 msgr2=0x7ff54c10d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 0x7ff54c10d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 -- 172.21.15.161:0/3313520828 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534009c40 con 0x7ff54c102910 2023-07-14T11:37:24.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 0x7ff54c10d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:24.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.130+0000 7ff54affd700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c10d890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff53400b6c0 tx=0x7ff534004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.131+0000 7ff548ff9700 1 -- 172.21.15.161:0/3313520828 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53400e040 con 0x7ff54c102910 2023-07-14T11:37:24.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.131+0000 7ff548ff9700 1 -- 172.21.15.161:0/3313520828 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff534008e80 con 0x7ff54c102910 2023-07-14T11:37:24.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.131+0000 7ff548ff9700 1 -- 172.21.15.161:0/3313520828 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5340052a0 con 0x7ff54c102910 2023-07-14T11:37:24.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.131+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c1a8bb0 con 0x7ff54c102910 2023-07-14T11:37:24.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.131+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1a9030 con 0x7ff54c102910 2023-07-14T11:37:24.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.135+0000 7ff548ff9700 1 -- 172.21.15.161:0/3313520828 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff53401d470 con 0x7ff54c102910 2023-07-14T11:37:24.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.136+0000 7ff5659e3b80 1 Processor -- start 2023-07-14T11:37:24.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.136+0000 7ff5659e3b80 1 -- start start 2023-07-14T11:37:24.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba33cacb0 con 0x556ba34c4be0 2023-07-14T11:37:24.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba341da40 con 0x556ba34c42d0 2023-07-14T11:37:24.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba3416ed0 con 0x556ba34cfad0 2023-07-14T11:37:24.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff54bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff54bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51422/0 (socket says 172.21.15.161:51422) 2023-07-14T11:37:24.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff54bfff700 1 -- 172.21.15.161:0/1376916476 learned_addr learned my addr 172.21.15.161:0/1376916476 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.137+0000 7ff54affd700 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff54b7fe700 1 -- 172.21.15.161:0/1376916476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 msgr2=0x556ba34d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff54b7fe700 1 -- 172.21.15.161:0/1376916476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 msgr2=0x556ba34c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff54b7fe700 1 -- 172.21.15.161:0/1376916476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba3244de0 con 0x556ba34c4be0 2023-07-14T11:37:24.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34cf590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff5440114f0 tx=0x7ff544011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6be6380c0140c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff548ff9700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff52c0a13c0 0x7ff52c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff541ffb700 1 -- 172.21.15.161:0/1376916476 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544004230 con 0x556ba34c4be0 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff541ffb700 1 -- 172.21.15.161:0/1376916476 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544004cf0 con 0x556ba34c4be0 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff541ffb700 1 -- 172.21.15.161:0/1376916476 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544005630 con 0x556ba34c4be0 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.138+0000 7ff548ff9700 1 -- 172.21.15.161:0/3313520828 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff5340ce400 con 0x7ff54c102910 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff54bfff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff52c0a13c0 0x7ff52c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff5659e3b80 1 -- 172.21.15.161:0/1376916476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 msgr2=0x556ba34cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34cf590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff5440114f0 tx=0x7ff544011800 comp rx=0 tx=0).stop 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff5659e3b80 1 -- 172.21.15.161:0/1376916476 shutdown_connections 2023-07-14T11:37:24.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/1376916476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34cf590 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff5659e3b80 1 -- 172.21.15.161:0/1376916476 >> 172.21.15.161:0/1376916476 conn(0x556ba34bada0 msgr2=0x556ba34c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.139+0000 7ff54bfff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff52c0a13c0 0x7ff52c0a3870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009eb0 tx=0x7ff53c009310 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.140+0000 7ff5659e3b80 1 -- 172.21.15.161:0/1376916476 shutdown_connections 2023-07-14T11:37:24.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.140+0000 7ff5659e3b80 1 -- 172.21.15.161:0/1376916476 wait complete. 2023-07-14T11:37:24.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.140+0000 7ff5659e3b80 1 -- start start 2023-07-14T11:37:24.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.140+0000 7ff5659e3b80 1 Processor -- start 2023-07-14T11:37:24.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34c42d0 0x556ba34cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34cfad0 0x556ba34cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba33b6ac0 con 0x556ba34cfad0 2023-07-14T11:37:24.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba33b67c0 con 0x556ba34c4be0 2023-07-14T11:37:24.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba329a920 con 0x556ba34c42d0 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff5659e3b80 10 client.0 _ll_register_callbacks cb 0x556ba33b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff54affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34c42d0 0x556ba34cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.141+0000 7ff54affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51428/0 (socket says 172.21.15.161:51428) 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54affd700 1 -- 172.21.15.161:0/4091289706 learned_addr learned my addr 172.21.15.161:0/4091289706 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54bfff700 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34cfad0 0x556ba34cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54affd700 1 -- 172.21.15.161:0/4091289706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34c42d0 msgr2=0x556ba34cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54affd700 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34c42d0 0x556ba34cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54affd700 1 -- 172.21.15.161:0/4091289706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34cfad0 msgr2=0x556ba34cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54affd700 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34cfad0 0x556ba34cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54affd700 1 -- 172.21.15.161:0/4091289706 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff544009c90 con 0x556ba34c4be0 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff54affd700 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34cc7c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff534097c10 tx=0x7ff5340092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c42243ca3b088f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff53affd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff53affd700 1 -- 172.21.15.161:0/4091289706 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53409af70 con 0x556ba34c4be0 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.142+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ba33704f0 con 0x556ba34c4be0 2023-07-14T11:37:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.143+0000 7ff53affd700 1 -- 172.21.15.161:0/4091289706 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff53401de40 con 0x556ba34c4be0 2023-07-14T11:37:24.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.143+0000 7ff53affd700 1 -- 172.21.15.161:0/4091289706 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53409ace0 con 0x556ba34c4be0 2023-07-14T11:37:24.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.143+0000 7ff5659e3b80 10 client.27245 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:24.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.143+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x556ba322dc60 con 0x556ba34c4be0 2023-07-14T11:37:24.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.143+0000 7ff53affd700 1 -- 172.21.15.161:0/4091289706 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ba33704f0 con 0x556ba34c4be0 2023-07-14T11:37:24.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff53affd700 1 -- 172.21.15.161:0/4091289706 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff5340d1c40 con 0x556ba34c4be0 2023-07-14T11:37:24.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff5659e3b80 10 client.27245 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:24.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff5659e3b80 10 client.27245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:37:24.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff5659e3b80 20 client.27245 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:24.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba34df120 0x556ba34e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff5659e3b80 4 client.27245 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:24.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ba340feb0 con 0x556ba34df120 2023-07-14T11:37:24.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.144+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba34df120 0x556ba34e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.145+0000 7ff53affd700 10 client.27245 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.145+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba34df120 0x556ba34e1530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=959a90ff56e30d5b server_cookie=3c1b77456df9cb8b in_seq=0 out_seq=0 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff53affd700 1 -- 172.21.15.161:0/4091289706 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ba340feb0 con 0x556ba34df120 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff53affd700 10 client.27245 handle_command_reply: tid=0 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 client.27245 shutdown 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 msgr2=0x556ba34cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34cc7c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff534097c10 tx=0x7ff5340092e0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 shutdown_connections 2023-07-14T11:37:24.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba34df120 0x556ba34e1530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34cfad0 0x556ba34cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34c42d0 0x556ba34cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/4091289706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34cc7c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 >> 172.21.15.161:0/4091289706 conn(0x556ba34bada0 msgr2=0x556ba34c0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff53affd700 0 client.27245 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:24.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.148+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 shutdown_connections 2023-07-14T11:37:24.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.149+0000 7ff5659e3b80 1 -- 172.21.15.161:0/4091289706 wait complete. 2023-07-14T11:37:24.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.149+0000 7ff5659e3b80 20 client.27245 trim_cache size 0 max 16384 2023-07-14T11:37:24.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.164+0000 7ff5659e3b80 1 Processor -- start 2023-07-14T11:37:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.164+0000 7ff5659e3b80 1 -- start start 2023-07-14T11:37:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.164+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c42d0 0x556ba34d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba34cd4b0 con 0x556ba34c42d0 2023-07-14T11:37:24.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba34c5dd0 con 0x556ba34c4be0 2023-07-14T11:37:24.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba34d2270 con 0x556ba34cfad0 2023-07-14T11:37:24.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34db6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51444/0 (socket says 172.21.15.161:51444) 2023-07-14T11:37:24.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 -- 172.21.15.161:0/346330991 learned_addr learned my addr 172.21.15.161:0/346330991 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c42d0 0x556ba34d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 -- 172.21.15.161:0/346330991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 msgr2=0x556ba34cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 -- 172.21.15.161:0/346330991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c42d0 msgr2=0x556ba34d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c42d0 0x556ba34d6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54affd700 1 -- 172.21.15.161:0/346330991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba326b250 con 0x556ba34c4be0 2023-07-14T11:37:24.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.165+0000 7ff54bfff700 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:24.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.166+0000 7ff54affd700 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34db6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff534000f80 tx=0x7ff534097ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e20034656eec8e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.168+0000 7ff5427fc700 1 -- 172.21.15.161:0/346330991 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53409b1a0 con 0x556ba34c4be0 2023-07-14T11:37:24.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.168+0000 7ff5427fc700 1 -- 172.21.15.161:0/346330991 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff53400be00 con 0x556ba34c4be0 2023-07-14T11:37:24.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.168+0000 7ff5427fc700 1 -- 172.21.15.161:0/346330991 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5340a3330 con 0x556ba34c4be0 2023-07-14T11:37:24.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 -- 172.21.15.161:0/346330991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 msgr2=0x556ba34db6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34db6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff534000f80 tx=0x7ff534097ac0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 -- 172.21.15.161:0/346330991 shutdown_connections 2023-07-14T11:37:24.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c42d0 0x556ba34d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/346330991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c4be0 0x556ba34db6f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 -- 172.21.15.161:0/346330991 >> 172.21.15.161:0/346330991 conn(0x556ba34bada0 msgr2=0x556ba34e5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 -- 172.21.15.161:0/346330991 shutdown_connections 2023-07-14T11:37:24.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.169+0000 7ff5659e3b80 1 -- 172.21.15.161:0/346330991 wait complete. 2023-07-14T11:37:24.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 -- start start 2023-07-14T11:37:24.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 Processor -- start 2023-07-14T11:37:24.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba3532100 con 0x556ba34c4be0 2023-07-14T11:37:24.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba3532270 con 0x556ba34c42d0 2023-07-14T11:37:24.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba3532430 con 0x556ba34cfad0 2023-07-14T11:37:24.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.170+0000 7ff5659e3b80 10 client.0 _ll_register_callbacks cb 0x556ba33b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:24.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34d24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51460/0 (socket says 172.21.15.161:51460) 2023-07-14T11:37:24.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 -- 172.21.15.161:0/3809344343 learned_addr learned my addr 172.21.15.161:0/3809344343 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54affd700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54bfff700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 -- 172.21.15.161:0/3809344343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 msgr2=0x556ba34d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 -- 172.21.15.161:0/3809344343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 msgr2=0x556ba34d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54b7fe700 1 -- 172.21.15.161:0/3809344343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c012d70 con 0x556ba34c42d0 2023-07-14T11:37:24.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.171+0000 7ff54affd700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:24.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.172+0000 7ff54b7fe700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34d24f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff5440097c0 tx=0x7ff544004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c817634fce9a936 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.172+0000 7ff53affd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:37:24.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.172+0000 7ff53affd700 1 -- 172.21.15.161:0/3809344343 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544029d20 con 0x556ba34c42d0 2023-07-14T11:37:24.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.172+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff544004cf0 con 0x556ba34c42d0 2023-07-14T11:37:24.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.172+0000 7ff53affd700 1 -- 172.21.15.161:0/3809344343 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff544029150 con 0x556ba34c42d0 2023-07-14T11:37:24.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.172+0000 7ff53affd700 1 -- 172.21.15.161:0/3809344343 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff544029e80 con 0x556ba34c42d0 2023-07-14T11:37:24.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 10 client.27251 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:24.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x556ba322dc60 con 0x556ba34c42d0 2023-07-14T11:37:24.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff53affd700 1 -- 172.21.15.161:0/3809344343 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544004cf0 con 0x556ba34c42d0 2023-07-14T11:37:24.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff53affd700 1 -- 172.21.15.161:0/3809344343 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff544003840 con 0x556ba34c42d0 2023-07-14T11:37:24.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 10 client.27251 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:24.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 10 client.27251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:37:24.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 20 client.27251 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:24.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba3533b40 0x556ba3535f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 4 client.27251 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:37:24.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.173+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ba33d2050 con 0x556ba3533b40 2023-07-14T11:37:24.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.174+0000 7ff54affd700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba3533b40 0x556ba3535f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.175+0000 7ff53affd700 10 client.27251 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:24.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.175+0000 7ff54affd700 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba3533b40 0x556ba3535f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e202f0bdd60e493e server_cookie=56ba6757f4c546ff in_seq=0 out_seq=0 2023-07-14T11:37:24.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff53affd700 1 -- 172.21.15.161:0/3809344343 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ba33d2050 con 0x556ba3533b40 2023-07-14T11:37:24.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff53affd700 10 client.27251 handle_command_reply: tid=0 2023-07-14T11:37:24.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff5659e3b80 1 client.27251 shutdown 2023-07-14T11:37:24.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 msgr2=0x556ba34d24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34d24f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff5440097c0 tx=0x7ff544004b50 comp rx=0 tx=0).stop 2023-07-14T11:37:24.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 shutdown_connections 2023-07-14T11:37:24.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x556ba3533b40 0x556ba3535f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556ba34cfad0 0x556ba34d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff53affd700 0 client.27251 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:24.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.177+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ba34c42d0 0x556ba34d24f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.178+0000 7ff5659e3b80 1 --2- 172.21.15.161:0/3809344343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ba34c4be0 0x556ba34d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.178+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 >> 172.21.15.161:0/3809344343 conn(0x556ba34bada0 msgr2=0x556ba34db6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.178+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 shutdown_connections 2023-07-14T11:37:24.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.178+0000 7ff5659e3b80 1 -- 172.21.15.161:0/3809344343 wait complete. 2023-07-14T11:37:24.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.178+0000 7ff5659e3b80 20 client.27251 trim_cache size 0 max 16384 2023-07-14T11:37:24.183 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:37:24.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff52c0a13c0 msgr2=0x7ff52c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff52c0a13c0 0x7ff52c0a3870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009eb0 tx=0x7ff53c009310 comp rx=0 tx=0).stop 2023-07-14T11:37:24.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 msgr2=0x7ff54c10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c10d890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff53400b6c0 tx=0x7ff534004d00 comp rx=0 tx=0).stop 2023-07-14T11:37:24.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 shutdown_connections 2023-07-14T11:37:24.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff52c0a13c0 0x7ff52c0a3870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff54c101f70 0x7ff54c10d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff54c106fb0 0x7ff54c1a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 --2- 172.21.15.161:0/3313520828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff54c102910 0x7ff54c10d890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 >> 172.21.15.161:0/3313520828 conn(0x7ff54c0fd7e0 msgr2=0x7ff54c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 shutdown_connections 2023-07-14T11:37:24.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.179+0000 7ff551fff700 1 -- 172.21.15.161:0/3313520828 wait complete. 2023-07-14T11:37:24.198 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:23 smithi161 ceph-mon[95526]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 705 op/s 2023-07-14T11:37:24.198 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:23 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1481630560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:37:24.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:37:24.416 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:23 smithi175 ceph-mon[129081]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 705 op/s 2023-07-14T11:37:24.416 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:23 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1481630560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:37:24.461 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:23 smithi182 ceph-mon[129366]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 705 op/s 2023-07-14T11:37:24.461 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:23 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1481630560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:37:24.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.468+0000 7ff0529ff700 1 -- 172.21.15.161:0/3486577096 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 msgr2=0x7ff04c10b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.468+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3486577096 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c10b810 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff04400b0a0 tx=0x7ff04400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.469+0000 7ff0529ff700 1 -- 172.21.15.161:0/3486577096 shutdown_connections 2023-07-14T11:37:24.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.469+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3486577096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c1049e0 0x7ff04c104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.469+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3486577096 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c10b810 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.469+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3486577096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04c0fe9d0 0x7ff04c0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.469+0000 7ff0529ff700 1 -- 172.21.15.161:0/3486577096 >> 172.21.15.161:0/3486577096 conn(0x7ff04c0fa510 msgr2=0x7ff04c0fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.469+0000 7ff0529ff700 1 -- 172.21.15.161:0/3486577096 shutdown_connections 2023-07-14T11:37:24.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.469+0000 7ff0529ff700 1 -- 172.21.15.161:0/3486577096 wait complete. 2023-07-14T11:37:24.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.470+0000 7ff0529ff700 1 Processor -- start 2023-07-14T11:37:24.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.470+0000 7ff0529ff700 1 -- start start 2023-07-14T11:37:24.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff0529ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04c0fe9d0 0x7ff04c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff0529ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.472 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff0529ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c1049e0 0x7ff04c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff0529ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c19cd10 con 0x7ff04c1049e0 2023-07-14T11:37:24.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff0529ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c19ce80 con 0x7ff04c0fe9d0 2023-07-14T11:37:24.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff0529ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c19d040 con 0x7ff04c0ff380 2023-07-14T11:37:24.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff04b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04c0fe9d0 0x7ff04c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04c0fe9d0 0x7ff04c197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51478/0 (socket says 172.21.15.161:51478) 2023-07-14T11:37:24.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff04b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36294/0 (socket says 172.21.15.161:36294) 2023-07-14T11:37:24.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.471+0000 7ff04bfff700 1 -- 172.21.15.161:0/3536526465 learned_addr learned my addr 172.21.15.161:0/3536526465 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c1049e0 0x7ff04c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff04b7fe700 1 -- 172.21.15.161:0/3536526465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04c0fe9d0 msgr2=0x7ff04c197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04c0fe9d0 0x7ff04c197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff04b7fe700 1 -- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c1049e0 msgr2=0x7ff04c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c1049e0 0x7ff04c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff04b7fe700 1 -- 172.21.15.161:0/3536526465 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044009d00 con 0x7ff04c0ff380 2023-07-14T11:37:24.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c198270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff04000e9f0 tx=0x7ff04000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff0497fa700 1 -- 172.21.15.161:0/3536526465 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000eee0 con 0x7ff04c0ff380 2023-07-14T11:37:24.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff0497fa700 1 -- 172.21.15.161:0/3536526465 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04000cbd0 con 0x7ff04c0ff380 2023-07-14T11:37:24.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c19d320 con 0x7ff04c0ff380 2023-07-14T11:37:24.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff0497fa700 1 -- 172.21.15.161:0/3536526465 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040018680 con 0x7ff04c0ff380 2023-07-14T11:37:24.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.472+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c1a4b60 con 0x7ff04c0ff380 2023-07-14T11:37:24.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.476+0000 7ff0497fa700 1 -- 172.21.15.161:0/3536526465 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff04000cd40 con 0x7ff04c0ff380 2023-07-14T11:37:24.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.478+0000 7ff0662e2b80 1 Processor -- start 2023-07-14T11:37:24.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.478+0000 7ff0662e2b80 1 -- start start 2023-07-14T11:37:24.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.478+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3555eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.478+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.478+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e35636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.478+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e3453cb0 con 0x5620e35563f0 2023-07-14T11:37:24.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.479+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e34a6a40 con 0x5620e3555ae0 2023-07-14T11:37:24.479 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.479+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e349fed0 con 0x5620e35612e0 2023-07-14T11:37:24.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.479+0000 7ff0497fa700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff0340a1340 0x7ff0340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.479+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.479+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51164/0 (socket says 172.21.15.161:51164) 2023-07-14T11:37:24.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.479+0000 7ff04bfff700 1 -- 172.21.15.161:0/3912187510 learned_addr learned my addr 172.21.15.161:0/3912187510 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff0497fa700 1 -- 172.21.15.161:0/3536526465 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff040014070 con 0x7ff04c0ff380 2023-07-14T11:37:24.481 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3555eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff0340a1340 0x7ff0340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff04bfff700 1 -- 172.21.15.161:0/3912187510 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 msgr2=0x5620e35636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e35636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff04bfff700 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e35636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff04bfff700 1 -- 172.21.15.161:0/3912187510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 msgr2=0x5620e3555eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff04bfff700 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3555eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.482 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.480+0000 7ff04bfff700 1 -- 172.21.15.161:0/3912187510 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e32cdde0 con 0x5620e35563f0 2023-07-14T11:37:24.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.481+0000 7ff04bfff700 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009df0 tx=0x7ff03c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fc2e17487e1e14 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.481+0000 7ff0327fc700 1 -- 172.21.15.161:0/3912187510 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c004230 con 0x5620e35563f0 2023-07-14T11:37:24.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.481+0000 7ff0327fc700 1 -- 172.21.15.161:0/3912187510 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03c004390 con 0x5620e35563f0 2023-07-14T11:37:24.483 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.481+0000 7ff0327fc700 1 -- 172.21.15.161:0/3912187510 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c019860 con 0x5620e35563f0 2023-07-14T11:37:24.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3912187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 msgr2=0x5620e3560da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff0340a1340 0x7ff0340a37f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff044009fd0 tx=0x7ff04400b700 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009df0 tx=0x7ff03c011720 comp rx=0 tx=0).stop 2023-07-14T11:37:24.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3912187510 shutdown_connections 2023-07-14T11:37:24.487 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e35636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3555eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3912187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560da0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3912187510 >> 172.21.15.161:0/3912187510 conn(0x5620e3543da0 msgr2=0x5620e35496a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.482+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3912187510 shutdown_connections 2023-07-14T11:37:24.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.483+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3912187510 wait complete. 2023-07-14T11:37:24.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.484+0000 7ff0662e2b80 1 -- start start 2023-07-14T11:37:24.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.484+0000 7ff0662e2b80 1 Processor -- start 2023-07-14T11:37:24.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e3555ae0 0x5620e3560ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e355df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e35612e0 0x5620e355e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e343fac0 con 0x5620e35563f0 2023-07-14T11:37:24.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e343f7c0 con 0x5620e35612e0 2023-07-14T11:37:24.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e3323920 con 0x5620e3555ae0 2023-07-14T11:37:24.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff0662e2b80 10 client.0 _ll_register_callbacks cb 0x5620e343d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:24.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e3555ae0 0x5620e3560ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e3555ae0 0x5620e3560ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36312/0 (socket says 172.21.15.161:36312) 2023-07-14T11:37:24.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff04bfff700 1 -- 172.21.15.161:0/3536594062 learned_addr learned my addr 172.21.15.161:0/3536594062 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e355df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.485+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e35612e0 0x5620e355e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff04bfff700 1 -- 172.21.15.161:0/3536594062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e35612e0 msgr2=0x5620e355e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff04bfff700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e35612e0 0x5620e355e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff04bfff700 1 -- 172.21.15.161:0/3536594062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 msgr2=0x5620e355df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff04bfff700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e355df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff04bfff700 1 -- 172.21.15.161:0/3536594062 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c009a50 con 0x5620e3555ae0 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e355df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff04bfff700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e3555ae0 0x5620e3560ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff03c003930 tx=0x7ff03c003a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bffc183d7f494c65 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff02b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff02b7fe700 1 -- 172.21.15.161:0/3536594062 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c005460 con 0x5620e3555ae0 2023-07-14T11:37:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620e33f94f0 con 0x5620e3555ae0 2023-07-14T11:37:24.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.486+0000 7ff02b7fe700 1 -- 172.21.15.161:0/3536594062 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff03c0055c0 con 0x5620e3555ae0 2023-07-14T11:37:24.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.487+0000 7ff02b7fe700 1 -- 172.21.15.161:0/3536594062 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03c004500 con 0x5620e3555ae0 2023-07-14T11:37:24.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.487+0000 7ff0662e2b80 10 client.27301 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:24.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.487+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5620e32b6c60 con 0x5620e3555ae0 2023-07-14T11:37:24.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.487+0000 7ff02b7fe700 1 -- 172.21.15.161:0/3536594062 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620e33f94f0 con 0x5620e3555ae0 2023-07-14T11:37:24.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff02b7fe700 1 -- 172.21.15.161:0/3536594062 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff03c021040 con 0x5620e3555ae0 2023-07-14T11:37:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff0662e2b80 10 client.27301 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff0662e2b80 10 client.27301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:37:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff0662e2b80 20 client.27301 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e3581b70 0x5620e3583f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff0662e2b80 4 client.27301 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620e3498eb0 con 0x5620e3581b70 2023-07-14T11:37:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e3581b70 0x5620e3583f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff02b7fe700 10 client.27301 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.488+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e3581b70 0x5620e3583f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adc2eb44f8be8d2 server_cookie=d0b1b1d8416298a4 in_seq=0 out_seq=0 2023-07-14T11:37:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.490+0000 7ff02b7fe700 1 -- 172.21.15.161:0/3536594062 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620e3498eb0 con 0x5620e3581b70 2023-07-14T11:37:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.490+0000 7ff02b7fe700 10 client.27301 handle_command_reply: tid=0 2023-07-14T11:37:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.490+0000 7ff0662e2b80 1 client.27301 shutdown 2023-07-14T11:37:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e3555ae0 msgr2=0x5620e3560ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e3555ae0 0x5620e3560ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff03c003930 tx=0x7ff03c003a10 comp rx=0 tx=0).stop 2023-07-14T11:37:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 shutdown_connections 2023-07-14T11:37:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e3581b70 0x5620e3583f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e35612e0 0x5620e355e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e3555ae0 0x5620e3560ce0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3536594062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e355df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 >> 172.21.15.161:0/3536594062 conn(0x5620e3543da0 msgr2=0x5620e3549230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff02b7fe700 0 client.27301 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 shutdown_connections 2023-07-14T11:37:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3536594062 wait complete. 2023-07-14T11:37:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.491+0000 7ff0662e2b80 20 client.27301 trim_cache size 0 max 16384 2023-07-14T11:37:24.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.510+0000 7ff0662e2b80 1 Processor -- start 2023-07-14T11:37:24.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff0662e2b80 1 -- start start 2023-07-14T11:37:24.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3559240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e357e590 con 0x5620e35563f0 2023-07-14T11:37:24.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e3584290 con 0x5620e3555ae0 2023-07-14T11:37:24.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e355eb70 con 0x5620e35612e0 2023-07-14T11:37:24.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff050f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3559240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51502/0 (socket says 172.21.15.161:51502) 2023-07-14T11:37:24.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff04bfff700 1 -- 172.21.15.161:0/3582281972 learned_addr learned my addr 172.21.15.161:0/3582281972 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.511+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff050f9c700 1 -- 172.21.15.161:0/3582281972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 msgr2=0x5620e3567d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff050f9c700 1 -- 172.21.15.161:0/3582281972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 msgr2=0x5620e3560810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff050f9c700 1 -- 172.21.15.161:0/3582281972 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e32f4250 con 0x5620e35612e0 2023-07-14T11:37:24.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff04bfff700 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:24.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff050f9c700 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3559240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff044009040 tx=0x7ff0440096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d199e66e58d1d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff032ffd700 1 -- 172.21.15.161:0/3582281972 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400e040 con 0x5620e35612e0 2023-07-14T11:37:24.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff032ffd700 1 -- 172.21.15.161:0/3582281972 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff044004e30 con 0x5620e35612e0 2023-07-14T11:37:24.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.512+0000 7ff032ffd700 1 -- 172.21.15.161:0/3582281972 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04402bb70 con 0x5620e35612e0 2023-07-14T11:37:24.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.513+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3582281972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 msgr2=0x5620e3559240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.513+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3559240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff044009040 tx=0x7ff0440096a0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3582281972 shutdown_connections 2023-07-14T11:37:24.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3559240 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/3582281972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3560810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3582281972 >> 172.21.15.161:0/3582281972 conn(0x5620e3543da0 msgr2=0x5620e3583b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3582281972 shutdown_connections 2023-07-14T11:37:24.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 -- 172.21.15.161:0/3582281972 wait complete. 2023-07-14T11:37:24.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.514+0000 7ff0662e2b80 1 -- start start 2023-07-14T11:37:24.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 1 Processor -- start 2023-07-14T11:37:24.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3567f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3563f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e3564690 con 0x5620e35563f0 2023-07-14T11:37:24.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e3564800 con 0x5620e3555ae0 2023-07-14T11:37:24.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e35649c0 con 0x5620e35612e0 2023-07-14T11:37:24.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff0662e2b80 10 client.0 _ll_register_callbacks cb 0x7ff03c009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:24.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.515+0000 7ff04b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3567f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff050f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3563f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff04b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3567f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51190/0 (socket says 172.21.15.161:51190) 2023-07-14T11:37:24.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff04b7fe700 1 -- 172.21.15.161:0/670958006 learned_addr learned my addr 172.21.15.161:0/670958006 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff04b7fe700 1 -- 172.21.15.161:0/670958006 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 msgr2=0x5620e3563f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3563f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff04b7fe700 1 -- 172.21.15.161:0/670958006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 msgr2=0x5620e3567a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.516+0000 7ff04b7fe700 1 -- 172.21.15.161:0/670958006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620e32b6c60 con 0x5620e35563f0 2023-07-14T11:37:24.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.517+0000 7ff04b7fe700 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3567f40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff040009800 tx=0x7ff04009df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a41bd53daf5586 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.517+0000 7ff02b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:37:24.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.517+0000 7ff02b7fe700 1 -- 172.21.15.161:0/670958006 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400caa60 con 0x5620e35563f0 2023-07-14T11:37:24.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.517+0000 7ff02b7fe700 1 -- 172.21.15.161:0/670958006 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff04009a1c0 con 0x5620e35563f0 2023-07-14T11:37:24.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.518+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff03c004390 con 0x5620e35563f0 2023-07-14T11:37:24.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.518+0000 7ff02b7fe700 1 -- 172.21.15.161:0/670958006 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff04009a380 con 0x5620e35563f0 2023-07-14T11:37:24.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.518+0000 7ff0662e2b80 10 client.17232 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:24.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.518+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7ff04c10c110 con 0x5620e35563f0 2023-07-14T11:37:24.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.518+0000 7ff02b7fe700 1 -- 172.21.15.161:0/670958006 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03c004390 con 0x5620e35563f0 2023-07-14T11:37:24.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff02b7fe700 1 -- 172.21.15.161:0/670958006 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff040094b70 con 0x5620e35563f0 2023-07-14T11:37:24.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff0662e2b80 10 client.17232 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:24.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff0662e2b80 10 client.17232 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:37:24.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff0662e2b80 20 client.17232 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:24.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e35d6d80 0x5620e35d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff0662e2b80 4 client.17232 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:37:24.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620e345b050 con 0x5620e35d6d80 2023-07-14T11:37:24.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.519+0000 7ff04bfff700 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e35d6d80 0x5620e35d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.520+0000 7ff04bfff700 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e35d6d80 0x5620e35d9190 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f2d276c7e3c4945 server_cookie=9d27d400cf023809 in_seq=0 out_seq=0 2023-07-14T11:37:24.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.520+0000 7ff02b7fe700 10 client.17232 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:24.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.522+0000 7ff02b7fe700 1 -- 172.21.15.161:0/670958006 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620e345b050 con 0x5620e35d6d80 2023-07-14T11:37:24.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.522+0000 7ff02b7fe700 10 client.17232 handle_command_reply: tid=0 2023-07-14T11:37:24.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.522+0000 7ff0662e2b80 1 client.17232 shutdown 2023-07-14T11:37:24.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.522+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 msgr2=0x5620e3567f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.522+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3567f40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff040009800 tx=0x7ff04009df90 comp rx=0 tx=0).stop 2023-07-14T11:37:24.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 shutdown_connections 2023-07-14T11:37:24.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x5620e35d6d80 0x5620e35d9190 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5620e35612e0 0x5620e3563f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620e3555ae0 0x5620e3567a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 --2- 172.21.15.161:0/670958006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620e35563f0 0x5620e3567f40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 >> 172.21.15.161:0/670958006 conn(0x5620e3543da0 msgr2=0x5620e3559240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff02b7fe700 0 client.17232 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:24.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 shutdown_connections 2023-07-14T11:37:24.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 1 -- 172.21.15.161:0/670958006 wait complete. 2023-07-14T11:37:24.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.523+0000 7ff0662e2b80 20 client.17232 trim_cache size 0 max 16384 2023-07-14T11:37:24.528 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:37:24.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.524+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff0340a1340 msgr2=0x7ff0340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.524+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff0340a1340 0x7ff0340a37f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff044009fd0 tx=0x7ff04400b700 comp rx=0 tx=0).stop 2023-07-14T11:37:24.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.524+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 msgr2=0x7ff04c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.524+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c198270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff04000e9f0 tx=0x7ff04000ed00 comp rx=0 tx=0).stop 2023-07-14T11:37:24.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 shutdown_connections 2023-07-14T11:37:24.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff0340a1340 0x7ff0340a37f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c1049e0 0x7ff04c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04c0fe9d0 0x7ff04c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 --2- 172.21.15.161:0/3536526465 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff04c0ff380 0x7ff04c198270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 >> 172.21.15.161:0/3536526465 conn(0x7ff04c0fa510 msgr2=0x7ff04c0688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 shutdown_connections 2023-07-14T11:37:24.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.525+0000 7ff0529ff700 1 -- 172.21.15.161:0/3536526465 wait complete. 2023-07-14T11:37:24.547 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:37:24.581 INFO:tasks.workunit.client.0.smithi161.stdout: 10 75510 34.00 MB/sec execute 6 sec latency 5940.190 ms 2023-07-14T11:37:24.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 -- 172.21.15.161:0/4073021646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810cb50 msgr2=0x7f72f8106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 --2- 172.21.15.161:0/4073021646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810cb50 0x7f72f8106340 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009920 tx=0x7f72e4009c30 comp rx=0 tx=0).stop 2023-07-14T11:37:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 -- 172.21.15.161:0/4073021646 shutdown_connections 2023-07-14T11:37:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 --2- 172.21.15.161:0/4073021646 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 0x7f72f8112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 --2- 172.21.15.161:0/4073021646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f810c240 0x7f72f810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 --2- 172.21.15.161:0/4073021646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810cb50 0x7f72f8106340 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 -- 172.21.15.161:0/4073021646 >> 172.21.15.161:0/4073021646 conn(0x7f72f80759e0 msgr2=0x7f72f8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.799+0000 7f72fefff700 1 -- 172.21.15.161:0/4073021646 shutdown_connections 2023-07-14T11:37:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.801+0000 7f72fefff700 1 -- 172.21.15.161:0/4073021646 wait complete. 2023-07-14T11:37:24.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.802+0000 7f72fefff700 1 Processor -- start 2023-07-14T11:37:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.802+0000 7f72fefff700 1 -- start start 2023-07-14T11:37:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fefff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 0x7f72f819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fefff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810c240 0x7f72f819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fefff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f810cb50 0x7f72f81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fefff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f81a13d0 con 0x7f72f810c240 2023-07-14T11:37:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fefff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f81a1540 con 0x7f72f810cb50 2023-07-14T11:37:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fefff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f81a1700 con 0x7f72f8106a70 2023-07-14T11:37:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fcd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 0x7f72f819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fcd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 0x7f72f819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36352/0 (socket says 172.21.15.161:36352) 2023-07-14T11:37:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3726121352 learned_addr learned my addr 172.21.15.161:0/3726121352 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72fd59c700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f810cb50 0x7f72f81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.803+0000 7f72f7fff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810c240 0x7f72f819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3726121352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f810cb50 msgr2=0x7f72f81a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f810cb50 0x7f72f81a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810c240 msgr2=0x7f72f819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810c240 0x7f72f819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3726121352 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4009520 con 0x7f72f8106a70 2023-07-14T11:37:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72f7fff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810c240 0x7f72f819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:37:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 0x7f72f819c3f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00efc0 tx=0x7f72ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72f5ffb700 1 -- 172.21.15.161:0/3726121352 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec018070 con 0x7f72f8106a70 2023-07-14T11:37:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f81a1980 con 0x7f72f8106a70 2023-07-14T11:37:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72f5ffb700 1 -- 172.21.15.161:0/3726121352 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72ec00f040 con 0x7f72f8106a70 2023-07-14T11:37:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72f5ffb700 1 -- 172.21.15.161:0/3726121352 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec004310 con 0x7f72f8106a70 2023-07-14T11:37:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.804+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f81a9220 con 0x7f72f8106a70 2023-07-14T11:37:24.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.808+0000 7f72f5ffb700 1 -- 172.21.15.161:0/3726121352 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f72ec00ca70 con 0x7f72f8106a70 2023-07-14T11:37:24.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 Processor -- start 2023-07-14T11:37:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 -- start start 2023-07-14T11:37:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da562d0 0x55e59da566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da61ad0 0x55e59da63eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59d95ccb0 con 0x55e59da562d0 2023-07-14T11:37:24.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59d9afa40 con 0x55e59da56be0 2023-07-14T11:37:24.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.810+0000 7f73128d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59d9a8ed0 con 0x55e59da61ad0 2023-07-14T11:37:24.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da562d0 0x55e59da566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da562d0 0x55e59da566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51210/0 (socket says 172.21.15.161:51210) 2023-07-14T11:37:24.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 -- 172.21.15.161:0/1940656509 learned_addr learned my addr 172.21.15.161:0/1940656509 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72f5ffb700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72e80a12c0 0x7f72e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72f7fff700 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da61ad0 0x55e59da63eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 -- 172.21.15.161:0/1940656509 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da61ad0 msgr2=0x55e59da63eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da61ad0 0x55e59da63eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 -- 172.21.15.161:0/1940656509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 msgr2=0x55e59da61590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da61590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72fd59c700 1 -- 172.21.15.161:0/1940656509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59d7d6de0 con 0x55e59da562d0 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.811+0000 7f72f5ffb700 1 -- 172.21.15.161:0/3726121352 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f72ec0ccf40 con 0x7f72f8106a70 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.812+0000 7f72fd59c700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72e80a12c0 0x7f72e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.812+0000 7f72fd59c700 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da562d0 0x55e59da566a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009e70 tx=0x7f72f0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd556cb53155c07d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.812+0000 7f72e2ffd700 1 -- 172.21.15.161:0/1940656509 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f001bc00 con 0x55e59da562d0 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.812+0000 7f72fd59c700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72e80a12c0 0x7f72e80a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f72f0023000 tx=0x7f72f0024000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.812+0000 7f72e2ffd700 1 -- 172.21.15.161:0/1940656509 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72f001bd60 con 0x55e59da562d0 2023-07-14T11:37:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.812+0000 7f72e2ffd700 1 -- 172.21.15.161:0/1940656509 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f000f3c0 con 0x55e59da562d0 2023-07-14T11:37:24.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 -- 172.21.15.161:0/1940656509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da562d0 msgr2=0x55e59da566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da562d0 0x55e59da566a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009e70 tx=0x7f72f0011720 comp rx=0 tx=0).stop 2023-07-14T11:37:24.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 -- 172.21.15.161:0/1940656509 shutdown_connections 2023-07-14T11:37:24.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da562d0 0x55e59da566a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da61ad0 0x55e59da63eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 --2- 172.21.15.161:0/1940656509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da61590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 -- 172.21.15.161:0/1940656509 >> 172.21.15.161:0/1940656509 conn(0x55e59da4cda0 msgr2=0x55e59da526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 -- 172.21.15.161:0/1940656509 shutdown_connections 2023-07-14T11:37:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.813+0000 7f73128d0b80 1 -- 172.21.15.161:0/1940656509 wait complete. 2023-07-14T11:37:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.814+0000 7f73128d0b80 1 -- start start 2023-07-14T11:37:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.814+0000 7f73128d0b80 1 Processor -- start 2023-07-14T11:37:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.815+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.815+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 0x55e59da5e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.815+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.815+0000 7f73128d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59d948ac0 con 0x55e59da56be0 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.815+0000 7f73128d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59d9487c0 con 0x55e59da61ad0 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.815+0000 7f73128d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59d82c920 con 0x55e59da562d0 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.815+0000 7f73128d0b80 10 client.0 _ll_register_callbacks cb 0x55e59d946760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.816+0000 7f72fd59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.816+0000 7f72fd59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51552/0 (socket says 172.21.15.161:51552) 2023-07-14T11:37:24.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.816+0000 7f72fd59c700 1 -- 172.21.15.161:0/470731566 learned_addr learned my addr 172.21.15.161:0/470731566 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.816+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.816+0000 7f72fd59c700 1 -- 172.21.15.161:0/470731566 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 msgr2=0x55e59da614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.816+0000 7f72fd59c700 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.816+0000 7f72fd59c700 1 -- 172.21.15.161:0/470731566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 msgr2=0x55e59da5e7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.817+0000 7f72fd59c700 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 0x55e59da5e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.817+0000 7f72fd59c700 1 -- 172.21.15.161:0/470731566 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f00099d0 con 0x55e59da61ad0 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.817+0000 7f72fd59c700 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5ee70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009e40 tx=0x7f72f001bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c294ccd263d6862 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.817+0000 7f72d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.817+0000 7f72d7fff700 1 -- 172.21.15.161:0/470731566 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f002c070 con 0x55e59da61ad0 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.817+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e59d9024f0 con 0x55e59da61ad0 2023-07-14T11:37:24.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.817+0000 7f72d7fff700 1 -- 172.21.15.161:0/470731566 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f72f0026650 con 0x55e59da61ad0 2023-07-14T11:37:24.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f72d7fff700 1 -- 172.21.15.161:0/470731566 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72f0026810 con 0x55e59da61ad0 2023-07-14T11:37:24.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f73128d0b80 10 client.27278 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:24.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e59d7bfc60 con 0x55e59da61ad0 2023-07-14T11:37:24.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f72d7fff700 1 -- 172.21.15.161:0/470731566 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e59d9024f0 con 0x55e59da61ad0 2023-07-14T11:37:24.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f72d7fff700 1 -- 172.21.15.161:0/470731566 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f72f0030510 con 0x55e59da61ad0 2023-07-14T11:37:24.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f73128d0b80 10 client.27278 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:24.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f73128d0b80 10 client.27278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:24.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.818+0000 7f73128d0b80 20 client.27278 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:24.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.819+0000 7f73128d0b80 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59da713a0 0x55e59da737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.819+0000 7f73128d0b80 4 client.27278 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:24.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.819+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e59d9a1eb0 con 0x55e59da713a0 2023-07-14T11:37:24.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.819+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59da713a0 0x55e59da737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.819+0000 7f72d7fff700 10 client.27278 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:24.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.819+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59da713a0 0x55e59da737b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35da1dabae3c5242 server_cookie=a14848e61c35771c in_seq=0 out_seq=0 2023-07-14T11:37:24.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f72d7fff700 1 -- 172.21.15.161:0/470731566 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e59d9a1eb0 con 0x55e59da713a0 2023-07-14T11:37:24.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f72d7fff700 10 client.27278 handle_command_reply: tid=0 2023-07-14T11:37:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 client.27278 shutdown 2023-07-14T11:37:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 msgr2=0x55e59da5ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5ee70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009e40 tx=0x7f72f001bed0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 shutdown_connections 2023-07-14T11:37:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59da713a0 0x55e59da737b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5ee70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 --2- 172.21.15.161:0/470731566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 0x55e59da5e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f72d7fff700 0 client.27278 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:24.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 >> 172.21.15.161:0/470731566 conn(0x55e59da4cda0 msgr2=0x55e59da52230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 shutdown_connections 2023-07-14T11:37:24.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.823+0000 7f73128d0b80 1 -- 172.21.15.161:0/470731566 wait complete. 2023-07-14T11:37:24.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.824+0000 7f73128d0b80 20 client.27278 trim_cache size 0 max 16384 2023-07-14T11:37:24.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 Processor -- start 2023-07-14T11:37:24.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 -- start start 2023-07-14T11:37:24.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da61080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 0x55e59da5df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59da57d60 con 0x55e59da56be0 2023-07-14T11:37:24.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59da6ddc0 con 0x55e59da61ad0 2023-07-14T11:37:24.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.839+0000 7f73128d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59da5fd10 con 0x55e59da562d0 2023-07-14T11:37:24.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fcd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da61080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fcd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da61080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36388/0 (socket says 172.21.15.161:36388) 2023-07-14T11:37:24.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 0x55e59da5df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fcd9b700 1 -- 172.21.15.161:0/468889277 learned_addr learned my addr 172.21.15.161:0/468889277 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fd59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fd59c700 1 -- 172.21.15.161:0/468889277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 msgr2=0x55e59da61080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fd59c700 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da61080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fd59c700 1 -- 172.21.15.161:0/468889277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 msgr2=0x55e59da5df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fd59c700 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 0x55e59da5df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fd59c700 1 -- 172.21.15.161:0/468889277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59d7fd250 con 0x55e59da61ad0 2023-07-14T11:37:24.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.840+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da61080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:24.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.841+0000 7f72fd59c700 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5e4d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f72f0030840 tx=0x7f72f0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3525af10a57ac615 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.841+0000 7f72e37fe700 1 -- 172.21.15.161:0/468889277 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f002c070 con 0x55e59da61ad0 2023-07-14T11:37:24.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.841+0000 7f72e37fe700 1 -- 172.21.15.161:0/468889277 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72f0030510 con 0x55e59da61ad0 2023-07-14T11:37:24.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.841+0000 7f72e37fe700 1 -- 172.21.15.161:0/468889277 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f000fb80 con 0x55e59da61ad0 2023-07-14T11:37:24.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 -- 172.21.15.161:0/468889277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 msgr2=0x55e59da5e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5e4d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f72f0030840 tx=0x7f72f0011720 comp rx=0 tx=0).stop 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 -- 172.21.15.161:0/468889277 shutdown_connections 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da61ad0 0x55e59da5e4d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da61080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 --2- 172.21.15.161:0/468889277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da56be0 0x55e59da5df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 -- 172.21.15.161:0/468889277 >> 172.21.15.161:0/468889277 conn(0x55e59da4cda0 msgr2=0x55e59da59b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 -- 172.21.15.161:0/468889277 shutdown_connections 2023-07-14T11:37:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 -- 172.21.15.161:0/468889277 wait complete. 2023-07-14T11:37:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.842+0000 7f73128d0b80 1 -- start start 2023-07-14T11:37:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.843+0000 7f73128d0b80 1 Processor -- start 2023-07-14T11:37:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.843+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da67d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.843+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.843+0000 7f73128d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da61ad0 0x55e59da66490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.843+0000 7f73128d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59da66a90 con 0x55e59da61ad0 2023-07-14T11:37:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.843+0000 7f73128d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59da66c00 con 0x55e59da56be0 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.843+0000 7f73128d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59da66dc0 con 0x55e59da562d0 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f73128d0b80 10 client.0 _ll_register_callbacks cb 0x7f72ec0960e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da67d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da67d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36392/0 (socket says 172.21.15.161:36392) 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3637718991 learned_addr learned my addr 172.21.15.161:0/3637718991 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fd59c700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da61ad0 0x55e59da66490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72f7fff700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 msgr2=0x55e59da682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3637718991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da61ad0 msgr2=0x55e59da66490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da61ad0 0x55e59da66490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fcd9b700 1 -- 172.21.15.161:0/3637718991 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f81a9220 con 0x55e59da562d0 2023-07-14T11:37:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72f7fff700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:37:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.844+0000 7f72fd59c700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da61ad0 0x55e59da66490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f72fcd9b700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da67d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0d0650 tx=0x7f72ec096190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1546327b1254720 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f72d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f72d7fff700 1 -- 172.21.15.161:0/3637718991 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec099860 con 0x55e59da562d0 2023-07-14T11:37:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72f001bd60 con 0x55e59da562d0 2023-07-14T11:37:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f72d7fff700 1 -- 172.21.15.161:0/3637718991 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f72ec099d20 con 0x55e59da562d0 2023-07-14T11:37:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f72d7fff700 1 -- 172.21.15.161:0/3637718991 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72ec099680 con 0x55e59da562d0 2023-07-14T11:37:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 10 client.27325 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e59d7bfc60 con 0x55e59da562d0 2023-07-14T11:37:24.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f72d7fff700 1 -- 172.21.15.161:0/3637718991 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72ec063080 con 0x55e59da562d0 2023-07-14T11:37:24.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f72d7fff700 1 -- 172.21.15.161:0/3637718991 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f72ec0b2210 con 0x55e59da562d0 2023-07-14T11:37:24.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 10 client.27325 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:24.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 10 client.27325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:24.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 20 client.27325 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:24.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59dac61f0 0x55e59dac8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:24.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 4 client.27325 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:37:24.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.845+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e59d964050 con 0x55e59dac61f0 2023-07-14T11:37:24.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.846+0000 7f72f7fff700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59dac61f0 0x55e59dac8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:24.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.846+0000 7f72d7fff700 10 client.27325 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:24.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.846+0000 7f72f7fff700 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59dac61f0 0x55e59dac8600 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254f75f5f255a4ae server_cookie=a066ac2867c8ed48 in_seq=0 out_seq=0 2023-07-14T11:37:24.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f72d7fff700 1 -- 172.21.15.161:0/3637718991 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e59d964050 con 0x55e59dac61f0 2023-07-14T11:37:24.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f72d7fff700 10 client.27325 handle_command_reply: tid=0 2023-07-14T11:37:24.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 client.27325 shutdown 2023-07-14T11:37:24.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 msgr2=0x55e59da67d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da67d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0d0650 tx=0x7f72ec096190 comp rx=0 tx=0).stop 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 shutdown_connections 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e59dac61f0 0x55e59dac8600 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e59da61ad0 0x55e59da66490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e59da562d0 0x55e59da67d90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 --2- 172.21.15.161:0/3637718991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e59da56be0 0x55e59da682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f72d7fff700 0 client.27325 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 >> 172.21.15.161:0/3637718991 conn(0x55e59da4cda0 msgr2=0x55e59da518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 shutdown_connections 2023-07-14T11:37:24.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.850+0000 7f73128d0b80 1 -- 172.21.15.161:0/3637718991 wait complete. 2023-07-14T11:37:24.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.851+0000 7f73128d0b80 20 client.27325 trim_cache size 0 max 16384 2023-07-14T11:37:24.922 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:37:24.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.851+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72e80a12c0 msgr2=0x7f72e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.851+0000 7f72fefff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72e80a12c0 0x7f72e80a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f72f0023000 tx=0x7f72f0024000 comp rx=0 tx=0).stop 2023-07-14T11:37:24.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.851+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 msgr2=0x7f72f819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:24.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.851+0000 7f72fefff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 0x7f72f819c3f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00efc0 tx=0x7f72ec00c3a0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 shutdown_connections 2023-07-14T11:37:24.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72e80a12c0 0x7f72e80a3770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f810cb50 0x7f72f81a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72f8106a70 0x7f72f819c3f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 --2- 172.21.15.161:0/3726121352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72f810c240 0x7f72f819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:24.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 >> 172.21.15.161:0/3726121352 conn(0x7f72f80759e0 msgr2=0x7f72f80771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:24.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 shutdown_connections 2023-07-14T11:37:24.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:24.852+0000 7f72fefff700 1 -- 172.21.15.161:0/3726121352 wait complete. 2023-07-14T11:37:25.581 INFO:tasks.workunit.client.0.smithi161.stdout: 10 76024 34.00 MB/sec execute 7 sec latency 6864.018 ms 2023-07-14T11:37:25.925 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:37:25.926 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:37:26.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 -- 172.21.15.161:0/3257908471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 msgr2=0x7f8c00104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 --2- 172.21.15.161:0/3257908471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 0x7f8c00104dc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009990 tx=0x7f8bf0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 -- 172.21.15.161:0/3257908471 shutdown_connections 2023-07-14T11:37:26.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 --2- 172.21.15.161:0/3257908471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 0x7f8c00104dc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 --2- 172.21.15.161:0/3257908471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c000ff390 0x7f8c00068860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 --2- 172.21.15.161:0/3257908471 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 0x7f8c000fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 -- 172.21.15.161:0/3257908471 >> 172.21.15.161:0/3257908471 conn(0x7f8c000fa500 msgr2=0x7f8c000fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 -- 172.21.15.161:0/3257908471 shutdown_connections 2023-07-14T11:37:26.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.218+0000 7f8c04dff700 1 -- 172.21.15.161:0/3257908471 wait complete. 2023-07-14T11:37:26.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 Processor -- start 2023-07-14T11:37:26.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 -- start start 2023-07-14T11:37:26.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 0x7f8c00197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c000ff390 0x7f8c00198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 0x7f8c0019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0019ce30 con 0x7f8c001049f0 2023-07-14T11:37:26.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0019cfa0 con 0x7f8c000ff390 2023-07-14T11:37:26.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.219+0000 7f8c04dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0019d160 con 0x7f8c000fe9e0 2023-07-14T11:37:26.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 0x7f8c00197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 0x7f8c00197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36402/0 (socket says 172.21.15.161:36402) 2023-07-14T11:37:26.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1813805923 learned_addr learned my addr 172.21.15.161:0/1813805923 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfdd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c000ff390 0x7f8c00198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 0x7f8c0019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1813805923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c000ff390 msgr2=0x7f8c00198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c000ff390 0x7f8c00198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 msgr2=0x7f8c0019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 0x7f8c0019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1813805923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00095f0 con 0x7f8c000fe9e0 2023-07-14T11:37:26.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.220+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 0x7f8c00197e50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0004730 tx=0x7f8bf0004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.221+0000 7f8bef7fe700 1 -- 172.21.15.161:0/1813805923 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf001d070 con 0x7f8c000fe9e0 2023-07-14T11:37:26.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.221+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0019d3e0 con 0x7f8c000fe9e0 2023-07-14T11:37:26.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.221+0000 7f8bef7fe700 1 -- 172.21.15.161:0/1813805923 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf00047d0 con 0x7f8c000fe9e0 2023-07-14T11:37:26.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.221+0000 7f8bef7fe700 1 -- 172.21.15.161:0/1813805923 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00053b0 con 0x7f8c000fe9e0 2023-07-14T11:37:26.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.221+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c001a4c80 con 0x7f8c000fe9e0 2023-07-14T11:37:26.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.224+0000 7f8bef7fe700 1 -- 172.21.15.161:0/1813805923 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8bf0004940 con 0x7f8c000fe9e0 2023-07-14T11:37:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 Processor -- start 2023-07-14T11:37:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 -- start start 2023-07-14T11:37:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8bef7fe700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8be80a1340 0x7f8be80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63e350 0x558d5e63e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63ec60 0x558d5e649610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8bef7fe700 1 -- 172.21.15.161:0/1813805923 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8bf00ce1e0 con 0x7f8c000fe9e0 2023-07-14T11:37:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e649b50 0x558d5e64bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e544d30 con 0x558d5e63e350 2023-07-14T11:37:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e597ac0 con 0x558d5e63ec60 2023-07-14T11:37:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e590f50 con 0x558d5e649b50 2023-07-14T11:37:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.226+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8be80a1340 0x7f8be80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63ec60 0x558d5e649610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfed9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e649b50 0x558d5e64bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63ec60 0x558d5e649610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51592/0 (socket says 172.21.15.161:51592) 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/713795300 learned_addr learned my addr 172.21.15.161:0/713795300 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63e350 0x558d5e63e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/713795300 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e649b50 msgr2=0x558d5e64bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e649b50 0x558d5e64bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/713795300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63e350 msgr2=0x558d5e63e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63e350 0x558d5e63e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/713795300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5e3bee60 con 0x558d5e63ec60 2023-07-14T11:37:26.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8be80a1340 0x7f8be80a37f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009900 tx=0x7f8bf8009ea0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.227+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63ec60 0x558d5e649610 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009d20 tx=0x7f8bf4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693b4453bf1a2ed2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.228+0000 7f8becff9700 1 -- 172.21.15.161:0/713795300 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4019630 con 0x558d5e63ec60 2023-07-14T11:37:26.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.228+0000 7f8becff9700 1 -- 172.21.15.161:0/713795300 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf4019c50 con 0x558d5e63ec60 2023-07-14T11:37:26.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.228+0000 7f8becff9700 1 -- 172.21.15.161:0/713795300 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400ec90 con 0x558d5e63ec60 2023-07-14T11:37:26.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 -- 172.21.15.161:0/713795300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63ec60 msgr2=0x558d5e649610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63ec60 0x558d5e649610 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009d20 tx=0x7f8bf4011720 comp rx=0 tx=0).stop 2023-07-14T11:37:26.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 -- 172.21.15.161:0/713795300 shutdown_connections 2023-07-14T11:37:26.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63e350 0x558d5e63e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e649b50 0x558d5e64bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/713795300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63ec60 0x558d5e649610 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 -- 172.21.15.161:0/713795300 >> 172.21.15.161:0/713795300 conn(0x558d5e634e20 msgr2=0x558d5e63a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 -- 172.21.15.161:0/713795300 shutdown_connections 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.229+0000 7f8c187e0b80 1 -- 172.21.15.161:0/713795300 wait complete. 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 -- start start 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 Processor -- start 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63e350 0x558d5e6495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63ec60 0x558d5e6468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e649b50 0x558d5e646fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e530b40 con 0x558d5e63ec60 2023-07-14T11:37:26.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e530840 con 0x558d5e649b50 2023-07-14T11:37:26.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.230+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e4149a0 con 0x558d5e63e350 2023-07-14T11:37:26.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8c187e0b80 10 client.0 _ll_register_callbacks cb 0x558d5e52e7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:26.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63e350 0x558d5e6495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63e350 0x558d5e6495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36412/0 (socket says 172.21.15.161:36412) 2023-07-14T11:37:26.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1894499507 learned_addr learned my addr 172.21.15.161:0/1894499507 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e649b50 0x558d5e646fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e649b50 msgr2=0x558d5e646fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e649b50 0x558d5e646fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1894499507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63ec60 msgr2=0x558d5e6468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63ec60 0x558d5e6468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 -- 172.21.15.161:0/1894499507 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf4009980 con 0x558d5e63e350 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63e350 0x558d5e6495b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00095c0 tx=0x7f8bf009f0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fcb28652666001b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.231+0000 7f8be5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8be5ffb700 1 -- 172.21.15.161:0/1894499507 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf009b3e0 con 0x558d5e63e350 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8be5ffb700 1 -- 172.21.15.161:0/1894499507 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8bf009a470 con 0x558d5e63e350 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d5e4ea570 con 0x558d5e63e350 2023-07-14T11:37:26.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8be5ffb700 1 -- 172.21.15.161:0/1894499507 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf0005760 con 0x558d5e63e350 2023-07-14T11:37:26.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8c187e0b80 10 client.27337 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:26.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558d5e3a7ce0 con 0x558d5e63e350 2023-07-14T11:37:26.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8be5ffb700 1 -- 172.21.15.161:0/1894499507 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d5e4ea570 con 0x558d5e63e350 2023-07-14T11:37:26.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8be5ffb700 1 -- 172.21.15.161:0/1894499507 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8bf0095b90 con 0x558d5e63e350 2023-07-14T11:37:26.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8c187e0b80 10 client.27337 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:26.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8c187e0b80 10 client.27337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:26.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.232+0000 7f8c187e0b80 20 client.27337 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:26.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.233+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e659080 0x558d5e65b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.233+0000 7f8c187e0b80 4 client.27337 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:26.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.233+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d5e589f30 con 0x558d5e659080 2023-07-14T11:37:26.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.233+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e659080 0x558d5e65b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.233+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e659080 0x558d5e65b490 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2428fb576d3dd4 server_cookie=3f5cadd03dbcb410 in_seq=0 out_seq=0 2023-07-14T11:37:26.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.233+0000 7f8be5ffb700 10 client.27337 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.239+0000 7f8be5ffb700 1 -- 172.21.15.161:0/1894499507 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558d5e589f30 con 0x558d5e659080 2023-07-14T11:37:26.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.239+0000 7f8be5ffb700 10 client.27337 handle_command_reply: tid=0 2023-07-14T11:37:26.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.239+0000 7f8c187e0b80 1 client.27337 shutdown 2023-07-14T11:37:26.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.239+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63e350 msgr2=0x558d5e6495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63e350 0x558d5e6495b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00095c0 tx=0x7f8bf009f0e0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 shutdown_connections 2023-07-14T11:37:26.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e659080 0x558d5e65b490 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e649b50 0x558d5e646fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63e350 0x558d5e6495b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8be5ffb700 0 client.27337 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1894499507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e63ec60 0x558d5e6468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 >> 172.21.15.161:0/1894499507 conn(0x558d5e634e20 msgr2=0x558d5e63a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 shutdown_connections 2023-07-14T11:37:26.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1894499507 wait complete. 2023-07-14T11:37:26.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.240+0000 7f8c187e0b80 20 client.27337 trim_cache size 0 max 16384 2023-07-14T11:37:26.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 Processor -- start 2023-07-14T11:37:26.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 -- start start 2023-07-14T11:37:26.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 0x558d5e63a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e641230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e650530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e647e10 con 0x558d5e649b50 2023-07-14T11:37:26.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e632020 con 0x558d5e63e350 2023-07-14T11:37:26.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.266+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e6349f0 con 0x558d5e63ec60 2023-07-14T11:37:26.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e641230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e641230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36424/0 (socket says 172.21.15.161:36424) 2023-07-14T11:37:26.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/1876591392 learned_addr learned my addr 172.21.15.161:0/1876591392 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e650530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 0x558d5e63a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/1876591392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 msgr2=0x558d5e63a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 0x558d5e63a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/1876591392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 msgr2=0x558d5e650530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e650530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/1876591392 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5e639160 con 0x558d5e63ec60 2023-07-14T11:37:26.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e650530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:26.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.267+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e641230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40092a0 tx=0x7f8bf400e750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352c6bc999fffe96 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.268+0000 7f8bed7fa700 1 -- 172.21.15.161:0/1876591392 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4028d90 con 0x558d5e63ec60 2023-07-14T11:37:26.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.268+0000 7f8bed7fa700 1 -- 172.21.15.161:0/1876591392 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf400e860 con 0x558d5e63ec60 2023-07-14T11:37:26.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.268+0000 7f8bed7fa700 1 -- 172.21.15.161:0/1876591392 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4019670 con 0x558d5e63ec60 2023-07-14T11:37:26.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.268+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1876591392 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 msgr2=0x558d5e641230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.268+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e641230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40092a0 tx=0x7f8bf400e750 comp rx=0 tx=0).stop 2023-07-14T11:37:26.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1876591392 shutdown_connections 2023-07-14T11:37:26.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e650530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 0x558d5e63a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/1876591392 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e641230 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1876591392 >> 172.21.15.161:0/1876591392 conn(0x558d5e634e20 msgr2=0x558d5e63a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1876591392 shutdown_connections 2023-07-14T11:37:26.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 -- 172.21.15.161:0/1876591392 wait complete. 2023-07-14T11:37:26.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 -- start start 2023-07-14T11:37:26.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.269+0000 7f8c187e0b80 1 Processor -- start 2023-07-14T11:37:26.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.270+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 0x558d5e64f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.270+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e64f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.270+0000 7f8c187e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e6b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.270+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e6b5ca0 con 0x558d5e649b50 2023-07-14T11:37:26.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.270+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e6b5de0 con 0x558d5e63e350 2023-07-14T11:37:26.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.270+0000 7f8c187e0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5e6a2ee0 con 0x558d5e63ec60 2023-07-14T11:37:26.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.270+0000 7f8c187e0b80 10 client.0 _ll_register_callbacks cb 0x7f8bf009af80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:26.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e64f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e64f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36432/0 (socket says 172.21.15.161:36432) 2023-07-14T11:37:26.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/244319080 learned_addr learned my addr 172.21.15.161:0/244319080 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e6b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/244319080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 msgr2=0x558d5e64f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 0x558d5e64f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/244319080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 msgr2=0x558d5e6b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e6b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 -- 172.21.15.161:0/244319080 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d5e3bee60 con 0x558d5e63ec60 2023-07-14T11:37:26.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfed9d700 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e6b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:37:26.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.271+0000 7f8bfdd9b700 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e64f570 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8bf401c040 tx=0x7f8bf4005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e237bf5feb74fb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.272+0000 7f8be5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:26.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.272+0000 7f8be5ffb700 1 -- 172.21.15.161:0/244319080 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40283d0 con 0x558d5e63ec60 2023-07-14T11:37:26.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.272+0000 7f8be5ffb700 1 -- 172.21.15.161:0/244319080 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8bf4028530 con 0x558d5e63ec60 2023-07-14T11:37:26.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.272+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bf4019c50 con 0x558d5e63ec60 2023-07-14T11:37:26.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.272+0000 7f8be5ffb700 1 -- 172.21.15.161:0/244319080 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf400e9d0 con 0x558d5e63ec60 2023-07-14T11:37:26.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.272+0000 7f8c187e0b80 10 client.27343 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:26.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.272+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558d5e3e52d0 con 0x558d5e63ec60 2023-07-14T11:37:26.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8be5ffb700 1 -- 172.21.15.161:0/244319080 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf4019c50 con 0x558d5e63ec60 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8be5ffb700 1 -- 172.21.15.161:0/244319080 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8bf4005020 con 0x558d5e63ec60 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8c187e0b80 10 client.27343 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8c187e0b80 10 client.27343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8c187e0b80 20 client.27343 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e6a4310 0x558d5e6a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8c187e0b80 4 client.27343 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558d5e54c0d0 con 0x558d5e6a4310 2023-07-14T11:37:26.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.273+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e6a4310 0x558d5e6a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.274+0000 7f8bfe59c700 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e6a4310 0x558d5e6a6720 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3878906284bce132 server_cookie=7afb07000126cc64 in_seq=0 out_seq=0 2023-07-14T11:37:26.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.275+0000 7f8be5ffb700 10 client.27343 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8be5ffb700 1 -- 172.21.15.161:0/244319080 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558d5e54c0d0 con 0x558d5e6a4310 2023-07-14T11:37:26.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8be5ffb700 10 client.27343 handle_command_reply: tid=0 2023-07-14T11:37:26.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 client.27343 shutdown 2023-07-14T11:37:26.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 msgr2=0x558d5e64f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e64f570 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8bf401c040 tx=0x7f8bf4005d50 comp rx=0 tx=0).stop 2023-07-14T11:37:26.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 shutdown_connections 2023-07-14T11:37:26.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558d5e6a4310 0x558d5e6a6720 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5e649b50 0x558d5e6b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d5e63e350 0x558d5e64f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 --2- 172.21.15.161:0/244319080 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558d5e63ec60 0x558d5e64f570 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.281+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 >> 172.21.15.161:0/244319080 conn(0x558d5e634e20 msgr2=0x558d5e640220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.282+0000 7f8be5ffb700 0 client.27343 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.282+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 shutdown_connections 2023-07-14T11:37:26.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.282+0000 7f8c187e0b80 1 -- 172.21.15.161:0/244319080 wait complete. 2023-07-14T11:37:26.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.283+0000 7f8c187e0b80 20 client.27343 trim_cache size 0 max 16384 2023-07-14T11:37:26.286 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:37:26.286 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:37:26.287 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "a9340d58-8623-470b-9097-f8d5a6bd7bac", 2023-07-14T11:37:26.287 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:37:26.287 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:37:26.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.284+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8be80a1340 msgr2=0x7f8be80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.284+0000 7f8c04dff700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8be80a1340 0x7f8be80a37f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009900 tx=0x7f8bf8009ea0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.284+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 msgr2=0x7f8c00197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.284+0000 7f8c04dff700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 0x7f8c00197e50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0004730 tx=0x7f8bf0004760 comp rx=0 tx=0).stop 2023-07-14T11:37:26.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.284+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 shutdown_connections 2023-07-14T11:37:26.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.285+0000 7f8c04dff700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f8be80a1340 0x7f8be80a37f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.285+0000 7f8c04dff700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c001049f0 0x7f8c0019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.285+0000 7f8c04dff700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8c000fe9e0 0x7f8c00197e50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.285+0000 7f8c04dff700 1 --2- 172.21.15.161:0/1813805923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c000ff390 0x7f8c00198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.285+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 >> 172.21.15.161:0/1813805923 conn(0x7f8c000fa500 msgr2=0x7f8c001099f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.285+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 shutdown_connections 2023-07-14T11:37:26.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.285+0000 7f8c04dff700 1 -- 172.21.15.161:0/1813805923 wait complete. 2023-07-14T11:37:26.310 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:37:26.581 INFO:tasks.workunit.client.0.smithi161.stdout: 10 76429 31.84 MB/sec execute 8 sec latency 206.282 ms 2023-07-14T11:37:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.598+0000 7f72f2bff700 1 -- 172.21.15.161:0/427800886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec100490 msgr2=0x7f72ec10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.598+0000 7f72f2bff700 1 --2- 172.21.15.161:0/427800886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec100490 0x7f72ec10c920 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b0a0 tx=0x7f72e400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.599+0000 7f72f2bff700 1 -- 172.21.15.161:0/427800886 shutdown_connections 2023-07-14T11:37:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.599+0000 7f72f2bff700 1 --2- 172.21.15.161:0/427800886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72ec105af0 0x7f72ec105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.599+0000 7f72f2bff700 1 --2- 172.21.15.161:0/427800886 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec100490 0x7f72ec10c920 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.599+0000 7f72f2bff700 1 --2- 172.21.15.161:0/427800886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72ec0ffae0 0x7f72ec0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.599+0000 7f72f2bff700 1 -- 172.21.15.161:0/427800886 >> 172.21.15.161:0/427800886 conn(0x7f72ec0fb5e0 msgr2=0x7f72ec0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.599+0000 7f72f2bff700 1 -- 172.21.15.161:0/427800886 shutdown_connections 2023-07-14T11:37:26.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.599+0000 7f72f2bff700 1 -- 172.21.15.161:0/427800886 wait complete. 2023-07-14T11:37:26.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.600+0000 7f72f2bff700 1 Processor -- start 2023-07-14T11:37:26.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.601+0000 7f72f2bff700 1 -- start start 2023-07-14T11:37:26.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f2bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72ec0ffae0 0x7f72ec19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f2bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72ec100490 0x7f72ec19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f2bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec105af0 0x7f72ec195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f2bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1961b0 con 0x7f72ec100490 2023-07-14T11:37:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f2bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec196320 con 0x7f72ec0ffae0 2023-07-14T11:37:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f2bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1964e0 con 0x7f72ec105af0 2023-07-14T11:37:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec105af0 0x7f72ec195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec105af0 0x7f72ec195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36450/0 (socket says 172.21.15.161:36450) 2023-07-14T11:37:26.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72f119c700 1 -- 172.21.15.161:0/3319462880 learned_addr learned my addr 172.21.15.161:0/3319462880 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.602+0000 7f72ebfff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72ec100490 0x7f72ec19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72f119c700 1 -- 172.21.15.161:0/3319462880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72ec0ffae0 msgr2=0x7f72ec19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72f119c700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72ec0ffae0 0x7f72ec19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72f119c700 1 -- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72ec100490 msgr2=0x7f72ec19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72f119c700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72ec100490 0x7f72ec19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72f119c700 1 -- 172.21.15.161:0/3319462880 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0009520 con 0x7f72ec105af0 2023-07-14T11:37:26.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72ebfff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72ec100490 0x7f72ec19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:37:26.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72f119c700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec105af0 0x7f72ec195b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009fd0 tx=0x7f72e400bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.603+0000 7f72e9ffb700 1 -- 172.21.15.161:0/3319462880 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400e040 con 0x7f72ec105af0 2023-07-14T11:37:26.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.604+0000 7f72e9ffb700 1 -- 172.21.15.161:0/3319462880 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e4004e60 con 0x7f72ec105af0 2023-07-14T11:37:26.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.604+0000 7f72e9ffb700 1 -- 172.21.15.161:0/3319462880 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4005260 con 0x7f72ec105af0 2023-07-14T11:37:26.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.604+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e4009d00 con 0x7f72ec105af0 2023-07-14T11:37:26.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.604+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec072740 con 0x7f72ec105af0 2023-07-14T11:37:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.608+0000 7f72e9ffb700 1 -- 172.21.15.161:0/3319462880 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f72e401d470 con 0x7f72ec105af0 2023-07-14T11:37:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.610+0000 7f72e9ffb700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72dc0a5750 0x7f72dc0a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.610+0000 7f7306522b80 1 Processor -- start 2023-07-14T11:37:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.610+0000 7f72e9ffb700 1 -- 172.21.15.161:0/3319462880 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f72e40ce650 con 0x7f72ec105af0 2023-07-14T11:37:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.610+0000 7f72f099b700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72dc0a5750 0x7f72dc0a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.610+0000 7f7306522b80 1 -- start start 2023-07-14T11:37:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f7306522b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d102d0 0x55efb5d106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f7306522b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d10be0 0x55efb5d1b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f7306522b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d1deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f7306522b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5c16cb0 con 0x55efb5d102d0 2023-07-14T11:37:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f7306522b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5c69a40 con 0x55efb5d10be0 2023-07-14T11:37:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f7306522b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5c62ed0 con 0x55efb5d1bad0 2023-07-14T11:37:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d10be0 0x55efb5d1b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72f119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d1deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d10be0 0x55efb5d1b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51660/0 (socket says 172.21.15.161:51660) 2023-07-14T11:37:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72f119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d1deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36460/0 (socket says 172.21.15.161:36460) 2023-07-14T11:37:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72ebfff700 1 -- 172.21.15.161:0/4177396639 learned_addr learned my addr 172.21.15.161:0/4177396639 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72f119c700 1 -- 172.21.15.161:0/4177396639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d10be0 msgr2=0x55efb5d1b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72f119c700 1 --2- 172.21.15.161:0/4177396639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d10be0 0x55efb5d1b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72f119c700 1 -- 172.21.15.161:0/4177396639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d102d0 msgr2=0x55efb5d106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72f119c700 1 --2- 172.21.15.161:0/4177396639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d102d0 0x55efb5d106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.611+0000 7f72f119c700 1 -- 172.21.15.161:0/4177396639 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb5a90de0 con 0x55efb5d1bad0 2023-07-14T11:37:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.612+0000 7f72f119c700 1 --2- 172.21.15.161:0/4177396639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d1deb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f72e40cb9e0 tx=0x7f72e401dcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd27b860c0d86261 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.612+0000 7f72f099b700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72dc0a5750 0x7f72dc0a7c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009d10 tx=0x7f72e00097b0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.612+0000 7f72d6ffd700 1 -- 172.21.15.161:0/4177396639 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40cd070 con 0x55efb5d1bad0 2023-07-14T11:37:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.612+0000 7f72d6ffd700 1 -- 172.21.15.161:0/4177396639 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e409b3f0 con 0x55efb5d1bad0 2023-07-14T11:37:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.612+0000 7f72d6ffd700 1 -- 172.21.15.161:0/4177396639 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40a1110 con 0x55efb5d1bad0 2023-07-14T11:37:26.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 -- 172.21.15.161:0/4177396639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 msgr2=0x55efb5d1deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 --2- 172.21.15.161:0/4177396639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d1deb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f72e40cb9e0 tx=0x7f72e401dcd0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 -- 172.21.15.161:0/4177396639 shutdown_connections 2023-07-14T11:37:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 --2- 172.21.15.161:0/4177396639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d102d0 0x55efb5d106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 --2- 172.21.15.161:0/4177396639 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d1deb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 --2- 172.21.15.161:0/4177396639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d10be0 0x55efb5d1b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 -- 172.21.15.161:0/4177396639 >> 172.21.15.161:0/4177396639 conn(0x55efb5d06da0 msgr2=0x55efb5d0c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 -- 172.21.15.161:0/4177396639 shutdown_connections 2023-07-14T11:37:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.613+0000 7f7306522b80 1 -- 172.21.15.161:0/4177396639 wait complete. 2023-07-14T11:37:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.614+0000 7f7306522b80 1 -- start start 2023-07-14T11:37:26.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.614+0000 7f7306522b80 1 Processor -- start 2023-07-14T11:37:26.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f7306522b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d102d0 0x55efb5d1bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f7306522b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f7306522b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d1bad0 0x55efb5d26f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f7306522b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5c02ac0 con 0x55efb5d10be0 2023-07-14T11:37:26.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f7306522b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5c027c0 con 0x55efb5d1bad0 2023-07-14T11:37:26.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f7306522b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5ae6920 con 0x55efb5d102d0 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f7306522b80 10 client.0 _ll_register_callbacks cb 0x55efb5c00760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51322/0 (socket says 172.21.15.161:51322) 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f72ebfff700 1 -- 172.21.15.161:0/2798603556 learned_addr learned my addr 172.21.15.161:0/2798603556 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f72f119c700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d1bad0 0x55efb5d26f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f72f099b700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d102d0 0x55efb5d1bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.615+0000 7f72ebfff700 1 -- 172.21.15.161:0/2798603556 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d102d0 msgr2=0x55efb5d1bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.616+0000 7f72ebfff700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d102d0 0x55efb5d1bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.616+0000 7f72ebfff700 1 -- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d1bad0 msgr2=0x55efb5d26f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.616+0000 7f72ebfff700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d1bad0 0x55efb5d26f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.616+0000 7f72ebfff700 1 -- 172.21.15.161:0/2798603556 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec072740 con 0x55efb5d10be0 2023-07-14T11:37:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.616+0000 7f72f119c700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d1bad0 0x55efb5d26f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:37:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.616+0000 7f72ebfff700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1c3e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f72d8012810 tx=0x7f72d8012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10636a8f9c448e71 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.616+0000 7f72cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:37:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.617+0000 7f72cbfff700 1 -- 172.21.15.161:0/2798603556 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8017070 con 0x55efb5d10be0 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.617+0000 7f72cbfff700 1 -- 172.21.15.161:0/2798603556 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f72d80117d0 con 0x55efb5d10be0 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.617+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efb5bbc4f0 con 0x55efb5d10be0 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.617+0000 7f72cbfff700 1 -- 172.21.15.161:0/2798603556 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72d8011990 con 0x55efb5d10be0 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.617+0000 7f7306522b80 10 client.17268 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.617+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55efb5a79c60 con 0x55efb5d10be0 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f72cbfff700 1 -- 172.21.15.161:0/2798603556 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55efb5bbc4f0 con 0x55efb5d10be0 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f72cbfff700 1 -- 172.21.15.161:0/2798603556 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f72d800fc70 con 0x55efb5d10be0 2023-07-14T11:37:26.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f7306522b80 10 client.17268 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:26.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f7306522b80 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:26.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f7306522b80 20 client.17268 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:26.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f7306522b80 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d2b310 0x55efb5d2d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f7306522b80 4 client.17268 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:26.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.618+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efb5c5beb0 con 0x55efb5d2b310 2023-07-14T11:37:26.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.619+0000 7f72f099b700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d2b310 0x55efb5d2d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.619+0000 7f72cbfff700 10 client.17268 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.619+0000 7f72f099b700 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d2b310 0x55efb5d2d720 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d6edec09e2e220 server_cookie=e87bf8e02434515b in_seq=0 out_seq=0 2023-07-14T11:37:26.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f72cbfff700 1 -- 172.21.15.161:0/2798603556 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55efb5c5beb0 con 0x55efb5d2b310 2023-07-14T11:37:26.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f72cbfff700 10 client.17268 handle_command_reply: tid=0 2023-07-14T11:37:26.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 client.17268 shutdown 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 msgr2=0x55efb5d1c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1c3e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f72d8012810 tx=0x7f72d8012b20 comp rx=0 tx=0).stop 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 shutdown_connections 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d2b310 0x55efb5d2d720 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d1bad0 0x55efb5d26f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d102d0 0x55efb5d1bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 --2- 172.21.15.161:0/2798603556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1c3e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 >> 172.21.15.161:0/2798603556 conn(0x55efb5d06da0 msgr2=0x55efb5d0c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f72cbfff700 0 client.17268 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 shutdown_connections 2023-07-14T11:37:26.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.625+0000 7f7306522b80 1 -- 172.21.15.161:0/2798603556 wait complete. 2023-07-14T11:37:26.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.626+0000 7f7306522b80 20 client.17268 trim_cache size 0 max 16384 2023-07-14T11:37:26.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 Processor -- start 2023-07-14T11:37:26.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 -- start start 2023-07-14T11:37:26.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d15400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d26600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d26b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5d11540 con 0x55efb5d10be0 2023-07-14T11:37:26.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5d1cb40 con 0x55efb5d102d0 2023-07-14T11:37:26.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.647+0000 7f7306522b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5d289b0 con 0x55efb5d1bad0 2023-07-14T11:37:26.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d26b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d26b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36476/0 (socket says 172.21.15.161:36476) 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 -- 172.21.15.161:0/630213188 learned_addr learned my addr 172.21.15.161:0/630213188 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72ebfff700 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d26600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 -- 172.21.15.161:0/630213188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 msgr2=0x55efb5d15400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d15400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 -- 172.21.15.161:0/630213188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 msgr2=0x55efb5d26600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d26600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 -- 172.21.15.161:0/630213188 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb5d32350 con 0x55efb5d1bad0 2023-07-14T11:37:26.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.648+0000 7f72f119c700 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d26b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72e409b0f0 tx=0x7f72e4086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df73e440e5728e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f72d77fe700 1 -- 172.21.15.161:0/630213188 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40cd040 con 0x55efb5d1bad0 2023-07-14T11:37:26.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f72d77fe700 1 -- 172.21.15.161:0/630213188 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e409f420 con 0x55efb5d1bad0 2023-07-14T11:37:26.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f72d77fe700 1 -- 172.21.15.161:0/630213188 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40a10b0 con 0x55efb5d1bad0 2023-07-14T11:37:26.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f7306522b80 1 -- 172.21.15.161:0/630213188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 msgr2=0x55efb5d26b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f7306522b80 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d26b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72e409b0f0 tx=0x7f72e4086040 comp rx=0 tx=0).stop 2023-07-14T11:37:26.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f7306522b80 1 -- 172.21.15.161:0/630213188 shutdown_connections 2023-07-14T11:37:26.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f7306522b80 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d26b40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f7306522b80 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d15400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f7306522b80 1 --2- 172.21.15.161:0/630213188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d26600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.649+0000 7f7306522b80 1 -- 172.21.15.161:0/630213188 >> 172.21.15.161:0/630213188 conn(0x55efb5d06da0 msgr2=0x55efb5d1bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.650+0000 7f7306522b80 1 -- 172.21.15.161:0/630213188 shutdown_connections 2023-07-14T11:37:26.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.650+0000 7f7306522b80 1 -- 172.21.15.161:0/630213188 wait complete. 2023-07-14T11:37:26.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.650+0000 7f7306522b80 1 -- start start 2023-07-14T11:37:26.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.650+0000 7f7306522b80 1 Processor -- start 2023-07-14T11:37:26.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.651+0000 7f7306522b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.651+0000 7f7306522b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.651+0000 7f7306522b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.651+0000 7f7306522b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5d786e0 con 0x55efb5d10be0 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.651+0000 7f7306522b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5d78850 con 0x55efb5d102d0 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.651+0000 7f7306522b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb5d78a10 con 0x55efb5d1bad0 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.651+0000 7f7306522b80 10 client.0 _ll_register_callbacks cb 0x7f72d8009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f099b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f099b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d1ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51682/0 (socket says 172.21.15.161:51682) 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f099b700 1 -- 172.21.15.161:0/1047198611 learned_addr learned my addr 172.21.15.161:0/1047198611 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:26.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f119c700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f119c700 1 -- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 msgr2=0x55efb5d1ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f119c700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d1ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f119c700 1 -- 172.21.15.161:0/1047198611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 msgr2=0x55efb5d1f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:26.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f119c700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f119c700 1 -- 172.21.15.161:0/1047198611 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efb5a90de0 con 0x55efb5d1bad0 2023-07-14T11:37:26.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72ebfff700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:26.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f099b700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d1ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:26.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72f119c700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d781a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55efb5d067e0 tx=0x7f72e409b280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4daf43a9244a6022 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:26.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.652+0000 7f72cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:26.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f72cbfff700 1 -- 172.21.15.161:0/1047198611 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40cd070 con 0x55efb5d1bad0 2023-07-14T11:37:26.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f72cbfff700 1 -- 172.21.15.161:0/1047198611 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f72e40a12b0 con 0x55efb5d1bad0 2023-07-14T11:37:26.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72e409b3f0 con 0x55efb5d1bad0 2023-07-14T11:37:26.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f72cbfff700 1 -- 172.21.15.161:0/1047198611 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e40ce240 con 0x55efb5d1bad0 2023-07-14T11:37:26.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 10 client.27361 fetch_fsmap learned FSMap version 38 2023-07-14T11:37:26.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55efb5ab7250 con 0x55efb5d1bad0 2023-07-14T11:37:26.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f72cbfff700 1 -- 172.21.15.161:0/1047198611 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e40c9070 con 0x55efb5d1bad0 2023-07-14T11:37:26.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f72cbfff700 1 -- 172.21.15.161:0/1047198611 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f72e409a970 con 0x55efb5d1bad0 2023-07-14T11:37:26.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 10 client.27361 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T11:37:26.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 10 client.27361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:26.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 20 client.27361 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:26.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d7fca0 0x55efb5d82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:26.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 4 client.27361 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:37:26.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.653+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55efb5c1e050 con 0x55efb5d7fca0 2023-07-14T11:37:26.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.654+0000 7f72f099b700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d7fca0 0x55efb5d82080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:26.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.654+0000 7f72f099b700 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d7fca0 0x55efb5d82080 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63278329e1bf1e3 server_cookie=43aa1c061b8477bd in_seq=0 out_seq=0 2023-07-14T11:37:26.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.654+0000 7f72cbfff700 10 client.27361 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f72cbfff700 1 -- 172.21.15.161:0/1047198611 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55efb5c1e050 con 0x55efb5d7fca0 2023-07-14T11:37:26.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f72cbfff700 10 client.27361 handle_command_reply: tid=0 2023-07-14T11:37:26.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 client.27361 shutdown 2023-07-14T11:37:26.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 msgr2=0x55efb5d781a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d781a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55efb5d067e0 tx=0x7f72e409b280 comp rx=0 tx=0).stop 2023-07-14T11:37:26.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 shutdown_connections 2023-07-14T11:37:26.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55efb5d7fca0 0x55efb5d82080 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55efb5d1bad0 0x55efb5d781a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efb5d102d0 0x55efb5d1ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 --2- 172.21.15.161:0/1047198611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efb5d10be0 0x55efb5d1f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f72cbfff700 0 client.27361 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:26.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.660+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 >> 172.21.15.161:0/1047198611 conn(0x55efb5d06da0 msgr2=0x55efb5d19460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 shutdown_connections 2023-07-14T11:37:26.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f7306522b80 1 -- 172.21.15.161:0/1047198611 wait complete. 2023-07-14T11:37:26.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f7306522b80 20 client.27361 trim_cache size 0 max 16384 2023-07-14T11:37:26.662 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:37:26.662 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (193 inodes in the stack)", 2023-07-14T11:37:26.662 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:37:26.663 INFO:teuthology.orchestra.run.smithi161.stdout: "a9340d58-8623-470b-9097-f8d5a6bd7bac": { 2023-07-14T11:37:26.663 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:37:26.663 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "a9340d58-8623-470b-9097-f8d5a6bd7bac", 2023-07-14T11:37:26.663 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:37:26.663 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:37:26.663 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:37:26.663 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:37:26.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72dc0a5750 msgr2=0x7f72dc0a7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72dc0a5750 0x7f72dc0a7c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009d10 tx=0x7f72e00097b0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec105af0 msgr2=0x7f72ec195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:26.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec105af0 0x7f72ec195b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009fd0 tx=0x7f72e400bd90 comp rx=0 tx=0).stop 2023-07-14T11:37:26.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 shutdown_connections 2023-07-14T11:37:26.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f72dc0a5750 0x7f72dc0a7c00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f72ec105af0 0x7f72ec195b40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72ec0ffae0 0x7f72ec19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.661+0000 7f72f2bff700 1 --2- 172.21.15.161:0/3319462880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72ec100490 0x7f72ec19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:26.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.662+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 >> 172.21.15.161:0/3319462880 conn(0x7f72ec0fb5e0 msgr2=0x7f72ec1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:26.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.662+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 shutdown_connections 2023-07-14T11:37:26.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:26.662+0000 7f72f2bff700 1 -- 172.21.15.161:0/3319462880 wait complete. 2023-07-14T11:37:26.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:26 smithi175 ceph-mon[129081]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 705 op/s 2023-07-14T11:37:26.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:26 smithi161 ceph-mon[95526]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 705 op/s 2023-07-14T11:37:26.681 INFO:tasks.cephfs.filesystem:scrub status for tag:a9340d58-8623-470b-9097-f8d5a6bd7bac - {'path': '/', 'tag': 'a9340d58-8623-470b-9097-f8d5a6bd7bac', 'options': 'recursive,force'} 2023-07-14T11:37:26.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:26 smithi182 ceph-mon[129366]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 705 op/s 2023-07-14T11:37:27.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 76992 31.23 MB/sec execute 9 sec latency 96.040 ms 2023-07-14T11:37:27.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:37:27.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:37:27.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:37:27.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:37:27.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:37:27.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:37:27.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:27 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:37:27.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:37:27.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:37:27.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:37:27.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:37:27.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:37:27.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:27 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:37:27.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:37:27.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:37:27.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:37:27.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:37:27.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:37:27.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:27.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:27 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:28.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 77757 32.50 MB/sec execute 10 sec latency 80.949 ms 2023-07-14T11:37:28.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:28 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:28.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:28 smithi175 ceph-mon[129081]: pgmap v477: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 15 MiB/s wr, 657 op/s 2023-07-14T11:37:28.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:28 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:28.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:28 smithi182 ceph-mon[129366]: pgmap v477: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 15 MiB/s wr, 657 op/s 2023-07-14T11:37:28.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:28 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:37:28.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:28 smithi161 ceph-mon[95526]: pgmap v477: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 15 MiB/s wr, 657 op/s 2023-07-14T11:37:29.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 78669 33.78 MB/sec execute 11 sec latency 60.764 ms 2023-07-14T11:37:30.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 79351 34.03 MB/sec execute 12 sec latency 116.869 ms 2023-07-14T11:37:30.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:30 smithi175 ceph-mon[129081]: pgmap v478: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 580 op/s 2023-07-14T11:37:30.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:30 smithi182 ceph-mon[129366]: pgmap v478: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 580 op/s 2023-07-14T11:37:30.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:30 smithi161 ceph-mon[95526]: pgmap v478: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 580 op/s 2023-07-14T11:37:31.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 79898 34.03 MB/sec execute 13 sec latency 103.536 ms 2023-07-14T11:37:32.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 80749 34.77 MB/sec execute 14 sec latency 70.579 ms 2023-07-14T11:37:32.665 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:32 smithi175 ceph-mon[129081]: pgmap v479: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 636 op/s 2023-07-14T11:37:32.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:32 smithi182 ceph-mon[129366]: pgmap v479: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 636 op/s 2023-07-14T11:37:32.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:32 smithi161 ceph-mon[95526]: pgmap v479: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 636 op/s 2023-07-14T11:37:33.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 81454 34.73 MB/sec execute 15 sec latency 77.771 ms 2023-07-14T11:37:34.416 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:34 smithi175 ceph-mon[129081]: pgmap v480: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 591 op/s 2023-07-14T11:37:34.462 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:34 smithi182 ceph-mon[129366]: pgmap v480: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 591 op/s 2023-07-14T11:37:34.492 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:33 smithi161 ceph-mon[95526]: pgmap v480: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 591 op/s 2023-07-14T11:37:34.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 82187 35.21 MB/sec execute 16 sec latency 62.662 ms 2023-07-14T11:37:35.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 83060 35.68 MB/sec execute 17 sec latency 56.868 ms 2023-07-14T11:37:36.582 INFO:tasks.workunit.client.0.smithi161.stdout: 10 83870 36.41 MB/sec execute 18 sec latency 59.293 ms 2023-07-14T11:37:36.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:36 smithi175 ceph-mon[129081]: pgmap v481: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.7 MiB/s wr, 408 op/s 2023-07-14T11:37:36.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:36 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:37:36.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:36 smithi182 ceph-mon[129366]: pgmap v481: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.7 MiB/s wr, 408 op/s 2023-07-14T11:37:36.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:36 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:37:36.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:36 smithi161 ceph-mon[95526]: pgmap v481: 129 pgs: 129 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.7 MiB/s wr, 408 op/s 2023-07-14T11:37:36.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:36 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:37:37.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 84592 36.69 MB/sec execute 19 sec latency 67.217 ms 2023-07-14T11:37:37.665 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:37 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:37:37.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:37 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:37:37.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:37 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:37:37.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:37 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:37:37.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:37 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:37:37.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:37 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:37:38.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 85167 36.28 MB/sec execute 20 sec latency 66.872 ms 2023-07-14T11:37:38.665 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:38 smithi175 ceph-mon[129081]: pgmap v482: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 610 op/s 2023-07-14T11:37:38.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:38 smithi182 ceph-mon[129366]: pgmap v482: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 610 op/s 2023-07-14T11:37:38.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:38 smithi161 ceph-mon[95526]: pgmap v482: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 610 op/s 2023-07-14T11:37:39.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 85646 35.54 MB/sec execute 21 sec latency 154.415 ms 2023-07-14T11:37:40.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 86390 35.84 MB/sec execute 22 sec latency 68.519 ms 2023-07-14T11:37:40.665 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:40 smithi175 ceph-mon[129081]: pgmap v483: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2023-07-14T11:37:40.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:40 smithi182 ceph-mon[129366]: pgmap v483: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2023-07-14T11:37:40.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:40 smithi161 ceph-mon[95526]: pgmap v483: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2023-07-14T11:37:41.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 87257 36.04 MB/sec execute 23 sec latency 47.130 ms 2023-07-14T11:37:42.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 88109 36.77 MB/sec execute 24 sec latency 63.696 ms 2023-07-14T11:37:42.665 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:42 smithi175 ceph-mon[129081]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.7 MiB/s wr, 514 op/s 2023-07-14T11:37:42.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:42 smithi182 ceph-mon[129366]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.7 MiB/s wr, 514 op/s 2023-07-14T11:37:42.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:42 smithi161 ceph-mon[95526]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.7 MiB/s wr, 514 op/s 2023-07-14T11:37:43.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 88763 36.69 MB/sec execute 25 sec latency 58.288 ms 2023-07-14T11:37:44.416 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:44 smithi175 ceph-mon[129081]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 501 op/s 2023-07-14T11:37:44.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:44 smithi182 ceph-mon[129366]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 501 op/s 2023-07-14T11:37:44.492 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:44 smithi161 ceph-mon[95526]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 501 op/s 2023-07-14T11:37:44.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 89237 36.34 MB/sec execute 26 sec latency 243.869 ms 2023-07-14T11:37:45.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 89983 36.36 MB/sec execute 27 sec latency 73.092 ms 2023-07-14T11:37:46.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 90795 36.73 MB/sec execute 28 sec latency 75.752 ms 2023-07-14T11:37:46.666 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:46 smithi175 ceph-mon[129081]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.1 MiB/s wr, 391 op/s 2023-07-14T11:37:46.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:46 smithi182 ceph-mon[129366]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.1 MiB/s wr, 391 op/s 2023-07-14T11:37:46.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:46 smithi161 ceph-mon[95526]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.1 MiB/s wr, 391 op/s 2023-07-14T11:37:47.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 91489 36.66 MB/sec execute 29 sec latency 94.192 ms 2023-07-14T11:37:48.584 INFO:tasks.workunit.client.0.smithi161.stdout: 10 91923 36.32 MB/sec execute 30 sec latency 481.656 ms 2023-07-14T11:37:48.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:48 smithi161 ceph-mon[95526]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 593 op/s 2023-07-14T11:37:48.920 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:48 smithi175 ceph-mon[129081]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 593 op/s 2023-07-14T11:37:48.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:48 smithi182 ceph-mon[129366]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 593 op/s 2023-07-14T11:37:49.583 INFO:tasks.workunit.client.0.smithi161.stdout: 10 92013 35.18 MB/sec execute 31 sec latency 916.684 ms 2023-07-14T11:37:49.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:49 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:49.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:49 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:49.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:49 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:50.585 INFO:tasks.workunit.client.0.smithi161.stdout: 10 92350 34.54 MB/sec execute 32 sec latency 342.322 ms 2023-07-14T11:37:50.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:50 smithi175 ceph-mon[129081]: pgmap v488: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.6 MiB/s wr, 392 op/s 2023-07-14T11:37:50.963 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:50 smithi182 ceph-mon[129366]: pgmap v488: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.6 MiB/s wr, 392 op/s 2023-07-14T11:37:50.993 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:50 smithi161 ceph-mon[95526]: pgmap v488: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.6 MiB/s wr, 392 op/s 2023-07-14T11:37:51.586 INFO:tasks.workunit.client.0.smithi161.stdout: 10 92603 33.88 MB/sec execute 33 sec latency 129.446 ms 2023-07-14T11:37:51.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:51 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:51.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:51 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:37:51.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:51 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:51.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:51 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:37:51.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:51 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:37:51.992 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:51 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:37:52.586 INFO:tasks.workunit.client.0.smithi161.stdout: 10 93140 33.98 MB/sec execute 34 sec latency 130.402 ms 2023-07-14T11:37:52.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:52 smithi175 ceph-mon[129081]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 552 op/s 2023-07-14T11:37:52.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:52 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:37:52.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:52 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:37:52.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:52 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:37:52.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:52 smithi182 ceph-mon[129366]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 552 op/s 2023-07-14T11:37:52.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:52 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:37:52.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:52 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:37:52.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:52 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:37:52.992 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:52 smithi161 ceph-mon[95526]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 552 op/s 2023-07-14T11:37:52.992 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:52 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:37:52.992 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:52 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:37:52.993 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:52 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:37:53.592 INFO:tasks.workunit.client.0.smithi161.stdout: 10 93918 34.07 MB/sec execute 35 sec latency 60.353 ms 2023-07-14T11:37:54.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:54 smithi175 ceph-mon[129081]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 622 op/s 2023-07-14T11:37:54.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:54 smithi182 ceph-mon[129366]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 622 op/s 2023-07-14T11:37:54.493 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:54 smithi161 ceph-mon[95526]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 622 op/s 2023-07-14T11:37:54.591 INFO:tasks.workunit.client.0.smithi161.stdout: 10 94006 33.18 MB/sec execute 36 sec latency 831.934 ms 2023-07-14T11:37:55.592 INFO:tasks.workunit.client.0.smithi161.stdout: 10 94115 32.49 MB/sec execute 37 sec latency 955.941 ms 2023-07-14T11:37:56.592 INFO:tasks.workunit.client.0.smithi161.stdout: 10 94406 32.07 MB/sec execute 38 sec latency 142.716 ms 2023-07-14T11:37:56.665 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:56 smithi175 ceph-mon[129081]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 546 op/s 2023-07-14T11:37:56.682 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:37:56.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:56 smithi182 ceph-mon[129366]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 546 op/s 2023-07-14T11:37:56.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:56 smithi161 ceph-mon[95526]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 546 op/s 2023-07-14T11:37:57.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.073+0000 7f24a11ff700 1 -- 172.21.15.161:0/3195968617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06c9a0 msgr2=0x7f249c06ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.073+0000 7f24a11ff700 1 --2- 172.21.15.161:0/3195968617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06c9a0 0x7f249c06ce10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f248c01a9a0 tx=0x7f248c01acb0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.084+0000 7f24a11ff700 1 -- 172.21.15.161:0/3195968617 shutdown_connections 2023-07-14T11:37:57.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.084+0000 7f24a11ff700 1 --2- 172.21.15.161:0/3195968617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f249c06d3a0 0x7f249c11b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.084+0000 7f24a11ff700 1 --2- 172.21.15.161:0/3195968617 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f249c10e280 0x7f249c10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.084+0000 7f24a11ff700 1 --2- 172.21.15.161:0/3195968617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06c9a0 0x7f249c06ce10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.084+0000 7f24a11ff700 1 -- 172.21.15.161:0/3195968617 >> 172.21.15.161:0/3195968617 conn(0x7f249c06bf70 msgr2=0x7f249c070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:57.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.093+0000 7f24a11ff700 1 -- 172.21.15.161:0/3195968617 shutdown_connections 2023-07-14T11:37:57.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.097+0000 7f24a11ff700 1 -- 172.21.15.161:0/3195968617 wait complete. 2023-07-14T11:37:57.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.098+0000 7f24a11ff700 1 Processor -- start 2023-07-14T11:37:57.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.104+0000 7f24a11ff700 1 -- start start 2023-07-14T11:37:57.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.105+0000 7f24a11ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f249c06c9a0 0x7f249c116e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.105+0000 7f24a11ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06d3a0 0x7f249c1173b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.105+0000 7f24a11ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f249c10e280 0x7f249c1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.105+0000 7f24a11ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c1a2550 con 0x7f249c10e280 2023-07-14T11:37:57.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.105+0000 7f24a11ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c1a26c0 con 0x7f249c06d3a0 2023-07-14T11:37:57.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.105+0000 7f24a11ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c1a2880 con 0x7f249c06c9a0 2023-07-14T11:37:57.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.107+0000 7f249a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06d3a0 0x7f249c1173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.107+0000 7f249a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06d3a0 0x7f249c1173b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:41992/0 (socket says 172.21.15.161:41992) 2023-07-14T11:37:57.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.107+0000 7f249a59c700 1 -- 172.21.15.161:0/85615480 learned_addr learned my addr 172.21.15.161:0/85615480 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:57.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.108+0000 7f249a59c700 1 -- 172.21.15.161:0/85615480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f249c06c9a0 msgr2=0x7f249c116e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:37:57.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.108+0000 7f249a59c700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f249c06c9a0 0x7f249c116e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.108+0000 7f249a59c700 1 -- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f249c10e280 msgr2=0x7f249c1a1f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:37:57.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.108+0000 7f249a59c700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f249c10e280 0x7f249c1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.108+0000 7f249a59c700 1 -- 172.21.15.161:0/85615480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c01a600 con 0x7f249c06d3a0 2023-07-14T11:37:57.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.109+0000 7f249a59c700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06d3a0 0x7f249c1173b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f248c01a8f0 tx=0x7f248c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:57.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.111+0000 7f248bfff700 1 -- 172.21.15.161:0/85615480 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c02d070 con 0x7f249c06d3a0 2023-07-14T11:37:57.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.111+0000 7f248bfff700 1 -- 172.21.15.161:0/85615480 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f248c0054e0 con 0x7f249c06d3a0 2023-07-14T11:37:57.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.111+0000 7f248bfff700 1 -- 172.21.15.161:0/85615480 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c004ad0 con 0x7f249c06d3a0 2023-07-14T11:37:57.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.111+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c1a2b00 con 0x7f249c06d3a0 2023-07-14T11:37:57.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.111+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c1a9030 con 0x7f249c06d3a0 2023-07-14T11:37:57.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.124+0000 7f24b4b03b80 1 Processor -- start 2023-07-14T11:37:57.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.125+0000 7f24b4b03b80 1 -- start start 2023-07-14T11:37:57.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.125+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 0x5631acdab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.125+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.125+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdb6ad0 0x5631acdb8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.125+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631accb1cb0 con 0x5631acdab2d0 2023-07-14T11:37:57.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.125+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acd04a40 con 0x5631acdb6ad0 2023-07-14T11:37:57.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.125+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631accfded0 con 0x5631acdabbe0 2023-07-14T11:37:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.127+0000 7f249ad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.127+0000 7f249ad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41042/0 (socket says 172.21.15.161:41042) 2023-07-14T11:37:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.127+0000 7f249ad9d700 1 -- 172.21.15.161:0/3662001248 learned_addr learned my addr 172.21.15.161:0/3662001248 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.130+0000 7f248bfff700 1 -- 172.21.15.161:0/85615480 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f248c004080 con 0x7f249c06d3a0 2023-07-14T11:37:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.131+0000 7f249ad9d700 1 -- 172.21.15.161:0/3662001248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdb6ad0 msgr2=0x5631acdb8eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:37:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.131+0000 7f249ad9d700 1 --2- 172.21.15.161:0/3662001248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdb6ad0 0x5631acdb8eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.131+0000 7f249ad9d700 1 -- 172.21.15.161:0/3662001248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 msgr2=0x5631acdab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.131+0000 7f249ad9d700 1 --2- 172.21.15.161:0/3662001248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 0x5631acdab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.131+0000 7f249ad9d700 1 -- 172.21.15.161:0/3662001248 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631acb2bde0 con 0x5631acdabbe0 2023-07-14T11:37:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.134+0000 7f249ad9d700 1 --2- 172.21.15.161:0/3662001248 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb6590 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2494009d80 tx=0x7f249400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac596bf8141bdbaa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.135+0000 7f24897fa700 1 -- 172.21.15.161:0/3662001248 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400e070 con 0x5631acdabbe0 2023-07-14T11:37:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.135+0000 7f24897fa700 1 -- 172.21.15.161:0/3662001248 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249400bb70 con 0x5631acdabbe0 2023-07-14T11:37:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.135+0000 7f24897fa700 1 -- 172.21.15.161:0/3662001248 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24940115f0 con 0x5631acdabbe0 2023-07-14T11:37:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.138+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3662001248 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 msgr2=0x5631acdb6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.138+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3662001248 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb6590 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2494009d80 tx=0x7f249400b330 comp rx=0 tx=0).stop 2023-07-14T11:37:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.139+0000 7f248bfff700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f24840a14d0 0x7f24840a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.139+0000 7f248bfff700 1 -- 172.21.15.161:0/85615480 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f248c0dec40 con 0x7f249c06d3a0 2023-07-14T11:37:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.139+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3662001248 shutdown_connections 2023-07-14T11:37:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.139+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3662001248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 0x5631acdab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.139+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3662001248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdb6ad0 0x5631acdb8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.139+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3662001248 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb6590 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.139+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3662001248 >> 172.21.15.161:0/3662001248 conn(0x5631acda1da0 msgr2=0x5631acda76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.144+0000 7f249b59e700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f24840a14d0 0x7f24840a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.144+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3662001248 shutdown_connections 2023-07-14T11:37:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.144+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3662001248 wait complete. 2023-07-14T11:37:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.145+0000 7f24b4b03b80 1 -- start start 2023-07-14T11:37:57.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.145+0000 7f24b4b03b80 1 Processor -- start 2023-07-14T11:37:57.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.146+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdb6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.146+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.146+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdb3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.146+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acc9dac0 con 0x5631acdb6ad0 2023-07-14T11:37:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.146+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acc9d7c0 con 0x5631acdab2d0 2023-07-14T11:37:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.146+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acb81920 con 0x5631acdabbe0 2023-07-14T11:37:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.146+0000 7f24b4b03b80 10 client.0 _ll_register_callbacks cb 0x5631acc9b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:57.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.147+0000 7f249ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdb6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.147+0000 7f249ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdb6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:42006/0 (socket says 172.21.15.161:42006) 2023-07-14T11:37:57.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.147+0000 7f249ad9d700 1 -- 172.21.15.161:0/1731709158 learned_addr learned my addr 172.21.15.161:0/1731709158 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:57.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.150+0000 7f249b59e700 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdb3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.150+0000 7f249ad9d700 1 -- 172.21.15.161:0/1731709158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 msgr2=0x5631acdb37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:57.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.150+0000 7f249ad9d700 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.151+0000 7f249ad9d700 1 -- 172.21.15.161:0/1731709158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 msgr2=0x5631acdb3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:57.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.151+0000 7f249ad9d700 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdb3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.151+0000 7f249ad9d700 1 -- 172.21.15.161:0/1731709158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2494009a30 con 0x5631acdab2d0 2023-07-14T11:37:57.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.155+0000 7f249ad9d700 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdb6560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f249400bf10 tx=0x7f249400bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c975bf3f7cea25fe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:57.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.155+0000 7f24827fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:37:57.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.156+0000 7f24827fc700 1 -- 172.21.15.161:0/1731709158 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400e040 con 0x5631acdab2d0 2023-07-14T11:37:57.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.156+0000 7f24827fc700 1 -- 172.21.15.161:0/1731709158 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f249401b800 con 0x5631acdab2d0 2023-07-14T11:37:57.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.157+0000 7f249b59e700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f24840a14d0 0x7f24840a3980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2490009da0 tx=0x7f2490009190 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:57.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.157+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631acc574f0 con 0x5631acdab2d0 2023-07-14T11:37:57.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.157+0000 7f24827fc700 1 -- 172.21.15.161:0/1731709158 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f249401b9c0 con 0x5631acdab2d0 2023-07-14T11:37:57.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.157+0000 7f24b4b03b80 10 client.27317 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:57.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.157+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5631acb14c60 con 0x5631acdab2d0 2023-07-14T11:37:57.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24827fc700 1 -- 172.21.15.161:0/1731709158 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631acc574f0 con 0x5631acdab2d0 2023-07-14T11:37:57.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24827fc700 1 -- 172.21.15.161:0/1731709158 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f249402eba0 con 0x5631acdab2d0 2023-07-14T11:37:57.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24b4b03b80 10 client.27317 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:57.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24b4b03b80 10 client.27317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:57.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24b4b03b80 20 client.27317 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:57.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631acdc63b0 0x5631acdc87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24b4b03b80 4 client.27317 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:57.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.166+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631accf6eb0 con 0x5631acdc63b0 2023-07-14T11:37:57.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.167+0000 7f249a59c700 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631acdc63b0 0x5631acdc87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.176+0000 7f249a59c700 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631acdc63b0 0x5631acdc87c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52078b499278896 server_cookie=58009b123f88abb7 in_seq=0 out_seq=0 2023-07-14T11:37:57.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.176+0000 7f24827fc700 10 client.27317 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:57.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.188+0000 7f24827fc700 1 -- 172.21.15.161:0/1731709158 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5631accf6eb0 con 0x5631acdc63b0 2023-07-14T11:37:57.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.188+0000 7f24827fc700 10 client.27317 handle_command_reply: tid=0 2023-07-14T11:37:57.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.188+0000 7f24b4b03b80 1 client.27317 shutdown 2023-07-14T11:37:57.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.197+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 msgr2=0x5631acdb6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.197+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdb6560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f249400bf10 tx=0x7f249400bf40 comp rx=0 tx=0).stop 2023-07-14T11:37:57.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.198+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 shutdown_connections 2023-07-14T11:37:57.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.198+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631acdc63b0 0x5631acdc87c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.198+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdb3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.198+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdb6560 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.198+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/1731709158 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdb37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.198+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 >> 172.21.15.161:0/1731709158 conn(0x5631acda1da0 msgr2=0x5631acda7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:57.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.199+0000 7f24827fc700 0 client.27317 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:57.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.203+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 shutdown_connections 2023-07-14T11:37:57.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.210+0000 7f24b4b03b80 1 -- 172.21.15.161:0/1731709158 wait complete. 2023-07-14T11:37:57.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.217+0000 7f24b4b03b80 20 client.27317 trim_cache size 0 max 16384 2023-07-14T11:37:57.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.250+0000 7f24b4b03b80 1 Processor -- start 2023-07-14T11:37:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.252+0000 7f24b4b03b80 1 -- start start 2023-07-14T11:37:57.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.252+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 0x5631acdb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.252+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdabbe0 0x5631acdb30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.252+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdb6ad0 0x5631acdb3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.252+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acdb47d0 con 0x5631acdab2d0 2023-07-14T11:37:57.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.252+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acda3ec0 con 0x5631acdabbe0 2023-07-14T11:37:57.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.252+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acdab8f0 con 0x5631acdb6ad0 2023-07-14T11:37:57.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.254+0000 7f249a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdabbe0 0x5631acdb30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.254+0000 7f249a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdabbe0 0x5631acdb30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:42014/0 (socket says 172.21.15.161:42014) 2023-07-14T11:37:57.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.254+0000 7f249a59c700 1 -- 172.21.15.161:0/2076662124 learned_addr learned my addr 172.21.15.161:0/2076662124 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:57.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.256+0000 7f249b59e700 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdb6ad0 0x5631acdb3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.258+0000 7f249b59e700 1 -- 172.21.15.161:0/2076662124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdabbe0 msgr2=0x5631acdb30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.258+0000 7f249b59e700 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdabbe0 0x5631acdb30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.258+0000 7f249b59e700 1 -- 172.21.15.161:0/2076662124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 msgr2=0x5631acdb6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.258+0000 7f249b59e700 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 0x5631acdb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.259+0000 7f249b59e700 1 -- 172.21.15.161:0/2076662124 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631acb52250 con 0x5631acdb6ad0 2023-07-14T11:37:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.259+0000 7f249b59e700 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdb6ad0 0x5631acdb3610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2490012880 tx=0x7f2490004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a343ea2af278e97 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.260+0000 7f2489ffb700 1 -- 172.21.15.161:0/2076662124 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000c070 con 0x5631acdb6ad0 2023-07-14T11:37:57.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.260+0000 7f2489ffb700 1 -- 172.21.15.161:0/2076662124 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2490019bf0 con 0x5631acdb6ad0 2023-07-14T11:37:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.260+0000 7f2489ffb700 1 -- 172.21.15.161:0/2076662124 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490022d50 con 0x5631acdb6ad0 2023-07-14T11:37:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 -- 172.21.15.161:0/2076662124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdb6ad0 msgr2=0x5631acdb3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdb6ad0 0x5631acdb3610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2490012880 tx=0x7f2490004860 comp rx=0 tx=0).stop 2023-07-14T11:37:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 -- 172.21.15.161:0/2076662124 shutdown_connections 2023-07-14T11:37:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdab2d0 0x5631acdb6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdb6ad0 0x5631acdb3610 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/2076662124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdabbe0 0x5631acdb30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 -- 172.21.15.161:0/2076662124 >> 172.21.15.161:0/2076662124 conn(0x5631acda1da0 msgr2=0x5631acdc2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 -- 172.21.15.161:0/2076662124 shutdown_connections 2023-07-14T11:37:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.261+0000 7f24b4b03b80 1 -- 172.21.15.161:0/2076662124 wait complete. 2023-07-14T11:37:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.262+0000 7f24b4b03b80 1 -- start start 2023-07-14T11:37:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.262+0000 7f24b4b03b80 1 Processor -- start 2023-07-14T11:37:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.263+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdbbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.263+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdbc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.263+0000 7f24b4b03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdbc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.263+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631acdbdb20 con 0x5631acdb6ad0 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.263+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ace1a260 con 0x5631acdab2d0 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.263+0000 7f24b4b03b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ace1a3a0 con 0x5631acdabbe0 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.263+0000 7f24b4b03b80 10 client.0 _ll_register_callbacks cb 0x7f2494009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.267+0000 7f249b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdbc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.267+0000 7f249ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdbbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.267+0000 7f249b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdbc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40232/0 (socket says 172.21.15.161:40232) 2023-07-14T11:37:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.267+0000 7f249b59e700 1 -- 172.21.15.161:0/3831178260 learned_addr learned my addr 172.21.15.161:0/3831178260 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.267+0000 7f249a59c700 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdbc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.271+0000 7f249ad9d700 1 -- 172.21.15.161:0/3831178260 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 msgr2=0x5631acdbc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:57.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.271+0000 7f249ad9d700 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdbc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.271+0000 7f249ad9d700 1 -- 172.21.15.161:0/3831178260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 msgr2=0x5631acdbc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.271+0000 7f249ad9d700 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdbc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.271+0000 7f249ad9d700 1 -- 172.21.15.161:0/3831178260 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2490009e80 con 0x5631acdab2d0 2023-07-14T11:37:57.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.276+0000 7f249ad9d700 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdbbba0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f24940115a0 tx=0x7f2494024430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce954e311e2f118e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:57.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.276+0000 7f24827fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:37:57.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.276+0000 7f24827fc700 1 -- 172.21.15.161:0/3831178260 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400e030 con 0x5631acdab2d0 2023-07-14T11:37:57.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.276+0000 7f24827fc700 1 -- 172.21.15.161:0/3831178260 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f2494024580 con 0x5631acdab2d0 2023-07-14T11:37:57.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.276+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f249400bb70 con 0x5631acdab2d0 2023-07-14T11:37:57.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.277+0000 7f24827fc700 1 -- 172.21.15.161:0/3831178260 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2494012b20 con 0x5631acdab2d0 2023-07-14T11:37:57.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.277+0000 7f24b4b03b80 10 client.27323 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:57.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.277+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5631acb2a2b0 con 0x5631acdab2d0 2023-07-14T11:37:57.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.278+0000 7f24827fc700 1 -- 172.21.15.161:0/3831178260 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249400bb70 con 0x5631acdab2d0 2023-07-14T11:37:57.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.279+0000 7f24827fc700 1 -- 172.21.15.161:0/3831178260 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2494011bf0 con 0x5631acdab2d0 2023-07-14T11:37:57.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.279+0000 7f24b4b03b80 10 client.27323 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:57.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.279+0000 7f24b4b03b80 10 client.27323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:57.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.279+0000 7f24b4b03b80 20 client.27323 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:57.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.279+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631ace1b740 0x5631ace1db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.279+0000 7f24b4b03b80 4 client.27323 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:37:57.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.279+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5631accb9050 con 0x5631ace1b740 2023-07-14T11:37:57.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.284+0000 7f249a59c700 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631ace1b740 0x5631ace1db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.290+0000 7f249a59c700 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631ace1b740 0x5631ace1db50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f36100c4fe44da server_cookie=cedb765d99057433 in_seq=0 out_seq=0 2023-07-14T11:37:57.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.293+0000 7f24827fc700 10 client.27323 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:57.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.293+0000 7f24827fc700 1 -- 172.21.15.161:0/3831178260 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5631accb9050 con 0x5631ace1b740 2023-07-14T11:37:57.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.293+0000 7f24827fc700 10 client.27323 handle_command_reply: tid=0 2023-07-14T11:37:57.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.293+0000 7f24b4b03b80 1 client.27323 shutdown 2023-07-14T11:37:57.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.296+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 msgr2=0x5631acdbbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.296+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdbbba0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f24940115a0 tx=0x7f2494024430 comp rx=0 tx=0).stop 2023-07-14T11:37:57.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.300+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 shutdown_connections 2023-07-14T11:37:57.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.300+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5631ace1b740 0x5631ace1db50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.300+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631acdb6ad0 0x5631acdbc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.300+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631acdab2d0 0x5631acdbbba0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.300+0000 7f24b4b03b80 1 --2- 172.21.15.161:0/3831178260 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5631acdabbe0 0x5631acdbc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.300+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 >> 172.21.15.161:0/3831178260 conn(0x5631acda1da0 msgr2=0x5631acdb30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:57.299 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.300+0000 7f24827fc700 0 client.27323 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:57.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.309+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 shutdown_connections 2023-07-14T11:37:57.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.314+0000 7f24b4b03b80 1 -- 172.21.15.161:0/3831178260 wait complete. 2023-07-14T11:37:57.326 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:37:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:37:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:37:57.326 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:37:57.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.322+0000 7f24b4b03b80 20 client.27323 trim_cache size 0 max 16384 2023-07-14T11:37:57.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.330+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f24840a14d0 msgr2=0x7f24840a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.330+0000 7f24a11ff700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f24840a14d0 0x7f24840a3980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2490009da0 tx=0x7f2490009190 comp rx=0 tx=0).stop 2023-07-14T11:37:57.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.330+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06d3a0 msgr2=0x7f249c1173b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.330+0000 7f24a11ff700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06d3a0 0x7f249c1173b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f248c01a8f0 tx=0x7f248c003cd0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.332+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 shutdown_connections 2023-07-14T11:37:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.332+0000 7f24a11ff700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f24840a14d0 0x7f24840a3980 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.332+0000 7f24a11ff700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f249c10e280 0x7f249c1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.332+0000 7f24a11ff700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f249c06c9a0 0x7f249c116e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.332+0000 7f24a11ff700 1 --2- 172.21.15.161:0/85615480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f249c06d3a0 0x7f249c1173b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.332+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 >> 172.21.15.161:0/85615480 conn(0x7f249c06bf70 msgr2=0x7f249c070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:57.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.337+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 shutdown_connections 2023-07-14T11:37:57.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.337+0000 7f24a11ff700 1 -- 172.21.15.161:0/85615480 wait complete. 2023-07-14T11:37:57.382 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:37:57.383 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:37:57.602 INFO:tasks.workunit.client.0.smithi161.stdout: 10 94716 31.61 MB/sec execute 39 sec latency 122.874 ms 2023-07-14T11:37:57.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.834+0000 7f4611dff700 1 -- 172.21.15.161:0/1660906443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c110ac0 msgr2=0x7f460c112f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.834+0000 7f4611dff700 1 --2- 172.21.15.161:0/1660906443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c110ac0 0x7f460c112f00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f460400d490 tx=0x7f460400d7a0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.834+0000 7f4611dff700 1 -- 172.21.15.161:0/1660906443 shutdown_connections 2023-07-14T11:37:57.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.834+0000 7f4611dff700 1 --2- 172.21.15.161:0/1660906443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c110ac0 0x7f460c112f00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.834+0000 7f4611dff700 1 --2- 172.21.15.161:0/1660906443 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f460c107780 0x7f460c107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.834+0000 7f4611dff700 1 --2- 172.21.15.161:0/1660906443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c108090 0x7f460c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.834+0000 7f4611dff700 1 -- 172.21.15.161:0/1660906443 >> 172.21.15.161:0/1660906443 conn(0x7f460c06b090 msgr2=0x7f460c070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:57.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.836+0000 7f4611dff700 1 -- 172.21.15.161:0/1660906443 shutdown_connections 2023-07-14T11:37:57.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.837+0000 7f4611dff700 1 -- 172.21.15.161:0/1660906443 wait complete. 2023-07-14T11:37:57.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.838+0000 7f4611dff700 1 Processor -- start 2023-07-14T11:37:57.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.839+0000 7f4611dff700 1 -- start start 2023-07-14T11:37:57.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.839+0000 7f4611dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f460c107780 0x7f460c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.839+0000 7f4611dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c108090 0x7f460c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.839+0000 7f4611dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c110ac0 0x7f460c1b9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.839+0000 7f4611dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1b95f0 con 0x7f460c110ac0 2023-07-14T11:37:57.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.839+0000 7f4611dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1b9760 con 0x7f460c108090 2023-07-14T11:37:57.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.839+0000 7f4611dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1b9920 con 0x7f460c107780 2023-07-14T11:37:57.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.840+0000 7f460b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f460c107780 0x7f460c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.840+0000 7f460b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f460c107780 0x7f460c19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49878/0 (socket says 172.21.15.161:49878) 2023-07-14T11:37:57.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.840+0000 7f460b7fe700 1 -- 172.21.15.161:0/998987193 learned_addr learned my addr 172.21.15.161:0/998987193 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:57.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.841+0000 7f460affd700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c108090 0x7f460c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.844+0000 7f460bfff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c110ac0 0x7f460c1b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.845+0000 7f460bfff700 1 -- 172.21.15.161:0/998987193 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f460c107780 msgr2=0x7f460c19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.845+0000 7f460bfff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f460c107780 0x7f460c19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.845+0000 7f460bfff700 1 -- 172.21.15.161:0/998987193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c108090 msgr2=0x7f460c19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:57.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.845+0000 7f460bfff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c108090 0x7f460c19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:57.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.845+0000 7f460bfff700 1 -- 172.21.15.161:0/998987193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604008cf0 con 0x7f460c110ac0 2023-07-14T11:37:57.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.845+0000 7f460bfff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c110ac0 0x7f460c1b9020 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f460400d490 tx=0x7f4604006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:57.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.846+0000 7f4608ff9700 1 -- 172.21.15.161:0/998987193 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460401aa20 con 0x7f460c110ac0 2023-07-14T11:37:57.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.846+0000 7f4608ff9700 1 -- 172.21.15.161:0/998987193 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4604012070 con 0x7f460c110ac0 2023-07-14T11:37:57.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.846+0000 7f4608ff9700 1 -- 172.21.15.161:0/998987193 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460401c9b0 con 0x7f460c110ac0 2023-07-14T11:37:57.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.849+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c1b9ba0 con 0x7f460c110ac0 2023-07-14T11:37:57.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.849+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c1ba030 con 0x7f460c110ac0 2023-07-14T11:37:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.850+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c10b3e0 con 0x7f460c110ac0 2023-07-14T11:37:57.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.856+0000 7f4608ff9700 1 -- 172.21.15.161:0/998987193 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f460401cb10 con 0x7f460c110ac0 2023-07-14T11:37:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.858+0000 7f4608ff9700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45f40a1440 0x7f45f40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.858+0000 7f4608ff9700 1 -- 172.21.15.161:0/998987193 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f46040cd6e0 con 0x7f460c110ac0 2023-07-14T11:37:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.860+0000 7f460b7fe700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45f40a1440 0x7f45f40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:57.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.863+0000 7f4608ff9700 1 -- 172.21.15.161:0/998987193 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f46040967c0 con 0x7f460c110ac0 2023-07-14T11:37:57.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:57.864+0000 7f460b7fe700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45f40a1440 0x7f45f40a38f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009ad0 tx=0x7f45fc009210 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.307+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f460c067300 con 0x7f460c110ac0 2023-07-14T11:37:58.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.312+0000 7f4608ff9700 1 -- 172.21.15.161:0/998987193 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+10868 (secure 0 0 0) 0x7f4604099de0 con 0x7f460c110ac0 2023-07-14T11:37:58.314 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:37:58.314 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:37:50.523332+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:37:58.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.324+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45f40a1440 msgr2=0x7f45f40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.324+0000 7f4611dff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45f40a1440 0x7f45f40a38f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009ad0 tx=0x7f45fc009210 comp rx=0 tx=0).stop 2023-07-14T11:37:58.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.324+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c110ac0 msgr2=0x7f460c1b9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.324+0000 7f4611dff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c110ac0 0x7f460c1b9020 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f460400d490 tx=0x7f4604006070 comp rx=0 tx=0).stop 2023-07-14T11:37:58.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.325+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 shutdown_connections 2023-07-14T11:37:58.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.325+0000 7f4611dff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45f40a1440 0x7f45f40a38f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.325+0000 7f4611dff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c110ac0 0x7f460c1b9020 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.325+0000 7f4611dff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f460c107780 0x7f460c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.325+0000 7f4611dff700 1 --2- 172.21.15.161:0/998987193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460c108090 0x7f460c19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.325+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 >> 172.21.15.161:0/998987193 conn(0x7f460c06b090 msgr2=0x7f460c108520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:58.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.328+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 shutdown_connections 2023-07-14T11:37:58.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.328+0000 7f4611dff700 1 -- 172.21.15.161:0/998987193 wait complete. 2023-07-14T11:37:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 40 2023-07-14T11:37:58.354 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:37:58.602 INFO:tasks.workunit.client.0.smithi161.stdout: 10 95223 31.55 MB/sec execute 40 sec latency 116.304 ms 2023-07-14T11:37:58.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 -- 172.21.15.161:0/2553575019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8406d3a0 msgr2=0x7fda8411b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/2553575019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8406d3a0 0x7fda8411b470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0098f0 tx=0x7fda7c009c00 comp rx=0 tx=0).stop 2023-07-14T11:37:58.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 -- 172.21.15.161:0/2553575019 shutdown_connections 2023-07-14T11:37:58.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/2553575019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8410e280 0x7fda8410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/2553575019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8406d3a0 0x7fda8411b470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/2553575019 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 0x7fda8406ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 -- 172.21.15.161:0/2553575019 >> 172.21.15.161:0/2553575019 conn(0x7fda8406bf70 msgr2=0x7fda84070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:58.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 -- 172.21.15.161:0/2553575019 shutdown_connections 2023-07-14T11:37:58.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.622+0000 7fda8b9ff700 1 -- 172.21.15.161:0/2553575019 wait complete. 2023-07-14T11:37:58.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.623+0000 7fda8b9ff700 1 Processor -- start 2023-07-14T11:37:58.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8b9ff700 1 -- start start 2023-07-14T11:37:58.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8b9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 0x7fda841aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8b9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8406d3a0 0x7fda841aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8b9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8410e280 0x7fda841a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8b9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841a4f10 con 0x7fda8410e280 2023-07-14T11:37:58.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8b9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841a5080 con 0x7fda8406d3a0 2023-07-14T11:37:58.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8b9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841a5240 con 0x7fda8406c9a0 2023-07-14T11:37:58.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8979b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 0x7fda841aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8979b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 0x7fda841aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49900/0 (socket says 172.21.15.161:49900) 2023-07-14T11:37:58.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda8979b700 1 -- 172.21.15.161:0/4060887159 learned_addr learned my addr 172.21.15.161:0/4060887159 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:58.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.624+0000 7fda89f9c700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8410e280 0x7fda841a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda8979b700 1 -- 172.21.15.161:0/4060887159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8406d3a0 msgr2=0x7fda841aad70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:37:58.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda8979b700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8406d3a0 0x7fda841aad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda8979b700 1 -- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8410e280 msgr2=0x7fda841a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda8979b700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8410e280 0x7fda841a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda8979b700 1 -- 172.21.15.161:0/4060887159 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c009520 con 0x7fda8406c9a0 2023-07-14T11:37:58.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda8979b700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 0x7fda841aa830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda800060b0 tx=0x7fda8000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:58.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda7a7fc700 1 -- 172.21.15.161:0/4060887159 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ddd0 con 0x7fda8406c9a0 2023-07-14T11:37:58.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda841a5520 con 0x7fda8406c9a0 2023-07-14T11:37:58.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda7a7fc700 1 -- 172.21.15.161:0/4060887159 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda8000f040 con 0x7fda8406c9a0 2023-07-14T11:37:58.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.625+0000 7fda7a7fc700 1 -- 172.21.15.161:0/4060887159 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80014690 con 0x7fda8406c9a0 2023-07-14T11:37:58.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.626+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda841b1820 con 0x7fda8406c9a0 2023-07-14T11:37:58.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.631+0000 7fda7a7fc700 1 -- 172.21.15.161:0/4060887159 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fda80015400 con 0x7fda8406c9a0 2023-07-14T11:37:58.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.633+0000 7fda9f3d2b80 1 Processor -- start 2023-07-14T11:37:58.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.633+0000 7fda9f3d2b80 1 -- start start 2023-07-14T11:37:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c22196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2224590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c2226eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c211fcb0 con 0x55a7c2224ad0 2023-07-14T11:37:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2172a40 con 0x55a7c22192d0 2023-07-14T11:37:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c216bed0 con 0x55a7c2219be0 2023-07-14T11:37:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda89f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c22196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda8979b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2224590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda89f9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c22196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34482/0 (socket says 172.21.15.161:34482) 2023-07-14T11:37:58.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.634+0000 7fda89f9c700 1 -- 172.21.15.161:0/3605534808 learned_addr learned my addr 172.21.15.161:0/3605534808 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:58.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda7a7fc700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fda700a1340 0x7fda700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda8979b700 1 -- 172.21.15.161:0/3605534808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 msgr2=0x55a7c22196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:58.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda8979b700 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c22196a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda8979b700 1 -- 172.21.15.161:0/3605534808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 msgr2=0x55a7c2226eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:58.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda8979b700 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c2226eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda8979b700 1 -- 172.21.15.161:0/3605534808 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c1f99de0 con 0x55a7c2219be0 2023-07-14T11:37:58.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda89f9c700 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c22196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:58.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.635+0000 7fda8979b700 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2224590 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda800c9d30 tx=0x7fda800958a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d90338a5651d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:58.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.636+0000 7fda89f9c700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fda700a1340 0x7fda700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.636+0000 7fda7a7fc700 1 -- 172.21.15.161:0/4060887159 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fda800cc960 con 0x7fda8406c9a0 2023-07-14T11:37:58.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.636+0000 7fda6f7fe700 1 -- 172.21.15.161:0/3605534808 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80099480 con 0x55a7c2219be0 2023-07-14T11:37:58.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.636+0000 7fda6f7fe700 1 -- 172.21.15.161:0/3605534808 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda800cee90 con 0x55a7c2219be0 2023-07-14T11:37:58.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.636+0000 7fda6f7fe700 1 -- 172.21.15.161:0/3605534808 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800ab9d0 con 0x55a7c2219be0 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda89f9c700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fda700a1340 0x7fda700a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00a010 tx=0x7fda7c009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3605534808 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 msgr2=0x55a7c2224590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2224590 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda800c9d30 tx=0x7fda800958a0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3605534808 shutdown_connections 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c2226eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c22196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3605534808 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2224590 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3605534808 >> 172.21.15.161:0/3605534808 conn(0x55a7c220fda0 msgr2=0x55a7c22156a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:58.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3605534808 shutdown_connections 2023-07-14T11:37:58.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.637+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3605534808 wait complete. 2023-07-14T11:37:58.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.638+0000 7fda9f3d2b80 1 -- start start 2023-07-14T11:37:58.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.638+0000 7fda9f3d2b80 1 Processor -- start 2023-07-14T11:37:58.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.639+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c22192d0 0x55a7c2224490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.639+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c2221710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.639+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c2224ad0 0x55a7c2221d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.639+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c210bac0 con 0x55a7c2219be0 2023-07-14T11:37:58.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.639+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c210b7c0 con 0x55a7c2224ad0 2023-07-14T11:37:58.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.639+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c1fef920 con 0x55a7c22192d0 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.639+0000 7fda9f3d2b80 10 client.0 _ll_register_callbacks cb 0x55a7c2109760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.640+0000 7fda88f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c2221710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.640+0000 7fda88f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c2221710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42832/0 (socket says 172.21.15.161:42832) 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.640+0000 7fda88f9a700 1 -- 172.21.15.161:0/3711268451 learned_addr learned my addr 172.21.15.161:0/3711268451 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.640+0000 7fda89f9c700 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c2224ad0 0x55a7c2221d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.641+0000 7fda88f9a700 1 -- 172.21.15.161:0/3711268451 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c22192d0 msgr2=0x55a7c2224490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.641+0000 7fda88f9a700 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c22192d0 0x55a7c2224490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.641+0000 7fda88f9a700 1 -- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c2224ad0 msgr2=0x55a7c2221d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:58.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.641+0000 7fda88f9a700 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c2224ad0 0x55a7c2221d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.641+0000 7fda88f9a700 1 -- 172.21.15.161:0/3711268451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8411bfa0 con 0x55a7c2219be0 2023-07-14T11:37:58.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda88f9a700 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c2221710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e5d0 tx=0x7fda7400e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eada1e1a0077c48d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:58.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:37:58.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda6cff9700 1 -- 172.21.15.161:0/3711268451 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74005440 con 0x55a7c2219be0 2023-07-14T11:37:58.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda6cff9700 1 -- 172.21.15.161:0/3711268451 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fda74036070 con 0x55a7c2219be0 2023-07-14T11:37:58.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7c20c54f0 con 0x55a7c2219be0 2023-07-14T11:37:58.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda6cff9700 1 -- 172.21.15.161:0/3711268451 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda7400ee70 con 0x55a7c2219be0 2023-07-14T11:37:58.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda9f3d2b80 10 client.17292 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:58.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.642+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fda841b1820 con 0x55a7c2219be0 2023-07-14T11:37:58.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda6cff9700 1 -- 172.21.15.161:0/3711268451 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7c20c54f0 con 0x55a7c2219be0 2023-07-14T11:37:58.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda6cff9700 1 -- 172.21.15.161:0/3711268451 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fda7403b870 con 0x55a7c2219be0 2023-07-14T11:37:58.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda9f3d2b80 10 client.17292 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda9f3d2b80 10 client.17292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:37:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda9f3d2b80 20 client.17292 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c2232dc0 0x55a7c22351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda9f3d2b80 4 client.17292 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7c2164eb0 con 0x55a7c2232dc0 2023-07-14T11:37:58.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.643+0000 7fda8979b700 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c2232dc0 0x55a7c22351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.644+0000 7fda8979b700 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c2232dc0 0x55a7c22351d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c442302599c87524 server_cookie=e45054641a6b4d3c in_seq=0 out_seq=0 2023-07-14T11:37:58.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.644+0000 7fda6cff9700 10 client.17292 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:58.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda6cff9700 1 -- 172.21.15.161:0/3711268451 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a7c2164eb0 con 0x55a7c2232dc0 2023-07-14T11:37:58.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda6cff9700 10 client.17292 handle_command_reply: tid=0 2023-07-14T11:37:58.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 client.17292 shutdown 2023-07-14T11:37:58.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 msgr2=0x55a7c2221710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c2221710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e5d0 tx=0x7fda7400e8e0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 shutdown_connections 2023-07-14T11:37:58.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c2232dc0 0x55a7c22351d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c2224ad0 0x55a7c2221d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c22192d0 0x55a7c2224490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/3711268451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c2221710 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 >> 172.21.15.161:0/3711268451 conn(0x55a7c220fda0 msgr2=0x55a7c2215230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:58.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.649+0000 7fda6cff9700 0 client.17292 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:58.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.650+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 shutdown_connections 2023-07-14T11:37:58.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.650+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/3711268451 wait complete. 2023-07-14T11:37:58.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.651+0000 7fda9f3d2b80 20 client.17292 trim_cache size 0 max 16384 2023-07-14T11:37:58.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.672+0000 7fda9f3d2b80 1 Processor -- start 2023-07-14T11:37:58.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.672+0000 7fda9f3d2b80 1 -- start start 2023-07-14T11:37:58.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.672+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2224030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.672+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c221c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.672+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2224ad0 0x55a7c221cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c221add0 con 0x55a7c2219be0 2023-07-14T11:37:58.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c221fc70 con 0x55a7c22192d0 2023-07-14T11:37:58.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c221c3f0 con 0x55a7c2224ad0 2023-07-14T11:37:58.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2224ad0 0x55a7c221cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda8979b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2224030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2224ad0 0x55a7c221cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49924/0 (socket says 172.21.15.161:49924) 2023-07-14T11:37:58.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 -- 172.21.15.161:0/1627893418 learned_addr learned my addr 172.21.15.161:0/1627893418 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:58.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda88f9a700 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c221c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 -- 172.21.15.161:0/1627893418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 msgr2=0x55a7c2224030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:58.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2224030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 -- 172.21.15.161:0/1627893418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 msgr2=0x55a7c221c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c221c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.673+0000 7fda89f9c700 1 -- 172.21.15.161:0/1627893418 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c1f82c60 con 0x55a7c2224ad0 2023-07-14T11:37:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.674+0000 7fda89f9c700 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2224ad0 0x55a7c221cf30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009810 tx=0x7fda7c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37b81f91539c8dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.674+0000 7fda6ffff700 1 -- 172.21.15.161:0/1627893418 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c02c040 con 0x55a7c2224ad0 2023-07-14T11:37:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.674+0000 7fda6ffff700 1 -- 172.21.15.161:0/1627893418 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda7c01d030 con 0x55a7c2224ad0 2023-07-14T11:37:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.674+0000 7fda6ffff700 1 -- 172.21.15.161:0/1627893418 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c026980 con 0x55a7c2224ad0 2023-07-14T11:37:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.675+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1627893418 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2224ad0 msgr2=0x55a7c221cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.675+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2224ad0 0x55a7c221cf30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009810 tx=0x7fda7c018000 comp rx=0 tx=0).stop 2023-07-14T11:37:58.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.675+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1627893418 shutdown_connections 2023-07-14T11:37:58.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2224ad0 0x55a7c221cf30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2224030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1627893418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2219be0 0x55a7c221c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1627893418 >> 172.21.15.161:0/1627893418 conn(0x55a7c220fda0 msgr2=0x55a7c2239b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1627893418 shutdown_connections 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1627893418 wait complete. 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 -- start start 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.676+0000 7fda9f3d2b80 1 Processor -- start 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2228d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2229270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda9f3d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c22297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2287e30 con 0x55a7c2224ad0 2023-07-14T11:37:58.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2287fa0 con 0x55a7c22192d0 2023-07-14T11:37:58.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda9f3d2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2288160 con 0x55a7c2219be0 2023-07-14T11:37:58.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda9f3d2b80 10 client.0 _ll_register_callbacks cb 0x55a7c2109760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:58.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda88f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2229270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda8979b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2228d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda88f9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2229270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49934/0 (socket says 172.21.15.161:49934) 2023-07-14T11:37:58.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.677+0000 7fda88f9a700 1 -- 172.21.15.161:0/1097389734 learned_addr learned my addr 172.21.15.161:0/1097389734 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda89f9c700 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c22297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda8979b700 1 -- 172.21.15.161:0/1097389734 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 msgr2=0x55a7c2229270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda8979b700 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2229270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda8979b700 1 -- 172.21.15.161:0/1097389734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 msgr2=0x55a7c22297b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda8979b700 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c22297b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda8979b700 1 -- 172.21.15.161:0/1097389734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c01f040 con 0x55a7c22192d0 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda89f9c700 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c22297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:37:58.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda8979b700 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2228d30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fda80000f80 tx=0x7fda80015c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d33f15265b598e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:58.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:37:58.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda6cff9700 1 -- 172.21.15.161:0/1097389734 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800ccba0 con 0x55a7c22192d0 2023-07-14T11:37:58.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda6cff9700 1 -- 172.21.15.161:0/1097389734 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fda800996b0 con 0x55a7c22192d0 2023-07-14T11:37:58.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.678+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda800cee90 con 0x55a7c22192d0 2023-07-14T11:37:58.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.679+0000 7fda6cff9700 1 -- 172.21.15.161:0/1097389734 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda80099480 con 0x55a7c22192d0 2023-07-14T11:37:58.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.679+0000 7fda9f3d2b80 10 client.27344 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:58.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.679+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a7c1fc0250 con 0x55a7c22192d0 2023-07-14T11:37:58.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.679+0000 7fda6cff9700 1 -- 172.21.15.161:0/1097389734 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda800cee90 con 0x55a7c22192d0 2023-07-14T11:37:58.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.679+0000 7fda6cff9700 1 -- 172.21.15.161:0/1097389734 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fda800156b0 con 0x55a7c22192d0 2023-07-14T11:37:58.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda9f3d2b80 10 client.27344 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:58.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda9f3d2b80 10 client.27344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:37:58.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda9f3d2b80 20 client.27344 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:58.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c22894d0 0x55a7c228b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:58.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda9f3d2b80 4 client.27344 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:37:58.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a7c2127050 con 0x55a7c22894d0 2023-07-14T11:37:58.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda88f9a700 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c22894d0 0x55a7c228b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:58.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda88f9a700 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c22894d0 0x55a7c228b8b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5237a5d43df307dc server_cookie=d4bccddf1bf00a2a in_seq=0 out_seq=0 2023-07-14T11:37:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.680+0000 7fda6cff9700 10 client.27344 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.681+0000 7fda6cff9700 1 -- 172.21.15.161:0/1097389734 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a7c2127050 con 0x55a7c22894d0 2023-07-14T11:37:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.681+0000 7fda6cff9700 10 client.27344 handle_command_reply: tid=0 2023-07-14T11:37:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 client.27344 shutdown 2023-07-14T11:37:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 msgr2=0x55a7c2228d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2228d30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fda80000f80 tx=0x7fda80015c90 comp rx=0 tx=0).stop 2023-07-14T11:37:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 shutdown_connections 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x55a7c22894d0 0x55a7c228b8b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c2224ad0 0x55a7c22297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a7c22192d0 0x55a7c2228d30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 --2- 172.21.15.161:0/1097389734 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a7c2219be0 0x55a7c2229270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda6cff9700 0 client.27344 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 >> 172.21.15.161:0/1097389734 conn(0x55a7c220fda0 msgr2=0x55a7c221c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 shutdown_connections 2023-07-14T11:37:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 1 -- 172.21.15.161:0/1097389734 wait complete. 2023-07-14T11:37:58.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.682+0000 7fda9f3d2b80 20 client.27344 trim_cache size 0 max 16384 2023-07-14T11:37:58.686 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:37:58.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fda700a1340 msgr2=0x7fda700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fda700a1340 0x7fda700a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00a010 tx=0x7fda7c009260 comp rx=0 tx=0).stop 2023-07-14T11:37:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 msgr2=0x7fda841aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 0x7fda841aa830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda800060b0 tx=0x7fda8000d3b0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 shutdown_connections 2023-07-14T11:37:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fda700a1340 0x7fda700a37f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8410e280 0x7fda841a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fda8406c9a0 0x7fda841aa830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 --2- 172.21.15.161:0/4060887159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8406d3a0 0x7fda841aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.683+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 >> 172.21.15.161:0/4060887159 conn(0x7fda8406bf70 msgr2=0x7fda8406f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.684+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 shutdown_connections 2023-07-14T11:37:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:58.684+0000 7fda8b9ff700 1 -- 172.21.15.161:0/4060887159 wait complete. 2023-07-14T11:37:58.702 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:58 smithi161 ceph-mon[95526]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 890 op/s 2023-07-14T11:37:58.702 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:37:58 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/998987193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:37:58.704 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:37:58.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:58 smithi182 ceph-mon[129366]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 890 op/s 2023-07-14T11:37:58.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:37:58 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/998987193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:37:58.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:58 smithi175 ceph-mon[129081]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 890 op/s 2023-07-14T11:37:58.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:37:58 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/998987193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:37:59.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 -- 172.21.15.161:0/2326613200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 msgr2=0x7fd0bc103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/2326613200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc103870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009990 tx=0x7fd0ac009ca0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 -- 172.21.15.161:0/2326613200 shutdown_connections 2023-07-14T11:37:59.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/2326613200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0bc0fea90 0x7fd0bc0fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/2326613200 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0bc103e40 0x7fd0bc1042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/2326613200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc103870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 -- 172.21.15.161:0/2326613200 >> 172.21.15.161:0/2326613200 conn(0x7fd0bc0fa4b0 msgr2=0x7fd0bc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 -- 172.21.15.161:0/2326613200 shutdown_connections 2023-07-14T11:37:59.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.011+0000 7fd0c37ff700 1 -- 172.21.15.161:0/2326613200 wait complete. 2023-07-14T11:37:59.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.012+0000 7fd0c37ff700 1 Processor -- start 2023-07-14T11:37:59.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c37ff700 1 -- start start 2023-07-14T11:37:59.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c37ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0bc0fea90 0x7fd0bc06f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c37ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc06fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c37ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0bc103e40 0x7fd0bc06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c37ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc06af60 con 0x7fd0bc0fea90 2023-07-14T11:37:59.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c37ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc06b0d0 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c37ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc06b290 con 0x7fd0bc103e40 2023-07-14T11:37:59.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.013+0000 7fd0c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc06fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc06fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34536/0 (socket says 172.21.15.161:34536) 2023-07-14T11:37:59.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c0d9a700 1 -- 172.21.15.161:0/212983539 learned_addr learned my addr 172.21.15.161:0/212983539 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c159b700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0bc0fea90 0x7fd0bc06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c0d9a700 1 -- 172.21.15.161:0/212983539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0bc103e40 msgr2=0x7fd0bc06a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c1d9c700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0bc103e40 0x7fd0bc06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c0d9a700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0bc103e40 0x7fd0bc06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c0d9a700 1 -- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0bc0fea90 msgr2=0x7fd0bc06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c0d9a700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0bc0fea90 0x7fd0bc06f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.014 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.014+0000 7fd0c0d9a700 1 -- 172.21.15.161:0/212983539 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac0095f0 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.016+0000 7fd0c0d9a700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc06fc70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac005b40 tx=0x7fd0ac003710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.018+0000 7fd0b27fc700 1 -- 172.21.15.161:0/212983539 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac01d070 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.018+0000 7fd0b27fc700 1 -- 172.21.15.161:0/212983539 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0ac004090 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.018+0000 7fd0b27fc700 1 -- 172.21.15.161:0/212983539 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac022dc0 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.018+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc06b510 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.019+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc06e670 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.025+0000 7fd0b27fc700 1 -- 172.21.15.161:0/212983539 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd0ac022470 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.026+0000 7fd0d703eb80 1 Processor -- start 2023-07-14T11:37:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.026+0000 7fd0d703eb80 1 -- start start 2023-07-14T11:37:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.026+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.026+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 0x55a86df74590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.026+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df76eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.027+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86de6fcb0 con 0x55a86df74ad0 2023-07-14T11:37:59.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.027+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86dec2a40 con 0x55a86df692d0 2023-07-14T11:37:59.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.027+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86debbed0 con 0x55a86df69be0 2023-07-14T11:37:59.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.027+0000 7fd0c0d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df76eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.027+0000 7fd0c1d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.027+0000 7fd0c1d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34546/0 (socket says 172.21.15.161:34546) 2023-07-14T11:37:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.027+0000 7fd0c1d9c700 1 -- 172.21.15.161:0/3812107022 learned_addr learned my addr 172.21.15.161:0/3812107022 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0c1d9c700 1 -- 172.21.15.161:0/3812107022 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 msgr2=0x55a86df74590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0c1d9c700 1 --2- 172.21.15.161:0/3812107022 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 0x55a86df74590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0c1d9c700 1 -- 172.21.15.161:0/3812107022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 msgr2=0x55a86df76eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0c1d9c700 1 --2- 172.21.15.161:0/3812107022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df76eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0c1d9c700 1 -- 172.21.15.161:0/3812107022 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a86dce9de0 con 0x55a86df692d0 2023-07-14T11:37:59.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0c1d9c700 1 --2- 172.21.15.161:0/3812107022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df696a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4007d90 tx=0x7fd0b400c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f337c6a658ca6b3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0a77fe700 1 -- 172.21.15.161:0/3812107022 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400ccc0 con 0x55a86df692d0 2023-07-14T11:37:59.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0a77fe700 1 -- 172.21.15.161:0/3812107022 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0b400ce20 con 0x55a86df692d0 2023-07-14T11:37:59.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.028+0000 7fd0a77fe700 1 -- 172.21.15.161:0/3812107022 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4010680 con 0x55a86df692d0 2023-07-14T11:37:59.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0b27fc700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0a80a1410 0x7fd0a80a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3812107022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 msgr2=0x55a86df696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3812107022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df696a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4007d90 tx=0x7fd0b400c470 comp rx=0 tx=0).stop 2023-07-14T11:37:59.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3812107022 shutdown_connections 2023-07-14T11:37:59.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3812107022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df76eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3812107022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df696a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3812107022 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 0x55a86df74590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3812107022 >> 172.21.15.161:0/3812107022 conn(0x55a86df5fda0 msgr2=0x55a86df656a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0b27fc700 1 -- 172.21.15.161:0/212983539 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd0ac0ce450 con 0x7fd0bc0ff3a0 2023-07-14T11:37:59.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3812107022 shutdown_connections 2023-07-14T11:37:59.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.029+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3812107022 wait complete. 2023-07-14T11:37:59.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.030+0000 7fd0c159b700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0a80a1410 0x7fd0a80a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.030+0000 7fd0d703eb80 1 -- start start 2023-07-14T11:37:59.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.030+0000 7fd0d703eb80 1 Processor -- start 2023-07-14T11:37:59.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.030+0000 7fd0c159b700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0a80a1410 0x7fd0a80a38c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8006fd0 tx=0x7fd0b8008040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df72260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86de5bac0 con 0x55a86df692d0 2023-07-14T11:37:59.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86de5b7c0 con 0x55a86df69be0 2023-07-14T11:37:59.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86dd3f920 con 0x55a86df74ad0 2023-07-14T11:37:59.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0d703eb80 10 client.0 _ll_register_callbacks cb 0x55a86de59760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:59.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42886/0 (socket says 172.21.15.161:42886) 2023-07-14T11:37:59.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0c159b700 1 -- 172.21.15.161:0/1240896224 learned_addr learned my addr 172.21.15.161:0/1240896224 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.031+0000 7fd0c1d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df72260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0c0d9a700 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0c159b700 1 -- 172.21.15.161:0/1240896224 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 msgr2=0x55a86df72260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0c159b700 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df72260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0c159b700 1 -- 172.21.15.161:0/1240896224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 msgr2=0x55a86df71d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0c159b700 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df71d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0c159b700 1 -- 172.21.15.161:0/1240896224 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b40079c0 con 0x55a86df692d0 2023-07-14T11:37:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0c159b700 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df717e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8014cf0 tx=0x7fd0b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c079a27fe3a11c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:37:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/1240896224 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8012870 con 0x55a86df692d0 2023-07-14T11:37:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a86de154f0 con 0x55a86df692d0 2023-07-14T11:37:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.032+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/1240896224 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd0b80129d0 con 0x55a86df692d0 2023-07-14T11:37:59.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/1240896224 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0b8012b90 con 0x55a86df692d0 2023-07-14T11:37:59.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 10 client.17307 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:59.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a86dcd2c60 con 0x55a86df692d0 2023-07-14T11:37:59.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/1240896224 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a86de154f0 con 0x55a86df692d0 2023-07-14T11:37:59.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/1240896224 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd0b80263a0 con 0x55a86df692d0 2023-07-14T11:37:59.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 10 client.17307 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 10 client.17307 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:37:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 20 client.17307 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86df844c0 0x55a86df868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 4 client.17307 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.033+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a86deb4eb0 con 0x55a86df844c0 2023-07-14T11:37:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.034+0000 7fd0c0d9a700 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86df844c0 0x55a86df868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.034+0000 7fd0a4ff9700 10 client.17307 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.034+0000 7fd0c0d9a700 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86df844c0 0x55a86df868d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=561c82c27b28b2a6 server_cookie=6f2c9ffa78c52831 in_seq=0 out_seq=0 2023-07-14T11:37:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/1240896224 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a86deb4eb0 con 0x55a86df844c0 2023-07-14T11:37:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0a4ff9700 10 client.17307 handle_command_reply: tid=0 2023-07-14T11:37:59.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 client.17307 shutdown 2023-07-14T11:37:59.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 msgr2=0x55a86df717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df717e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8014cf0 tx=0x7fd0b8004750 comp rx=0 tx=0).stop 2023-07-14T11:37:59.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 shutdown_connections 2023-07-14T11:37:59.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86df844c0 0x55a86df868d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df717e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df72260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0a4ff9700 0 client.17307 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:59.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.036+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/1240896224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df71d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.037+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 >> 172.21.15.161:0/1240896224 conn(0x55a86df5fda0 msgr2=0x55a86df65230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.037+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 shutdown_connections 2023-07-14T11:37:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.037+0000 7fd0d703eb80 1 -- 172.21.15.161:0/1240896224 wait complete. 2023-07-14T11:37:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.037+0000 7fd0d703eb80 20 client.17307 trim_cache size 0 max 16384 2023-07-14T11:37:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 Processor -- start 2023-07-14T11:37:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 -- start start 2023-07-14T11:37:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 0x55a86df6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df6d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86df6a370 con 0x55a86df74ad0 2023-07-14T11:37:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86df6a540 con 0x55a86df692d0 2023-07-14T11:37:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86df66d30 con 0x55a86df69be0 2023-07-14T11:37:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0c159b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0c159b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df72110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34574/0 (socket says 172.21.15.161:34574) 2023-07-14T11:37:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0c159b700 1 -- 172.21.15.161:0/3105909170 learned_addr learned my addr 172.21.15.161:0/3105909170 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.052+0000 7fd0c1d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df6d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0c159b700 1 -- 172.21.15.161:0/3105909170 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 msgr2=0x55a86df6cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0c159b700 1 --2- 172.21.15.161:0/3105909170 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 0x55a86df6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0c159b700 1 -- 172.21.15.161:0/3105909170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 msgr2=0x55a86df6d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0c159b700 1 --2- 172.21.15.161:0/3105909170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df6d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0c159b700 1 -- 172.21.15.161:0/3105909170 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a86dd10250 con 0x55a86df692d0 2023-07-14T11:37:59.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0c159b700 1 --2- 172.21.15.161:0/3105909170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df72110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8000c00 tx=0x7fd0b800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44373137f64f3d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0a7fff700 1 -- 172.21.15.161:0/3105909170 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80264a0 con 0x55a86df692d0 2023-07-14T11:37:59.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0a7fff700 1 -- 172.21.15.161:0/3105909170 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0b8026920 con 0x55a86df692d0 2023-07-14T11:37:59.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.053+0000 7fd0a7fff700 1 -- 172.21.15.161:0/3105909170 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8012b80 con 0x55a86df692d0 2023-07-14T11:37:59.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.054+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3105909170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 msgr2=0x55a86df72110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.054+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3105909170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df72110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8000c00 tx=0x7fd0b800bf10 comp rx=0 tx=0).stop 2023-07-14T11:37:59.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.054+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3105909170 shutdown_connections 2023-07-14T11:37:59.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.054+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3105909170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df74ad0 0x55a86df6d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.054+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3105909170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df692d0 0x55a86df72110 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.054+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/3105909170 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df69be0 0x55a86df6cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.054+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3105909170 >> 172.21.15.161:0/3105909170 conn(0x55a86df5fda0 msgr2=0x55a86df80720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.055+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3105909170 shutdown_connections 2023-07-14T11:37:59.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.055+0000 7fd0d703eb80 1 -- 172.21.15.161:0/3105909170 wait complete. 2023-07-14T11:37:59.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.055+0000 7fd0d703eb80 1 -- start start 2023-07-14T11:37:59.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.055+0000 7fd0d703eb80 1 Processor -- start 2023-07-14T11:37:59.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df7b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0d703eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df79c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86df7a1c0 con 0x55a86df692d0 2023-07-14T11:37:59.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86df7a300 con 0x55a86df69be0 2023-07-14T11:37:59.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0d703eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86df7a4c0 con 0x55a86df74ad0 2023-07-14T11:37:59.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0d703eb80 10 client.0 _ll_register_callbacks cb 0x7fd0b8014690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:59.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df7b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0c1d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df79c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df7b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42894/0 (socket says 172.21.15.161:42894) 2023-07-14T11:37:59.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.056+0000 7fd0c159b700 1 -- 172.21.15.161:0/2183288815 learned_addr learned my addr 172.21.15.161:0/2183288815 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c0d9a700 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c1d9c700 1 -- 172.21.15.161:0/2183288815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 msgr2=0x55a86df796f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c1d9c700 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df796f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c1d9c700 1 -- 172.21.15.161:0/2183288815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 msgr2=0x55a86df7b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c1d9c700 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df7b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c1d9c700 1 -- 172.21.15.161:0/2183288815 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a86dcd2c60 con 0x55a86df74ad0 2023-07-14T11:37:59.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c159b700 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df7b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:37:59.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0c1d9c700 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df79c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400d010 tx=0x7fd0b4003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ba1a34a33727dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.057+0000 7fd0a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:59.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.058+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/2183288815 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4021040 con 0x55a86df74ad0 2023-07-14T11:37:59.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.058+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0b400ce20 con 0x55a86df74ad0 2023-07-14T11:37:59.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.058+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/2183288815 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd0b400d040 con 0x55a86df74ad0 2023-07-14T11:37:59.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.058+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/2183288815 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0b4011840 con 0x55a86df74ad0 2023-07-14T11:37:59.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.058+0000 7fd0d703eb80 10 client.27394 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:59.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.058+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a86dce82b0 con 0x55a86df74ad0 2023-07-14T11:37:59.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/2183288815 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0b400ce20 con 0x55a86df74ad0 2023-07-14T11:37:59.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/2183288815 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd0b4024030 con 0x55a86df74ad0 2023-07-14T11:37:59.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0d703eb80 10 client.27394 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:59.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0d703eb80 10 client.27394 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:37:59.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0d703eb80 20 client.27394 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:59.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86dfd9840 0x55a86dfdbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0d703eb80 4 client.27394 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:37:59.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a86de77050 con 0x55a86dfd9840 2023-07-14T11:37:59.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.059+0000 7fd0c159b700 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86dfd9840 0x55a86dfdbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.060+0000 7fd0a4ff9700 10 client.27394 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:59.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.060+0000 7fd0c159b700 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86dfd9840 0x55a86dfdbc50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6280c3a15d559cd2 server_cookie=8a2d995c46aac1df in_seq=0 out_seq=0 2023-07-14T11:37:59.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.060+0000 7fd0a4ff9700 1 -- 172.21.15.161:0/2183288815 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a86de77050 con 0x55a86dfd9840 2023-07-14T11:37:59.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.060+0000 7fd0a4ff9700 10 client.27394 handle_command_reply: tid=0 2023-07-14T11:37:59.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.060+0000 7fd0d703eb80 1 client.27394 shutdown 2023-07-14T11:37:59.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.060+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 msgr2=0x55a86df79c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.060+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df79c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400d010 tx=0x7fd0b4003cd0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 shutdown_connections 2023-07-14T11:37:59.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55a86dfd9840 0x55a86dfdbc50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a86df692d0 0x55a86df7b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a86df74ad0 0x55a86df79c30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 --2- 172.21.15.161:0/2183288815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86df69be0 0x55a86df796f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 >> 172.21.15.161:0/2183288815 conn(0x55a86df5fda0 msgr2=0x55a86df6cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0a4ff9700 0 client.27394 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:37:59.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 shutdown_connections 2023-07-14T11:37:59.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 1 -- 172.21.15.161:0/2183288815 wait complete. 2023-07-14T11:37:59.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.061+0000 7fd0d703eb80 20 client.27394 trim_cache size 0 max 16384 2023-07-14T11:37:59.076 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:37:59.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0a80a1410 msgr2=0x7fd0a80a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0a80a1410 0x7fd0a80a38c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8006fd0 tx=0x7fd0b8008040 comp rx=0 tx=0).stop 2023-07-14T11:37:59.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 msgr2=0x7fd0bc06fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc06fc70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac005b40 tx=0x7fd0ac003710 comp rx=0 tx=0).stop 2023-07-14T11:37:59.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 shutdown_connections 2023-07-14T11:37:59.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd0a80a1410 0x7fd0a80a38c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0bc0fea90 0x7fd0bc06f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd0bc103e40 0x7fd0bc06a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 --2- 172.21.15.161:0/212983539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0bc0ff3a0 0x7fd0bc06fc70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.062+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 >> 172.21.15.161:0/212983539 conn(0x7fd0bc0fa4b0 msgr2=0x7fd0bc101840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.063+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 shutdown_connections 2023-07-14T11:37:59.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.063+0000 7fd0c37ff700 1 -- 172.21.15.161:0/212983539 wait complete. 2023-07-14T11:37:59.082 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:37:59.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.337+0000 7f648f7ff700 1 -- 172.21.15.161:0/389620640 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6488100480 msgr2=0x7f648810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.337+0000 7f648f7ff700 1 --2- 172.21.15.161:0/389620640 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6488100480 0x7f648810c910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6480009f90 tx=0x7f648000b330 comp rx=0 tx=0).stop 2023-07-14T11:37:59.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.337+0000 7f648f7ff700 1 -- 172.21.15.161:0/389620640 shutdown_connections 2023-07-14T11:37:59.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.337+0000 7f648f7ff700 1 --2- 172.21.15.161:0/389620640 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6488100480 0x7f648810c910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.337+0000 7f648f7ff700 1 --2- 172.21.15.161:0/389620640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6488105ae0 0x7f6488105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.337+0000 7f648f7ff700 1 --2- 172.21.15.161:0/389620640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64880ffad0 0x7f64880fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.337+0000 7f648f7ff700 1 -- 172.21.15.161:0/389620640 >> 172.21.15.161:0/389620640 conn(0x7f64880fb5f0 msgr2=0x7f64880fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.338+0000 7f648f7ff700 1 -- 172.21.15.161:0/389620640 shutdown_connections 2023-07-14T11:37:59.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.338+0000 7f648f7ff700 1 -- 172.21.15.161:0/389620640 wait complete. 2023-07-14T11:37:59.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 Processor -- start 2023-07-14T11:37:59.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 -- start start 2023-07-14T11:37:59.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64880ffad0 0x7f6488197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6488100480 0x7f64881983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6488105ae0 0x7f648819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648819ce60 con 0x7f6488100480 2023-07-14T11:37:59.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648819cfd0 con 0x7f6488105ae0 2023-07-14T11:37:59.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648f7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648819d190 con 0x7f64880ffad0 2023-07-14T11:37:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648d59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64880ffad0 0x7f6488197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648d59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64880ffad0 0x7f6488197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50014/0 (socket says 172.21.15.161:50014) 2023-07-14T11:37:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.339+0000 7f648d59b700 1 -- 172.21.15.161:0/903204040 learned_addr learned my addr 172.21.15.161:0/903204040 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648dd9c700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6488105ae0 0x7f648819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648cd9a700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6488100480 0x7f64881983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648d59b700 1 -- 172.21.15.161:0/903204040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6488105ae0 msgr2=0x7f648819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648d59b700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6488105ae0 0x7f648819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648d59b700 1 -- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6488100480 msgr2=0x7f64881983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648d59b700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6488100480 0x7f64881983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648d59b700 1 -- 172.21.15.161:0/903204040 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64840095f0 con 0x7f64880ffad0 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648d59b700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64880ffad0 0x7f6488197e80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f648400ec10 tx=0x7f648400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f647e7fc700 1 -- 172.21.15.161:0/903204040 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648400cc90 con 0x7f64880ffad0 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f647e7fc700 1 -- 172.21.15.161:0/903204040 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64840044e0 con 0x7f64880ffad0 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6480009c40 con 0x7f64880ffad0 2023-07-14T11:37:59.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.340+0000 7f647e7fc700 1 -- 172.21.15.161:0/903204040 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484010430 con 0x7f64880ffad0 2023-07-14T11:37:59.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.341+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64881a4ad0 con 0x7f64880ffad0 2023-07-14T11:37:59.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.344+0000 7f647e7fc700 1 -- 172.21.15.161:0/903204040 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6484003fb0 con 0x7f64880ffad0 2023-07-14T11:37:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.345+0000 7f64a3213b80 1 Processor -- start 2023-07-14T11:37:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.345+0000 7f64a3213b80 1 -- start start 2023-07-14T11:37:59.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f64a3213b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29ab2cb0 con 0x556c29bac2d0 2023-07-14T11:37:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f64a3213b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29b05a40 con 0x556c29bb7ad0 2023-07-14T11:37:59.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f64a3213b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29afeed0 con 0x556c29bacbe0 2023-07-14T11:37:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648d59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f647e7fc700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f64740a12c0 0x7f64740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648dd9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34598/0 (socket says 172.21.15.161:34598) 2023-07-14T11:37:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 -- 172.21.15.161:0/3931148046 learned_addr learned my addr 172.21.15.161:0/3931148046 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f647e7fc700 1 -- 172.21.15.161:0/903204040 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6484014070 con 0x7f64880ffad0 2023-07-14T11:37:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 -- 172.21.15.161:0/3931148046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 msgr2=0x556c29bb7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 -- 172.21.15.161:0/3931148046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 msgr2=0x556c29bac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648cd9a700 1 -- 172.21.15.161:0/3931148046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c2992cde0 con 0x556c29bb7ad0 2023-07-14T11:37:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.346+0000 7f648dd9c700 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:37:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.347+0000 7f648dd9c700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f64740a12c0 0x7f64740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.347+0000 7f648cd9a700 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb9eb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6478009d20 tx=0x7f6478011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2325fb6298c802c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.347+0000 7f64737fe700 1 -- 172.21.15.161:0/3931148046 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478019630 con 0x556c29bb7ad0 2023-07-14T11:37:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.347+0000 7f64737fe700 1 -- 172.21.15.161:0/3931148046 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6478019c50 con 0x556c29bb7ad0 2023-07-14T11:37:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.347+0000 7f64737fe700 1 -- 172.21.15.161:0/3931148046 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800eb90 con 0x556c29bb7ad0 2023-07-14T11:37:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.347+0000 7f648dd9c700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f64740a12c0 0x7f64740a3770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6480009790 tx=0x7f6480009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.348+0000 7f64a3213b80 1 -- 172.21.15.161:0/3931148046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 msgr2=0x556c29bb9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.348+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb9eb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6478009d20 tx=0x7f6478011720 comp rx=0 tx=0).stop 2023-07-14T11:37:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.348+0000 7f64a3213b80 1 -- 172.21.15.161:0/3931148046 shutdown_connections 2023-07-14T11:37:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.348+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.348+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb9eb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.348+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3931148046 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.348+0000 7f64a3213b80 1 -- 172.21.15.161:0/3931148046 >> 172.21.15.161:0/3931148046 conn(0x556c29ba2da0 msgr2=0x556c29ba86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.349+0000 7f64a3213b80 1 -- 172.21.15.161:0/3931148046 shutdown_connections 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.349+0000 7f64a3213b80 1 -- 172.21.15.161:0/3931148046 wait complete. 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.349+0000 7f64a3213b80 1 -- start start 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.349+0000 7f64a3213b80 1 Processor -- start 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.350+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bb7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.350+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.350+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.350+0000 7f64a3213b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29a9eac0 con 0x556c29bac2d0 2023-07-14T11:37:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.350+0000 7f64a3213b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29a9e7c0 con 0x556c29bb7ad0 2023-07-14T11:37:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.350+0000 7f64a3213b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29982920 con 0x556c29bacbe0 2023-07-14T11:37:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.350+0000 7f64a3213b80 10 client.0 _ll_register_callbacks cb 0x556c29a9c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648dd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50022/0 (socket says 172.21.15.161:50022) 2023-07-14T11:37:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 -- 172.21.15.161:0/2865083715 learned_addr learned my addr 172.21.15.161:0/2865083715 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648d59b700 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bb7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 -- 172.21.15.161:0/2865083715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 msgr2=0x556c29bb4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 -- 172.21.15.161:0/2865083715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 msgr2=0x556c29bb7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bb7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 -- 172.21.15.161:0/2865083715 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6478009980 con 0x556c29bacbe0 2023-07-14T11:37:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.351+0000 7f648cd9a700 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb4790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f647801c040 tx=0x7f6478019a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6bee1d1c0bf2978 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f6470ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:59.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f6470ff9700 1 -- 172.21.15.161:0/2865083715 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478005450 con 0x556c29bacbe0 2023-07-14T11:37:59.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f6470ff9700 1 -- 172.21.15.161:0/2865083715 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6478019dc0 con 0x556c29bacbe0 2023-07-14T11:37:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c29a584f0 con 0x556c29bacbe0 2023-07-14T11:37:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f6470ff9700 1 -- 172.21.15.161:0/2865083715 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6478005ce0 con 0x556c29bacbe0 2023-07-14T11:37:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f64a3213b80 10 client.27406 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556c29915c60 con 0x556c29bacbe0 2023-07-14T11:37:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.352+0000 7f6470ff9700 1 -- 172.21.15.161:0/2865083715 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c29a584f0 con 0x556c29bacbe0 2023-07-14T11:37:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f6470ff9700 1 -- 172.21.15.161:0/2865083715 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f6478004a90 con 0x556c29bacbe0 2023-07-14T11:37:59.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f64a3213b80 10 client.27406 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f64a3213b80 10 client.27406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f64a3213b80 20 client.27406 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f64a3213b80 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29bcfae0 0x556c29bd1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f64a3213b80 4 client.27406 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:37:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c29af7eb0 con 0x556c29bcfae0 2023-07-14T11:37:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f648d59b700 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29bcfae0 0x556c29bd1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f648d59b700 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29bcfae0 0x556c29bd1ef0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cbcc09b6521abc4 server_cookie=f848755ad9034e1e in_seq=0 out_seq=0 2023-07-14T11:37:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.353+0000 7f6470ff9700 10 client.27406 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.357+0000 7f6470ff9700 1 -- 172.21.15.161:0/2865083715 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556c29af7eb0 con 0x556c29bcfae0 2023-07-14T11:37:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.357+0000 7f6470ff9700 10 client.27406 handle_command_reply: tid=0 2023-07-14T11:37:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.357+0000 7f64a3213b80 1 client.27406 shutdown 2023-07-14T11:37:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.357+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 msgr2=0x556c29bb4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.357+0000 7f64a3213b80 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb4790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f647801c040 tx=0x7f6478019a30 comp rx=0 tx=0).stop 2023-07-14T11:37:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 shutdown_connections 2023-07-14T11:37:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29bcfae0 0x556c29bd1ef0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bb7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bb4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 --2- 172.21.15.161:0/2865083715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bb4790 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 >> 172.21.15.161:0/2865083715 conn(0x556c29ba2da0 msgr2=0x556c29ba8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f6470ff9700 0 client.27406 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 shutdown_connections 2023-07-14T11:37:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 1 -- 172.21.15.161:0/2865083715 wait complete. 2023-07-14T11:37:59.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.358+0000 7f64a3213b80 20 client.27406 trim_cache size 0 max 16384 2023-07-14T11:37:59.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 Processor -- start 2023-07-14T11:37:59.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 -- start start 2023-07-14T11:37:59.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bac2d0 0x556c29bb4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bcbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bb7ad0 0x556c29bb7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29bb54d0 con 0x556c29bb7ad0 2023-07-14T11:37:59.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29bb9eb0 con 0x556c29bac2d0 2023-07-14T11:37:59.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.374+0000 7f64a3213b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29bcc4b0 con 0x556c29bacbe0 2023-07-14T11:37:59.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bcbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bcbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50038/0 (socket says 172.21.15.161:50038) 2023-07-14T11:37:59.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648cd9a700 1 -- 172.21.15.161:0/69287555 learned_addr learned my addr 172.21.15.161:0/69287555 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648dd9c700 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bb7ad0 0x556c29bb7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648dd9c700 1 -- 172.21.15.161:0/69287555 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 msgr2=0x556c29bcbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648dd9c700 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bcbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648dd9c700 1 -- 172.21.15.161:0/69287555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bac2d0 msgr2=0x556c29bb4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648dd9c700 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bac2d0 0x556c29bb4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.375+0000 7f648dd9c700 1 -- 172.21.15.161:0/69287555 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c29953250 con 0x556c29bb7ad0 2023-07-14T11:37:59.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.376+0000 7f648dd9c700 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bb7ad0 0x556c29bb7040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f648000b6c0 tx=0x7f648001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531fcb684c48a192 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.376+0000 7f6473fff700 1 -- 172.21.15.161:0/69287555 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648000e040 con 0x556c29bb7ad0 2023-07-14T11:37:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.376+0000 7f6473fff700 1 -- 172.21.15.161:0/69287555 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648000b9f0 con 0x556c29bb7ad0 2023-07-14T11:37:59.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.377+0000 7f6473fff700 1 -- 172.21.15.161:0/69287555 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800053f0 con 0x556c29bb7ad0 2023-07-14T11:37:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.377+0000 7f64a3213b80 1 -- 172.21.15.161:0/69287555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bb7ad0 msgr2=0x556c29bb7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.377+0000 7f64a3213b80 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bb7ad0 0x556c29bb7040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f648000b6c0 tx=0x7f648001e000 comp rx=0 tx=0).stop 2023-07-14T11:37:59.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 -- 172.21.15.161:0/69287555 shutdown_connections 2023-07-14T11:37:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bb7ad0 0x556c29bb7040 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bac2d0 0x556c29bb4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 --2- 172.21.15.161:0/69287555 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bcbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 -- 172.21.15.161:0/69287555 >> 172.21.15.161:0/69287555 conn(0x556c29ba2da0 msgr2=0x556c29bd6870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 -- 172.21.15.161:0/69287555 shutdown_connections 2023-07-14T11:37:59.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 -- 172.21.15.161:0/69287555 wait complete. 2023-07-14T11:37:59.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 -- start start 2023-07-14T11:37:59.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.378+0000 7f64a3213b80 1 Processor -- start 2023-07-14T11:37:59.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bc69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bc6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f64a3213b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bc2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f64a3213b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29bc36d0 con 0x556c29bac2d0 2023-07-14T11:37:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f64a3213b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29bc3840 con 0x556c29bb7ad0 2023-07-14T11:37:59.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f64a3213b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c29bc3a00 con 0x556c29bacbe0 2023-07-14T11:37:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f64a3213b80 10 client.0 _ll_register_callbacks cb 0x7f64840963f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:37:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f648cd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bc6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f648dd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bc2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f648dd9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bc2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34622/0 (socket says 172.21.15.161:34622) 2023-07-14T11:37:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f648dd9c700 1 -- 172.21.15.161:0/3884090206 learned_addr learned my addr 172.21.15.161:0/3884090206 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:37:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.379+0000 7f648d59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bc69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f648cd9a700 1 -- 172.21.15.161:0/3884090206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 msgr2=0x556c29bc2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f648cd9a700 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bc2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f648cd9a700 1 -- 172.21.15.161:0/3884090206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 msgr2=0x556c29bc69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:37:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f648cd9a700 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bc69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f648cd9a700 1 -- 172.21.15.161:0/3884090206 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648810d210 con 0x556c29bacbe0 2023-07-14T11:37:59.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f648cd9a700 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bc6f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f6478011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa65eb6388edc83e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:37:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f6470ff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:37:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f6470ff9700 1 -- 172.21.15.161:0/3884090206 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800ec00 con 0x556c29bacbe0 2023-07-14T11:37:59.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6478019c50 con 0x556c29bacbe0 2023-07-14T11:37:59.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.380+0000 7f6470ff9700 1 -- 172.21.15.161:0/3884090206 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6478000e40 con 0x556c29bacbe0 2023-07-14T11:37:59.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f6470ff9700 1 -- 172.21.15.161:0/3884090206 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6478019aa0 con 0x556c29bacbe0 2023-07-14T11:37:59.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 10 client.27412 fetch_fsmap learned FSMap version 40 2023-07-14T11:37:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556c2992b2b0 con 0x556c29bacbe0 2023-07-14T11:37:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f6470ff9700 1 -- 172.21.15.161:0/3884090206 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6478019c50 con 0x556c29bacbe0 2023-07-14T11:37:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f6470ff9700 1 -- 172.21.15.161:0/3884090206 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f647800ed60 con 0x556c29bacbe0 2023-07-14T11:37:59.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 10 client.27412 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:37:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 10 client.27412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:37:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 20 client.27412 populate_metadata read hostname 'smithi161' 2023-07-14T11:37:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29c24960 0x556c29c26d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:37:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 4 client.27412 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:37:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.381+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c29aba050 con 0x556c29c24960 2023-07-14T11:37:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.382+0000 7f648d59b700 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29c24960 0x556c29c26d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:37:59.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.382+0000 7f6470ff9700 10 client.27412 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:59.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.382+0000 7f648d59b700 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29c24960 0x556c29c26d70 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7924bdff51da6edd server_cookie=f7de73acfb82c765 in_seq=0 out_seq=0 2023-07-14T11:37:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.385+0000 7f6470ff9700 1 -- 172.21.15.161:0/3884090206 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c29aba050 con 0x556c29c24960 2023-07-14T11:37:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.385+0000 7f6470ff9700 10 client.27412 handle_command_reply: tid=0 2023-07-14T11:37:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.385+0000 7f64a3213b80 1 client.27412 shutdown 2023-07-14T11:37:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.385+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 msgr2=0x556c29bc6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.385+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bc6f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f6478011aa0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 shutdown_connections 2023-07-14T11:37:59.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556c29c24960 0x556c29c26d70 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c29bac2d0 0x556c29bc69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c29bb7ad0 0x556c29bc2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 --2- 172.21.15.161:0/3884090206 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556c29bacbe0 0x556c29bc6f30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 >> 172.21.15.161:0/3884090206 conn(0x556c29ba2da0 msgr2=0x556c29bb4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f6470ff9700 0 client.27412 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:37:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 shutdown_connections 2023-07-14T11:37:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 1 -- 172.21.15.161:0/3884090206 wait complete. 2023-07-14T11:37:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.386+0000 7f64a3213b80 20 client.27412 trim_cache size 0 max 16384 2023-07-14T11:37:59.400 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:37:59.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f64740a12c0 msgr2=0x7f64740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f64740a12c0 0x7f64740a3770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6480009790 tx=0x7f6480009920 comp rx=0 tx=0).stop 2023-07-14T11:37:59.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64880ffad0 msgr2=0x7f6488197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:37:59.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64880ffad0 0x7f6488197e80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f648400ec10 tx=0x7f648400ef20 comp rx=0 tx=0).stop 2023-07-14T11:37:59.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 shutdown_connections 2023-07-14T11:37:59.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f64740a12c0 0x7f64740a3770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6488105ae0 0x7f648819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f64880ffad0 0x7f6488197e80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 --2- 172.21.15.161:0/903204040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6488100480 0x7f64881983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:37:59.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 >> 172.21.15.161:0/903204040 conn(0x7f64880fb5f0 msgr2=0x7f648810ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:37:59.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 shutdown_connections 2023-07-14T11:37:59.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:37:59.387+0000 7f648f7ff700 1 -- 172.21.15.161:0/903204040 wait complete. 2023-07-14T11:37:59.602 INFO:tasks.workunit.client.0.smithi161.stdout: 10 95999 31.83 MB/sec execute 41 sec latency 78.069 ms 2023-07-14T11:38:00.407 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:38:00.407 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:38:00.602 INFO:tasks.workunit.client.0.smithi161.stdout: 10 96613 31.89 MB/sec execute 42 sec latency 95.862 ms 2023-07-14T11:38:00.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:00 smithi182 ceph-mon[129366]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 688 op/s 2023-07-14T11:38:00.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 -- 172.21.15.161:0/3411060234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070107a50 msgr2=0x7f4070107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 --2- 172.21.15.161:0/3411060234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070107a50 0x7f4070107ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f406000b0a0 tx=0x7f406000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 -- 172.21.15.161:0/3411060234 shutdown_connections 2023-07-14T11:38:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 --2- 172.21.15.161:0/3411060234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f407006d8e0 0x7f4070107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 --2- 172.21.15.161:0/3411060234 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4070112920 0x7f4070114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 --2- 172.21.15.161:0/3411060234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070107a50 0x7f4070107ec0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 -- 172.21.15.161:0/3411060234 >> 172.21.15.161:0/3411060234 conn(0x7f407006bf70 msgr2=0x7f4070070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.732+0000 7f4077dff700 1 -- 172.21.15.161:0/3411060234 shutdown_connections 2023-07-14T11:38:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.733+0000 7f4077dff700 1 -- 172.21.15.161:0/3411060234 wait complete. 2023-07-14T11:38:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.734+0000 7f4077dff700 1 Processor -- start 2023-07-14T11:38:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4077dff700 1 -- start start 2023-07-14T11:38:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4077dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f407006d8e0 0x7f4070108ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4077dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070107a50 0x7f40701090e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4077dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070112920 0x7f40701b9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4077dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701b96b0 con 0x7f4070107a50 2023-07-14T11:38:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4077dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701b97f0 con 0x7f4070112920 2023-07-14T11:38:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4077dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701b9960 con 0x7f407006d8e0 2023-07-14T11:38:00.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.736+0000 7f4075b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f407006d8e0 0x7f4070108ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.737+0000 7f4075b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f407006d8e0 0x7f4070108ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50072/0 (socket says 172.21.15.161:50072) 2023-07-14T11:38:00.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.737+0000 7f4075b9b700 1 -- 172.21.15.161:0/248708995 learned_addr learned my addr 172.21.15.161:0/248708995 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:00.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.737+0000 7f407539a700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070107a50 0x7f40701090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.737+0000 7f407539a700 1 -- 172.21.15.161:0/248708995 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f407006d8e0 msgr2=0x7f4070108ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.737+0000 7f407539a700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f407006d8e0 0x7f4070108ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.737+0000 7f407539a700 1 -- 172.21.15.161:0/248708995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070112920 msgr2=0x7f40701b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.738+0000 7f407639c700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070112920 0x7f40701b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.738+0000 7f407539a700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070112920 0x7f40701b9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.738+0000 7f407539a700 1 -- 172.21.15.161:0/248708995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4060009d00 con 0x7f4070107a50 2023-07-14T11:38:00.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.738+0000 7f407539a700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070107a50 0x7f40701090e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f406000b780 tx=0x7f4060007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:00.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.739+0000 7f406effd700 1 -- 172.21.15.161:0/248708995 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000e040 con 0x7f4070107a50 2023-07-14T11:38:00.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.739+0000 7f406effd700 1 -- 172.21.15.161:0/248708995 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f406000be40 con 0x7f4070107a50 2023-07-14T11:38:00.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.739+0000 7f406effd700 1 -- 172.21.15.161:0/248708995 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406001b6e0 con 0x7f4070107a50 2023-07-14T11:38:00.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.739+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40701b9be0 con 0x7f4070107a50 2023-07-14T11:38:00.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.739+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40701ba040 con 0x7f4070107a50 2023-07-14T11:38:00.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:00 smithi161 ceph-mon[95526]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 688 op/s 2023-07-14T11:38:00.744 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.746+0000 7f406effd700 1 -- 172.21.15.161:0/248708995 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f406001b840 con 0x7f4070107a50 2023-07-14T11:38:00.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 Processor -- start 2023-07-14T11:38:00.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 -- start start 2023-07-14T11:38:00.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d65fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fdc60 0x5619d6608610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d660af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d6503d30 con 0x5619d65fdc60 2023-07-14T11:38:00.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d6556ac0 con 0x5619d6608b50 2023-07-14T11:38:00.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.747+0000 7f408b795b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d654ff50 con 0x5619d65fd350 2023-07-14T11:38:00.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d65fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d65fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50082/0 (socket says 172.21.15.161:50082) 2023-07-14T11:38:00.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 -- 172.21.15.161:0/70978052 learned_addr learned my addr 172.21.15.161:0/70978052 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:00.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f4075b9b700 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fdc60 0x5619d6608610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 -- 172.21.15.161:0/70978052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 msgr2=0x5619d660af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407539a700 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d660af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d660af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 -- 172.21.15.161:0/70978052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fdc60 msgr2=0x5619d6608610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fdc60 0x5619d6608610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407639c700 1 -- 172.21.15.161:0/70978052 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619d637de60 con 0x5619d65fd350 2023-07-14T11:38:00.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f4075b9b700 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fdc60 0x5619d6608610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:38:00.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.748+0000 7f407539a700 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d660af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:38:00.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.749+0000 7f407639c700 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d65fd720 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4064009bf0 tx=0x7f4064009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404ab016cdb3b28c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:00.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.749+0000 7f4057fff700 1 -- 172.21.15.161:0/70978052 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064009770 con 0x5619d65fd350 2023-07-14T11:38:00.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.749+0000 7f406effd700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40580a1490 0x7f40580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.749+0000 7f4057fff700 1 -- 172.21.15.161:0/70978052 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4064011440 con 0x5619d65fd350 2023-07-14T11:38:00.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.749+0000 7f4057fff700 1 -- 172.21.15.161:0/70978052 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406401b670 con 0x5619d65fd350 2023-07-14T11:38:00.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.749+0000 7f406effd700 1 -- 172.21.15.161:0/248708995 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f40600cdeb0 con 0x7f4070107a50 2023-07-14T11:38:00.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f4075b9b700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40580a1490 0x7f40580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f408b795b80 1 -- 172.21.15.161:0/70978052 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 msgr2=0x5619d65fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f408b795b80 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d65fd720 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4064009bf0 tx=0x7f4064009f00 comp rx=0 tx=0).stop 2023-07-14T11:38:00.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f408b795b80 1 -- 172.21.15.161:0/70978052 shutdown_connections 2023-07-14T11:38:00.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f408b795b80 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d660af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f408b795b80 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d65fd720 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f408b795b80 1 --2- 172.21.15.161:0/70978052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fdc60 0x5619d6608610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.751+0000 7f408b795b80 1 -- 172.21.15.161:0/70978052 >> 172.21.15.161:0/70978052 conn(0x5619d65f3e20 msgr2=0x5619d65f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:00.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.752+0000 7f4075b9b700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40580a1490 0x7f40580a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4068009f00 tx=0x7f4068009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:00.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.752+0000 7f408b795b80 1 -- 172.21.15.161:0/70978052 shutdown_connections 2023-07-14T11:38:00.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.752+0000 7f408b795b80 1 -- 172.21.15.161:0/70978052 wait complete. 2023-07-14T11:38:00.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.752+0000 7f408b795b80 1 -- start start 2023-07-14T11:38:00.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.753+0000 7f408b795b80 1 Processor -- start 2023-07-14T11:38:00.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.753+0000 7f408b795b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fd350 0x5619d66085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.753+0000 7f408b795b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d6605860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f408b795b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f408b795b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d64efb40 con 0x5619d6608b50 2023-07-14T11:38:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f408b795b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d64ef840 con 0x5619d65fd350 2023-07-14T11:38:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f408b795b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d63d39a0 con 0x5619d65fdc60 2023-07-14T11:38:00.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f408b795b80 10 client.0 _ll_register_callbacks cb 0x5619d64ed7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:00.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f4075b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fd350 0x5619d66085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f4075b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fd350 0x5619d66085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34668/0 (socket says 172.21.15.161:34668) 2023-07-14T11:38:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f4075b9b700 1 -- 172.21.15.161:0/679901689 learned_addr learned my addr 172.21.15.161:0/679901689 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f407539a700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d6605860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f407639c700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.754+0000 7f407539a700 1 -- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fd350 msgr2=0x5619d66085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f407539a700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fd350 0x5619d66085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f407539a700 1 -- 172.21.15.161:0/679901689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 msgr2=0x5619d6605fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f407539a700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f407539a700 1 -- 172.21.15.161:0/679901689 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40640098d0 con 0x5619d65fdc60 2023-07-14T11:38:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f407639c700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:38:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f4075b9b700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fd350 0x5619d66085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:38:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f407539a700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d6605860 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f40600cb240 tx=0x7f4060009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96411256bb370a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f40557fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f40557fa700 1 -- 172.21.15.161:0/679901689 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060003e80 con 0x5619d65fdc60 2023-07-14T11:38:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619d64a9570 con 0x5619d65fdc60 2023-07-14T11:38:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f40557fa700 1 -- 172.21.15.161:0/679901689 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f406001de40 con 0x5619d65fdc60 2023-07-14T11:38:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.755+0000 7f40557fa700 1 -- 172.21.15.161:0/679901689 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f406009acd0 con 0x5619d65fdc60 2023-07-14T11:38:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 10 client.27424 fetch_fsmap learned FSMap version 40 2023-07-14T11:38:00.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5619d6366ce0 con 0x5619d65fdc60 2023-07-14T11:38:00.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f40557fa700 1 -- 172.21.15.161:0/679901689 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40600b7020 con 0x5619d65fdc60 2023-07-14T11:38:00.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f40557fa700 1 -- 172.21.15.161:0/679901689 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f406001baf0 con 0x5619d65fdc60 2023-07-14T11:38:00.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 10 client.27424 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:38:00.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 10 client.27424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:00.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 20 client.27424 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d66181f0 0x5619d661a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 4 client.27424 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.756+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619d6548f30 con 0x5619d66181f0 2023-07-14T11:38:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.757+0000 7f4075b9b700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d66181f0 0x5619d661a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.758+0000 7f4075b9b700 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d66181f0 0x5619d661a600 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b7cb6f563cecb6 server_cookie=36e44e1f31f96f36 in_seq=0 out_seq=0 2023-07-14T11:38:00.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.758+0000 7f40557fa700 10 client.27424 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:00.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.761+0000 7f40557fa700 1 -- 172.21.15.161:0/679901689 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5619d6548f30 con 0x5619d66181f0 2023-07-14T11:38:00.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.761+0000 7f40557fa700 10 client.27424 handle_command_reply: tid=0 2023-07-14T11:38:00.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.761+0000 7f408b795b80 1 client.27424 shutdown 2023-07-14T11:38:00.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.762+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 msgr2=0x5619d6605860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.762+0000 7f408b795b80 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d6605860 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f40600cb240 tx=0x7f4060009c90 comp rx=0 tx=0).stop 2023-07-14T11:38:00.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 shutdown_connections 2023-07-14T11:38:00.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d66181f0 0x5619d661a600 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fd350 0x5619d66085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 --2- 172.21.15.161:0/679901689 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d6605860 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 >> 172.21.15.161:0/679901689 conn(0x5619d65f3e20 msgr2=0x5619d65f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:00.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f40557fa700 0 client.27424 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:00.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 shutdown_connections 2023-07-14T11:38:00.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.770+0000 7f408b795b80 1 -- 172.21.15.161:0/679901689 wait complete. 2023-07-14T11:38:00.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.771+0000 7f408b795b80 20 client.27424 trim_cache size 0 max 16384 2023-07-14T11:38:00.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.798+0000 7f408b795b80 1 Processor -- start 2023-07-14T11:38:00.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.798+0000 7f408b795b80 1 -- start start 2023-07-14T11:38:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f408b795b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d660f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f408b795b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fdc60 0x5619d6605040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f408b795b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f408b795b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d6606e40 con 0x5619d6608b50 2023-07-14T11:38:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f408b795b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d6606c50 con 0x5619d65fdc60 2023-07-14T11:38:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f408b795b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d65f39f0 con 0x5619d65fd350 2023-07-14T11:38:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f4075b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d660f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f4075b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d660f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50106/0 (socket says 172.21.15.161:50106) 2023-07-14T11:38:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.799+0000 7f4075b9b700 1 -- 172.21.15.161:0/2575515304 learned_addr learned my addr 172.21.15.161:0/2575515304 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f407539a700 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fdc60 0x5619d6605040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f4075b9b700 1 -- 172.21.15.161:0/2575515304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fdc60 msgr2=0x5619d6605040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f4075b9b700 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fdc60 0x5619d6605040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f407639c700 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f4075b9b700 1 -- 172.21.15.161:0/2575515304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 msgr2=0x5619d6605580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f4075b9b700 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f4075b9b700 1 -- 172.21.15.161:0/2575515304 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619d65f5660 con 0x5619d65fd350 2023-07-14T11:38:00.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f407639c700 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:38:00.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.800+0000 7f4075b9b700 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d660f600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4068012ac0 tx=0x7f406801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5acc5cb6d2c1d76 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:00.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.801+0000 7f406cbff700 1 -- 172.21.15.161:0/2575515304 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068017070 con 0x5619d65fd350 2023-07-14T11:38:00.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.801+0000 7f406cbff700 1 -- 172.21.15.161:0/2575515304 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4068024040 con 0x5619d65fd350 2023-07-14T11:38:00.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.801+0000 7f406cbff700 1 -- 172.21.15.161:0/2575515304 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068032630 con 0x5619d65fd350 2023-07-14T11:38:00.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.802+0000 7f408b795b80 1 -- 172.21.15.161:0/2575515304 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 msgr2=0x5619d660f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.802+0000 7f408b795b80 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d660f600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4068012ac0 tx=0x7f406801b000 comp rx=0 tx=0).stop 2023-07-14T11:38:00.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.802+0000 7f408b795b80 1 -- 172.21.15.161:0/2575515304 shutdown_connections 2023-07-14T11:38:00.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.802+0000 7f408b795b80 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d6608b50 0x5619d6605580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.802+0000 7f408b795b80 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fd350 0x5619d660f600 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.802+0000 7f408b795b80 1 --2- 172.21.15.161:0/2575515304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d65fdc60 0x5619d6605040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.802+0000 7f408b795b80 1 -- 172.21.15.161:0/2575515304 >> 172.21.15.161:0/2575515304 conn(0x5619d65f3e20 msgr2=0x5619d65ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:00.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.803+0000 7f408b795b80 1 -- 172.21.15.161:0/2575515304 shutdown_connections 2023-07-14T11:38:00.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.803+0000 7f408b795b80 1 -- 172.21.15.161:0/2575515304 wait complete. 2023-07-14T11:38:00.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.803+0000 7f408b795b80 1 -- start start 2023-07-14T11:38:00.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 1 Processor -- start 2023-07-14T11:38:00.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fd350 0x5619d660e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d660ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d6665080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d6665630 con 0x5619d65fd350 2023-07-14T11:38:00.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d66657a0 con 0x5619d6608b50 2023-07-14T11:38:00.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d6665960 con 0x5619d65fdc60 2023-07-14T11:38:00.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.804+0000 7f408b795b80 10 client.0 _ll_register_callbacks cb 0x5619d64ed7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:00.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.805+0000 7f407639c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d6665080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.805+0000 7f407639c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d6665080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:34694/0 (socket says 172.21.15.161:34694) 2023-07-14T11:38:00.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.805+0000 7f407639c700 1 -- 172.21.15.161:0/1162015715 learned_addr learned my addr 172.21.15.161:0/1162015715 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:00.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.805+0000 7f407539a700 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d660ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.805+0000 7f4075b9b700 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fd350 0x5619d660e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f407639c700 1 -- 172.21.15.161:0/1162015715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 msgr2=0x5619d660ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f407639c700 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d660ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f407639c700 1 -- 172.21.15.161:0/1162015715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fd350 msgr2=0x5619d660e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:00.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f407639c700 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fd350 0x5619d660e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f407639c700 1 -- 172.21.15.161:0/1162015715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4068012e50 con 0x5619d6608b50 2023-07-14T11:38:00.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f407639c700 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d6665080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f406400fb90 tx=0x7f406400fbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70812aed1e4f6b27 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:00.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f40557fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:38:00.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f40557fa700 1 -- 172.21.15.161:0/1162015715 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064020070 con 0x5619d6608b50 2023-07-14T11:38:00.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f40557fa700 1 -- 172.21.15.161:0/1162015715 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f40640050b0 con 0x5619d6608b50 2023-07-14T11:38:00.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.806+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4064011440 con 0x5619d6608b50 2023-07-14T11:38:00.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.807+0000 7f40557fa700 1 -- 172.21.15.161:0/1162015715 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4064005270 con 0x5619d6608b50 2023-07-14T11:38:00.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.807+0000 7f408b795b80 10 client.27389 fetch_fsmap learned FSMap version 40 2023-07-14T11:38:00.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.807+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5619d63a42d0 con 0x5619d6608b50 2023-07-14T11:38:00.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.807+0000 7f40557fa700 1 -- 172.21.15.161:0/1162015715 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4064011440 con 0x5619d6608b50 2023-07-14T11:38:00.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f40557fa700 1 -- 172.21.15.161:0/1162015715 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f40640115b0 con 0x5619d6608b50 2023-07-14T11:38:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f408b795b80 10 client.27389 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:38:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f408b795b80 10 client.27389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f408b795b80 20 client.27389 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f408b795b80 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d666e310 0x5619d6670720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f408b795b80 4 client.27389 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:38:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5619d650b0d0 con 0x5619d666e310 2023-07-14T11:38:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.808+0000 7f4075b9b700 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d666e310 0x5619d6670720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:00.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.809+0000 7f4075b9b700 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d666e310 0x5619d6670720 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67091090ec1aaed9 server_cookie=1eb98eabd5a5a2e3 in_seq=0 out_seq=0 2023-07-14T11:38:00.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.809+0000 7f40557fa700 10 client.27389 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:00.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.824+0000 7f40557fa700 1 -- 172.21.15.161:0/1162015715 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5619d650b0d0 con 0x5619d666e310 2023-07-14T11:38:00.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.824+0000 7f40557fa700 10 client.27389 handle_command_reply: tid=0 2023-07-14T11:38:00.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.824+0000 7f408b795b80 1 client.27389 shutdown 2023-07-14T11:38:00.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.824+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 msgr2=0x5619d6665080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.824+0000 7f408b795b80 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d6665080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f406400fb90 tx=0x7f406400fbc0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.824 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:38:00.824 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:38:00.824 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "db756cfb-aaf8-4007-ada0-62667c4da737", 2023-07-14T11:38:00.825 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:38:00.825 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:38:00.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 shutdown_connections 2023-07-14T11:38:00.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5619d666e310 0x5619d6670720 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619d65fd350 0x5619d660e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619d6608b50 0x5619d6665080 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 --2- 172.21.15.161:0/1162015715 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5619d65fdc60 0x5619d660ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 >> 172.21.15.161:0/1162015715 conn(0x5619d65f3e20 msgr2=0x5619d65f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:00.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f40557fa700 0 client.27389 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:00.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 shutdown_connections 2023-07-14T11:38:00.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.825+0000 7f408b795b80 1 -- 172.21.15.161:0/1162015715 wait complete. 2023-07-14T11:38:00.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.826+0000 7f408b795b80 20 client.27389 trim_cache size 0 max 16384 2023-07-14T11:38:00.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.826+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40580a1490 msgr2=0x7f40580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.826+0000 7f4077dff700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40580a1490 0x7f40580a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4068009f00 tx=0x7f4068009260 comp rx=0 tx=0).stop 2023-07-14T11:38:00.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.826+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070107a50 msgr2=0x7f40701090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:00.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.826+0000 7f4077dff700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070107a50 0x7f40701090e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f406000b780 tx=0x7f4060007990 comp rx=0 tx=0).stop 2023-07-14T11:38:00.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 shutdown_connections 2023-07-14T11:38:00.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f40580a1490 0x7f40580a3940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070112920 0x7f40701b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f407006d8e0 0x7f4070108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 --2- 172.21.15.161:0/248708995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070107a50 0x7f40701090e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:00.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 >> 172.21.15.161:0/248708995 conn(0x7f407006bf70 msgr2=0x7f407019d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:00.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 shutdown_connections 2023-07-14T11:38:00.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:00.827+0000 7f4077dff700 1 -- 172.21.15.161:0/248708995 wait complete. 2023-07-14T11:38:00.856 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:38:00.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:00 smithi175 ceph-mon[129081]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 688 op/s 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 -- 172.21.15.161:0/503666083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3140ffad0 msgr2=0x7fa3140fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/503666083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3140ffad0 0x7fa3140fff40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa3040098f0 tx=0x7fa304009c00 comp rx=0 tx=0).stop 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 -- 172.21.15.161:0/503666083 shutdown_connections 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/503666083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa314105ae0 0x7fa314105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/503666083 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa314100480 0x7fa31410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/503666083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3140ffad0 0x7fa3140fff40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 -- 172.21.15.161:0/503666083 >> 172.21.15.161:0/503666083 conn(0x7fa3140fb5f0 msgr2=0x7fa3140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:01.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.144+0000 7fa31c3ff700 1 -- 172.21.15.161:0/503666083 shutdown_connections 2023-07-14T11:38:01.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.145+0000 7fa31c3ff700 1 -- 172.21.15.161:0/503666083 wait complete. 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.145+0000 7fa31c3ff700 1 Processor -- start 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.146+0000 7fa31c3ff700 1 -- start start 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.146+0000 7fa31c3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3140ffad0 0x7fa314072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.146+0000 7fa31c3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa314100480 0x7fa314072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.146+0000 7fa31c3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa314105ae0 0x7fa314073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.146+0000 7fa31c3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3141a8660 con 0x7fa314100480 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.146+0000 7fa31c3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3141a87a0 con 0x7fa3140ffad0 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.146+0000 7fa31c3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3141a88e0 con 0x7fa314105ae0 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31a99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa314105ae0 0x7fa314073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31a99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa314105ae0 0x7fa314073290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50124/0 (socket says 172.21.15.161:50124) 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31a99c700 1 -- 172.21.15.161:0/3033781739 learned_addr learned my addr 172.21.15.161:0/3033781739 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31999a700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa314100480 0x7fa314072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31999a700 1 -- 172.21.15.161:0/3033781739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa314105ae0 msgr2=0x7fa314073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31999a700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa314105ae0 0x7fa314073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31999a700 1 -- 172.21.15.161:0/3033781739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3140ffad0 msgr2=0x7fa314072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31999a700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3140ffad0 0x7fa314072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.147+0000 7fa31999a700 1 -- 172.21.15.161:0/3033781739 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009520 con 0x7fa314100480 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.148+0000 7fa31999a700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa314100480 0x7fa314072d50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa304009230 tx=0x7fa30400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.148+0000 7fa30b7fe700 1 -- 172.21.15.161:0/3033781739 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30401c070 con 0x7fa314100480 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.148+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3141a8b60 con 0x7fa314100480 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.148+0000 7fa30b7fe700 1 -- 172.21.15.161:0/3033781739 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30400bbd0 con 0x7fa314100480 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.148+0000 7fa30b7fe700 1 -- 172.21.15.161:0/3033781739 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304003970 con 0x7fa314100480 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.148+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3141a8f80 con 0x7fa314100480 2023-07-14T11:38:01.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.155+0000 7fa32fdd4b80 1 Processor -- start 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.155+0000 7fa32fdd4b80 1 -- start start 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.155+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x5620678616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.155+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.156+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.156+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067767cb0 con 0x5620678612d0 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.156+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620677baa40 con 0x56206787dae0 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.156+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620677b3ed0 con 0x562067861be0 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.156+0000 7fa31a99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x5620678616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.157+0000 7fa31a99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x5620678616a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43018/0 (socket says 172.21.15.161:43018) 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.157+0000 7fa31a99c700 1 -- 172.21.15.161:0/3702252266 learned_addr learned my addr 172.21.15.161:0/3702252266 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.157+0000 7fa31a99c700 1 -- 172.21.15.161:0/3702252266 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 msgr2=0x56206787d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.157+0000 7fa31a99c700 1 --2- 172.21.15.161:0/3702252266 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.157+0000 7fa31a99c700 1 -- 172.21.15.161:0/3702252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 msgr2=0x56206787fec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.157+0000 7fa31a99c700 1 --2- 172.21.15.161:0/3702252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787fec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.157+0000 7fa31a99c700 1 -- 172.21.15.161:0/3702252266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620675e1de0 con 0x5620678612d0 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.158+0000 7fa30b7fe700 1 -- 172.21.15.161:0/3033781739 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa304020030 con 0x7fa314100480 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.158+0000 7fa31a99c700 1 --2- 172.21.15.161:0/3702252266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x5620678616a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e1c0 tx=0x7fa30c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11689c9b909ba2d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.160+0000 7fa308bfe700 1 -- 172.21.15.161:0/3702252266 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c014480 con 0x5620678612d0 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.160+0000 7fa308bfe700 1 -- 172.21.15.161:0/3702252266 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30c00f040 con 0x5620678612d0 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.160+0000 7fa308bfe700 1 -- 172.21.15.161:0/3702252266 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c013860 con 0x5620678612d0 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.161+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/3702252266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 msgr2=0x5620678616a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.161+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/3702252266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x5620678616a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e1c0 tx=0x7fa30c009710 comp rx=0 tx=0).stop 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.161+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/3702252266 shutdown_connections 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.161+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/3702252266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x5620678616a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.162+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/3702252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.162+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/3702252266 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.162+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/3702252266 >> 172.21.15.161:0/3702252266 conn(0x562067857da0 msgr2=0x56206785d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.162+0000 7fa30b7fe700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa3000a1400 0x7fa3000a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.162+0000 7fa30b7fe700 1 -- 172.21.15.161:0/3033781739 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa3040cdf90 con 0x7fa314100480 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.163+0000 7fa31a19b700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa3000a1400 0x7fa3000a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.164+0000 7fa31a19b700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa3000a1400 0x7fa3000a38b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa310005950 tx=0x7fa3100058e0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.168+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/3702252266 shutdown_connections 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.168+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/3702252266 wait complete. 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.169+0000 7fa32fdd4b80 1 -- start start 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.169+0000 7fa32fdd4b80 1 Processor -- start 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.170+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x56206787d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.170+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.170+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.170+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067753ac0 con 0x5620678612d0 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.170+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620677537c0 con 0x56206787dae0 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.170+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067637920 con 0x562067861be0 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.170+0000 7fa32fdd4b80 10 client.0 _ll_register_callbacks cb 0x562067751760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.171+0000 7fa31a19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x56206787d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.171+0000 7fa31a19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x56206787d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43020/0 (socket says 172.21.15.161:43020) 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.171+0000 7fa31a19b700 1 -- 172.21.15.161:0/2079311596 learned_addr learned my addr 172.21.15.161:0/2079311596 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.171+0000 7fa31999a700 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.172+0000 7fa31a99c700 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.172+0000 7fa31999a700 1 -- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 msgr2=0x56206787ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.172+0000 7fa31999a700 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.172+0000 7fa31999a700 1 -- 172.21.15.161:0/2079311596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 msgr2=0x56206787d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.172+0000 7fa31999a700 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x56206787d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.172+0000 7fa31999a700 1 -- 172.21.15.161:0/2079311596 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c009d20 con 0x562067861be0 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.173+0000 7fa31999a700 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa3040cb320 tx=0x7fa30409a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c0e5427715be2f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.173+0000 7fa2fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.173+0000 7fa2fdffb700 1 -- 172.21.15.161:0/2079311596 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3040a30f0 con 0x562067861be0 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.173+0000 7fa2fdffb700 1 -- 172.21.15.161:0/2079311596 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa30409d070 con 0x562067861be0 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.175+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56206770d4f0 con 0x562067861be0 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.178+0000 7fa2fdffb700 1 -- 172.21.15.161:0/2079311596 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3040052b0 con 0x562067861be0 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.178+0000 7fa32fdd4b80 10 client.27436 fetch_fsmap learned FSMap version 40 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.178+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5620675cac60 con 0x562067861be0 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa2fdffb700 1 -- 172.21.15.161:0/2079311596 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56206770d4f0 con 0x562067861be0 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa2fdffb700 1 -- 172.21.15.161:0/2079311596 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa3040caf70 con 0x562067861be0 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa32fdd4b80 10 client.27436 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa32fdd4b80 10 client.27436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa32fdd4b80 20 client.27436 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206788cf70 0x56206788f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa32fdd4b80 4 client.27436 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.179+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620677aceb0 con 0x56206788cf70 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.185+0000 7fa31a19b700 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206788cf70 0x56206788f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.186+0000 7fa31a19b700 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206788cf70 0x56206788f380 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99448e1862b413b server_cookie=3bac97ff91acb083 in_seq=0 out_seq=0 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.186+0000 7fa2fdffb700 10 client.27436 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.190+0000 7fa2fdffb700 1 -- 172.21.15.161:0/2079311596 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620677aceb0 con 0x56206788cf70 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.190+0000 7fa2fdffb700 10 client.27436 handle_command_reply: tid=0 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 client.27436 shutdown 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 msgr2=0x56206787a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa3040cb320 tx=0x7fa30409a780 comp rx=0 tx=0).stop 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 shutdown_connections 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206788cf70 0x56206788f380 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x56206787d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa2fdffb700 0 client.27436 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.191+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2079311596 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x56206787a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.192+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 >> 172.21.15.161:0/2079311596 conn(0x562067857da0 msgr2=0x56206785d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.192+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 shutdown_connections 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.192+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2079311596 wait complete. 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.192+0000 7fa32fdd4b80 20 client.27436 trim_cache size 0 max 16384 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.222+0000 7fa32fdd4b80 1 Processor -- start 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.223+0000 7fa32fdd4b80 1 -- start start 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.223+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067879fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.223+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x562067875be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.223+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.223+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206787b700 con 0x5620678612d0 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.223+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620678631e0 con 0x56206787dae0 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.223+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067854fa0 con 0x562067861be0 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.224+0000 7fa31999a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x562067875be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.224+0000 7fa31999a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x562067875be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:50146/0 (socket says 172.21.15.161:50146) 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.224+0000 7fa31999a700 1 -- 172.21.15.161:0/2470060411 learned_addr learned my addr 172.21.15.161:0/2470060411 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.225+0000 7fa31a99c700 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.225+0000 7fa31a19b700 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067879fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.225+0000 7fa31999a700 1 -- 172.21.15.161:0/2470060411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 msgr2=0x56206787a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.225+0000 7fa31999a700 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.225+0000 7fa31999a700 1 -- 172.21.15.161:0/2470060411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 msgr2=0x562067879fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.225+0000 7fa31999a700 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067879fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.225+0000 7fa31999a700 1 -- 172.21.15.161:0/2470060411 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620678595e0 con 0x562067861be0 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.226+0000 7fa31999a700 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x562067875be0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa304009c40 tx=0x7fa3040992e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cff96590bd2564e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.228+0000 7fa3093ff700 1 -- 172.21.15.161:0/2470060411 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3040a30f0 con 0x562067861be0 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.228+0000 7fa3093ff700 1 -- 172.21.15.161:0/2470060411 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30409b130 con 0x562067861be0 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.228+0000 7fa3093ff700 1 -- 172.21.15.161:0/2470060411 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3040052b0 con 0x562067861be0 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.229+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2470060411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 msgr2=0x562067875be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.229+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x562067875be0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa304009c40 tx=0x7fa3040992e0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.234+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2470060411 shutdown_connections 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.234+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067879fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.234+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56206787dae0 0x56206787a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.234+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/2470060411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562067861be0 0x562067875be0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.234+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2470060411 >> 172.21.15.161:0/2470060411 conn(0x562067857da0 msgr2=0x56206787d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.234+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2470060411 shutdown_connections 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.235+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/2470060411 wait complete. 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.235+0000 7fa32fdd4b80 1 -- start start 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 1 Processor -- start 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067884360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562067861be0 0x562067882620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56206787dae0 0x562067882b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620678830a0 con 0x5620678612d0 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067883210 con 0x562067861be0 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620678833d0 con 0x56206787dae0 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.236+0000 7fa32fdd4b80 10 client.0 _ll_register_callbacks cb 0x7fa30409abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.239+0000 7fa31a19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067884360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.239+0000 7fa31a19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067884360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43032/0 (socket says 172.21.15.161:43032) 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.239+0000 7fa31a19b700 1 -- 172.21.15.161:0/1395882891 learned_addr learned my addr 172.21.15.161:0/1395882891 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.239+0000 7fa31999a700 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562067861be0 0x562067882620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.240+0000 7fa31a19b700 1 -- 172.21.15.161:0/1395882891 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56206787dae0 msgr2=0x562067882b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.240+0000 7fa31a19b700 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56206787dae0 0x562067882b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.240+0000 7fa31a19b700 1 -- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562067861be0 msgr2=0x562067882620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.240+0000 7fa31a19b700 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562067861be0 0x562067882620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.240+0000 7fa31a19b700 1 -- 172.21.15.161:0/1395882891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620675cac60 con 0x5620678612d0 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.240+0000 7fa31a19b700 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067884360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa3100095a0 tx=0x7fa310011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39efc262efc79858 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.241+0000 7fa2fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.241+0000 7fa2fdffb700 1 -- 172.21.15.161:0/1395882891 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310022040 con 0x5620678612d0 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.241+0000 7fa2fdffb700 1 -- 172.21.15.161:0/1395882891 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa310024860 con 0x5620678612d0 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.241+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa30c00f040 con 0x5620678612d0 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.242+0000 7fa2fdffb700 1 -- 172.21.15.161:0/1395882891 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa310024a20 con 0x5620678612d0 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.242+0000 7fa32fdd4b80 10 client.17361 fetch_fsmap learned FSMap version 40 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.242+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562067608250 con 0x5620678612d0 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa2fdffb700 1 -- 172.21.15.161:0/1395882891 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30c00f040 con 0x5620678612d0 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa2fdffb700 1 -- 172.21.15.161:0/1395882891 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa310034070 con 0x5620678612d0 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa32fdd4b80 10 client.17361 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa32fdd4b80 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa32fdd4b80 20 client.17361 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206786f930 0x562067871d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa32fdd4b80 4 client.17361 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.243+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56206776f050 con 0x56206786f930 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.244+0000 7fa31999a700 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206786f930 0x562067871d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.246+0000 7fa2fdffb700 10 client.17361 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.246+0000 7fa31999a700 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206786f930 0x562067871d40 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b4fd8bc2135d3f server_cookie=c13b982a5a1ad0d6 in_seq=0 out_seq=0 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.251+0000 7fa2fdffb700 1 -- 172.21.15.161:0/1395882891 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x56206776f050 con 0x56206786f930 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.251+0000 7fa2fdffb700 10 client.17361 handle_command_reply: tid=0 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.251+0000 7fa32fdd4b80 1 client.17361 shutdown 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.252+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 msgr2=0x562067884360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.252+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067884360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa3100095a0 tx=0x7fa310011000 comp rx=0 tx=0).stop 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.252+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 shutdown_connections 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.252+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56206786f930 0x562067871d40 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.252+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620678612d0 0x562067884360 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.252+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56206787dae0 0x562067882b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.253+0000 7fa32fdd4b80 1 --2- 172.21.15.161:0/1395882891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562067861be0 0x562067882620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.253+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 >> 172.21.15.161:0/1395882891 conn(0x562067857da0 msgr2=0x562067875be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.253+0000 7fa2fdffb700 0 client.17361 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.253+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 shutdown_connections 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.253+0000 7fa32fdd4b80 1 -- 172.21.15.161:0/1395882891 wait complete. 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.253+0000 7fa32fdd4b80 20 client.17361 trim_cache size 0 max 16384 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (77 inodes in the stack)", 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi161.stdout: "db756cfb-aaf8-4007-ada0-62667c4da737": { 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "db756cfb-aaf8-4007-ada0-62667c4da737", 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa3000a1400 msgr2=0x7fa3000a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa3000a1400 0x7fa3000a38b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa310005950 tx=0x7fa3100058e0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa314100480 msgr2=0x7fa314072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa314100480 0x7fa314072d50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa304009230 tx=0x7fa30400fa70 comp rx=0 tx=0).stop 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 shutdown_connections 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa3000a1400 0x7fa3000a38b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa314105ae0 0x7fa314073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3140ffad0 0x7fa314072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 --2- 172.21.15.161:0/3033781739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa314100480 0x7fa314072d50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.257+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 >> 172.21.15.161:0/3033781739 conn(0x7fa3140fb5f0 msgr2=0x7fa3140fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.258+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 shutdown_connections 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:01.258+0000 7fa31c3ff700 1 -- 172.21.15.161:0/3033781739 wait complete. 2023-07-14T11:38:01.284 INFO:tasks.cephfs.filesystem:scrub status for tag:db756cfb-aaf8-4007-ada0-62667c4da737 - {'path': '/', 'tag': 'db756cfb-aaf8-4007-ada0-62667c4da737', 'options': 'recursive,force'} 2023-07-14T11:38:01.602 INFO:tasks.workunit.client.0.smithi161.stdout: 10 96923 31.56 MB/sec execute 43 sec latency 239.137 ms 2023-07-14T11:38:01.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:01 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:38:01.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:01 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:01.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:01 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:01.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:01 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:38:01.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:01 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:01.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:01 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:01 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:01 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:01 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:02.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 97058 30.94 MB/sec execute 44 sec latency 371.024 ms 2023-07-14T11:38:02.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:02.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:02.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:02.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:02 smithi182 ceph-mon[129366]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 800 op/s 2023-07-14T11:38:02.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:02.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:02.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:02.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:02 smithi161 ceph-mon[95526]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 800 op/s 2023-07-14T11:38:02.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:02.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:02.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:02.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:02.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:02 smithi175 ceph-mon[129081]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 800 op/s 2023-07-14T11:38:03.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 97104 30.33 MB/sec execute 45 sec latency 326.033 ms 2023-07-14T11:38:04.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 97127 29.72 MB/sec execute 46 sec latency 335.022 ms 2023-07-14T11:38:04.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:04 smithi182 ceph-mon[129366]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 791 op/s 2023-07-14T11:38:04.712 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:04 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:38:04.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:04 smithi161 ceph-mon[95526]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 791 op/s 2023-07-14T11:38:04.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:04 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:38:04.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:04 smithi175 ceph-mon[129081]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 791 op/s 2023-07-14T11:38:04.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:04 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:38:05.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 97461 29.52 MB/sec execute 47 sec latency 358.736 ms 2023-07-14T11:38:06.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 97975 29.38 MB/sec execute 48 sec latency 220.608 ms 2023-07-14T11:38:06.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:06 smithi182 ceph-mon[129366]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 607 op/s 2023-07-14T11:38:06.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:06 smithi161 ceph-mon[95526]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 607 op/s 2023-07-14T11:38:06.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:06 smithi175 ceph-mon[129081]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 607 op/s 2023-07-14T11:38:07.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 98520 29.43 MB/sec execute 49 sec latency 289.963 ms 2023-07-14T11:38:07.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:07 smithi161 ceph-mon[95526]: osdmap e112: 12 total, 12 up, 12 in 2023-07-14T11:38:07.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:07 smithi175 ceph-mon[129081]: osdmap e112: 12 total, 12 up, 12 in 2023-07-14T11:38:07.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:07 smithi182 ceph-mon[129366]: osdmap e112: 12 total, 12 up, 12 in 2023-07-14T11:38:08.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 99159 29.58 MB/sec execute 50 sec latency 66.224 ms 2023-07-14T11:38:08.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:08 smithi182 ceph-mon[129366]: pgmap v498: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 614 op/s 2023-07-14T11:38:08.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:08 smithi161 ceph-mon[95526]: pgmap v498: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 614 op/s 2023-07-14T11:38:08.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:08 smithi175 ceph-mon[129081]: pgmap v498: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 614 op/s 2023-07-14T11:38:09.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 99606 29.42 MB/sec execute 51 sec latency 314.155 ms 2023-07-14T11:38:10.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 100197 29.35 MB/sec execute 52 sec latency 110.273 ms 2023-07-14T11:38:10.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:10 smithi161 ceph-mon[95526]: pgmap v499: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 614 op/s 2023-07-14T11:38:10.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:10 smithi175 ceph-mon[129081]: pgmap v499: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 614 op/s 2023-07-14T11:38:10.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:10 smithi182 ceph-mon[129366]: pgmap v499: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 614 op/s 2023-07-14T11:38:11.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 101026 29.63 MB/sec execute 53 sec latency 79.119 ms 2023-07-14T11:38:11.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:11 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:38:11.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:11 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:38:11.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:11 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:38:12.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 101833 30.14 MB/sec execute 54 sec latency 77.540 ms 2023-07-14T11:38:12.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:12 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:38:12.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:12 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:38:12.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:12 smithi175 ceph-mon[129081]: pgmap v500: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 600 op/s 2023-07-14T11:38:12.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:12 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:38:12.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:12 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:38:12.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:12 smithi182 ceph-mon[129366]: pgmap v500: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 600 op/s 2023-07-14T11:38:12.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:12 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:38:12.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:12 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:38:12.991 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:12 smithi161 ceph-mon[95526]: pgmap v500: 129 pgs: 9 active+clean+snaptrim_wait, 120 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 600 op/s 2023-07-14T11:38:13.604 INFO:tasks.workunit.client.0.smithi161.stdout: 10 102478 30.24 MB/sec execute 55 sec latency 69.861 ms 2023-07-14T11:38:14.269 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:14 smithi182 ceph-mon[129366]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 521 op/s 2023-07-14T11:38:14.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:14 smithi175 ceph-mon[129081]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 521 op/s 2023-07-14T11:38:14.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:14 smithi161 ceph-mon[95526]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 521 op/s 2023-07-14T11:38:14.603 INFO:tasks.workunit.client.0.smithi161.stdout: 10 102996 30.10 MB/sec execute 56 sec latency 93.051 ms 2023-07-14T11:38:15.604 INFO:tasks.workunit.client.0.smithi161.stdout: 10 103157 29.74 MB/sec execute 57 sec latency 377.453 ms 2023-07-14T11:38:16.604 INFO:tasks.workunit.client.0.smithi161.stdout: 10 103546 29.51 MB/sec execute 58 sec latency 116.593 ms 2023-07-14T11:38:16.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:16 smithi182 ceph-mon[129366]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 521 op/s 2023-07-14T11:38:16.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:16 smithi161 ceph-mon[95526]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 521 op/s 2023-07-14T11:38:16.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:16 smithi175 ceph-mon[129081]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 521 op/s 2023-07-14T11:38:17.604 INFO:tasks.workunit.client.0.smithi161.stdout: 10 104373 29.59 MB/sec execute 59 sec latency 74.538 ms 2023-07-14T11:38:18.604 INFO:tasks.workunit.client.0.smithi161.stdout: 10 105290 30.18 MB/sec execute 60 sec latency 68.234 ms 2023-07-14T11:38:18.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:18 smithi182 ceph-mon[129366]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.6 MiB/s wr, 514 op/s 2023-07-14T11:38:18.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:18 smithi182 ceph-mon[129366]: osdmap e113: 12 total, 12 up, 12 in 2023-07-14T11:38:18.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:18 smithi161 ceph-mon[95526]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.6 MiB/s wr, 514 op/s 2023-07-14T11:38:18.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:18 smithi161 ceph-mon[95526]: osdmap e113: 12 total, 12 up, 12 in 2023-07-14T11:38:18.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:18 smithi175 ceph-mon[129081]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.6 MiB/s wr, 514 op/s 2023-07-14T11:38:18.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:18 smithi175 ceph-mon[129081]: osdmap e113: 12 total, 12 up, 12 in 2023-07-14T11:38:19.604 INFO:tasks.workunit.client.0.smithi161.stdout: 10 106110 30.44 MB/sec execute 61 sec latency 51.613 ms 2023-07-14T11:38:20.605 INFO:tasks.workunit.client.0.smithi161.stdout: 10 106255 30.08 MB/sec execute 62 sec latency 418.627 ms 2023-07-14T11:38:20.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:20 smithi182 ceph-mon[129366]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.7 MiB/s wr, 487 op/s 2023-07-14T11:38:20.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:20 smithi161 ceph-mon[95526]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.7 MiB/s wr, 487 op/s 2023-07-14T11:38:20.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:20 smithi175 ceph-mon[129081]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.7 MiB/s wr, 487 op/s 2023-07-14T11:38:21.605 INFO:tasks.workunit.client.0.smithi161.stdout: 10 106371 29.66 MB/sec execute 63 sec latency 434.752 ms 2023-07-14T11:38:22.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 106619 29.44 MB/sec execute 64 sec latency 118.279 ms 2023-07-14T11:38:22.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:22 smithi161 ceph-mon[95526]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 13 MiB/s wr, 529 op/s 2023-07-14T11:38:22.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:22 smithi175 ceph-mon[129081]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 13 MiB/s wr, 529 op/s 2023-07-14T11:38:22.959 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:22 smithi182 ceph-mon[129366]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 13 MiB/s wr, 529 op/s 2023-07-14T11:38:23.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 107002 29.22 MB/sec execute 65 sec latency 121.715 ms 2023-07-14T11:38:24.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:24 smithi175 ceph-mon[129081]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-14T11:38:24.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:24 smithi182 ceph-mon[129366]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-14T11:38:24.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:24 smithi161 ceph-mon[95526]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-14T11:38:24.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 107777 29.32 MB/sec execute 66 sec latency 86.354 ms 2023-07-14T11:38:25.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 108522 29.66 MB/sec execute 67 sec latency 135.017 ms 2023-07-14T11:38:26.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 109086 29.58 MB/sec execute 68 sec latency 309.994 ms 2023-07-14T11:38:26.664 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:26 smithi175 ceph-mon[129081]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-14T11:38:26.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:26 smithi182 ceph-mon[129366]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-14T11:38:26.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:26 smithi161 ceph-mon[95526]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-14T11:38:27.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 109641 29.53 MB/sec execute 69 sec latency 100.090 ms 2023-07-14T11:38:28.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 110371 29.83 MB/sec execute 70 sec latency 100.909 ms 2023-07-14T11:38:28.709 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:28 smithi182 ceph-mon[129366]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 644 op/s 2023-07-14T11:38:28.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:28 smithi161 ceph-mon[95526]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 644 op/s 2023-07-14T11:38:28.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:28 smithi175 ceph-mon[129081]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 644 op/s 2023-07-14T11:38:29.609 INFO:tasks.workunit.client.0.smithi161.stdout: 10 111017 29.85 MB/sec execute 71 sec latency 106.950 ms 2023-07-14T11:38:30.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 111635 29.86 MB/sec execute 72 sec latency 79.101 ms 2023-07-14T11:38:30.664 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:30 smithi175 ceph-mon[129081]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 569 op/s 2023-07-14T11:38:30.713 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:30 smithi182 ceph-mon[129366]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 569 op/s 2023-07-14T11:38:30.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:30 smithi161 ceph-mon[95526]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 569 op/s 2023-07-14T11:38:31.285 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:38:31.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 112291 29.83 MB/sec execute 73 sec latency 81.766 ms 2023-07-14T11:38:31.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 -- 172.21.15.161:0/1292874986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 msgr2=0x7faa1410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/1292874986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 0x7faa1410c920 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009f90 tx=0x7faa0c00b330 comp rx=0 tx=0).stop 2023-07-14T11:38:31.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 -- 172.21.15.161:0/1292874986 shutdown_connections 2023-07-14T11:38:31.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/1292874986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 0x7faa1410c920 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/1292874986 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faa14105af0 0x7faa14105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/1292874986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 0x7faa140fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 -- 172.21.15.161:0/1292874986 >> 172.21.15.161:0/1292874986 conn(0x7faa140fb5e0 msgr2=0x7faa140fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:31.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 -- 172.21.15.161:0/1292874986 shutdown_connections 2023-07-14T11:38:31.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.615+0000 7faa1c3ff700 1 -- 172.21.15.161:0/1292874986 wait complete. 2023-07-14T11:38:31.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 Processor -- start 2023-07-14T11:38:31.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 -- start start 2023-07-14T11:38:31.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 0x7faa1419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 0x7faa1419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faa14105af0 0x7faa14195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa141961b0 con 0x7faa14100490 2023-07-14T11:38:31.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa14196320 con 0x7faa140ffae0 2023-07-14T11:38:31.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1c3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa141964e0 con 0x7faa14105af0 2023-07-14T11:38:31.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 0x7faa1419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 0x7faa1419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60080/0 (socket says 172.21.15.161:60080) 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1a99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faa14105af0 0x7faa14195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1a19b700 1 -- 172.21.15.161:0/91849021 learned_addr learned my addr 172.21.15.161:0/91849021 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.617+0000 7faa1999a700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 0x7faa1419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1a99c700 1 -- 172.21.15.161:0/91849021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 msgr2=0x7faa1419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1a99c700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 0x7faa1419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1a99c700 1 -- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 msgr2=0x7faa1419c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1a99c700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 0x7faa1419c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1a99c700 1 -- 172.21.15.161:0/91849021 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c009c40 con 0x7faa14105af0 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1a19b700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 0x7faa1419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1999a700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 0x7faa1419c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1a99c700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faa14105af0 0x7faa14195b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b6c0 tx=0x7faa0c0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa0b7fe700 1 -- 172.21.15.161:0/91849021 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00e040 con 0x7faa14105af0 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa0b7fe700 1 -- 172.21.15.161:0/91849021 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0c01d470 con 0x7faa14105af0 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa0b7fe700 1 -- 172.21.15.161:0/91849021 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0052a0 con 0x7faa14105af0 2023-07-14T11:38:31.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.618+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa14196760 con 0x7faa14105af0 2023-07-14T11:38:31.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.619+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa141a2ac0 con 0x7faa14105af0 2023-07-14T11:38:31.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.623+0000 7faa0b7fe700 1 -- 172.21.15.161:0/91849021 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7faa0c01d5e0 con 0x7faa14105af0 2023-07-14T11:38:31.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.624+0000 7faa2fc82b80 1 Processor -- start 2023-07-14T11:38:31.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa2fc82b80 1 -- start start 2023-07-14T11:38:31.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa0b7fe700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7faa000a1340 0x7faa000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb766e0 0x55bf4cb76ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4ca78cb0 con 0x55bf4cb76ff0 2023-07-14T11:38:31.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cacba40 con 0x55bf4cb766e0 2023-07-14T11:38:31.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cac4ed0 con 0x55bf4cb81ee0 2023-07-14T11:38:31.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.625+0000 7faa0b7fe700 1 -- 172.21.15.161:0/91849021 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7faa0c0ce470 con 0x7faa14105af0 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1999a700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7faa000a1340 0x7faa000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb766e0 0x55bf4cb76ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb766e0 0x55bf4cb76ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60090/0 (socket says 172.21.15.161:60090) 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 -- 172.21.15.161:0/679506201 learned_addr learned my addr 172.21.15.161:0/679506201 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1999a700 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a99c700 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 -- 172.21.15.161:0/679506201 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb81ee0 msgr2=0x55bf4cb842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 -- 172.21.15.161:0/679506201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 msgr2=0x55bf4cb819a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb819a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1a19b700 1 -- 172.21.15.161:0/679506201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf4c8f2de0 con 0x55bf4cb766e0 2023-07-14T11:38:31.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.626+0000 7faa1999a700 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb819a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:38:31.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.627+0000 7faa1a19b700 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb766e0 0x55bf4cb76ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faa100115f0 tx=0x7faa10011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d8d9da1c2a4399 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:31.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.627+0000 7faa1999a700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7faa000a1340 0x7faa000a37f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa040097c0 tx=0x7faa04009750 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:31.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.629+0000 7faa08bfe700 1 -- 172.21.15.161:0/679506201 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000f680 con 0x55bf4cb766e0 2023-07-14T11:38:31.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.629+0000 7faa08bfe700 1 -- 172.21.15.161:0/679506201 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa1000fca0 con 0x55bf4cb766e0 2023-07-14T11:38:31.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.629+0000 7faa08bfe700 1 -- 172.21.15.161:0/679506201 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10019d50 con 0x55bf4cb766e0 2023-07-14T11:38:31.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 -- 172.21.15.161:0/679506201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb766e0 msgr2=0x55bf4cb76ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb766e0 0x55bf4cb76ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faa100115f0 tx=0x7faa10011900 comp rx=0 tx=0).stop 2023-07-14T11:38:31.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 -- 172.21.15.161:0/679506201 shutdown_connections 2023-07-14T11:38:31.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb766e0 0x55bf4cb76ab0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/679506201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb819a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 -- 172.21.15.161:0/679506201 >> 172.21.15.161:0/679506201 conn(0x55bf4cb68da0 msgr2=0x55bf4cb6e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:31.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.630+0000 7faa2fc82b80 1 -- 172.21.15.161:0/679506201 shutdown_connections 2023-07-14T11:38:31.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.631+0000 7faa2fc82b80 1 -- 172.21.15.161:0/679506201 wait complete. 2023-07-14T11:38:31.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.631+0000 7faa2fc82b80 1 -- start start 2023-07-14T11:38:31.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 1 Processor -- start 2023-07-14T11:38:31.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4ca64ac0 con 0x55bf4cb81ee0 2023-07-14T11:38:31.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4ca647c0 con 0x55bf4cb76ff0 2023-07-14T11:38:31.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4c948920 con 0x55bf4cb766e0 2023-07-14T11:38:31.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa2fc82b80 10 client.0 _ll_register_callbacks cb 0x55bf4ca62760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:31.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41736/0 (socket says 172.21.15.161:41736) 2023-07-14T11:38:31.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.632+0000 7faa1a19b700 1 -- 172.21.15.161:0/872943677 learned_addr learned my addr 172.21.15.161:0/872943677 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:31.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a99c700 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a19b700 1 -- 172.21.15.161:0/872943677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 msgr2=0x55bf4cb7eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a19b700 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a19b700 1 -- 172.21.15.161:0/872943677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 msgr2=0x55bf4cb7f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a19b700 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a19b700 1 -- 172.21.15.161:0/872943677 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa10009dd0 con 0x55bf4cb766e0 2023-07-14T11:38:31.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a99c700 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:38:31.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7faa1a19b700 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faa1001af50 tx=0x7faa1001af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64b1e5982414a23 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:31.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.633+0000 7fa9fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:31.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7fa9fdffb700 1 -- 172.21.15.161:0/872943677 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10004220 con 0x55bf4cb766e0 2023-07-14T11:38:31.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf4ca1e4f0 con 0x55bf4cb766e0 2023-07-14T11:38:31.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7fa9fdffb700 1 -- 172.21.15.161:0/872943677 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7faa10004380 con 0x55bf4cb766e0 2023-07-14T11:38:31.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7fa9fdffb700 1 -- 172.21.15.161:0/872943677 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa10004540 con 0x55bf4cb766e0 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 10 client.27451 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55bf4c8dbc60 con 0x55bf4cb766e0 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7fa9fdffb700 1 -- 172.21.15.161:0/872943677 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf4ca1e4f0 con 0x55bf4cb766e0 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7fa9fdffb700 1 -- 172.21.15.161:0/872943677 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faa10005510 con 0x55bf4cb766e0 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 10 client.27451 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 10 client.27451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 20 client.27451 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:31.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cb917a0 0x55bf4cb93bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 4 client.27451 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:31.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.634+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf4cabdeb0 con 0x55bf4cb917a0 2023-07-14T11:38:31.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.637+0000 7faa1999a700 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cb917a0 0x55bf4cb93bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.638+0000 7faa1999a700 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cb917a0 0x55bf4cb93bb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b67a3bd54f8055 server_cookie=d15e8e17e80ec230 in_seq=0 out_seq=0 2023-07-14T11:38:31.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.638+0000 7fa9fdffb700 10 client.27451 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:31.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7fa9fdffb700 1 -- 172.21.15.161:0/872943677 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf4cabdeb0 con 0x55bf4cb917a0 2023-07-14T11:38:31.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7fa9fdffb700 10 client.27451 handle_command_reply: tid=0 2023-07-14T11:38:31.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 client.27451 shutdown 2023-07-14T11:38:31.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 msgr2=0x55bf4cb81930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faa1001af50 tx=0x7faa1001af80 comp rx=0 tx=0).stop 2023-07-14T11:38:31.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 shutdown_connections 2023-07-14T11:38:31.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cb917a0 0x55bf4cb93bb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7fa9fdffb700 0 client.27451 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:31.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81930 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/872943677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 >> 172.21.15.161:0/872943677 conn(0x55bf4cb68da0 msgr2=0x55bf4cb6e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:31.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 shutdown_connections 2023-07-14T11:38:31.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.642+0000 7faa2fc82b80 1 -- 172.21.15.161:0/872943677 wait complete. 2023-07-14T11:38:31.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.643+0000 7faa2fc82b80 20 client.27451 trim_cache size 0 max 16384 2023-07-14T11:38:31.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 Processor -- start 2023-07-14T11:38:31.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 -- start start 2023-07-14T11:38:31.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cb7eeb0 con 0x55bf4cb76ff0 2023-07-14T11:38:31.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cb76da0 con 0x55bf4cb81ee0 2023-07-14T11:38:31.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cb7a730 con 0x55bf4cb766e0 2023-07-14T11:38:31.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa1999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa1999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39114/0 (socket says 172.21.15.161:39114) 2023-07-14T11:38:31.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41740/0 (socket says 172.21.15.161:41740) 2023-07-14T11:38:31.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.664+0000 7faa1999a700 1 -- 172.21.15.161:0/3354577121 learned_addr learned my addr 172.21.15.161:0/3354577121 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:31.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa1a99c700 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa1a19b700 1 -- 172.21.15.161:0/3354577121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb81ee0 msgr2=0x55bf4cb7e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa1a19b700 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa1a19b700 1 -- 172.21.15.161:0/3354577121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 msgr2=0x55bf4cb7e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa1a19b700 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa1a19b700 1 -- 172.21.15.161:0/3354577121 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf4c919250 con 0x55bf4cb766e0 2023-07-14T11:38:31.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa1a19b700 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81460 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa1000eea0 tx=0x7faa10009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b725ae6c3928b9eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:31.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa093ff700 1 -- 172.21.15.161:0/3354577121 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000f680 con 0x55bf4cb766e0 2023-07-14T11:38:31.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa093ff700 1 -- 172.21.15.161:0/3354577121 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa10004940 con 0x55bf4cb766e0 2023-07-14T11:38:31.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.665+0000 7faa093ff700 1 -- 172.21.15.161:0/3354577121 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000ebd0 con 0x55bf4cb766e0 2023-07-14T11:38:31.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 -- 172.21.15.161:0/3354577121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 msgr2=0x55bf4cb81460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81460 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa1000eea0 tx=0x7faa10009310 comp rx=0 tx=0).stop 2023-07-14T11:38:31.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 -- 172.21.15.161:0/3354577121 shutdown_connections 2023-07-14T11:38:31.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb7e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb81460 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/3354577121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb7e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 -- 172.21.15.161:0/3354577121 >> 172.21.15.161:0/3354577121 conn(0x55bf4cb68da0 msgr2=0x55bf4cb8db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:31.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 -- 172.21.15.161:0/3354577121 shutdown_connections 2023-07-14T11:38:31.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.666+0000 7faa2fc82b80 1 -- 172.21.15.161:0/3354577121 wait complete. 2023-07-14T11:38:31.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.667+0000 7faa2fc82b80 1 -- start start 2023-07-14T11:38:31.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.667+0000 7faa2fc82b80 1 Processor -- start 2023-07-14T11:38:31.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb88910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa2fc82b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cb86ec0 con 0x55bf4cb81ee0 2023-07-14T11:38:31.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cb87030 con 0x55bf4cb76ff0 2023-07-14T11:38:31.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa2fc82b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4cb871f0 con 0x55bf4cb766e0 2023-07-14T11:38:31.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa2fc82b80 10 client.0 _ll_register_callbacks cb 0x55bf4ca62760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:31.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa1a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb883d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41748/0 (socket says 172.21.15.161:41748) 2023-07-14T11:38:31.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa1a19b700 1 -- 172.21.15.161:0/15487935 learned_addr learned my addr 172.21.15.161:0/15487935 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:31.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa1999a700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb88910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.668+0000 7faa1a99c700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7faa1999a700 1 -- 172.21.15.161:0/15487935 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 msgr2=0x55bf4cb883d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7faa1999a700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb883d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7faa1999a700 1 -- 172.21.15.161:0/15487935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 msgr2=0x55bf4cb867e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:31.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7faa1999a700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb867e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7faa1999a700 1 -- 172.21.15.161:0/15487935 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf4c8dbc60 con 0x55bf4cb76ff0 2023-07-14T11:38:31.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7faa1a19b700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb883d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:31.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7faa1999a700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb88910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55bf4cb802a0 tx=0x7faa040047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8c7585ecc0eafd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:31.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.669+0000 7fa9fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:38:31.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.670+0000 7fa9fdffb700 1 -- 172.21.15.161:0/15487935 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04035040 con 0x55bf4cb76ff0 2023-07-14T11:38:31.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.670+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa1000fca0 con 0x55bf4cb76ff0 2023-07-14T11:38:31.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.670+0000 7fa9fdffb700 1 -- 172.21.15.161:0/15487935 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7faa040395c0 con 0x55bf4cb76ff0 2023-07-14T11:38:31.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.670+0000 7fa9fdffb700 1 -- 172.21.15.161:0/15487935 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa04039780 con 0x55bf4cb76ff0 2023-07-14T11:38:31.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.670+0000 7faa2fc82b80 10 client.27416 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:31.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.670+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55bf4c8f12b0 con 0x55bf4cb76ff0 2023-07-14T11:38:31.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.670+0000 7fa9fdffb700 1 -- 172.21.15.161:0/15487935 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa1000fca0 con 0x55bf4cb76ff0 2023-07-14T11:38:31.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7fa9fdffb700 1 -- 172.21.15.161:0/15487935 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faa04043900 con 0x55bf4cb76ff0 2023-07-14T11:38:31.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7faa2fc82b80 10 client.27416 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:31.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7faa2fc82b80 10 client.27416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:31.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7faa2fc82b80 20 client.27416 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:31.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cbe6890 0x55bf4cbe8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7faa2fc82b80 4 client.27416 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:38:31.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf4ca80050 con 0x55bf4cbe6890 2023-07-14T11:38:31.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.671+0000 7faa1a19b700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cbe6890 0x55bf4cbe8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.672+0000 7faa1a19b700 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cbe6890 0x55bf4cbe8ca0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b921d7a1d71484 server_cookie=1906847891ac4fa7 in_seq=0 out_seq=0 2023-07-14T11:38:31.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.672+0000 7fa9fdffb700 10 client.27416 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:31.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.672+0000 7fa9fdffb700 1 -- 172.21.15.161:0/15487935 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bf4ca80050 con 0x55bf4cbe6890 2023-07-14T11:38:31.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.672+0000 7fa9fdffb700 10 client.27416 handle_command_reply: tid=0 2023-07-14T11:38:31.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.672+0000 7faa2fc82b80 1 client.27416 shutdown 2023-07-14T11:38:31.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 msgr2=0x55bf4cb88910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb88910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55bf4cb802a0 tx=0x7faa040047c0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 shutdown_connections 2023-07-14T11:38:31.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bf4cbe6890 0x55bf4cbe8ca0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf4cb81ee0 0x55bf4cb867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7fa9fdffb700 0 client.27416 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:31.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bf4cb766e0 0x55bf4cb883d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 --2- 172.21.15.161:0/15487935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf4cb76ff0 0x55bf4cb88910 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 >> 172.21.15.161:0/15487935 conn(0x55bf4cb68da0 msgr2=0x55bf4cb78710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:31.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 shutdown_connections 2023-07-14T11:38:31.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 1 -- 172.21.15.161:0/15487935 wait complete. 2023-07-14T11:38:31.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.673+0000 7faa2fc82b80 20 client.27416 trim_cache size 0 max 16384 2023-07-14T11:38:31.688 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:38:31.689 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:38:31.689 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:38:31.689 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:38:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7faa000a1340 msgr2=0x7faa000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7faa000a1340 0x7faa000a37f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa040097c0 tx=0x7faa04009750 comp rx=0 tx=0).stop 2023-07-14T11:38:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faa14105af0 msgr2=0x7faa14195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faa14105af0 0x7faa14195b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b6c0 tx=0x7faa0c0094b0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 shutdown_connections 2023-07-14T11:38:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7faa000a1340 0x7faa000a37f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7faa14105af0 0x7faa14195b40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa140ffae0 0x7faa1419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 --2- 172.21.15.161:0/91849021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa14100490 0x7faa1419c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 >> 172.21.15.161:0/91849021 conn(0x7faa140fb5e0 msgr2=0x7faa14106940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 shutdown_connections 2023-07-14T11:38:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.674+0000 7faa1c3ff700 1 -- 172.21.15.161:0/91849021 wait complete. 2023-07-14T11:38:31.698 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:38:31.698 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:38:31.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.994+0000 7fc256bff700 1 -- 172.21.15.161:0/1850296942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2501015b0 msgr2=0x7fc250109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.994+0000 7fc256bff700 1 --2- 172.21.15.161:0/1850296942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2501015b0 0x7fc250109aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009990 tx=0x7fc23c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.995+0000 7fc256bff700 1 -- 172.21.15.161:0/1850296942 shutdown_connections 2023-07-14T11:38:31.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.995+0000 7fc256bff700 1 --2- 172.21.15.161:0/1850296942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc250100ca0 0x7fc250101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.995+0000 7fc256bff700 1 --2- 172.21.15.161:0/1850296942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc25010a070 0x7fc25010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.995+0000 7fc256bff700 1 --2- 172.21.15.161:0/1850296942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2501015b0 0x7fc250109aa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.995+0000 7fc256bff700 1 -- 172.21.15.161:0/1850296942 >> 172.21.15.161:0/1850296942 conn(0x7fc2500fa490 msgr2=0x7fc2500fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:31.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.995+0000 7fc256bff700 1 -- 172.21.15.161:0/1850296942 shutdown_connections 2023-07-14T11:38:31.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.995+0000 7fc256bff700 1 -- 172.21.15.161:0/1850296942 wait complete. 2023-07-14T11:38:31.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.996+0000 7fc256bff700 1 Processor -- start 2023-07-14T11:38:31.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.996+0000 7fc256bff700 1 -- start start 2023-07-14T11:38:31.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.997+0000 7fc256bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc250100ca0 0x7fc250101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.997+0000 7fc256bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2501015b0 0x7fc250102050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.997+0000 7fc256bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc25010a070 0x7fc250102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:31.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.997+0000 7fc256bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2501a8580 con 0x7fc2501015b0 2023-07-14T11:38:31.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.997+0000 7fc256bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2501a86f0 con 0x7fc250100ca0 2023-07-14T11:38:31.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.997+0000 7fc256bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2501a88b0 con 0x7fc25010a070 2023-07-14T11:38:31.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.998+0000 7fc25519c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc25010a070 0x7fc250102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.998+0000 7fc25519c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc25010a070 0x7fc250102590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41760/0 (socket says 172.21.15.161:41760) 2023-07-14T11:38:31.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.998+0000 7fc25519c700 1 -- 172.21.15.161:0/3184208826 learned_addr learned my addr 172.21.15.161:0/3184208826 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:31.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.998+0000 7fc24ffff700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2501015b0 0x7fc250102050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.998+0000 7fc25499b700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc250100ca0 0x7fc250101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:31.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc25519c700 1 -- 172.21.15.161:0/3184208826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc250100ca0 msgr2=0x7fc250101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc25519c700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc250100ca0 0x7fc250101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc25519c700 1 -- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2501015b0 msgr2=0x7fc250102050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:31.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc25519c700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2501015b0 0x7fc250102050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:31.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc25519c700 1 -- 172.21.15.161:0/3184208826 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc248009c40 con 0x7fc25010a070 2023-07-14T11:38:31.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc25499b700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc250100ca0 0x7fc250101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:38:31.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc25519c700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc25010a070 0x7fc250102590 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c370 tx=0x7fc24800c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:31.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc24dffb700 1 -- 172.21.15.161:0/3184208826 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248017070 con 0x7fc25010a070 2023-07-14T11:38:31.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc24dffb700 1 -- 172.21.15.161:0/3184208826 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc24800f040 con 0x7fc25010a070 2023-07-14T11:38:32.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c0095f0 con 0x7fc25010a070 2023-07-14T11:38:32.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:31.999+0000 7fc24dffb700 1 -- 172.21.15.161:0/3184208826 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24800e050 con 0x7fc25010a070 2023-07-14T11:38:32.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.000+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2501a8f00 con 0x7fc25010a070 2023-07-14T11:38:32.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.001+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc250067300 con 0x7fc25010a070 2023-07-14T11:38:32.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.003+0000 7fc24dffb700 1 -- 172.21.15.161:0/3184208826 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc248004520 con 0x7fc25010a070 2023-07-14T11:38:32.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.005+0000 7fc24dffb700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2400a13c0 0x7fc2400a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.005+0000 7fc24dffb700 1 -- 172.21.15.161:0/3184208826 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fc2480cc960 con 0x7fc25010a070 2023-07-14T11:38:32.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.005+0000 7fc25499b700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2400a13c0 0x7fc2400a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.006+0000 7fc25499b700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2400a13c0 0x7fc2400a3870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc244005d90 tx=0x7fc244005d00 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:32.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.011+0000 7fc24dffb700 1 -- 172.21.15.161:0/3184208826 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc2480959b0 con 0x7fc25010a070 2023-07-14T11:38:32.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.371+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc250103040 con 0x7fc25010a070 2023-07-14T11:38:32.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.372+0000 7fc24dffb700 1 -- 172.21.15.161:0/3184208826 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10867 (secure 0 0 0) 0x7fc248098fd0 con 0x7fc25010a070 2023-07-14T11:38:32.371 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:38:32.371 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:38:03.446499+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:38:32.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.376+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2400a13c0 msgr2=0x7fc2400a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.376+0000 7fc256bff700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2400a13c0 0x7fc2400a3870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc244005d90 tx=0x7fc244005d00 comp rx=0 tx=0).stop 2023-07-14T11:38:32.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc25010a070 msgr2=0x7fc250102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc25010a070 0x7fc250102590 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c370 tx=0x7fc24800c680 comp rx=0 tx=0).stop 2023-07-14T11:38:32.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 shutdown_connections 2023-07-14T11:38:32.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2400a13c0 0x7fc2400a3870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc25010a070 0x7fc250102590 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc250100ca0 0x7fc250101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 --2- 172.21.15.161:0/3184208826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2501015b0 0x7fc250102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 >> 172.21.15.161:0/3184208826 conn(0x7fc2500fa490 msgr2=0x7fc2500fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:32.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 shutdown_connections 2023-07-14T11:38:32.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.377+0000 7fc256bff700 1 -- 172.21.15.161:0/3184208826 wait complete. 2023-07-14T11:38:32.377 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 41 2023-07-14T11:38:32.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:38:32.608 INFO:tasks.workunit.client.0.smithi161.stdout: 10 113169 30.21 MB/sec execute 74 sec latency 74.996 ms 2023-07-14T11:38:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.693+0000 7f5483bff700 1 -- 172.21.15.161:0/959223942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 msgr2=0x7f547c10b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.693+0000 7f5483bff700 1 --2- 172.21.15.161:0/959223942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c10b810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f54740098f0 tx=0x7f5474009c00 comp rx=0 tx=0).stop 2023-07-14T11:38:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.693+0000 7f5483bff700 1 -- 172.21.15.161:0/959223942 shutdown_connections 2023-07-14T11:38:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.694+0000 7f5483bff700 1 --2- 172.21.15.161:0/959223942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f547c1049e0 0x7f547c104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.694+0000 7f5483bff700 1 --2- 172.21.15.161:0/959223942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c10b810 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.694+0000 7f5483bff700 1 --2- 172.21.15.161:0/959223942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547c0fe9d0 0x7f547c0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.694+0000 7f5483bff700 1 -- 172.21.15.161:0/959223942 >> 172.21.15.161:0/959223942 conn(0x7f547c0fa4b0 msgr2=0x7f547c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:32.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.694+0000 7f5483bff700 1 -- 172.21.15.161:0/959223942 shutdown_connections 2023-07-14T11:38:32.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.694+0000 7f5483bff700 1 -- 172.21.15.161:0/959223942 wait complete. 2023-07-14T11:38:32.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.695+0000 7f5483bff700 1 Processor -- start 2023-07-14T11:38:32.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.695+0000 7f5483bff700 1 -- start start 2023-07-14T11:38:32.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.696+0000 7f5483bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547c0fe9d0 0x7f547c1937b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.696+0000 7f5483bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c193cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.696+0000 7f5483bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f547c1049e0 0x7f547c194230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.696+0000 7f5483bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c1a8450 con 0x7f547c1049e0 2023-07-14T11:38:32.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.696+0000 7f5483bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c1a85c0 con 0x7f547c0fe9d0 2023-07-14T11:38:32.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.696+0000 7f5483bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c1a8780 con 0x7f547c0ff380 2023-07-14T11:38:32.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.696+0000 7f548119a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c193cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c193cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41788/0 (socket says 172.21.15.161:41788) 2023-07-14T11:38:32.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 -- 172.21.15.161:0/2568490622 learned_addr learned my addr 172.21.15.161:0/2568490622 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:32.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548219c700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f547c1049e0 0x7f547c194230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 -- 172.21.15.161:0/2568490622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547c0fe9d0 msgr2=0x7f547c1937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547c0fe9d0 0x7f547c1937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 -- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f547c1049e0 msgr2=0x7f547c194230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f547c1049e0 0x7f547c194230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 -- 172.21.15.161:0/2568490622 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5474009520 con 0x7f547c0ff380 2023-07-14T11:38:32.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.697+0000 7f548119a700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c193cf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f546c0097d0 tx=0x7f546c009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:32.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.698+0000 7f5472ffd700 1 -- 172.21.15.161:0/2568490622 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c010040 con 0x7f547c0ff380 2023-07-14T11:38:32.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.698+0000 7f5472ffd700 1 -- 172.21.15.161:0/2568490622 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546c014cd0 con 0x7f547c0ff380 2023-07-14T11:38:32.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.698+0000 7f5472ffd700 1 -- 172.21.15.161:0/2568490622 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c00edb0 con 0x7f547c0ff380 2023-07-14T11:38:32.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.698+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c1a8a60 con 0x7f547c0ff380 2023-07-14T11:38:32.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.698+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c1a8f20 con 0x7f547c0ff380 2023-07-14T11:38:32.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.704+0000 7f5472ffd700 1 -- 172.21.15.161:0/2568490622 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f546c00e450 con 0x7f547c0ff380 2023-07-14T11:38:32.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.704+0000 7f549748cb80 1 Processor -- start 2023-07-14T11:38:32.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.705+0000 7f549748cb80 1 -- start start 2023-07-14T11:38:32.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.705+0000 7f549748cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c882d0 0x562af3c886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.705+0000 7f549748cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c93590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.705+0000 7f549748cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c93ad0 0x562af3c95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.705+0000 7f549748cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3b8ecb0 con 0x562af3c93ad0 2023-07-14T11:38:32.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.705+0000 7f549748cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3be1a40 con 0x562af3c882d0 2023-07-14T11:38:32.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.705+0000 7f549748cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3bdaed0 con 0x562af3c88be0 2023-07-14T11:38:32.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.706+0000 7f548219c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c882d0 0x562af3c886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.706+0000 7f548119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c93ad0 0x562af3c95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.706+0000 7f548219c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c882d0 0x562af3c886a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60138/0 (socket says 172.21.15.161:60138) 2023-07-14T11:38:32.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.706+0000 7f548219c700 1 -- 172.21.15.161:0/2695926168 learned_addr learned my addr 172.21.15.161:0/2695926168 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:32.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.706+0000 7f548199b700 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c93590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.707+0000 7f548219c700 1 -- 172.21.15.161:0/2695926168 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 msgr2=0x562af3c93590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:32.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.707+0000 7f548219c700 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c93590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.707+0000 7f548219c700 1 -- 172.21.15.161:0/2695926168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c93ad0 msgr2=0x562af3c95eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.707+0000 7f548219c700 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c93ad0 0x562af3c95eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.707+0000 7f548219c700 1 -- 172.21.15.161:0/2695926168 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af3a08de0 con 0x562af3c882d0 2023-07-14T11:38:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.707+0000 7f548219c700 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c882d0 0x562af3c886a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5474016000 tx=0x7f5474009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b60eddde3beb017 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.708+0000 7f5472ffd700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54680a1410 0x7f54680a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.708+0000 7f5472ffd700 1 -- 172.21.15.161:0/2568490622 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f546c0cd020 con 0x7f547c0ff380 2023-07-14T11:38:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.708+0000 7f5467fff700 1 -- 172.21.15.161:0/2695926168 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547401c040 con 0x562af3c882d0 2023-07-14T11:38:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.708+0000 7f5467fff700 1 -- 172.21.15.161:0/2695926168 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f547400ba20 con 0x562af3c882d0 2023-07-14T11:38:32.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.708+0000 7f5467fff700 1 -- 172.21.15.161:0/2695926168 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474005530 con 0x562af3c882d0 2023-07-14T11:38:32.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:32 smithi182 ceph-mon[129366]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 657 op/s 2023-07-14T11:38:32.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:32 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3184208826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:38:32.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.709+0000 7f548199b700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54680a1410 0x7f54680a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.710+0000 7f549748cb80 1 -- 172.21.15.161:0/2695926168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c882d0 msgr2=0x562af3c886a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.710+0000 7f549748cb80 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c882d0 0x562af3c886a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5474016000 tx=0x7f5474009400 comp rx=0 tx=0).stop 2023-07-14T11:38:32.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.710+0000 7f548199b700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54680a1410 0x7f54680a38c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5478009fd0 tx=0x7f547800b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:32.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.711+0000 7f549748cb80 1 -- 172.21.15.161:0/2695926168 shutdown_connections 2023-07-14T11:38:32.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.711+0000 7f549748cb80 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c93ad0 0x562af3c95eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.711+0000 7f549748cb80 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c882d0 0x562af3c886a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.711+0000 7f549748cb80 1 --2- 172.21.15.161:0/2695926168 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c93590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.711+0000 7f549748cb80 1 -- 172.21.15.161:0/2695926168 >> 172.21.15.161:0/2695926168 conn(0x562af3c7eda0 msgr2=0x562af3c846a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.711+0000 7f549748cb80 1 -- 172.21.15.161:0/2695926168 shutdown_connections 2023-07-14T11:38:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.711+0000 7f549748cb80 1 -- 172.21.15.161:0/2695926168 wait complete. 2023-07-14T11:38:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.712+0000 7f549748cb80 1 -- start start 2023-07-14T11:38:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.712+0000 7f549748cb80 1 Processor -- start 2023-07-14T11:38:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.713+0000 7f549748cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c90830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.713+0000 7f549748cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c90d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.713+0000 7f549748cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.713+0000 7f549748cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3b7aac0 con 0x562af3c882d0 2023-07-14T11:38:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.713+0000 7f549748cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3b7a7c0 con 0x562af3c88be0 2023-07-14T11:38:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.713+0000 7f549748cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3a5e920 con 0x562af3c93ad0 2023-07-14T11:38:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.713+0000 7f549748cb80 10 client.0 _ll_register_callbacks cb 0x562af3b78760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:32.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c90830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c90830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39180/0 (socket says 172.21.15.161:39180) 2023-07-14T11:38:32.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548199b700 1 -- 172.21.15.161:0/2640563192 learned_addr learned my addr 172.21.15.161:0/2640563192 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:32.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548119a700 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c90d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548119a700 1 -- 172.21.15.161:0/2640563192 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 msgr2=0x562af3c912b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:38:32.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548119a700 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c912b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548119a700 1 -- 172.21.15.161:0/2640563192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 msgr2=0x562af3c90830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:32.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548119a700 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c90830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548119a700 1 -- 172.21.15.161:0/2640563192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c10c110 con 0x562af3c88be0 2023-07-14T11:38:32.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.714+0000 7f548199b700 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c90830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:32.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.715+0000 7f548119a700 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c90d70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f546c0ca3f0 tx=0x7f546c00efc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39222632fd6be1a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:32.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.715+0000 7f54657fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:38:32.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.715+0000 7f54657fa700 1 -- 172.21.15.161:0/2640563192 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0998b0 con 0x562af3c88be0 2023-07-14T11:38:32.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.715+0000 7f54657fa700 1 -- 172.21.15.161:0/2640563192 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f546c099d70 con 0x562af3c88be0 2023-07-14T11:38:32.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.716+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562af3b344f0 con 0x562af3c88be0 2023-07-14T11:38:32.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.716+0000 7f54657fa700 1 -- 172.21.15.161:0/2640563192 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f546c0996d0 con 0x562af3c88be0 2023-07-14T11:38:32.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.716+0000 7f549748cb80 10 client.27428 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:32.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.716+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562af39f1c60 con 0x562af3c88be0 2023-07-14T11:38:32.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.717+0000 7f54657fa700 1 -- 172.21.15.161:0/2640563192 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562af3b344f0 con 0x562af3c88be0 2023-07-14T11:38:32.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.717+0000 7f54657fa700 1 -- 172.21.15.161:0/2640563192 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f546c0ca740 con 0x562af3c88be0 2023-07-14T11:38:32.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.717+0000 7f549748cb80 10 client.27428 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:32.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.717+0000 7f549748cb80 10 client.27428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:38:32.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.717+0000 7f549748cb80 20 client.27428 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:32.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.717+0000 7f549748cb80 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3ca3430 0x562af3ca5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.718+0000 7f549748cb80 4 client.27428 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:32.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.718+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562af3bd3eb0 con 0x562af3ca3430 2023-07-14T11:38:32.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.719+0000 7f548199b700 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3ca3430 0x562af3ca5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.720+0000 7f54657fa700 10 client.27428 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:38:32.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.720+0000 7f548199b700 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3ca3430 0x562af3ca5840 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e71d9814736432a1 server_cookie=3d1a95e34463803d in_seq=0 out_seq=0 2023-07-14T11:38:32.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.723+0000 7f54657fa700 1 -- 172.21.15.161:0/2640563192 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562af3bd3eb0 con 0x562af3ca3430 2023-07-14T11:38:32.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.723+0000 7f54657fa700 10 client.27428 handle_command_reply: tid=0 2023-07-14T11:38:32.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.723+0000 7f549748cb80 1 client.27428 shutdown 2023-07-14T11:38:32.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.724+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 msgr2=0x562af3c90d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.724+0000 7f549748cb80 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c90d70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f546c0ca3f0 tx=0x7f546c00efc0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.725+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 shutdown_connections 2023-07-14T11:38:32.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.725+0000 7f549748cb80 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3ca3430 0x562af3ca5840 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.726+0000 7f549748cb80 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c90830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.726+0000 7f549748cb80 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c912b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.726+0000 7f549748cb80 1 --2- 172.21.15.161:0/2640563192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c90d70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.726+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 >> 172.21.15.161:0/2640563192 conn(0x562af3c7eda0 msgr2=0x562af3c84290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:32.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.726+0000 7f54657fa700 0 client.27428 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:38:32.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.726+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 shutdown_connections 2023-07-14T11:38:32.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.727+0000 7f549748cb80 1 -- 172.21.15.161:0/2640563192 wait complete. 2023-07-14T11:38:32.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.727+0000 7f549748cb80 20 client.27428 trim_cache size 0 max 16384 2023-07-14T11:38:32.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:32 smithi161 ceph-mon[95526]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 657 op/s 2023-07-14T11:38:32.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:32 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3184208826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:38:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.747+0000 7f549748cb80 1 Processor -- start 2023-07-14T11:38:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.747+0000 7f549748cb80 1 -- start start 2023-07-14T11:38:32.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f549748cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c9a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f549748cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c8bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f549748cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.747 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f549748cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3c80ec0 con 0x562af3c882d0 2023-07-14T11:38:32.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f549748cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3c89e30 con 0x562af3c88be0 2023-07-14T11:38:32.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f549748cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3c89540 con 0x562af3c93ad0 2023-07-14T11:38:32.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f548219c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f548219c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c8c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41812/0 (socket says 172.21.15.161:41812) 2023-07-14T11:38:32.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f548219c700 1 -- 172.21.15.161:0/2053627320 learned_addr learned my addr 172.21.15.161:0/2053627320 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:32.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f548199b700 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c9a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.748+0000 7f548119a700 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c8bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.749+0000 7f548219c700 1 -- 172.21.15.161:0/2053627320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 msgr2=0x562af3c8bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:32.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.749+0000 7f548219c700 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c8bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.749+0000 7f548219c700 1 -- 172.21.15.161:0/2053627320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 msgr2=0x562af3c9a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:32.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.749+0000 7f548219c700 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c9a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.749+0000 7f548219c700 1 -- 172.21.15.161:0/2053627320 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af3a2f250 con 0x562af3c93ad0 2023-07-14T11:38:32.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.749+0000 7f548219c700 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c8c0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f547400b260 tx=0x7f5474009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120bd46a451e3a73 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:32.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.750+0000 7f5470bff700 1 -- 172.21.15.161:0/2053627320 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547401c040 con 0x562af3c93ad0 2023-07-14T11:38:32.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.750+0000 7f5470bff700 1 -- 172.21.15.161:0/2053627320 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f547400f920 con 0x562af3c93ad0 2023-07-14T11:38:32.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.750+0000 7f5470bff700 1 -- 172.21.15.161:0/2053627320 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54740046d0 con 0x562af3c93ad0 2023-07-14T11:38:32.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.750+0000 7f549748cb80 1 -- 172.21.15.161:0/2053627320 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 msgr2=0x562af3c8c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.750+0000 7f549748cb80 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c8c0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f547400b260 tx=0x7f5474009c00 comp rx=0 tx=0).stop 2023-07-14T11:38:32.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 -- 172.21.15.161:0/2053627320 shutdown_connections 2023-07-14T11:38:32.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c9a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c93ad0 0x562af3c8c0b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 --2- 172.21.15.161:0/2053627320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c88be0 0x562af3c8bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 -- 172.21.15.161:0/2053627320 >> 172.21.15.161:0/2053627320 conn(0x562af3c7eda0 msgr2=0x562af3c82e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:32.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 -- 172.21.15.161:0/2053627320 shutdown_connections 2023-07-14T11:38:32.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 -- 172.21.15.161:0/2053627320 wait complete. 2023-07-14T11:38:32.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.751+0000 7f549748cb80 1 -- start start 2023-07-14T11:38:32.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 1 Processor -- start 2023-07-14T11:38:32.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c98680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c98bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c93ad0 0x562af3cf62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3c992d0 con 0x562af3c882d0 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3cf6830 con 0x562af3c93ad0 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af3cf69a0 con 0x562af3c88be0 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f549748cb80 10 client.0 _ll_register_callbacks cb 0x562af3b78760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f548119a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c98bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f548119a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c98bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41824/0 (socket says 172.21.15.161:41824) 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f548119a700 1 -- 172.21.15.161:0/1138824840 learned_addr learned my addr 172.21.15.161:0/1138824840 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:32.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.752+0000 7f548199b700 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f548219c700 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c93ad0 0x562af3cf62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f548119a700 1 -- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c93ad0 msgr2=0x562af3cf62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:32.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f548119a700 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c93ad0 0x562af3cf62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f548119a700 1 -- 172.21.15.161:0/1138824840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 msgr2=0x562af3c98680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:32.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f548119a700 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c98680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.756 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f548119a700 1 -- 172.21.15.161:0/1138824840 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562af3a08de0 con 0x562af3c88be0 2023-07-14T11:38:32.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f548119a700 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c98bc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f546c099d00 tx=0x7f546c0d0600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4556b7786af5000e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:32.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f54657fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:32.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f54657fa700 1 -- 172.21.15.161:0/1138824840 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c099fd0 con 0x562af3c88be0 2023-07-14T11:38:32.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f54657fa700 1 -- 172.21.15.161:0/1138824840 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f546c099d70 con 0x562af3c88be0 2023-07-14T11:38:32.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.753+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f547400ba20 con 0x562af3c88be0 2023-07-14T11:38:32.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f54657fa700 1 -- 172.21.15.161:0/1138824840 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f546c0996d0 con 0x562af3c88be0 2023-07-14T11:38:32.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 10 client.27478 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:32.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562af3a072b0 con 0x562af3c88be0 2023-07-14T11:38:32.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f54657fa700 1 -- 172.21.15.161:0/1138824840 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f547400ba20 con 0x562af3c88be0 2023-07-14T11:38:32.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f54657fa700 1 -- 172.21.15.161:0/1138824840 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f546c099260 con 0x562af3c88be0 2023-07-14T11:38:32.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 10 client.27478 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:32.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 10 client.27478 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:38:32.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 20 client.27478 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:32.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3cf7bf0 0x562af3cf9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:32.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 4 client.27478 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:38:32.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.754+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562af3b96050 con 0x562af3cf7bf0 2023-07-14T11:38:32.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.755+0000 7f548199b700 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3cf7bf0 0x562af3cf9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:32.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.758+0000 7f54657fa700 10 client.27478 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:38:32.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.758+0000 7f548199b700 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3cf7bf0 0x562af3cf9fd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25588e841c8767c3 server_cookie=ecc1b39a8ee9bf26 in_seq=0 out_seq=0 2023-07-14T11:38:32.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f54657fa700 1 -- 172.21.15.161:0/1138824840 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562af3b96050 con 0x562af3cf7bf0 2023-07-14T11:38:32.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f54657fa700 10 client.27478 handle_command_reply: tid=0 2023-07-14T11:38:32.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 client.27478 shutdown 2023-07-14T11:38:32.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 msgr2=0x562af3c98bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c98bc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f546c099d00 tx=0x7f546c0d0600 comp rx=0 tx=0).stop 2023-07-14T11:38:32.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 shutdown_connections 2023-07-14T11:38:32.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x562af3cf7bf0 0x562af3cf9fd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af3c882d0 0x562af3c98680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562af3c93ad0 0x562af3cf62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f54657fa700 0 client.27478 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:38:32.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 --2- 172.21.15.161:0/1138824840 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562af3c88be0 0x562af3c98bc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 >> 172.21.15.161:0/1138824840 conn(0x562af3c7eda0 msgr2=0x562af3c8bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:32.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 shutdown_connections 2023-07-14T11:38:32.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.763+0000 7f549748cb80 1 -- 172.21.15.161:0/1138824840 wait complete. 2023-07-14T11:38:32.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.764+0000 7f549748cb80 20 client.27478 trim_cache size 0 max 16384 2023-07-14T11:38:32.764 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:38:32.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.764+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54680a1410 msgr2=0x7f54680a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.764+0000 7f5483bff700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54680a1410 0x7f54680a38c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5478009fd0 tx=0x7f547800b040 comp rx=0 tx=0).stop 2023-07-14T11:38:32.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.764+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 msgr2=0x7f547c193cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:32.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.764+0000 7f5483bff700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c193cf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f546c0097d0 tx=0x7f546c009b90 comp rx=0 tx=0).stop 2023-07-14T11:38:32.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 shutdown_connections 2023-07-14T11:38:32.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f54680a1410 0x7f54680a38c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f547c1049e0 0x7f547c194230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547c0fe9d0 0x7f547c1937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 --2- 172.21.15.161:0/2568490622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f547c0ff380 0x7f547c193cf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:32.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 >> 172.21.15.161:0/2568490622 conn(0x7f547c0fa4b0 msgr2=0x7f547c0fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:32.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 shutdown_connections 2023-07-14T11:38:32.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:32.765+0000 7f5483bff700 1 -- 172.21.15.161:0/2568490622 wait complete. 2023-07-14T11:38:32.785 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:38:32.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:32 smithi175 ceph-mon[129081]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 657 op/s 2023-07-14T11:38:32.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:32 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3184208826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:38:33.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66bd599700 1 -- 172.21.15.161:0/3187275214 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b400fe90 con 0x7f66b8105af0 2023-07-14T11:38:33.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66c07ff700 1 -- 172.21.15.161:0/3187275214 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 msgr2=0x7f66b8105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3187275214 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 0x7f66b8105ec0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f66b40098f0 tx=0x7f66b4009c00 comp rx=0 tx=0).stop 2023-07-14T11:38:33.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66c07ff700 1 -- 172.21.15.161:0/3187275214 shutdown_connections 2023-07-14T11:38:33.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3187275214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66b8100490 0x7f66b810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3187275214 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 0x7f66b8105ec0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3187275214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b80ffae0 0x7f66b80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.068+0000 7f66c07ff700 1 -- 172.21.15.161:0/3187275214 >> 172.21.15.161:0/3187275214 conn(0x7f66b80fb5e0 msgr2=0x7f66b80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.069+0000 7f66c07ff700 1 -- 172.21.15.161:0/3187275214 shutdown_connections 2023-07-14T11:38:33.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.069+0000 7f66c07ff700 1 -- 172.21.15.161:0/3187275214 wait complete. 2023-07-14T11:38:33.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.070+0000 7f66c07ff700 1 Processor -- start 2023-07-14T11:38:33.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.070+0000 7f66c07ff700 1 -- start start 2023-07-14T11:38:33.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66c07ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66b80ffae0 0x7f66b8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66c07ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b8100490 0x7f66b81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66c07ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 0x7f66b819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66c07ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819ce50 con 0x7f66b8100490 2023-07-14T11:38:33.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66c07ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819cfc0 con 0x7f66b80ffae0 2023-07-14T11:38:33.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66c07ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819d180 con 0x7f66b8105af0 2023-07-14T11:38:33.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66bdd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b8100490 0x7f66b81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66bdd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b8100490 0x7f66b81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39214/0 (socket says 172.21.15.161:39214) 2023-07-14T11:38:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66bdd9a700 1 -- 172.21.15.161:0/3719221597 learned_addr learned my addr 172.21.15.161:0/3719221597 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.071+0000 7f66be59b700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66b80ffae0 0x7f66b8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 0x7f66b819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66bdd9a700 1 -- 172.21.15.161:0/3719221597 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 msgr2=0x7f66b819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 0x7f66b819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66bdd9a700 1 -- 172.21.15.161:0/3719221597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66b80ffae0 msgr2=0x7f66b8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66b80ffae0 0x7f66b8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66bdd9a700 1 -- 172.21.15.161:0/3719221597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b0009d00 con 0x7f66b8100490 2023-07-14T11:38:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66be59b700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66b80ffae0 0x7f66b8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.072+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 0x7f66b819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:38:33.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.073+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b8100490 0x7f66b81983b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f66a800e9a0 tx=0x7f66a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.073+0000 7f66af7fe700 1 -- 172.21.15.161:0/3719221597 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8009790 con 0x7f66b8100490 2023-07-14T11:38:33.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.073+0000 7f66af7fe700 1 -- 172.21.15.161:0/3719221597 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a800cbd0 con 0x7f66b8100490 2023-07-14T11:38:33.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.073+0000 7f66af7fe700 1 -- 172.21.15.161:0/3719221597 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8018680 con 0x7f66b8100490 2023-07-14T11:38:33.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.073+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b4009520 con 0x7f66b8100490 2023-07-14T11:38:33.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.074+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b81a4ac0 con 0x7f66b8100490 2023-07-14T11:38:33.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.080+0000 7f66af7fe700 1 -- 172.21.15.161:0/3719221597 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f66a800cd40 con 0x7f66b8100490 2023-07-14T11:38:33.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.080+0000 7f66d403bb80 1 Processor -- start 2023-07-14T11:38:33.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.081+0000 7f66d403bb80 1 -- start start 2023-07-14T11:38:33.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.082+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560194fc46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.082+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 0x560194fd7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.082+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fda6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.082+0000 7f66d403bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194ecacb0 con 0x560194fc42d0 2023-07-14T11:38:33.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.082+0000 7f66d403bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194f1da40 con 0x560194fd82e0 2023-07-14T11:38:33.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.082+0000 7f66d403bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194f16ed0 con 0x560194fc4be0 2023-07-14T11:38:33.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.083+0000 7f66bed9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560194fc46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.083+0000 7f66bed9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560194fc46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39220/0 (socket says 172.21.15.161:39220) 2023-07-14T11:38:33.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.083+0000 7f66bed9c700 1 -- 172.21.15.161:0/2666892401 learned_addr learned my addr 172.21.15.161:0/2666892401 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.083+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fda6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.084+0000 7f66af7fe700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66a40a1340 0x7f66a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.084+0000 7f66bdd9a700 1 -- 172.21.15.161:0/2666892401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 msgr2=0x560194fd7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.084+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 0x560194fd7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.084+0000 7f66bdd9a700 1 -- 172.21.15.161:0/2666892401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 msgr2=0x560194fc46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.084+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560194fc46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.084+0000 7f66bdd9a700 1 -- 172.21.15.161:0/2666892401 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560194d44de0 con 0x560194fd82e0 2023-07-14T11:38:33.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.085+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66a40a1340 0x7f66a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.085+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fda6c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f66a80ca550 tx=0x7f66a8099a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e2baa5390fbe2a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.086+0000 7f66af7fe700 1 -- 172.21.15.161:0/3719221597 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f66a8014070 con 0x7f66b8100490 2023-07-14T11:38:33.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.087+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66a40a1340 0x7f66a40a37f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66b0007760 tx=0x7f66b000d040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.089+0000 7f66acff9700 1 -- 172.21.15.161:0/2666892401 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800b2c0 con 0x560194fd82e0 2023-07-14T11:38:33.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.089+0000 7f66acff9700 1 -- 172.21.15.161:0/2666892401 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a8099ed0 con 0x560194fd82e0 2023-07-14T11:38:33.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.089+0000 7f66acff9700 1 -- 172.21.15.161:0/2666892401 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a80a23b0 con 0x560194fd82e0 2023-07-14T11:38:33.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.090+0000 7f66d403bb80 1 -- 172.21.15.161:0/2666892401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 msgr2=0x560194fda6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.090+0000 7f66d403bb80 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fda6c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f66a80ca550 tx=0x7f66a8099a10 comp rx=0 tx=0).stop 2023-07-14T11:38:33.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.091+0000 7f66d403bb80 1 -- 172.21.15.161:0/2666892401 shutdown_connections 2023-07-14T11:38:33.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.091+0000 7f66d403bb80 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560194fc46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.091+0000 7f66d403bb80 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fda6c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.091+0000 7f66d403bb80 1 --2- 172.21.15.161:0/2666892401 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 0x560194fd7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.091+0000 7f66d403bb80 1 -- 172.21.15.161:0/2666892401 >> 172.21.15.161:0/2666892401 conn(0x560194fbada0 msgr2=0x560194fc06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.091+0000 7f66d403bb80 1 -- 172.21.15.161:0/2666892401 shutdown_connections 2023-07-14T11:38:33.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.092+0000 7f66d403bb80 1 -- 172.21.15.161:0/2666892401 wait complete. 2023-07-14T11:38:33.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.092+0000 7f66d403bb80 1 -- start start 2023-07-14T11:38:33.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.093+0000 7f66d403bb80 1 Processor -- start 2023-07-14T11:38:33.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.094+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc42d0 0x560194fd7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.094+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 0x560194fd4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.094+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fd82e0 0x560194fd5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.094+0000 7f66d403bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194eb6ac0 con 0x560194fd82e0 2023-07-14T11:38:33.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.094+0000 7f66d403bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194eb67c0 con 0x560194fc42d0 2023-07-14T11:38:33.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.094+0000 7f66d403bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194d9a920 con 0x560194fc4be0 2023-07-14T11:38:33.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.094+0000 7f66d403bb80 10 client.0 _ll_register_callbacks cb 0x560194eb4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:33.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.095+0000 7f66be59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc42d0 0x560194fd7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.095+0000 7f66be59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc42d0 0x560194fd7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60202/0 (socket says 172.21.15.161:60202) 2023-07-14T11:38:33.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.095+0000 7f66be59b700 1 -- 172.21.15.161:0/3447639427 learned_addr learned my addr 172.21.15.161:0/3447639427 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.095+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fd82e0 0x560194fd5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.096+0000 7f66bed9c700 1 -- 172.21.15.161:0/3447639427 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 msgr2=0x560194fd4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.096+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 0x560194fd4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.096+0000 7f66bed9c700 1 -- 172.21.15.161:0/3447639427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc42d0 msgr2=0x560194fd7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.096+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc42d0 0x560194fd7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.096+0000 7f66bed9c700 1 -- 172.21.15.161:0/3447639427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b81a4ac0 con 0x560194fd82e0 2023-07-14T11:38:33.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.098+0000 7f66bed9c700 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fd82e0 0x560194fd5730 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f66b00095b0 tx=0x7f66b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5779a7e48a47d76 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.098+0000 7f66a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:38:33.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.098+0000 7f66a1ffb700 1 -- 172.21.15.161:0/3447639427 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0018e90 con 0x560194fd82e0 2023-07-14T11:38:33.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.098+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560194e704f0 con 0x560194fd82e0 2023-07-14T11:38:33.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.098+0000 7f66a1ffb700 1 -- 172.21.15.161:0/3447639427 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f66b0004bb0 con 0x560194fd82e0 2023-07-14T11:38:33.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.099+0000 7f66a1ffb700 1 -- 172.21.15.161:0/3447639427 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66b0004d70 con 0x560194fd82e0 2023-07-14T11:38:33.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.099+0000 7f66d403bb80 10 client.17415 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:33.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.099+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560194d2dc60 con 0x560194fd82e0 2023-07-14T11:38:33.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.099+0000 7f66a1ffb700 1 -- 172.21.15.161:0/3447639427 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560194e704f0 con 0x560194fd82e0 2023-07-14T11:38:33.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.100+0000 7f66a1ffb700 1 -- 172.21.15.161:0/3447639427 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f66b0023b90 con 0x560194fd82e0 2023-07-14T11:38:33.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.100+0000 7f66d403bb80 10 client.17415 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:33.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.100+0000 7f66d403bb80 10 client.17415 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:38:33.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.100+0000 7f66d403bb80 20 client.17415 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:33.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.100+0000 7f66d403bb80 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194ff8c10 0x560194ffb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.100+0000 7f66d403bb80 4 client.17415 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:33.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.100+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560194f0feb0 con 0x560194ff8c10 2023-07-14T11:38:33.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.102+0000 7f66be59b700 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194ff8c10 0x560194ffb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.104+0000 7f66a1ffb700 10 client.17415 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:38:33.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.104+0000 7f66be59b700 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194ff8c10 0x560194ffb020 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29c65856928a7aa7 server_cookie=df6026bdc5a1e28a in_seq=0 out_seq=0 2023-07-14T11:38:33.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.106+0000 7f66a1ffb700 1 -- 172.21.15.161:0/3447639427 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560194f0feb0 con 0x560194ff8c10 2023-07-14T11:38:33.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.106+0000 7f66a1ffb700 10 client.17415 handle_command_reply: tid=0 2023-07-14T11:38:33.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 client.17415 shutdown 2023-07-14T11:38:33.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fd82e0 msgr2=0x560194fd5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fd82e0 0x560194fd5730 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f66b00095b0 tx=0x7f66b0004ad0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 shutdown_connections 2023-07-14T11:38:33.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194ff8c10 0x560194ffb020 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fd82e0 0x560194fd5730 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc42d0 0x560194fd7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 --2- 172.21.15.161:0/3447639427 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc4be0 0x560194fd4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 >> 172.21.15.161:0/3447639427 conn(0x560194fbada0 msgr2=0x560194fc0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.107+0000 7f66a1ffb700 0 client.17415 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:38:33.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.109+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 shutdown_connections 2023-07-14T11:38:33.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.109+0000 7f66d403bb80 1 -- 172.21.15.161:0/3447639427 wait complete. 2023-07-14T11:38:33.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.110+0000 7f66d403bb80 20 client.17415 trim_cache size 0 max 16384 2023-07-14T11:38:33.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.137+0000 7f66d403bb80 1 Processor -- start 2023-07-14T11:38:33.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.138+0000 7f66d403bb80 1 -- start start 2023-07-14T11:38:33.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.138+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc42d0 0x560194fd47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.138+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc4be0 0x560194fbee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.139+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fd7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.139+0000 7f66d403bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194fd3570 con 0x560194fc4be0 2023-07-14T11:38:33.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.139+0000 7f66d403bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194fd0c30 con 0x560194fd82e0 2023-07-14T11:38:33.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.139+0000 7f66d403bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194fd4d10 con 0x560194fc42d0 2023-07-14T11:38:33.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.141+0000 7f66bdd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc4be0 0x560194fbee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.141+0000 7f66bdd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc4be0 0x560194fbee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39244/0 (socket says 172.21.15.161:39244) 2023-07-14T11:38:33.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.141+0000 7f66bdd9a700 1 -- 172.21.15.161:0/1586939305 learned_addr learned my addr 172.21.15.161:0/1586939305 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.142+0000 7f66bdd9a700 1 -- 172.21.15.161:0/1586939305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc42d0 msgr2=0x560194fd47d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:38:33.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.142+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/1586939305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc42d0 0x560194fd47d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.142+0000 7f66bdd9a700 1 -- 172.21.15.161:0/1586939305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 msgr2=0x560194fd7910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:38:33.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.142+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/1586939305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fd7910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.142+0000 7f66bdd9a700 1 -- 172.21.15.161:0/1586939305 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560194d6b250 con 0x560194fc4be0 2023-07-14T11:38:33.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.147+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/1586939305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc4be0 0x560194fbee10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f66a8096940 tx=0x7f66a809a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f66ca7d0d56ddc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.151+0000 7f66ad7fa700 1 -- 172.21.15.161:0/1586939305 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800b2c0 con 0x560194fc4be0 2023-07-14T11:38:33.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.151+0000 7f66ad7fa700 1 -- 172.21.15.161:0/1586939305 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a809df00 con 0x560194fc4be0 2023-07-14T11:38:33.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.151+0000 7f66ad7fa700 1 -- 172.21.15.161:0/1586939305 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8099620 con 0x560194fc4be0 2023-07-14T11:38:33.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.152+0000 7f66d403bb80 1 -- 172.21.15.161:0/1586939305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc4be0 msgr2=0x560194fbee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.152+0000 7f66d403bb80 1 --2- 172.21.15.161:0/1586939305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc4be0 0x560194fbee10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f66a8096940 tx=0x7f66a809a040 comp rx=0 tx=0).stop 2023-07-14T11:38:33.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.153+0000 7f66d403bb80 1 -- 172.21.15.161:0/1586939305 shutdown_connections 2023-07-14T11:38:33.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.153+0000 7f66d403bb80 1 --2- 172.21.15.161:0/1586939305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fd82e0 0x560194fd7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.153+0000 7f66d403bb80 1 --2- 172.21.15.161:0/1586939305 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fc42d0 0x560194fd47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.153+0000 7f66d403bb80 1 --2- 172.21.15.161:0/1586939305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc4be0 0x560194fbee10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.153+0000 7f66d403bb80 1 -- 172.21.15.161:0/1586939305 >> 172.21.15.161:0/1586939305 conn(0x560194fbada0 msgr2=0x560194ff4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.158+0000 7f66d403bb80 1 -- 172.21.15.161:0/1586939305 shutdown_connections 2023-07-14T11:38:33.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.159+0000 7f66d403bb80 1 -- 172.21.15.161:0/1586939305 wait complete. 2023-07-14T11:38:33.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.160+0000 7f66d403bb80 1 -- start start 2023-07-14T11:38:33.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.160+0000 7f66d403bb80 1 Processor -- start 2023-07-14T11:38:33.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.170+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560195047e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.170+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc4be0 0x560195048340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.170+0000 7f66d403bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fd82e0 0x560194fdb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.170+0000 7f66d403bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194fdbbc0 con 0x560194fc42d0 2023-07-14T11:38:33.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.170+0000 7f66d403bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194fdbd30 con 0x560194fc4be0 2023-07-14T11:38:33.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.170+0000 7f66d403bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560194fdbef0 con 0x560194fd82e0 2023-07-14T11:38:33.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.170+0000 7f66d403bb80 10 client.0 _ll_register_callbacks cb 0x7f66b0009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:33.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.172+0000 7f66be59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560195047e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.172+0000 7f66be59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560195047e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39256/0 (socket says 172.21.15.161:39256) 2023-07-14T11:38:33.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.172+0000 7f66be59b700 1 -- 172.21.15.161:0/601322479 learned_addr learned my addr 172.21.15.161:0/601322479 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.172+0000 7f66bed9c700 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fd82e0 0x560194fdb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.173+0000 7f66be59b700 1 -- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fd82e0 msgr2=0x560194fdb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.173+0000 7f66be59b700 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fd82e0 0x560194fdb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.171 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.173+0000 7f66be59b700 1 -- 172.21.15.161:0/601322479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc4be0 msgr2=0x560195048340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.173+0000 7f66be59b700 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc4be0 0x560195048340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.173+0000 7f66be59b700 1 -- 172.21.15.161:0/601322479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560194d44de0 con 0x560194fc42d0 2023-07-14T11:38:33.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.174+0000 7f66be59b700 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560195047e00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f66b40098c0 tx=0x7f66b400f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73883dc59cffaa57 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.174+0000 7f66a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:38:33.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.174+0000 7f66a1ffb700 1 -- 172.21.15.161:0/601322479 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b401c040 con 0x560194fc42d0 2023-07-14T11:38:33.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.174+0000 7f66a1ffb700 1 -- 172.21.15.161:0/601322479 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f66b400f910 con 0x560194fc42d0 2023-07-14T11:38:33.173 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.175+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66a8099ed0 con 0x560194fc42d0 2023-07-14T11:38:33.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.175+0000 7f66a1ffb700 1 -- 172.21.15.161:0/601322479 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66b4024e50 con 0x560194fc42d0 2023-07-14T11:38:33.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.176+0000 7f66d403bb80 10 client.17421 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:33.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.176+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560194d432b0 con 0x560194fc42d0 2023-07-14T11:38:33.174 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66a1ffb700 1 -- 172.21.15.161:0/601322479 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a8099ed0 con 0x560194fc42d0 2023-07-14T11:38:33.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66a1ffb700 1 -- 172.21.15.161:0/601322479 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f66b4015020 con 0x560194fc42d0 2023-07-14T11:38:33.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66d403bb80 10 client.17421 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:33.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66d403bb80 10 client.17421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:38:33.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66d403bb80 20 client.17421 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:33.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66d403bb80 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194fcc1d0 0x560194fce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66d403bb80 4 client.17421 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:38:33.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.177+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560194ed2050 con 0x560194fcc1d0 2023-07-14T11:38:33.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.181+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194fcc1d0 0x560194fce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.182+0000 7f66bdd9a700 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194fcc1d0 0x560194fce5e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc665b60d65ce213 server_cookie=30b9857884edf39c in_seq=0 out_seq=0 2023-07-14T11:38:33.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.186+0000 7f66a1ffb700 10 client.17421 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:38:33.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.186+0000 7f66a1ffb700 1 -- 172.21.15.161:0/601322479 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560194ed2050 con 0x560194fcc1d0 2023-07-14T11:38:33.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.186+0000 7f66a1ffb700 10 client.17421 handle_command_reply: tid=0 2023-07-14T11:38:33.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.186+0000 7f66d403bb80 1 client.17421 shutdown 2023-07-14T11:38:33.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 msgr2=0x560195047e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560195047e00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f66b40098c0 tx=0x7f66b400f630 comp rx=0 tx=0).stop 2023-07-14T11:38:33.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 shutdown_connections 2023-07-14T11:38:33.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x560194fcc1d0 0x560194fce5e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560194fc42d0 0x560195047e00 secure :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f66b40098c0 tx=0x7f66b400f630 comp rx=0 tx=0).stop 2023-07-14T11:38:33.185 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560194fd82e0 0x560194fdb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 --2- 172.21.15.161:0/601322479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560194fc4be0 0x560195048340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 >> 172.21.15.161:0/601322479 conn(0x560194fbada0 msgr2=0x560194fd03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66a1ffb700 0 client.17421 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:38:33.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.187+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 shutdown_connections 2023-07-14T11:38:33.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.188+0000 7f66d403bb80 1 -- 172.21.15.161:0/601322479 wait complete. 2023-07-14T11:38:33.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.189+0000 7f66d403bb80 20 client.17421 trim_cache size 0 max 16384 2023-07-14T11:38:33.187 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:38:33.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.192+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66a40a1340 msgr2=0x7f66a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.192+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66a40a1340 0x7f66a40a37f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66b0007760 tx=0x7f66b000d040 comp rx=0 tx=0).stop 2023-07-14T11:38:33.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.192+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b8100490 msgr2=0x7f66b81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.192+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b8100490 0x7f66b81983b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f66a800e9a0 tx=0x7f66a800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.193+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 shutdown_connections 2023-07-14T11:38:33.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.193+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66a40a1340 0x7f66a40a37f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.193+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66b8105af0 0x7f66b819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.193+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66b80ffae0 0x7f66b8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.193+0000 7f66c07ff700 1 --2- 172.21.15.161:0/3719221597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66b8100490 0x7f66b81983b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.193+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 >> 172.21.15.161:0/3719221597 conn(0x7f66b80fb5e0 msgr2=0x7f66b80fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.194+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 shutdown_connections 2023-07-14T11:38:33.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.194+0000 7f66c07ff700 1 -- 172.21.15.161:0/3719221597 wait complete. 2023-07-14T11:38:33.235 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:38:33.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/3832701649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c810c450 msgr2=0x7fb9c810e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/3832701649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c810c450 0x7fb9c810e830 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000b0a0 tx=0x7fb9c000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/3832701649 shutdown_connections 2023-07-14T11:38:33.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/3832701649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c810c450 0x7fb9c810e830 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/3832701649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c80fe8a0 0x7fb9c80fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/3832701649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c80ff1b0 0x7fb9c810bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/3832701649 >> 172.21.15.161:0/3832701649 conn(0x7fb9c80fa4b0 msgr2=0x7fb9c80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/3832701649 shutdown_connections 2023-07-14T11:38:33.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.571+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/3832701649 wait complete. 2023-07-14T11:38:33.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 Processor -- start 2023-07-14T11:38:33.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 -- start start 2023-07-14T11:38:33.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c80fe8a0 0x7fb9c8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c80ff1b0 0x7fb9c81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c810c450 0x7fb9c819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c819cf00 con 0x7fb9c80fe8a0 2023-07-14T11:38:33.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c819d070 con 0x7fb9c810c450 2023-07-14T11:38:33.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.573+0000 7fb9ce5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c819d230 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c80ff1b0 0x7fb9c81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c80ff1b0 0x7fb9c81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41912/0 (socket says 172.21.15.161:41912) 2023-07-14T11:38:33.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 -- 172.21.15.161:0/173608795 learned_addr learned my addr 172.21.15.161:0/173608795 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c80fe8a0 0x7fb9c8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 -- 172.21.15.161:0/173608795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c810c450 msgr2=0x7fb9c819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.574 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c810c450 0x7fb9c819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 -- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c80fe8a0 msgr2=0x7fb9c8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c80fe8a0 0x7fb9c8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c77fe700 1 -- 172.21.15.161:0/173608795 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0009d00 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.574+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c80fe8a0 0x7fb9c8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:33.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.575+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c80ff1b0 0x7fb9c81984d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00e9a0 tx=0x7fb9bc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.575+0000 7fb9c57fa700 1 -- 172.21.15.161:0/173608795 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc009790 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.575+0000 7fb9c57fa700 1 -- 172.21.15.161:0/173608795 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9bc00cbd0 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.575+0000 7fb9c57fa700 1 -- 172.21.15.161:0/173608795 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc018680 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.575+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c819d510 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.575+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c81a4d50 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.580+0000 7fb9c57fa700 1 -- 172.21.15.161:0/173608795 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb9bc00cd40 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.578 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.581+0000 7fb9e1ea8b80 1 Processor -- start 2023-07-14T11:38:33.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.581+0000 7fb9e1ea8b80 1 -- start start 2023-07-14T11:38:33.579 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.581+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.581+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdf2be0 0x561a0cdfd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.581+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdfdad0 0x561a0cdffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0ccf8cb0 con 0x561a0cdf22d0 2023-07-14T11:38:33.580 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cd4ba40 con 0x561a0cdf2be0 2023-07-14T11:38:33.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cd44ed0 con 0x561a0cdfdad0 2023-07-14T11:38:33.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdf2be0 0x561a0cdfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.581 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdf2be0 0x561a0cdfd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60238/0 (socket says 172.21.15.161:60238) 2023-07-14T11:38:33.582 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9c7fff700 1 -- 172.21.15.161:0/470600125 learned_addr learned my addr 172.21.15.161:0/470600125 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9ccb9c700 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdfdad0 0x561a0cdffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.582+0000 7fb9c7fff700 1 -- 172.21.15.161:0/470600125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdfdad0 msgr2=0x561a0cdffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdfdad0 0x561a0cdffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9c7fff700 1 -- 172.21.15.161:0/470600125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 msgr2=0x561a0cdf26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9c7fff700 1 -- 172.21.15.161:0/470600125 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a0cb72de0 con 0x561a0cdf2be0 2023-07-14T11:38:33.588 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9c57fa700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9b00a13c0 0x7fb9b00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9ccb9c700 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdfdad0 0x561a0cdffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9c57fa700 1 -- 172.21.15.161:0/173608795 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fb9bc014070 con 0x7fb9c80ff1b0 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.583+0000 7fb9ccb9c700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9b00a13c0 0x7fb9b00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.584+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdf2be0 0x561a0cdfd590 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80115f0 tx=0x7fb9b8011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee2b0a2cdecd63a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.584+0000 7fb9ae7fc700 1 -- 172.21.15.161:0/470600125 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8019630 con 0x561a0cdf2be0 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.584+0000 7fb9ae7fc700 1 -- 172.21.15.161:0/470600125 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b8019c50 con 0x561a0cdf2be0 2023-07-14T11:38:33.589 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.584+0000 7fb9ae7fc700 1 -- 172.21.15.161:0/470600125 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800ec20 con 0x561a0cdf2be0 2023-07-14T11:38:33.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/470600125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdf2be0 msgr2=0x561a0cdfd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdf2be0 0x561a0cdfd590 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80115f0 tx=0x7fb9b8011900 comp rx=0 tx=0).stop 2023-07-14T11:38:33.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9ccb9c700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9b00a13c0 0x7fb9b00a3870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000b780 tx=0x7fb9c0009780 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/470600125 shutdown_connections 2023-07-14T11:38:33.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdfdad0 0x561a0cdffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.590 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/470600125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdf2be0 0x561a0cdfd590 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/470600125 >> 172.21.15.161:0/470600125 conn(0x561a0cde8da0 msgr2=0x561a0cdee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/470600125 shutdown_connections 2023-07-14T11:38:33.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.585+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/470600125 wait complete. 2023-07-14T11:38:33.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.586+0000 7fb9e1ea8b80 1 -- start start 2023-07-14T11:38:33.591 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.586+0000 7fb9e1ea8b80 1 Processor -- start 2023-07-14T11:38:33.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdfd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdfacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cce4ac0 con 0x561a0cdf22d0 2023-07-14T11:38:33.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cce47c0 con 0x561a0cdfdad0 2023-07-14T11:38:33.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cbc8920 con 0x561a0cdf2be0 2023-07-14T11:38:33.592 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9e1ea8b80 10 client.0 _ll_register_callbacks cb 0x561a0cce2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41940/0 (socket says 172.21.15.161:41940) 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9c77fe700 1 -- 172.21.15.161:0/980405836 learned_addr learned my addr 172.21.15.161:0/980405836 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9ccb9c700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdfacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.587+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdfd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9c77fe700 1 -- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 msgr2=0x561a0cdfacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdfacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.593 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9c77fe700 1 -- 172.21.15.161:0/980405836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 msgr2=0x561a0cdfd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdfd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9c77fe700 1 -- 172.21.15.161:0/980405836 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c810f3b0 con 0x561a0cdf2be0 2023-07-14T11:38:33.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdfd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:38:33.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfa700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009230 tx=0x7fb9bc0998d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46e7311ecd366f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:33.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9a77fe700 1 -- 172.21.15.161:0/980405836 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc099660 con 0x561a0cdf2be0 2023-07-14T11:38:33.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9a77fe700 1 -- 172.21.15.161:0/980405836 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fb9bc099f70 con 0x561a0cdf2be0 2023-07-14T11:38:33.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a0cc9e4f0 con 0x561a0cdf2be0 2023-07-14T11:38:33.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9a77fe700 1 -- 172.21.15.161:0/980405836 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9bc09dfa0 con 0x561a0cdf2be0 2023-07-14T11:38:33.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9e1ea8b80 10 client.27493 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:33.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.588+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fb9b8009dd0 con 0x561a0cdf2be0 2023-07-14T11:38:33.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9a77fe700 1 -- 172.21.15.161:0/980405836 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9bc0b1030 con 0x561a0cdf2be0 2023-07-14T11:38:33.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9a77fe700 1 -- 172.21.15.161:0/980405836 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb9bc0b9d20 con 0x561a0cdf2be0 2023-07-14T11:38:33.595 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9e1ea8b80 10 client.27493 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:33.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9e1ea8b80 10 client.27493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:33.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9e1ea8b80 20 client.27493 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:33.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce15ae0 0x561a0ce17ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9e1ea8b80 4 client.27493 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:33.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a0cd3deb0 con 0x561a0ce15ae0 2023-07-14T11:38:33.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.589+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce15ae0 0x561a0ce17ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.590+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce15ae0 0x561a0ce17ef0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd428ab24239451 server_cookie=7cef63049cdff6d8 in_seq=0 out_seq=0 2023-07-14T11:38:33.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.590+0000 7fb9a77fe700 10 client.27493 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:33.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.592+0000 7fb9a77fe700 1 -- 172.21.15.161:0/980405836 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561a0cd3deb0 con 0x561a0ce15ae0 2023-07-14T11:38:33.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.592+0000 7fb9a77fe700 10 client.27493 handle_command_reply: tid=0 2023-07-14T11:38:33.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.592+0000 7fb9e1ea8b80 1 client.27493 shutdown 2023-07-14T11:38:33.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 msgr2=0x561a0cdfa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfa700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009230 tx=0x7fb9bc0998d0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 shutdown_connections 2023-07-14T11:38:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce15ae0 0x561a0ce17ef0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdfd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdfacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/980405836 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfa700 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 >> 172.21.15.161:0/980405836 conn(0x561a0cde8da0 msgr2=0x561a0cdee230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9a77fe700 0 client.27493 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 shutdown_connections 2023-07-14T11:38:33.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/980405836 wait complete. 2023-07-14T11:38:33.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.593+0000 7fb9e1ea8b80 20 client.27493 trim_cache size 0 max 16384 2023-07-14T11:38:33.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 Processor -- start 2023-07-14T11:38:33.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 -- start start 2023-07-14T11:38:33.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0ce0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0ce13190 con 0x561a0cdf22d0 2023-07-14T11:38:33.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cdefc70 con 0x561a0cdfdad0 2023-07-14T11:38:33.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.612+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0ce0d2b0 con 0x561a0cdf2be0 2023-07-14T11:38:33.612 INFO:tasks.workunit.client.0.smithi161.stdout: 10 113847 30.19 MB/sec execute 75 sec latency 64.170 ms 2023-07-14T11:38:33.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.613+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.613+0000 7fb9ccb9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.613+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41944/0 (socket says 172.21.15.161:41944) 2023-07-14T11:38:33.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.613+0000 7fb9c77fe700 1 -- 172.21.15.161:0/555320103 learned_addr learned my addr 172.21.15.161:0/555320103 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.613+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0ce0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9c77fe700 1 -- 172.21.15.161:0/555320103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 msgr2=0x561a0cdf5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9c77fe700 1 -- 172.21.15.161:0/555320103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 msgr2=0x561a0ce0cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0ce0cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9c77fe700 1 -- 172.21.15.161:0/555320103 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a0cb5bc60 con 0x561a0cdf2be0 2023-07-14T11:38:33.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfcfb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc096470 tx=0x7fb9bc00e930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9855fe3e89f7e15e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9aeffd700 1 -- 172.21.15.161:0/555320103 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0a2130 con 0x561a0cdf2be0 2023-07-14T11:38:33.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9aeffd700 1 -- 172.21.15.161:0/555320103 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9bc0b1020 con 0x561a0cdf2be0 2023-07-14T11:38:33.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.614+0000 7fb9aeffd700 1 -- 172.21.15.161:0/555320103 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0996c0 con 0x561a0cdf2be0 2023-07-14T11:38:33.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/555320103 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 msgr2=0x561a0cdfcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfcfb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc096470 tx=0x7fb9bc00e930 comp rx=0 tx=0).stop 2023-07-14T11:38:33.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/555320103 shutdown_connections 2023-07-14T11:38:33.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0ce0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/555320103 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdfcfb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/555320103 >> 172.21.15.161:0/555320103 conn(0x561a0cde8da0 msgr2=0x561a0ce11e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/555320103 shutdown_connections 2023-07-14T11:38:33.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.615+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/555320103 wait complete. 2023-07-14T11:38:33.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.616+0000 7fb9e1ea8b80 1 -- start start 2023-07-14T11:38:33.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.616+0000 7fb9e1ea8b80 1 Processor -- start 2023-07-14T11:38:33.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.617+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.617+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdf8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.617+0000 7fb9e1ea8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.617+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cdf6ec0 con 0x561a0cdf22d0 2023-07-14T11:38:33.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.617+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cdf7030 con 0x561a0cdfdad0 2023-07-14T11:38:33.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.617+0000 7fb9e1ea8b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0cdf71f0 con 0x561a0cdf2be0 2023-07-14T11:38:33.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.618+0000 7fb9e1ea8b80 10 client.0 _ll_register_callbacks cb 0x7fb9bc099ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:33.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.618+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdf8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.618+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdf8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41960/0 (socket says 172.21.15.161:41960) 2023-07-14T11:38:33.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.618+0000 7fb9c77fe700 1 -- 172.21.15.161:0/1798481903 learned_addr learned my addr 172.21.15.161:0/1798481903 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:33.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.618+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.618+0000 7fb9ccb9c700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9c77fe700 1 -- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 msgr2=0x561a0cdf6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9c77fe700 1 -- 172.21.15.161:0/1798481903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 msgr2=0x561a0cdf7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:33.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9c77fe700 1 -- 172.21.15.161:0/1798481903 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b8009dd0 con 0x561a0cdf2be0 2023-07-14T11:38:33.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:38:33.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9c77fe700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdf8240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c5b0 tx=0x7fb9bc099690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c99cf9459b05399 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:33.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:33.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9a77fe700 1 -- 172.21.15.161:0/1798481903 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc099f70 con 0x561a0cdf2be0 2023-07-14T11:38:33.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.619+0000 7fb9a77fe700 1 -- 172.21.15.161:0/1798481903 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fb9bc09dfa0 con 0x561a0cdf2be0 2023-07-14T11:38:33.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.620+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b8019c50 con 0x561a0cdf2be0 2023-07-14T11:38:33.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.620+0000 7fb9a77fe700 1 -- 172.21.15.161:0/1798481903 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9bc0a23c0 con 0x561a0cdf2be0 2023-07-14T11:38:33.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.620+0000 7fb9e1ea8b80 10 client.27499 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:33.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.620+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561a0cb99250 con 0x561a0cdf2be0 2023-07-14T11:38:33.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.620+0000 7fb9a77fe700 1 -- 172.21.15.161:0/1798481903 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b8019c50 con 0x561a0cdf2be0 2023-07-14T11:38:33.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9a77fe700 1 -- 172.21.15.161:0/1798481903 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb9bc0b9d20 con 0x561a0cdf2be0 2023-07-14T11:38:33.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9e1ea8b80 10 client.27499 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:33.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9e1ea8b80 10 client.27499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:33.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9e1ea8b80 20 client.27499 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:33.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce6a440 0x561a0ce6c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:33.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9e1ea8b80 4 client.27499 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:38:33.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a0cd00050 con 0x561a0ce6a440 2023-07-14T11:38:33.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.621+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce6a440 0x561a0ce6c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:33.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.622+0000 7fb9a77fe700 10 client.27499 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:33.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.622+0000 7fb9c7fff700 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce6a440 0x561a0ce6c850 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41de12df2ddadb9f server_cookie=f6374c8f782d5eac in_seq=0 out_seq=0 2023-07-14T11:38:33.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.623+0000 7fb9a77fe700 1 -- 172.21.15.161:0/1798481903 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a0cd00050 con 0x561a0ce6a440 2023-07-14T11:38:33.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.623+0000 7fb9a77fe700 10 client.27499 handle_command_reply: tid=0 2023-07-14T11:38:33.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.623+0000 7fb9e1ea8b80 1 client.27499 shutdown 2023-07-14T11:38:33.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 msgr2=0x561a0cdf8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdf8240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c5b0 tx=0x7fb9bc099690 comp rx=0 tx=0).stop 2023-07-14T11:38:33.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 shutdown_connections 2023-07-14T11:38:33.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561a0ce6a440 0x561a0ce6c850 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0cdf22d0 0x561a0cdf7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a0cdfdad0 0x561a0cdf6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 --2- 172.21.15.161:0/1798481903 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561a0cdf2be0 0x561a0cdf8240 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 >> 172.21.15.161:0/1798481903 conn(0x561a0cde8da0 msgr2=0x561a0cdf5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9a77fe700 0 client.27499 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:33.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.624+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 shutdown_connections 2023-07-14T11:38:33.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.625+0000 7fb9e1ea8b80 1 -- 172.21.15.161:0/1798481903 wait complete. 2023-07-14T11:38:33.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.625+0000 7fb9e1ea8b80 20 client.27499 trim_cache size 0 max 16384 2023-07-14T11:38:33.634 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:38:33.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.626+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9b00a13c0 msgr2=0x7fb9b00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.626+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9b00a13c0 0x7fb9b00a3870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000b780 tx=0x7fb9c0009780 comp rx=0 tx=0).stop 2023-07-14T11:38:33.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.626+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c80ff1b0 msgr2=0x7fb9c81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:33.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.626+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c80ff1b0 0x7fb9c81984d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00e9a0 tx=0x7fb9bc00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 shutdown_connections 2023-07-14T11:38:33.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb9b00a13c0 0x7fb9b00a3870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c80fe8a0 0x7fb9c8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c810c450 0x7fb9c819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 --2- 172.21.15.161:0/173608795 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb9c80ff1b0 0x7fb9c81984d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:33.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 >> 172.21.15.161:0/173608795 conn(0x7fb9c80fa4b0 msgr2=0x7fb9c810d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:33.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 shutdown_connections 2023-07-14T11:38:33.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:33.627+0000 7fb9ce5ff700 1 -- 172.21.15.161:0/173608795 wait complete. 2023-07-14T11:38:34.413 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:34 smithi175 ceph-mon[129081]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 594 op/s 2023-07-14T11:38:34.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:34 smithi182 ceph-mon[129366]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 594 op/s 2023-07-14T11:38:34.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:34 smithi161 ceph-mon[95526]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 594 op/s 2023-07-14T11:38:34.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 114612 30.47 MB/sec execute 76 sec latency 57.155 ms 2023-07-14T11:38:34.655 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:38:34.656 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:38:34.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 -- 172.21.15.161:0/2228853096 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 msgr2=0x7f520c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 --2- 172.21.15.161:0/2228853096 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 0x7f520c109b00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5200009990 tx=0x7f5200009ca0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 -- 172.21.15.161:0/2228853096 shutdown_connections 2023-07-14T11:38:34.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 --2- 172.21.15.161:0/2228853096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f520c100d00 0x7f520c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 --2- 172.21.15.161:0/2228853096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c10a040 0x7f520c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 --2- 172.21.15.161:0/2228853096 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 0x7f520c109b00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 -- 172.21.15.161:0/2228853096 >> 172.21.15.161:0/2228853096 conn(0x7f520c0fa4b0 msgr2=0x7f520c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:34.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 -- 172.21.15.161:0/2228853096 shutdown_connections 2023-07-14T11:38:34.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.910+0000 7f5211bff700 1 -- 172.21.15.161:0/2228853096 wait complete. 2023-07-14T11:38:34.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.911+0000 7f5211bff700 1 Processor -- start 2023-07-14T11:38:34.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f5211bff700 1 -- start start 2023-07-14T11:38:34.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f5211bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c100d00 0x7f520c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f5211bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 0x7f520c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f5211bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f520c10a040 0x7f520c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f5211bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c19cdf0 con 0x7f520c10a040 2023-07-14T11:38:34.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f5211bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c19cf60 con 0x7f520c100d00 2023-07-14T11:38:34.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f5211bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c19d120 con 0x7f520c101610 2023-07-14T11:38:34.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c100d00 0x7f520c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c100d00 0x7f520c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60272/0 (socket says 172.21.15.161:60272) 2023-07-14T11:38:34.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f520b7fe700 1 -- 172.21.15.161:0/943260867 learned_addr learned my addr 172.21.15.161:0/943260867 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:34.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.912+0000 7f520affd700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 0x7f520c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f520bfff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f520c10a040 0x7f520c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f520b7fe700 1 -- 172.21.15.161:0/943260867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 msgr2=0x7f520c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f520b7fe700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 0x7f520c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f520b7fe700 1 -- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f520c10a040 msgr2=0x7f520c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f520b7fe700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f520c10a040 0x7f520c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f520b7fe700 1 -- 172.21.15.161:0/943260867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009520 con 0x7f520c100d00 2023-07-14T11:38:34.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f520b7fe700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c100d00 0x7f520c197e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00eb30 tx=0x7f51fc00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:34.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f5208ff9700 1 -- 172.21.15.161:0/943260867 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc009880 con 0x7f520c100d00 2023-07-14T11:38:34.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f5208ff9700 1 -- 172.21.15.161:0/943260867 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51fc0044e0 con 0x7f520c100d00 2023-07-14T11:38:34.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52000095f0 con 0x7f520c100d00 2023-07-14T11:38:34.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.913+0000 7f5208ff9700 1 -- 172.21.15.161:0/943260867 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc003d80 con 0x7f520c100d00 2023-07-14T11:38:34.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.914+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c1a4a60 con 0x7f520c100d00 2023-07-14T11:38:34.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.919+0000 7f5208ff9700 1 -- 172.21.15.161:0/943260867 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f51fc003f20 con 0x7f520c100d00 2023-07-14T11:38:34.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.919+0000 7f5225597b80 1 Processor -- start 2023-07-14T11:38:34.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f5225597b80 1 -- start start 2023-07-14T11:38:34.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f5225597b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0350 0x55ea8f2d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f5225597b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f5225597b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f5225597b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f1d6d30 con 0x55ea8f2d0350 2023-07-14T11:38:34.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f5225597b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f229ac0 con 0x55ea8f2dbb50 2023-07-14T11:38:34.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f5225597b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f222f50 con 0x55ea8f2d0c60 2023-07-14T11:38:34.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f520bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0350 0x55ea8f2d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f520affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f520affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2ddf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60280/0 (socket says 172.21.15.161:60280) 2023-07-14T11:38:34.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.920+0000 7f520affd700 1 -- 172.21.15.161:0/233532367 learned_addr learned my addr 172.21.15.161:0/233532367 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:34.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f5208ff9700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f51f40a1340 0x7f51f40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f520affd700 1 -- 172.21.15.161:0/233532367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0c60 msgr2=0x55ea8f2db610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f520affd700 1 --2- 172.21.15.161:0/233532367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2db610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f520affd700 1 -- 172.21.15.161:0/233532367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0350 msgr2=0x55ea8f2d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f520affd700 1 --2- 172.21.15.161:0/233532367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0350 0x55ea8f2d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f520affd700 1 -- 172.21.15.161:0/233532367 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea8f050e60 con 0x55ea8f2dbb50 2023-07-14T11:38:34.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f5208ff9700 1 -- 172.21.15.161:0/943260867 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f51fc014070 con 0x7f520c100d00 2023-07-14T11:38:34.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f520affd700 1 --2- 172.21.15.161:0/233532367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2ddf30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5200015000 tx=0x7f5200005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6b0ce89c90ce7c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:34.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f520bfff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f51f40a1340 0x7f51f40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.921+0000 7f51f1ffb700 1 -- 172.21.15.161:0/233532367 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520001d070 con 0x55ea8f2dbb50 2023-07-14T11:38:34.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.922+0000 7f51f1ffb700 1 -- 172.21.15.161:0/233532367 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f520001e040 con 0x55ea8f2dbb50 2023-07-14T11:38:34.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.922+0000 7f51f1ffb700 1 -- 172.21.15.161:0/233532367 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000048c0 con 0x55ea8f2dbb50 2023-07-14T11:38:34.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.922+0000 7f5225597b80 1 -- 172.21.15.161:0/233532367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 msgr2=0x55ea8f2ddf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.922+0000 7f5225597b80 1 --2- 172.21.15.161:0/233532367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2ddf30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5200015000 tx=0x7f5200005ea0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.922+0000 7f520bfff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f51f40a1340 0x7f51f40a37f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52040075c0 tx=0x7f5204009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:34.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 -- 172.21.15.161:0/233532367 shutdown_connections 2023-07-14T11:38:34.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 --2- 172.21.15.161:0/233532367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0350 0x55ea8f2d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 --2- 172.21.15.161:0/233532367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2ddf30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 --2- 172.21.15.161:0/233532367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2db610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 -- 172.21.15.161:0/233532367 >> 172.21.15.161:0/233532367 conn(0x55ea8f2c6e20 msgr2=0x55ea8f2cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:34.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 -- 172.21.15.161:0/233532367 shutdown_connections 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 -- 172.21.15.161:0/233532367 wait complete. 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.923+0000 7f5225597b80 1 -- start start 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 1 Processor -- start 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2db520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f1c2b40 con 0x55ea8f2d0c60 2023-07-14T11:38:34.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f1c2840 con 0x55ea8f2dbb50 2023-07-14T11:38:34.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f0a69a0 con 0x55ea8f2d0350 2023-07-14T11:38:34.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f5225597b80 10 client.0 _ll_register_callbacks cb 0x55ea8f1c07e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:34.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f520bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2db520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41984/0 (socket says 172.21.15.161:41984) 2023-07-14T11:38:34.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f520bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60294/0 (socket says 172.21.15.161:60294) 2023-07-14T11:38:34.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f520b7fe700 1 -- 172.21.15.161:0/3456494869 learned_addr learned my addr 172.21.15.161:0/3456494869 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:34.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.924+0000 7f520affd700 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f520b7fe700 1 -- 172.21.15.161:0/3456494869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 msgr2=0x55ea8f2d8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f520b7fe700 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f520b7fe700 1 -- 172.21.15.161:0/3456494869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 msgr2=0x55ea8f2d8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f520b7fe700 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f520b7fe700 1 -- 172.21.15.161:0/3456494869 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c10d170 con 0x55ea8f2d0350 2023-07-14T11:38:34.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f520b7fe700 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2db520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00eb00 tx=0x7f51fc09a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5f060dd39107268 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:34.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f51eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:34.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f51eaffd700 1 -- 172.21.15.161:0/3456494869 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc099c90 con 0x55ea8f2d0350 2023-07-14T11:38:34.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea8f17c570 con 0x55ea8f2d0350 2023-07-14T11:38:34.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.925+0000 7f51eaffd700 1 -- 172.21.15.161:0/3456494869 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f51fc01e2e0 con 0x55ea8f2d0350 2023-07-14T11:38:34.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f51eaffd700 1 -- 172.21.15.161:0/3456494869 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc096670 con 0x55ea8f2d0350 2023-07-14T11:38:34.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 10 client.27511 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:34.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ea8f039ce0 con 0x55ea8f2d0350 2023-07-14T11:38:34.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f51eaffd700 1 -- 172.21.15.161:0/3456494869 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51fc0a6030 con 0x55ea8f2d0350 2023-07-14T11:38:34.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f51eaffd700 1 -- 172.21.15.161:0/3456494869 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f51fc0cd200 con 0x55ea8f2d0350 2023-07-14T11:38:34.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 10 client.27511 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:34.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 10 client.27511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:34.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 20 client.27511 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:34.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f2eb3f0 0x55ea8f2ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 4 client.27511 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:34.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.926+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea8f21bf30 con 0x55ea8f2eb3f0 2023-07-14T11:38:34.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.927+0000 7f520affd700 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f2eb3f0 0x55ea8f2ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.927+0000 7f520affd700 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f2eb3f0 0x55ea8f2ed800 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50294bbbcdb2712b server_cookie=34e1cf47d6587c5b in_seq=0 out_seq=0 2023-07-14T11:38:34.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.927+0000 7f51eaffd700 10 client.27511 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f51eaffd700 1 -- 172.21.15.161:0/3456494869 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea8f21bf30 con 0x55ea8f2eb3f0 2023-07-14T11:38:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f51eaffd700 10 client.27511 handle_command_reply: tid=0 2023-07-14T11:38:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 client.27511 shutdown 2023-07-14T11:38:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 msgr2=0x55ea8f2db520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2db520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00eb00 tx=0x7f51fc09a100 comp rx=0 tx=0).stop 2023-07-14T11:38:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 shutdown_connections 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f2eb3f0 0x55ea8f2ed800 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2db520 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f51eaffd700 0 client.27511 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 --2- 172.21.15.161:0/3456494869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.930+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 >> 172.21.15.161:0/3456494869 conn(0x55ea8f2c6e20 msgr2=0x55ea8f2cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.931+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 shutdown_connections 2023-07-14T11:38:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.931+0000 7f5225597b80 1 -- 172.21.15.161:0/3456494869 wait complete. 2023-07-14T11:38:34.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.931+0000 7f5225597b80 20 client.27511 trim_cache size 0 max 16384 2023-07-14T11:38:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 Processor -- start 2023-07-14T11:38:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 -- start start 2023-07-14T11:38:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f2cd580 con 0x55ea8f2d0c60 2023-07-14T11:38:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f2d9770 con 0x55ea8f2dbb50 2023-07-14T11:38:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f5225597b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f2e8a70 con 0x55ea8f2d0350 2023-07-14T11:38:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f520bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f520affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f520bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60302/0 (socket says 172.21.15.161:60302) 2023-07-14T11:38:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:42000/0 (socket says 172.21.15.161:42000) 2023-07-14T11:38:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.950+0000 7f520bfff700 1 -- 172.21.15.161:0/2338524448 learned_addr learned my addr 172.21.15.161:0/2338524448 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.951+0000 7f520b7fe700 1 -- 172.21.15.161:0/2338524448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 msgr2=0x55ea8f2d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.951+0000 7f520b7fe700 1 --2- 172.21.15.161:0/2338524448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.951+0000 7f520b7fe700 1 -- 172.21.15.161:0/2338524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 msgr2=0x55ea8f2d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.951+0000 7f520b7fe700 1 --2- 172.21.15.161:0/2338524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.951+0000 7f520b7fe700 1 -- 172.21.15.161:0/2338524448 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea8f2c8660 con 0x55ea8f2d0350 2023-07-14T11:38:34.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.951+0000 7f520b7fe700 1 --2- 172.21.15.161:0/2338524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e2530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096da0 tx=0x7f51fc0988e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eef535528d4a1aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:34.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.951+0000 7f51f27fc700 1 -- 172.21.15.161:0/2338524448 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0b1070 con 0x55ea8f2d0350 2023-07-14T11:38:34.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f51f27fc700 1 -- 172.21.15.161:0/2338524448 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51fc0a6070 con 0x55ea8f2d0350 2023-07-14T11:38:34.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f51f27fc700 1 -- 172.21.15.161:0/2338524448 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00cbd0 con 0x55ea8f2d0350 2023-07-14T11:38:34.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f5225597b80 1 -- 172.21.15.161:0/2338524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 msgr2=0x55ea8f2e2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f5225597b80 1 --2- 172.21.15.161:0/2338524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e2530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096da0 tx=0x7f51fc0988e0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f5225597b80 1 -- 172.21.15.161:0/2338524448 shutdown_connections 2023-07-14T11:38:34.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f5225597b80 1 --2- 172.21.15.161:0/2338524448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2dbb50 0x55ea8f2d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f5225597b80 1 --2- 172.21.15.161:0/2338524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e2530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f5225597b80 1 --2- 172.21.15.161:0/2338524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.952+0000 7f5225597b80 1 -- 172.21.15.161:0/2338524448 >> 172.21.15.161:0/2338524448 conn(0x55ea8f2c6e20 msgr2=0x55ea8f2d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.953+0000 7f5225597b80 1 -- 172.21.15.161:0/2338524448 shutdown_connections 2023-07-14T11:38:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.953+0000 7f5225597b80 1 -- 172.21.15.161:0/2338524448 wait complete. 2023-07-14T11:38:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.953+0000 7f5225597b80 1 -- start start 2023-07-14T11:38:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.953+0000 7f5225597b80 1 Processor -- start 2023-07-14T11:38:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f5225597b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f5225597b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f5225597b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2dbb50 0x55ea8f33f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f5225597b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f2e1e90 con 0x55ea8f2d0c60 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f5225597b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f33f820 con 0x55ea8f2d0350 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f5225597b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea8f33f9e0 con 0x55ea8f2dbb50 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f5225597b80 10 client.0 _ll_register_callbacks cb 0x55ea8f1c07e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60304/0 (socket says 172.21.15.161:60304) 2023-07-14T11:38:34.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 -- 172.21.15.161:0/936193366 learned_addr learned my addr 172.21.15.161:0/936193366 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:34.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520bfff700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2dbb50 0x55ea8f33f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520affd700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 -- 172.21.15.161:0/936193366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2dbb50 msgr2=0x55ea8f33f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2dbb50 0x55ea8f33f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 -- 172.21.15.161:0/936193366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 msgr2=0x55ea8f2e1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:34.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2e1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520b7fe700 1 -- 172.21.15.161:0/936193366 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea8f039ce0 con 0x55ea8f2d0350 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.954+0000 7f520bfff700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2dbb50 0x55ea8f33f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f520b7fe700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e1200 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096830 tx=0x7f51fc009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f46b7611a48fcf4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f51eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f51eaffd700 1 -- 172.21.15.161:0/936193366 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0b1050 con 0x55ea8f2d0350 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f51eaffd700 1 -- 172.21.15.161:0/936193366 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f51fc00cbd0 con 0x55ea8f2d0350 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f520001e040 con 0x55ea8f2d0350 2023-07-14T11:38:34.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f51eaffd700 1 -- 172.21.15.161:0/936193366 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc099c90 con 0x55ea8f2d0350 2023-07-14T11:38:34.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f5225597b80 10 client.27479 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:34.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ea8f0772d0 con 0x55ea8f2d0350 2023-07-14T11:38:34.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.955+0000 7f51eaffd700 1 -- 172.21.15.161:0/936193366 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f520001e040 con 0x55ea8f2d0350 2023-07-14T11:38:34.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f51eaffd700 1 -- 172.21.15.161:0/936193366 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f51fc0a22f0 con 0x55ea8f2d0350 2023-07-14T11:38:34.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f5225597b80 10 client.27479 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:34.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f5225597b80 10 client.27479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:34.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f5225597b80 20 client.27479 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:34.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f5225597b80 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f340c50 0x55ea8f343030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:34.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f5225597b80 4 client.27479 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:38:34.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ea8f1de0d0 con 0x55ea8f340c50 2023-07-14T11:38:34.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f520affd700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f340c50 0x55ea8f343030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:34.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f51eaffd700 10 client.27479 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:34.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.956+0000 7f520affd700 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f340c50 0x55ea8f343030 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac44befcf9c3e15 server_cookie=7fd1fe93e7fb1e23 in_seq=0 out_seq=0 2023-07-14T11:38:34.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f51eaffd700 1 -- 172.21.15.161:0/936193366 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ea8f1de0d0 con 0x55ea8f340c50 2023-07-14T11:38:34.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f51eaffd700 10 client.27479 handle_command_reply: tid=0 2023-07-14T11:38:34.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 client.27479 shutdown 2023-07-14T11:38:34.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2d0350 msgr2=0x55ea8f2e1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e1200 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096830 tx=0x7f51fc009f60 comp rx=0 tx=0).stop 2023-07-14T11:38:34.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 shutdown_connections 2023-07-14T11:38:34.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ea8f340c50 0x55ea8f343030 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ea8f2dbb50 0x55ea8f33f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea8f2d0350 0x55ea8f2e1200 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f51eaffd700 0 client.27479 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:34.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 --2- 172.21.15.161:0/936193366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea8f2d0c60 0x55ea8f2e1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.972+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 >> 172.21.15.161:0/936193366 conn(0x55ea8f2c6e20 msgr2=0x55ea8f2d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:34.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.973+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 shutdown_connections 2023-07-14T11:38:34.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.973+0000 7f5225597b80 1 -- 172.21.15.161:0/936193366 wait complete. 2023-07-14T11:38:34.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.973+0000 7f5225597b80 20 client.27479 trim_cache size 0 max 16384 2023-07-14T11:38:34.973 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:38:34.973 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:38:34.973 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "eb3d05be-6184-4b4f-8076-e33e9a553189", 2023-07-14T11:38:34.973 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:38:34.973 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:38:34.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.975+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f51f40a1340 msgr2=0x7f51f40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.975+0000 7f5211bff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f51f40a1340 0x7f51f40a37f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52040075c0 tx=0x7f5204009040 comp rx=0 tx=0).stop 2023-07-14T11:38:34.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.975+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c100d00 msgr2=0x7f520c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:34.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.975+0000 7f5211bff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c100d00 0x7f520c197e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00eb30 tx=0x7f51fc00ee40 comp rx=0 tx=0).stop 2023-07-14T11:38:34.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.976+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 shutdown_connections 2023-07-14T11:38:34.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.976+0000 7f5211bff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f51f40a1340 0x7f51f40a37f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.976+0000 7f5211bff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f520c10a040 0x7f520c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.976+0000 7f5211bff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f520c100d00 0x7f520c197e10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.976+0000 7f5211bff700 1 --2- 172.21.15.161:0/943260867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f520c101610 0x7f520c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:34.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.976+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 >> 172.21.15.161:0/943260867 conn(0x7f520c0fa4b0 msgr2=0x7f520c10c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:34.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.976+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 shutdown_connections 2023-07-14T11:38:34.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:34.977+0000 7f5211bff700 1 -- 172.21.15.161:0/943260867 wait complete. 2023-07-14T11:38:34.999 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:38:35.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.259+0000 7ff353bff700 1 -- 172.21.15.161:0/1180501342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0731c0 msgr2=0x7ff34c1105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.259+0000 7ff353bff700 1 --2- 172.21.15.161:0/1180501342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0731c0 0x7ff34c1105a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009990 tx=0x7ff33c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.259+0000 7ff353bff700 1 -- 172.21.15.161:0/1180501342 shutdown_connections 2023-07-14T11:38:35.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.259+0000 7ff353bff700 1 --2- 172.21.15.161:0/1180501342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff34c0728b0 0x7ff34c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.259+0000 7ff353bff700 1 --2- 172.21.15.161:0/1180501342 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff34c110ae0 0x7ff34c112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.259+0000 7ff353bff700 1 --2- 172.21.15.161:0/1180501342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0731c0 0x7ff34c1105a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.259+0000 7ff353bff700 1 -- 172.21.15.161:0/1180501342 >> 172.21.15.161:0/1180501342 conn(0x7ff34c0fb5f0 msgr2=0x7ff34c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:35.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.260+0000 7ff353bff700 1 -- 172.21.15.161:0/1180501342 shutdown_connections 2023-07-14T11:38:35.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.260+0000 7ff353bff700 1 -- 172.21.15.161:0/1180501342 wait complete. 2023-07-14T11:38:35.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 Processor -- start 2023-07-14T11:38:35.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 -- start start 2023-07-14T11:38:35.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0728b0 0x7ff34c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff34c0731c0 0x7ff34c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff34c110ae0 0x7ff34c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c1a1210 con 0x7ff34c110ae0 2023-07-14T11:38:35.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c1a1380 con 0x7ff34c0728b0 2023-07-14T11:38:35.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.261+0000 7ff353bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c1a1540 con 0x7ff34c0731c0 2023-07-14T11:38:35.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0728b0 0x7ff34c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0728b0 0x7ff34c19c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60320/0 (socket says 172.21.15.161:60320) 2023-07-14T11:38:35.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 -- 172.21.15.161:0/2925664009 learned_addr learned my addr 172.21.15.161:0/2925664009 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:35.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35119a700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff34c0731c0 0x7ff34c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35219c700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff34c110ae0 0x7ff34c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 -- 172.21.15.161:0/2925664009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff34c0731c0 msgr2=0x7ff34c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff34c0731c0 0x7ff34c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 -- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff34c110ae0 msgr2=0x7ff34c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff34c110ae0 0x7ff34c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 -- 172.21.15.161:0/2925664009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c0095f0 con 0x7ff34c0728b0 2023-07-14T11:38:35.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35119a700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff34c0731c0 0x7ff34c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:38:35.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff35199b700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0728b0 0x7ff34c19c230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff34800e9a0 tx=0x7ff34800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:35.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.262+0000 7ff342ffd700 1 -- 172.21.15.161:0/2925664009 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348009790 con 0x7ff34c0728b0 2023-07-14T11:38:35.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.263+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c1a1820 con 0x7ff34c0728b0 2023-07-14T11:38:35.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.263+0000 7ff342ffd700 1 -- 172.21.15.161:0/2925664009 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348010430 con 0x7ff34c0728b0 2023-07-14T11:38:35.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.263+0000 7ff342ffd700 1 -- 172.21.15.161:0/2925664009 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348018680 con 0x7ff34c0728b0 2023-07-14T11:38:35.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.263+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c1a9060 con 0x7ff34c0728b0 2023-07-14T11:38:35.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.267+0000 7ff342ffd700 1 -- 172.21.15.161:0/2925664009 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff348010810 con 0x7ff34c0728b0 2023-07-14T11:38:35.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.269+0000 7ff3675a5b80 1 Processor -- start 2023-07-14T11:38:35.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.269+0000 7ff3675a5b80 1 -- start start 2023-07-14T11:38:35.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e80590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e80ad0 0x55dd25e82eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d7bcb0 con 0x55dd25e75be0 2023-07-14T11:38:35.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25dcea40 con 0x55dd25e80ad0 2023-07-14T11:38:35.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25dc7ed0 con 0x55dd25e752d0 2023-07-14T11:38:35.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff342ffd700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff3380a12c0 0x7ff3380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff35219c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff35119a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e80ad0 0x55dd25e82eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff35119a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e80ad0 0x55dd25e82eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60332/0 (socket says 172.21.15.161:60332) 2023-07-14T11:38:35.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff35119a700 1 -- 172.21.15.161:0/1022246880 learned_addr learned my addr 172.21.15.161:0/1022246880 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:35.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff35219c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e756a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:42042/0 (socket says 172.21.15.161:42042) 2023-07-14T11:38:35.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff342ffd700 1 -- 172.21.15.161:0/2925664009 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff348014070 con 0x7ff34c0728b0 2023-07-14T11:38:35.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.270+0000 7ff35199b700 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e80590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.271+0000 7ff35119a700 1 -- 172.21.15.161:0/1022246880 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 msgr2=0x55dd25e756a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.271+0000 7ff35119a700 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e756a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.271+0000 7ff35119a700 1 -- 172.21.15.161:0/1022246880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 msgr2=0x55dd25e80590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.271+0000 7ff35119a700 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e80590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.271+0000 7ff35219c700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff3380a12c0 0x7ff3380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.271+0000 7ff35119a700 1 -- 172.21.15.161:0/1022246880 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd25bf5de0 con 0x55dd25e80ad0 2023-07-14T11:38:35.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.271+0000 7ff35119a700 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e80ad0 0x55dd25e82eb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009990 tx=0x7ff33c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b2b144005e692f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:35.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.272+0000 7ff35219c700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff3380a12c0 0x7ff3380a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff344009ec0 tx=0x7ff344004b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:35.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.272+0000 7ff337fff700 1 -- 172.21.15.161:0/1022246880 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c01d070 con 0x55dd25e80ad0 2023-07-14T11:38:35.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.272+0000 7ff337fff700 1 -- 172.21.15.161:0/1022246880 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c01e040 con 0x55dd25e80ad0 2023-07-14T11:38:35.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.272+0000 7ff337fff700 1 -- 172.21.15.161:0/1022246880 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c004d90 con 0x55dd25e80ad0 2023-07-14T11:38:35.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.272+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1022246880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e80ad0 msgr2=0x55dd25e82eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.272+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e80ad0 0x55dd25e82eb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009990 tx=0x7ff33c00b610 comp rx=0 tx=0).stop 2023-07-14T11:38:35.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1022246880 shutdown_connections 2023-07-14T11:38:35.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e80ad0 0x55dd25e82eb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1022246880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e80590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1022246880 >> 172.21.15.161:0/1022246880 conn(0x55dd25e6bda0 msgr2=0x55dd25e716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:35.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1022246880 shutdown_connections 2023-07-14T11:38:35.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1022246880 wait complete. 2023-07-14T11:38:35.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 -- start start 2023-07-14T11:38:35.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.273+0000 7ff3675a5b80 1 Processor -- start 2023-07-14T11:38:35.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e752d0 0x55dd25e80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e7d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e80ad0 0x55dd25e7dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d67ac0 con 0x55dd25e75be0 2023-07-14T11:38:35.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d677c0 con 0x55dd25e752d0 2023-07-14T11:38:35.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25c4b920 con 0x55dd25e80ad0 2023-07-14T11:38:35.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff3675a5b80 10 client.0 _ll_register_callbacks cb 0x55dd25d65760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:35.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff35119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e7d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff35199b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e752d0 0x55dd25e80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff35119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e7d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39408/0 (socket says 172.21.15.161:39408) 2023-07-14T11:38:35.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff35199b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e752d0 0x55dd25e80480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60338/0 (socket says 172.21.15.161:60338) 2023-07-14T11:38:35.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff35119a700 1 -- 172.21.15.161:0/2941327071 learned_addr learned my addr 172.21.15.161:0/2941327071 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:35.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.274+0000 7ff35219c700 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e80ad0 0x55dd25e7dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff35119a700 1 -- 172.21.15.161:0/2941327071 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e80ad0 msgr2=0x55dd25e7dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff35119a700 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e80ad0 0x55dd25e7dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff35119a700 1 -- 172.21.15.161:0/2941327071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e752d0 msgr2=0x55dd25e80480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff35119a700 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e752d0 0x55dd25e80480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff35119a700 1 -- 172.21.15.161:0/2941327071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c113780 con 0x55dd25e75be0 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff35119a700 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e7d770 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009ce0 tx=0x7ff33c022b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65a2ba03a68415 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff3357fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.275+0000 7ff3357fa700 1 -- 172.21.15.161:0/2941327071 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c01d030 con 0x55dd25e75be0 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.276+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd25d214f0 con 0x55dd25e75be0 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.276+0000 7ff3357fa700 1 -- 172.21.15.161:0/2941327071 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff33c00f660 con 0x55dd25e75be0 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.276+0000 7ff3357fa700 1 -- 172.21.15.161:0/2941327071 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33c004790 con 0x55dd25e75be0 2023-07-14T11:38:35.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.276+0000 7ff3675a5b80 10 client.17469 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:35.283 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.276+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55dd25bdec60 con 0x55dd25e75be0 2023-07-14T11:38:35.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.276+0000 7ff3357fa700 1 -- 172.21.15.161:0/2941327071 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd25d214f0 con 0x55dd25e75be0 2023-07-14T11:38:35.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3357fa700 1 -- 172.21.15.161:0/2941327071 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff33c02c590 con 0x55dd25e75be0 2023-07-14T11:38:35.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3675a5b80 10 client.17469 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:35.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3675a5b80 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:35.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3675a5b80 20 client.17469 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e8ff70 0x55dd25e92380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3675a5b80 4 client.17469 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:38:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd25dc0eb0 con 0x55dd25e8ff70 2023-07-14T11:38:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff35199b700 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e8ff70 0x55dd25e92380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff35199b700 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e8ff70 0x55dd25e92380 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead3ddd5cbc62339 server_cookie=fe3855514ff534f6 in_seq=0 out_seq=0 2023-07-14T11:38:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.277+0000 7ff3357fa700 10 client.17469 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.281+0000 7ff3357fa700 1 -- 172.21.15.161:0/2941327071 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd25dc0eb0 con 0x55dd25e8ff70 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.281+0000 7ff3357fa700 10 client.17469 handle_command_reply: tid=0 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.281+0000 7ff3675a5b80 1 client.17469 shutdown 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 msgr2=0x55dd25e7d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e7d770 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009ce0 tx=0x7ff33c022b20 comp rx=0 tx=0).stop 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 shutdown_connections 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e8ff70 0x55dd25e92380 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e80ad0 0x55dd25e7dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e752d0 0x55dd25e80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/2941327071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e75be0 0x55dd25e7d770 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 >> 172.21.15.161:0/2941327071 conn(0x55dd25e6bda0 msgr2=0x55dd25e71290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:35.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3357fa700 0 client.17469 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:35.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 shutdown_connections 2023-07-14T11:38:35.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.282+0000 7ff3675a5b80 1 -- 172.21.15.161:0/2941327071 wait complete. 2023-07-14T11:38:35.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.283+0000 7ff3675a5b80 20 client.17469 trim_cache size 0 max 16384 2023-07-14T11:38:35.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.310+0000 7ff3675a5b80 1 Processor -- start 2023-07-14T11:38:35.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.310+0000 7ff3675a5b80 1 -- start start 2023-07-14T11:38:35.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.311+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e6bda0 0x55dd25e874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.311+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e80020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.311+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 0x55dd25e7cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.311+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25e832b0 con 0x55dd25e6bda0 2023-07-14T11:38:35.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.311+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25e72b00 con 0x55dd25e75be0 2023-07-14T11:38:35.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.311+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25e76590 con 0x55dd25e752d0 2023-07-14T11:38:35.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.312+0000 7ff35199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e6bda0 0x55dd25e874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.312+0000 7ff35199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e6bda0 0x55dd25e874a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39422/0 (socket says 172.21.15.161:39422) 2023-07-14T11:38:35.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.312+0000 7ff35199b700 1 -- 172.21.15.161:0/3977719169 learned_addr learned my addr 172.21.15.161:0/3977719169 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:35.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.313+0000 7ff35199b700 1 -- 172.21.15.161:0/3977719169 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 msgr2=0x55dd25e80020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.313+0000 7ff35199b700 1 --2- 172.21.15.161:0/3977719169 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e80020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.313+0000 7ff35199b700 1 -- 172.21.15.161:0/3977719169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 msgr2=0x55dd25e7cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.313+0000 7ff35199b700 1 --2- 172.21.15.161:0/3977719169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 0x55dd25e7cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.313+0000 7ff35199b700 1 -- 172.21.15.161:0/3977719169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd25c1c250 con 0x55dd25e6bda0 2023-07-14T11:38:35.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.313+0000 7ff35199b700 1 --2- 172.21.15.161:0/3977719169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e6bda0 0x55dd25e874a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff3480ca6e0 tx=0x7ff348096380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afebceae850a356 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:35.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.315+0000 7ff340bff700 1 -- 172.21.15.161:0/3977719169 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348098140 con 0x55dd25e6bda0 2023-07-14T11:38:35.312 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.315+0000 7ff340bff700 1 -- 172.21.15.161:0/3977719169 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3480b1070 con 0x55dd25e6bda0 2023-07-14T11:38:35.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.315+0000 7ff340bff700 1 -- 172.21.15.161:0/3977719169 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34809a260 con 0x55dd25e6bda0 2023-07-14T11:38:35.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.316+0000 7ff3675a5b80 1 -- 172.21.15.161:0/3977719169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e6bda0 msgr2=0x55dd25e874a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.316+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/3977719169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e6bda0 0x55dd25e874a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff3480ca6e0 tx=0x7ff348096380 comp rx=0 tx=0).stop 2023-07-14T11:38:35.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.317+0000 7ff3675a5b80 1 -- 172.21.15.161:0/3977719169 shutdown_connections 2023-07-14T11:38:35.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.317+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/3977719169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e6bda0 0x55dd25e874a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.317+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/3977719169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 0x55dd25e7cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.317+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/3977719169 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e752d0 0x55dd25e80020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.317+0000 7ff3675a5b80 1 -- 172.21.15.161:0/3977719169 >> 172.21.15.161:0/3977719169 conn(0x55dd25e6d600 msgr2=0x55dd25e78a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:35.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.319+0000 7ff3675a5b80 1 -- 172.21.15.161:0/3977719169 shutdown_connections 2023-07-14T11:38:35.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.321+0000 7ff3675a5b80 1 -- 172.21.15.161:0/3977719169 wait complete. 2023-07-14T11:38:35.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.322+0000 7ff3675a5b80 1 -- start start 2023-07-14T11:38:35.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.322+0000 7ff3675a5b80 1 Processor -- start 2023-07-14T11:38:35.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.323+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e6bda0 0x55dd25e83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.323+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e752d0 0x55dd25e83fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.323+0000 7ff3675a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 0x55dd25e84500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.323+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25e87af0 con 0x55dd25e752d0 2023-07-14T11:38:35.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.323+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25ef4e00 con 0x55dd25e75be0 2023-07-14T11:38:35.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.323+0000 7ff3675a5b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25f05fa0 con 0x55dd25e6bda0 2023-07-14T11:38:35.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.323+0000 7ff3675a5b80 10 client.0 _ll_register_callbacks cb 0x7ff3480963f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:38:35.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e6bda0 0x55dd25e83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e6bda0 0x55dd25e83a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:42082/0 (socket says 172.21.15.161:42082) 2023-07-14T11:38:35.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 -- 172.21.15.161:0/1680178709 learned_addr learned my addr 172.21.15.161:0/1680178709 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:38:35.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35119a700 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e752d0 0x55dd25e83fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 -- 172.21.15.161:0/1680178709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 msgr2=0x55dd25e84500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 0x55dd25e84500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 -- 172.21.15.161:0/1680178709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e752d0 msgr2=0x55dd25e83fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:38:35.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e752d0 0x55dd25e83fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.324+0000 7ff35199b700 1 -- 172.21.15.161:0/1680178709 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c1a9060 con 0x55dd25e6bda0 2023-07-14T11:38:35.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.325+0000 7ff35199b700 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e6bda0 0x55dd25e83a80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff3480cac80 tx=0x7ff348010ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=466528089cbb3ac3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:38:35.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.325+0000 7ff3357fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:38:35.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.326+0000 7ff3357fa700 1 -- 172.21.15.161:0/1680178709 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480b82b0 con 0x55dd25e6bda0 2023-07-14T11:38:35.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.326+0000 7ff3357fa700 1 -- 172.21.15.161:0/1680178709 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7ff348099610 con 0x55dd25e6bda0 2023-07-14T11:38:35.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.326+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff33c01e040 con 0x55dd25e6bda0 2023-07-14T11:38:35.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.327+0000 7ff3357fa700 1 -- 172.21.15.161:0/1680178709 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3480b8410 con 0x55dd25e6bda0 2023-07-14T11:38:35.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.327+0000 7ff3675a5b80 10 client.27532 fetch_fsmap learned FSMap version 41 2023-07-14T11:38:35.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.327+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55dd25bf42b0 con 0x55dd25e6bda0 2023-07-14T11:38:35.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3357fa700 1 -- 172.21.15.161:0/1680178709 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c01e040 con 0x55dd25e6bda0 2023-07-14T11:38:35.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3357fa700 1 -- 172.21.15.161:0/1680178709 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff348095df0 con 0x55dd25e6bda0 2023-07-14T11:38:35.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3675a5b80 10 client.27532 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:38:35.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3675a5b80 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:38:35.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3675a5b80 20 client.27532 populate_metadata read hostname 'smithi161' 2023-07-14T11:38:35.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e80ad0 0x55dd25f09360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:38:35.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3675a5b80 4 client.27532 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:38:35.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.328+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd25d83050 con 0x55dd25e80ad0 2023-07-14T11:38:35.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.332+0000 7ff35119a700 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e80ad0 0x55dd25f09360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:38:35.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.333+0000 7ff3357fa700 10 client.27532 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:35.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.333+0000 7ff35119a700 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e80ad0 0x55dd25f09360 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49fbcd0d3cdfbd2 server_cookie=96b43287c5f49971 in_seq=0 out_seq=0 2023-07-14T11:38:35.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.341+0000 7ff3357fa700 1 -- 172.21.15.161:0/1680178709 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55dd25d83050 con 0x55dd25e80ad0 2023-07-14T11:38:35.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.341+0000 7ff3357fa700 10 client.27532 handle_command_reply: tid=0 2023-07-14T11:38:35.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.341+0000 7ff3675a5b80 1 client.27532 shutdown 2023-07-14T11:38:35.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.342+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e6bda0 msgr2=0x55dd25e83a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.342+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e6bda0 0x55dd25e83a80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff3480cac80 tx=0x7ff348010ac0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.343+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 shutdown_connections 2023-07-14T11:38:35.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.343+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55dd25e80ad0 0x55dd25f09360 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.343+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dd25e75be0 0x55dd25e84500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.343+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55dd25e6bda0 0x55dd25e83a80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.343+0000 7ff3675a5b80 1 --2- 172.21.15.161:0/1680178709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd25e752d0 0x55dd25e83fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.343+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 >> 172.21.15.161:0/1680178709 conn(0x55dd25e6d600 msgr2=0x55dd25e771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:35.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.343+0000 7ff3357fa700 0 client.27532 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:38:35.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.345+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 shutdown_connections 2023-07-14T11:38:35.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.345+0000 7ff3675a5b80 1 -- 172.21.15.161:0/1680178709 wait complete. 2023-07-14T11:38:35.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.347+0000 7ff3675a5b80 20 client.27532 trim_cache size 0 max 16384 2023-07-14T11:38:35.346 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:38:35.346 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (164 inodes in the stack)", 2023-07-14T11:38:35.346 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:38:35.346 INFO:teuthology.orchestra.run.smithi161.stdout: "eb3d05be-6184-4b4f-8076-e33e9a553189": { 2023-07-14T11:38:35.347 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:38:35.347 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "eb3d05be-6184-4b4f-8076-e33e9a553189", 2023-07-14T11:38:35.347 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:38:35.347 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:38:35.347 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:38:35.347 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:38:35.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.349+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff3380a12c0 msgr2=0x7ff3380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.349+0000 7ff353bff700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff3380a12c0 0x7ff3380a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff344009ec0 tx=0x7ff344004b40 comp rx=0 tx=0).stop 2023-07-14T11:38:35.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.349+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0728b0 msgr2=0x7ff34c19c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:38:35.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.349+0000 7ff353bff700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0728b0 0x7ff34c19c230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff34800e9a0 tx=0x7ff34800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 shutdown_connections 2023-07-14T11:38:35.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff3380a12c0 0x7ff3380a3770 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff34c110ae0 0x7ff34c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff34c0728b0 0x7ff34c19c230 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 --2- 172.21.15.161:0/2925664009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff34c0731c0 0x7ff34c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:38:35.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 >> 172.21.15.161:0/2925664009 conn(0x7ff34c0fb5f0 msgr2=0x7ff34c0fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:38:35.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 shutdown_connections 2023-07-14T11:38:35.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:38:35.350+0000 7ff353bff700 1 -- 172.21.15.161:0/2925664009 wait complete. 2023-07-14T11:38:35.376 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:35 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:38:35.377 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:35 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:35.377 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:35 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:35.378 INFO:tasks.cephfs.filesystem:scrub status for tag:eb3d05be-6184-4b4f-8076-e33e9a553189 - {'path': '/', 'tag': 'eb3d05be-6184-4b4f-8076-e33e9a553189', 'options': 'recursive,force'} 2023-07-14T11:38:35.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:35 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:38:35.416 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:35 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:35.416 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:35 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:35.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:35 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:38:35.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:35 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:35.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:35 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:35.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 115281 30.57 MB/sec execute 77 sec latency 80.153 ms 2023-07-14T11:38:36.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:36.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:36.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:36.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:38:36.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.5 MiB/s wr, 461 op/s 2023-07-14T11:38:36.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:36 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:36.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:36.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:36.459 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:38:36.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.5 MiB/s wr, 461 op/s 2023-07-14T11:38:36.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:36 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:36.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.5 MiB/s wr, 461 op/s 2023-07-14T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:36 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:38:36.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 115823 30.48 MB/sec execute 78 sec latency 86.478 ms 2023-07-14T11:38:37.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 116339 30.33 MB/sec execute 79 sec latency 125.134 ms 2023-07-14T11:38:38.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 117340 30.61 MB/sec execute 80 sec latency 58.637 ms 2023-07-14T11:38:38.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:38 smithi175 ceph-mon[129081]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 650 op/s 2023-07-14T11:38:38.708 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:38 smithi182 ceph-mon[129366]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 650 op/s 2023-07-14T11:38:38.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:38 smithi161 ceph-mon[95526]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 650 op/s 2023-07-14T11:38:39.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 118146 30.89 MB/sec execute 81 sec latency 71.473 ms 2023-07-14T11:38:40.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 118682 30.84 MB/sec execute 82 sec latency 84.209 ms 2023-07-14T11:38:40.708 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:40 smithi182 ceph-mon[129366]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.3 MiB/s wr, 380 op/s 2023-07-14T11:38:40.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:40 smithi161 ceph-mon[95526]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.3 MiB/s wr, 380 op/s 2023-07-14T11:38:40.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:40 smithi175 ceph-mon[129081]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.3 MiB/s wr, 380 op/s 2023-07-14T11:38:41.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 118837 30.56 MB/sec execute 83 sec latency 630.994 ms 2023-07-14T11:38:41.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:41 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:38:41.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:41 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:38:41.990 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:41 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:38:42.612 INFO:tasks.workunit.client.0.smithi161.stdout: 10 119071 30.42 MB/sec execute 84 sec latency 620.226 ms 2023-07-14T11:38:42.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:42 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:38:42.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:42 smithi175 ceph-mon[129081]: scrub summary: idle 2023-07-14T11:38:42.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:42 smithi175 ceph-mon[129081]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.6 MiB/s wr, 448 op/s 2023-07-14T11:38:42.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:42 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:38:42.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:42 smithi182 ceph-mon[129366]: scrub summary: idle 2023-07-14T11:38:42.959 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:42 smithi182 ceph-mon[129366]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.6 MiB/s wr, 448 op/s 2023-07-14T11:38:42.990 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:42 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:38:42.990 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:42 smithi161 ceph-mon[95526]: scrub summary: idle 2023-07-14T11:38:42.990 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:42 smithi161 ceph-mon[95526]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.6 MiB/s wr, 448 op/s 2023-07-14T11:38:43.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 119408 30.25 MB/sec execute 85 sec latency 194.708 ms 2023-07-14T11:38:44.413 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:44 smithi175 ceph-mon[129081]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 517 op/s 2023-07-14T11:38:44.458 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:44 smithi182 ceph-mon[129366]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 517 op/s 2023-07-14T11:38:44.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:44 smithi161 ceph-mon[95526]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 517 op/s 2023-07-14T11:38:44.611 INFO:tasks.workunit.client.0.smithi161.stdout: 10 120229 30.35 MB/sec execute 86 sec latency 66.013 ms 2023-07-14T11:38:45.612 INFO:tasks.workunit.client.0.smithi161.stdout: 10 120988 30.54 MB/sec execute 87 sec latency 74.961 ms 2023-07-14T11:38:46.612 INFO:tasks.workunit.client.0.smithi161.stdout: 10 121505 30.52 MB/sec execute 88 sec latency 175.564 ms 2023-07-14T11:38:46.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:46 smithi182 ceph-mon[129366]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 445 op/s 2023-07-14T11:38:46.748 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:46 smithi161 ceph-mon[95526]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 445 op/s 2023-07-14T11:38:46.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:46 smithi175 ceph-mon[129081]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 445 op/s 2023-07-14T11:38:47.613 INFO:tasks.workunit.client.0.smithi161.stdout: 10 121522 30.19 MB/sec execute 89 sec latency 1118.812 ms 2023-07-14T11:38:48.613 INFO:tasks.workunit.client.0.smithi161.stdout: 10 121668 29.92 MB/sec execute 90 sec latency 1331.764 ms 2023-07-14T11:38:48.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:48 smithi182 ceph-mon[129366]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 662 op/s 2023-07-14T11:38:48.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:48 smithi161 ceph-mon[95526]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 662 op/s 2023-07-14T11:38:48.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:48 smithi175 ceph-mon[129081]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 662 op/s 2023-07-14T11:38:49.613 INFO:tasks.workunit.client.0.smithi161.stdout: 10 121937 29.72 MB/sec execute 91 sec latency 197.787 ms 2023-07-14T11:38:50.613 INFO:tasks.workunit.client.0.smithi161.stdout: 10 122414 29.63 MB/sec execute 92 sec latency 67.899 ms 2023-07-14T11:38:50.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:50 smithi161 ceph-mon[95526]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 523 op/s 2023-07-14T11:38:50.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:50 smithi175 ceph-mon[129081]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 523 op/s 2023-07-14T11:38:50.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:50 smithi182 ceph-mon[129366]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 523 op/s 2023-07-14T11:38:51.613 INFO:tasks.workunit.client.0.smithi161.stdout: 10 123275 29.87 MB/sec execute 93 sec latency 66.332 ms 2023-07-14T11:38:52.613 INFO:tasks.workunit.client.0.smithi161.stdout: 10 123991 29.96 MB/sec execute 94 sec latency 76.362 ms 2023-07-14T11:38:52.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:52 smithi161 ceph-mon[95526]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 601 op/s 2023-07-14T11:38:52.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:52 smithi175 ceph-mon[129081]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 601 op/s 2023-07-14T11:38:52.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:52 smithi182 ceph-mon[129366]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 601 op/s 2023-07-14T11:38:53.617 INFO:tasks.workunit.client.0.smithi161.stdout: 10 124495 29.90 MB/sec execute 95 sec latency 85.722 ms 2023-07-14T11:38:53.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:53 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:38:53.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:53 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:38:53.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:53 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:38:54.617 INFO:tasks.workunit.client.0.smithi161.stdout: 10 125103 29.95 MB/sec execute 96 sec latency 108.099 ms 2023-07-14T11:38:54.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:54 smithi161 ceph-mon[95526]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 17 MiB/s wr, 749 op/s 2023-07-14T11:38:54.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:54 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:38:54.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:54 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:38:54.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:54 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:38:54.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:54 smithi175 ceph-mon[129081]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 17 MiB/s wr, 749 op/s 2023-07-14T11:38:54.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:54 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:38:54.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:54 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:38:54.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:54 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:38:54.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:54 smithi182 ceph-mon[129366]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 17 MiB/s wr, 749 op/s 2023-07-14T11:38:54.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:54 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:38:54.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:54 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:38:54.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:54 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:38:55.619 INFO:tasks.workunit.client.0.smithi161.stdout: 10 125805 30.03 MB/sec execute 97 sec latency 90.919 ms 2023-07-14T11:38:56.617 INFO:tasks.workunit.client.0.smithi161.stdout: 10 126486 30.11 MB/sec execute 98 sec latency 124.230 ms 2023-07-14T11:38:56.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:56 smithi182 ceph-mon[129366]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 560 op/s 2023-07-14T11:38:56.987 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:56 smithi175 ceph-mon[129081]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 560 op/s 2023-07-14T11:38:56.989 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:56 smithi161 ceph-mon[95526]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 560 op/s 2023-07-14T11:38:57.617 INFO:tasks.workunit.client.0.smithi161.stdout: 10 127246 30.17 MB/sec execute 99 sec latency 77.802 ms 2023-07-14T11:38:58.617 INFO:tasks.workunit.client.0.smithi161.stdout: 10 127979 30.34 MB/sec execute 100 sec latency 123.917 ms 2023-07-14T11:38:58.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:38:58 smithi175 ceph-mon[129081]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 687 op/s 2023-07-14T11:38:58.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:38:58 smithi182 ceph-mon[129366]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 687 op/s 2023-07-14T11:38:58.990 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:38:58 smithi161 ceph-mon[95526]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 687 op/s 2023-07-14T11:38:59.621 INFO:tasks.workunit.client.0.smithi161.stdout: 10 128621 30.38 MB/sec execute 101 sec latency 80.339 ms 2023-07-14T11:39:00.621 INFO:tasks.workunit.client.0.smithi161.stdout: 10 129212 30.36 MB/sec execute 102 sec latency 159.406 ms 2023-07-14T11:39:00.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:00 smithi175 ceph-mon[129081]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 522 op/s 2023-07-14T11:39:00.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:00 smithi182 ceph-mon[129366]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 522 op/s 2023-07-14T11:39:00.989 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:00 smithi161 ceph-mon[95526]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 522 op/s 2023-07-14T11:39:01.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 129594 30.25 MB/sec execute 103 sec latency 359.337 ms 2023-07-14T11:39:02.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 129820 30.01 MB/sec execute 104 sec latency 370.670 ms 2023-07-14T11:39:02.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:02 smithi175 ceph-mon[129081]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 575 op/s 2023-07-14T11:39:02.960 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:02 smithi182 ceph-mon[129366]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 575 op/s 2023-07-14T11:39:02.989 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:02 smithi161 ceph-mon[95526]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 575 op/s 2023-07-14T11:39:03.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 129852 29.75 MB/sec execute 105 sec latency 350.475 ms 2023-07-14T11:39:04.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:04 smithi175 ceph-mon[129081]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 599 op/s 2023-07-14T11:39:04.457 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:04 smithi182 ceph-mon[129366]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 599 op/s 2023-07-14T11:39:04.489 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:04 smithi161 ceph-mon[95526]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 599 op/s 2023-07-14T11:39:04.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 129858 29.49 MB/sec execute 106 sec latency 338.104 ms 2023-07-14T11:39:05.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:39:05.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 129923 29.27 MB/sec execute 107 sec latency 443.708 ms 2023-07-14T11:39:05.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.656+0000 7fbfa719d700 1 -- 172.21.15.161:0/4273035725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa00728b0 msgr2=0x7fbfa0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.656+0000 7fbfa719d700 1 --2- 172.21.15.161:0/4273035725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa00728b0 0x7fbfa0072c80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009990 tx=0x7fbf9c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.657+0000 7fbfa719d700 1 -- 172.21.15.161:0/4273035725 shutdown_connections 2023-07-14T11:39:05.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.657+0000 7fbfa719d700 1 --2- 172.21.15.161:0/4273035725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa00728b0 0x7fbfa0072c80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.657+0000 7fbfa719d700 1 --2- 172.21.15.161:0/4273035725 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa01109e0 0x7fbfa0112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.657+0000 7fbfa719d700 1 --2- 172.21.15.161:0/4273035725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa00731c0 0x7fbfa01104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.657+0000 7fbfa719d700 1 -- 172.21.15.161:0/4273035725 >> 172.21.15.161:0/4273035725 conn(0x7fbfa00fb4b0 msgr2=0x7fbfa00fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:05.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.657+0000 7fbfa719d700 1 -- 172.21.15.161:0/4273035725 shutdown_connections 2023-07-14T11:39:05.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.657+0000 7fbfa719d700 1 -- 172.21.15.161:0/4273035725 wait complete. 2023-07-14T11:39:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.658+0000 7fbfa719d700 1 Processor -- start 2023-07-14T11:39:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.658+0000 7fbfa719d700 1 -- start start 2023-07-14T11:39:05.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.659+0000 7fbfa719d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa00728b0 0x7fbfa019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.659+0000 7fbfa719d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa00731c0 0x7fbfa019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.659+0000 7fbfa719d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa01109e0 0x7fbfa01a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.659+0000 7fbfa719d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01a1030 con 0x7fbfa01109e0 2023-07-14T11:39:05.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.659+0000 7fbfa719d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01a11a0 con 0x7fbfa00728b0 2023-07-14T11:39:05.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.659+0000 7fbfa719d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01a1360 con 0x7fbfa00731c0 2023-07-14T11:39:05.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.659+0000 7fbfa599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa00731c0 0x7fbfa019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa00728b0 0x7fbfa019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa00731c0 0x7fbfa019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45874/0 (socket says 172.21.15.161:45874) 2023-07-14T11:39:05.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa599a700 1 -- 172.21.15.161:0/383430600 learned_addr learned my addr 172.21.15.161:0/383430600 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:05.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa00728b0 0x7fbfa019c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37538/0 (socket says 172.21.15.161:37538) 2023-07-14T11:39:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa699c700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa01109e0 0x7fbfa01a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa599a700 1 -- 172.21.15.161:0/383430600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa00728b0 msgr2=0x7fbfa019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa599a700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa00728b0 0x7fbfa019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa599a700 1 -- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa01109e0 msgr2=0x7fbfa01a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa599a700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa01109e0 0x7fbfa01a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa599a700 1 -- 172.21.15.161:0/383430600 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf98009c40 con 0x7fbfa00731c0 2023-07-14T11:39:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.660+0000 7fbfa699c700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa01109e0 0x7fbfa01a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:39:05.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.661+0000 7fbfa599a700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa00731c0 0x7fbfa019c590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b530 tx=0x7fbf9400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.661+0000 7fbf937fe700 1 -- 172.21.15.161:0/383430600 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94011830 con 0x7fbfa00731c0 2023-07-14T11:39:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.661+0000 7fbf937fe700 1 -- 172.21.15.161:0/383430600 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf94011e50 con 0x7fbfa00731c0 2023-07-14T11:39:05.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.661+0000 7fbf937fe700 1 -- 172.21.15.161:0/383430600 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400f2e0 con 0x7fbfa00731c0 2023-07-14T11:39:05.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.661+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c0095f0 con 0x7fbfa00731c0 2023-07-14T11:39:05.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.661+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa01a8ca0 con 0x7fbfa00731c0 2023-07-14T11:39:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.666+0000 7fbf937fe700 1 -- 172.21.15.161:0/383430600 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fbf94010380 con 0x7fbfa00731c0 2023-07-14T11:39:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.667+0000 7fbfbbc79b80 1 Processor -- start 2023-07-14T11:39:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.668+0000 7fbfbbc79b80 1 -- start start 2023-07-14T11:39:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.668+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.668+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.668+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.668+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9ce55cb0 con 0x55ab9cf50e00 2023-07-14T11:39:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.668+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cea8a40 con 0x55ab9cf504f0 2023-07-14T11:39:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.668+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cea1ed0 con 0x55ab9cf5ad30 2023-07-14T11:39:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44284/0 (socket says 172.21.15.161:44284) 2023-07-14T11:39:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 -- 172.21.15.161:0/1854254413 learned_addr learned my addr 172.21.15.161:0/1854254413 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa699c700 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 -- 172.21.15.161:0/1854254413 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 msgr2=0x55ab9cf5d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 -- 172.21.15.161:0/1854254413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 msgr2=0x55ab9cf508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbfa619b700 1 -- 172.21.15.161:0/1854254413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab9cccfde0 con 0x55ab9cf50e00 2023-07-14T11:39:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.669+0000 7fbf937fe700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbf8c0a1360 0x7fbf8c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.670+0000 7fbf937fe700 1 -- 172.21.15.161:0/383430600 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fbf940ccb30 con 0x7fbfa00731c0 2023-07-14T11:39:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.670+0000 7fbfa619b700 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5a7f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00b6c0 tx=0x7fbf9c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934b351c8fed525e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.670+0000 7fbf90bfe700 1 -- 172.21.15.161:0/1854254413 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c01d070 con 0x55ab9cf50e00 2023-07-14T11:39:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.670+0000 7fbfa699c700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbf8c0a1360 0x7fbf8c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.670+0000 7fbf90bfe700 1 -- 172.21.15.161:0/1854254413 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c01e040 con 0x55ab9cf50e00 2023-07-14T11:39:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.670+0000 7fbf90bfe700 1 -- 172.21.15.161:0/1854254413 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c004a90 con 0x55ab9cf50e00 2023-07-14T11:39:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/1854254413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 msgr2=0x55ab9cf5a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5a7f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00b6c0 tx=0x7fbf9c005bc0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfa699c700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbf8c0a1360 0x7fbf8c0a3810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0069910 tx=0x7fbf98009040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/1854254413 shutdown_connections 2023-07-14T11:39:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/1854254413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5a7f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.671+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/1854254413 >> 172.21.15.161:0/1854254413 conn(0x55ab9cf45ff0 msgr2=0x55ab9cf4b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.672+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/1854254413 shutdown_connections 2023-07-14T11:39:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.672+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/1854254413 wait complete. 2023-07-14T11:39:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.672+0000 7fbfbbc79b80 1 -- start start 2023-07-14T11:39:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 1 Processor -- start 2023-07-14T11:39:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf57960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf57f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9ce41ac0 con 0x55ab9cf50e00 2023-07-14T11:39:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9ce417c0 con 0x55ab9cf5ad30 2023-07-14T11:39:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cd25920 con 0x55ab9cf504f0 2023-07-14T11:39:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.673+0000 7fbfbbc79b80 10 client.0 _ll_register_callbacks cb 0x55ab9ce3f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:05.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45886/0 (socket says 172.21.15.161:45886) 2023-07-14T11:39:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 -- 172.21.15.161:0/3934936035 learned_addr learned my addr 172.21.15.161:0/3934936035 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa599a700 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf57960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa699c700 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf57f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 -- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf5ad30 msgr2=0x55ab9cf57f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf57f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 -- 172.21.15.161:0/3934936035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 msgr2=0x55ab9cf57960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf57960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.674+0000 7fbfa619b700 1 -- 172.21.15.161:0/3934936035 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa0113680 con 0x55ab9cf504f0 2023-07-14T11:39:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.675+0000 7fbfa619b700 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5a750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c005850 tx=0x7fbf9c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cff17e62af89dff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.675+0000 7fbf89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.675+0000 7fbf89ffb700 1 -- 172.21.15.161:0/3934936035 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c01d040 con 0x55ab9cf504f0 2023-07-14T11:39:05.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.675+0000 7fbf89ffb700 1 -- 172.21.15.161:0/3934936035 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fbf9c00fd40 con 0x55ab9cf504f0 2023-07-14T11:39:05.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab9cdfb4f0 con 0x55ab9cf504f0 2023-07-14T11:39:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbf89ffb700 1 -- 172.21.15.161:0/3934936035 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9c036400 con 0x55ab9cf504f0 2023-07-14T11:39:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbfbbc79b80 10 client.27544 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ab9ccb8c60 con 0x55ab9cf504f0 2023-07-14T11:39:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbf89ffb700 1 -- 172.21.15.161:0/3934936035 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c03a030 con 0x55ab9cf504f0 2023-07-14T11:39:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbf89ffb700 1 -- 172.21.15.161:0/3934936035 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbf9c0173f0 con 0x55ab9cf504f0 2023-07-14T11:39:05.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbfbbc79b80 10 client.27544 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:05.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.676+0000 7fbfbbc79b80 10 client.27544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:05.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.677+0000 7fbfbbc79b80 20 client.27544 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:05.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.677+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cf6a200 0x55ab9cf6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.677+0000 7fbfbbc79b80 4 client.27544 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:39:05.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.677+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab9ce9aeb0 con 0x55ab9cf6a200 2023-07-14T11:39:05.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.678+0000 7fbfa599a700 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cf6a200 0x55ab9cf6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.681+0000 7fbfa599a700 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cf6a200 0x55ab9cf6c610 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b4707499b440b8f server_cookie=22a1273a79a8ed3b in_seq=0 out_seq=0 2023-07-14T11:39:05.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.681+0000 7fbf89ffb700 10 client.27544 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:05.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.688+0000 7fbf89ffb700 1 -- 172.21.15.161:0/3934936035 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab9ce9aeb0 con 0x55ab9cf6a200 2023-07-14T11:39:05.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.688+0000 7fbf89ffb700 10 client.27544 handle_command_reply: tid=0 2023-07-14T11:39:05.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.688+0000 7fbfbbc79b80 1 client.27544 shutdown 2023-07-14T11:39:05.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf504f0 msgr2=0x55ab9cf5a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5a750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c005850 tx=0x7fbf9c00f690 comp rx=0 tx=0).stop 2023-07-14T11:39:05.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 shutdown_connections 2023-07-14T11:39:05.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cf6a200 0x55ab9cf6c610 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf57f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5a750 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/3934936035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf57960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 >> 172.21.15.161:0/3934936035 conn(0x55ab9cf45ff0 msgr2=0x55ab9cf4b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:05.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbf89ffb700 0 client.27544 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:05.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 shutdown_connections 2023-07-14T11:39:05.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/3934936035 wait complete. 2023-07-14T11:39:05.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.689+0000 7fbfbbc79b80 20 client.27544 trim_cache size 0 max 16384 2023-07-14T11:39:05.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.712+0000 7fbfbbc79b80 1 Processor -- start 2023-07-14T11:39:05.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.712+0000 7fbfbbc79b80 1 -- start start 2023-07-14T11:39:05.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf50e00 0x55ab9cf6c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cf521a0 con 0x55ab9cf5ad30 2023-07-14T11:39:05.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cf66660 con 0x55ab9cf504f0 2023-07-14T11:39:05.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cf55ec0 con 0x55ab9cf50e00 2023-07-14T11:39:05.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfa599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf50e00 0x55ab9cf6c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfa599a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf50e00 0x55ab9cf6c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45894/0 (socket says 172.21.15.161:45894) 2023-07-14T11:39:05.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfa699c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfa599a700 1 -- 172.21.15.161:0/2644331912 learned_addr learned my addr 172.21.15.161:0/2644331912 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:05.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.713+0000 7fbfa619b700 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.714+0000 7fbfa619b700 1 -- 172.21.15.161:0/2644331912 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf50e00 msgr2=0x55ab9cf6c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.714+0000 7fbfa619b700 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf50e00 0x55ab9cf6c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.714+0000 7fbfa619b700 1 -- 172.21.15.161:0/2644331912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf5ad30 msgr2=0x55ab9cf5a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.714+0000 7fbfa619b700 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.714+0000 7fbfa619b700 1 -- 172.21.15.161:0/2644331912 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab9ccf6250 con 0x55ab9cf504f0 2023-07-14T11:39:05.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.714+0000 7fbfa599a700 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf50e00 0x55ab9cf6c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:05.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.714+0000 7fbfa619b700 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf617b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c017850 tx=0x7fbf9c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106de14cfa0dd9c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:05.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.715+0000 7fbf913ff700 1 -- 172.21.15.161:0/2644331912 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c009290 con 0x55ab9cf504f0 2023-07-14T11:39:05.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.715+0000 7fbf913ff700 1 -- 172.21.15.161:0/2644331912 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c03a040 con 0x55ab9cf504f0 2023-07-14T11:39:05.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.715+0000 7fbf913ff700 1 -- 172.21.15.161:0/2644331912 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c03f980 con 0x55ab9cf504f0 2023-07-14T11:39:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.716+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2644331912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 msgr2=0x55ab9cf617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.716+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf617b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c017850 tx=0x7fbf9c000bc0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.716+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2644331912 shutdown_connections 2023-07-14T11:39:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.716+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf5a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.716+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf617b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.716+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2644331912 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf50e00 0x55ab9cf6c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.716+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2644331912 >> 172.21.15.161:0/2644331912 conn(0x55ab9cf45ff0 msgr2=0x55ab9cf65f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:05.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.717+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2644331912 shutdown_connections 2023-07-14T11:39:05.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.717+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2644331912 wait complete. 2023-07-14T11:39:05.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.717+0000 7fbfbbc79b80 1 -- start start 2023-07-14T11:39:05.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.718+0000 7fbfbbc79b80 1 Processor -- start 2023-07-14T11:39:05.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfbbc79b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf60210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cfbd120 con 0x55ab9cf50e00 2023-07-14T11:39:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cfbd290 con 0x55ab9cf504f0 2023-07-14T11:39:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfbbc79b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9cfbd450 con 0x55ab9cf5ad30 2023-07-14T11:39:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfbbc79b80 10 client.0 _ll_register_callbacks cb 0x7fbf940d0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa599a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa619b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37580/0 (socket says 172.21.15.161:37580) 2023-07-14T11:39:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa619b700 1 -- 172.21.15.161:0/2293793856 learned_addr learned my addr 172.21.15.161:0/2293793856 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa699c700 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf60210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa619b700 1 -- 172.21.15.161:0/2293793856 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 msgr2=0x55ab9cf60210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa619b700 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf60210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa619b700 1 -- 172.21.15.161:0/2293793856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 msgr2=0x55ab9cf5fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:05.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.719+0000 7fbfa619b700 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.720+0000 7fbfa619b700 1 -- 172.21.15.161:0/2293793856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf940d4040 con 0x55ab9cf504f0 2023-07-14T11:39:05.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.720+0000 7fbfa619b700 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5f790 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c005820 tx=0x7fbf9c017620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1dee0adb792ef2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.720+0000 7fbf89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:39:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.720+0000 7fbf89ffb700 1 -- 172.21.15.161:0/2293793856 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c01d030 con 0x55ab9cf504f0 2023-07-14T11:39:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.720+0000 7fbf89ffb700 1 -- 172.21.15.161:0/2293793856 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fbf9c000e40 con 0x55ab9cf504f0 2023-07-14T11:39:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.721+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf9c01e040 con 0x55ab9cf504f0 2023-07-14T11:39:05.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.721+0000 7fbf89ffb700 1 -- 172.21.15.161:0/2293793856 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9c009290 con 0x55ab9cf504f0 2023-07-14T11:39:05.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.721+0000 7fbfbbc79b80 10 client.27506 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:05.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.721+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ab9ccb8c60 con 0x55ab9cf504f0 2023-07-14T11:39:05.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.721+0000 7fbf89ffb700 1 -- 172.21.15.161:0/2293793856 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c01e040 con 0x55ab9cf504f0 2023-07-14T11:39:05.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.722+0000 7fbf89ffb700 1 -- 172.21.15.161:0/2293793856 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbf9c004cb0 con 0x55ab9cf504f0 2023-07-14T11:39:05.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.722+0000 7fbfbbc79b80 10 client.27506 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.722+0000 7fbfbbc79b80 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.722+0000 7fbfbbc79b80 20 client.27506 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.722+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cfbe710 0x55ab9cfc0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.722+0000 7fbfbbc79b80 4 client.27506 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:39:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.722+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab9ce5d050 con 0x55ab9cfbe710 2023-07-14T11:39:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.723+0000 7fbfa599a700 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cfbe710 0x55ab9cfc0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.726+0000 7fbf89ffb700 10 client.27506 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.726+0000 7fbfa599a700 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cfbe710 0x55ab9cfc0b20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abb02cb2c7db0de server_cookie=96a94a00add04d60 in_seq=0 out_seq=0 2023-07-14T11:39:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.726+0000 7fbf89ffb700 1 -- 172.21.15.161:0/2293793856 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ab9ce5d050 con 0x55ab9cfbe710 2023-07-14T11:39:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.726+0000 7fbf89ffb700 10 client.27506 handle_command_reply: tid=0 2023-07-14T11:39:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.726+0000 7fbfbbc79b80 1 client.27506 shutdown 2023-07-14T11:39:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 msgr2=0x55ab9cf5f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5f790 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c005820 tx=0x7fbf9c017620 comp rx=0 tx=0).stop 2023-07-14T11:39:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 shutdown_connections 2023-07-14T11:39:05.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab9cfbe710 0x55ab9cfc0b20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab9cf5ad30 0x55ab9cf60210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab9cf504f0 0x55ab9cf5f790 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 --2- 172.21.15.161:0/2293793856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab9cf50e00 0x55ab9cf5fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 >> 172.21.15.161:0/2293793856 conn(0x55ab9cf45ff0 msgr2=0x55ab9cf6c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:05.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbf89ffb700 0 client.27506 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:05.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 shutdown_connections 2023-07-14T11:39:05.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 1 -- 172.21.15.161:0/2293793856 wait complete. 2023-07-14T11:39:05.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.727+0000 7fbfbbc79b80 20 client.27506 trim_cache size 0 max 16384 2023-07-14T11:39:05.730 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:39:05.731 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "no active scrubs running", 2023-07-14T11:39:05.731 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": {} 2023-07-14T11:39:05.732 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:39:05.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbf8c0a1360 msgr2=0x7fbf8c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbf8c0a1360 0x7fbf8c0a3810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0069910 tx=0x7fbf98009040 comp rx=0 tx=0).stop 2023-07-14T11:39:05.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa00731c0 msgr2=0x7fbfa019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:05.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa00731c0 0x7fbfa019c590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b530 tx=0x7fbf9400b840 comp rx=0 tx=0).stop 2023-07-14T11:39:05.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 shutdown_connections 2023-07-14T11:39:05.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fbf8c0a1360 0x7fbf8c0a3810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa01109e0 0x7fbfa01a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfa00728b0 0x7fbfa019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 --2- 172.21.15.161:0/383430600 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fbfa00731c0 0x7fbfa019c590 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:05.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 >> 172.21.15.161:0/383430600 conn(0x7fbfa00fb4b0 msgr2=0x7fbfa00fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:05.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 shutdown_connections 2023-07-14T11:39:05.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:05.728+0000 7fbfa719d700 1 -- 172.21.15.161:0/383430600 wait complete. 2023-07-14T11:39:05.747 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T11:39:05.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:39:05.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/1110290567 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b98105b00 msgr2=0x7f6b98105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/1110290567 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b98105b00 0x7f6b98105ed0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009990 tx=0x7f6b88009ca0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/1110290567 shutdown_connections 2023-07-14T11:39:06.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/1110290567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b981004a0 0x7f6b9810c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/1110290567 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b98105b00 0x7f6b98105ed0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/1110290567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b980ffaf0 0x7f6b980fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/1110290567 >> 172.21.15.161:0/1110290567 conn(0x7f6b980fb630 msgr2=0x7f6b980fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/1110290567 shutdown_connections 2023-07-14T11:39:06.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.003+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/1110290567 wait complete. 2023-07-14T11:39:06.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 Processor -- start 2023-07-14T11:39:06.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 -- start start 2023-07-14T11:39:06.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980ffaf0 0x7f6b98197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b981004a0 0x7f6b98198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b98105b00 0x7f6b9819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9819ce30 con 0x7f6b98105b00 2023-07-14T11:39:06.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9819cfa0 con 0x7f6b980ffaf0 2023-07-14T11:39:06.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.005+0000 7f6b9f1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9819d160 con 0x7f6b981004a0 2023-07-14T11:39:06.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b98105b00 0x7f6b9819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b98105b00 0x7f6b9819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44320/0 (socket says 172.21.15.161:44320) 2023-07-14T11:39:06.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 -- 172.21.15.161:0/63708577 learned_addr learned my addr 172.21.15.161:0/63708577 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:06.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b97fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b981004a0 0x7f6b98198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 -- 172.21.15.161:0/63708577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b981004a0 msgr2=0x7f6b98198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b981004a0 0x7f6b98198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 -- 172.21.15.161:0/63708577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980ffaf0 msgr2=0x7f6b98197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980ffaf0 0x7f6b98197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.006+0000 7f6b9d79c700 1 -- 172.21.15.161:0/63708577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880095f0 con 0x7f6b98105b00 2023-07-14T11:39:06.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.007+0000 7f6b9d79c700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b98105b00 0x7f6b9819c7f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007ee0 tx=0x7f6b9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.007+0000 7f6b95ffb700 1 -- 172.21.15.161:0/63708577 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90017070 con 0x7f6b98105b00 2023-07-14T11:39:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.007+0000 7f6b95ffb700 1 -- 172.21.15.161:0/63708577 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9000f040 con 0x7f6b98105b00 2023-07-14T11:39:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.007+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9819d440 con 0x7f6b98105b00 2023-07-14T11:39:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.007+0000 7f6b95ffb700 1 -- 172.21.15.161:0/63708577 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90005740 con 0x7f6b98105b00 2023-07-14T11:39:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.007+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b981a4c80 con 0x7f6b98105b00 2023-07-14T11:39:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.009+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b98067300 con 0x7f6b98105b00 2023-07-14T11:39:06.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.011+0000 7f6b95ffb700 1 -- 172.21.15.161:0/63708577 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6b9000dc60 con 0x7f6b98105b00 2023-07-14T11:39:06.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.012+0000 7f6b95ffb700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6b800a1380 0x7f6b800a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.013+0000 7f6b95ffb700 1 -- 172.21.15.161:0/63708577 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6b900ccda0 con 0x7f6b98105b00 2023-07-14T11:39:06.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.013+0000 7f6b9cf9b700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6b800a1380 0x7f6b800a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.014+0000 7f6b9cf9b700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6b800a1380 0x7f6b800a3830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b880095c0 tx=0x7f6b88009510 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.015+0000 7f6b95ffb700 1 -- 172.21.15.161:0/63708577 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6b90095df0 con 0x7f6b98105b00 2023-07-14T11:39:06.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.390+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b981991a0 con 0x7f6b98105b00 2023-07-14T11:39:06.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.391+0000 7f6b95ffb700 1 -- 172.21.15.161:0/63708577 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f6b90099410 con 0x7f6b98105b00 2023-07-14T11:39:06.387 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:39:06.387 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:38:03.446499+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:39:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6b800a1380 msgr2=0x7f6b800a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6b800a1380 0x7f6b800a3830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b880095c0 tx=0x7f6b88009510 comp rx=0 tx=0).stop 2023-07-14T11:39:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b98105b00 msgr2=0x7f6b9819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b98105b00 0x7f6b9819c7f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007ee0 tx=0x7f6b9000d3b0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 shutdown_connections 2023-07-14T11:39:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6b800a1380 0x7f6b800a3830 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b98105b00 0x7f6b9819c7f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980ffaf0 0x7f6b98197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 --2- 172.21.15.161:0/63708577 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6b981004a0 0x7f6b98198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.395+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 >> 172.21.15.161:0/63708577 conn(0x7f6b980fb630 msgr2=0x7f6b9810ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.396+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 shutdown_connections 2023-07-14T11:39:06.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.396+0000 7f6b9f1ff700 1 -- 172.21.15.161:0/63708577 wait complete. 2023-07-14T11:39:06.394 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 41 2023-07-14T11:39:06.427 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T11:39:06.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 130333 29.19 MB/sec execute 108 sec latency 299.264 ms 2023-07-14T11:39:06.708 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:06 smithi182 ceph-mon[129366]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 384 op/s 2023-07-14T11:39:06.708 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:06 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/63708577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:39:06.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:06 smithi161 ceph-mon[95526]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 384 op/s 2023-07-14T11:39:06.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:06 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/63708577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:39:06.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.807+0000 7f9711dff700 1 -- 172.21.15.161:0/929069087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c100d00 msgr2=0x7f970c1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 --2- 172.21.15.161:0/929069087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c100d00 0x7f970c1010d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009990 tx=0x7f96fc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 -- 172.21.15.161:0/929069087 shutdown_connections 2023-07-14T11:39:06.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 --2- 172.21.15.161:0/929069087 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f970c10a040 0x7f970c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 --2- 172.21.15.161:0/929069087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c100d00 0x7f970c1010d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 --2- 172.21.15.161:0/929069087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c101610 0x7f970c109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 -- 172.21.15.161:0/929069087 >> 172.21.15.161:0/929069087 conn(0x7f970c0fa4b0 msgr2=0x7f970c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 -- 172.21.15.161:0/929069087 shutdown_connections 2023-07-14T11:39:06.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.808+0000 7f9711dff700 1 -- 172.21.15.161:0/929069087 wait complete. 2023-07-14T11:39:06.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.809+0000 7f9711dff700 1 Processor -- start 2023-07-14T11:39:06.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.809+0000 7f9711dff700 1 -- start start 2023-07-14T11:39:06.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f9711dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c100d00 0x7f970c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f9711dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c101610 0x7f970c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f9711dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f970c10a040 0x7f970c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f9711dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c19ce00 con 0x7f970c100d00 2023-07-14T11:39:06.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f9711dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c19cf70 con 0x7f970c101610 2023-07-14T11:39:06.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f9711dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c19d130 con 0x7f970c10a040 2023-07-14T11:39:06.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c101610 0x7f970c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c101610 0x7f970c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37614/0 (socket says 172.21.15.161:37614) 2023-07-14T11:39:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 -- 172.21.15.161:0/2723067546 learned_addr learned my addr 172.21.15.161:0/2723067546 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c100d00 0x7f970c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970bfff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f970c10a040 0x7f970c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 -- 172.21.15.161:0/2723067546 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f970c10a040 msgr2=0x7f970c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f970c10a040 0x7f970c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 -- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c100d00 msgr2=0x7f970c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c100d00 0x7f970c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970affd700 1 -- 172.21.15.161:0/2723067546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc0095f0 con 0x7f970c101610 2023-07-14T11:39:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.810+0000 7f970b7fe700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c100d00 0x7f970c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:06.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.811+0000 7f970affd700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c101610 0x7f970c198570 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f970000e9a0 tx=0x7f970000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.811+0000 7f9708ff9700 1 -- 172.21.15.161:0/2723067546 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700009790 con 0x7f970c101610 2023-07-14T11:39:06.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.811+0000 7f9708ff9700 1 -- 172.21.15.161:0/2723067546 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9700010430 con 0x7f970c101610 2023-07-14T11:39:06.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.811+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970c19d410 con 0x7f970c101610 2023-07-14T11:39:06.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.811+0000 7f9708ff9700 1 -- 172.21.15.161:0/2723067546 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700018680 con 0x7f970c101610 2023-07-14T11:39:06.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.811+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970c1a4c50 con 0x7f970c101610 2023-07-14T11:39:06.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.815+0000 7f9708ff9700 1 -- 172.21.15.161:0/2723067546 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f97000105a0 con 0x7f970c101610 2023-07-14T11:39:06.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.816+0000 7f972581fb80 1 Processor -- start 2023-07-14T11:39:06.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f972581fb80 1 -- start start 2023-07-14T11:39:06.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f972581fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c72d0 0x5601fe2c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f972581fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2c7be0 0x5601fe2d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f972581fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2d2ad0 0x5601fe2d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f972581fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe1cdcb0 con 0x5601fe2c7be0 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f972581fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe220a40 con 0x5601fe2d2ad0 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f972581fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe219ed0 con 0x5601fe2c72d0 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f970bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c72d0 0x5601fe2c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f970bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c72d0 0x5601fe2c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45938/0 (socket says 172.21.15.161:45938) 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f970b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2c7be0 0x5601fe2d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f970bfff700 1 -- 172.21.15.161:0/3218333057 learned_addr learned my addr 172.21.15.161:0/3218333057 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.817+0000 7f970affd700 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2d2ad0 0x5601fe2d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.818+0000 7f9708ff9700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f40a12c0 0x7f96f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.818+0000 7f970bfff700 1 -- 172.21.15.161:0/3218333057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2d2ad0 msgr2=0x5601fe2d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.818+0000 7f970bfff700 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2d2ad0 0x5601fe2d4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.818+0000 7f970bfff700 1 -- 172.21.15.161:0/3218333057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2c7be0 msgr2=0x5601fe2d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.818+0000 7f970bfff700 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2c7be0 0x5601fe2d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.818+0000 7f970bfff700 1 -- 172.21.15.161:0/3218333057 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fe047de0 con 0x5601fe2c72d0 2023-07-14T11:39:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.818+0000 7f9708ff9700 1 -- 172.21.15.161:0/2723067546 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f9700014070 con 0x7f970c101610 2023-07-14T11:39:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.819+0000 7f970bfff700 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c72d0 0x5601fe2c76a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9704011680 tx=0x7f9704009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9ea905245abc1d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.819+0000 7f970bfff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f40a12c0 0x7f96f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.820+0000 7f970bfff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f40a12c0 0x7f96f40a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9704020000 tx=0x7f9704023000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.823+0000 7f96f1ffb700 1 -- 172.21.15.161:0/3218333057 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970400e040 con 0x5601fe2c72d0 2023-07-14T11:39:06.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.823+0000 7f96f1ffb700 1 -- 172.21.15.161:0/3218333057 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f970400f040 con 0x5601fe2c72d0 2023-07-14T11:39:06.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.823+0000 7f96f1ffb700 1 -- 172.21.15.161:0/3218333057 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97040156b0 con 0x5601fe2c72d0 2023-07-14T11:39:06.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.824+0000 7f972581fb80 1 -- 172.21.15.161:0/3218333057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c72d0 msgr2=0x5601fe2c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.824+0000 7f972581fb80 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c72d0 0x5601fe2c76a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9704011680 tx=0x7f9704009920 comp rx=0 tx=0).stop 2023-07-14T11:39:06.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.825+0000 7f972581fb80 1 -- 172.21.15.161:0/3218333057 shutdown_connections 2023-07-14T11:39:06.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.825+0000 7f972581fb80 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2d2ad0 0x5601fe2d4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.825+0000 7f972581fb80 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c72d0 0x5601fe2c76a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.825+0000 7f972581fb80 1 --2- 172.21.15.161:0/3218333057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2c7be0 0x5601fe2d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.825+0000 7f972581fb80 1 -- 172.21.15.161:0/3218333057 >> 172.21.15.161:0/3218333057 conn(0x5601fe2bdda0 msgr2=0x5601fe2c36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.829+0000 7f972581fb80 1 -- 172.21.15.161:0/3218333057 shutdown_connections 2023-07-14T11:39:06.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.829+0000 7f972581fb80 1 -- 172.21.15.161:0/3218333057 wait complete. 2023-07-14T11:39:06.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.830+0000 7f972581fb80 1 -- start start 2023-07-14T11:39:06.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.830+0000 7f972581fb80 1 Processor -- start 2023-07-14T11:39:06.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.831+0000 7f972581fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.831+0000 7f972581fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.831+0000 7f972581fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.831+0000 7f972581fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe1b9ac0 con 0x5601fe2d2ad0 2023-07-14T11:39:06.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.831+0000 7f972581fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe1b97c0 con 0x5601fe2c72d0 2023-07-14T11:39:06.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.831+0000 7f972581fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe09d920 con 0x5601fe2c7be0 2023-07-14T11:39:06.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.831+0000 7f972581fb80 10 client.0 _ll_register_callbacks cb 0x5601fe1b7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:06.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45944/0 (socket says 172.21.15.161:45944) 2023-07-14T11:39:06.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 -- 172.21.15.161:0/1684951882 learned_addr learned my addr 172.21.15.161:0/1684951882 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:06.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970b7fe700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970bfff700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 -- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 msgr2=0x5601fe2d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 -- 172.21.15.161:0/1684951882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 msgr2=0x5601fe2cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970affd700 1 -- 172.21.15.161:0/1684951882 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9704011330 con 0x5601fe2c7be0 2023-07-14T11:39:06.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.832+0000 7f970b7fe700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:39:06.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.833+0000 7f970affd700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2cf7e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f97000a1ed0 tx=0x7f97000cab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4b03d5daa91157 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.833+0000 7f96eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:06.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.833+0000 7f96eaffd700 1 -- 172.21.15.161:0/1684951882 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700099af0 con 0x5601fe2c7be0 2023-07-14T11:39:06.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.833+0000 7f96eaffd700 1 -- 172.21.15.161:0/1684951882 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f9700099fb0 con 0x5601fe2c7be0 2023-07-14T11:39:06.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.834+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601fe1734f0 con 0x5601fe2c7be0 2023-07-14T11:39:06.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.834+0000 7f96eaffd700 1 -- 172.21.15.161:0/1684951882 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9700099910 con 0x5601fe2c7be0 2023-07-14T11:39:06.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.834+0000 7f972581fb80 10 client.27568 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:06.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.834+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5601fe030c60 con 0x5601fe2c7be0 2023-07-14T11:39:06.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.835+0000 7f96eaffd700 1 -- 172.21.15.161:0/1684951882 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601fe1734f0 con 0x5601fe2c7be0 2023-07-14T11:39:06.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.836+0000 7f96eaffd700 1 -- 172.21.15.161:0/1684951882 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f97000a1f70 con 0x5601fe2c7be0 2023-07-14T11:39:06.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.836+0000 7f972581fb80 10 client.27568 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:06.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.836+0000 7f972581fb80 10 client.27568 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:39:06.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.836+0000 7f972581fb80 20 client.27568 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:06.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.836+0000 7f972581fb80 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe2eac80 0x5601fe2ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.836+0000 7f972581fb80 4 client.27568 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:39:06.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.836+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601fe212eb0 con 0x5601fe2eac80 2023-07-14T11:39:06.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.837+0000 7f970b7fe700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe2eac80 0x5601fe2ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.839+0000 7f970b7fe700 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe2eac80 0x5601fe2ed090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96f938502bdddf2d server_cookie=2f25c5fc753c2426 in_seq=0 out_seq=0 2023-07-14T11:39:06.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.839+0000 7f96eaffd700 10 client.27568 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:39:06.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.840+0000 7f96eaffd700 1 -- 172.21.15.161:0/1684951882 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5601fe212eb0 con 0x5601fe2eac80 2023-07-14T11:39:06.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.840+0000 7f96eaffd700 10 client.27568 handle_command_reply: tid=0 2023-07-14T11:39:06.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.840+0000 7f972581fb80 1 client.27568 shutdown 2023-07-14T11:39:06.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.840+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 msgr2=0x5601fe2cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.840+0000 7f972581fb80 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2cf7e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f97000a1ed0 tx=0x7f97000cab90 comp rx=0 tx=0).stop 2023-07-14T11:39:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 shutdown_connections 2023-07-14T11:39:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f972581fb80 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe2eac80 0x5601fe2ed090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f972581fb80 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f972581fb80 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f972581fb80 1 --2- 172.21.15.161:0/1684951882 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2cf7e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 >> 172.21.15.161:0/1684951882 conn(0x5601fe2bdda0 msgr2=0x5601fe2c3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f96eaffd700 0 client.27568 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:39:06.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.841+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 shutdown_connections 2023-07-14T11:39:06.841 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.842+0000 7f972581fb80 1 -- 172.21.15.161:0/1684951882 wait complete. 2023-07-14T11:39:06.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.843+0000 7f972581fb80 20 client.27568 trim_cache size 0 max 16384 2023-07-14T11:39:06.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.867+0000 7f972581fb80 1 Processor -- start 2023-07-14T11:39:06.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.868+0000 7f972581fb80 1 -- start start 2023-07-14T11:39:06.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.868+0000 7f972581fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.868+0000 7f972581fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.868+0000 7f972581fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.868+0000 7f972581fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe2eaab0 con 0x5601fe2d2ad0 2023-07-14T11:39:06.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.868+0000 7f972581fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe2e83d0 con 0x5601fe2c72d0 2023-07-14T11:39:06.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.868+0000 7f972581fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe2bfec0 con 0x5601fe2c7be0 2023-07-14T11:39:06.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37642/0 (socket says 172.21.15.161:37642) 2023-07-14T11:39:06.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 -- 172.21.15.161:0/366203871 learned_addr learned my addr 172.21.15.161:0/366203871 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:06.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970affd700 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970bfff700 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 -- 172.21.15.161:0/366203871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 msgr2=0x5601fe2ecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2ecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 -- 172.21.15.161:0/366203871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 msgr2=0x5601fe2cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970b7fe700 1 -- 172.21.15.161:0/366203871 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fe06e250 con 0x5601fe2c72d0 2023-07-14T11:39:06.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.869+0000 7f970affd700 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:39:06.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.870+0000 7f970b7fe700 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2cefa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f96fc015000 tx=0x7f96fc005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc4680012012077 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.870+0000 7f96f27fc700 1 -- 172.21.15.161:0/366203871 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc01d040 con 0x5601fe2c72d0 2023-07-14T11:39:06.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.870+0000 7f96f27fc700 1 -- 172.21.15.161:0/366203871 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96fc01e040 con 0x5601fe2c72d0 2023-07-14T11:39:06.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.870+0000 7f96f27fc700 1 -- 172.21.15.161:0/366203871 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00f920 con 0x5601fe2c72d0 2023-07-14T11:39:06.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 -- 172.21.15.161:0/366203871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 msgr2=0x5601fe2cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.872 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2cefa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f96fc015000 tx=0x7f96fc005fb0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 -- 172.21.15.161:0/366203871 shutdown_connections 2023-07-14T11:39:06.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2cefa0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 --2- 172.21.15.161:0/366203871 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 -- 172.21.15.161:0/366203871 >> 172.21.15.161:0/366203871 conn(0x5601fe2bdda0 msgr2=0x5601fe2d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.873 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 -- 172.21.15.161:0/366203871 shutdown_connections 2023-07-14T11:39:06.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 -- 172.21.15.161:0/366203871 wait complete. 2023-07-14T11:39:06.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.871+0000 7f972581fb80 1 -- start start 2023-07-14T11:39:06.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.872+0000 7f972581fb80 1 Processor -- start 2023-07-14T11:39:06.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.872+0000 7f972581fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f972581fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f972581fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f972581fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe2e08e0 con 0x5601fe2d2ad0 2023-07-14T11:39:06.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f972581fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe2e0a50 con 0x5601fe2c72d0 2023-07-14T11:39:06.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f972581fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fe2e0c10 con 0x5601fe2c7be0 2023-07-14T11:39:06.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f972581fb80 10 client.0 _ll_register_callbacks cb 0x5601fe1b7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:06.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2e1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37654/0 (socket says 172.21.15.161:37654) 2023-07-14T11:39:06.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 -- 172.21.15.161:0/1388962638 learned_addr learned my addr 172.21.15.161:0/1388962638 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:06.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 -- 172.21.15.161:0/1388962638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 msgr2=0x5601fe2dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 -- 172.21.15.161:0/1388962638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 msgr2=0x5601fe2e03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:06.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.873+0000 7f970b7fe700 1 -- 172.21.15.161:0/1388962638 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970c10d170 con 0x5601fe2c72d0 2023-07-14T11:39:06.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.874+0000 7f970b7fe700 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2e1b30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b2e0 tx=0x7f96fc0286d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11752b1f00efa785 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:06.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.874+0000 7f96eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:39:06.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.874+0000 7f96eaffd700 1 -- 172.21.15.161:0/1388962638 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc01d040 con 0x5601fe2c72d0 2023-07-14T11:39:06.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.874+0000 7f96eaffd700 1 -- 172.21.15.161:0/1388962638 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f96fc028d40 con 0x5601fe2c72d0 2023-07-14T11:39:06.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.874+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f970400f040 con 0x5601fe2c72d0 2023-07-14T11:39:06.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.874+0000 7f96eaffd700 1 -- 172.21.15.161:0/1388962638 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96fc00f5a0 con 0x5601fe2c72d0 2023-07-14T11:39:06.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.874+0000 7f972581fb80 10 client.27527 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:06.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.875+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5601fe0462b0 con 0x5601fe2c72d0 2023-07-14T11:39:06.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.875+0000 7f96eaffd700 1 -- 172.21.15.161:0/1388962638 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f970400f040 con 0x5601fe2c72d0 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.875+0000 7f96eaffd700 1 -- 172.21.15.161:0/1388962638 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f96fc03b070 con 0x5601fe2c72d0 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.875+0000 7f972581fb80 10 client.27527 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.875+0000 7f972581fb80 10 client.27527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.875+0000 7f972581fb80 20 client.27527 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.875+0000 7f972581fb80 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe339b20 0x5601fe339f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.876+0000 7f972581fb80 4 client.27527 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.876+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 --> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5601fe1d5050 con 0x5601fe339b20 2023-07-14T11:39:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.876+0000 7f970affd700 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe339b20 0x5601fe339f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:06.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.877+0000 7f970affd700 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe339b20 0x5601fe339f20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43f037998fe4eb28 server_cookie=53e1a208b524ce3e in_seq=0 out_seq=0 2023-07-14T11:39:06.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.877+0000 7f96eaffd700 10 client.27527 ms_handle_connect on v2:172.21.15.175:6836/4186191407 2023-07-14T11:39:06.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.881+0000 7f96eaffd700 1 -- 172.21.15.161:0/1388962638 <== mds.2 v2:172.21.15.175:6836/4186191407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5601fe1d5050 con 0x5601fe339b20 2023-07-14T11:39:06.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.881+0000 7f96eaffd700 10 client.27527 handle_command_reply: tid=0 2023-07-14T11:39:06.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.881+0000 7f972581fb80 1 client.27527 shutdown 2023-07-14T11:39:06.880 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.881+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 msgr2=0x5601fe2e1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.881+0000 7f972581fb80 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2e1b30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b2e0 tx=0x7f96fc0286d0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 shutdown_connections 2023-07-14T11:39:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:6836/4186191407,v1:172.21.15.175:6837/4186191407] conn(0x5601fe339b20 0x5601fe339f20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601fe2d2ad0 0x5601fe2e03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601fe2c72d0 0x5601fe2e1b30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 --2- 172.21.15.161:0/1388962638 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601fe2c7be0 0x5601fe2dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 >> 172.21.15.161:0/1388962638 conn(0x5601fe2bdda0 msgr2=0x5601fe2cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f96eaffd700 0 client.27527 ms_handle_reset on v2:172.21.15.175:6836/4186191407 2023-07-14T11:39:06.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 shutdown_connections 2023-07-14T11:39:06.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 1 -- 172.21.15.161:0/1388962638 wait complete. 2023-07-14T11:39:06.883 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.882+0000 7f972581fb80 20 client.27527 trim_cache size 0 max 16384 2023-07-14T11:39:06.884 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:39:06.885 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f40a12c0 msgr2=0x7f96f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f40a12c0 0x7f96f40a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9704020000 tx=0x7f9704023000 comp rx=0 tx=0).stop 2023-07-14T11:39:06.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c101610 msgr2=0x7f970c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:06.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c101610 0x7f970c198570 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f970000e9a0 tx=0x7f970000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.886 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 shutdown_connections 2023-07-14T11:39:06.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f96f40a12c0 0x7f96f40a3770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c100d00 0x7f970c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f970c10a040 0x7f970c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 --2- 172.21.15.161:0/2723067546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f970c101610 0x7f970c198570 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:06.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.883+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 >> 172.21.15.161:0/2723067546 conn(0x7f970c0fa4b0 msgr2=0x7f970c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:06.887 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.884+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 shutdown_connections 2023-07-14T11:39:06.888 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:06.884+0000 7f9711dff700 1 -- 172.21.15.161:0/2723067546 wait complete. 2023-07-14T11:39:06.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T11:39:06.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:06 smithi175 ceph-mon[129081]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 384 op/s 2023-07-14T11:39:06.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:06 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/63708577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:39:07.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.212+0000 7fde72bff700 1 -- 172.21.15.161:0/394884122 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fde6c101f70 msgr2=0x7fde6c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.212+0000 7fde72bff700 1 --2- 172.21.15.161:0/394884122 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fde6c101f70 0x7fde6c102340 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00b0d0 tx=0x7fde5c00b3e0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.213+0000 7fde72bff700 1 -- 172.21.15.161:0/394884122 shutdown_connections 2023-07-14T11:39:07.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.213+0000 7fde72bff700 1 --2- 172.21.15.161:0/394884122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c106fb0 0x7fde6c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.213+0000 7fde72bff700 1 --2- 172.21.15.161:0/394884122 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fde6c101f70 0x7fde6c102340 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.213+0000 7fde72bff700 1 --2- 172.21.15.161:0/394884122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde6c102910 0x7fde6c1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.213+0000 7fde72bff700 1 -- 172.21.15.161:0/394884122 >> 172.21.15.161:0/394884122 conn(0x7fde6c0fd800 msgr2=0x7fde6c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.213+0000 7fde72bff700 1 -- 172.21.15.161:0/394884122 shutdown_connections 2023-07-14T11:39:07.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.213+0000 7fde72bff700 1 -- 172.21.15.161:0/394884122 wait complete. 2023-07-14T11:39:07.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.214+0000 7fde72bff700 1 Processor -- start 2023-07-14T11:39:07.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.214+0000 7fde72bff700 1 -- start start 2023-07-14T11:39:07.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde72bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c101f70 0x7fde6c10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde72bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fde6c102910 0x7fde6c10b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde72bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde6c106fb0 0x7fde6c10bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde72bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c10c180 con 0x7fde6c106fb0 2023-07-14T11:39:07.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde72bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c10c2f0 con 0x7fde6c101f70 2023-07-14T11:39:07.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde72bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c10c4b0 con 0x7fde6c102910 2023-07-14T11:39:07.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7099b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c101f70 0x7fde6c10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7119c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde6c106fb0 0x7fde6c10bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7099b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c101f70 0x7fde6c10e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37674/0 (socket says 172.21.15.161:37674) 2023-07-14T11:39:07.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7099b700 1 -- 172.21.15.161:0/929999816 learned_addr learned my addr 172.21.15.161:0/929999816 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7119c700 1 -- 172.21.15.161:0/929999816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fde6c102910 msgr2=0x7fde6c10b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7119c700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fde6c102910 0x7fde6c10b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7119c700 1 -- 172.21.15.161:0/929999816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c101f70 msgr2=0x7fde6c10e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7119c700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c101f70 0x7fde6c10e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7119c700 1 -- 172.21.15.161:0/929999816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c009d00 con 0x7fde6c106fb0 2023-07-14T11:39:07.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.215+0000 7fde7099b700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c101f70 0x7fde6c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:07.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.216+0000 7fde7119c700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde6c106fb0 0x7fde6c10bc40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e970 tx=0x7fde6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.216+0000 7fde6a7fc700 1 -- 172.21.15.161:0/929999816 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6400cbd0 con 0x7fde6c106fb0 2023-07-14T11:39:07.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.216+0000 7fde6a7fc700 1 -- 172.21.15.161:0/929999816 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde6400ee60 con 0x7fde6c106fb0 2023-07-14T11:39:07.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.216+0000 7fde6a7fc700 1 -- 172.21.15.161:0/929999816 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64005020 con 0x7fde6c106fb0 2023-07-14T11:39:07.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.217+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c1a8c90 con 0x7fde6c106fb0 2023-07-14T11:39:07.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.217+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c1a9120 con 0x7fde6c106fb0 2023-07-14T11:39:07.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.223+0000 7fde86594b80 1 Processor -- start 2023-07-14T11:39:07.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.227+0000 7fde6a7fc700 1 -- 172.21.15.161:0/929999816 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fde64005180 con 0x7fde6c106fb0 2023-07-14T11:39:07.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.227+0000 7fde86594b80 1 -- start start 2023-07-14T11:39:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.228+0000 7fde86594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a15566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.228+0000 7fde86594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1556be0 0x55d8a1561590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.228+0000 7fde86594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1561ad0 0x55d8a1563eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.228+0000 7fde86594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a145ccb0 con 0x55d8a15562d0 2023-07-14T11:39:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.228+0000 7fde86594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a14afa40 con 0x55d8a1561ad0 2023-07-14T11:39:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.228+0000 7fde86594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a14a8ed0 con 0x55d8a1556be0 2023-07-14T11:39:07.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1556be0 0x55d8a1561590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1556be0 0x55d8a1561590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45996/0 (socket says 172.21.15.161:45996) 2023-07-14T11:39:07.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 -- 172.21.15.161:0/1159647827 learned_addr learned my addr 172.21.15.161:0/1159647827 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde63fff700 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1561ad0 0x55d8a1563eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde63fff700 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a15566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 -- 172.21.15.161:0/1159647827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1561ad0 msgr2=0x55d8a1563eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1561ad0 0x55d8a1563eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 -- 172.21.15.161:0/1159647827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 msgr2=0x55d8a15566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a15566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.229+0000 7fde7099b700 1 -- 172.21.15.161:0/1159647827 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a12d6de0 con 0x55d8a1556be0 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.230+0000 7fde7099b700 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1556be0 0x55d8a1561590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009c00 tx=0x7fde5c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17cd8bf5103bb2b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.230+0000 7fde63fff700 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1561ad0 0x55d8a1563eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.230+0000 7fde62ffd700 1 -- 172.21.15.161:0/1159647827 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00e040 con 0x55d8a1556be0 2023-07-14T11:39:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.230+0000 7fde62ffd700 1 -- 172.21.15.161:0/1159647827 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5c006160 con 0x55d8a1556be0 2023-07-14T11:39:07.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.230+0000 7fde62ffd700 1 -- 172.21.15.161:0/1159647827 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0050d0 con 0x55d8a1556be0 2023-07-14T11:39:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.230+0000 7fde6a7fc700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fde580a13c0 0x7fde580a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.230+0000 7fde6a7fc700 1 -- 172.21.15.161:0/929999816 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fde64014070 con 0x7fde6c106fb0 2023-07-14T11:39:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.231+0000 7fde63fff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fde580a13c0 0x7fde580a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.231+0000 7fde86594b80 1 -- 172.21.15.161:0/1159647827 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1556be0 msgr2=0x55d8a1561590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.231+0000 7fde86594b80 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1556be0 0x55d8a1561590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009c00 tx=0x7fde5c009c30 comp rx=0 tx=0).stop 2023-07-14T11:39:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde86594b80 1 -- 172.21.15.161:0/1159647827 shutdown_connections 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde86594b80 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a15566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde86594b80 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1561ad0 0x55d8a1563eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde86594b80 1 --2- 172.21.15.161:0/1159647827 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1556be0 0x55d8a1561590 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde86594b80 1 -- 172.21.15.161:0/1159647827 >> 172.21.15.161:0/1159647827 conn(0x55d8a154cda0 msgr2=0x55d8a15526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde63fff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fde580a13c0 0x7fde580a3870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde54005fd0 tx=0x7fde54014040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde86594b80 1 -- 172.21.15.161:0/1159647827 shutdown_connections 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.232+0000 7fde86594b80 1 -- 172.21.15.161:0/1159647827 wait complete. 2023-07-14T11:39:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.233+0000 7fde86594b80 1 -- start start 2023-07-14T11:39:07.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.233+0000 7fde86594b80 1 Processor -- start 2023-07-14T11:39:07.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde86594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a15562d0 0x55d8a1561540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde86594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1556be0 0x55d8a155e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde86594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a155ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde86594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a1448ac0 con 0x55d8a1556be0 2023-07-14T11:39:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde86594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a14487c0 con 0x55d8a15562d0 2023-07-14T11:39:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde86594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a132c920 con 0x55d8a1561ad0 2023-07-14T11:39:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde86594b80 10 client.0 _ll_register_callbacks cb 0x55d8a1446760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde7119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a155ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde7119c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a155ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46006/0 (socket says 172.21.15.161:46006) 2023-07-14T11:39:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde7119c700 1 -- 172.21.15.161:0/2951996833 learned_addr learned my addr 172.21.15.161:0/2951996833 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.234+0000 7fde7099b700 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a15562d0 0x55d8a1561540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde63fff700 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1556be0 0x55d8a155e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde7119c700 1 -- 172.21.15.161:0/2951996833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a15562d0 msgr2=0x55d8a1561540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde7119c700 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a15562d0 0x55d8a1561540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde7119c700 1 -- 172.21.15.161:0/2951996833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1556be0 msgr2=0x55d8a155e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde7119c700 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1556be0 0x55d8a155e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde7119c700 1 -- 172.21.15.161:0/2951996833 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c10f3c0 con 0x55d8a1561ad0 2023-07-14T11:39:07.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde7119c700 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a155ee40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fde640960d0 tx=0x7fde640ca7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0769b1253d2fdf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:07.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.235+0000 7fde4bfff700 1 -- 172.21.15.161:0/2951996833 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640b1070 con 0x55d8a1561ad0 2023-07-14T11:39:07.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.236+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8a14024f0 con 0x55d8a1561ad0 2023-07-14T11:39:07.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.236+0000 7fde4bfff700 1 -- 172.21.15.161:0/2951996833 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fde6409dc00 con 0x55d8a1561ad0 2023-07-14T11:39:07.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.236+0000 7fde4bfff700 1 -- 172.21.15.161:0/2951996833 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde640a2020 con 0x55d8a1561ad0 2023-07-14T11:39:07.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.236+0000 7fde86594b80 10 client.27580 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:07.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.236+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d8a12bfc60 con 0x55d8a1561ad0 2023-07-14T11:39:07.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde4bfff700 1 -- 172.21.15.161:0/2951996833 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8a14024f0 con 0x55d8a1561ad0 2023-07-14T11:39:07.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde4bfff700 1 -- 172.21.15.161:0/2951996833 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fde64005430 con 0x55d8a1561ad0 2023-07-14T11:39:07.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde86594b80 10 client.27580 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:07.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde86594b80 10 client.27580 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:39:07.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde86594b80 20 client.27580 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:07.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde86594b80 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15713f0 0x55d8a15737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde86594b80 4 client.27580 mds_command: new command op to 24544 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:39:07.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8a14a1eb0 con 0x55d8a15713f0 2023-07-14T11:39:07.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.237+0000 7fde7099b700 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15713f0 0x55d8a15737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.238+0000 7fde7099b700 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15713f0 0x55d8a15737d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a02c108e59382488 server_cookie=519b551f1376d244 in_seq=0 out_seq=0 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.238+0000 7fde4bfff700 10 client.27580 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde4bfff700 1 -- 172.21.15.161:0/2951996833 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d8a14a1eb0 con 0x55d8a15713f0 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde4bfff700 10 client.27580 handle_command_reply: tid=0 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 client.27580 shutdown 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 msgr2=0x55d8a155ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a155ee40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fde640960d0 tx=0x7fde640ca7b0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 shutdown_connections 2023-07-14T11:39:07.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15713f0 0x55d8a15737d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a155ee40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a15562d0 0x55d8a1561540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 --2- 172.21.15.161:0/2951996833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1556be0 0x55d8a155e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.241+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 >> 172.21.15.161:0/2951996833 conn(0x55d8a154cda0 msgr2=0x55d8a1552230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.242+0000 7fde4bfff700 0 client.27580 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:39:07.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.242+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 shutdown_connections 2023-07-14T11:39:07.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.242+0000 7fde86594b80 1 -- 172.21.15.161:0/2951996833 wait complete. 2023-07-14T11:39:07.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.242+0000 7fde86594b80 20 client.27580 trim_cache size 0 max 16384 2023-07-14T11:39:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 Processor -- start 2023-07-14T11:39:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 -- start start 2023-07-14T11:39:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a1561070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a1559a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a1559f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a1557da0 con 0x55d8a15562d0 2023-07-14T11:39:07.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a15711d0 con 0x55d8a1556be0 2023-07-14T11:39:07.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.263+0000 7fde86594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a15581a0 con 0x55d8a1561ad0 2023-07-14T11:39:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde7099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a1561070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde63fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a1559a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde7099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a1561070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44400/0 (socket says 172.21.15.161:44400) 2023-07-14T11:39:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde7099b700 1 -- 172.21.15.161:0/1206090115 learned_addr learned my addr 172.21.15.161:0/1206090115 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde7119c700 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a1559f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde63fff700 1 -- 172.21.15.161:0/1206090115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 msgr2=0x55d8a1559f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde63fff700 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a1559f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde63fff700 1 -- 172.21.15.161:0/1206090115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 msgr2=0x55d8a1561070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde63fff700 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a1561070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde63fff700 1 -- 172.21.15.161:0/1206090115 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a12fd250 con 0x55d8a1556be0 2023-07-14T11:39:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.264+0000 7fde7119c700 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a1559f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.265+0000 7fde7099b700 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a1561070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.265+0000 7fde63fff700 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a1559a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fde54005d90 tx=0x7fde54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b58b894ce52e5c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.265+0000 7fde637fe700 1 -- 172.21.15.161:0/1206090115 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54011ab0 con 0x55d8a1556be0 2023-07-14T11:39:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.265+0000 7fde637fe700 1 -- 172.21.15.161:0/1206090115 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde54011c10 con 0x55d8a1556be0 2023-07-14T11:39:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.265+0000 7fde637fe700 1 -- 172.21.15.161:0/1206090115 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540103d0 con 0x55d8a1556be0 2023-07-14T11:39:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 -- 172.21.15.161:0/1206090115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 msgr2=0x55d8a1559a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a1559a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fde54005d90 tx=0x7fde54004750 comp rx=0 tx=0).stop 2023-07-14T11:39:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 -- 172.21.15.161:0/1206090115 shutdown_connections 2023-07-14T11:39:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a15562d0 0x55d8a1561070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a1561ad0 0x55d8a1559f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 --2- 172.21.15.161:0/1206090115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a1559a30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 -- 172.21.15.161:0/1206090115 >> 172.21.15.161:0/1206090115 conn(0x55d8a154cda0 msgr2=0x55d8a156d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 -- 172.21.15.161:0/1206090115 shutdown_connections 2023-07-14T11:39:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.266+0000 7fde86594b80 1 -- 172.21.15.161:0/1206090115 wait complete. 2023-07-14T11:39:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.267+0000 7fde86594b80 1 -- start start 2023-07-14T11:39:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.267+0000 7fde86594b80 1 Processor -- start 2023-07-14T11:39:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde86594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a15562d0 0x55d8a1566360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde86594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a15668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde86594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1561ad0 0x55d8a15be220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde86594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a15be760 con 0x55d8a1561ad0 2023-07-14T11:39:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde86594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a15be8d0 con 0x55d8a1556be0 2023-07-14T11:39:07.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde86594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a15bea90 con 0x55d8a15562d0 2023-07-14T11:39:07.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde86594b80 10 client.0 _ll_register_callbacks cb 0x7fde64099710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:07.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde7099b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a15562d0 0x55d8a1566360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde7099b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a15562d0 0x55d8a1566360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46026/0 (socket says 172.21.15.161:46026) 2023-07-14T11:39:07.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde7099b700 1 -- 172.21.15.161:0/3021912677 learned_addr learned my addr 172.21.15.161:0/3021912677 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.268+0000 7fde63fff700 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a15668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.269+0000 7fde7099b700 1 -- 172.21.15.161:0/3021912677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 msgr2=0x55d8a15668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.269+0000 7fde7099b700 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a15668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.269+0000 7fde7099b700 1 -- 172.21.15.161:0/3021912677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1561ad0 msgr2=0x55d8a15be220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.269+0000 7fde7099b700 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1561ad0 0x55d8a15be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.269+0000 7fde7099b700 1 -- 172.21.15.161:0/3021912677 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8a12d6de0 con 0x55d8a15562d0 2023-07-14T11:39:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.269+0000 7fde7099b700 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a15562d0 0x55d8a1566360 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009c00 tx=0x7fde5c003ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f70e2305a6b9c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.269+0000 7fde4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.270+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021912677 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c02b040 con 0x55d8a15562d0 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.270+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021912677 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fde5c0315c0 con 0x55d8a15562d0 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.270+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde5c006160 con 0x55d8a15562d0 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.271+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021912677 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde5c00e040 con 0x55d8a15562d0 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.271+0000 7fde86594b80 10 client.27586 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.271+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d8a12bfc60 con 0x55d8a15562d0 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.271+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021912677 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5c006160 con 0x55d8a15562d0 2023-07-14T11:39:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021912677 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fde5c03b070 con 0x55d8a15562d0 2023-07-14T11:39:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde86594b80 10 client.27586 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde86594b80 10 client.27586 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T11:39:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde86594b80 20 client.27586 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde86594b80 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15c5ed0 0x55d8a15c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde86594b80 4 client.27586 mds_command: new command op to 24544 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:39:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 --> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8a1464050 con 0x55d8a15c5ed0 2023-07-14T11:39:07.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.272+0000 7fde63fff700 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15c5ed0 0x55d8a15c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.273+0000 7fde63fff700 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15c5ed0 0x55d8a15c82e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d71fc5d57193892 server_cookie=21def18ee00ea67 in_seq=0 out_seq=0 2023-07-14T11:39:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.273+0000 7fde4bfff700 10 client.27586 ms_handle_connect on v2:172.21.15.182:6838/392865750 2023-07-14T11:39:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021912677 <== mds.1 v2:172.21.15.182:6838/392865750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8a1464050 con 0x55d8a15c5ed0 2023-07-14T11:39:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde4bfff700 10 client.27586 handle_command_reply: tid=0 2023-07-14T11:39:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 client.27586 shutdown 2023-07-14T11:39:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a15562d0 msgr2=0x55d8a1566360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a15562d0 0x55d8a1566360 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009c00 tx=0x7fde5c003ad0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 shutdown_connections 2023-07-14T11:39:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:6838/392865750,v1:172.21.15.182:6839/392865750] conn(0x55d8a15c5ed0 0x55d8a15c82e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8a1561ad0 0x55d8a15be220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d8a15562d0 0x55d8a1566360 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 --2- 172.21.15.161:0/3021912677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8a1556be0 0x55d8a15668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 >> 172.21.15.161:0/3021912677 conn(0x55d8a154cda0 msgr2=0x55d8a1559a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.277+0000 7fde4bfff700 0 client.27586 ms_handle_reset on v2:172.21.15.182:6838/392865750 2023-07-14T11:39:07.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.278+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 shutdown_connections 2023-07-14T11:39:07.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.278+0000 7fde86594b80 1 -- 172.21.15.161:0/3021912677 wait complete. 2023-07-14T11:39:07.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.278+0000 7fde86594b80 20 client.27586 trim_cache size 0 max 16384 2023-07-14T11:39:07.276 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:39:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fde580a13c0 msgr2=0x7fde580a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fde580a13c0 0x7fde580a3870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde54005fd0 tx=0x7fde54014040 comp rx=0 tx=0).stop 2023-07-14T11:39:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde6c106fb0 msgr2=0x7fde6c10bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde6c106fb0 0x7fde6c10bc40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e970 tx=0x7fde6400ec80 comp rx=0 tx=0).stop 2023-07-14T11:39:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 shutdown_connections 2023-07-14T11:39:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fde580a13c0 0x7fde580a3870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde6c106fb0 0x7fde6c10bc40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c101f70 0x7fde6c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 --2- 172.21.15.161:0/929999816 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fde6c102910 0x7fde6c10b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 >> 172.21.15.161:0/929999816 conn(0x7fde6c0fd800 msgr2=0x7fde6c1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 shutdown_connections 2023-07-14T11:39:07.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.279+0000 7fde72bff700 1 -- 172.21.15.161:0/929999816 wait complete. 2023-07-14T11:39:07.302 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T11:39:07.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 131107 29.23 MB/sec execute 109 sec latency 67.939 ms 2023-07-14T11:39:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.634+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1549059515 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe474105b00 msgr2=0x7fe474105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.634+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1549059515 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe474105b00 0x7fe474105ed0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe4700098f0 tx=0x7fe470009c00 comp rx=0 tx=0).stop 2023-07-14T11:39:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.635+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1549059515 shutdown_connections 2023-07-14T11:39:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.635+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1549059515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4741004a0 0x7fe47410c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.635+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1549059515 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe474105b00 0x7fe474105ed0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.635+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1549059515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4740ffaf0 0x7fe4740fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.635+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1549059515 >> 172.21.15.161:0/1549059515 conn(0x7fe4740fb650 msgr2=0x7fe4740fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.635+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1549059515 shutdown_connections 2023-07-14T11:39:07.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.635+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1549059515 wait complete. 2023-07-14T11:39:07.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.636+0000 7fe47c7ff700 1 Processor -- start 2023-07-14T11:39:07.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.636+0000 7fe47c7ff700 1 -- start start 2023-07-14T11:39:07.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47c7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe4740ffaf0 0x7fe474197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47c7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4741004a0 0x7fe474198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47c7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe474105b00 0x7fe47419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47c7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419ce30 con 0x7fe4741004a0 2023-07-14T11:39:07.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47c7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419cfa0 con 0x7fe474105b00 2023-07-14T11:39:07.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47c7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419d160 con 0x7fe4740ffaf0 2023-07-14T11:39:07.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe4740ffaf0 0x7fe474197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47ad9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe474105b00 0x7fe47419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe4740ffaf0 0x7fe474197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46048/0 (socket says 172.21.15.161:46048) 2023-07-14T11:39:07.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 -- 172.21.15.161:0/1144920382 learned_addr learned my addr 172.21.15.161:0/1144920382 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe479d9a700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4741004a0 0x7fe474198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 -- 172.21.15.161:0/1144920382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe474105b00 msgr2=0x7fe47419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe474105b00 0x7fe47419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 -- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4741004a0 msgr2=0x7fe474198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4741004a0 0x7fe474198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.637+0000 7fe47a59b700 1 -- 172.21.15.161:0/1144920382 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470009520 con 0x7fe4740ffaf0 2023-07-14T11:39:07.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.638+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe4740ffaf0 0x7fe474197e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe470009810 tx=0x7fe47000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.640+0000 7fe46b7fe700 1 -- 172.21.15.161:0/1144920382 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47001c070 con 0x7fe4740ffaf0 2023-07-14T11:39:07.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.640+0000 7fe46b7fe700 1 -- 172.21.15.161:0/1144920382 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe470005040 con 0x7fe4740ffaf0 2023-07-14T11:39:07.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.640+0000 7fe46b7fe700 1 -- 172.21.15.161:0/1144920382 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470004a30 con 0x7fe4740ffaf0 2023-07-14T11:39:07.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.640+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47419d3e0 con 0x7fe4740ffaf0 2023-07-14T11:39:07.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.640+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47419d660 con 0x7fe4740ffaf0 2023-07-14T11:39:07.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.645+0000 7fe46b7fe700 1 -- 172.21.15.161:0/1144920382 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe470020070 con 0x7fe4740ffaf0 2023-07-14T11:39:07.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.646+0000 7fe4901feb80 1 Processor -- start 2023-07-14T11:39:07.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.648+0000 7fe4901feb80 1 -- start start 2023-07-14T11:39:07.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c3856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c390590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c392eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe4901feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c28bcb0 con 0x55cd8c385be0 2023-07-14T11:39:07.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe4901feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c2dea40 con 0x55cd8c390ad0 2023-07-14T11:39:07.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe4901feb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c2d7ed0 con 0x55cd8c3852d0 2023-07-14T11:39:07.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe47a59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c390590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe47a59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c390590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44442/0 (socket says 172.21.15.161:44442) 2023-07-14T11:39:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe47a59b700 1 -- 172.21.15.161:0/1902992909 learned_addr learned my addr 172.21.15.161:0/1902992909 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.649+0000 7fe46b7fe700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe4600a12c0 0x7fe4600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe479d9a700 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c392eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe46b7fe700 1 -- 172.21.15.161:0/1144920382 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fe4700cdd40 con 0x7fe4740ffaf0 2023-07-14T11:39:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe47a59b700 1 -- 172.21.15.161:0/1902992909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 msgr2=0x55cd8c3856a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c3856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe47a59b700 1 -- 172.21.15.161:0/1902992909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 msgr2=0x55cd8c392eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c392eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe47a59b700 1 -- 172.21.15.161:0/1902992909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd8c105de0 con 0x55cd8c385be0 2023-07-14T11:39:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.650+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe4600a12c0 0x7fe4600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.651+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c390590 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe47009a7c0 tx=0x7fe470009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ed58c4a909d9c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.651+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe4600a12c0 0x7fe4600a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009ec0 tx=0x7fe46c009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.651+0000 7fe468ff9700 1 -- 172.21.15.161:0/1902992909 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470060930 con 0x55cd8c385be0 2023-07-14T11:39:07.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.651+0000 7fe468ff9700 1 -- 172.21.15.161:0/1902992909 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe47009aac0 con 0x55cd8c385be0 2023-07-14T11:39:07.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.651+0000 7fe468ff9700 1 -- 172.21.15.161:0/1902992909 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4700cb330 con 0x55cd8c385be0 2023-07-14T11:39:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.652+0000 7fe4901feb80 1 -- 172.21.15.161:0/1902992909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 msgr2=0x55cd8c390590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.652+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c390590 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe47009a7c0 tx=0x7fe470009260 comp rx=0 tx=0).stop 2023-07-14T11:39:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.652+0000 7fe4901feb80 1 -- 172.21.15.161:0/1902992909 shutdown_connections 2023-07-14T11:39:07.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.653+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c392eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.653+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c3856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.653+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1902992909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c390590 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.653+0000 7fe4901feb80 1 -- 172.21.15.161:0/1902992909 >> 172.21.15.161:0/1902992909 conn(0x55cd8c37bda0 msgr2=0x55cd8c3816a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.653+0000 7fe4901feb80 1 -- 172.21.15.161:0/1902992909 shutdown_connections 2023-07-14T11:39:07.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.653+0000 7fe4901feb80 1 -- 172.21.15.161:0/1902992909 wait complete. 2023-07-14T11:39:07.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.654+0000 7fe4901feb80 1 -- start start 2023-07-14T11:39:07.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.654+0000 7fe4901feb80 1 Processor -- start 2023-07-14T11:39:07.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c395b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c38fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe4901feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c277ac0 con 0x55cd8c390ad0 2023-07-14T11:39:07.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe4901feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c2777c0 con 0x55cd8c385be0 2023-07-14T11:39:07.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe4901feb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c15b920 con 0x55cd8c3852d0 2023-07-14T11:39:07.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe4901feb80 10 client.0 _ll_register_callbacks cb 0x55cd8c275760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:07.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe47a59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c395b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe47a59b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c395b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:46070/0 (socket says 172.21.15.161:46070) 2023-07-14T11:39:07.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.655+0000 7fe47a59b700 1 -- 172.21.15.161:0/1253605026 learned_addr learned my addr 172.21.15.161:0/1253605026 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c38fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe479d9a700 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe47a59b700 1 -- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 msgr2=0x55cd8c3960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe47a59b700 1 -- 172.21.15.161:0/1253605026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 msgr2=0x55cd8c38fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c38fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe47a59b700 1 -- 172.21.15.161:0/1253605026 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47419d3e0 con 0x55cd8c3852d0 2023-07-14T11:39:07.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe47a59b700 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c395b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe470005e90 tx=0x7fe4700cab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83297ffea6808d26 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe45dffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:07.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.656+0000 7fe45dffb700 1 -- 172.21.15.161:0/1253605026 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4700d1360 con 0x55cd8c3852d0 2023-07-14T11:39:07.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.657+0000 7fe45dffb700 1 -- 172.21.15.161:0/1253605026 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fe4700a2e80 con 0x55cd8c3852d0 2023-07-14T11:39:07.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.657+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd8c2314f0 con 0x55cd8c3852d0 2023-07-14T11:39:07.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.657+0000 7fe45dffb700 1 -- 172.21.15.161:0/1253605026 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4700b3200 con 0x55cd8c3852d0 2023-07-14T11:39:07.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.657+0000 7fe4901feb80 10 client.27595 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:07.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.657+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55cd8c0eec60 con 0x55cd8c3852d0 2023-07-14T11:39:07.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe45dffb700 1 -- 172.21.15.161:0/1253605026 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd8c2314f0 con 0x55cd8c3852d0 2023-07-14T11:39:07.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe45dffb700 1 -- 172.21.15.161:0/1253605026 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe47009ae30 con 0x55cd8c3852d0 2023-07-14T11:39:07.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe4901feb80 10 client.27595 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:07.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe4901feb80 10 client.27595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:07.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe4901feb80 20 client.27595 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:07.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c39fb80 0x55cd8c3a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe4901feb80 4 client.27595 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:39:07.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd8c2d0eb0 con 0x55cd8c39fb80 2023-07-14T11:39:07.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.658+0000 7fe479d9a700 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c39fb80 0x55cd8c3a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.659+0000 7fe479d9a700 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c39fb80 0x55cd8c3a1f80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8e3d2ee86e2533 server_cookie=fe496cc793e7a0c9 in_seq=0 out_seq=0 2023-07-14T11:39:07.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.659+0000 7fe45dffb700 10 client.27595 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:07.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe45dffb700 1 -- 172.21.15.161:0/1253605026 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd8c2d0eb0 con 0x55cd8c39fb80 2023-07-14T11:39:07.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe45dffb700 10 client.27595 handle_command_reply: tid=0 2023-07-14T11:39:07.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 client.27595 shutdown 2023-07-14T11:39:07.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 msgr2=0x55cd8c395b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c395b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe470005e90 tx=0x7fe4700cab00 comp rx=0 tx=0).stop 2023-07-14T11:39:07.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 shutdown_connections 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c39fb80 0x55cd8c3a1f80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c38fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c395b90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1253605026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe45dffb700 0 client.27595 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.660+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 >> 172.21.15.161:0/1253605026 conn(0x55cd8c37bda0 msgr2=0x55cd8c381290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.661+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 shutdown_connections 2023-07-14T11:39:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.661+0000 7fe4901feb80 1 -- 172.21.15.161:0/1253605026 wait complete. 2023-07-14T11:39:07.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.661+0000 7fe4901feb80 20 client.27595 trim_cache size 0 max 16384 2023-07-14T11:39:07.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.681+0000 7fe4901feb80 1 Processor -- start 2023-07-14T11:39:07.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.682+0000 7fe4901feb80 1 -- start start 2023-07-14T11:39:07.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.682+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c38a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.682+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c395740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.682+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c395c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.682+0000 7fe4901feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c38f000 con 0x55cd8c385be0 2023-07-14T11:39:07.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.682+0000 7fe4901feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c38f170 con 0x55cd8c390ad0 2023-07-14T11:39:07.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.682+0000 7fe4901feb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c390700 con 0x55cd8c3852d0 2023-07-14T11:39:07.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c395c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c395c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:37768/0 (socket says 172.21.15.161:37768) 2023-07-14T11:39:07.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 -- 172.21.15.161:0/1617428466 learned_addr learned my addr 172.21.15.161:0/1617428466 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 -- 172.21.15.161:0/1617428466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 msgr2=0x55cd8c38a1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:39:07.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/1617428466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c38a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 -- 172.21.15.161:0/1617428466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 msgr2=0x55cd8c395740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:39:07.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/1617428466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c395740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.683+0000 7fe47ad9c700 1 -- 172.21.15.161:0/1617428466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd8c12c250 con 0x55cd8c390ad0 2023-07-14T11:39:07.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.684+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/1617428466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c395c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe46c016500 tx=0x7fe46c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e902899da8c6f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.684+0000 7fe4697fa700 1 -- 172.21.15.161:0/1617428466 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c01b040 con 0x55cd8c390ad0 2023-07-14T11:39:07.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.684+0000 7fe4697fa700 1 -- 172.21.15.161:0/1617428466 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46c00f040 con 0x55cd8c390ad0 2023-07-14T11:39:07.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.685+0000 7fe4697fa700 1 -- 172.21.15.161:0/1617428466 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00e070 con 0x55cd8c390ad0 2023-07-14T11:39:07.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.686+0000 7fe4901feb80 1 -- 172.21.15.161:0/1617428466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 msgr2=0x55cd8c395c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.686+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1617428466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c395c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe46c016500 tx=0x7fe46c015000 comp rx=0 tx=0).stop 2023-07-14T11:39:07.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.688+0000 7fe4901feb80 1 -- 172.21.15.161:0/1617428466 shutdown_connections 2023-07-14T11:39:07.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.688+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1617428466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c390ad0 0x55cd8c395c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.688+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1617428466 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c38a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.688+0000 7fe4901feb80 1 --2- 172.21.15.161:0/1617428466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c385be0 0x55cd8c395740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.688+0000 7fe4901feb80 1 -- 172.21.15.161:0/1617428466 >> 172.21.15.161:0/1617428466 conn(0x55cd8c37bda0 msgr2=0x55cd8c3a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.688+0000 7fe4901feb80 1 -- 172.21.15.161:0/1617428466 shutdown_connections 2023-07-14T11:39:07.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.688+0000 7fe4901feb80 1 -- 172.21.15.161:0/1617428466 wait complete. 2023-07-14T11:39:07.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.689+0000 7fe4901feb80 1 -- start start 2023-07-14T11:39:07.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.689+0000 7fe4901feb80 1 Processor -- start 2023-07-14T11:39:07.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c3f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe4901feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c3f0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe4901feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c3f0ff0 con 0x55cd8c390ad0 2023-07-14T11:39:07.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe4901feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c40d240 con 0x55cd8c385be0 2023-07-14T11:39:07.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe4901feb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8c40d380 con 0x55cd8c3852d0 2023-07-14T11:39:07.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe4901feb80 10 client.0 _ll_register_callbacks cb 0x7fe47009a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:07.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe47ad9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c3f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe47ad9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c3f0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44456/0 (socket says 172.21.15.161:44456) 2023-07-14T11:39:07.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe47ad9c700 1 -- 172.21.15.161:0/2609085911 learned_addr learned my addr 172.21.15.161:0/2609085911 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:07.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe47a59b700 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c3f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.690+0000 7fe479d9a700 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe47ad9c700 1 -- 172.21.15.161:0/2609085911 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 msgr2=0x55cd8c3f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c3f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe47ad9c700 1 -- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 msgr2=0x55cd8c3f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3f01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe47ad9c700 1 -- 172.21.15.161:0/2609085911 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c008dd0 con 0x55cd8c390ad0 2023-07-14T11:39:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe47ad9c700 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c3f0910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0168c0 tx=0x7fe46c0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5d87988d70033f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe45dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T11:39:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.691+0000 7fe45dffb700 1 -- 172.21.15.161:0/2609085911 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c003e10 con 0x55cd8c390ad0 2023-07-14T11:39:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe45dffb700 1 -- 172.21.15.161:0/2609085911 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fe46c003f70 con 0x55cd8c390ad0 2023-07-14T11:39:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe47009aac0 con 0x55cd8c390ad0 2023-07-14T11:39:07.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe45dffb700 1 -- 172.21.15.161:0/2609085911 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe46c0048e0 con 0x55cd8c390ad0 2023-07-14T11:39:07.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 10 client.17541 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:07.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55cd8c1042b0 con 0x55cd8c390ad0 2023-07-14T11:39:07.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe45dffb700 1 -- 172.21.15.161:0/2609085911 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe47009aac0 con 0x55cd8c390ad0 2023-07-14T11:39:07.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe45dffb700 1 -- 172.21.15.161:0/2609085911 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe46c02b570 con 0x55cd8c390ad0 2023-07-14T11:39:07.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 10 client.17541 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:07.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:07.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 20 client.17541 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:07.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c40e8a0 0x55cd8c410c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:07.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 4 client.17541 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-14T11:39:07.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.692+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd8c293050 con 0x55cd8c40e8a0 2023-07-14T11:39:07.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.693+0000 7fe47a59b700 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c40e8a0 0x55cd8c410c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:07.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.694+0000 7fe47a59b700 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c40e8a0 0x55cd8c410c80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6431cfaef3ae449b server_cookie=f861c220d29cd174 in_seq=0 out_seq=0 2023-07-14T11:39:07.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.694+0000 7fe45dffb700 10 client.17541 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:07.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.699+0000 7fe45dffb700 1 -- 172.21.15.161:0/2609085911 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd8c293050 con 0x55cd8c40e8a0 2023-07-14T11:39:07.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.699+0000 7fe45dffb700 10 client.17541 handle_command_reply: tid=0 2023-07-14T11:39:07.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.699+0000 7fe4901feb80 1 client.17541 shutdown 2023-07-14T11:39:07.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 msgr2=0x55cd8c3f0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c3f0910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0168c0 tx=0x7fe46c0046f0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 shutdown_connections 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55cd8c40e8a0 0x55cd8c410c80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd8c390ad0 0x55cd8c3f0910 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55cd8c3852d0 0x55cd8c3f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 --2- 172.21.15.161:0/2609085911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd8c385be0 0x55cd8c3f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 >> 172.21.15.161:0/2609085911 conn(0x55cd8c37bda0 msgr2=0x55cd8c38dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe45dffb700 0 client.17541 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:07.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.700+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 shutdown_connections 2023-07-14T11:39:07.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.701+0000 7fe4901feb80 1 -- 172.21.15.161:0/2609085911 wait complete. 2023-07-14T11:39:07.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.701+0000 7fe4901feb80 20 client.17541 trim_cache size 0 max 16384 2023-07-14T11:39:07.698 INFO:teuthology.orchestra.run.smithi161.stdout:[] 2023-07-14T11:39:07.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe4600a12c0 msgr2=0x7fe4600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe4600a12c0 0x7fe4600a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009ec0 tx=0x7fe46c009920 comp rx=0 tx=0).stop 2023-07-14T11:39:07.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe4740ffaf0 msgr2=0x7fe474197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:07.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe4740ffaf0 0x7fe474197e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe470009810 tx=0x7fe47000fa70 comp rx=0 tx=0).stop 2023-07-14T11:39:07.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 shutdown_connections 2023-07-14T11:39:07.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe4600a12c0 0x7fe4600a3770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe474105b00 0x7fe47419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe4740ffaf0 0x7fe474197e50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 --2- 172.21.15.161:0/1144920382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4741004a0 0x7fe474198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:07.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.702+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 >> 172.21.15.161:0/1144920382 conn(0x7fe4740fb650 msgr2=0x7fe47410ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:07.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.703+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 shutdown_connections 2023-07-14T11:39:07.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:07.703+0000 7fe47c7ff700 1 -- 172.21.15.161:0/1144920382 wait complete. 2023-07-14T11:39:07.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:07 smithi182 ceph-mon[129366]: osdmap e114: 12 total, 12 up, 12 in 2023-07-14T11:39:07.721 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:07 smithi161 ceph-mon[95526]: osdmap e114: 12 total, 12 up, 12 in 2023-07-14T11:39:07.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:07 smithi175 ceph-mon[129081]: osdmap e114: 12 total, 12 up, 12 in 2023-07-14T11:39:08.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 131989 29.59 MB/sec execute 110 sec latency 39.615 ms 2023-07-14T11:39:08.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:08 smithi182 ceph-mon[129366]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 570 op/s 2023-07-14T11:39:08.725 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T11:39:08.726 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T11:39:08.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:08 smithi161 ceph-mon[95526]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 570 op/s 2023-07-14T11:39:08.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:08 smithi175 ceph-mon[129081]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 570 op/s 2023-07-14T11:39:08.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 -- 172.21.15.161:0/2440269205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4101610 msgr2=0x7fd7d4109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:08.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/2440269205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4101610 0x7fd7d4109b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009990 tx=0x7fd7c8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 -- 172.21.15.161:0/2440269205 shutdown_connections 2023-07-14T11:39:08.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/2440269205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d4100d00 0x7fd7d41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/2440269205 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7d410a040 0x7fd7d410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/2440269205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4101610 0x7fd7d4109b00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 -- 172.21.15.161:0/2440269205 >> 172.21.15.161:0/2440269205 conn(0x7fd7d40fa4b0 msgr2=0x7fd7d40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:08.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 -- 172.21.15.161:0/2440269205 shutdown_connections 2023-07-14T11:39:08.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.981+0000 7fd7d97ff700 1 -- 172.21.15.161:0/2440269205 wait complete. 2023-07-14T11:39:08.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.982+0000 7fd7d97ff700 1 Processor -- start 2023-07-14T11:39:08.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d97ff700 1 -- start start 2023-07-14T11:39:08.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d97ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4100d00 0x7fd7d4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d97ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d4101610 0x7fd7d4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d97ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7d410a040 0x7fd7d419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d97ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d419ce10 con 0x7fd7d4101610 2023-07-14T11:39:08.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d97ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d419cf80 con 0x7fd7d4100d00 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d97ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d419d140 con 0x7fd7d410a040 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4100d00 0x7fd7d4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7d410a040 0x7fd7d419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4100d00 0x7fd7d4197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44790/0 (socket says 172.21.15.161:44790) 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/1331067548 learned_addr learned my addr 172.21.15.161:0/1331067548 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.983+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d4101610 0x7fd7d4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d37fe700 1 -- 172.21.15.161:0/1331067548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4100d00 msgr2=0x7fd7d4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:08.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d37fe700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4100d00 0x7fd7d4197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d37fe700 1 -- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d4101610 msgr2=0x7fd7d4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:08.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d37fe700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d4101610 0x7fd7d4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d37fe700 1 -- 172.21.15.161:0/1331067548 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80095f0 con 0x7fd7d410a040 2023-07-14T11:39:08.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d4101610 0x7fd7d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:08.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4100d00 0x7fd7d4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:08.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d37fe700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7d410a040 0x7fd7d419c7d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00efc0 tx=0x7fd7cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:08.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7bbfff700 1 -- 172.21.15.161:0/1331067548 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc018070 con 0x7fd7d410a040 2023-07-14T11:39:08.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d419d3c0 con 0x7fd7d410a040 2023-07-14T11:39:08.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7bbfff700 1 -- 172.21.15.161:0/1331067548 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc00f040 con 0x7fd7d410a040 2023-07-14T11:39:08.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7bbfff700 1 -- 172.21.15.161:0/1331067548 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc00cd40 con 0x7fd7d410a040 2023-07-14T11:39:08.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.984+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d41a4c60 con 0x7fd7d410a040 2023-07-14T11:39:08.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.988+0000 7fd7bbfff700 1 -- 172.21.15.161:0/1331067548 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd7cc003a00 con 0x7fd7d410a040 2023-07-14T11:39:08.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.989+0000 7fd7ed0cab80 1 Processor -- start 2023-07-14T11:39:08.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.989+0000 7fd7ed0cab80 1 -- start start 2023-07-14T11:39:08.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da610720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da61b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da61df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da516d30 con 0x5601da610c60 2023-07-14T11:39:08.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da569ac0 con 0x5601da610350 2023-07-14T11:39:08.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da562f50 con 0x5601da61bb50 2023-07-14T11:39:08.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da610720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da610720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44798/0 (socket says 172.21.15.161:44798) 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 -- 172.21.15.161:0/412562801 learned_addr learned my addr 172.21.15.161:0/412562801 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da61df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da61b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 -- 172.21.15.161:0/412562801 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 msgr2=0x5601da61df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da61df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 -- 172.21.15.161:0/412562801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 msgr2=0x5601da61b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da61b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.990+0000 7fd7d27fc700 1 -- 172.21.15.161:0/412562801 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601da390e60 con 0x5601da610350 2023-07-14T11:39:08.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da61b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:39:08.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7bbfff700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7bc0a1340 0x7fd7bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da610720 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009990 tx=0x7fd7c801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a90d6d50be56fa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:08.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7bbfff700 1 -- 172.21.15.161:0/1331067548 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fd7cc0ccbd0 con 0x7fd7d410a040 2023-07-14T11:39:08.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7bc0a1340 0x7fd7bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7b97fa700 1 -- 172.21.15.161:0/412562801 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8005050 con 0x5601da610350 2023-07-14T11:39:08.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7b97fa700 1 -- 172.21.15.161:0/412562801 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c80051b0 con 0x5601da610350 2023-07-14T11:39:08.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.991+0000 7fd7b97fa700 1 -- 172.21.15.161:0/412562801 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80296c0 con 0x5601da610350 2023-07-14T11:39:08.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/412562801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 msgr2=0x5601da610720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:08.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da610720 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009990 tx=0x7fd7c801a040 comp rx=0 tx=0).stop 2023-07-14T11:39:08.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7bc0a1340 0x7fd7bc0a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009fd0 tx=0x7fd7c4009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:08.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/412562801 shutdown_connections 2023-07-14T11:39:08.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da61df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da610720 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/412562801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da61b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/412562801 >> 172.21.15.161:0/412562801 conn(0x5601da606e20 msgr2=0x5601da60c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:08.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/412562801 shutdown_connections 2023-07-14T11:39:08.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.992+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/412562801 wait complete. 2023-07-14T11:39:08.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.993+0000 7fd7ed0cab80 1 -- start start 2023-07-14T11:39:08.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.993+0000 7fd7ed0cab80 1 Processor -- start 2023-07-14T11:39:08.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da6188f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da618e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da619370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da502b40 con 0x5601da610c60 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da502840 con 0x5601da610350 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da3e69a0 con 0x5601da61bb50 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7ed0cab80 10 client.0 _ll_register_callbacks cb 0x5601da5007e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da6188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da619370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7d27fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da618e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da6188f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44802/0 (socket says 172.21.15.161:44802) 2023-07-14T11:39:08.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/2904302396 learned_addr learned my addr 172.21.15.161:0/2904302396 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:08.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7d37fe700 1 -- 172.21.15.161:0/2904302396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 msgr2=0x5601da6188f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:08.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.994+0000 7fd7d37fe700 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da6188f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7d37fe700 1 -- 172.21.15.161:0/2904302396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 msgr2=0x5601da618e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:08.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7d37fe700 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da618e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:08.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7d37fe700 1 -- 172.21.15.161:0/2904302396 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c8020040 con 0x5601da61bb50 2023-07-14T11:39:08.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:08.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7d37fe700 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da619370 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0c9ee0 tx=0x7fd7cc099430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e134101cc0f882 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:08.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/2904302396 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0998f0 con 0x5601da61bb50 2023-07-14T11:39:08.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/2904302396 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fd7cc09d920 con 0x5601da61bb50 2023-07-14T11:39:08.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601da4bc570 con 0x5601da61bb50 2023-07-14T11:39:08.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/2904302396 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0a1cb0 con 0x5601da61bb50 2023-07-14T11:39:08.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7ed0cab80 10 client.27610 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:08.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.995+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5601da379ce0 con 0x5601da61bb50 2023-07-14T11:39:08.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/2904302396 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601da4bc570 con 0x5601da61bb50 2023-07-14T11:39:08.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/2904302396 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd7cc0b1f00 con 0x5601da61bb50 2023-07-14T11:39:08.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7ed0cab80 10 client.27610 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:08.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7ed0cab80 10 client.27610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:08.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7ed0cab80 20 client.27610 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:08.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da62b3b0 0x5601da62d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:08.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7ed0cab80 4 client.27610 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:39:08.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.996+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601da55bf30 con 0x5601da62b3b0 2023-07-14T11:39:08.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.997+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da62b3b0 0x5601da62d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:08.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.997+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da62b3b0 0x5601da62d7c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4623834472a976e server_cookie=6e82ad695084d699 in_seq=0 out_seq=0 2023-07-14T11:39:08.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:08.997+0000 7fd7b6ff5700 10 client.27610 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:08.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/2904302396 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5601da55bf30 con 0x5601da62b3b0 2023-07-14T11:39:08.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7b6ff5700 10 client.27610 handle_command_reply: tid=0 2023-07-14T11:39:09.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 client.27610 shutdown 2023-07-14T11:39:09.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 msgr2=0x5601da619370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da619370 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0c9ee0 tx=0x7fd7cc099430 comp rx=0 tx=0).stop 2023-07-14T11:39:09.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 shutdown_connections 2023-07-14T11:39:09.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da62b3b0 0x5601da62d7c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da619370 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da6188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7b6ff5700 0 client.27610 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:09.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2904302396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da618e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 >> 172.21.15.161:0/2904302396 conn(0x5601da606e20 msgr2=0x5601da60c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 shutdown_connections 2023-07-14T11:39:09.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.000+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2904302396 wait complete. 2023-07-14T11:39:09.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.001+0000 7fd7ed0cab80 20 client.27610 trim_cache size 0 max 16384 2023-07-14T11:39:09.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.022+0000 7fd7ed0cab80 1 Processor -- start 2023-07-14T11:39:09.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.022+0000 7fd7ed0cab80 1 -- start start 2023-07-14T11:39:09.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da615b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da6160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da612220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da4bc5a0 con 0x5601da610c60 2023-07-14T11:39:09.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da608f70 con 0x5601da610350 2023-07-14T11:39:09.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da6111e0 con 0x5601da61bb50 2023-07-14T11:39:09.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da615b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da615b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44818/0 (socket says 172.21.15.161:44818) 2023-07-14T11:39:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/2941469430 learned_addr learned my addr 172.21.15.161:0/2941469430 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da6160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/2941469430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 msgr2=0x5601da612220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da612220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/2941469430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 msgr2=0x5601da6160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da6160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.023+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/2941469430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601da618460 con 0x5601da610350 2023-07-14T11:39:09.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.024+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da615b80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x5601da62e0a0 tx=0x7fd7c4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e7f5db1ddf7cc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.024+0000 7fd7b9ffb700 1 -- 172.21.15.161:0/2941469430 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40322b0 con 0x5601da610350 2023-07-14T11:39:09.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.024+0000 7fd7b9ffb700 1 -- 172.21.15.161:0/2941469430 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c4028040 con 0x5601da610350 2023-07-14T11:39:09.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.024+0000 7fd7b9ffb700 1 -- 172.21.15.161:0/2941469430 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c403b6d0 con 0x5601da610350 2023-07-14T11:39:09.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2941469430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 msgr2=0x5601da615b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da615b80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x5601da62e0a0 tx=0x7fd7c4016000 comp rx=0 tx=0).stop 2023-07-14T11:39:09.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2941469430 shutdown_connections 2023-07-14T11:39:09.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da61bb50 0x5601da612220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610350 0x5601da615b80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/2941469430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da610c60 0x5601da6160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2941469430 >> 172.21.15.161:0/2941469430 conn(0x5601da606e20 msgr2=0x5601da617930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2941469430 shutdown_connections 2023-07-14T11:39:09.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.025+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/2941469430 wait complete. 2023-07-14T11:39:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.026+0000 7fd7ed0cab80 1 -- start start 2023-07-14T11:39:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.026+0000 7fd7ed0cab80 1 Processor -- start 2023-07-14T11:39:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da610350 0x5601da6799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610c60 0x5601da679ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7ed0cab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da61bb50 0x5601da6217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da621df0 con 0x5601da61bb50 2023-07-14T11:39:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da621f60 con 0x5601da610c60 2023-07-14T11:39:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7ed0cab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601da622120 con 0x5601da610350 2023-07-14T11:39:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7ed0cab80 10 client.0 _ll_register_callbacks cb 0x7fd7cc095c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da610350 0x5601da6799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610c60 0x5601da679ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da610350 0x5601da6799b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33342/0 (socket says 172.21.15.161:33342) 2023-07-14T11:39:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/151037240 learned_addr learned my addr 172.21.15.161:0/151037240 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:09.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d37fe700 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da61bb50 0x5601da6217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/151037240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610c60 msgr2=0x5601da679ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610c60 0x5601da679ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/151037240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da61bb50 msgr2=0x5601da6217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da61bb50 0x5601da6217e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.027+0000 7fd7d2ffd700 1 -- 172.21.15.161:0/151037240 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c4019070 con 0x5601da610350 2023-07-14T11:39:09.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.028+0000 7fd7d2ffd700 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da610350 0x5601da6799b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4005e90 tx=0x7fd7c4032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af620371da1c1cb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.028+0000 7fd7b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.028+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/151037240 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4011950 con 0x5601da610350 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.028+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7c80051b0 con 0x5601da610350 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.028+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/151037240 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fd7c4011ab0 con 0x5601da610350 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.029+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/151037240 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c4011c70 con 0x5601da610350 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.029+0000 7fd7ed0cab80 10 client.27613 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.029+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5601da3b72d0 con 0x5601da610350 2023-07-14T11:39:09.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.029+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/151037240 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c80051b0 con 0x5601da610350 2023-07-14T11:39:09.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.029+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/151037240 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd7c4041020 con 0x5601da610350 2023-07-14T11:39:09.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.029+0000 7fd7ed0cab80 10 client.27613 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:09.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.029+0000 7fd7ed0cab80 10 client.27613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:09.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.030+0000 7fd7ed0cab80 20 client.27613 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:09.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.030+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da681810 0x5601da683c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.030+0000 7fd7ed0cab80 4 client.27613 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T11:39:09.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.030+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5601da51e0d0 con 0x5601da681810 2023-07-14T11:39:09.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.030+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da681810 0x5601da683c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.031+0000 7fd7d27fc700 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da681810 0x5601da683c20 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0d7daa59385607 server_cookie=c4fe26a5ff41788a in_seq=0 out_seq=0 2023-07-14T11:39:09.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.031+0000 7fd7b6ff5700 10 client.27613 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:09.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7b6ff5700 1 -- 172.21.15.161:0/151037240 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5601da51e0d0 con 0x5601da681810 2023-07-14T11:39:09.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7b6ff5700 10 client.27613 handle_command_reply: tid=0 2023-07-14T11:39:09.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 client.27613 shutdown 2023-07-14T11:39:09.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da610350 msgr2=0x5601da6799b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da610350 0x5601da6799b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4005e90 tx=0x7fd7c4032e90 comp rx=0 tx=0).stop 2023-07-14T11:39:09.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 shutdown_connections 2023-07-14T11:39:09.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5601da681810 0x5601da683c20 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601da61bb50 0x5601da6217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5601da610350 0x5601da6799b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 --2- 172.21.15.161:0/151037240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5601da610c60 0x5601da679ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7b6ff5700 0 client.27613 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:09.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.039+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 >> 172.21.15.161:0/151037240 conn(0x5601da606e20 msgr2=0x5601da618840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.040+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 shutdown_connections 2023-07-14T11:39:09.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.040+0000 7fd7ed0cab80 1 -- 172.21.15.161:0/151037240 wait complete. 2023-07-14T11:39:09.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.040+0000 7fd7ed0cab80 20 client.27613 trim_cache size 0 max 16384 2023-07-14T11:39:09.041 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:39:09.042 INFO:teuthology.orchestra.run.smithi161.stdout: "return_code": 0, 2023-07-14T11:39:09.042 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:39:09.042 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "asynchronous" 2023-07-14T11:39:09.042 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:39:09.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7bc0a1340 msgr2=0x7fd7bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7bc0a1340 0x7fd7bc0a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009fd0 tx=0x7fd7c4009260 comp rx=0 tx=0).stop 2023-07-14T11:39:09.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7d410a040 msgr2=0x7fd7d419c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7d410a040 0x7fd7d419c7d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00efc0 tx=0x7fd7cc00c3a0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 shutdown_connections 2023-07-14T11:39:09.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7bc0a1340 0x7fd7bc0a37f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7d410a040 0x7fd7d419c7d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d4100d00 0x7fd7d4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 --2- 172.21.15.161:0/1331067548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d4101610 0x7fd7d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 >> 172.21.15.161:0/1331067548 conn(0x7fd7d40fa4b0 msgr2=0x7fd7d410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 shutdown_connections 2023-07-14T11:39:09.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.041+0000 7fd7d97ff700 1 -- 172.21.15.161:0/1331067548 wait complete. 2023-07-14T11:39:09.057 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:39:09.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.343+0000 7f60d8bff700 1 -- 172.21.15.161:0/1241871973 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4101f70 msgr2=0x7f60d4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.343+0000 7f60d8bff700 1 --2- 172.21.15.161:0/1241871973 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4101f70 0x7f60d4102340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f60c40099c0 tx=0x7f60c4009cd0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.344+0000 7f60d8bff700 1 -- 172.21.15.161:0/1241871973 shutdown_connections 2023-07-14T11:39:09.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.344+0000 7f60d8bff700 1 --2- 172.21.15.161:0/1241871973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4106fb0 0x7f60d410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.344+0000 7f60d8bff700 1 --2- 172.21.15.161:0/1241871973 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4101f70 0x7f60d4102340 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.344+0000 7f60d8bff700 1 --2- 172.21.15.161:0/1241871973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d4102910 0x7f60d41068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.344+0000 7f60d8bff700 1 -- 172.21.15.161:0/1241871973 >> 172.21.15.161:0/1241871973 conn(0x7f60d40fd7e0 msgr2=0x7f60d40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.344+0000 7f60d8bff700 1 -- 172.21.15.161:0/1241871973 shutdown_connections 2023-07-14T11:39:09.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.344+0000 7f60d8bff700 1 -- 172.21.15.161:0/1241871973 wait complete. 2023-07-14T11:39:09.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 Processor -- start 2023-07-14T11:39:09.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 -- start start 2023-07-14T11:39:09.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4101f70 0x7f60d4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4102910 0x7f60d4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d4106fb0 0x7f60d419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d419ce10 con 0x7f60d4106fb0 2023-07-14T11:39:09.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d419cf80 con 0x7f60d4101f70 2023-07-14T11:39:09.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d8bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d419d140 con 0x7f60d4102910 2023-07-14T11:39:09.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.345+0000 7f60d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4101f70 0x7f60d4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d4106fb0 0x7f60d419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4101f70 0x7f60d4197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44848/0 (socket says 172.21.15.161:44848) 2023-07-14T11:39:09.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 -- 172.21.15.161:0/4234812128 learned_addr learned my addr 172.21.15.161:0/4234812128 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:09.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4102910 0x7f60d4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 -- 172.21.15.161:0/4234812128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4102910 msgr2=0x7f60d4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4102910 0x7f60d4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 -- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d4106fb0 msgr2=0x7f60d419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d4106fb0 0x7f60d419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 -- 172.21.15.161:0/4234812128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40095f0 con 0x7f60d4101f70 2023-07-14T11:39:09.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4102910 0x7f60d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:09.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.346+0000 7f60d259c700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4101f70 0x7f60d4197e30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f60c40098e0 tx=0x7f60c400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.347+0000 7f60c37fe700 1 -- 172.21.15.161:0/4234812128 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c401d070 con 0x7f60d4101f70 2023-07-14T11:39:09.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.347+0000 7f60c37fe700 1 -- 172.21.15.161:0/4234812128 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c4022470 con 0x7f60d4101f70 2023-07-14T11:39:09.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.347+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d419d3c0 con 0x7f60d4101f70 2023-07-14T11:39:09.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.347+0000 7f60c37fe700 1 -- 172.21.15.161:0/4234812128 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4017650 con 0x7f60d4101f70 2023-07-14T11:39:09.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.347+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d41a4c60 con 0x7f60d4101f70 2023-07-14T11:39:09.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.352+0000 7f60c37fe700 1 -- 172.21.15.161:0/4234812128 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f60c4022850 con 0x7f60d4101f70 2023-07-14T11:39:09.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.353+0000 7f60ec4ecb80 1 Processor -- start 2023-07-14T11:39:09.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.353+0000 7f60ec4ecb80 1 -- start start 2023-07-14T11:39:09.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e56c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e577590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e579eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e472cb0 con 0x55b35e56cbe0 2023-07-14T11:39:09.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e4c5a40 con 0x55b35e577ad0 2023-07-14T11:39:09.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e4beed0 con 0x55b35e56c2d0 2023-07-14T11:39:09.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60d259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e577590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.354+0000 7f60c37fe700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60bc0a12c0 0x7f60bc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.355+0000 7f60d259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e577590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56266/0 (socket says 172.21.15.161:56266) 2023-07-14T11:39:09.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.355+0000 7f60d259c700 1 -- 172.21.15.161:0/974577936 learned_addr learned my addr 172.21.15.161:0/974577936 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:09.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.355+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e56c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.355+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e579eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.355+0000 7f60c37fe700 1 -- 172.21.15.161:0/4234812128 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f60c40cdbd0 con 0x7f60d4101f70 2023-07-14T11:39:09.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.355+0000 7f60d259c700 1 -- 172.21.15.161:0/974577936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 msgr2=0x55b35e56c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60d259c700 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e56c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60d259c700 1 -- 172.21.15.161:0/974577936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 msgr2=0x55b35e579eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60d259c700 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e579eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60d259c700 1 -- 172.21.15.161:0/974577936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b35e2ecde0 con 0x55b35e56cbe0 2023-07-14T11:39:09.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60bc0a12c0 0x7f60bc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60d259c700 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e577590 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60c40cb100 tx=0x7f60c4022bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5eca078acaf9a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60c0ff9700 1 -- 172.21.15.161:0/974577936 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c409a3c0 con 0x55b35e56cbe0 2023-07-14T11:39:09.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60c0ff9700 1 -- 172.21.15.161:0/974577936 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c409aa60 con 0x55b35e56cbe0 2023-07-14T11:39:09.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.356+0000 7f60c0ff9700 1 -- 172.21.15.161:0/974577936 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4099f50 con 0x55b35e56cbe0 2023-07-14T11:39:09.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/974577936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 msgr2=0x55b35e577590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e577590 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60c40cb100 tx=0x7f60c4022bf0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60bc0a12c0 0x7f60bc0a3770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009ec0 tx=0x7f60cc004ad0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/974577936 shutdown_connections 2023-07-14T11:39:09.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e579eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e56c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/974577936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e577590 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.357+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/974577936 >> 172.21.15.161:0/974577936 conn(0x55b35e562da0 msgr2=0x55b35e5686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.358+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/974577936 shutdown_connections 2023-07-14T11:39:09.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.358+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/974577936 wait complete. 2023-07-14T11:39:09.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.358+0000 7f60ec4ecb80 1 -- start start 2023-07-14T11:39:09.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.359+0000 7f60ec4ecb80 1 Processor -- start 2023-07-14T11:39:09.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.359+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e577ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.359+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.359+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e578920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.359+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e45eac0 con 0x55b35e56cbe0 2023-07-14T11:39:09.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.359+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e45e7c0 con 0x55b35e577ad0 2023-07-14T11:39:09.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e342920 con 0x55b35e56c2d0 2023-07-14T11:39:09.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60ec4ecb80 10 client.0 _ll_register_callbacks cb 0x55b35e45c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:09.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e577ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e577ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:33374/0 (socket says 172.21.15.161:33374) 2023-07-14T11:39:09.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 -- 172.21.15.161:0/3821226503 learned_addr learned my addr 172.21.15.161:0/3821226503 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:09.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e578920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 -- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 msgr2=0x55b35e578920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e578920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 -- 172.21.15.161:0/3821226503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 msgr2=0x55b35e5783e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5783e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 -- 172.21.15.161:0/3821226503 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d41a4c60 con 0x55b35e56c2d0 2023-07-14T11:39:09.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e578920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:39:09.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60d259c700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e577ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c010 tx=0x7f60c40966e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde68d83df124423 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.360+0000 7f60b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:09.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60b9ffb700 1 -- 172.21.15.161:0/3821226503 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c409a150 con 0x55b35e56c2d0 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60b9ffb700 1 -- 172.21.15.161:0/3821226503 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f60c40a2e20 con 0x55b35e56c2d0 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b35e4184f0 con 0x55b35e56c2d0 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60b9ffb700 1 -- 172.21.15.161:0/3821226503 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60c409ea90 con 0x55b35e56c2d0 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60ec4ecb80 10 client.27625 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b35e2d5c60 con 0x55b35e56c2d0 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60b9ffb700 1 -- 172.21.15.161:0/3821226503 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c40b2070 con 0x55b35e56c2d0 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60b9ffb700 1 -- 172.21.15.161:0/3821226503 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f60c400fad0 con 0x55b35e56c2d0 2023-07-14T11:39:09.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60ec4ecb80 10 client.27625 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:09.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.361+0000 7f60ec4ecb80 10 client.27625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:09.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.362+0000 7f60ec4ecb80 20 client.27625 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:09.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.362+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e586c40 0x55b35e589050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.362+0000 7f60ec4ecb80 4 client.27625 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:39:09.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.362+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b35e4b7eb0 con 0x55b35e586c40 2023-07-14T11:39:09.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.362+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e586c40 0x55b35e589050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.362+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e586c40 0x55b35e589050 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0698d8d820a1cc server_cookie=967ff54e7b867a1c in_seq=0 out_seq=0 2023-07-14T11:39:09.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.362+0000 7f60b9ffb700 10 client.27625 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:09.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.364+0000 7f60b9ffb700 1 -- 172.21.15.161:0/3821226503 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b35e4b7eb0 con 0x55b35e586c40 2023-07-14T11:39:09.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.364+0000 7f60b9ffb700 10 client.27625 handle_command_reply: tid=0 2023-07-14T11:39:09.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.364+0000 7f60ec4ecb80 1 client.27625 shutdown 2023-07-14T11:39:09.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 msgr2=0x55b35e577ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e577ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c010 tx=0x7f60c40966e0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 shutdown_connections 2023-07-14T11:39:09.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e586c40 0x55b35e589050 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e577ad0 0x55b35e578920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e56c2d0 0x55b35e577ea0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3821226503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5783e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 >> 172.21.15.161:0/3821226503 conn(0x55b35e562da0 msgr2=0x55b35e568290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60b9ffb700 0 client.27625 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:09.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 shutdown_connections 2023-07-14T11:39:09.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3821226503 wait complete. 2023-07-14T11:39:09.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.365+0000 7f60ec4ecb80 20 client.27625 trim_cache size 0 max 16384 2023-07-14T11:39:09.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.386+0000 7f60ec4ecb80 1 Processor -- start 2023-07-14T11:39:09.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60ec4ecb80 1 -- start start 2023-07-14T11:39:09.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e57bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e571400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e571e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e56d540 con 0x55b35e56cbe0 2023-07-14T11:39:09.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e562970 con 0x55b35e56c2d0 2023-07-14T11:39:09.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e5694d0 con 0x55b35e577ad0 2023-07-14T11:39:09.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e57bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60d2d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e571e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60d1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e571400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e57bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44876/0 (socket says 172.21.15.161:44876) 2023-07-14T11:39:09.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.387+0000 7f60d259c700 1 -- 172.21.15.161:0/3629844039 learned_addr learned my addr 172.21.15.161:0/3629844039 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:09.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60d2d9d700 1 -- 172.21.15.161:0/3629844039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 msgr2=0x55b35e57bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e57bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60d2d9d700 1 -- 172.21.15.161:0/3629844039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 msgr2=0x55b35e571400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e571400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60d2d9d700 1 -- 172.21.15.161:0/3629844039 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b35e5645e0 con 0x55b35e577ad0 2023-07-14T11:39:09.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60d259c700 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e57bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e571e80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f60cc017680 tx=0x7f60cc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8bc02eedbef0ba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60c17fa700 1 -- 172.21.15.161:0/3629844039 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00e040 con 0x55b35e577ad0 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60c17fa700 1 -- 172.21.15.161:0/3629844039 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60cc00f040 con 0x55b35e577ad0 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.388+0000 7f60c17fa700 1 -- 172.21.15.161:0/3629844039 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0169d0 con 0x55b35e577ad0 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3629844039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 msgr2=0x55b35e571e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e571e80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f60cc017680 tx=0x7f60cc017000 comp rx=0 tx=0).stop 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3629844039 shutdown_connections 2023-07-14T11:39:09.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e571e80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e57bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/3629844039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e571400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3629844039 >> 172.21.15.161:0/3629844039 conn(0x55b35e562da0 msgr2=0x55b35e582680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3629844039 shutdown_connections 2023-07-14T11:39:09.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.389+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/3629844039 wait complete. 2023-07-14T11:39:09.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.390+0000 7f60ec4ecb80 1 -- start start 2023-07-14T11:39:09.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.390+0000 7f60ec4ecb80 1 Processor -- start 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e5d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5d4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60ec4ecb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e5ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e5cf090 con 0x55b35e56cbe0 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e5cf200 con 0x55b35e56c2d0 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60ec4ecb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35e5cf3c0 con 0x55b35e577ad0 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60ec4ecb80 10 client.0 _ll_register_callbacks cb 0x7f60cc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:09.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e5d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e5d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44890/0 (socket says 172.21.15.161:44890) 2023-07-14T11:39:09.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 -- 172.21.15.161:0/637284073 learned_addr learned my addr 172.21.15.161:0/637284073 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:09.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d2d9d700 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e5ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 -- 172.21.15.161:0/637284073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 msgr2=0x55b35e5ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e5ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 -- 172.21.15.161:0/637284073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 msgr2=0x55b35e5d4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:09.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5d4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.391+0000 7f60d259c700 1 -- 172.21.15.161:0/637284073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b35e2d5c60 con 0x55b35e56c2d0 2023-07-14T11:39:09.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60d259c700 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e5d43d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f60c40974d0 tx=0x7f60c4096e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506ab37c58b1f093 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:09.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:39:09.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60b9ffb700 1 -- 172.21.15.161:0/637284073 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c409aec0 con 0x55b35e56c2d0 2023-07-14T11:39:09.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60b9ffb700 1 -- 172.21.15.161:0/637284073 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f60c40a2e20 con 0x55b35e56c2d0 2023-07-14T11:39:09.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60c409aa60 con 0x55b35e56c2d0 2023-07-14T11:39:09.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60b9ffb700 1 -- 172.21.15.161:0/637284073 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60c409a3c0 con 0x55b35e56c2d0 2023-07-14T11:39:09.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60ec4ecb80 10 client.27596 fetch_fsmap learned FSMap version 41 2023-07-14T11:39:09.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.392+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f60cc007eb0 con 0x55b35e56c2d0 2023-07-14T11:39:09.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60b9ffb700 1 -- 172.21.15.161:0/637284073 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c409aa60 con 0x55b35e56c2d0 2023-07-14T11:39:09.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60b9ffb700 1 -- 172.21.15.161:0/637284073 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f60c4099ad0 con 0x55b35e56c2d0 2023-07-14T11:39:09.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60ec4ecb80 10 client.27596 fetch_fsmap finished waiting for FSMap version 41 2023-07-14T11:39:09.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60ec4ecb80 10 client.27596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:09.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60ec4ecb80 20 client.27596 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:09.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e5dbe00 0x55b35e5de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:09.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60ec4ecb80 4 client.27596 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:39:09.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.393+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b35e47a050 con 0x55b35e5dbe00 2023-07-14T11:39:09.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.394+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e5dbe00 0x55b35e5de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:09.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.394+0000 7f60d1d9b700 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e5dbe00 0x55b35e5de210 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fedf467e6da0f3a3 server_cookie=da9ff8a58e7e0115 in_seq=0 out_seq=0 2023-07-14T11:39:09.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.394+0000 7f60b9ffb700 10 client.27596 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:09.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60b9ffb700 1 -- 172.21.15.161:0/637284073 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55b35e47a050 con 0x55b35e5dbe00 2023-07-14T11:39:09.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60b9ffb700 10 client.27596 handle_command_reply: tid=0 2023-07-14T11:39:09.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 client.27596 shutdown 2023-07-14T11:39:09.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 msgr2=0x55b35e5d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e5d43d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f60c40974d0 tx=0x7f60c4096e50 comp rx=0 tx=0).stop 2023-07-14T11:39:09.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 shutdown_connections 2023-07-14T11:39:09.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b35e5dbe00 0x55b35e5de210 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b35e577ad0 0x55b35e5ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b35e56c2d0 0x55b35e5d43d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60b9ffb700 0 client.27596 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:09.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 --2- 172.21.15.161:0/637284073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b35e56cbe0 0x55b35e5d4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.402+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 >> 172.21.15.161:0/637284073 conn(0x55b35e562da0 msgr2=0x55b35e571e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.403+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 shutdown_connections 2023-07-14T11:39:09.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.403+0000 7f60ec4ecb80 1 -- 172.21.15.161:0/637284073 wait complete. 2023-07-14T11:39:09.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.403+0000 7f60ec4ecb80 20 client.27596 trim_cache size 0 max 16384 2023-07-14T11:39:09.403 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:39:09.404 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (143 inodes in the stack)", 2023-07-14T11:39:09.404 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:39:09.404 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:39:09.404 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:39:09.404 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:39:09.404 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:39:09.404 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:39:09.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:39:09.405 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:39:09.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60bc0a12c0 msgr2=0x7f60bc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60bc0a12c0 0x7f60bc0a3770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009ec0 tx=0x7f60cc004ad0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4101f70 msgr2=0x7f60d4197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:09.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4101f70 0x7f60d4197e30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f60c40098e0 tx=0x7f60c400f690 comp rx=0 tx=0).stop 2023-07-14T11:39:09.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 shutdown_connections 2023-07-14T11:39:09.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60bc0a12c0 0x7f60bc0a3770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d4106fb0 0x7f60d419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d4101f70 0x7f60d4197e30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 --2- 172.21.15.161:0/4234812128 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60d4102910 0x7f60d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:09.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 >> 172.21.15.161:0/4234812128 conn(0x7f60d40fd7e0 msgr2=0x7f60d4105910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:09.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 shutdown_connections 2023-07-14T11:39:09.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:09.404+0000 7f60d8bff700 1 -- 172.21.15.161:0/4234812128 wait complete. 2023-07-14T11:39:09.420 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:39:09.620 INFO:tasks.workunit.client.0.smithi161.stdout: 10 132532 29.53 MB/sec execute 111 sec latency 67.415 ms 2023-07-14T11:39:09.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:09 smithi182 ceph-mon[129366]: scrub queued for path: / 2023-07-14T11:39:09.708 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:09 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:09.708 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:09 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:39:09.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:09 smithi161 ceph-mon[95526]: scrub queued for path: / 2023-07-14T11:39:09.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:09 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:09.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:09 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:39:09.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:09 smithi175 ceph-mon[129081]: scrub queued for path: / 2023-07-14T11:39:09.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:09 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:09.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:09 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:39:10.623 INFO:tasks.workunit.client.0.smithi161.stdout: 10 133052 29.51 MB/sec execute 112 sec latency 105.183 ms 2023-07-14T11:39:10.710 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:10 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:39:10.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:10 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:39:10.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:10 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:10 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:39:10.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:10 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:10 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.711 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:10 smithi182 ceph-mon[129366]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 525 op/s 2023-07-14T11:39:10.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:10 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:39:10.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:10 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:39:10.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:10 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:10 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:39:10.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:10 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:10 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:10 smithi161 ceph-mon[95526]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 525 op/s 2023-07-14T11:39:10.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:10 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:39:10.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:10 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:39:10.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:10 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:10 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:39:10.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:10 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:10 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:39:10.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:10 smithi175 ceph-mon[129081]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 525 op/s 2023-07-14T11:39:11.623 INFO:tasks.workunit.client.0.smithi161.stdout: 10 133569 29.50 MB/sec execute 113 sec latency 128.955 ms 2023-07-14T11:39:12.623 INFO:tasks.workunit.client.0.smithi161.stdout: 10 134069 29.51 MB/sec execute 114 sec latency 135.422 ms 2023-07-14T11:39:12.744 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:12 smithi161 ceph-mon[95526]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2023-07-14T11:39:12.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:12 smithi175 ceph-mon[129081]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2023-07-14T11:39:12.961 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:12 smithi182 ceph-mon[129366]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2023-07-14T11:39:13.629 INFO:tasks.workunit.client.0.smithi161.stdout: 10 134103 29.26 MB/sec execute 115 sec latency 1141.129 ms 2023-07-14T11:39:14.420 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:14 smithi175 ceph-mon[129081]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 495 op/s 2023-07-14T11:39:14.457 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:14 smithi182 ceph-mon[129366]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 495 op/s 2023-07-14T11:39:14.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:14 smithi161 ceph-mon[95526]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 495 op/s 2023-07-14T11:39:14.629 INFO:tasks.workunit.client.0.smithi161.stdout: 10 134208 29.03 MB/sec execute 116 sec latency 2141.195 ms 2023-07-14T11:39:15.630 INFO:tasks.workunit.client.0.smithi161.stdout: 10 134415 28.88 MB/sec execute 117 sec latency 3142.197 ms 2023-07-14T11:39:16.630 INFO:tasks.workunit.client.0.smithi161.stdout: 10 134950 28.83 MB/sec execute 118 sec latency 4142.254 ms 2023-07-14T11:39:16.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:16 smithi182 ceph-mon[129366]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 495 op/s 2023-07-14T11:39:16.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:16 smithi161 ceph-mon[95526]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 495 op/s 2023-07-14T11:39:16.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:16 smithi175 ceph-mon[129081]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 495 op/s 2023-07-14T11:39:17.630 INFO:tasks.workunit.client.0.smithi161.stdout: 10 135737 28.99 MB/sec execute 119 sec latency 5142.338 ms 2023-07-14T11:39:17.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:17 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:39:17.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:17 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:39:17.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:17 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:39:18.631 INFO:tasks.workunit.client.0.smithi161.stdout: 10 136080 28.92 MB/sec execute 120 sec latency 6143.342 ms 2023-07-14T11:39:18.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:18 smithi161 ceph-mon[95526]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 575 op/s 2023-07-14T11:39:18.745 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:18 smithi161 ceph-mon[95526]: osdmap e115: 12 total, 12 up, 12 in 2023-07-14T11:39:18.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:18 smithi175 ceph-mon[129081]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 575 op/s 2023-07-14T11:39:18.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:18 smithi175 ceph-mon[129081]: osdmap e115: 12 total, 12 up, 12 in 2023-07-14T11:39:18.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:18 smithi182 ceph-mon[129366]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 575 op/s 2023-07-14T11:39:18.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:18 smithi182 ceph-mon[129366]: osdmap e115: 12 total, 12 up, 12 in 2023-07-14T11:39:19.632 INFO:tasks.workunit.client.0.smithi161.stdout: 10 136169 28.74 MB/sec execute 121 sec latency 7144.345 ms 2023-07-14T11:39:20.636 INFO:tasks.workunit.client.0.smithi161.stdout: 10 136408 28.61 MB/sec execute 122 sec latency 8147.734 ms 2023-07-14T11:39:20.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:20 smithi175 ceph-mon[129081]: pgmap v537: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 549 op/s 2023-07-14T11:39:20.957 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:20 smithi182 ceph-mon[129366]: pgmap v537: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 549 op/s 2023-07-14T11:39:20.993 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:20 smithi161 ceph-mon[95526]: pgmap v537: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 549 op/s 2023-07-14T11:39:21.635 INFO:tasks.workunit.client.0.smithi161.stdout: 10 136675 28.50 MB/sec execute 123 sec latency 8564.855 ms 2023-07-14T11:39:22.635 INFO:tasks.workunit.client.0.smithi161.stdout: 10 137319 28.50 MB/sec execute 124 sec latency 1143.181 ms 2023-07-14T11:39:22.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:22 smithi175 ceph-mon[129081]: pgmap v538: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 554 op/s 2023-07-14T11:39:22.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:22 smithi182 ceph-mon[129366]: pgmap v538: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 554 op/s 2023-07-14T11:39:22.988 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:22 smithi161 ceph-mon[95526]: pgmap v538: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 554 op/s 2023-07-14T11:39:23.636 INFO:tasks.workunit.client.0.smithi161.stdout: 10 137933 28.50 MB/sec execute 125 sec latency 2143.239 ms 2023-07-14T11:39:24.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:24 smithi175 ceph-mon[129081]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 694 op/s 2023-07-14T11:39:24.457 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:24 smithi182 ceph-mon[129366]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 694 op/s 2023-07-14T11:39:24.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:24 smithi161 ceph-mon[95526]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 694 op/s 2023-07-14T11:39:24.636 INFO:tasks.workunit.client.0.smithi161.stdout: 10 138399 28.48 MB/sec execute 126 sec latency 3143.293 ms 2023-07-14T11:39:25.636 INFO:tasks.workunit.client.0.smithi161.stdout: 10 139051 28.57 MB/sec execute 127 sec latency 3986.386 ms 2023-07-14T11:39:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: 10 139499 28.56 MB/sec execute 128 sec latency 4040.549 ms 2023-07-14T11:39:26.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:26 smithi182 ceph-mon[129366]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 694 op/s 2023-07-14T11:39:26.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:26 smithi161 ceph-mon[95526]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 694 op/s 2023-07-14T11:39:26.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:26 smithi175 ceph-mon[129081]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 694 op/s 2023-07-14T11:39:27.636 INFO:tasks.workunit.client.0.smithi161.stdout: 10 139955 28.55 MB/sec execute 129 sec latency 5040.759 ms 2023-07-14T11:39:28.636 INFO:tasks.workunit.client.0.smithi161.stdout: 10 140376 28.47 MB/sec execute 130 sec latency 5226.724 ms 2023-07-14T11:39:28.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:28 smithi182 ceph-mon[129366]: pgmap v541: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 662 op/s 2023-07-14T11:39:28.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:28 smithi161 ceph-mon[95526]: pgmap v541: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 662 op/s 2023-07-14T11:39:28.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:28 smithi175 ceph-mon[129081]: pgmap v541: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 662 op/s 2023-07-14T11:39:29.639 INFO:tasks.workunit.client.0.smithi161.stdout: 10 140737 28.35 MB/sec execute 131 sec latency 205.415 ms 2023-07-14T11:39:30.637 INFO:tasks.workunit.client.0.smithi161.stdout: 10 141375 28.41 MB/sec execute 132 sec latency 105.289 ms 2023-07-14T11:39:30.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:30 smithi182 ceph-mon[129366]: pgmap v542: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 577 op/s 2023-07-14T11:39:30.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:30 smithi161 ceph-mon[95526]: pgmap v542: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 577 op/s 2023-07-14T11:39:30.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:30 smithi175 ceph-mon[129081]: pgmap v542: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 577 op/s 2023-07-14T11:39:31.637 INFO:tasks.workunit.client.0.smithi161.stdout: 10 142121 28.49 MB/sec execute 133 sec latency 126.319 ms 2023-07-14T11:39:32.637 INFO:tasks.workunit.client.0.smithi161.stdout: 10 142901 28.63 MB/sec execute 134 sec latency 101.959 ms 2023-07-14T11:39:32.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:32 smithi182 ceph-mon[129366]: pgmap v543: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 606 op/s 2023-07-14T11:39:32.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:32 smithi161 ceph-mon[95526]: pgmap v543: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 606 op/s 2023-07-14T11:39:32.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:32 smithi175 ceph-mon[129081]: pgmap v543: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 606 op/s 2023-07-14T11:39:33.637 INFO:tasks.workunit.client.0.smithi161.stdout: 10 143373 28.60 MB/sec execute 135 sec latency 113.577 ms 2023-07-14T11:39:34.415 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:34 smithi175 ceph-mon[129081]: pgmap v544: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 13 MiB/s wr, 637 op/s 2023-07-14T11:39:34.457 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:34 smithi182 ceph-mon[129366]: pgmap v544: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 13 MiB/s wr, 637 op/s 2023-07-14T11:39:34.488 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:34 smithi161 ceph-mon[95526]: pgmap v544: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 13 MiB/s wr, 637 op/s 2023-07-14T11:39:34.637 INFO:tasks.workunit.client.0.smithi161.stdout: 10 143815 28.54 MB/sec execute 136 sec latency 136.614 ms 2023-07-14T11:39:35.637 INFO:tasks.workunit.client.0.smithi161.stdout: 10 144462 28.63 MB/sec execute 137 sec latency 86.549 ms 2023-07-14T11:39:36.637 INFO:tasks.workunit.client.0.smithi161.stdout: 10 145094 28.63 MB/sec execute 138 sec latency 105.281 ms 2023-07-14T11:39:36.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:36 smithi182 ceph-mon[129366]: pgmap v545: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.9 MiB/s wr, 433 op/s 2023-07-14T11:39:36.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:36 smithi161 ceph-mon[95526]: pgmap v545: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.9 MiB/s wr, 433 op/s 2023-07-14T11:39:36.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:36 smithi175 ceph-mon[129081]: pgmap v545: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.9 MiB/s wr, 433 op/s 2023-07-14T11:39:37.638 INFO:tasks.workunit.client.0.smithi161.stdout: 10 145708 28.67 MB/sec execute 139 sec latency 129.753 ms 2023-07-14T11:39:38.638 INFO:tasks.workunit.client.0.smithi161.stdout: 10 146164 28.66 MB/sec execute 140 sec latency 127.092 ms 2023-07-14T11:39:38.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:38 smithi182 ceph-mon[129366]: pgmap v546: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 582 op/s 2023-07-14T11:39:38.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:38 smithi161 ceph-mon[95526]: pgmap v546: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 582 op/s 2023-07-14T11:39:38.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:38 smithi175 ceph-mon[129081]: pgmap v546: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 582 op/s 2023-07-14T11:39:39.421 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:39:39.638 INFO:tasks.workunit.client.0.smithi161.stdout: 10 146714 28.62 MB/sec execute 141 sec latency 84.064 ms 2023-07-14T11:39:39.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 -- 172.21.15.161:0/110559490 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f08940fe5a0 msgr2=0x7f089410c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 --2- 172.21.15.161:0/110559490 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f08940fe5a0 0x7f089410c940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f088c009f90 tx=0x7f088c00b330 comp rx=0 tx=0).stop 2023-07-14T11:39:39.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 -- 172.21.15.161:0/110559490 shutdown_connections 2023-07-14T11:39:39.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 --2- 172.21.15.161:0/110559490 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f08940fe5a0 0x7f089410c940 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 --2- 172.21.15.161:0/110559490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0894105bc0 0x7f0894105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 --2- 172.21.15.161:0/110559490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08940fdbf0 0x7f08940fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 -- 172.21.15.161:0/110559490 >> 172.21.15.161:0/110559490 conn(0x7f0894075b70 msgr2=0x7f0894075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:39.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 -- 172.21.15.161:0/110559490 shutdown_connections 2023-07-14T11:39:39.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.687+0000 7f0899fff700 1 -- 172.21.15.161:0/110559490 wait complete. 2023-07-14T11:39:39.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 Processor -- start 2023-07-14T11:39:39.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 -- start start 2023-07-14T11:39:39.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08940fdbf0 0x7f089410b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08940fe5a0 0x7f089410bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0894105bc0 0x7f08941b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08941b0d90 con 0x7f08940fe5a0 2023-07-14T11:39:39.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08941b0ed0 con 0x7f08940fdbf0 2023-07-14T11:39:39.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.689+0000 7f0899fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08941b1040 con 0x7f0894105bc0 2023-07-14T11:39:39.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f08937fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08940fdbf0 0x7f089410b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f08937fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08940fdbf0 0x7f089410b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:58776/0 (socket says 172.21.15.161:58776) 2023-07-14T11:39:39.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f08937fe700 1 -- 172.21.15.161:0/2807970674 learned_addr learned my addr 172.21.15.161:0/2807970674 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:39.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f0893fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0894105bc0 0x7f08941b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f0892ffd700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08940fe5a0 0x7f089410bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f0893fff700 1 -- 172.21.15.161:0/2807970674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08940fdbf0 msgr2=0x7f089410b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f0893fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08940fdbf0 0x7f089410b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f0893fff700 1 -- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08940fe5a0 msgr2=0x7f089410bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f0893fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08940fe5a0 0x7f089410bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.690+0000 7f0893fff700 1 -- 172.21.15.161:0/2807970674 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c009c40 con 0x7f0894105bc0 2023-07-14T11:39:39.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.691+0000 7f0893fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0894105bc0 0x7f08941b0850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f088c00b6c0 tx=0x7f088c009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:39.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.691+0000 7f0890ff9700 1 -- 172.21.15.161:0/2807970674 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c00e040 con 0x7f0894105bc0 2023-07-14T11:39:39.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.691+0000 7f0890ff9700 1 -- 172.21.15.161:0/2807970674 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088c01d470 con 0x7f0894105bc0 2023-07-14T11:39:39.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.691+0000 7f0890ff9700 1 -- 172.21.15.161:0/2807970674 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c005280 con 0x7f0894105bc0 2023-07-14T11:39:39.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.691+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08941b12c0 con 0x7f0894105bc0 2023-07-14T11:39:39.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.691+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08941b16f0 con 0x7f0894105bc0 2023-07-14T11:39:39.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.695+0000 7f0890ff9700 1 -- 172.21.15.161:0/2807970674 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f088c01d5e0 con 0x7f0894105bc0 2023-07-14T11:39:39.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.696+0000 7f08ad89eb80 1 Processor -- start 2023-07-14T11:39:39.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.696+0000 7f08ad89eb80 1 -- start start 2023-07-14T11:39:39.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 0x559dfd6f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f2be0 0x559dfd6fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6fdad0 0x559dfd6ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd5f8cb0 con 0x559dfd6f22d0 2023-07-14T11:39:39.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd64ba40 con 0x559dfd6f2be0 2023-07-14T11:39:39.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd644ed0 con 0x559dfd6fdad0 2023-07-14T11:39:39.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f0890ff9700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f087c0a1340 0x7f087c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f0890ff9700 1 -- 172.21.15.161:0/2807970674 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f088c0ce490 con 0x7f0894105bc0 2023-07-14T11:39:39.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6fdad0 0x559dfd6ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6fdad0 0x559dfd6ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:51192/0 (socket says 172.21.15.161:51192) 2023-07-14T11:39:39.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f0892ffd700 1 -- 172.21.15.161:0/3320798656 learned_addr learned my addr 172.21.15.161:0/3320798656 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:39.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.697+0000 7f0893fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f087c0a1340 0x7f087c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.698+0000 7f0892ffd700 1 -- 172.21.15.161:0/3320798656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f2be0 msgr2=0x559dfd6fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.698+0000 7f0892ffd700 1 --2- 172.21.15.161:0/3320798656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f2be0 0x559dfd6fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.698+0000 7f0892ffd700 1 -- 172.21.15.161:0/3320798656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 msgr2=0x559dfd6f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.698+0000 7f0892ffd700 1 --2- 172.21.15.161:0/3320798656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 0x559dfd6f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.698+0000 7f0892ffd700 1 -- 172.21.15.161:0/3320798656 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dfd472de0 con 0x559dfd6fdad0 2023-07-14T11:39:39.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.698+0000 7f0892ffd700 1 --2- 172.21.15.161:0/3320798656 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6fdad0 0x559dfd6ffeb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f088800bdb0 tx=0x7f08880126d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae340e897e7f5020 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:39.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.698+0000 7f0893fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f087c0a1340 0x7f087c0a37f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f088c0cb880 tx=0x7f088c01dc90 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:39.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.699+0000 7f0879ffb700 1 -- 172.21.15.161:0/3320798656 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088800b930 con 0x559dfd6fdad0 2023-07-14T11:39:39.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.699+0000 7f0879ffb700 1 -- 172.21.15.161:0/3320798656 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0888012e80 con 0x559dfd6fdad0 2023-07-14T11:39:39.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.699+0000 7f0879ffb700 1 -- 172.21.15.161:0/3320798656 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888010450 con 0x559dfd6fdad0 2023-07-14T11:39:39.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 -- 172.21.15.161:0/3320798656 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6fdad0 msgr2=0x559dfd6ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/3320798656 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6fdad0 0x559dfd6ffeb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f088800bdb0 tx=0x7f08880126d0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 -- 172.21.15.161:0/3320798656 shutdown_connections 2023-07-14T11:39:39.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/3320798656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 0x559dfd6f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/3320798656 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6fdad0 0x559dfd6ffeb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/3320798656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f2be0 0x559dfd6fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 -- 172.21.15.161:0/3320798656 >> 172.21.15.161:0/3320798656 conn(0x559dfd6e8da0 msgr2=0x559dfd6ee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:39.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 -- 172.21.15.161:0/3320798656 shutdown_connections 2023-07-14T11:39:39.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.700+0000 7f08ad89eb80 1 -- 172.21.15.161:0/3320798656 wait complete. 2023-07-14T11:39:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.701+0000 7f08ad89eb80 1 -- start start 2023-07-14T11:39:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.701+0000 7f08ad89eb80 1 Processor -- start 2023-07-14T11:39:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd6fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6fa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd6faf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd5e4ac0 con 0x559dfd6fdad0 2023-07-14T11:39:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd5e47c0 con 0x559dfd6f22d0 2023-07-14T11:39:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd4c8920 con 0x559dfd6f2be0 2023-07-14T11:39:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f08ad89eb80 10 client.0 _ll_register_callbacks cb 0x559dfd5e2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd6faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd6faf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48156/0 (socket says 172.21.15.161:48156) 2023-07-14T11:39:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.702+0000 7f0892ffd700 1 -- 172.21.15.161:0/4081298412 learned_addr learned my addr 172.21.15.161:0/4081298412 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f08937fe700 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6fa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f0892ffd700 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd6fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f08937fe700 1 -- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 msgr2=0x559dfd6fd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f08937fe700 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd6fd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f08937fe700 1 -- 172.21.15.161:0/4081298412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 msgr2=0x559dfd6faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f08937fe700 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd6faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f08937fe700 1 -- 172.21.15.161:0/4081298412 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088800ba90 con 0x559dfd6f2be0 2023-07-14T11:39:39.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f08937fe700 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6fa850 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08840116c0 tx=0x7f0884009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7cd4b7b8efbc41b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:39.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f0872ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:39.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.703+0000 7f0872ffd700 1 -- 172.21.15.161:0/4081298412 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884004440 con 0x559dfd6f2be0 2023-07-14T11:39:39.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.704+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dfd63deb0 con 0x559dfd6f2be0 2023-07-14T11:39:39.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.704+0000 7f0872ffd700 1 -- 172.21.15.161:0/4081298412 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f0884003680 con 0x559dfd6f2be0 2023-07-14T11:39:39.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.704+0000 7f0872ffd700 1 -- 172.21.15.161:0/4081298412 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08840045a0 con 0x559dfd6f2be0 2023-07-14T11:39:39.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.704+0000 7f08ad89eb80 10 client.27643 fetch_fsmap learned FSMap version 42 2023-07-14T11:39:39.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.704+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559dfd45bc60 con 0x559dfd6f2be0 2023-07-14T11:39:39.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.704+0000 7f0872ffd700 1 -- 172.21.15.161:0/4081298412 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559dfd63deb0 con 0x559dfd6f2be0 2023-07-14T11:39:39.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.704+0000 7f0872ffd700 1 -- 172.21.15.161:0/4081298412 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f088401aa90 con 0x559dfd6f2be0 2023-07-14T11:39:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.705+0000 7f08ad89eb80 10 client.27643 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T11:39:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.705+0000 7f08ad89eb80 10 client.27643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.705+0000 7f08ad89eb80 20 client.27643 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.705+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd70d500 0x559dfd70f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.705+0000 7f08ad89eb80 4 client.27643 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:39:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.705+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dfd600050 con 0x559dfd70d500 2023-07-14T11:39:39.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.706+0000 7f0892ffd700 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd70d500 0x559dfd70f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.706+0000 7f0872ffd700 10 client.27643 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:39.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.706+0000 7f0892ffd700 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd70d500 0x559dfd70f910 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f23a6f0bef7f80 server_cookie=e3bf88efd873df7a in_seq=0 out_seq=0 2023-07-14T11:39:39.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f0872ffd700 1 -- 172.21.15.161:0/4081298412 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559dfd600050 con 0x559dfd70d500 2023-07-14T11:39:39.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f0872ffd700 10 client.27643 handle_command_reply: tid=0 2023-07-14T11:39:39.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 client.27643 shutdown 2023-07-14T11:39:39.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 msgr2=0x559dfd6fa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6fa850 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08840116c0 tx=0x7f0884009f90 comp rx=0 tx=0).stop 2023-07-14T11:39:39.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 shutdown_connections 2023-07-14T11:39:39.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd70d500 0x559dfd70f910 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd6faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd6fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/4081298412 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6fa850 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 >> 172.21.15.161:0/4081298412 conn(0x559dfd6e8da0 msgr2=0x559dfd6ee290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.709+0000 7f0872ffd700 0 client.27643 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.710+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 shutdown_connections 2023-07-14T11:39:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.710+0000 7f08ad89eb80 1 -- 172.21.15.161:0/4081298412 wait complete. 2023-07-14T11:39:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.710+0000 7f08ad89eb80 20 client.27643 trim_cache size 0 max 16384 2023-07-14T11:39:39.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 Processor -- start 2023-07-14T11:39:39.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 -- start start 2023-07-14T11:39:39.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 0x559dfd6f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6f6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6fdad0 0x559dfd6fa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd6ef2e0 con 0x559dfd6f22d0 2023-07-14T11:39:39.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd6f3540 con 0x559dfd6fdad0 2023-07-14T11:39:39.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd6e5fb0 con 0x559dfd6f2be0 2023-07-14T11:39:39.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08937fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08937fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6f6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:51204/0 (socket says 172.21.15.161:51204) 2023-07-14T11:39:39.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 0x559dfd6f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.730+0000 7f08937fe700 1 -- 172.21.15.161:0/2661285502 learned_addr learned my addr 172.21.15.161:0/2661285502 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:39.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.731+0000 7f0892ffd700 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6fdad0 0x559dfd6fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.731+0000 7f08937fe700 1 -- 172.21.15.161:0/2661285502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6fdad0 msgr2=0x559dfd6fa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.731+0000 7f08937fe700 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6fdad0 0x559dfd6fa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.731+0000 7f08937fe700 1 -- 172.21.15.161:0/2661285502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 msgr2=0x559dfd6f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.731+0000 7f08937fe700 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 0x559dfd6f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.731+0000 7f08937fe700 1 -- 172.21.15.161:0/2661285502 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dfd499250 con 0x559dfd6f2be0 2023-07-14T11:39:39.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.731+0000 7f08937fe700 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6f6110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f088401bff0 tx=0x7f088401ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe762918853341bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:39.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.732+0000 7f087a7fc700 1 -- 172.21.15.161:0/2661285502 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840052b0 con 0x559dfd6f2be0 2023-07-14T11:39:39.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.732+0000 7f087a7fc700 1 -- 172.21.15.161:0/2661285502 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088401a900 con 0x559dfd6f2be0 2023-07-14T11:39:39.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.732+0000 7f087a7fc700 1 -- 172.21.15.161:0/2661285502 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884003cd0 con 0x559dfd6f2be0 2023-07-14T11:39:39.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 -- 172.21.15.161:0/2661285502 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 msgr2=0x559dfd6f6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6f6110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f088401bff0 tx=0x7f088401ac70 comp rx=0 tx=0).stop 2023-07-14T11:39:39.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 -- 172.21.15.161:0/2661285502 shutdown_connections 2023-07-14T11:39:39.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6f22d0 0x559dfd6f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6fdad0 0x559dfd6fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/2661285502 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd6f6110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 -- 172.21.15.161:0/2661285502 >> 172.21.15.161:0/2661285502 conn(0x559dfd6e8da0 msgr2=0x559dfd708cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:39.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 -- 172.21.15.161:0/2661285502 shutdown_connections 2023-07-14T11:39:39.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 -- 172.21.15.161:0/2661285502 wait complete. 2023-07-14T11:39:39.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.733+0000 7f08ad89eb80 1 -- start start 2023-07-14T11:39:39.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 1 Processor -- start 2023-07-14T11:39:39.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd704550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd702930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd702e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd703550 con 0x559dfd6fdad0 2023-07-14T11:39:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd7036c0 con 0x559dfd6f22d0 2023-07-14T11:39:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfd7613d0 con 0x559dfd6f2be0 2023-07-14T11:39:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.734+0000 7f08ad89eb80 10 client.0 _ll_register_callbacks cb 0x559dfd5e2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:39:39.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd702930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd702930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:51206/0 (socket says 172.21.15.161:51206) 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 -- 172.21.15.161:0/363481922 learned_addr learned my addr 172.21.15.161:0/363481922 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f0892ffd700 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd704550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 -- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 msgr2=0x559dfd704550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd704550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 -- 172.21.15.161:0/363481922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 msgr2=0x559dfd702e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd702e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 -- 172.21.15.161:0/363481922 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dfd45bc60 con 0x559dfd6f2be0 2023-07-14T11:39:39.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f08937fe700 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd702930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0884000c00 tx=0x7f0884003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61bf24e49123440f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:39:39.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.735+0000 7f0872ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:39:39.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f0872ffd700 1 -- 172.21.15.161:0/363481922 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884003a70 con 0x559dfd6f2be0 2023-07-14T11:39:39.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f0872ffd700 1 -- 172.21.15.161:0/363481922 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f0884000e40 con 0x559dfd6f2be0 2023-07-14T11:39:39.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dfd59e4f0 con 0x559dfd6f2be0 2023-07-14T11:39:39.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f0872ffd700 1 -- 172.21.15.161:0/363481922 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0884003bd0 con 0x559dfd6f2be0 2023-07-14T11:39:39.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 10 client.27649 fetch_fsmap learned FSMap version 42 2023-07-14T11:39:39.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559dfd4712b0 con 0x559dfd6f2be0 2023-07-14T11:39:39.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f0872ffd700 1 -- 172.21.15.161:0/363481922 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0884042020 con 0x559dfd6f2be0 2023-07-14T11:39:39.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f0872ffd700 1 -- 172.21.15.161:0/363481922 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0884028420 con 0x559dfd6f2be0 2023-07-14T11:39:39.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 10 client.27649 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T11:39:39.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 10 client.27649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:39:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 20 client.27649 populate_metadata read hostname 'smithi161' 2023-07-14T11:39:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd7626b0 0x559dfd764ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:39:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 4 client.27649 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:39:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.736+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559dfd493f70 con 0x559dfd7626b0 2023-07-14T11:39:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.806+0000 7f0892ffd700 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd7626b0 0x559dfd764ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:39:39.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.806+0000 7f0892ffd700 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd7626b0 0x559dfd764ac0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9a0881a0e4d1d9 server_cookie=5665e6001e27db38 in_seq=0 out_seq=0 2023-07-14T11:39:39.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.806+0000 7f0872ffd700 10 client.27649 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:39.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f0872ffd700 1 -- 172.21.15.161:0/363481922 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559dfd493f70 con 0x559dfd7626b0 2023-07-14T11:39:39.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f0872ffd700 10 client.27649 handle_command_reply: tid=0 2023-07-14T11:39:39.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 client.27649 shutdown 2023-07-14T11:39:39.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 msgr2=0x559dfd702930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd702930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0884000c00 tx=0x7f0884003680 comp rx=0 tx=0).stop 2023-07-14T11:39:39.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 shutdown_connections 2023-07-14T11:39:39.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x559dfd7626b0 0x559dfd764ac0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559dfd6fdad0 0x559dfd702e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dfd6f22d0 0x559dfd704550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 --2- 172.21.15.161:0/363481922 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x559dfd6f2be0 0x559dfd702930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 >> 172.21.15.161:0/363481922 conn(0x559dfd6e8da0 msgr2=0x559dfd6f5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:39.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f0872ffd700 0 client.27649 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:39:39.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 shutdown_connections 2023-07-14T11:39:39.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.808+0000 7f08ad89eb80 1 -- 172.21.15.161:0/363481922 wait complete. 2023-07-14T11:39:39.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.809+0000 7f08ad89eb80 20 client.27649 trim_cache size 0 max 16384 2023-07-14T11:39:39.806 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:39:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:39:39.807 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:39:39.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.810+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f087c0a1340 msgr2=0x7f087c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.810+0000 7f0899fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f087c0a1340 0x7f087c0a37f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f088c0cb880 tx=0x7f088c01dc90 comp rx=0 tx=0).stop 2023-07-14T11:39:39.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.810+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0894105bc0 msgr2=0x7f08941b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.810+0000 7f0899fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0894105bc0 0x7f08941b0850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f088c00b6c0 tx=0x7f088c009490 comp rx=0 tx=0).stop 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.810+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 shutdown_connections 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.811+0000 7f0899fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f087c0a1340 0x7f087c0a37f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.811+0000 7f0899fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0894105bc0 0x7f08941b0850 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.811+0000 7f0899fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08940fdbf0 0x7f089410b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.811+0000 7f0899fff700 1 --2- 172.21.15.161:0/2807970674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08940fe5a0 0x7f089410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.811+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 >> 172.21.15.161:0/2807970674 conn(0x7f0894075b70 msgr2=0x7f0894101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:39:39.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.811+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 shutdown_connections 2023-07-14T11:39:39.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:39:39.811+0000 7f0899fff700 1 -- 172.21.15.161:0/2807970674 wait complete. 2023-07-14T11:39:39.828 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:39:40.638 INFO:tasks.workunit.client.0.smithi161.stdout: 10 147273 28.64 MB/sec execute 142 sec latency 128.383 ms 2023-07-14T11:39:40.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:40 smithi182 ceph-mon[129366]: pgmap v547: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.5 MiB/s wr, 378 op/s 2023-07-14T11:39:40.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:40 smithi161 ceph-mon[95526]: pgmap v547: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.5 MiB/s wr, 378 op/s 2023-07-14T11:39:40.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:40 smithi175 ceph-mon[129081]: pgmap v547: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.5 MiB/s wr, 378 op/s 2023-07-14T11:39:41.638 INFO:tasks.workunit.client.0.smithi161.stdout: 10 147869 28.70 MB/sec execute 143 sec latency 106.220 ms 2023-07-14T11:39:42.638 INFO:tasks.workunit.client.0.smithi161.stdout: 10 148578 28.75 MB/sec execute 144 sec latency 74.834 ms 2023-07-14T11:39:42.707 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:42 smithi182 ceph-mon[129366]: pgmap v548: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 437 op/s 2023-07-14T11:39:42.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:42 smithi161 ceph-mon[95526]: pgmap v548: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 437 op/s 2023-07-14T11:39:42.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:42 smithi175 ceph-mon[129081]: pgmap v548: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 437 op/s 2023-07-14T11:39:43.638 INFO:tasks.workunit.client.0.smithi161.stdout: 10 149035 28.73 MB/sec execute 145 sec latency 270.150 ms 2023-07-14T11:39:44.421 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:44 smithi175 ceph-mon[129081]: pgmap v549: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 461 op/s 2023-07-14T11:39:44.461 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:44 smithi182 ceph-mon[129366]: pgmap v549: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 461 op/s 2023-07-14T11:39:44.490 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:44 smithi161 ceph-mon[95526]: pgmap v549: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 461 op/s 2023-07-14T11:39:44.641 INFO:tasks.workunit.client.0.smithi161.stdout: 10 149328 28.65 MB/sec execute 146 sec latency 344.129 ms 2023-07-14T11:39:45.641 INFO:tasks.workunit.client.0.smithi161.stdout: 10 149338 28.46 MB/sec execute 147 sec latency 1116.659 ms 2023-07-14T11:39:46.641 INFO:tasks.workunit.client.0.smithi161.stdout: 10 149422 28.29 MB/sec execute 148 sec latency 1472.815 ms 2023-07-14T11:39:46.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:46 smithi161 ceph-mon[95526]: pgmap v550: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 334 op/s 2023-07-14T11:39:46.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:46 smithi175 ceph-mon[129081]: pgmap v550: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 334 op/s 2023-07-14T11:39:46.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:46 smithi182 ceph-mon[129366]: pgmap v550: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 334 op/s 2023-07-14T11:39:47.641 INFO:tasks.workunit.client.0.smithi161.stdout: 10 149664 28.18 MB/sec execute 149 sec latency 211.638 ms 2023-07-14T11:39:48.643 INFO:tasks.workunit.client.0.smithi161.stdout: 10 149854 28.06 MB/sec execute 150 sec latency 256.580 ms 2023-07-14T11:39:48.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:48 smithi175 ceph-mon[129081]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 567 op/s 2023-07-14T11:39:48.957 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:48 smithi182 ceph-mon[129366]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 567 op/s 2023-07-14T11:39:48.989 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:48 smithi161 ceph-mon[95526]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 567 op/s 2023-07-14T11:39:49.642 INFO:tasks.workunit.client.0.smithi161.stdout: 10 150172 27.97 MB/sec execute 151 sec latency 178.249 ms 2023-07-14T11:39:50.642 INFO:tasks.workunit.client.0.smithi161.stdout: 10 150439 27.88 MB/sec execute 152 sec latency 532.868 ms 2023-07-14T11:39:50.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:50 smithi175 ceph-mon[129081]: pgmap v552: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 452 op/s 2023-07-14T11:39:50.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:50 smithi182 ceph-mon[129366]: pgmap v552: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 452 op/s 2023-07-14T11:39:50.988 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:50 smithi161 ceph-mon[95526]: pgmap v552: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 452 op/s 2023-07-14T11:39:51.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 150818 27.83 MB/sec execute 153 sec latency 157.913 ms 2023-07-14T11:39:52.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 151524 27.93 MB/sec execute 154 sec latency 77.530 ms 2023-07-14T11:39:52.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:52 smithi175 ceph-mon[129081]: pgmap v553: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 490 op/s 2023-07-14T11:39:52.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:52 smithi182 ceph-mon[129366]: pgmap v553: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 490 op/s 2023-07-14T11:39:52.988 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:52 smithi161 ceph-mon[95526]: pgmap v553: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 490 op/s 2023-07-14T11:39:53.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 152125 27.94 MB/sec execute 155 sec latency 79.650 ms 2023-07-14T11:39:54.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:54 smithi175 ceph-mon[129081]: pgmap v554: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 640 op/s 2023-07-14T11:39:54.456 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:54 smithi182 ceph-mon[129366]: pgmap v554: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 640 op/s 2023-07-14T11:39:54.488 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:54 smithi161 ceph-mon[95526]: pgmap v554: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 640 op/s 2023-07-14T11:39:54.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 152724 27.99 MB/sec execute 156 sec latency 86.556 ms 2023-07-14T11:39:55.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:55 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:39:55.457 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:55 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:39:55.492 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:55 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:39:55.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 153188 27.97 MB/sec execute 157 sec latency 146.032 ms 2023-07-14T11:39:56.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:56 smithi175 ceph-mon[129081]: pgmap v555: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 16 MiB/s wr, 554 op/s 2023-07-14T11:39:56.456 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:56 smithi182 ceph-mon[129366]: pgmap v555: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 16 MiB/s wr, 554 op/s 2023-07-14T11:39:56.488 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:56 smithi161 ceph-mon[95526]: pgmap v555: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 16 MiB/s wr, 554 op/s 2023-07-14T11:39:56.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 153519 27.86 MB/sec execute 158 sec latency 245.733 ms 2023-07-14T11:39:57.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 154264 27.95 MB/sec execute 159 sec latency 102.141 ms 2023-07-14T11:39:58.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 154917 27.98 MB/sec execute 160 sec latency 92.176 ms 2023-07-14T11:39:58.706 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:39:58 smithi182 ceph-mon[129366]: pgmap v556: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 698 op/s 2023-07-14T11:39:58.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:39:58 smithi161 ceph-mon[95526]: pgmap v556: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 698 op/s 2023-07-14T11:39:58.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:39:58 smithi175 ceph-mon[129081]: pgmap v556: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 698 op/s 2023-07-14T11:39:59.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 155608 28.08 MB/sec execute 161 sec latency 95.161 ms 2023-07-14T11:40:00.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 156032 28.04 MB/sec execute 162 sec latency 255.066 ms 2023-07-14T11:40:00.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:00 smithi161 ceph-mon[95526]: pgmap v557: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.6 MiB/s wr, 483 op/s 2023-07-14T11:40:00.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:00 smithi161 ceph-mon[95526]: overall HEALTH_OK 2023-07-14T11:40:00.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:00 smithi175 ceph-mon[129081]: pgmap v557: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.6 MiB/s wr, 483 op/s 2023-07-14T11:40:00.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:00 smithi175 ceph-mon[129081]: overall HEALTH_OK 2023-07-14T11:40:00.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:00 smithi182 ceph-mon[129366]: pgmap v557: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.6 MiB/s wr, 483 op/s 2023-07-14T11:40:00.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:00 smithi182 ceph-mon[129366]: overall HEALTH_OK 2023-07-14T11:40:01.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 156036 27.87 MB/sec execute 163 sec latency 1230.548 ms 2023-07-14T11:40:02.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 156038 27.71 MB/sec execute 164 sec latency 2230.597 ms 2023-07-14T11:40:02.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:02 smithi161 ceph-mon[95526]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.5 MiB/s wr, 495 op/s 2023-07-14T11:40:02.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:02 smithi175 ceph-mon[129081]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.5 MiB/s wr, 495 op/s 2023-07-14T11:40:02.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:02 smithi182 ceph-mon[129366]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.5 MiB/s wr, 495 op/s 2023-07-14T11:40:03.644 INFO:tasks.workunit.client.0.smithi161.stdout: 10 156039 27.54 MB/sec execute 165 sec latency 3230.766 ms 2023-07-14T11:40:03.740 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:03 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:40:03.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:03 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:40:03.741 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:03 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:40:03.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:03 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:40:03.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:03 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:40:03.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:03 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:40:03.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:03 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:40:03.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:03 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:40:03.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:03 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:40:04.645 INFO:tasks.workunit.client.0.smithi161.stdout: 10 156039 27.37 MB/sec execute 166 sec latency 4230.858 ms 2023-07-14T11:40:04.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:04 smithi161 ceph-mon[95526]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 544 op/s 2023-07-14T11:40:04.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:04 smithi175 ceph-mon[129081]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 544 op/s 2023-07-14T11:40:04.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:04 smithi182 ceph-mon[129366]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 544 op/s 2023-07-14T11:40:05.390 INFO:tasks.workunit.client.0.smithi161.stdout:[211329] write failed on handle 13755 (Resource temporarily unavailable) 2023-07-14T11:40:05.391 INFO:tasks.workunit.client.0.smithi161.stdout:Child failed with status 1 2023-07-14T11:40:05.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:40:05.477 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-07-14T11:40:05.478 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-14T11:40:06.417 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3fc0089cc1082313a98853d48b892acdc1385b8b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-07-14T11:40:06.501 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3745c23d90bb46c39b9ea03f6dc52fe6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3fc0089cc1082313a98853d48b892acdc1385b8b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-07-14T11:40:06.504 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-07-14T11:40:06.515 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-07-14T11:40:06.516 INFO:teuthology.task.exec:Executing custom commands... 2023-07-14T11:40:06.516 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T11:40:06.516 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-07-14T11:40:06.645 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-07-14T11:40:06.707 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:06 smithi161 ceph-mon[95526]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 316 op/s 2023-07-14T11:40:06.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-14T11:40:06.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:06 smithi175 ceph-mon[129081]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 316 op/s 2023-07-14T11:40:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 -- 172.21.15.161:0/3667743527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 msgr2=0x7f0ae0105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/3667743527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 0x7f0ae0105f30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009990 tx=0x7f0ad0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:40:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 -- 172.21.15.161:0/3667743527 shutdown_connections 2023-07-14T11:40:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/3667743527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ae01004e0 0x7f0ae010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/3667743527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 0x7f0ae0105f30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/3667743527 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae00ffb30 0x7f0ae00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 -- 172.21.15.161:0/3667743527 >> 172.21.15.161:0/3667743527 conn(0x7f0ae00fb5f0 msgr2=0x7f0ae00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 -- 172.21.15.161:0/3667743527 shutdown_connections 2023-07-14T11:40:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.925+0000 7f0ae65ff700 1 -- 172.21.15.161:0/3667743527 wait complete. 2023-07-14T11:40:06.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.927+0000 7f0ae65ff700 1 Processor -- start 2023-07-14T11:40:06.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.927+0000 7f0ae65ff700 1 -- start start 2023-07-14T11:40:06.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0ae65ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ae00ffb30 0x7f0ae019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:06.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0ae65ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae01004e0 0x7f0ae019c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:06.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0ae65ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 0x7f0ae0195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:06.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0ae65ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae01960e0 con 0x7f0ae00ffb30 2023-07-14T11:40:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0ae65ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0196250 con 0x7f0ae0105b60 2023-07-14T11:40:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0ae65ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0196410 con 0x7f0ae01004e0 2023-07-14T11:40:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ae00ffb30 0x7f0ae019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae01004e0 0x7f0ae019c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae01004e0 0x7f0ae019c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56664/0 (socket says 172.21.15.161:56664) 2023-07-14T11:40:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 -- 172.21.15.161:0/2964211817 learned_addr learned my addr 172.21.15.161:0/2964211817 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0ae4b9c700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 0x7f0ae0195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 -- 172.21.15.161:0/2964211817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 msgr2=0x7f0ae0195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 0x7f0ae0195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 -- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ae00ffb30 msgr2=0x7f0ae019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ae00ffb30 0x7f0ae019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.928+0000 7f0adf7fe700 1 -- 172.21.15.161:0/2964211817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8009c40 con 0x7f0ae01004e0 2023-07-14T11:40:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.929+0000 7f0adf7fe700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae01004e0 0x7f0ae019c0f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e9a0 tx=0x7f0ad400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.929+0000 7f0add7fa700 1 -- 172.21.15.161:0/2964211817 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4009790 con 0x7f0ae01004e0 2023-07-14T11:40:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.929+0000 7f0add7fa700 1 -- 172.21.15.161:0/2964211817 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ad400ce80 con 0x7f0ae01004e0 2023-07-14T11:40:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.929+0000 7f0add7fa700 1 -- 172.21.15.161:0/2964211817 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4018680 con 0x7f0ae01004e0 2023-07-14T11:40:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.929+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad00095f0 con 0x7f0ae01004e0 2023-07-14T11:40:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.929+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01a29c0 con 0x7f0ae01004e0 2023-07-14T11:40:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.930+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae00ff250 con 0x7f0ae01004e0 2023-07-14T11:40:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.932+0000 7f0add7fa700 1 -- 172.21.15.161:0/2964211817 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0ad4010430 con 0x7f0ae01004e0 2023-07-14T11:40:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.933+0000 7f0add7fa700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0ac80a12c0 0x7f0ac80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.934+0000 7f0add7fa700 1 -- 172.21.15.161:0/2964211817 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f0ad4014070 con 0x7f0ae01004e0 2023-07-14T11:40:06.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.934+0000 7f0adffff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0ac80a12c0 0x7f0ac80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:06.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.934+0000 7f0adffff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0ac80a12c0 0x7f0ac80a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b2e0 tx=0x7f0ad0005fb0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:06.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:06.936+0000 7f0add7fa700 1 -- 172.21.15.161:0/2964211817 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0ad40966d0 con 0x7f0ae01004e0 2023-07-14T11:40:06.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:06 smithi182 ceph-mon[129366]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 316 op/s 2023-07-14T11:40:07.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.165+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 --> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ae00fd260 con 0x7f0ac80a12c0 2023-07-14T11:40:07.172 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.177+0000 7f0add7fa700 1 -- 172.21.15.161:0/2964211817 <== mgr.15543 v2:172.21.15.161:6800/4046273551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f0ae00fd260 con 0x7f0ac80a12c0 2023-07-14T11:40:07.173 INFO:teuthology.orchestra.run.smithi161.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-14T00:00:00", "created": "2023-07-14T11:22:16", "first": "2023-07-14T11:23:00", "last": "2023-07-14T11:40:00", "last_pruned": "2023-07-14T11:40:00", "created_count": 18, "pruned_count": 12, "active": true} 2023-07-14T11:40:07.175 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0ac80a12c0 msgr2=0x7f0ac80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:07.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0ac80a12c0 0x7f0ac80a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b2e0 tx=0x7f0ad0005fb0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae01004e0 msgr2=0x7f0ae019c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:07.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae01004e0 0x7f0ae019c0f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e9a0 tx=0x7f0ad400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 shutdown_connections 2023-07-14T11:40:07.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f0ac80a12c0 0x7f0ac80a3770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ae00ffb30 0x7f0ae019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0105b60 0x7f0ae0195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 --2- 172.21.15.161:0/2964211817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f0ae01004e0 0x7f0ae019c0f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 >> 172.21.15.161:0/2964211817 conn(0x7f0ae00fb5f0 msgr2=0x7f0ae00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:07.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 shutdown_connections 2023-07-14T11:40:07.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.179+0000 7f0ae65ff700 1 -- 172.21.15.161:0/2964211817 wait complete. 2023-07-14T11:40:07.195 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-14T11:40:07.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/2820407283 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a14105b60 msgr2=0x7f6a14105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:07.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/2820407283 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a14105b60 0x7f6a14105f30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6a040098f0 tx=0x7f6a04009c00 comp rx=0 tx=0).stop 2023-07-14T11:40:07.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/2820407283 shutdown_connections 2023-07-14T11:40:07.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/2820407283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a141004e0 0x7f6a1410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/2820407283 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a14105b60 0x7f6a14105f30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/2820407283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 0x7f6a140fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/2820407283 >> 172.21.15.161:0/2820407283 conn(0x7f6a140fb5f0 msgr2=0x7f6a140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:07.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/2820407283 shutdown_connections 2023-07-14T11:40:07.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.396+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/2820407283 wait complete. 2023-07-14T11:40:07.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.397+0000 7f6a1a5ff700 1 Processor -- start 2023-07-14T11:40:07.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.397+0000 7f6a1a5ff700 1 -- start start 2023-07-14T11:40:07.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.397+0000 7f6a1a5ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 0x7f6a14197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:07.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.397+0000 7f6a1a5ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a141004e0 0x7f6a14198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:07.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.397+0000 7f6a1a5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a14105b60 0x7f6a1419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:07.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a1a5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1419d0d0 con 0x7f6a14105b60 2023-07-14T11:40:07.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a1a5ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1419d240 con 0x7f6a140ffb30 2023-07-14T11:40:07.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a1a5ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1419d400 con 0x7f6a141004e0 2023-07-14T11:40:07.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 0x7f6a14197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:07.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a0bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a141004e0 0x7f6a14198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:07.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 0x7f6a14197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:41136/0 (socket says 172.21.15.161:41136) 2023-07-14T11:40:07.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 -- 172.21.15.161:0/507622281 learned_addr learned my addr 172.21.15.161:0/507622281 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:07.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a18b9c700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a14105b60 0x7f6a1419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:07.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 -- 172.21.15.161:0/507622281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a141004e0 msgr2=0x7f6a14198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:07.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a141004e0 0x7f6a14198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 -- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a14105b60 msgr2=0x7f6a1419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a14105b60 0x7f6a1419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 -- 172.21.15.161:0/507622281 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0095f0 con 0x7f6a140ffb30 2023-07-14T11:40:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a0bfff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a141004e0 0x7f6a14198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a18b9c700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a14105b60 0x7f6a1419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.398+0000 7f6a13fff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 0x7f6a14197ee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009810 tx=0x7f6a0400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.399+0000 7f6a11ffb700 1 -- 172.21.15.161:0/507622281 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0401c070 con 0x7f6a140ffb30 2023-07-14T11:40:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.399+0000 7f6a11ffb700 1 -- 172.21.15.161:0/507622281 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0400bb20 con 0x7f6a140ffb30 2023-07-14T11:40:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.399+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a04009520 con 0x7f6a140ffb30 2023-07-14T11:40:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.399+0000 7f6a11ffb700 1 -- 172.21.15.161:0/507622281 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04004930 con 0x7f6a140ffb30 2023-07-14T11:40:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.399+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1419d910 con 0x7f6a140ffb30 2023-07-14T11:40:07.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.400+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a140ff250 con 0x7f6a140ffb30 2023-07-14T11:40:07.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.402+0000 7f6a11ffb700 1 -- 172.21.15.161:0/507622281 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6a04020070 con 0x7f6a140ffb30 2023-07-14T11:40:07.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.403+0000 7f6a11ffb700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6a000a12c0 0x7f6a000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:07.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.403+0000 7f6a11ffb700 1 -- 172.21.15.161:0/507622281 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f6a040ce200 con 0x7f6a140ffb30 2023-07-14T11:40:07.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.403+0000 7f6a0bfff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6a000a12c0 0x7f6a000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:07.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.404+0000 7f6a0bfff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6a000a12c0 0x7f6a000a3770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6a14199700 tx=0x7f69fc009310 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:07.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.406+0000 7f6a11ffb700 1 -- 172.21.15.161:0/507622281 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6a04097260 con 0x7f6a140ffb30 2023-07-14T11:40:07.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.642+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 --> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6a141994b0 con 0x7f6a000a12c0 2023-07-14T11:40:07.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.653+0000 7f6a11ffb700 1 -- 172.21.15.161:0/507622281 <== mgr.15543 v2:172.21.15.161:6800/4046273551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f6a141994b0 con 0x7f6a000a12c0 2023-07-14T11:40:07.649 INFO:teuthology.orchestra.run.smithi161.stdout:/ 1M 6M3h 2023-07-14T11:40:07.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6a000a12c0 msgr2=0x7f6a000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6a000a12c0 0x7f6a000a3770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6a14199700 tx=0x7f69fc009310 comp rx=0 tx=0).stop 2023-07-14T11:40:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 msgr2=0x7f6a14197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:07.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 0x7f6a14197ee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009810 tx=0x7f6a0400fa00 comp rx=0 tx=0).stop 2023-07-14T11:40:07.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 shutdown_connections 2023-07-14T11:40:07.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6a000a12c0 0x7f6a000a3770 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a14105b60 0x7f6a1419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a140ffb30 0x7f6a14197ee0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 --2- 172.21.15.161:0/507622281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6a141004e0 0x7f6a14198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:07.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 >> 172.21.15.161:0/507622281 conn(0x7f6a140fb5f0 msgr2=0x7f6a14072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:07.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 shutdown_connections 2023-07-14T11:40:07.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:07.656+0000 7f6a1a5ff700 1 -- 172.21.15.161:0/507622281 wait complete. 2023-07-14T11:40:07.672 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-14T11:40:07.686 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-14T11:40:07.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:07 smithi161 ceph-mon[95526]: osdmap e116: 12 total, 12 up, 12 in 2023-07-14T11:40:07.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:07 smithi175 ceph-mon[129081]: osdmap e116: 12 total, 12 up, 12 in 2023-07-14T11:40:07.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:07 smithi182 ceph-mon[129366]: osdmap e116: 12 total, 12 up, 12 in 2023-07-14T11:40:08.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:08 smithi161 ceph-mon[95526]: from='client.27655 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:40:08.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:08 smithi161 ceph-mon[95526]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2023-07-14T11:40:08.739 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:08 smithi161 ceph-mon[95526]: from='client.27623 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:40:08.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:08 smithi175 ceph-mon[129081]: from='client.27655 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:40:08.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:08 smithi175 ceph-mon[129081]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2023-07-14T11:40:08.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:08 smithi175 ceph-mon[129081]: from='client.27623 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:40:08.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:08 smithi182 ceph-mon[129366]: from='client.27655 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:40:08.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:08 smithi182 ceph-mon[129366]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2023-07-14T11:40:08.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:08 smithi182 ceph-mon[129366]: from='client.27623 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:40:09.829 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:40:10.041 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.045+0000 7f9816fff700 1 -- 172.21.15.161:0/1387219778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98100ffae0 msgr2=0x7f98100fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.045+0000 7f9816fff700 1 --2- 172.21.15.161:0/1387219778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98100ffae0 0x7f98100fff50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009990 tx=0x7f97fc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.045+0000 7f9816fff700 1 -- 172.21.15.161:0/1387219778 shutdown_connections 2023-07-14T11:40:10.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.045+0000 7f9816fff700 1 --2- 172.21.15.161:0/1387219778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9810100490 0x7f981010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.045+0000 7f9816fff700 1 --2- 172.21.15.161:0/1387219778 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9810105af0 0x7f9810105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.045+0000 7f9816fff700 1 --2- 172.21.15.161:0/1387219778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98100ffae0 0x7f98100fff50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.045+0000 7f9816fff700 1 -- 172.21.15.161:0/1387219778 >> 172.21.15.161:0/1387219778 conn(0x7f98100fb5e0 msgr2=0x7f98100fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:10.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.046+0000 7f9816fff700 1 -- 172.21.15.161:0/1387219778 shutdown_connections 2023-07-14T11:40:10.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.046+0000 7f9816fff700 1 -- 172.21.15.161:0/1387219778 wait complete. 2023-07-14T11:40:10.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 Processor -- start 2023-07-14T11:40:10.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 -- start start 2023-07-14T11:40:10.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98100ffae0 0x7f9810197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9810100490 0x7f9810198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9810105af0 0x7f981019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981019ce10 con 0x7f98100ffae0 2023-07-14T11:40:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981019cf80 con 0x7f9810100490 2023-07-14T11:40:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9816fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981019d140 con 0x7f9810105af0 2023-07-14T11:40:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f980ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9810100490 0x7f9810198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f980ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9810100490 0x7f9810198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43604/0 (socket says 172.21.15.161:43604) 2023-07-14T11:40:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f981559c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9810105af0 0x7f981019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f980ffff700 1 -- 172.21.15.161:0/1669411987 learned_addr learned my addr 172.21.15.161:0/1669411987 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.047+0000 7f9814d9b700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98100ffae0 0x7f9810197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f981559c700 1 -- 172.21.15.161:0/1669411987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9810100490 msgr2=0x7f9810198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f981559c700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9810100490 0x7f9810198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f981559c700 1 -- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98100ffae0 msgr2=0x7f9810197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f981559c700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98100ffae0 0x7f9810197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f981559c700 1 -- 172.21.15.161:0/1669411987 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0095f0 con 0x7f9810105af0 2023-07-14T11:40:10.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f980ffff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9810100490 0x7f9810198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:10.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f9814d9b700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98100ffae0 0x7f9810197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:10.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f981559c700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9810105af0 0x7f981019c7d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f980800efc0 tx=0x7f980800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:10.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f980dffb700 1 -- 172.21.15.161:0/1669411987 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808018070 con 0x7f9810105af0 2023-07-14T11:40:10.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981019d3c0 con 0x7f9810105af0 2023-07-14T11:40:10.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f980dffb700 1 -- 172.21.15.161:0/1669411987 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980800f040 con 0x7f9810105af0 2023-07-14T11:40:10.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f980dffb700 1 -- 172.21.15.161:0/1669411987 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800cd40 con 0x7f9810105af0 2023-07-14T11:40:10.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.048+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101a4c60 con 0x7f9810105af0 2023-07-14T11:40:10.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.052+0000 7f980dffb700 1 -- 172.21.15.161:0/1669411987 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9808003a00 con 0x7f9810105af0 2023-07-14T11:40:10.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 Processor -- start 2023-07-14T11:40:10.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 -- start start 2023-07-14T11:40:10.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f980dffb700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98000a12c0 0x7f98000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f802d0 0x55df37f806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f980dffb700 1 -- 172.21.15.161:0/1669411987 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f98080ccd20 con 0x7f9810105af0 2023-07-14T11:40:10.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f8bad0 0x55df37f8deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37e86cb0 con 0x55df37f802d0 2023-07-14T11:40:10.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37ed9a40 con 0x55df37f80be0 2023-07-14T11:40:10.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f982a838b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37ed2ed0 con 0x55df37f8bad0 2023-07-14T11:40:10.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f980ffff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98000a12c0 0x7f98000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f9814d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f802d0 0x55df37f806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.053+0000 7f9814d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f802d0 0x55df37f806a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39408/0 (socket says 172.21.15.161:39408) 2023-07-14T11:40:10.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f9814d9b700 1 -- 172.21.15.161:0/982874992 learned_addr learned my addr 172.21.15.161:0/982874992 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:10.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f981559c700 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f8bad0 0x55df37f8deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f980ffff700 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f981559c700 1 -- 172.21.15.161:0/982874992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 msgr2=0x55df37f8b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f981559c700 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f8b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f981559c700 1 -- 172.21.15.161:0/982874992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f802d0 msgr2=0x55df37f806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f981559c700 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f802d0 0x55df37f806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f981559c700 1 -- 172.21.15.161:0/982874992 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df37d00de0 con 0x55df37f8bad0 2023-07-14T11:40:10.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f9814d9b700 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f802d0 0x55df37f806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:10.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f981559c700 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f8bad0 0x55df37f8deb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f98080ca030 tx=0x7f9808099810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea6f8cbed970f4f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:10.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f980ffff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98000a12c0 0x7f98000a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9810199440 tx=0x7f97fc005c20 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:10.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f97faffd700 1 -- 172.21.15.161:0/982874992 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980809aea0 con 0x55df37f8bad0 2023-07-14T11:40:10.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f97faffd700 1 -- 172.21.15.161:0/982874992 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9808004ad0 con 0x55df37f8bad0 2023-07-14T11:40:10.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.054+0000 7f97faffd700 1 -- 172.21.15.161:0/982874992 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808099f00 con 0x55df37f8bad0 2023-07-14T11:40:10.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 -- 172.21.15.161:0/982874992 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f8bad0 msgr2=0x55df37f8deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f8bad0 0x55df37f8deb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f98080ca030 tx=0x7f9808099810 comp rx=0 tx=0).stop 2023-07-14T11:40:10.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 -- 172.21.15.161:0/982874992 shutdown_connections 2023-07-14T11:40:10.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f802d0 0x55df37f806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f8bad0 0x55df37f8deb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 --2- 172.21.15.161:0/982874992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f8b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 -- 172.21.15.161:0/982874992 >> 172.21.15.161:0/982874992 conn(0x55df37f76da0 msgr2=0x55df37f7c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:10.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 -- 172.21.15.161:0/982874992 shutdown_connections 2023-07-14T11:40:10.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.055+0000 7f982a838b80 1 -- 172.21.15.161:0/982874992 wait complete. 2023-07-14T11:40:10.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 -- start start 2023-07-14T11:40:10.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 Processor -- start 2023-07-14T11:40:10.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f88880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f88fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37e72ac0 con 0x55df37f8bad0 2023-07-14T11:40:10.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37e727c0 con 0x55df37f802d0 2023-07-14T11:40:10.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.056+0000 7f982a838b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37d56920 con 0x55df37f80be0 2023-07-14T11:40:10.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f982a838b80 10 client.0 _ll_register_callbacks cb 0x55df37e70760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:40:10.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f88880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f9814d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f88880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:32888/0 (socket says 172.21.15.161:32888) 2023-07-14T11:40:10.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 -- 172.21.15.161:0/1065989541 learned_addr learned my addr 172.21.15.161:0/1065989541 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:10.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f981559c700 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f88fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 -- 172.21.15.161:0/1065989541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 msgr2=0x55df37f8b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f8b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 -- 172.21.15.161:0/1065989541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 msgr2=0x55df37f88fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f88fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 -- 172.21.15.161:0/1065989541 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9804009d50 con 0x55df37f80be0 2023-07-14T11:40:10.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f981559c700 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f88fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:40:10.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f980ffff700 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f88880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f97fc005f50 tx=0x7f97fc00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcd1fc840919749 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:10.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f97effff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:40:10.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.057+0000 7f97effff700 1 -- 172.21.15.161:0/1065989541 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc02a970 con 0x55df37f80be0 2023-07-14T11:40:10.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df37e2c4f0 con 0x55df37f80be0 2023-07-14T11:40:10.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f97effff700 1 -- 172.21.15.161:0/1065989541 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f97fc02aad0 con 0x55df37f80be0 2023-07-14T11:40:10.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f97effff700 1 -- 172.21.15.161:0/1065989541 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97fc02ac90 con 0x55df37f80be0 2023-07-14T11:40:10.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 10 client.27673 fetch_fsmap learned FSMap version 42 2023-07-14T11:40:10.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f98101a4c60 con 0x55df37f80be0 2023-07-14T11:40:10.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f97effff700 1 -- 172.21.15.161:0/1065989541 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df37e2c4f0 con 0x55df37f80be0 2023-07-14T11:40:10.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f97effff700 1 -- 172.21.15.161:0/1065989541 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f97fc034bc0 con 0x55df37f80be0 2023-07-14T11:40:10.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 10 client.27673 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T11:40:10.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 10 client.27673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:40:10.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 20 client.27673 populate_metadata read hostname 'smithi161' 2023-07-14T11:40:10.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37f9b4e0 0x55df37f9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 4 client.27673 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:40:10.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.058+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df37ecbeb0 con 0x55df37f9b4e0 2023-07-14T11:40:10.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.059+0000 7f9814d9b700 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37f9b4e0 0x55df37f9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.059+0000 7f9814d9b700 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37f9b4e0 0x55df37f9d8f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67124b2427f0d7e server_cookie=9f9fb6895e958f86 in_seq=0 out_seq=0 2023-07-14T11:40:10.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.059+0000 7f97effff700 10 client.27673 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:10.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f97effff700 1 -- 172.21.15.161:0/1065989541 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55df37ecbeb0 con 0x55df37f9b4e0 2023-07-14T11:40:10.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f97effff700 10 client.27673 handle_command_reply: tid=0 2023-07-14T11:40:10.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 client.27673 shutdown 2023-07-14T11:40:10.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 msgr2=0x55df37f88880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f88880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f97fc005f50 tx=0x7f97fc00b260 comp rx=0 tx=0).stop 2023-07-14T11:40:10.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 shutdown_connections 2023-07-14T11:40:10.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37f9b4e0 0x55df37f9d8f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f88fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f97effff700 0 client.27673 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:10.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f8b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 --2- 172.21.15.161:0/1065989541 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f88880 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 >> 172.21.15.161:0/1065989541 conn(0x55df37f76da0 msgr2=0x55df37f7c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:10.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 shutdown_connections 2023-07-14T11:40:10.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 1 -- 172.21.15.161:0/1065989541 wait complete. 2023-07-14T11:40:10.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.060+0000 7f982a838b80 20 client.27673 trim_cache size 0 max 16384 2023-07-14T11:40:10.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.077+0000 7f982a838b80 1 Processor -- start 2023-07-14T11:40:10.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.077+0000 7f982a838b80 1 -- start start 2023-07-14T11:40:10.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f982a838b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f982a838b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f83b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f982a838b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f982a838b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37f86de0 con 0x55df37f8bad0 2023-07-14T11:40:10.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f982a838b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37f811c0 con 0x55df37f802d0 2023-07-14T11:40:10.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f982a838b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37f89aa0 con 0x55df37f80be0 2023-07-14T11:40:10.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f83b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f9814d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f83b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:32900/0 (socket says 172.21.15.161:32900) 2023-07-14T11:40:10.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 -- 172.21.15.161:0/155971618 learned_addr learned my addr 172.21.15.161:0/155971618 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:10.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f981559c700 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 -- 172.21.15.161:0/155971618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 msgr2=0x55df37f924f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f924f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 -- 172.21.15.161:0/155971618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 msgr2=0x55df37f840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f840b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.078+0000 7f980ffff700 1 -- 172.21.15.161:0/155971618 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df37ce9c60 con 0x55df37f80be0 2023-07-14T11:40:10.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f980ffff700 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f83b70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f97fc034ef0 tx=0x7f97fc035f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0101a62222b8eb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:10.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f97fb7fe700 1 -- 172.21.15.161:0/155971618 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00b390 con 0x55df37f80be0 2023-07-14T11:40:10.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f97fb7fe700 1 -- 172.21.15.161:0/155971618 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97fc00bd40 con 0x55df37f80be0 2023-07-14T11:40:10.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f97fb7fe700 1 -- 172.21.15.161:0/155971618 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc017750 con 0x55df37f80be0 2023-07-14T11:40:10.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 -- 172.21.15.161:0/155971618 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 msgr2=0x55df37f83b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f83b70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f97fc034ef0 tx=0x7f97fc035f80 comp rx=0 tx=0).stop 2023-07-14T11:40:10.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 -- 172.21.15.161:0/155971618 shutdown_connections 2023-07-14T11:40:10.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f840b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f802d0 0x55df37f924f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 --2- 172.21.15.161:0/155971618 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f80be0 0x55df37f83b70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 -- 172.21.15.161:0/155971618 >> 172.21.15.161:0/155971618 conn(0x55df37f76da0 msgr2=0x55df37f96c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:10.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 -- 172.21.15.161:0/155971618 shutdown_connections 2023-07-14T11:40:10.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.079+0000 7f982a838b80 1 -- 172.21.15.161:0/155971618 wait complete. 2023-07-14T11:40:10.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 -- start start 2023-07-14T11:40:10.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 Processor -- start 2023-07-14T11:40:10.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f802d0 0x55df37f925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f90920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f90ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37f915d0 con 0x55df37f8bad0 2023-07-14T11:40:10.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37f91740 con 0x55df37f80be0 2023-07-14T11:40:10.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.080+0000 7f982a838b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37fef3b0 con 0x55df37f802d0 2023-07-14T11:40:10.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f982a838b80 10 client.0 _ll_register_callbacks cb 0x7f97fc00f840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:40:10.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f90920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f9814d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f802d0 0x55df37f925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f90920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43648/0 (socket says 172.21.15.161:43648) 2023-07-14T11:40:10.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 -- 172.21.15.161:0/201210924 learned_addr learned my addr 172.21.15.161:0/201210924 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:10.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f9814d9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f802d0 0x55df37f925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:32910/0 (socket says 172.21.15.161:32910) 2023-07-14T11:40:10.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f981559c700 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f90ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 -- 172.21.15.161:0/201210924 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f802d0 msgr2=0x55df37f925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f802d0 0x55df37f925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 -- 172.21.15.161:0/201210924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 msgr2=0x55df37f90ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:10.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f90ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 -- 172.21.15.161:0/201210924 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101a4c60 con 0x55df37f80be0 2023-07-14T11:40:10.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f9814d9b700 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f802d0 0x55df37f925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:10.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f980ffff700 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f90920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006010 tx=0x7f97fc00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c722461219cc784 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:10.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.081+0000 7f97effff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:40:10.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f97effff700 1 -- 172.21.15.161:0/201210924 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc005040 con 0x55df37f80be0 2023-07-14T11:40:10.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df37e8e050 con 0x55df37f80be0 2023-07-14T11:40:10.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f97effff700 1 -- 172.21.15.161:0/201210924 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f97fc0051a0 con 0x55df37f80be0 2023-07-14T11:40:10.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f97effff700 1 -- 172.21.15.161:0/201210924 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97fc005360 con 0x55df37f80be0 2023-07-14T11:40:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 10 client.27641 fetch_fsmap learned FSMap version 42 2023-07-14T11:40:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55df37d27250 con 0x55df37f80be0 2023-07-14T11:40:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f97effff700 1 -- 172.21.15.161:0/201210924 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df37e8e050 con 0x55df37f80be0 2023-07-14T11:40:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f97effff700 1 -- 172.21.15.161:0/201210924 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f97fc0346c0 con 0x55df37f80be0 2023-07-14T11:40:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 10 client.27641 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T11:40:10.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 10 client.27641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:40:10.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 20 client.27641 populate_metadata read hostname 'smithi161' 2023-07-14T11:40:10.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37ff05e0 0x55df37ff29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:10.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 4 client.27641 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:40:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.082+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df37d21f70 con 0x55df37ff05e0 2023-07-14T11:40:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f9814d9b700 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37ff05e0 0x55df37ff29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f97effff700 10 client.27641 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:10.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f9814d9b700 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37ff05e0 0x55df37ff29c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd5ef70c55eca57 server_cookie=2c47a7e4f7a167ea in_seq=0 out_seq=0 2023-07-14T11:40:10.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f97effff700 1 -- 172.21.15.161:0/201210924 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55df37d21f70 con 0x55df37ff05e0 2023-07-14T11:40:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f97effff700 10 client.27641 handle_command_reply: tid=0 2023-07-14T11:40:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 client.27641 shutdown 2023-07-14T11:40:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 msgr2=0x55df37f90920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f90920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006010 tx=0x7f97fc00b410 comp rx=0 tx=0).stop 2023-07-14T11:40:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 shutdown_connections 2023-07-14T11:40:10.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55df37ff05e0 0x55df37ff29c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df37f8bad0 0x55df37f90ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55df37f802d0 0x55df37f925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 --2- 172.21.15.161:0/201210924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df37f80be0 0x55df37f90920 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f97effff700 0 client.27641 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:10.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 >> 172.21.15.161:0/201210924 conn(0x55df37f76da0 msgr2=0x55df37f83b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:10.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.083+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 shutdown_connections 2023-07-14T11:40:10.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f982a838b80 1 -- 172.21.15.161:0/201210924 wait complete. 2023-07-14T11:40:10.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f982a838b80 20 client.27641 trim_cache size 0 max 16384 2023-07-14T11:40:10.119 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:40:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:40:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:40:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:40:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:40:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:40:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:40:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:40:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:40:10.122 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:40:10.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98000a12c0 msgr2=0x7f98000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98000a12c0 0x7f98000a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9810199440 tx=0x7f97fc005c20 comp rx=0 tx=0).stop 2023-07-14T11:40:10.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9810105af0 msgr2=0x7f981019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:10.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9810105af0 0x7f981019c7d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f980800efc0 tx=0x7f980800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 shutdown_connections 2023-07-14T11:40:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f98000a12c0 0x7f98000a3770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98100ffae0 0x7f9810197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9810105af0 0x7f981019c7d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 --2- 172.21.15.161:0/1669411987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9810100490 0x7f9810198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 >> 172.21.15.161:0/1669411987 conn(0x7f98100fb5e0 msgr2=0x7f98100fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:10.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 shutdown_connections 2023-07-14T11:40:10.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:10.084+0000 7f9816fff700 1 -- 172.21.15.161:0/1669411987 wait complete. 2023-07-14T11:40:10.131 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:40:10.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:10 smithi161 ceph-mon[95526]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 224 op/s 2023-07-14T11:40:10.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:10 smithi175 ceph-mon[129081]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 224 op/s 2023-07-14T11:40:10.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:10 smithi182 ceph-mon[129366]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 224 op/s 2023-07-14T11:40:12.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:12 smithi175 ceph-mon[129081]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2023-07-14T11:40:12.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:12 smithi182 ceph-mon[129366]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2023-07-14T11:40:12.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:12 smithi161 ceph-mon[95526]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2023-07-14T11:40:14.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:14 smithi175 ceph-mon[129081]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 613 KiB/s wr, 46 op/s 2023-07-14T11:40:14.456 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:14 smithi182 ceph-mon[129366]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 613 KiB/s wr, 46 op/s 2023-07-14T11:40:14.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:14 smithi161 ceph-mon[95526]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 613 KiB/s wr, 46 op/s 2023-07-14T11:40:16.742 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:16 smithi161 ceph-mon[95526]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 613 KiB/s wr, 46 op/s 2023-07-14T11:40:16.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:16 smithi175 ceph-mon[129081]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 613 KiB/s wr, 46 op/s 2023-07-14T11:40:16.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:16 smithi182 ceph-mon[129366]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 613 KiB/s wr, 46 op/s 2023-07-14T11:40:18.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:18 smithi161 ceph-mon[95526]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.5 MiB/s wr, 91 op/s 2023-07-14T11:40:18.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:18 smithi161 ceph-mon[95526]: osdmap e117: 12 total, 12 up, 12 in 2023-07-14T11:40:18.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:18 smithi175 ceph-mon[129081]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.5 MiB/s wr, 91 op/s 2023-07-14T11:40:18.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:18 smithi175 ceph-mon[129081]: osdmap e117: 12 total, 12 up, 12 in 2023-07-14T11:40:18.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:18 smithi182 ceph-mon[129366]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.5 MiB/s wr, 91 op/s 2023-07-14T11:40:18.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:18 smithi182 ceph-mon[129366]: osdmap e117: 12 total, 12 up, 12 in 2023-07-14T11:40:20.743 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:20 smithi161 ceph-mon[95526]: pgmap v569: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 4.1 MiB/s wr, 93 op/s 2023-07-14T11:40:20.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:20 smithi175 ceph-mon[129081]: pgmap v569: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 4.1 MiB/s wr, 93 op/s 2023-07-14T11:40:20.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:20 smithi182 ceph-mon[129366]: pgmap v569: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 4.1 MiB/s wr, 93 op/s 2023-07-14T11:40:22.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:22 smithi161 ceph-mon[95526]: pgmap v570: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 5.3 MiB/s wr, 116 op/s 2023-07-14T11:40:22.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:22 smithi175 ceph-mon[129081]: pgmap v570: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 5.3 MiB/s wr, 116 op/s 2023-07-14T11:40:22.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:22 smithi182 ceph-mon[129366]: pgmap v570: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 5.3 MiB/s wr, 116 op/s 2023-07-14T11:40:24.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:24 smithi175 ceph-mon[129081]: pgmap v571: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 7.2 MiB/s wr, 184 op/s 2023-07-14T11:40:24.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:24 smithi182 ceph-mon[129366]: pgmap v571: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 7.2 MiB/s wr, 184 op/s 2023-07-14T11:40:24.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:24 smithi161 ceph-mon[95526]: pgmap v571: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 7.2 MiB/s wr, 184 op/s 2023-07-14T11:40:26.738 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:26 smithi161 ceph-mon[95526]: pgmap v572: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 7.2 MiB/s wr, 184 op/s 2023-07-14T11:40:26.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:26 smithi175 ceph-mon[129081]: pgmap v572: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 7.2 MiB/s wr, 184 op/s 2023-07-14T11:40:26.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:26 smithi182 ceph-mon[129366]: pgmap v572: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 7.2 MiB/s wr, 184 op/s 2023-07-14T11:40:28.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:28 smithi161 ceph-mon[95526]: pgmap v573: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 140 op/s 2023-07-14T11:40:28.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:28 smithi175 ceph-mon[129081]: pgmap v573: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 140 op/s 2023-07-14T11:40:28.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:28 smithi182 ceph-mon[129366]: pgmap v573: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 140 op/s 2023-07-14T11:40:30.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:30 smithi161 ceph-mon[95526]: pgmap v574: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 126 op/s 2023-07-14T11:40:30.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:30 smithi175 ceph-mon[129081]: pgmap v574: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 126 op/s 2023-07-14T11:40:30.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:30 smithi182 ceph-mon[129366]: pgmap v574: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 126 op/s 2023-07-14T11:40:32.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:32 smithi161 ceph-mon[95526]: pgmap v575: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2023-07-14T11:40:32.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:32 smithi175 ceph-mon[129081]: pgmap v575: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2023-07-14T11:40:32.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:32 smithi182 ceph-mon[129366]: pgmap v575: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2023-07-14T11:40:34.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:34 smithi175 ceph-mon[129081]: pgmap v576: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2023-07-14T11:40:34.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:34 smithi182 ceph-mon[129366]: pgmap v576: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2023-07-14T11:40:34.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:34 smithi161 ceph-mon[95526]: pgmap v576: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.0 MiB/s wr, 96 op/s 2023-07-14T11:40:36.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:36 smithi161 ceph-mon[95526]: pgmap v577: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 360 KiB/s wr, 36 op/s 2023-07-14T11:40:36.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:36 smithi175 ceph-mon[129081]: pgmap v577: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 360 KiB/s wr, 36 op/s 2023-07-14T11:40:36.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:36 smithi182 ceph-mon[129366]: pgmap v577: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 360 KiB/s wr, 36 op/s 2023-07-14T11:40:38.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:38 smithi161 ceph-mon[95526]: pgmap v578: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 405 KiB/s wr, 42 op/s 2023-07-14T11:40:38.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:38 smithi175 ceph-mon[129081]: pgmap v578: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 405 KiB/s wr, 42 op/s 2023-07-14T11:40:38.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:38 smithi182 ceph-mon[129366]: pgmap v578: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 405 KiB/s wr, 42 op/s 2023-07-14T11:40:40.132 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:40:40.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2166417061 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 msgr2=0x7f9f28105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2166417061 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 0x7f9f28105ec0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18009ca0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2166417061 shutdown_connections 2023-07-14T11:40:40.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2166417061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f28100490 0x7f9f2810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2166417061 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 0x7f9f28105ec0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2166417061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f280ffae0 0x7f9f280fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2166417061 >> 172.21.15.161:0/2166417061 conn(0x7f9f280fb5e0 msgr2=0x7f9f280fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:40.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2166417061 shutdown_connections 2023-07-14T11:40:40.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.356+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2166417061 wait complete. 2023-07-14T11:40:40.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.358+0000 7f9f2dfff700 1 Processor -- start 2023-07-14T11:40:40.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.358+0000 7f9f2dfff700 1 -- start start 2023-07-14T11:40:40.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.358+0000 7f9f2dfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f280ffae0 0x7f9f28072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f2dfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f28100490 0x7f9f28072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f2dfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 0x7f9f281a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f2dfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f28073580 con 0x7f9f280ffae0 2023-07-14T11:40:40.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f2dfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281a42e0 con 0x7f9f28100490 2023-07-14T11:40:40.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f2dfff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281a4420 con 0x7f9f28105af0 2023-07-14T11:40:40.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f277fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f280ffae0 0x7f9f28072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f28100490 0x7f9f28072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f28100490 0x7f9f28072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:53154/0 (socket says 172.21.15.161:53154) 2023-07-14T11:40:40.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 -- 172.21.15.161:0/2354682960 learned_addr learned my addr 172.21.15.161:0/2354682960 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:40.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f27fff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 0x7f9f281a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 -- 172.21.15.161:0/2354682960 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 msgr2=0x7f9f281a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 0x7f9f281a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 -- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f280ffae0 msgr2=0x7f9f28072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f280ffae0 0x7f9f28072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 -- 172.21.15.161:0/2354682960 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180095f0 con 0x7f9f28100490 2023-07-14T11:40:40.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f27fff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 0x7f9f281a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:40.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.359+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f28100490 0x7f9f28072d50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000e9a0 tx=0x7f9f1000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:40.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.360+0000 7f9f257fa700 1 -- 172.21.15.161:0/2354682960 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10009790 con 0x7f9f28100490 2023-07-14T11:40:40.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.360+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f281a4680 con 0x7f9f28100490 2023-07-14T11:40:40.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.360+0000 7f9f257fa700 1 -- 172.21.15.161:0/2354682960 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f10010430 con 0x7f9f28100490 2023-07-14T11:40:40.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.360+0000 7f9f257fa700 1 -- 172.21.15.161:0/2354682960 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10018680 con 0x7f9f28100490 2023-07-14T11:40:40.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.360+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f281a4b60 con 0x7f9f28100490 2023-07-14T11:40:40.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.363+0000 7f9f257fa700 1 -- 172.21.15.161:0/2354682960 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9f100105a0 con 0x7f9f28100490 2023-07-14T11:40:40.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.364+0000 7f9f418e2b80 1 Processor -- start 2023-07-14T11:40:40.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.364+0000 7f9f418e2b80 1 -- start start 2023-07-14T11:40:40.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8c2d0 0x565224d8c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8cbe0 0x565224d97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d99eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224c92cb0 con 0x565224d8cbe0 2023-07-14T11:40:40.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224ce5a40 con 0x565224d97ad0 2023-07-14T11:40:40.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224cdeed0 con 0x565224d8c2d0 2023-07-14T11:40:40.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f27fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8c2d0 0x565224d8c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f27fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8c2d0 0x565224d8c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60768/0 (socket says 172.21.15.161:60768) 2023-07-14T11:40:40.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f27fff700 1 -- 172.21.15.161:0/468277598 learned_addr learned my addr 172.21.15.161:0/468277598 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:40.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f257fa700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9f140a12c0 0x7f9f140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d99eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f277fe700 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8cbe0 0x565224d97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f277fe700 1 -- 172.21.15.161:0/468277598 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8c2d0 msgr2=0x565224d8c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f277fe700 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8c2d0 0x565224d8c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f257fa700 1 -- 172.21.15.161:0/2354682960 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f9f10014070 con 0x7f9f28100490 2023-07-14T11:40:40.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f277fe700 1 -- 172.21.15.161:0/468277598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 msgr2=0x565224d99eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f277fe700 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d99eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f27fff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9f140a12c0 0x7f9f140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f277fe700 1 -- 172.21.15.161:0/468277598 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565224b0cde0 con 0x565224d8cbe0 2023-07-14T11:40:40.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.365+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d99eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:40.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.366+0000 7f9f277fe700 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8cbe0 0x565224d97590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9f18000c00 tx=0x7f9f1800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ba04d17a3eeab2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:40.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.366+0000 7f9f27fff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9f140a12c0 0x7f9f140a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009550 tx=0x7f9f20004b40 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:40.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.366+0000 7f9f1dffb700 1 -- 172.21.15.161:0/468277598 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1801d070 con 0x565224d8cbe0 2023-07-14T11:40:40.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.366+0000 7f9f1dffb700 1 -- 172.21.15.161:0/468277598 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f1801e040 con 0x565224d8cbe0 2023-07-14T11:40:40.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.366+0000 7f9f1dffb700 1 -- 172.21.15.161:0/468277598 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004860 con 0x565224d8cbe0 2023-07-14T11:40:40.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 -- 172.21.15.161:0/468277598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8cbe0 msgr2=0x565224d97590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8cbe0 0x565224d97590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9f18000c00 tx=0x7f9f1800b610 comp rx=0 tx=0).stop 2023-07-14T11:40:40.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 -- 172.21.15.161:0/468277598 shutdown_connections 2023-07-14T11:40:40.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d99eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8c2d0 0x565224d8c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/468277598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8cbe0 0x565224d97590 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 -- 172.21.15.161:0/468277598 >> 172.21.15.161:0/468277598 conn(0x565224d82da0 msgr2=0x565224d886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:40.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 -- 172.21.15.161:0/468277598 shutdown_connections 2023-07-14T11:40:40.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 -- 172.21.15.161:0/468277598 wait complete. 2023-07-14T11:40:40.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.367+0000 7f9f418e2b80 1 -- start start 2023-07-14T11:40:40.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.368+0000 7f9f418e2b80 1 Processor -- start 2023-07-14T11:40:40.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d97510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d94790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224d94ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224c7eac0 con 0x565224d97ad0 2023-07-14T11:40:40.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224c7e7c0 con 0x565224d8c2d0 2023-07-14T11:40:40.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224b62920 con 0x565224d8cbe0 2023-07-14T11:40:40.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f418e2b80 10 client.0 _ll_register_callbacks cb 0x565224c7c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:40:40.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d97510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f1ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d94790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d97510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:53170/0 (socket says 172.21.15.161:53170) 2023-07-14T11:40:40.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 -- 172.21.15.161:0/3360017676 learned_addr learned my addr 172.21.15.161:0/3360017676 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:40.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f27fff700 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224d94ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 -- 172.21.15.161:0/3360017676 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 msgr2=0x565224d94790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d94790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 -- 172.21.15.161:0/3360017676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 msgr2=0x565224d94ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224d94ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 -- 172.21.15.161:0/3360017676 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f281a4b60 con 0x565224d8c2d0 2023-07-14T11:40:40.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d94790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:40.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f277fe700 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d97510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800bf90 tx=0x7f9f1800bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0ed99b73a7d7bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:40.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.369+0000 7f9f06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:40:40.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3360017676 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1801d040 con 0x565224d8c2d0 2023-07-14T11:40:40.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565224c384f0 con 0x565224d8c2d0 2023-07-14T11:40:40.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3360017676 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f9f1800f9f0 con 0x565224d8c2d0 2023-07-14T11:40:40.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3360017676 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f1800fbb0 con 0x565224d8c2d0 2023-07-14T11:40:40.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 10 client.27653 fetch_fsmap learned FSMap version 42 2023-07-14T11:40:40.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f9f2810d220 con 0x565224d8c2d0 2023-07-14T11:40:40.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3360017676 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565224c384f0 con 0x565224d8c2d0 2023-07-14T11:40:40.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3360017676 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f18022400 con 0x565224d8c2d0 2023-07-14T11:40:40.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 10 client.27653 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T11:40:40.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 10 client.27653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:40:40.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 20 client.27653 populate_metadata read hostname 'smithi161' 2023-07-14T11:40:40.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224da6e90 0x565224da9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 4 client.27653 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:40:40.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.370+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565224cd7eb0 con 0x565224da6e90 2023-07-14T11:40:40.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.371+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224da6e90 0x565224da9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.371+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224da6e90 0x565224da9270 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10243435315bec5 server_cookie=d46b2219396e5240 in_seq=0 out_seq=0 2023-07-14T11:40:40.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.371+0000 7f9f06ffd700 10 client.27653 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:40.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3360017676 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565224cd7eb0 con 0x565224da6e90 2023-07-14T11:40:40.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f06ffd700 10 client.27653 handle_command_reply: tid=0 2023-07-14T11:40:40.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 client.27653 shutdown 2023-07-14T11:40:40.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 msgr2=0x565224d97510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d97510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800bf90 tx=0x7f9f1800bfc0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 shutdown_connections 2023-07-14T11:40:40.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224da6e90 0x565224da9270 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224d94ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d97510 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3360017676 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d94790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f06ffd700 0 client.27653 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:40.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 >> 172.21.15.161:0/3360017676 conn(0x565224d82da0 msgr2=0x565224d88230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:40.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 shutdown_connections 2023-07-14T11:40:40.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3360017676 wait complete. 2023-07-14T11:40:40.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.372+0000 7f9f418e2b80 20 client.27653 trim_cache size 0 max 16384 2023-07-14T11:40:40.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.389+0000 7f9f418e2b80 1 Processor -- start 2023-07-14T11:40:40.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.389+0000 7f9f418e2b80 1 -- start start 2023-07-14T11:40:40.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8c2d0 0x565224d97040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d91f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224d8de70 con 0x565224d8c2d0 2023-07-14T11:40:40.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224d89d50 con 0x565224d97ad0 2023-07-14T11:40:40.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224da3d40 con 0x565224d8cbe0 2023-07-14T11:40:40.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8c2d0 0x565224d97040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8c2d0 0x565224d97040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44996/0 (socket says 172.21.15.161:44996) 2023-07-14T11:40:40.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 -- 172.21.15.161:0/1262125315 learned_addr learned my addr 172.21.15.161:0/1262125315 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:40.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d91a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f27fff700 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d91f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 -- 172.21.15.161:0/1262125315 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 msgr2=0x565224d91a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d91a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 -- 172.21.15.161:0/1262125315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 msgr2=0x565224d91f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d91f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f277fe700 1 -- 172.21.15.161:0/1262125315 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565224af5c60 con 0x565224d8c2d0 2023-07-14T11:40:40.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.390+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d91a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:40:40.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.391+0000 7f9f277fe700 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8c2d0 0x565224d97040 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbb78ae5cf5f2e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:40.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.391+0000 7f9f1e7fc700 1 -- 172.21.15.161:0/1262125315 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18022ad0 con 0x565224d8c2d0 2023-07-14T11:40:40.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.391+0000 7f9f1e7fc700 1 -- 172.21.15.161:0/1262125315 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f1800fcb0 con 0x565224d8c2d0 2023-07-14T11:40:40.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.391+0000 7f9f1e7fc700 1 -- 172.21.15.161:0/1262125315 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180349f0 con 0x565224d8c2d0 2023-07-14T11:40:40.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 -- 172.21.15.161:0/1262125315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8c2d0 msgr2=0x565224d97040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8c2d0 0x565224d97040 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18005fd0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 -- 172.21.15.161:0/1262125315 shutdown_connections 2023-07-14T11:40:40.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d8c2d0 0x565224d97040 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d97ad0 0x565224d91f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/1262125315 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d91a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 -- 172.21.15.161:0/1262125315 >> 172.21.15.161:0/1262125315 conn(0x565224d82da0 msgr2=0x565224d8fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:40.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 -- 172.21.15.161:0/1262125315 shutdown_connections 2023-07-14T11:40:40.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 -- 172.21.15.161:0/1262125315 wait complete. 2023-07-14T11:40:40.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 -- start start 2023-07-14T11:40:40.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.392+0000 7f9f418e2b80 1 Processor -- start 2023-07-14T11:40:40.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d9a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d9abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f418e2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224df9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224d9b350 con 0x565224d97ad0 2023-07-14T11:40:40.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224dfa240 con 0x565224d8c2d0 2023-07-14T11:40:40.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f418e2b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x565224dfa400 con 0x565224d8cbe0 2023-07-14T11:40:40.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f418e2b80 10 client.0 _ll_register_callbacks cb 0x7f9f10099d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:40:40.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f1ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d9abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d9a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d9a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:53186/0 (socket says 172.21.15.161:53186) 2023-07-14T11:40:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f1ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d9abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60798/0 (socket says 172.21.15.161:60798) 2023-07-14T11:40:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 -- 172.21.15.161:0/3774097011 learned_addr learned my addr 172.21.15.161:0/3774097011 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:40:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f27fff700 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224df9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 -- 172.21.15.161:0/3774097011 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 msgr2=0x565224d9abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d9abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 -- 172.21.15.161:0/3774097011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 msgr2=0x565224df9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:40:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224df9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.393+0000 7f9f277fe700 1 -- 172.21.15.161:0/3774097011 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f281a4680 con 0x565224d8c2d0 2023-07-14T11:40:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f277fe700 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d9a670 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005f50 tx=0x7f9f18022c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f4d446df65c7055 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:40:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:40:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3774097011 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1801d070 con 0x565224d8c2d0 2023-07-14T11:40:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3774097011 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f9f18000e40 con 0x565224d8c2d0 2023-07-14T11:40:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f1801e040 con 0x565224d8c2d0 2023-07-14T11:40:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3774097011 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f18034460 con 0x565224d8c2d0 2023-07-14T11:40:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f418e2b80 10 client.27659 fetch_fsmap learned FSMap version 42 2023-07-14T11:40:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.394+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f9f2810d220 con 0x565224d8c2d0 2023-07-14T11:40:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3774097011 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f18055020 con 0x565224d8c2d0 2023-07-14T11:40:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3774097011 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f1800b850 con 0x565224d8c2d0 2023-07-14T11:40:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f418e2b80 10 client.27659 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T11:40:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f418e2b80 10 client.27659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:40:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f418e2b80 20 client.27659 populate_metadata read hostname 'smithi161' 2023-07-14T11:40:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224dfb6e0 0x565224dfdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:40:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f418e2b80 4 client.27659 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:40:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565224c9a050 con 0x565224dfb6e0 2023-07-14T11:40:40.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224dfb6e0 0x565224dfdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:40:40.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f06ffd700 10 client.27659 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:40.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f1ffff700 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224dfb6e0 0x565224dfdac0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d507e7169f8cbb02 server_cookie=3e0b13739a28be79 in_seq=0 out_seq=0 2023-07-14T11:40:40.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f06ffd700 1 -- 172.21.15.161:0/3774097011 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x565224c9a050 con 0x565224dfb6e0 2023-07-14T11:40:40.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.395+0000 7f9f06ffd700 10 client.27659 handle_command_reply: tid=0 2023-07-14T11:40:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 client.27659 shutdown 2023-07-14T11:40:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 msgr2=0x565224d9a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d9a670 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005f50 tx=0x7f9f18022c50 comp rx=0 tx=0).stop 2023-07-14T11:40:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 shutdown_connections 2023-07-14T11:40:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565224dfb6e0 0x565224dfdac0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565224d97ad0 0x565224df9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565224d8c2d0 0x565224d9a670 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 --2- 172.21.15.161:0/3774097011 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565224d8cbe0 0x565224d9abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f06ffd700 0 client.27659 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:40:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 >> 172.21.15.161:0/3774097011 conn(0x565224d82da0 msgr2=0x565224d8e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 shutdown_connections 2023-07-14T11:40:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 1 -- 172.21.15.161:0/3774097011 wait complete. 2023-07-14T11:40:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f418e2b80 20 client.27659 trim_cache size 0 max 16384 2023-07-14T11:40:40.412 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:40:40.412 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:40:40.412 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:40:40.413 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:40:40.413 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:40:40.413 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:40:40.413 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:40:40.413 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:40:40.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:40:40.414 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:40:40.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9f140a12c0 msgr2=0x7f9f140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.396+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9f140a12c0 0x7f9f140a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009550 tx=0x7f9f20004b40 comp rx=0 tx=0).stop 2023-07-14T11:40:40.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f28100490 msgr2=0x7f9f28072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:40:40.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f28100490 0x7f9f28072d50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000e9a0 tx=0x7f9f1000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 shutdown_connections 2023-07-14T11:40:40.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f9f140a12c0 0x7f9f140a3770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f280ffae0 0x7f9f28072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9f28105af0 0x7f9f281a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 --2- 172.21.15.161:0/2354682960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f28100490 0x7f9f28072d50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:40:40.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 >> 172.21.15.161:0/2354682960 conn(0x7f9f280fb5e0 msgr2=0x7f9f280fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:40:40.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 shutdown_connections 2023-07-14T11:40:40.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:40:40.397+0000 7f9f2dfff700 1 -- 172.21.15.161:0/2354682960 wait complete. 2023-07-14T11:40:40.418 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:40:40.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:40 smithi161 ceph-mon[95526]: pgmap v579: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-07-14T11:40:40.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:40 smithi175 ceph-mon[129081]: pgmap v579: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-07-14T11:40:40.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:40 smithi182 ceph-mon[129366]: pgmap v579: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-07-14T11:40:42.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:42 smithi161 ceph-mon[95526]: pgmap v580: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-07-14T11:40:42.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:42 smithi175 ceph-mon[129081]: pgmap v580: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-07-14T11:40:42.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:42 smithi182 ceph-mon[129366]: pgmap v580: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-07-14T11:40:44.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:44 smithi175 ceph-mon[129081]: pgmap v581: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-07-14T11:40:44.457 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:44 smithi182 ceph-mon[129366]: pgmap v581: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-07-14T11:40:44.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:44 smithi161 ceph-mon[95526]: pgmap v581: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-07-14T11:40:46.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:46 smithi161 ceph-mon[95526]: pgmap v582: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-07-14T11:40:46.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:46 smithi175 ceph-mon[129081]: pgmap v582: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-07-14T11:40:46.958 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:46 smithi182 ceph-mon[129366]: pgmap v582: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-07-14T11:40:47.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:47 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:40:47.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:47 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:40:47.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:47 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:40:48.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:48 smithi161 ceph-mon[95526]: pgmap v583: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-07-14T11:40:48.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:48 smithi175 ceph-mon[129081]: pgmap v583: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-07-14T11:40:48.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:48 smithi182 ceph-mon[129366]: pgmap v583: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-07-14T11:40:50.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:50 smithi161 ceph-mon[95526]: pgmap v584: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 52 KiB/s wr, 8 op/s 2023-07-14T11:40:50.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:50 smithi175 ceph-mon[129081]: pgmap v584: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 52 KiB/s wr, 8 op/s 2023-07-14T11:40:50.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:50 smithi182 ceph-mon[129366]: pgmap v584: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 52 KiB/s wr, 8 op/s 2023-07-14T11:40:52.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:52 smithi175 ceph-mon[129081]: pgmap v585: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-07-14T11:40:52.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:52 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:40:52.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:52 smithi182 ceph-mon[129366]: pgmap v585: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-07-14T11:40:52.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:52 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:40:52.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:52 smithi161 ceph-mon[95526]: pgmap v585: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-07-14T11:40:52.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:52 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:40:54.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:54 smithi175 ceph-mon[129081]: pgmap v586: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-07-14T11:40:54.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:54 smithi182 ceph-mon[129366]: pgmap v586: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-07-14T11:40:54.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:54 smithi161 ceph-mon[95526]: pgmap v586: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-07-14T11:40:56.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:56 smithi161 ceph-mon[95526]: pgmap v587: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 71 KiB/s wr, 8 op/s 2023-07-14T11:40:56.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:56 smithi175 ceph-mon[129081]: pgmap v587: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 71 KiB/s wr, 8 op/s 2023-07-14T11:40:56.956 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:56 smithi182 ceph-mon[129366]: pgmap v587: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 71 KiB/s wr, 8 op/s 2023-07-14T11:40:58.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:40:58 smithi161 ceph-mon[95526]: pgmap v588: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 97 KiB/s wr, 13 op/s 2023-07-14T11:40:58.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:40:58 smithi175 ceph-mon[129081]: pgmap v588: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 97 KiB/s wr, 13 op/s 2023-07-14T11:40:58.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:40:58 smithi182 ceph-mon[129366]: pgmap v588: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 97 KiB/s wr, 13 op/s 2023-07-14T11:41:00.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:00 smithi161 ceph-mon[95526]: pgmap v589: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 8 op/s 2023-07-14T11:41:00.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:00 smithi175 ceph-mon[129081]: pgmap v589: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 8 op/s 2023-07-14T11:41:00.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:00 smithi182 ceph-mon[129366]: pgmap v589: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 8 op/s 2023-07-14T11:41:02.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:02 smithi161 ceph-mon[95526]: pgmap v590: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-07-14T11:41:02.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:02 smithi175 ceph-mon[129081]: pgmap v590: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-07-14T11:41:02.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:02 smithi182 ceph-mon[129366]: pgmap v590: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-07-14T11:41:04.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:04 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:41:04.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:04 smithi175 ceph-mon[129081]: pgmap v591: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 117 KiB/s wr, 24 op/s 2023-07-14T11:41:04.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:04 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:41:04.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:04 smithi182 ceph-mon[129366]: pgmap v591: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 117 KiB/s wr, 24 op/s 2023-07-14T11:41:04.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:04 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:41:04.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:04 smithi161 ceph-mon[95526]: pgmap v591: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 117 KiB/s wr, 24 op/s 2023-07-14T11:41:05.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:05 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:41:05.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:05 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:41:05.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:05 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:41:05.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:05 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:41:05.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:05 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:41:05.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:05 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:41:05.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:05 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:41:05.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:05 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:41:05.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:05 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:41:06.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:06 smithi175 ceph-mon[129081]: pgmap v592: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 92 KiB/s wr, 22 op/s 2023-07-14T11:41:06.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:06 smithi182 ceph-mon[129366]: pgmap v592: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 92 KiB/s wr, 22 op/s 2023-07-14T11:41:06.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:06 smithi161 ceph-mon[95526]: pgmap v592: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 92 KiB/s wr, 22 op/s 2023-07-14T11:41:07.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:07 smithi175 ceph-mon[129081]: osdmap e118: 12 total, 12 up, 12 in 2023-07-14T11:41:07.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:07 smithi182 ceph-mon[129366]: osdmap e118: 12 total, 12 up, 12 in 2023-07-14T11:41:07.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:07 smithi161 ceph-mon[95526]: osdmap e118: 12 total, 12 up, 12 in 2023-07-14T11:41:08.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:08 smithi175 ceph-mon[129081]: pgmap v594: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 129 KiB/s wr, 27 op/s 2023-07-14T11:41:08.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:08 smithi182 ceph-mon[129366]: pgmap v594: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 129 KiB/s wr, 27 op/s 2023-07-14T11:41:08.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:08 smithi161 ceph-mon[95526]: pgmap v594: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 129 KiB/s wr, 27 op/s 2023-07-14T11:41:10.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:41:10.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 -- 172.21.15.161:0/32021343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510105ae0 msgr2=0x7f8510105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 --2- 172.21.15.161:0/32021343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510105ae0 0x7f8510105eb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8500009990 tx=0x7f8500009ca0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 -- 172.21.15.161:0/32021343 shutdown_connections 2023-07-14T11:41:10.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 --2- 172.21.15.161:0/32021343 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8510100480 0x7f851010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 --2- 172.21.15.161:0/32021343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510105ae0 0x7f8510105eb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 --2- 172.21.15.161:0/32021343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85100ffad0 0x7f85100fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 -- 172.21.15.161:0/32021343 >> 172.21.15.161:0/32021343 conn(0x7f85100fb5f0 msgr2=0x7f85100fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:10.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 -- 172.21.15.161:0/32021343 shutdown_connections 2023-07-14T11:41:10.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.642+0000 7f8515bff700 1 -- 172.21.15.161:0/32021343 wait complete. 2023-07-14T11:41:10.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.644+0000 7f8515bff700 1 Processor -- start 2023-07-14T11:41:10.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.644+0000 7f8515bff700 1 -- start start 2023-07-14T11:41:10.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.644+0000 7f8515bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85100ffad0 0x7f851019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.644+0000 7f8515bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510100480 0x7f851019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f8515bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8510105ae0 0x7f8510195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f8515bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85101960d0 con 0x7f85100ffad0 2023-07-14T11:41:10.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f8515bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510196240 con 0x7f8510100480 2023-07-14T11:41:10.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f8515bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510196400 con 0x7f8510105ae0 2023-07-14T11:41:10.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510100480 0x7f851019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85100ffad0 0x7f851019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8510105ae0 0x7f8510195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85100ffad0 0x7f851019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34924/0 (socket says 172.21.15.161:34924) 2023-07-14T11:41:10.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510100480 0x7f851019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:36668/0 (socket says 172.21.15.161:36668) 2023-07-14T11:41:10.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850f7fe700 1 -- 172.21.15.161:0/2961612066 learned_addr learned my addr 172.21.15.161:0/2961612066 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:10.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850ffff700 1 -- 172.21.15.161:0/2961612066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510100480 msgr2=0x7f851019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850ffff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510100480 0x7f851019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850ffff700 1 -- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85100ffad0 msgr2=0x7f851019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850ffff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85100ffad0 0x7f851019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850ffff700 1 -- 172.21.15.161:0/2961612066 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000095f0 con 0x7f8510105ae0 2023-07-14T11:41:10.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850effd700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510100480 0x7f851019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:41:10.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.645+0000 7f850ffff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8510105ae0 0x7f8510195a60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f850800efc0 tx=0x7f850800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:10.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.646+0000 7f850cff9700 1 -- 172.21.15.161:0/2961612066 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508018070 con 0x7f8510105ae0 2023-07-14T11:41:10.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.646+0000 7f850cff9700 1 -- 172.21.15.161:0/2961612066 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850800f040 con 0x7f8510105ae0 2023-07-14T11:41:10.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.646+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8510196680 con 0x7f8510105ae0 2023-07-14T11:41:10.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.646+0000 7f850cff9700 1 -- 172.21.15.161:0/2961612066 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850800cd40 con 0x7f8510105ae0 2023-07-14T11:41:10.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.646+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85101a29e0 con 0x7f8510105ae0 2023-07-14T11:41:10.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.649+0000 7f850cff9700 1 -- 172.21.15.161:0/2961612066 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8508003a00 con 0x7f8510105ae0 2023-07-14T11:41:10.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.650+0000 7f8529427b80 1 Processor -- start 2023-07-14T11:41:10.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.650+0000 7f8529427b80 1 -- start start 2023-07-14T11:41:10.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.650+0000 7f8529427b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f0c2d0 0x562c30f0c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.650+0000 7f8529427b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0cbe0 0x562c30f1b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f8529427b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f1bee0 0x562c30f1e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f8529427b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30e12cb0 con 0x562c30f0cbe0 2023-07-14T11:41:10.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f8529427b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30e65a40 con 0x562c30f0c2d0 2023-07-14T11:41:10.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f8529427b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30e5eed0 con 0x562c30f1bee0 2023-07-14T11:41:10.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850cff9700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84f80a1340 0x7f84f80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f0c2d0 0x562c30f0c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f0c2d0 0x562c30f0c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:36674/0 (socket says 172.21.15.161:36674) 2023-07-14T11:41:10.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 -- 172.21.15.161:0/1398885707 learned_addr learned my addr 172.21.15.161:0/1398885707 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:10.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850f7fe700 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0cbe0 0x562c30f1b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f1bee0 0x562c30f1e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850cff9700 1 -- 172.21.15.161:0/2961612066 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f85080ccbd0 con 0x7f8510105ae0 2023-07-14T11:41:10.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 -- 172.21.15.161:0/1398885707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f1bee0 msgr2=0x562c30f1e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:10.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f1bee0 0x562c30f1e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 -- 172.21.15.161:0/1398885707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0cbe0 msgr2=0x562c30f1b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:10.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0cbe0 0x562c30f1b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 -- 172.21.15.161:0/1398885707 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c30c8cde0 con 0x562c30f0c2d0 2023-07-14T11:41:10.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850ffff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84f80a1340 0x7f84f80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.651+0000 7f850effd700 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f0c2d0 0x562c30f0c6a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8504011c70 tx=0x7f8504009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac4e3277d2a50cf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:10.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f84f5ffb700 1 -- 172.21.15.161:0/1398885707 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400f6d0 con 0x562c30f0c2d0 2023-07-14T11:41:10.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f84f5ffb700 1 -- 172.21.15.161:0/1398885707 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850400fcf0 con 0x562c30f0c2d0 2023-07-14T11:41:10.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f84f5ffb700 1 -- 172.21.15.161:0/1398885707 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850401fc30 con 0x562c30f0c2d0 2023-07-14T11:41:10.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f8529427b80 1 -- 172.21.15.161:0/1398885707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f0c2d0 msgr2=0x562c30f0c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f850ffff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84f80a1340 0x7f84f80a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f850800c6f0 tx=0x7f8508004dc0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:10.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f8529427b80 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f0c2d0 0x562c30f0c6a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8504011c70 tx=0x7f8504009c30 comp rx=0 tx=0).stop 2023-07-14T11:41:10.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f8529427b80 1 -- 172.21.15.161:0/1398885707 shutdown_connections 2023-07-14T11:41:10.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f8529427b80 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f1bee0 0x562c30f1e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f8529427b80 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f0c2d0 0x562c30f0c6a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f8529427b80 1 --2- 172.21.15.161:0/1398885707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0cbe0 0x562c30f1b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.652+0000 7f8529427b80 1 -- 172.21.15.161:0/1398885707 >> 172.21.15.161:0/1398885707 conn(0x562c30f02da0 msgr2=0x562c30f086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:10.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.653+0000 7f8529427b80 1 -- 172.21.15.161:0/1398885707 shutdown_connections 2023-07-14T11:41:10.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.653+0000 7f8529427b80 1 -- 172.21.15.161:0/1398885707 wait complete. 2023-07-14T11:41:10.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.653+0000 7f8529427b80 1 -- start start 2023-07-14T11:41:10.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.653+0000 7f8529427b80 1 Processor -- start 2023-07-14T11:41:10.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f8529427b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f1b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f8529427b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f18be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f8529427b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f8529427b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30dfeac0 con 0x562c30f0c2d0 2023-07-14T11:41:10.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f8529427b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30dfe7c0 con 0x562c30f1bee0 2023-07-14T11:41:10.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f8529427b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30ce2920 con 0x562c30f0cbe0 2023-07-14T11:41:10.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f8529427b80 10 client.0 _ll_register_callbacks cb 0x562c30dfc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:41:10.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f18be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f18be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45114/0 (socket says 172.21.15.161:45114) 2023-07-14T11:41:10.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 -- 172.21.15.161:0/1183937703 learned_addr learned my addr 172.21.15.161:0/1183937703 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:10.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850effd700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f1b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850effd700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 -- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 msgr2=0x562c30f192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:10.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f192d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 -- 172.21.15.161:0/1183937703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 msgr2=0x562c30f1b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:10.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f1b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850f7fe700 1 -- 172.21.15.161:0/1183937703 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8504019040 con 0x562c30f0cbe0 2023-07-14T11:41:10.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.654+0000 7f850effd700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:41:10.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f850f7fe700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f18be0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8500005f20 tx=0x7f850000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e1e4183f074264 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:10.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f84eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:41:10.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f84eeffd700 1 -- 172.21.15.161:0/1183937703 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001d070 con 0x562c30f0cbe0 2023-07-14T11:41:10.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c30db84f0 con 0x562c30f0cbe0 2023-07-14T11:41:10.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f84eeffd700 1 -- 172.21.15.161:0/1183937703 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f8500004700 con 0x562c30f0cbe0 2023-07-14T11:41:10.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f84eeffd700 1 -- 172.21.15.161:0/1183937703 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85000056f0 con 0x562c30f0cbe0 2023-07-14T11:41:10.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f8529427b80 10 client.27709 fetch_fsmap learned FSMap version 44 2023-07-14T11:41:10.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.655+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562c30c75c60 con 0x562c30f0cbe0 2023-07-14T11:41:10.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f84eeffd700 1 -- 172.21.15.161:0/1183937703 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c30db84f0 con 0x562c30f0cbe0 2023-07-14T11:41:10.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f84eeffd700 1 -- 172.21.15.161:0/1183937703 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f850000f8d0 con 0x562c30f0cbe0 2023-07-14T11:41:10.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f8529427b80 10 client.27709 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:41:10.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f8529427b80 10 client.27709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:41:10.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f8529427b80 20 client.27709 populate_metadata read hostname 'smithi161' 2023-07-14T11:41:10.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f8529427b80 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f2b840 0x562c30f2dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f8529427b80 4 client.27709 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:41:10.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c30e57eb0 con 0x562c30f2b840 2023-07-14T11:41:10.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f850effd700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f2b840 0x562c30f2dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f850effd700 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f2b840 0x562c30f2dc50 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7437289a51ba8b9d server_cookie=3ffa706125e7259c in_seq=0 out_seq=0 2023-07-14T11:41:10.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.656+0000 7f84eeffd700 10 client.27709 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:10.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f84eeffd700 1 -- 172.21.15.161:0/1183937703 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562c30e57eb0 con 0x562c30f2b840 2023-07-14T11:41:10.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f84eeffd700 10 client.27709 handle_command_reply: tid=0 2023-07-14T11:41:10.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 client.27709 shutdown 2023-07-14T11:41:10.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 msgr2=0x562c30f18be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f18be0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8500005f20 tx=0x7f850000b610 comp rx=0 tx=0).stop 2023-07-14T11:41:10.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 shutdown_connections 2023-07-14T11:41:10.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f2b840 0x562c30f2dc50 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f1b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 --2- 172.21.15.161:0/1183937703 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f18be0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f84eeffd700 0 client.27709 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:10.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 >> 172.21.15.161:0/1183937703 conn(0x562c30f02da0 msgr2=0x562c30f08290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:10.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.657+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 shutdown_connections 2023-07-14T11:41:10.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.658+0000 7f8529427b80 1 -- 172.21.15.161:0/1183937703 wait complete. 2023-07-14T11:41:10.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.658+0000 7f8529427b80 20 client.27709 trim_cache size 0 max 16384 2023-07-14T11:41:10.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 Processor -- start 2023-07-14T11:41:10.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 -- start start 2023-07-14T11:41:10.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f18400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f27b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f1b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30f18f60 con 0x562c30f0c2d0 2023-07-14T11:41:10.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30effef0 con 0x562c30f1bee0 2023-07-14T11:41:10.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f8529427b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30f171a0 con 0x562c30f0cbe0 2023-07-14T11:41:10.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f27b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f850effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f18400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f27b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45120/0 (socket says 172.21.15.161:45120) 2023-07-14T11:41:10.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f850f7fe700 1 -- 172.21.15.161:0/429674383 learned_addr learned my addr 172.21.15.161:0/429674383 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:10.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.675+0000 7f850effd700 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f1b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f850f7fe700 1 -- 172.21.15.161:0/429674383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 msgr2=0x562c30f1b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:10.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f850f7fe700 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f1b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f850f7fe700 1 -- 172.21.15.161:0/429674383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 msgr2=0x562c30f18400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:10.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f850f7fe700 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f18400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f850f7fe700 1 -- 172.21.15.161:0/429674383 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c30cb3250 con 0x562c30f0cbe0 2023-07-14T11:41:10.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f850f7fe700 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f27b70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f850000fc60 tx=0x7f8500005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8078a283de0a93d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:10.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f84f67fc700 1 -- 172.21.15.161:0/429674383 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001d070 con 0x562c30f0cbe0 2023-07-14T11:41:10.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f84f67fc700 1 -- 172.21.15.161:0/429674383 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850001e040 con 0x562c30f0cbe0 2023-07-14T11:41:10.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.676+0000 7f84f67fc700 1 -- 172.21.15.161:0/429674383 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850000e740 con 0x562c30f0cbe0 2023-07-14T11:41:10.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 -- 172.21.15.161:0/429674383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 msgr2=0x562c30f27b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f27b70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f850000fc60 tx=0x7f8500005c70 comp rx=0 tx=0).stop 2023-07-14T11:41:10.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 -- 172.21.15.161:0/429674383 shutdown_connections 2023-07-14T11:41:10.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f18400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f1b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 --2- 172.21.15.161:0/429674383 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f27b70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 -- 172.21.15.161:0/429674383 >> 172.21.15.161:0/429674383 conn(0x562c30f02da0 msgr2=0x562c30f06e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:10.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 -- 172.21.15.161:0/429674383 shutdown_connections 2023-07-14T11:41:10.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 -- 172.21.15.161:0/429674383 wait complete. 2023-07-14T11:41:10.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 -- start start 2023-07-14T11:41:10.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.677+0000 7f8529427b80 1 Processor -- start 2023-07-14T11:41:10.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f8529427b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f22900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f8529427b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f20cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f8529427b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f21230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f8529427b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30f21930 con 0x562c30f0c2d0 2023-07-14T11:41:10.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f8529427b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30f21aa0 con 0x562c30f1bee0 2023-07-14T11:41:10.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f8529427b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c30f7f710 con 0x562c30f0cbe0 2023-07-14T11:41:10.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f8529427b80 10 client.0 _ll_register_callbacks cb 0x7f850000bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:41:10.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f20cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f22900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f20cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45126/0 (socket says 172.21.15.161:45126) 2023-07-14T11:41:10.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 -- 172.21.15.161:0/1777344439 learned_addr learned my addr 172.21.15.161:0/1777344439 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:10.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 -- 172.21.15.161:0/1777344439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 msgr2=0x562c30f21230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:41:10.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f21230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 -- 172.21.15.161:0/1777344439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 msgr2=0x562c30f22900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:10.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f22900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850f7fe700 1 -- 172.21.15.161:0/1777344439 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c30c75c60 con 0x562c30f0cbe0 2023-07-14T11:41:10.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.678+0000 7f850effd700 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f22900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:41:10.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f850f7fe700 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f20cf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f850000bdb0 tx=0x7f850000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c961eff9541f2808 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:10.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f84eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:41:10.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f84eeffd700 1 -- 172.21.15.161:0/1777344439 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001d040 con 0x562c30f0cbe0 2023-07-14T11:41:10.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f850400fcf0 con 0x562c30f0cbe0 2023-07-14T11:41:10.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f84eeffd700 1 -- 172.21.15.161:0/1777344439 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f850000f870 con 0x562c30f0cbe0 2023-07-14T11:41:10.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f84eeffd700 1 -- 172.21.15.161:0/1777344439 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f850000fa30 con 0x562c30f0cbe0 2023-07-14T11:41:10.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f8529427b80 10 client.27715 fetch_fsmap learned FSMap version 44 2023-07-14T11:41:10.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562c30c8b2b0 con 0x562c30f0cbe0 2023-07-14T11:41:10.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f84eeffd700 1 -- 172.21.15.161:0/1777344439 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850400fcf0 con 0x562c30f0cbe0 2023-07-14T11:41:10.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.679+0000 7f84eeffd700 1 -- 172.21.15.161:0/1777344439 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f85000046c0 con 0x562c30f0cbe0 2023-07-14T11:41:10.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f8529427b80 10 client.27715 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:41:10.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f8529427b80 10 client.27715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:41:10.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f8529427b80 20 client.27715 populate_metadata read hostname 'smithi161' 2023-07-14T11:41:10.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f8529427b80 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f80920 0x562c30f82d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:10.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f8529427b80 4 client.27715 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:41:10.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c30e1a050 con 0x562c30f80920 2023-07-14T11:41:10.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f850effd700 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f80920 0x562c30f82d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:10.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f850effd700 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f80920 0x562c30f82d00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a06e647adfb87f server_cookie=b58c9434ea7cbdbc in_seq=0 out_seq=0 2023-07-14T11:41:10.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f84eeffd700 10 client.27715 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:10.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f84eeffd700 1 -- 172.21.15.161:0/1777344439 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562c30e1a050 con 0x562c30f80920 2023-07-14T11:41:10.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f84eeffd700 10 client.27715 handle_command_reply: tid=0 2023-07-14T11:41:10.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.680+0000 7f8529427b80 1 client.27715 shutdown 2023-07-14T11:41:10.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 msgr2=0x562c30f20cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f20cf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f850000bdb0 tx=0x7f850000b610 comp rx=0 tx=0).stop 2023-07-14T11:41:10.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 shutdown_connections 2023-07-14T11:41:10.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x562c30f80920 0x562c30f82d00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c30f0c2d0 0x562c30f22900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f84eeffd700 0 client.27715 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:10.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c30f1bee0 0x562c30f21230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 --2- 172.21.15.161:0/1777344439 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x562c30f0cbe0 0x562c30f20cf0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 >> 172.21.15.161:0/1777344439 conn(0x562c30f02da0 msgr2=0x562c30f13f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:10.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 shutdown_connections 2023-07-14T11:41:10.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 1 -- 172.21.15.161:0/1777344439 wait complete. 2023-07-14T11:41:10.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8529427b80 20 client.27715 trim_cache size 0 max 16384 2023-07-14T11:41:10.692 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:41:10.693 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T11:41:10.693 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:41:10.693 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:41:10.693 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:41:10.694 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:41:10.694 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:41:10.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:41:10.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:41:10.694 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:41:10.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84f80a1340 msgr2=0x7f84f80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8515bff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84f80a1340 0x7f84f80a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f850800c6f0 tx=0x7f8508004dc0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8510105ae0 msgr2=0x7f8510195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:10.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.681+0000 7f8515bff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8510105ae0 0x7f8510195a60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f850800efc0 tx=0x7f850800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 shutdown_connections 2023-07-14T11:41:10.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f84f80a1340 0x7f84f80a37f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85100ffad0 0x7f851019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8510105ae0 0x7f8510195a60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 --2- 172.21.15.161:0/2961612066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8510100480 0x7f851019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:10.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 >> 172.21.15.161:0/2961612066 conn(0x7f85100fb5f0 msgr2=0x7f85100fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:10.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 shutdown_connections 2023-07-14T11:41:10.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:10.682+0000 7f8515bff700 1 -- 172.21.15.161:0/2961612066 wait complete. 2023-07-14T11:41:10.698 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:41:10.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:10 smithi161 ceph-mon[95526]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 129 KiB/s wr, 27 op/s 2023-07-14T11:41:10.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:10 smithi175 ceph-mon[129081]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 129 KiB/s wr, 27 op/s 2023-07-14T11:41:10.959 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:10 smithi182 ceph-mon[129366]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 129 KiB/s wr, 27 op/s 2023-07-14T11:41:12.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:12 smithi161 ceph-mon[95526]: pgmap v596: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 228 KiB/s wr, 30 op/s 2023-07-14T11:41:12.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:12 smithi175 ceph-mon[129081]: pgmap v596: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 228 KiB/s wr, 30 op/s 2023-07-14T11:41:12.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:12 smithi182 ceph-mon[129366]: pgmap v596: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 228 KiB/s wr, 30 op/s 2023-07-14T11:41:14.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:14 smithi175 ceph-mon[129081]: osdmap e119: 12 total, 12 up, 12 in 2023-07-14T11:41:14.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:14 smithi175 ceph-mon[129081]: pgmap v598: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 23 op/s 2023-07-14T11:41:14.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:14 smithi182 ceph-mon[129366]: osdmap e119: 12 total, 12 up, 12 in 2023-07-14T11:41:14.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:14 smithi182 ceph-mon[129366]: pgmap v598: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 23 op/s 2023-07-14T11:41:14.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:14 smithi161 ceph-mon[95526]: osdmap e119: 12 total, 12 up, 12 in 2023-07-14T11:41:14.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:14 smithi161 ceph-mon[95526]: pgmap v598: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 23 op/s 2023-07-14T11:41:16.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:16 smithi161 ceph-mon[95526]: pgmap v599: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 20 op/s 2023-07-14T11:41:16.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:16 smithi175 ceph-mon[129081]: pgmap v599: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 20 op/s 2023-07-14T11:41:16.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:16 smithi182 ceph-mon[129366]: pgmap v599: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 20 op/s 2023-07-14T11:41:18.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:18 smithi161 ceph-mon[95526]: pgmap v600: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 270 KiB/s wr, 18 op/s 2023-07-14T11:41:18.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:18 smithi175 ceph-mon[129081]: pgmap v600: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 270 KiB/s wr, 18 op/s 2023-07-14T11:41:18.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:18 smithi182 ceph-mon[129366]: pgmap v600: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 270 KiB/s wr, 18 op/s 2023-07-14T11:41:20.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:20 smithi161 ceph-mon[95526]: pgmap v601: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 270 KiB/s wr, 18 op/s 2023-07-14T11:41:20.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:20 smithi175 ceph-mon[129081]: pgmap v601: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 270 KiB/s wr, 18 op/s 2023-07-14T11:41:20.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:20 smithi182 ceph-mon[129366]: pgmap v601: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 270 KiB/s wr, 18 op/s 2023-07-14T11:41:21.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:21 smithi175 ceph-mon[129081]: scrub summary: idle+waiting paths [/] 2023-07-14T11:41:21.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:21 smithi182 ceph-mon[129366]: scrub summary: idle+waiting paths [/] 2023-07-14T11:41:21.986 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:21 smithi161 ceph-mon[95526]: scrub summary: idle+waiting paths [/] 2023-07-14T11:41:22.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:22 smithi175 ceph-mon[129081]: scrub summary: active paths [/] 2023-07-14T11:41:22.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:22 smithi175 ceph-mon[129081]: pgmap v602: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 188 KiB/s wr, 16 op/s 2023-07-14T11:41:22.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:22 smithi182 ceph-mon[129366]: scrub summary: active paths [/] 2023-07-14T11:41:22.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:22 smithi182 ceph-mon[129366]: pgmap v602: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 188 KiB/s wr, 16 op/s 2023-07-14T11:41:22.986 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:22 smithi161 ceph-mon[95526]: scrub summary: active paths [/] 2023-07-14T11:41:22.987 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:22 smithi161 ceph-mon[95526]: pgmap v602: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 188 KiB/s wr, 16 op/s 2023-07-14T11:41:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:24 smithi175 ceph-mon[129081]: pgmap v603: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-07-14T11:41:24.456 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:24 smithi182 ceph-mon[129366]: pgmap v603: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-07-14T11:41:24.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:24 smithi161 ceph-mon[95526]: pgmap v603: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-07-14T11:41:26.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:26 smithi161 ceph-mon[95526]: pgmap v604: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 103 KiB/s wr, 9 op/s 2023-07-14T11:41:26.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:26 smithi175 ceph-mon[129081]: pgmap v604: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 103 KiB/s wr, 9 op/s 2023-07-14T11:41:26.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:26 smithi182 ceph-mon[129366]: pgmap v604: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 103 KiB/s wr, 9 op/s 2023-07-14T11:41:28.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:28 smithi161 ceph-mon[95526]: pgmap v605: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-07-14T11:41:28.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:28 smithi175 ceph-mon[129081]: pgmap v605: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-07-14T11:41:28.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:28 smithi182 ceph-mon[129366]: pgmap v605: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-07-14T11:41:30.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:30 smithi161 ceph-mon[95526]: pgmap v606: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 8 op/s 2023-07-14T11:41:30.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:30 smithi175 ceph-mon[129081]: pgmap v606: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 8 op/s 2023-07-14T11:41:30.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:30 smithi182 ceph-mon[129366]: pgmap v606: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 8 op/s 2023-07-14T11:41:32.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:32 smithi175 ceph-mon[129081]: pgmap v607: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-07-14T11:41:32.955 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:32 smithi182 ceph-mon[129366]: pgmap v607: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-07-14T11:41:32.986 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:32 smithi161 ceph-mon[95526]: pgmap v607: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-07-14T11:41:34.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:34 smithi175 ceph-mon[129081]: pgmap v608: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-07-14T11:41:34.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:34 smithi182 ceph-mon[129366]: pgmap v608: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-07-14T11:41:34.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:34 smithi161 ceph-mon[95526]: pgmap v608: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-07-14T11:41:36.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:36 smithi161 ceph-mon[95526]: pgmap v609: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 KiB/s wr, 7 op/s 2023-07-14T11:41:36.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:36 smithi175 ceph-mon[129081]: pgmap v609: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 KiB/s wr, 7 op/s 2023-07-14T11:41:36.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:36 smithi182 ceph-mon[129366]: pgmap v609: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 KiB/s wr, 7 op/s 2023-07-14T11:41:38.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:38 smithi161 ceph-mon[95526]: pgmap v610: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-07-14T11:41:38.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:38 smithi175 ceph-mon[129081]: pgmap v610: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-07-14T11:41:38.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:38 smithi182 ceph-mon[129366]: pgmap v610: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-07-14T11:41:40.700 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:41:40.771 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:40 smithi161 ceph-mon[95526]: pgmap v611: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 6 op/s 2023-07-14T11:41:40.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:40 smithi175 ceph-mon[129081]: pgmap v611: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 6 op/s 2023-07-14T11:41:40.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 -- 172.21.15.161:0/523744235 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 msgr2=0x7f66e0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 --2- 172.21.15.161:0/523744235 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 0x7f66e0105eb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009990 tx=0x7f66dc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 -- 172.21.15.161:0/523744235 shutdown_connections 2023-07-14T11:41:40.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 --2- 172.21.15.161:0/523744235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e0100480 0x7f66e010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 --2- 172.21.15.161:0/523744235 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 0x7f66e0105eb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 --2- 172.21.15.161:0/523744235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e00ffad0 0x7f66e00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 -- 172.21.15.161:0/523744235 >> 172.21.15.161:0/523744235 conn(0x7f66e00fb5f0 msgr2=0x7f66e00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:40.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 -- 172.21.15.161:0/523744235 shutdown_connections 2023-07-14T11:41:40.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.924+0000 7f66e87ff700 1 -- 172.21.15.161:0/523744235 wait complete. 2023-07-14T11:41:40.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.925+0000 7f66e87ff700 1 Processor -- start 2023-07-14T11:41:40.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e87ff700 1 -- start start 2023-07-14T11:41:40.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e87ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e00ffad0 0x7f66e019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e87ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e0100480 0x7f66e019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e87ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 0x7f66e01959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e87ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0196060 con 0x7f66e00ffad0 2023-07-14T11:41:40.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e87ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e01961d0 con 0x7f66e0100480 2023-07-14T11:41:40.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e87ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0196390 con 0x7f66e0105ae0 2023-07-14T11:41:40.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e659b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e00ffad0 0x7f66e019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e5d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e0100480 0x7f66e019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e6d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 0x7f66e01959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e659b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e00ffad0 0x7f66e019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35876/0 (socket says 172.21.15.161:35876) 2023-07-14T11:41:40.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.926+0000 7f66e659b700 1 -- 172.21.15.161:0/4034934906 learned_addr learned my addr 172.21.15.161:0/4034934906 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:40.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e6d9c700 1 -- 172.21.15.161:0/4034934906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e0100480 msgr2=0x7f66e019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e0100480 0x7f66e019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e6d9c700 1 -- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e00ffad0 msgr2=0x7f66e019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e00ffad0 0x7f66e019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e6d9c700 1 -- 172.21.15.161:0/4034934906 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc0095f0 con 0x7f66e0105ae0 2023-07-14T11:41:40.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e0100480 0x7f66e019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:41:40.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 0x7f66e01959f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f66d800efc0 tx=0x7f66d800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:40.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66d77fe700 1 -- 172.21.15.161:0/4034934906 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8018070 con 0x7f66e0105ae0 2023-07-14T11:41:40.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66d77fe700 1 -- 172.21.15.161:0/4034934906 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d800f040 con 0x7f66e0105ae0 2023-07-14T11:41:40.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e0196610 con 0x7f66e0105ae0 2023-07-14T11:41:40.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66d77fe700 1 -- 172.21.15.161:0/4034934906 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d800cd40 con 0x7f66e0105ae0 2023-07-14T11:41:40.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.927+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e01a2970 con 0x7f66e0105ae0 2023-07-14T11:41:40.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.930+0000 7f66d77fe700 1 -- 172.21.15.161:0/4034934906 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f66d8003a00 con 0x7f66e0105ae0 2023-07-14T11:41:40.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.931+0000 7f66fc1efb80 1 Processor -- start 2023-07-14T11:41:40.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66fc1efb80 1 -- start start 2023-07-14T11:41:40.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66d77fe700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66cc0a1340 0x7f66cc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66d77fe700 1 -- 172.21.15.161:0/4034934906 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f66d80ccb50 con 0x7f66e0105ae0 2023-07-14T11:41:40.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66cc0a1340 0x7f66cc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab356539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab35653ee0 0x55ab356562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab3554acb0 con 0x55ab356442d0 2023-07-14T11:41:40.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab3559da40 con 0x55ab35644be0 2023-07-14T11:41:40.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab35596ed0 con 0x55ab35653ee0 2023-07-14T11:41:40.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66e659b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66e5d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab356539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66e659b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35890/0 (socket says 172.21.15.161:35890) 2023-07-14T11:41:40.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66e5d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab356539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35536/0 (socket says 172.21.15.161:35536) 2023-07-14T11:41:40.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66e6d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab35653ee0 0x55ab356562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.932+0000 7f66e5d9a700 1 -- 172.21.15.161:0/451296444 learned_addr learned my addr 172.21.15.161:0/451296444 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:40.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66e6d9c700 1 -- 172.21.15.161:0/451296444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 msgr2=0x55ab356539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/451296444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab356539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66e6d9c700 1 -- 172.21.15.161:0/451296444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 msgr2=0x55ab356446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/451296444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66e6d9c700 1 -- 172.21.15.161:0/451296444 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab353c4de0 con 0x55ab35653ee0 2023-07-14T11:41:40.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66cc0a1340 0x7f66cc0a37f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f66d00119d0 tx=0x7f66d0011d90 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:40.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/451296444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab35653ee0 0x55ab356562c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f66d80c9ee0 tx=0x7f66d80996c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232b7370e6a29764 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:40.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66d4ff9700 1 -- 172.21.15.161:0/451296444 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8099250 con 0x55ab35653ee0 2023-07-14T11:41:40.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66d4ff9700 1 -- 172.21.15.161:0/451296444 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d80b7e90 con 0x55ab35653ee0 2023-07-14T11:41:40.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.933+0000 7f66d4ff9700 1 -- 172.21.15.161:0/451296444 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d80b90d0 con 0x55ab35653ee0 2023-07-14T11:41:40.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 -- 172.21.15.161:0/451296444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab35653ee0 msgr2=0x55ab356562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/451296444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab35653ee0 0x55ab356562c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f66d80c9ee0 tx=0x7f66d80996c0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 -- 172.21.15.161:0/451296444 shutdown_connections 2023-07-14T11:41:40.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/451296444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/451296444 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab35653ee0 0x55ab356562c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/451296444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab356539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 -- 172.21.15.161:0/451296444 >> 172.21.15.161:0/451296444 conn(0x55ab3563ada0 msgr2=0x55ab356406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:40.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 -- 172.21.15.161:0/451296444 shutdown_connections 2023-07-14T11:41:40.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.934+0000 7f66fc1efb80 1 -- 172.21.15.161:0/451296444 wait complete. 2023-07-14T11:41:40.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 -- start start 2023-07-14T11:41:40.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 Processor -- start 2023-07-14T11:41:40.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab35650c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab35651190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab35653ee0 0x55ab356516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab35536ac0 con 0x55ab35653ee0 2023-07-14T11:41:40.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab355367c0 con 0x55ab35644be0 2023-07-14T11:41:40.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab3541a920 con 0x55ab356442d0 2023-07-14T11:41:40.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66fc1efb80 10 client.0 _ll_register_callbacks cb 0x55ab35534760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:41:40.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.935+0000 7f66e5d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab35651190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab35651190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35542/0 (socket says 172.21.15.161:35542) 2023-07-14T11:41:40.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e659b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab35650c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 -- 172.21.15.161:0/3093366101 learned_addr learned my addr 172.21.15.161:0/3093366101 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:40.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab35653ee0 0x55ab356516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 -- 172.21.15.161:0/3093366101 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 msgr2=0x55ab35650c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab35650c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 -- 172.21.15.161:0/3093366101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab35653ee0 msgr2=0x55ab356516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab35653ee0 0x55ab356516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 -- 172.21.15.161:0/3093366101 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e01a2970 con 0x55ab35644be0 2023-07-14T11:41:40.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab35651190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f66d000fe00 tx=0x7f66d00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6158780668ff6fd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:40.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:41:40.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66c9ffb700 1 -- 172.21.15.161:0/3093366101 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0004030 con 0x55ab35644be0 2023-07-14T11:41:40.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66c9ffb700 1 -- 172.21.15.161:0/3093366101 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f66d0004190 con 0x55ab35644be0 2023-07-14T11:41:40.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.936+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab354f04f0 con 0x55ab35644be0 2023-07-14T11:41:40.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66c9ffb700 1 -- 172.21.15.161:0/3093366101 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66d0004350 con 0x55ab35644be0 2023-07-14T11:41:40.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 10 client.27686 fetch_fsmap learned FSMap version 44 2023-07-14T11:41:40.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55ab353adc60 con 0x55ab35644be0 2023-07-14T11:41:40.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66c9ffb700 1 -- 172.21.15.161:0/3093366101 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab354f04f0 con 0x55ab35644be0 2023-07-14T11:41:40.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66c9ffb700 1 -- 172.21.15.161:0/3093366101 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f66d0026070 con 0x55ab35644be0 2023-07-14T11:41:40.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 10 client.27686 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:41:40.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 10 client.27686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:41:40.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 20 client.27686 populate_metadata read hostname 'smithi161' 2023-07-14T11:41:40.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35663910 0x55ab35665d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 4 client.27686 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:41:40.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab3558feb0 con 0x55ab35663910 2023-07-14T11:41:40.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.937+0000 7f66e659b700 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35663910 0x55ab35665d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.938+0000 7f66e659b700 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35663910 0x55ab35665d20 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b06cff36c636cea server_cookie=75ace39dfc75669f in_seq=0 out_seq=0 2023-07-14T11:41:40.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.938+0000 7f66c9ffb700 10 client.27686 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:40.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.938+0000 7f66c9ffb700 1 -- 172.21.15.161:0/3093366101 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab3558feb0 con 0x55ab35663910 2023-07-14T11:41:40.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66c9ffb700 10 client.27686 handle_command_reply: tid=0 2023-07-14T11:41:40.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 client.27686 shutdown 2023-07-14T11:41:40.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 msgr2=0x55ab35651190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab35651190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f66d000fe00 tx=0x7f66d00047c0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 shutdown_connections 2023-07-14T11:41:40.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35663910 0x55ab35665d20 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab35653ee0 0x55ab356516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab35650c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3093366101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab35644be0 0x55ab35651190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66c9ffb700 0 client.27686 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:40.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 >> 172.21.15.161:0/3093366101 conn(0x55ab3563ada0 msgr2=0x55ab35640230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:40.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 shutdown_connections 2023-07-14T11:41:40.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3093366101 wait complete. 2023-07-14T11:41:40.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.939+0000 7f66fc1efb80 20 client.27686 trim_cache size 0 max 16384 2023-07-14T11:41:40.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 Processor -- start 2023-07-14T11:41:40.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 -- start start 2023-07-14T11:41:40.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563f170 0x55ab3565a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab3563ada0 0x55ab35646af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab3564dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab3565ae40 con 0x55ab3563ada0 2023-07-14T11:41:40.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab35651a50 con 0x55ab3563f170 2023-07-14T11:41:40.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab356448f0 con 0x55ab356442d0 2023-07-14T11:41:40.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66e659b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563f170 0x55ab3565a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66e6d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab3564dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66e659b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563f170 0x55ab3565a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35544/0 (socket says 172.21.15.161:35544) 2023-07-14T11:41:40.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66e6d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab3564dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:48520/0 (socket says 172.21.15.161:48520) 2023-07-14T11:41:40.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66e6d9c700 1 -- 172.21.15.161:0/3988404294 learned_addr learned my addr 172.21.15.161:0/3988404294 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:40.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.956+0000 7f66e5d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab3563ada0 0x55ab35646af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66e6d9c700 1 -- 172.21.15.161:0/3988404294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563f170 msgr2=0x55ab3565a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/3988404294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563f170 0x55ab3565a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66e6d9c700 1 -- 172.21.15.161:0/3988404294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab3563ada0 msgr2=0x55ab35646af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/3988404294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab3563ada0 0x55ab35646af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66e6d9c700 1 -- 172.21.15.161:0/3988404294 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab353eb250 con 0x55ab356442d0 2023-07-14T11:41:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/3988404294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab3564dee0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f66d80c9f40 tx=0x7f66d80060e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a115c6f5b51dd5e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:40.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66d57fa700 1 -- 172.21.15.161:0/3988404294 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d80b2070 con 0x55ab356442d0 2023-07-14T11:41:40.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66d57fa700 1 -- 172.21.15.161:0/3988404294 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d8099250 con 0x55ab356442d0 2023-07-14T11:41:40.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.957+0000 7f66d57fa700 1 -- 172.21.15.161:0/3988404294 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8099b50 con 0x55ab356442d0 2023-07-14T11:41:40.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3988404294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 msgr2=0x55ab3564dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3988404294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab3564dee0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f66d80c9f40 tx=0x7f66d80060e0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3988404294 shutdown_connections 2023-07-14T11:41:40.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3988404294 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab356442d0 0x55ab3564dee0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3988404294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563f170 0x55ab3565a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/3988404294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab3563ada0 0x55ab35646af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3988404294 >> 172.21.15.161:0/3988404294 conn(0x55ab3565ff60 msgr2=0x55ab356463e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:40.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3988404294 shutdown_connections 2023-07-14T11:41:40.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 -- 172.21.15.161:0/3988404294 wait complete. 2023-07-14T11:41:40.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 -- start start 2023-07-14T11:41:40.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.958+0000 7f66fc1efb80 1 Processor -- start 2023-07-14T11:41:40.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563ada0 0x55ab356b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab3563f170 0x55ab356b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66fc1efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab356ac280 con 0x55ab356442d0 2023-07-14T11:41:40.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab356ac3f0 con 0x55ab3563ada0 2023-07-14T11:41:40.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66fc1efb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab356ac5b0 con 0x55ab3563f170 2023-07-14T11:41:40.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66fc1efb80 10 client.0 _ll_register_callbacks cb 0x7f66d000fbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:41:40.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66e659b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563ada0 0x55ab356b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66e659b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563ada0 0x55ab356b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35554/0 (socket says 172.21.15.161:35554) 2023-07-14T11:41:40.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66e659b700 1 -- 172.21.15.161:0/644815223 learned_addr learned my addr 172.21.15.161:0/644815223 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:41:40.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab3563f170 0x55ab356b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66e659b700 1 -- 172.21.15.161:0/644815223 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab3563f170 msgr2=0x55ab356b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.959+0000 7f66e659b700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab3563f170 0x55ab356b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66e659b700 1 -- 172.21.15.161:0/644815223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 msgr2=0x55ab356abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:41:40.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66e659b700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66e659b700 1 -- 172.21.15.161:0/644815223 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab353c4de0 con 0x55ab3563ada0 2023-07-14T11:41:40.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66e6d9c700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:41:40.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66e659b700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563ada0 0x55ab356b1970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f66dc006010 tx=0x7f66dc00bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962c818a2f29b71f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:41:40.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:41:40.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66c9ffb700 1 -- 172.21.15.161:0/644815223 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc01d040 con 0x55ab3563ada0 2023-07-14T11:41:40.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab35660330 con 0x55ab3563ada0 2023-07-14T11:41:40.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66c9ffb700 1 -- 172.21.15.161:0/644815223 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f66dc0290b0 con 0x55ab3563ada0 2023-07-14T11:41:40.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66c9ffb700 1 -- 172.21.15.161:0/644815223 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66dc00bea0 con 0x55ab3563ada0 2023-07-14T11:41:40.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66fc1efb80 10 client.27692 fetch_fsmap learned FSMap version 44 2023-07-14T11:41:40.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.960+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55ab353c32b0 con 0x55ab3563ada0 2023-07-14T11:41:40.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66c9ffb700 1 -- 172.21.15.161:0/644815223 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab35660330 con 0x55ab3563ada0 2023-07-14T11:41:40.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66c9ffb700 1 -- 172.21.15.161:0/644815223 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f66dc004500 con 0x55ab3563ada0 2023-07-14T11:41:40.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66fc1efb80 10 client.27692 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:41:40.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66fc1efb80 10 client.27692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:41:40.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66fc1efb80 20 client.27692 populate_metadata read hostname 'smithi161' 2023-07-14T11:41:40.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35644be0 0x55ab356bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:41:40.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66fc1efb80 4 client.27692 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:41:40.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f66d80b7e90 con 0x55ab35644be0 2023-07-14T11:41:40.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35644be0 0x55ab356bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:41:40.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66c9ffb700 10 client.27692 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:40.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.961+0000 7f66e5d9a700 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35644be0 0x55ab356bb730 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c562dc3ea54b0086 server_cookie=6922cfd7b46d0ee1 in_seq=0 out_seq=0 2023-07-14T11:41:40.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66c9ffb700 1 -- 172.21.15.161:0/644815223 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f66d80b7e90 con 0x55ab35644be0 2023-07-14T11:41:40.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66c9ffb700 10 client.27692 handle_command_reply: tid=0 2023-07-14T11:41:40.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 client.27692 shutdown 2023-07-14T11:41:40.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563ada0 msgr2=0x55ab356b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563ada0 0x55ab356b1970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f66dc006010 tx=0x7f66dc00bbc0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 shutdown_connections 2023-07-14T11:41:40.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ab35644be0 0x55ab356bb730 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab356442d0 0x55ab356abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3563ada0 0x55ab356b1970 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 --2- 172.21.15.161:0/644815223 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ab3563f170 0x55ab356b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66c9ffb700 0 client.27692 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:41:40.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 >> 172.21.15.161:0/644815223 conn(0x55ab3565ff60 msgr2=0x55ab35646c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:40.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 shutdown_connections 2023-07-14T11:41:40.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 1 -- 172.21.15.161:0/644815223 wait complete. 2023-07-14T11:41:40.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.962+0000 7f66fc1efb80 20 client.27692 trim_cache size 0 max 16384 2023-07-14T11:41:40.989 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:41:40.989 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:41:40.990 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:41:40.990 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:41:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:41:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:41:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:41:40.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:41:40.993 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:41:40.993 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:41:40.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66cc0a1340 msgr2=0x7f66cc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.996 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66cc0a1340 0x7f66cc0a37f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f66d00119d0 tx=0x7f66d0011d90 comp rx=0 tx=0).stop 2023-07-14T11:41:40.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 msgr2=0x7f66e01959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:41:40.997 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 0x7f66e01959f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f66d800efc0 tx=0x7f66d800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 shutdown_connections 2023-07-14T11:41:40.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f66cc0a1340 0x7f66cc0a37f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.998 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e00ffad0 0x7f66e019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f66e0105ae0 0x7f66e01959f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:40.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 --2- 172.21.15.161:0/4034934906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e0100480 0x7f66e019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:41:41.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 >> 172.21.15.161:0/4034934906 conn(0x7f66e00fb5f0 msgr2=0x7f66e00fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:41:41.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 shutdown_connections 2023-07-14T11:41:41.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:41:40.963+0000 7f66e87ff700 1 -- 172.21.15.161:0/4034934906 wait complete. 2023-07-14T11:41:41.001 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:41:41.002 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:40 smithi182 ceph-mon[129366]: pgmap v611: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 6 op/s 2023-07-14T11:41:42.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:42 smithi175 ceph-mon[129081]: pgmap v612: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 8 op/s 2023-07-14T11:41:42.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:42 smithi182 ceph-mon[129366]: pgmap v612: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 8 op/s 2023-07-14T11:41:42.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:42 smithi161 ceph-mon[95526]: pgmap v612: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 8 op/s 2023-07-14T11:41:44.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:44 smithi175 ceph-mon[129081]: pgmap v613: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-07-14T11:41:44.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:44 smithi182 ceph-mon[129366]: pgmap v613: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-07-14T11:41:44.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:44 smithi161 ceph-mon[95526]: pgmap v613: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-07-14T11:41:46.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:46 smithi161 ceph-mon[95526]: pgmap v614: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 5 op/s 2023-07-14T11:41:46.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:46 smithi175 ceph-mon[129081]: pgmap v614: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 5 op/s 2023-07-14T11:41:46.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:46 smithi182 ceph-mon[129366]: pgmap v614: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 5 op/s 2023-07-14T11:41:48.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:48 smithi175 ceph-mon[129081]: pgmap v615: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-07-14T11:41:48.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:48 smithi182 ceph-mon[129366]: pgmap v615: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-07-14T11:41:48.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:48 smithi161 ceph-mon[95526]: pgmap v615: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-07-14T11:41:50.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:50 smithi175 ceph-mon[129081]: pgmap v616: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 5 op/s 2023-07-14T11:41:50.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:50 smithi182 ceph-mon[129366]: pgmap v616: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 5 op/s 2023-07-14T11:41:50.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:50 smithi161 ceph-mon[95526]: pgmap v616: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 5 op/s 2023-07-14T11:41:52.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:52 smithi175 ceph-mon[129081]: pgmap v617: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 8 op/s 2023-07-14T11:41:52.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:52 smithi182 ceph-mon[129366]: pgmap v617: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 8 op/s 2023-07-14T11:41:52.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:52 smithi161 ceph-mon[95526]: pgmap v617: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 8 op/s 2023-07-14T11:41:54.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:54 smithi175 ceph-mon[129081]: pgmap v618: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 KiB/s wr, 8 op/s 2023-07-14T11:41:54.453 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:54 smithi182 ceph-mon[129366]: pgmap v618: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 KiB/s wr, 8 op/s 2023-07-14T11:41:54.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:54 smithi161 ceph-mon[95526]: pgmap v618: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 KiB/s wr, 8 op/s 2023-07-14T11:41:56.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:56 smithi175 ceph-mon[129081]: pgmap v619: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 6 op/s 2023-07-14T11:41:56.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:56 smithi182 ceph-mon[129366]: pgmap v619: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 6 op/s 2023-07-14T11:41:56.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:56 smithi161 ceph-mon[95526]: pgmap v619: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 6 op/s 2023-07-14T11:41:58.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:41:58 smithi175 ceph-mon[129081]: pgmap v620: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-07-14T11:41:58.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:41:58 smithi182 ceph-mon[129366]: pgmap v620: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-07-14T11:41:58.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:41:58 smithi161 ceph-mon[95526]: pgmap v620: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-07-14T11:42:00.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:00 smithi175 ceph-mon[129081]: pgmap v621: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 7 op/s 2023-07-14T11:42:00.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:00 smithi182 ceph-mon[129366]: pgmap v621: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 7 op/s 2023-07-14T11:42:00.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:00 smithi161 ceph-mon[95526]: pgmap v621: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 7 op/s 2023-07-14T11:42:02.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:02 smithi175 ceph-mon[129081]: pgmap v622: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 8 op/s 2023-07-14T11:42:02.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:02 smithi182 ceph-mon[129366]: pgmap v622: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 8 op/s 2023-07-14T11:42:02.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:02 smithi161 ceph-mon[95526]: pgmap v622: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 8 op/s 2023-07-14T11:42:04.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:04 smithi175 ceph-mon[129081]: pgmap v623: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 22 op/s 2023-07-14T11:42:04.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:04 smithi182 ceph-mon[129366]: pgmap v623: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 22 op/s 2023-07-14T11:42:04.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:04 smithi161 ceph-mon[95526]: pgmap v623: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 22 op/s 2023-07-14T11:42:05.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:05 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:42:05.453 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:05 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:42:05.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:05 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:42:06.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:06 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:42:06.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:06 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:42:06.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:06 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:42:06.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:06 smithi175 ceph-mon[129081]: pgmap v624: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 21 op/s 2023-07-14T11:42:06.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:06 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:42:06.454 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:06 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:42:06.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:06 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:42:06.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:06 smithi182 ceph-mon[129366]: pgmap v624: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 21 op/s 2023-07-14T11:42:06.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:06 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:42:06.486 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:06 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:42:06.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:06 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:42:06.487 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:06 smithi161 ceph-mon[95526]: pgmap v624: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 21 op/s 2023-07-14T11:42:07.453 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:07 smithi182 ceph-mon[129366]: osdmap e120: 12 total, 12 up, 12 in 2023-07-14T11:42:07.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:07 smithi161 ceph-mon[95526]: osdmap e120: 12 total, 12 up, 12 in 2023-07-14T11:42:07.658 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:07 smithi175 ceph-mon[129081]: osdmap e120: 12 total, 12 up, 12 in 2023-07-14T11:42:08.453 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:08 smithi182 ceph-mon[129366]: pgmap v626: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:08.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:08 smithi161 ceph-mon[95526]: pgmap v626: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:08.658 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:08 smithi175 ceph-mon[129081]: pgmap v626: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:10.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:10 smithi175 ceph-mon[129081]: pgmap v627: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:10.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:10 smithi182 ceph-mon[129366]: pgmap v627: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:10.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:10 smithi161 ceph-mon[95526]: pgmap v627: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:11.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:42:11.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 -- 172.21.15.161:0/304229132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0105af0 msgr2=0x7ff4c0105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/304229132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0105af0 0x7ff4c0105ec0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009990 tx=0x7ff4bc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 -- 172.21.15.161:0/304229132 shutdown_connections 2023-07-14T11:42:11.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/304229132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c0100490 0x7ff4c010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/304229132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0105af0 0x7ff4c0105ec0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/304229132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 0x7ff4c00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 -- 172.21.15.161:0/304229132 >> 172.21.15.161:0/304229132 conn(0x7ff4c00fb5e0 msgr2=0x7ff4c00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:11.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 -- 172.21.15.161:0/304229132 shutdown_connections 2023-07-14T11:42:11.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.225+0000 7ff4c81ff700 1 -- 172.21.15.161:0/304229132 wait complete. 2023-07-14T11:42:11.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 Processor -- start 2023-07-14T11:42:11.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 -- start start 2023-07-14T11:42:11.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 0x7ff4c019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0100490 0x7ff4c019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c0105af0 0x7ff4c0195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c0196080 con 0x7ff4c0105af0 2023-07-14T11:42:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c01961f0 con 0x7ff4c00ffae0 2023-07-14T11:42:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.226+0000 7ff4c81ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c01963b0 con 0x7ff4c0100490 2023-07-14T11:42:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c5f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 0x7ff4c019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c579a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0100490 0x7ff4c019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c5f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 0x7ff4c019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43992/0 (socket says 172.21.15.161:43992) 2023-07-14T11:42:11.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/2862176295 learned_addr learned my addr 172.21.15.161:0/2862176295 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:11.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c679c700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c0105af0 0x7ff4c0195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c579a700 1 -- 172.21.15.161:0/2862176295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 msgr2=0x7ff4c019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 0x7ff4c019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c579a700 1 -- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c0105af0 msgr2=0x7ff4c0195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c0105af0 0x7ff4c0195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c579a700 1 -- 172.21.15.161:0/2862176295 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc0095f0 con 0x7ff4c0100490 2023-07-14T11:42:11.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 0x7ff4c019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:42:11.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0100490 0x7ff4c019bee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e9f0 tx=0x7ff4b000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:11.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4b6ffd700 1 -- 172.21.15.161:0/2862176295 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000c9c0 con 0x7ff4c0100490 2023-07-14T11:42:11.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4b6ffd700 1 -- 172.21.15.161:0/2862176295 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b0010430 con 0x7ff4c0100490 2023-07-14T11:42:11.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4b6ffd700 1 -- 172.21.15.161:0/2862176295 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0018680 con 0x7ff4c0100490 2023-07-14T11:42:11.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.227+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c0196690 con 0x7ff4c0100490 2023-07-14T11:42:11.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.228+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c01a2990 con 0x7ff4c0100490 2023-07-14T11:42:11.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.231+0000 7ff4b6ffd700 1 -- 172.21.15.161:0/2862176295 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff4b0010860 con 0x7ff4c0100490 2023-07-14T11:42:11.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 Processor -- start 2023-07-14T11:42:11.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 -- start start 2023-07-14T11:42:11.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4b6ffd700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff4ac0a1340 0x7ff4ac0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x56123130e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x561231319590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x56123131beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231214cb0 con 0x56123130ebe0 2023-07-14T11:42:11.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231267a40 con 0x56123130e2d0 2023-07-14T11:42:11.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231260ed0 con 0x561231319ad0 2023-07-14T11:42:11.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.232+0000 7ff4b6ffd700 1 -- 172.21.15.161:0/2862176295 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9512+0+0 (secure 0 0 0) 0x7ff4b0014070 con 0x7ff4c0100490 2023-07-14T11:42:11.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x56123130e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c579a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x561231319590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x56123130e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:43998/0 (socket says 172.21.15.161:43998) 2023-07-14T11:42:11.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/3051749729 learned_addr learned my addr 172.21.15.161:0/3051749729 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:11.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c679c700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff4ac0a1340 0x7ff4ac0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c679c700 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x56123131beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/3051749729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 msgr2=0x56123131beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x56123131beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/3051749729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 msgr2=0x561231319590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x561231319590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/3051749729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123108ede0 con 0x56123130e2d0 2023-07-14T11:42:11.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c579a700 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x561231319590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:42:11.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c679c700 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x56123131beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:42:11.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x56123130e6a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00b640 tx=0x7ff4bc00b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdee6c4d085a18fb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:11.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.233+0000 7ff4c679c700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff4ac0a1340 0x7ff4ac0a37f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009ec0 tx=0x7ff4b8009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:11.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4abfff700 1 -- 172.21.15.161:0/3051749729 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc01d070 con 0x56123130e2d0 2023-07-14T11:42:11.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4abfff700 1 -- 172.21.15.161:0/3051749729 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4bc01e040 con 0x56123130e2d0 2023-07-14T11:42:11.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4abfff700 1 -- 172.21.15.161:0/3051749729 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc004dc0 con 0x56123130e2d0 2023-07-14T11:42:11.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/3051749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 msgr2=0x56123130e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x56123130e6a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00b640 tx=0x7ff4bc00b670 comp rx=0 tx=0).stop 2023-07-14T11:42:11.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/3051749729 shutdown_connections 2023-07-14T11:42:11.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x56123131beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x56123130e6a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/3051749729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x561231319590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/3051749729 >> 172.21.15.161:0/3051749729 conn(0x561231304da0 msgr2=0x56123130a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:11.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.234+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/3051749729 shutdown_connections 2023-07-14T11:42:11.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.235+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/3051749729 wait complete. 2023-07-14T11:42:11.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.235+0000 7ff4dbaf6b80 1 -- start start 2023-07-14T11:42:11.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.235+0000 7ff4dbaf6b80 1 Processor -- start 2023-07-14T11:42:11.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130ebe0 0x561231316670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231316c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231200ac0 con 0x56123130e2d0 2023-07-14T11:42:11.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612312007c0 con 0x56123130ebe0 2023-07-14T11:42:11.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612310e4920 con 0x561231319ad0 2023-07-14T11:42:11.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4dbaf6b80 10 client.0 _ll_register_callbacks cb 0x5612311fe760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:42:11.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130ebe0 0x561231316670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130ebe0 0x561231316670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44014/0 (socket says 172.21.15.161:44014) 2023-07-14T11:42:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 -- 172.21.15.161:0/2040996564 learned_addr learned my addr 172.21.15.161:0/2040996564 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c679c700 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231316c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 -- 172.21.15.161:0/2040996564 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 msgr2=0x561231316c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231316c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 -- 172.21.15.161:0/2040996564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 msgr2=0x5612313193f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313193f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.236+0000 7ff4c579a700 1 -- 172.21.15.161:0/2040996564 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c01a2990 con 0x56123130ebe0 2023-07-14T11:42:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130ebe0 0x561231316670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0099bb0 tx=0x7ff4b0099ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc630d026474b90 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:42:11.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2040996564 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00ca9b0 con 0x56123130ebe0 2023-07-14T11:42:11.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2040996564 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9512+0+0 (secure 0 0 0) 0x7ff4b0099eb0 con 0x56123130ebe0 2023-07-14T11:42:11.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612311ba4f0 con 0x56123130ebe0 2023-07-14T11:42:11.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2040996564 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b009dee0 con 0x56123130ebe0 2023-07-14T11:42:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4dbaf6b80 10 client.27704 fetch_fsmap learned FSMap version 44 2023-07-14T11:42:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7ff4c010d220 con 0x56123130ebe0 2023-07-14T11:42:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2040996564 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612311ba4f0 con 0x56123130ebe0 2023-07-14T11:42:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2040996564 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4b00994a0 con 0x56123130ebe0 2023-07-14T11:42:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4dbaf6b80 10 client.27704 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:42:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.237+0000 7ff4dbaf6b80 10 client.27704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:42:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.238+0000 7ff4dbaf6b80 20 client.27704 populate_metadata read hostname 'smithi161' 2023-07-14T11:42:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.238+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56123132d6d0 0x56123132fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.238+0000 7ff4dbaf6b80 4 client.27704 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:42:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.238+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561231259eb0 con 0x56123132d6d0 2023-07-14T11:42:11.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.238+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56123132d6d0 0x56123132fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.238+0000 7ff4a97fa700 10 client.27704 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:11.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.238+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56123132d6d0 0x56123132fae0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=718508c1e3f18ac4 server_cookie=266fa16b3bdf14a7 in_seq=0 out_seq=0 2023-07-14T11:42:11.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2040996564 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561231259eb0 con 0x56123132d6d0 2023-07-14T11:42:11.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4a97fa700 10 client.27704 handle_command_reply: tid=0 2023-07-14T11:42:11.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 client.27704 shutdown 2023-07-14T11:42:11.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130ebe0 msgr2=0x561231316670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130ebe0 0x561231316670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0099bb0 tx=0x7ff4b0099ae0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 shutdown_connections 2023-07-14T11:42:11.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56123132d6d0 0x56123132fae0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231316c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2040996564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130ebe0 0x561231316670 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4a97fa700 0 client.27704 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:11.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 >> 172.21.15.161:0/2040996564 conn(0x561231304da0 msgr2=0x56123130a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:11.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 shutdown_connections 2023-07-14T11:42:11.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.239+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2040996564 wait complete. 2023-07-14T11:42:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.240+0000 7ff4dbaf6b80 20 client.27704 trim_cache size 0 max 16384 2023-07-14T11:42:11.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 Processor -- start 2023-07-14T11:42:11.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 -- start start 2023-07-14T11:42:11.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56123130ebe0 0x5612313119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561231319ad0 0x561231311f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231317cc0 con 0x56123130e2d0 2023-07-14T11:42:11.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123130fd90 con 0x561231319ad0 2023-07-14T11:42:11.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231314c50 con 0x56123130ebe0 2023-07-14T11:42:11.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4c5f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4c679c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561231319ad0 0x561231311f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4c579a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56123130ebe0 0x5612313119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4c679c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561231319ad0 0x561231311f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44018/0 (socket says 172.21.15.161:44018) 2023-07-14T11:42:11.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.257+0000 7ff4c679c700 1 -- 172.21.15.161:0/13090897 learned_addr learned my addr 172.21.15.161:0/13090897 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:11.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4c579a700 1 -- 172.21.15.161:0/13090897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561231319ad0 msgr2=0x561231311f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4c579a700 1 --2- 172.21.15.161:0/13090897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561231319ad0 0x561231311f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4c579a700 1 -- 172.21.15.161:0/13090897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 msgr2=0x5612313111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4c579a700 1 --2- 172.21.15.161:0/13090897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4c579a700 1 -- 172.21.15.161:0/13090897 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561231077c60 con 0x56123130ebe0 2023-07-14T11:42:11.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4c579a700 1 --2- 172.21.15.161:0/13090897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56123130ebe0 0x5612313119f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00058d0 tx=0x7ff4b001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c36f56977781e740 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:11.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4b4ff9700 1 -- 172.21.15.161:0/13090897 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00ca9b0 con 0x56123130ebe0 2023-07-14T11:42:11.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4b4ff9700 1 -- 172.21.15.161:0/13090897 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b0099eb0 con 0x56123130ebe0 2023-07-14T11:42:11.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.258+0000 7ff4b4ff9700 1 -- 172.21.15.161:0/13090897 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b009a350 con 0x56123130ebe0 2023-07-14T11:42:11.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/13090897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56123130ebe0 msgr2=0x5612313119f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/13090897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56123130ebe0 0x5612313119f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00058d0 tx=0x7ff4b001f000 comp rx=0 tx=0).stop 2023-07-14T11:42:11.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/13090897 shutdown_connections 2023-07-14T11:42:11.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/13090897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130e2d0 0x5612313111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/13090897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561231319ad0 0x561231311f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/13090897 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56123130ebe0 0x5612313119f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/13090897 >> 172.21.15.161:0/13090897 conn(0x561231304da0 msgr2=0x561231319020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/13090897 shutdown_connections 2023-07-14T11:42:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/13090897 wait complete. 2023-07-14T11:42:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 -- start start 2023-07-14T11:42:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.259+0000 7ff4dbaf6b80 1 Processor -- start 2023-07-14T11:42:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x561231324730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x5612313229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4dbaf6b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231322ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231323430 con 0x56123130ebe0 2023-07-14T11:42:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612313235a0 con 0x56123130e2d0 2023-07-14T11:42:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4dbaf6b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561231323760 con 0x561231319ad0 2023-07-14T11:42:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4dbaf6b80 10 client.0 _ll_register_callbacks cb 0x7ff4b0099d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:42:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x561231324730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x561231324730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:44028/0 (socket says 172.21.15.161:44028) 2023-07-14T11:42:11.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/2933679711 learned_addr learned my addr 172.21.15.161:0/2933679711 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:11.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c679c700 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231322ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x5612313229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/2933679711 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 msgr2=0x561231322ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231322ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/2933679711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 msgr2=0x5612313229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:11.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x5612313229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.260+0000 7ff4c5f9b700 1 -- 172.21.15.161:0/2933679711 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c010d220 con 0x56123130e2d0 2023-07-14T11:42:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4c5f9b700 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x561231324730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc006010 tx=0x7ff4bc01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1dd83650461b6d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:42:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2933679711 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc032040 con 0x56123130e2d0 2023-07-14T11:42:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2933679711 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9512+0+0 (secure 0 0 0) 0x7ff4bc034860 con 0x56123130e2d0 2023-07-14T11:42:11.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4bc01e040 con 0x56123130e2d0 2023-07-14T11:42:11.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2933679711 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4bc01d040 con 0x56123130e2d0 2023-07-14T11:42:11.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4dbaf6b80 10 client.27710 fetch_fsmap learned FSMap version 44 2023-07-14T11:42:11.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5612310b5250 con 0x56123130e2d0 2023-07-14T11:42:11.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.261+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2933679711 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4bc01e040 con 0x56123130e2d0 2023-07-14T11:42:11.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2933679711 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4bc00f8c0 con 0x56123130e2d0 2023-07-14T11:42:11.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4dbaf6b80 10 client.27710 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:42:11.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4dbaf6b80 10 client.27710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:42:11.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4dbaf6b80 20 client.27710 populate_metadata read hostname 'smithi161' 2023-07-14T11:42:11.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5612313825d0 0x5612313849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:11.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4dbaf6b80 4 client.27710 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:42:11.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56123121c050 con 0x5612313825d0 2023-07-14T11:42:11.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5612313825d0 0x5612313849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:11.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4a97fa700 10 client.27710 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4c579a700 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5612313825d0 0x5612313849e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642e7268b9fa3ab6 server_cookie=d6534516906fe856 in_seq=0 out_seq=0 2023-07-14T11:42:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4a97fa700 1 -- 172.21.15.161:0/2933679711 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56123121c050 con 0x5612313825d0 2023-07-14T11:42:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4a97fa700 10 client.27710 handle_command_reply: tid=0 2023-07-14T11:42:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.262+0000 7ff4dbaf6b80 1 client.27710 shutdown 2023-07-14T11:42:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 msgr2=0x561231324730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x561231324730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc006010 tx=0x7ff4bc01a040 comp rx=0 tx=0).stop 2023-07-14T11:42:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 shutdown_connections 2023-07-14T11:42:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5612313825d0 0x5612313849e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561231319ad0 0x561231322ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123130e2d0 0x561231324730 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 --2- 172.21.15.161:0/2933679711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56123130ebe0 0x5612313229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 >> 172.21.15.161:0/2933679711 conn(0x561231304da0 msgr2=0x5612313098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4a97fa700 0 client.27710 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 shutdown_connections 2023-07-14T11:42:11.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 1 -- 172.21.15.161:0/2933679711 wait complete. 2023-07-14T11:42:11.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.263+0000 7ff4dbaf6b80 20 client.27710 trim_cache size 0 max 16384 2023-07-14T11:42:11.274 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:42:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:42:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:42:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:42:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:42:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:42:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:42:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:42:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:42:11.277 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:42:11.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff4ac0a1340 msgr2=0x7ff4ac0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff4ac0a1340 0x7ff4ac0a37f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009ec0 tx=0x7ff4b8009920 comp rx=0 tx=0).stop 2023-07-14T11:42:11.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0100490 msgr2=0x7ff4c019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:11.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0100490 0x7ff4c019bee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e9f0 tx=0x7ff4b000ed00 comp rx=0 tx=0).stop 2023-07-14T11:42:11.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 shutdown_connections 2023-07-14T11:42:11.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ff4ac0a1340 0x7ff4ac0a37f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.279 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c0105af0 0x7ff4c0195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c00ffae0 0x7ff4c019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 --2- 172.21.15.161:0/2862176295 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ff4c0100490 0x7ff4c019bee0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:11.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 >> 172.21.15.161:0/2862176295 conn(0x7ff4c00fb5e0 msgr2=0x7ff4c00fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:11.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 shutdown_connections 2023-07-14T11:42:11.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:11.264+0000 7ff4c81ff700 1 -- 172.21.15.161:0/2862176295 wait complete. 2023-07-14T11:42:11.281 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:42:12.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:12 smithi175 ceph-mon[129081]: pgmap v628: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:12.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:12 smithi182 ceph-mon[129366]: pgmap v628: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:12.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:12 smithi161 ceph-mon[95526]: pgmap v628: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 57 KiB/s wr, 25 op/s 2023-07-14T11:42:14.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:14 smithi175 ceph-mon[129081]: pgmap v629: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-07-14T11:42:14.455 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:14 smithi182 ceph-mon[129366]: pgmap v629: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-07-14T11:42:14.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:14 smithi161 ceph-mon[95526]: pgmap v629: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-07-14T11:42:16.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:16 smithi175 ceph-mon[129081]: pgmap v630: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-07-14T11:42:16.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:16 smithi182 ceph-mon[129366]: pgmap v630: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-07-14T11:42:16.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:16 smithi161 ceph-mon[95526]: pgmap v630: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-07-14T11:42:18.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:18 smithi175 ceph-mon[129081]: pgmap v631: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-07-14T11:42:18.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:18 smithi175 ceph-mon[129081]: osdmap e121: 12 total, 12 up, 12 in 2023-07-14T11:42:18.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:18 smithi182 ceph-mon[129366]: pgmap v631: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-07-14T11:42:18.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:18 smithi182 ceph-mon[129366]: osdmap e121: 12 total, 12 up, 12 in 2023-07-14T11:42:18.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:18 smithi161 ceph-mon[95526]: pgmap v631: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-07-14T11:42:18.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:18 smithi161 ceph-mon[95526]: osdmap e121: 12 total, 12 up, 12 in 2023-07-14T11:42:20.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:20 smithi175 ceph-mon[129081]: pgmap v633: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:42:20.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:20 smithi182 ceph-mon[129366]: pgmap v633: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:42:20.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:20 smithi161 ceph-mon[95526]: pgmap v633: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:42:22.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:22 smithi175 ceph-mon[129081]: pgmap v634: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:42:22.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:22 smithi182 ceph-mon[129366]: pgmap v634: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:42:22.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:22 smithi161 ceph-mon[95526]: pgmap v634: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:42:24.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:24 smithi175 ceph-mon[129081]: pgmap v635: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:24.460 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:24 smithi182 ceph-mon[129366]: pgmap v635: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:24.484 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:24 smithi161 ceph-mon[95526]: pgmap v635: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:26.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:26 smithi175 ceph-mon[129081]: pgmap v636: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:26.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:26 smithi182 ceph-mon[129366]: pgmap v636: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:26.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:26 smithi161 ceph-mon[95526]: pgmap v636: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:28.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:28 smithi175 ceph-mon[129081]: pgmap v637: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:28.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:28 smithi182 ceph-mon[129366]: pgmap v637: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:28.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:28 smithi161 ceph-mon[95526]: pgmap v637: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:42:30.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:30 smithi175 ceph-mon[129081]: pgmap v638: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:42:30.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:30 smithi182 ceph-mon[129366]: pgmap v638: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:42:30.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:30 smithi161 ceph-mon[95526]: pgmap v638: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:42:32.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:32 smithi175 ceph-mon[129081]: pgmap v639: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:32.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:32 smithi182 ceph-mon[129366]: pgmap v639: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:32.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:32 smithi161 ceph-mon[95526]: pgmap v639: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:34.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:34 smithi175 ceph-mon[129081]: pgmap v640: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:34.456 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:34 smithi182 ceph-mon[129366]: pgmap v640: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:34.484 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:34 smithi161 ceph-mon[95526]: pgmap v640: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:36.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:36 smithi175 ceph-mon[129081]: pgmap v641: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:36.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:36 smithi182 ceph-mon[129366]: pgmap v641: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:36.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:36 smithi161 ceph-mon[95526]: pgmap v641: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:38.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:38 smithi175 ceph-mon[129081]: pgmap v642: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:38.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:38 smithi182 ceph-mon[129366]: pgmap v642: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:38.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:38 smithi161 ceph-mon[95526]: pgmap v642: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:40.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:40 smithi175 ceph-mon[129081]: pgmap v643: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:40.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:40 smithi182 ceph-mon[129366]: pgmap v643: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:40.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:40 smithi161 ceph-mon[95526]: pgmap v643: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:41.282 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:42:41.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 -- 172.21.15.161:0/1107980775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4100490 msgr2=0x7fefb410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/1107980775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4100490 0x7fefb410c920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fefac00b0a0 tx=0x7fefac00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 -- 172.21.15.161:0/1107980775 shutdown_connections 2023-07-14T11:42:41.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/1107980775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb4105af0 0x7fefb4105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/1107980775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4100490 0x7fefb410c920 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/1107980775 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb40ffae0 0x7fefb40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 -- 172.21.15.161:0/1107980775 >> 172.21.15.161:0/1107980775 conn(0x7fefb40fb5e0 msgr2=0x7fefb40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:41.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 -- 172.21.15.161:0/1107980775 shutdown_connections 2023-07-14T11:42:41.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.523+0000 7fefbc1ff700 1 -- 172.21.15.161:0/1107980775 wait complete. 2023-07-14T11:42:41.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.524+0000 7fefbc1ff700 1 Processor -- start 2023-07-14T11:42:41.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.524+0000 7fefbc1ff700 1 -- start start 2023-07-14T11:42:41.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefbc1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb40ffae0 0x7fefb4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefbc1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb4100490 0x7fefb4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefbc1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4105af0 0x7fefb419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefbc1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb419cf00 con 0x7fefb40ffae0 2023-07-14T11:42:41.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefbc1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb419d070 con 0x7fefb4105af0 2023-07-14T11:42:41.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefbc1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb419d230 con 0x7fefb4100490 2023-07-14T11:42:41.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4105af0 0x7fefb419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefb979a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb4100490 0x7fefb4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4105af0 0x7fefb419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:39046/0 (socket says 172.21.15.161:39046) 2023-07-14T11:42:41.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefb9f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb40ffae0 0x7fefb4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 -- 172.21.15.161:0/4131166693 learned_addr learned my addr 172.21.15.161:0/4131166693 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:41.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefb979a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb4100490 0x7fefb4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38158/0 (socket says 172.21.15.161:38158) 2023-07-14T11:42:41.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 -- 172.21.15.161:0/4131166693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb4100490 msgr2=0x7fefb4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb4100490 0x7fefb4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.523 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 -- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb40ffae0 msgr2=0x7fefb4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb40ffae0 0x7fefb4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefba79c700 1 -- 172.21.15.161:0/4131166693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac009d00 con 0x7fefb4105af0 2023-07-14T11:42:41.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.525+0000 7fefb979a700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb4100490 0x7fefb4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:42:41.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.526+0000 7fefba79c700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4105af0 0x7fefb419c8c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fefac009fd0 tx=0x7fefac00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:41.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.526+0000 7fefaaffd700 1 -- 172.21.15.161:0/4131166693 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00e040 con 0x7fefb4105af0 2023-07-14T11:42:41.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.526+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb419d4b0 con 0x7fefb4105af0 2023-07-14T11:42:41.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.526+0000 7fefaaffd700 1 -- 172.21.15.161:0/4131166693 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefac0094c0 con 0x7fefb4105af0 2023-07-14T11:42:41.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.526+0000 7fefaaffd700 1 -- 172.21.15.161:0/4131166693 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0052a0 con 0x7fefb4105af0 2023-07-14T11:42:41.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.526+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb41a4d50 con 0x7fefb4105af0 2023-07-14T11:42:41.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.529+0000 7fefaaffd700 1 -- 172.21.15.161:0/4131166693 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fefac01d470 con 0x7fefb4105af0 2023-07-14T11:42:41.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.530+0000 7fefaaffd700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fefa00a12c0 0x7fefa00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 Processor -- start 2023-07-14T11:42:41.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefaaffd700 1 -- 172.21.15.161:0/4131166693 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fefac0ce330 con 0x7fefb4105af0 2023-07-14T11:42:41.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fefa00a12c0 0x7fefa00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 -- start start 2023-07-14T11:42:41.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f0e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74e09cb0 con 0x563b74f03be0 2023-07-14T11:42:41.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74e5ca40 con 0x563b74f032d0 2023-07-14T11:42:41.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.531+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74e55ed0 con 0x563b74f0ead0 2023-07-14T11:42:41.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefb9f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefb9f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:39062/0 (socket says 172.21.15.161:39062) 2023-07-14T11:42:41.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefb9f9b700 1 -- 172.21.15.161:0/2697054810 learned_addr learned my addr 172.21.15.161:0/2697054810 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:41.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefba79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefb979a700 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f0e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fefa00a12c0 0x7fefa00a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009be0 tx=0x7fefb0009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:41.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefba79c700 1 -- 172.21.15.161:0/2697054810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 msgr2=0x563b74f036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefba79c700 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefba79c700 1 -- 172.21.15.161:0/2697054810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 msgr2=0x563b74f0e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefba79c700 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f0e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefba79c700 1 -- 172.21.15.161:0/2697054810 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b74c83de0 con 0x563b74f0ead0 2023-07-14T11:42:41.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fefba79c700 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f10eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fefac0cb6c0 tx=0x7fefac007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd3cc5324c7ca766 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:41.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fef9ffff700 1 -- 172.21.15.161:0/2697054810 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac09ac10 con 0x563b74f0ead0 2023-07-14T11:42:41.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fef9ffff700 1 -- 172.21.15.161:0/2697054810 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefac09aa30 con 0x563b74f0ead0 2023-07-14T11:42:41.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.532+0000 7fef9ffff700 1 -- 172.21.15.161:0/2697054810 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0cb940 con 0x563b74f0ead0 2023-07-14T11:42:41.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2697054810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 msgr2=0x563b74f10eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f10eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fefac0cb6c0 tx=0x7fefac007930 comp rx=0 tx=0).stop 2023-07-14T11:42:41.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2697054810 shutdown_connections 2023-07-14T11:42:41.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f10eb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2697054810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2697054810 >> 172.21.15.161:0/2697054810 conn(0x563b74ef9da0 msgr2=0x563b74eff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:41.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2697054810 shutdown_connections 2023-07-14T11:42:41.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.533+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2697054810 wait complete. 2023-07-14T11:42:41.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 -- start start 2023-07-14T11:42:41.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 Processor -- start 2023-07-14T11:42:41.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f032d0 0x563b74f0eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f0f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f0ead0 0x563b74f19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74df5ac0 con 0x563b74f0ead0 2023-07-14T11:42:41.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74df57c0 con 0x563b74f03be0 2023-07-14T11:42:41.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74cd9920 con 0x563b74f032d0 2023-07-14T11:42:41.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.534+0000 7fefcfb0fb80 10 client.0 _ll_register_callbacks cb 0x563b74df3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:42:41.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb979a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f0f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb9f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f032d0 0x563b74f0eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb979a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f0f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:39068/0 (socket says 172.21.15.161:39068) 2023-07-14T11:42:41.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb979a700 1 -- 172.21.15.161:0/1448695131 learned_addr learned my addr 172.21.15.161:0/1448695131 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:41.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefba79c700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f0ead0 0x563b74f19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb9f9b700 1 -- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 msgr2=0x563b74f0f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f0f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb9f9b700 1 -- 172.21.15.161:0/1448695131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f0ead0 msgr2=0x563b74f19f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f0ead0 0x563b74f19f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb9f9b700 1 -- 172.21.15.161:0/1448695131 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb41a4d50 con 0x563b74f032d0 2023-07-14T11:42:41.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb979a700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f0f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:42:41.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefba79c700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f0ead0 0x563b74f19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:42:41.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f032d0 0x563b74f0eea0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fefb00117d0 tx=0x7fefb00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a30193f4ef96d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:41.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.535+0000 7fef9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:42:41.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fef9d7fa700 1 -- 172.21.15.161:0/1448695131 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000e5c0 con 0x563b74f032d0 2023-07-14T11:42:41.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b74daf4f0 con 0x563b74f032d0 2023-07-14T11:42:41.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fef9d7fa700 1 -- 172.21.15.161:0/1448695131 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fefb000e720 con 0x563b74f032d0 2023-07-14T11:42:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fef9d7fa700 1 -- 172.21.15.161:0/1448695131 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefb000e8e0 con 0x563b74f032d0 2023-07-14T11:42:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 10 client.27763 fetch_fsmap learned FSMap version 44 2023-07-14T11:42:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x563b74c6cc60 con 0x563b74f032d0 2023-07-14T11:42:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fef9d7fa700 1 -- 172.21.15.161:0/1448695131 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563b74daf4f0 con 0x563b74f032d0 2023-07-14T11:42:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fef9d7fa700 1 -- 172.21.15.161:0/1448695131 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fefb0005020 con 0x563b74f032d0 2023-07-14T11:42:41.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 10 client.27763 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:42:41.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 10 client.27763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:42:41.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 20 client.27763 populate_metadata read hostname 'smithi161' 2023-07-14T11:42:41.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f1e280 0x563b74f20690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 4 client.27763 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:42:41.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.536+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b74e4eeb0 con 0x563b74f1e280 2023-07-14T11:42:41.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.537+0000 7fefb979a700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f1e280 0x563b74f20690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.537+0000 7fefb979a700 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f1e280 0x563b74f20690 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f220c6a44323638d server_cookie=1722bc52f6694260 in_seq=0 out_seq=0 2023-07-14T11:42:41.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.537+0000 7fef9d7fa700 10 client.27763 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:41.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fef9d7fa700 1 -- 172.21.15.161:0/1448695131 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b74e4eeb0 con 0x563b74f1e280 2023-07-14T11:42:41.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fef9d7fa700 10 client.27763 handle_command_reply: tid=0 2023-07-14T11:42:41.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 client.27763 shutdown 2023-07-14T11:42:41.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f032d0 msgr2=0x563b74f0eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f032d0 0x563b74f0eea0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fefb00117d0 tx=0x7fefb00047c0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 shutdown_connections 2023-07-14T11:42:41.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f1e280 0x563b74f20690 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f0ead0 0x563b74f19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fef9d7fa700 0 client.27763 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:41.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f032d0 0x563b74f0eea0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/1448695131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f0f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 >> 172.21.15.161:0/1448695131 conn(0x563b74ef9da0 msgr2=0x563b74eff290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:41.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 shutdown_connections 2023-07-14T11:42:41.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/1448695131 wait complete. 2023-07-14T11:42:41.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.538+0000 7fefcfb0fb80 20 client.27763 trim_cache size 0 max 16384 2023-07-14T11:42:41.554 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.555+0000 7fefcfb0fb80 1 Processor -- start 2023-07-14T11:42:41.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.555+0000 7fefcfb0fb80 1 -- start start 2023-07-14T11:42:41.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.555+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f13e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f19640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.555 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74f04e70 con 0x563b74f03be0 2023-07-14T11:42:41.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74f08a60 con 0x563b74f032d0 2023-07-14T11:42:41.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74f0fad0 con 0x563b74f0ead0 2023-07-14T11:42:41.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefba79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.556 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f19640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefba79c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:38186/0 (socket says 172.21.15.161:38186) 2023-07-14T11:42:41.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefba79c700 1 -- 172.21.15.161:0/2002300052 learned_addr learned my addr 172.21.15.161:0/2002300052 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:41.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f19640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42024/0 (socket says 172.21.15.161:42024) 2023-07-14T11:42:41.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f13e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 -- 172.21.15.161:0/2002300052 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 msgr2=0x563b74f19b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f19b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 -- 172.21.15.161:0/2002300052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 msgr2=0x563b74f13e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f13e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.558 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 -- 172.21.15.161:0/2002300052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b74caa250 con 0x563b74f03be0 2023-07-14T11:42:41.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.556+0000 7fefb979a700 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f19640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x563b74f20f60 tx=0x7fefa4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c81ba6a5ff81db server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:41.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefa8ff9700 1 -- 172.21.15.161:0/2002300052 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4017040 con 0x563b74f03be0 2023-07-14T11:42:41.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefa8ff9700 1 -- 172.21.15.161:0/2002300052 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefa4018040 con 0x563b74f03be0 2023-07-14T11:42:41.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefa8ff9700 1 -- 172.21.15.161:0/2002300052 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004d60 con 0x563b74f03be0 2023-07-14T11:42:41.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2002300052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 msgr2=0x563b74f19640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f19640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x563b74f20f60 tx=0x7fefa4009260 comp rx=0 tx=0).stop 2023-07-14T11:42:41.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2002300052 shutdown_connections 2023-07-14T11:42:41.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f032d0 0x563b74f13e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/2002300052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f03be0 0x563b74f19640 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2002300052 >> 172.21.15.161:0/2002300052 conn(0x563b74ef9da0 msgr2=0x563b74f1a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:41.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2002300052 shutdown_connections 2023-07-14T11:42:41.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.557+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/2002300052 wait complete. 2023-07-14T11:42:41.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.558+0000 7fefcfb0fb80 1 -- start start 2023-07-14T11:42:41.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.558+0000 7fefcfb0fb80 1 Processor -- start 2023-07-14T11:42:41.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.558+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f032d0 0x563b74f13ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.558+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f12180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.558+0000 7fefcfb0fb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74f12dd0 con 0x563b74f032d0 2023-07-14T11:42:41.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74f12f40 con 0x563b74f03be0 2023-07-14T11:42:41.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefcfb0fb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b74f72110 con 0x563b74f0ead0 2023-07-14T11:42:41.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefcfb0fb80 10 client.0 _ll_register_callbacks cb 0x563b74df3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:42:41.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb9f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f032d0 0x563b74f13ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f12180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f12180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:39086/0 (socket says 172.21.15.161:39086) 2023-07-14T11:42:41.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 -- 172.21.15.161:0/392504643 learned_addr learned my addr 172.21.15.161:0/392504643 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:42:41.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefba79c700 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 -- 172.21.15.161:0/392504643 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 msgr2=0x563b74f126c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f126c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 -- 172.21.15.161:0/392504643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f032d0 msgr2=0x563b74f13ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:42:41.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f032d0 0x563b74f13ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 -- 172.21.15.161:0/392504643 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa4012b20 con 0x563b74f03be0 2023-07-14T11:42:41.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fefb979a700 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f12180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009fe0 tx=0x7fefa4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8db6bc237cbdca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:42:41.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.559+0000 7fef9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:42:41.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fef9d7fa700 1 -- 172.21.15.161:0/392504643 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4017070 con 0x563b74f03be0 2023-07-14T11:42:41.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fef9d7fa700 1 -- 172.21.15.161:0/392504643 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fefa4000e40 con 0x563b74f03be0 2023-07-14T11:42:41.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefac09aa30 con 0x563b74f03be0 2023-07-14T11:42:41.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fef9d7fa700 1 -- 172.21.15.161:0/392504643 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa40122d0 con 0x563b74f03be0 2023-07-14T11:42:41.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 10 client.27728 fetch_fsmap learned FSMap version 44 2023-07-14T11:42:41.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x563b74c822b0 con 0x563b74f03be0 2023-07-14T11:42:41.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fef9d7fa700 1 -- 172.21.15.161:0/392504643 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefac09aa30 con 0x563b74f03be0 2023-07-14T11:42:41.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fef9d7fa700 1 -- 172.21.15.161:0/392504643 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fefa400f720 con 0x563b74f03be0 2023-07-14T11:42:41.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 10 client.27728 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:42:41.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 10 client.27728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:42:41.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 20 client.27728 populate_metadata read hostname 'smithi161' 2023-07-14T11:42:41.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f73760 0x563b74f75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:42:41.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 4 client.27728 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:42:41.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.560+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563b74e11050 con 0x563b74f73760 2023-07-14T11:42:41.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f73760 0x563b74f75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:42:41.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fef9d7fa700 10 client.27728 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:41.568 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefb9f9b700 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f73760 0x563b74f75b70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e816714592770eca server_cookie=74257ce9c1c39906 in_seq=0 out_seq=0 2023-07-14T11:42:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fef9d7fa700 1 -- 172.21.15.161:0/392504643 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563b74e11050 con 0x563b74f73760 2023-07-14T11:42:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fef9d7fa700 10 client.27728 handle_command_reply: tid=0 2023-07-14T11:42:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 client.27728 shutdown 2023-07-14T11:42:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 msgr2=0x563b74f12180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f12180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009fe0 tx=0x7fefa4004750 comp rx=0 tx=0).stop 2023-07-14T11:42:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 shutdown_connections 2023-07-14T11:42:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x563b74f73760 0x563b74f75b70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b74f032d0 0x563b74f13ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x563b74f0ead0 0x563b74f126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 --2- 172.21.15.161:0/392504643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b74f03be0 0x563b74f12180 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fef9d7fa700 0 client.27728 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:42:41.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.561+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 >> 172.21.15.161:0/392504643 conn(0x563b74ef9da0 msgr2=0x563b74f073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:41.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 shutdown_connections 2023-07-14T11:42:41.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefcfb0fb80 1 -- 172.21.15.161:0/392504643 wait complete. 2023-07-14T11:42:41.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefcfb0fb80 20 client.27728 trim_cache size 0 max 16384 2023-07-14T11:42:41.572 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:42:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:42:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:42:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:42:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:42:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:42:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:42:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:42:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:42:41.574 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:42:41.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fefa00a12c0 msgr2=0x7fefa00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fefa00a12c0 0x7fefa00a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009be0 tx=0x7fefb0009260 comp rx=0 tx=0).stop 2023-07-14T11:42:41.575 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4105af0 msgr2=0x7fefb419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:42:41.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4105af0 0x7fefb419c8c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fefac009fd0 tx=0x7fefac00bce0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 shutdown_connections 2023-07-14T11:42:41.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fefa00a12c0 0x7fefa00a3770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.576 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb40ffae0 0x7fefb4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefb4105af0 0x7fefb419c8c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 --2- 172.21.15.161:0/4131166693 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fefb4100490 0x7fefb4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:42:41.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 >> 172.21.15.161:0/4131166693 conn(0x7fefb40fb5e0 msgr2=0x7fefb410b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:42:41.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 shutdown_connections 2023-07-14T11:42:41.577 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:42:41.562+0000 7fefbc1ff700 1 -- 172.21.15.161:0/4131166693 wait complete. 2023-07-14T11:42:41.578 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:42:42.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:42 smithi175 ceph-mon[129081]: pgmap v644: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:42.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:42 smithi182 ceph-mon[129366]: pgmap v644: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:42.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:42 smithi161 ceph-mon[95526]: pgmap v644: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:44.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:44 smithi175 ceph-mon[129081]: pgmap v645: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:44.452 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:44 smithi182 ceph-mon[129366]: pgmap v645: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:44.484 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:44 smithi161 ceph-mon[95526]: pgmap v645: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:46.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:46 smithi175 ceph-mon[129081]: pgmap v646: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:46.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:46 smithi182 ceph-mon[129366]: pgmap v646: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:46.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:46 smithi161 ceph-mon[95526]: pgmap v646: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:48.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:48 smithi175 ceph-mon[129081]: pgmap v647: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:48.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:48 smithi182 ceph-mon[129366]: pgmap v647: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:48.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:48 smithi161 ceph-mon[95526]: pgmap v647: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:50.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:50 smithi175 ceph-mon[129081]: pgmap v648: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:50.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:50 smithi182 ceph-mon[129366]: pgmap v648: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:50.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:50 smithi161 ceph-mon[95526]: pgmap v648: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:52.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:52 smithi175 ceph-mon[129081]: pgmap v649: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:52.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:52 smithi182 ceph-mon[129366]: pgmap v649: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:52.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:52 smithi161 ceph-mon[95526]: pgmap v649: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:42:54.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:54 smithi175 ceph-mon[129081]: pgmap v650: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:54.452 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:54 smithi182 ceph-mon[129366]: pgmap v650: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:54.484 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:54 smithi161 ceph-mon[95526]: pgmap v650: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:42:56.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:56 smithi175 ceph-mon[129081]: pgmap v651: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:56.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:56 smithi182 ceph-mon[129366]: pgmap v651: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:56.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:56 smithi161 ceph-mon[95526]: pgmap v651: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:42:58.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:42:58 smithi175 ceph-mon[129081]: pgmap v652: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:58.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:42:58 smithi182 ceph-mon[129366]: pgmap v652: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:42:58.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:42:58 smithi161 ceph-mon[95526]: pgmap v652: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:00.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:00 smithi175 ceph-mon[129081]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:00.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:00 smithi182 ceph-mon[129366]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:00.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:00 smithi161 ceph-mon[95526]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:02.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:02 smithi175 ceph-mon[129081]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-14T11:43:02.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:02 smithi182 ceph-mon[129366]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-14T11:43:02.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:02 smithi161 ceph-mon[95526]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-14T11:43:04.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:04 smithi175 ceph-mon[129081]: pgmap v655: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:43:04.452 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:04 smithi182 ceph-mon[129366]: pgmap v655: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:43:04.484 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:04 smithi161 ceph-mon[95526]: pgmap v655: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:43:05.657 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:05 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:43:05.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:05 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:43:05.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:05 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:43:06.657 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:06 smithi175 ceph-mon[129081]: pgmap v656: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:43:06.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:06 smithi182 ceph-mon[129366]: pgmap v656: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:43:06.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:06 smithi161 ceph-mon[95526]: pgmap v656: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:43:07.657 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:07 smithi175 ceph-mon[129081]: osdmap e122: 12 total, 12 up, 12 in 2023-07-14T11:43:07.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:07 smithi182 ceph-mon[129366]: osdmap e122: 12 total, 12 up, 12 in 2023-07-14T11:43:07.733 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:07 smithi161 ceph-mon[95526]: osdmap e122: 12 total, 12 up, 12 in 2023-07-14T11:43:08.657 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:08 smithi175 ceph-mon[129081]: pgmap v658: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:08.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:08 smithi182 ceph-mon[129366]: pgmap v658: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:08.733 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:08 smithi161 ceph-mon[95526]: pgmap v658: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:09.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:09 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.737 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:09 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:09.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:09.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:09 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:43:10.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:43:10.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:43:10.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:43:10.703 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: pgmap v659: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:10.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.704 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:43:10.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:43:10.705 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:10.733 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:43:10.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:43:10.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:43:10.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:43:10.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: pgmap v659: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:10.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.735 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:43:10.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:43:10.736 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:10.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:43:10.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:43:10.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:43:10.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:43:10.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: pgmap v659: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:10.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:43:10.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:43:10.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:43:10.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:43:11.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:43:11.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:11 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:43:11.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:11 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:43:11.733 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:11 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:43:11.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:11 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:43:11.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.806+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2885542824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd540ffae0 msgr2=0x7fcd540fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.806+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2885542824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd540ffae0 0x7fcd540fff50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009990 tx=0x7fcd44009ca0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.807+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2885542824 shutdown_connections 2023-07-14T11:43:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.807+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2885542824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd54105af0 0x7fcd54105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.807+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2885542824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 0x7fcd5410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.807+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2885542824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd540ffae0 0x7fcd540fff50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.807+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2885542824 >> 172.21.15.161:0/2885542824 conn(0x7fcd540fb5e0 msgr2=0x7fcd540fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:11.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.807+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2885542824 shutdown_connections 2023-07-14T11:43:11.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.807+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2885542824 wait complete. 2023-07-14T11:43:11.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 Processor -- start 2023-07-14T11:43:11.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 -- start start 2023-07-14T11:43:11.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd540ffae0 0x7fcd54072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 0x7fcd54072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd54105af0 0x7fcd541a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54073580 con 0x7fcd540ffae0 2023-07-14T11:43:11.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541a4310 con 0x7fcd54100490 2023-07-14T11:43:11.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5bbff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541a4450 con 0x7fcd54105af0 2023-07-14T11:43:11.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5919a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 0x7fcd54072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd540ffae0 0x7fcd54072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5919a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 0x7fcd54072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59276/0 (socket says 172.21.15.161:59276) 2023-07-14T11:43:11.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5919a700 1 -- 172.21.15.161:0/2033342329 learned_addr learned my addr 172.21.15.161:0/2033342329 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:11.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.808+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd54105af0 0x7fcd541a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5919a700 1 -- 172.21.15.161:0/2033342329 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd54105af0 msgr2=0x7fcd541a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5919a700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd54105af0 0x7fcd541a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5919a700 1 -- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd540ffae0 msgr2=0x7fcd54072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5919a700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd540ffae0 0x7fcd54072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5919a700 1 -- 172.21.15.161:0/2033342329 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440095f0 con 0x7fcd54100490 2023-07-14T11:43:11.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd54105af0 0x7fcd541a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:43:11.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5999b700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd540ffae0 0x7fcd54072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:43:11.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5919a700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 0x7fcd54072d50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005910 tx=0x7fcd44005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:11.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd4affd700 1 -- 172.21.15.161:0/2033342329 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401d070 con 0x7fcd54100490 2023-07-14T11:43:11.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd4affd700 1 -- 172.21.15.161:0/2033342329 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd44022470 con 0x7fcd54100490 2023-07-14T11:43:11.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd4affd700 1 -- 172.21.15.161:0/2033342329 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44017650 con 0x7fcd54100490 2023-07-14T11:43:11.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd541a4650 con 0x7fcd54100490 2023-07-14T11:43:11.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.809+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd541a4a20 con 0x7fcd54100490 2023-07-14T11:43:11.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.813+0000 7fcd4affd700 1 -- 172.21.15.161:0/2033342329 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcd440225e0 con 0x7fcd54100490 2023-07-14T11:43:11.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 Processor -- start 2023-07-14T11:43:11.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 -- start start 2023-07-14T11:43:11.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd4affd700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcd400a12c0 0x7fcd400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d0520586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd4affd700 1 -- 172.21.15.161:0/2033342329 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fcd440ce460 con 0x7fcd54100490 2023-07-14T11:43:11.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d052063590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d052065eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d051f5ecb0 con 0x55d052058be0 2023-07-14T11:43:11.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d051fb1a40 con 0x55d052063ad0 2023-07-14T11:43:11.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.814+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d051faaed0 con 0x55d0520582d0 2023-07-14T11:43:11.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d0520586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcd400a12c0 0x7fcd400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d0520586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:55348/0 (socket says 172.21.15.161:55348) 2023-07-14T11:43:11.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 -- 172.21.15.161:0/1914465092 learned_addr learned my addr 172.21.15.161:0/1914465092 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5919a700 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d052063590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 -- 172.21.15.161:0/1914465092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 msgr2=0x55d052065eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d052065eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 -- 172.21.15.161:0/1914465092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 msgr2=0x55d052063590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d052063590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 -- 172.21.15.161:0/1914465092 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d051dd8de0 con 0x55d0520582d0 2023-07-14T11:43:11.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5919a700 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d052063590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:43:11.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5999b700 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d0520586a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcd500115f0 tx=0x7fcd50011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30228c2417b3a314 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:11.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcd400a12c0 0x7fcd400a3770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcd54074920 tx=0x7fcd4c009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:11.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.815+0000 7fcd3ffff700 1 -- 172.21.15.161:0/1914465092 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50019630 con 0x55d0520582d0 2023-07-14T11:43:11.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd3ffff700 1 -- 172.21.15.161:0/1914465092 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd50019c50 con 0x55d0520582d0 2023-07-14T11:43:11.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd3ffff700 1 -- 172.21.15.161:0/1914465092 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000ec20 con 0x55d0520582d0 2023-07-14T11:43:11.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/1914465092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 msgr2=0x55d0520586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d0520586a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcd500115f0 tx=0x7fcd50011900 comp rx=0 tx=0).stop 2023-07-14T11:43:11.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/1914465092 shutdown_connections 2023-07-14T11:43:11.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d052065eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d0520586a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/1914465092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d052063590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/1914465092 >> 172.21.15.161:0/1914465092 conn(0x55d05204eda0 msgr2=0x55d0520546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:11.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/1914465092 shutdown_connections 2023-07-14T11:43:11.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.816+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/1914465092 wait complete. 2023-07-14T11:43:11.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.817+0000 7fcd6f4d1b80 1 -- start start 2023-07-14T11:43:11.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.817+0000 7fcd6f4d1b80 1 Processor -- start 2023-07-14T11:43:11.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.817+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0520582d0 0x55d052063510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.817+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052058be0 0x55d052060800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.817+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052063ad0 0x55d052060f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d051f4aac0 con 0x55d0520582d0 2023-07-14T11:43:11.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d051f4a7c0 con 0x55d052058be0 2023-07-14T11:43:11.837 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d051e2e920 con 0x55d052063ad0 2023-07-14T11:43:11.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd6f4d1b80 10 client.0 _ll_register_callbacks cb 0x55d051f48760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:43:11.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052058be0 0x55d052060800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5a19c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052063ad0 0x55d052060f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0520582d0 0x55d052063510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052058be0 0x55d052060800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59306/0 (socket says 172.21.15.161:59306) 2023-07-14T11:43:11.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 -- 172.21.15.161:0/891381050 learned_addr learned my addr 172.21.15.161:0/891381050 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:11.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 -- 172.21.15.161:0/891381050 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052063ad0 msgr2=0x55d052060f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052063ad0 0x55d052060f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 -- 172.21.15.161:0/891381050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0520582d0 msgr2=0x55d052063510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0520582d0 0x55d052063510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 -- 172.21.15.161:0/891381050 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd50009d50 con 0x55d052058be0 2023-07-14T11:43:11.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052063ad0 0x55d052060f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:43:11.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.818+0000 7fcd5919a700 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052058be0 0x55d052060800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4409afd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a330c76b24994e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:11.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:43:11.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/891381050 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440a1070 con 0x55d052058be0 2023-07-14T11:43:11.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/891381050 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fcd4409b180 con 0x55d052058be0 2023-07-14T11:43:11.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d051f044f0 con 0x55d052058be0 2023-07-14T11:43:11.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/891381050 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd4409aae0 con 0x55d052058be0 2023-07-14T11:43:11.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd6f4d1b80 10 client.27737 fetch_fsmap learned FSMap version 44 2023-07-14T11:43:11.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d051dc1c60 con 0x55d052058be0 2023-07-14T11:43:11.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/891381050 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d051f044f0 con 0x55d052058be0 2023-07-14T11:43:11.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/891381050 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcd440a30b0 con 0x55d052058be0 2023-07-14T11:43:11.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd6f4d1b80 10 client.27737 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:43:11.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd6f4d1b80 10 client.27737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:43:11.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd6f4d1b80 20 client.27737 populate_metadata read hostname 'smithi161' 2023-07-14T11:43:11.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.819+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d052073100 0x55d052075510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.820+0000 7fcd6f4d1b80 4 client.27737 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:43:11.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.820+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d051fa3eb0 con 0x55d052073100 2023-07-14T11:43:11.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.820+0000 7fcd5999b700 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d052073100 0x55d052075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.820+0000 7fcd5999b700 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d052073100 0x55d052075510 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5bd216fb0ca6fed server_cookie=b8881294f8da688b in_seq=0 out_seq=0 2023-07-14T11:43:11.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.820+0000 7fcd3d7fa700 10 client.27737 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:11.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/891381050 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d051fa3eb0 con 0x55d052073100 2023-07-14T11:43:11.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd3d7fa700 10 client.27737 handle_command_reply: tid=0 2023-07-14T11:43:11.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 client.27737 shutdown 2023-07-14T11:43:11.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052058be0 msgr2=0x55d052060800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052058be0 0x55d052060800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4409afd0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 shutdown_connections 2023-07-14T11:43:11.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d052073100 0x55d052075510 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d0520582d0 0x55d052063510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052063ad0 0x55d052060f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/891381050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052058be0 0x55d052060800 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd3d7fa700 0 client.27737 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:11.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 >> 172.21.15.161:0/891381050 conn(0x55d05204eda0 msgr2=0x55d052054230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:11.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 shutdown_connections 2023-07-14T11:43:11.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/891381050 wait complete. 2023-07-14T11:43:11.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.821+0000 7fcd6f4d1b80 20 client.27737 trim_cache size 0 max 16384 2023-07-14T11:43:11.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 Processor -- start 2023-07-14T11:43:11.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 -- start start 2023-07-14T11:43:11.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d052063140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d05205baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d05205c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0520617b0 con 0x55d052058be0 2023-07-14T11:43:11.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d052055d20 con 0x55d052063ad0 2023-07-14T11:43:11.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.839+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d052059d60 con 0x55d0520582d0 2023-07-14T11:43:11.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d05205c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d052063140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d05205c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59318/0 (socket says 172.21.15.161:59318) 2023-07-14T11:43:11.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d052063140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:55366/0 (socket says 172.21.15.161:55366) 2023-07-14T11:43:11.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 -- 172.21.15.161:0/2468998875 learned_addr learned my addr 172.21.15.161:0/2468998875 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:11.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5919a700 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d05205baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 -- 172.21.15.161:0/2468998875 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 msgr2=0x55d052063140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d052063140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 -- 172.21.15.161:0/2468998875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 msgr2=0x55d05205baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d05205baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 -- 172.21.15.161:0/2468998875 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d051dff250 con 0x55d052063ad0 2023-07-14T11:43:11.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5999b700 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d052063140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:43:11.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.840+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d05205c030 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00e9c0 tx=0x7fcd4c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2720dcd520a9813 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:11.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd48bff700 1 -- 172.21.15.161:0/2468998875 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c017770 con 0x55d052063ad0 2023-07-14T11:43:11.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd48bff700 1 -- 172.21.15.161:0/2468998875 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4c00f040 con 0x55d052063ad0 2023-07-14T11:43:11.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd48bff700 1 -- 172.21.15.161:0/2468998875 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c024d70 con 0x55d052063ad0 2023-07-14T11:43:11.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2468998875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 msgr2=0x55d05205c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d05205c030 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00e9c0 tx=0x7fcd4c015000 comp rx=0 tx=0).stop 2023-07-14T11:43:11.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2468998875 shutdown_connections 2023-07-14T11:43:11.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d052063ad0 0x55d05205c030 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d0520582d0 0x55d052063140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2468998875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052058be0 0x55d05205baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2468998875 >> 172.21.15.161:0/2468998875 conn(0x55d05204eda0 msgr2=0x55d05206f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:11.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.841+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2468998875 shutdown_connections 2023-07-14T11:43:11.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.842+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2468998875 wait complete. 2023-07-14T11:43:11.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.842+0000 7fcd6f4d1b80 1 -- start start 2023-07-14T11:43:11.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.842+0000 7fcd6f4d1b80 1 Processor -- start 2023-07-14T11:43:11.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.842+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0520582d0 0x55d05206a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.842+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052058be0 0x55d052068710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd6f4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052063ad0 0x55d052068c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0520691d0 con 0x55d052063ad0 2023-07-14T11:43:11.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d052069310 con 0x55d0520582d0 2023-07-14T11:43:11.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd6f4d1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0520694d0 con 0x55d052058be0 2023-07-14T11:43:11.861 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd6f4d1b80 10 client.0 _ll_register_callbacks cb 0x7fcd440cbec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:43:11.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0520582d0 0x55d05206a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0520582d0 0x55d05206a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59326/0 (socket says 172.21.15.161:59326) 2023-07-14T11:43:11.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 -- 172.21.15.161:0/2033804690 learned_addr learned my addr 172.21.15.161:0/2033804690 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:11.862 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5919a700 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052058be0 0x55d052068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5a19c700 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052063ad0 0x55d052068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 -- 172.21.15.161:0/2033804690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052058be0 msgr2=0x55d052068710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052058be0 0x55d052068710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 -- 172.21.15.161:0/2033804690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052063ad0 msgr2=0x55d052068c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052063ad0 0x55d052068c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 -- 172.21.15.161:0/2033804690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d051dd8de0 con 0x55d0520582d0 2023-07-14T11:43:11.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd5999b700 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0520582d0 0x55d05206a3e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcd500115f0 tx=0x7fcd5000e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17dc6a73b0307ba9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:11.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:43:11.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.843+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/2033804690 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500050d0 con 0x55d0520582d0 2023-07-14T11:43:11.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd50019c50 con 0x55d0520582d0 2023-07-14T11:43:11.864 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/2033804690 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fcd50029070 con 0x55d0520582d0 2023-07-14T11:43:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/2033804690 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd50005230 con 0x55d0520582d0 2023-07-14T11:43:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 10 client.27743 fetch_fsmap learned FSMap version 44 2023-07-14T11:43:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fcd4c004880 con 0x55d0520582d0 2023-07-14T11:43:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/2033804690 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd50019c50 con 0x55d0520582d0 2023-07-14T11:43:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/2033804690 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcd50019430 con 0x55d0520582d0 2023-07-14T11:43:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 10 client.27743 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T11:43:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 10 client.27743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:43:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 20 client.27743 populate_metadata read hostname 'smithi161' 2023-07-14T11:43:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d0520c84e0 0x55d0520ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 4 client.27743 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:43:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.844+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d051f66050 con 0x55d0520c84e0 2023-07-14T11:43:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd5919a700 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d0520c84e0 0x55d0520ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd3d7fa700 10 client.27743 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd5919a700 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d0520c84e0 0x55d0520ca8f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b9936c88188f9b server_cookie=ea4695e5d72217d in_seq=0 out_seq=0 2023-07-14T11:43:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd3d7fa700 1 -- 172.21.15.161:0/2033804690 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d051f66050 con 0x55d0520c84e0 2023-07-14T11:43:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd3d7fa700 10 client.27743 handle_command_reply: tid=0 2023-07-14T11:43:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 client.27743 shutdown 2023-07-14T11:43:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0520582d0 msgr2=0x55d05206a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0520582d0 0x55d05206a3e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcd500115f0 tx=0x7fcd5000e780 comp rx=0 tx=0).stop 2023-07-14T11:43:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 shutdown_connections 2023-07-14T11:43:11.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55d0520c84e0 0x55d0520ca8f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d052063ad0 0x55d052068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0520582d0 0x55d05206a3e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd3d7fa700 0 client.27743 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:11.869 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 --2- 172.21.15.161:0/2033804690 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55d052058be0 0x55d052068710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.845+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 >> 172.21.15.161:0/2033804690 conn(0x55d05204eda0 msgr2=0x55d05205baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:11.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 shutdown_connections 2023-07-14T11:43:11.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd6f4d1b80 1 -- 172.21.15.161:0/2033804690 wait complete. 2023-07-14T11:43:11.870 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd6f4d1b80 20 client.27743 trim_cache size 0 max 16384 2023-07-14T11:43:11.871 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:43:11.871 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:43:11.871 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:43:11.871 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:43:11.872 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:43:11.872 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:43:11.872 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:43:11.872 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:43:11.872 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:43:11.872 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:43:11.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcd400a12c0 msgr2=0x7fcd400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcd400a12c0 0x7fcd400a3770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcd54074920 tx=0x7fcd4c009920 comp rx=0 tx=0).stop 2023-07-14T11:43:11.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 msgr2=0x7fcd54072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:11.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 0x7fcd54072d50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005910 tx=0x7fcd44005720 comp rx=0 tx=0).stop 2023-07-14T11:43:11.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 shutdown_connections 2023-07-14T11:43:11.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcd400a12c0 0x7fcd400a3770 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd540ffae0 0x7fcd54072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcd54105af0 0x7fcd541a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 --2- 172.21.15.161:0/2033342329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd54100490 0x7fcd54072d50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:11.875 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 >> 172.21.15.161:0/2033342329 conn(0x7fcd540fb5e0 msgr2=0x7fcd540fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:11.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 shutdown_connections 2023-07-14T11:43:11.876 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:11.846+0000 7fcd5bbff700 1 -- 172.21.15.161:0/2033342329 wait complete. 2023-07-14T11:43:11.876 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:43:11.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:11 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:43:11.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:11 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:43:12.702 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:12 smithi182 ceph-mon[129366]: pgmap v660: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:12.734 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:12 smithi161 ceph-mon[95526]: pgmap v660: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:12.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:12 smithi175 ceph-mon[129081]: pgmap v660: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-07-14T11:43:14.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:14 smithi175 ceph-mon[129081]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:43:14.452 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:14 smithi182 ceph-mon[129366]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:43:14.484 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:14 smithi161 ceph-mon[95526]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:43:16.908 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:16 smithi175 ceph-mon[129081]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:43:16.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:16 smithi182 ceph-mon[129366]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:43:16.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:16 smithi161 ceph-mon[95526]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:43:18.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:18 smithi175 ceph-mon[129081]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 185 B/s wr, 8 op/s 2023-07-14T11:43:18.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:18 smithi175 ceph-mon[129081]: osdmap e123: 12 total, 12 up, 12 in 2023-07-14T11:43:18.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:18 smithi182 ceph-mon[129366]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 185 B/s wr, 8 op/s 2023-07-14T11:43:18.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:18 smithi182 ceph-mon[129366]: osdmap e123: 12 total, 12 up, 12 in 2023-07-14T11:43:18.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:18 smithi161 ceph-mon[95526]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 185 B/s wr, 8 op/s 2023-07-14T11:43:18.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:18 smithi161 ceph-mon[95526]: osdmap e123: 12 total, 12 up, 12 in 2023-07-14T11:43:20.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:20 smithi175 ceph-mon[129081]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:43:20.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:20 smithi182 ceph-mon[129366]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:43:20.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:20 smithi161 ceph-mon[95526]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:43:22.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:22 smithi175 ceph-mon[129081]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:43:22.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:22 smithi182 ceph-mon[129366]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:43:22.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:22 smithi161 ceph-mon[95526]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:43:24.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:24 smithi175 ceph-mon[129081]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:24.452 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:24 smithi182 ceph-mon[129366]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:24.485 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:24 smithi161 ceph-mon[95526]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:26.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:26 smithi175 ceph-mon[129081]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:26.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:26 smithi182 ceph-mon[129366]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:26.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:26 smithi161 ceph-mon[95526]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:27.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:27 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:43:27.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:27 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:43:27.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:27 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:43:28.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:28 smithi175 ceph-mon[129081]: pgmap v669: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:28.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:28 smithi182 ceph-mon[129366]: pgmap v669: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:28.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:28 smithi161 ceph-mon[95526]: pgmap v669: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:43:30.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:30 smithi175 ceph-mon[129081]: pgmap v670: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:43:30.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:30 smithi182 ceph-mon[129366]: pgmap v670: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:43:31.011 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:30 smithi161 ceph-mon[95526]: pgmap v670: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:43:32.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:32 smithi175 ceph-mon[129081]: pgmap v671: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:33.118 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:32 smithi161 ceph-mon[95526]: pgmap v671: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:33.119 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:32 smithi182 ceph-mon[129366]: pgmap v671: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:34.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:34 smithi175 ceph-mon[129081]: pgmap v672: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:34.452 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:34 smithi182 ceph-mon[129366]: pgmap v672: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:34.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:34 smithi161 ceph-mon[95526]: pgmap v672: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:36.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:36 smithi175 ceph-mon[129081]: pgmap v673: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:36.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:36 smithi182 ceph-mon[129366]: pgmap v673: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:36.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:36 smithi161 ceph-mon[95526]: pgmap v673: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:38.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:38 smithi175 ceph-mon[129081]: pgmap v674: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:38.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:38 smithi182 ceph-mon[129366]: pgmap v674: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:38.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:38 smithi161 ceph-mon[95526]: pgmap v674: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:40.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:40 smithi175 ceph-mon[129081]: pgmap v675: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:40.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:40 smithi182 ceph-mon[129366]: pgmap v675: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:40.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:40 smithi161 ceph-mon[95526]: pgmap v675: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:41.878 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:43:42.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2996303209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec0ffae0 msgr2=0x7f6cec0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2996303209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec0ffae0 0x7f6cec0fff50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009990 tx=0x7f6cd8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2996303209 shutdown_connections 2023-07-14T11:43:42.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2996303209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec100490 0x7f6cec10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2996303209 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec105af0 0x7f6cec105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2996303209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec0ffae0 0x7f6cec0fff50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2996303209 >> 172.21.15.161:0/2996303209 conn(0x7f6cec0fb5e0 msgr2=0x7f6cec0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:42.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2996303209 shutdown_connections 2023-07-14T11:43:42.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.102+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2996303209 wait complete. 2023-07-14T11:43:42.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.103+0000 7f6cf2fff700 1 Processor -- start 2023-07-14T11:43:42.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf2fff700 1 -- start start 2023-07-14T11:43:42.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf2fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec0ffae0 0x7f6cec197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf2fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec100490 0x7f6cec198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf2fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec105af0 0x7f6cec19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf2fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19cde0 con 0x7f6cec0ffae0 2023-07-14T11:43:42.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf2fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19cf50 con 0x7f6cec105af0 2023-07-14T11:43:42.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf2fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19d110 con 0x7f6cec100490 2023-07-14T11:43:42.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec100490 0x7f6cec198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec100490 0x7f6cec198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45830/0 (socket says 172.21.15.161:45830) 2023-07-14T11:43:42.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cebfff700 1 -- 172.21.15.161:0/2307264517 learned_addr learned my addr 172.21.15.161:0/2307264517 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:42.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec0ffae0 0x7f6cec197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.104+0000 7f6cf159c700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec105af0 0x7f6cec19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cebfff700 1 -- 172.21.15.161:0/2307264517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec105af0 msgr2=0x7f6cec19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cebfff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec105af0 0x7f6cec19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cebfff700 1 -- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec0ffae0 msgr2=0x7f6cec197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cebfff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec0ffae0 0x7f6cec197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cebfff700 1 -- 172.21.15.161:0/2307264517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80095f0 con 0x7f6cec100490 2023-07-14T11:43:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cebfff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec100490 0x7f6cec198340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8005910 tx=0x7f6cd8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:42.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6ce9ffb700 1 -- 172.21.15.161:0/2307264517 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd801d070 con 0x7f6cec100490 2023-07-14T11:43:42.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec19d390 con 0x7f6cec100490 2023-07-14T11:43:42.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6ce9ffb700 1 -- 172.21.15.161:0/2307264517 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd8022470 con 0x7f6cec100490 2023-07-14T11:43:42.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6ce9ffb700 1 -- 172.21.15.161:0/2307264517 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8017650 con 0x7f6cec100490 2023-07-14T11:43:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.105+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec1a4c30 con 0x7f6cec100490 2023-07-14T11:43:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.108+0000 7f6ce9ffb700 1 -- 172.21.15.161:0/2307264517 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6cd80225e0 con 0x7f6cec100490 2023-07-14T11:43:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.109+0000 7f6d068d7b80 1 Processor -- start 2023-07-14T11:43:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6d068d7b80 1 -- start start 2023-07-14T11:43:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd6542d0 0x560dbd6546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6ce9ffb700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cdc0a12c0 0x7f6cdc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd654be0 0x560dbd65f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd661eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd55acb0 con 0x560dbd654be0 2023-07-14T11:43:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd5ada40 con 0x560dbd65fad0 2023-07-14T11:43:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd5a6ed0 con 0x560dbd6542d0 2023-07-14T11:43:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6ce9ffb700 1 -- 172.21.15.161:0/2307264517 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6cd80ce3e0 con 0x7f6cec100490 2023-07-14T11:43:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6cf159c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd6542d0 0x560dbd6546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6cf159c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd6542d0 0x560dbd6546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45832/0 (socket says 172.21.15.161:45832) 2023-07-14T11:43:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6cf159c700 1 -- 172.21.15.161:0/2416450322 learned_addr learned my addr 172.21.15.161:0/2416450322 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6cf159c700 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd661eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6cebfff700 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd654be0 0x560dbd65f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.110+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cdc0a12c0 0x7f6cdc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cf159c700 1 -- 172.21.15.161:0/2416450322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 msgr2=0x560dbd661eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cf159c700 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd661eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cf159c700 1 -- 172.21.15.161:0/2416450322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd654be0 msgr2=0x560dbd65f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cf159c700 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd654be0 0x560dbd65f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cf159c700 1 -- 172.21.15.161:0/2416450322 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dbd3d4de0 con 0x560dbd6542d0 2023-07-14T11:43:42.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cf159c700 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd6542d0 0x560dbd6546a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4008cd0 tx=0x7f6ce4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa9d38e5c5ce5f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:42.119 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cdc0a12c0 0x7f6cdc0a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009be0 tx=0x7f6ce0009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:42.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cd6ffd700 1 -- 172.21.15.161:0/2416450322 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce401f070 con 0x560dbd6542d0 2023-07-14T11:43:42.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cd6ffd700 1 -- 172.21.15.161:0/2416450322 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce400f040 con 0x560dbd6542d0 2023-07-14T11:43:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.111+0000 7f6cd6ffd700 1 -- 172.21.15.161:0/2416450322 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400e050 con 0x560dbd6542d0 2023-07-14T11:43:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 -- 172.21.15.161:0/2416450322 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd6542d0 msgr2=0x560dbd6546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd6542d0 0x560dbd6546a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4008cd0 tx=0x7f6ce4009920 comp rx=0 tx=0).stop 2023-07-14T11:43:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 -- 172.21.15.161:0/2416450322 shutdown_connections 2023-07-14T11:43:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd661eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd6542d0 0x560dbd6546a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/2416450322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd654be0 0x560dbd65f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 -- 172.21.15.161:0/2416450322 >> 172.21.15.161:0/2416450322 conn(0x560dbd64ada0 msgr2=0x560dbd6506a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:42.124 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 -- 172.21.15.161:0/2416450322 shutdown_connections 2023-07-14T11:43:42.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 -- 172.21.15.161:0/2416450322 wait complete. 2023-07-14T11:43:42.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.112+0000 7f6d068d7b80 1 -- start start 2023-07-14T11:43:42.125 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 1 Processor -- start 2023-07-14T11:43:42.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd65c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd65ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd546ac0 con 0x560dbd6542d0 2023-07-14T11:43:42.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd5467c0 con 0x560dbd654be0 2023-07-14T11:43:42.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd42a920 con 0x560dbd65fad0 2023-07-14T11:43:42.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6d068d7b80 10 client.0 _ll_register_callbacks cb 0x560dbd544760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:43:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd65c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6cf0d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd65c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35698/0 (socket says 172.21.15.161:35698) 2023-07-14T11:43:42.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6cf159c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd65ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.113+0000 7f6cebfff700 1 -- 172.21.15.161:0/1314026228 learned_addr learned my addr 172.21.15.161:0/1314026228 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:42.130 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6cebfff700 1 -- 172.21.15.161:0/1314026228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 msgr2=0x560dbd65ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6cebfff700 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd65ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.131 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6cebfff700 1 -- 172.21.15.161:0/1314026228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 msgr2=0x560dbd65f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6cebfff700 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6cebfff700 1 -- 172.21.15.161:0/1314026228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce40089b0 con 0x560dbd654be0 2023-07-14T11:43:42.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6cebfff700 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd65c770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80cb7d0 tx=0x7f6cd80a02f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3af320a978715 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:42.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6ccbfff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:43:42.133 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6ccbfff700 1 -- 172.21.15.161:0/1314026228 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd809b180 con 0x560dbd654be0 2023-07-14T11:43:42.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dbd5004f0 con 0x560dbd654be0 2023-07-14T11:43:42.134 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6ccbfff700 1 -- 172.21.15.161:0/1314026228 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6cd809f1b0 con 0x560dbd654be0 2023-07-14T11:43:42.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6ccbfff700 1 -- 172.21.15.161:0/1314026228 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd80a03e0 con 0x560dbd654be0 2023-07-14T11:43:42.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6d068d7b80 10 client.27755 fetch_fsmap learned FSMap version 45 2023-07-14T11:43:42.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x560dbd3bdc60 con 0x560dbd654be0 2023-07-14T11:43:42.135 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.114+0000 7f6ccbfff700 1 -- 172.21.15.161:0/1314026228 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dbd5004f0 con 0x560dbd654be0 2023-07-14T11:43:42.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6ccbfff700 1 -- 172.21.15.161:0/1314026228 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6cd809b3e0 con 0x560dbd654be0 2023-07-14T11:43:42.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6d068d7b80 10 client.27755 fetch_fsmap finished waiting for FSMap version 45 2023-07-14T11:43:42.136 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6d068d7b80 10 client.27755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:43:42.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6d068d7b80 20 client.27755 populate_metadata read hostname 'smithi161' 2023-07-14T11:43:42.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd66f000 0x560dbd671410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6d068d7b80 4 client.27755 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:43:42.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dbd59feb0 con 0x560dbd66f000 2023-07-14T11:43:42.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.115+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd66f000 0x560dbd671410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6ccbfff700 10 client.27755 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:42.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd66f000 0x560dbd671410 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ef664f368fa34e server_cookie=5e22775fd90e08e8 in_seq=0 out_seq=0 2023-07-14T11:43:42.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6ccbfff700 1 -- 172.21.15.161:0/1314026228 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560dbd59feb0 con 0x560dbd66f000 2023-07-14T11:43:42.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6ccbfff700 10 client.27755 handle_command_reply: tid=0 2023-07-14T11:43:42.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 client.27755 shutdown 2023-07-14T11:43:42.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 msgr2=0x560dbd65c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd65c770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80cb7d0 tx=0x7f6cd80a02f0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 shutdown_connections 2023-07-14T11:43:42.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd66f000 0x560dbd671410 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd65ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1314026228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd65c770 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6ccbfff700 0 client.27755 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:42.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.116+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 >> 172.21.15.161:0/1314026228 conn(0x560dbd64ada0 msgr2=0x560dbd650230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:42.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.117+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 shutdown_connections 2023-07-14T11:43:42.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.117+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1314026228 wait complete. 2023-07-14T11:43:42.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.117+0000 7f6d068d7b80 20 client.27755 trim_cache size 0 max 16384 2023-07-14T11:43:42.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.134+0000 7f6d068d7b80 1 Processor -- start 2023-07-14T11:43:42.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.134+0000 7f6d068d7b80 1 -- start start 2023-07-14T11:43:42.147 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd654be0 0x560dbd65bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd65c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.148 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd59feb0 con 0x560dbd6542d0 2023-07-14T11:43:42.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd5004f0 con 0x560dbd65fad0 2023-07-14T11:43:42.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd64cec0 con 0x560dbd654be0 2023-07-14T11:43:42.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd654be0 0x560dbd65bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd654be0 0x560dbd65bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45852/0 (socket says 172.21.15.161:45852) 2023-07-14T11:43:42.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6cebfff700 1 -- 172.21.15.161:0/1867137281 learned_addr learned my addr 172.21.15.161:0/1867137281 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:42.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6cf159c700 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd65c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.135+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cf159c700 1 -- 172.21.15.161:0/1867137281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd654be0 msgr2=0x560dbd65bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cf159c700 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd654be0 0x560dbd65bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cf159c700 1 -- 172.21.15.161:0/1867137281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 msgr2=0x560dbd65f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cf159c700 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cf159c700 1 -- 172.21.15.161:0/1867137281 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dbd3fb250 con 0x560dbd65fad0 2023-07-14T11:43:42.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cebfff700 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd654be0 0x560dbd65bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:43:42.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cf159c700 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd65c4d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4018cc0 tx=0x7f6ce4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400ce0d79ac88d17 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:42.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cd77fe700 1 -- 172.21.15.161:0/1867137281 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400e040 con 0x560dbd65fad0 2023-07-14T11:43:42.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cd77fe700 1 -- 172.21.15.161:0/1867137281 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce4018d90 con 0x560dbd65fad0 2023-07-14T11:43:42.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.136+0000 7f6cd77fe700 1 -- 172.21.15.161:0/1867137281 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce401f040 con 0x560dbd65fad0 2023-07-14T11:43:42.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1867137281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 msgr2=0x560dbd65c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd65c4d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4018cc0 tx=0x7f6ce4009b60 comp rx=0 tx=0).stop 2023-07-14T11:43:42.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1867137281 shutdown_connections 2023-07-14T11:43:42.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd65f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd65fad0 0x560dbd65c4d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/1867137281 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd654be0 0x560dbd65bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1867137281 >> 172.21.15.161:0/1867137281 conn(0x560dbd64ada0 msgr2=0x560dbd657b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:42.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1867137281 shutdown_connections 2023-07-14T11:43:42.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 -- 172.21.15.161:0/1867137281 wait complete. 2023-07-14T11:43:42.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 -- start start 2023-07-14T11:43:42.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.137+0000 7f6d068d7b80 1 Processor -- start 2023-07-14T11:43:42.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd666670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd664b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6d068d7b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd665140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd665820 con 0x560dbd6542d0 2023-07-14T11:43:42.156 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd665990 con 0x560dbd654be0 2023-07-14T11:43:42.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6d068d7b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbd6c2f00 con 0x560dbd65fad0 2023-07-14T11:43:42.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6d068d7b80 10 client.0 _ll_register_callbacks cb 0x7f6cd8097570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:43:42.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd664b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cf159c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd665140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd664b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35720/0 (socket says 172.21.15.161:35720) 2023-07-14T11:43:42.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cebfff700 1 -- 172.21.15.161:0/555621363 learned_addr learned my addr 172.21.15.161:0/555621363 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:43:42.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cf159c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd665140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:45858/0 (socket says 172.21.15.161:45858) 2023-07-14T11:43:42.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd666670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cebfff700 1 -- 172.21.15.161:0/555621363 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 msgr2=0x560dbd665140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.138+0000 7f6cebfff700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd665140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6cebfff700 1 -- 172.21.15.161:0/555621363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 msgr2=0x560dbd666670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:43:42.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6cebfff700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd666670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6cebfff700 1 -- 172.21.15.161:0/555621363 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dbd3bdc60 con 0x560dbd654be0 2023-07-14T11:43:42.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd666670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:43:42.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6cf159c700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd665140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:43:42.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6cebfff700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd664b10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80994d0 tx=0x7f6cd8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9565ae7ab7afc558 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:43:42.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6ccbfff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:43:42.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6ccbfff700 1 -- 172.21.15.161:0/555621363 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd809a870 con 0x560dbd654be0 2023-07-14T11:43:42.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6ccbfff700 1 -- 172.21.15.161:0/555621363 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6cd809f1b0 con 0x560dbd654be0 2023-07-14T11:43:42.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ce400f040 con 0x560dbd654be0 2023-07-14T11:43:42.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6ccbfff700 1 -- 172.21.15.161:0/555621363 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd809b180 con 0x560dbd654be0 2023-07-14T11:43:42.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6d068d7b80 10 client.27761 fetch_fsmap learned FSMap version 45 2023-07-14T11:43:42.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x560dbd3d4de0 con 0x560dbd654be0 2023-07-14T11:43:42.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.139+0000 7f6ccbfff700 1 -- 172.21.15.161:0/555621363 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce400f040 con 0x560dbd654be0 2023-07-14T11:43:42.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6ccbfff700 1 -- 172.21.15.161:0/555621363 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6cd80b91a0 con 0x560dbd654be0 2023-07-14T11:43:42.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6d068d7b80 10 client.27761 fetch_fsmap finished waiting for FSMap version 45 2023-07-14T11:43:42.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6d068d7b80 10 client.27761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:43:42.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6d068d7b80 20 client.27761 populate_metadata read hostname 'smithi161' 2023-07-14T11:43:42.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd6c41b0 0x560dbd6c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:43:42.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6d068d7b80 4 client.27761 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:43:42.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560dbd562050 con 0x560dbd6c41b0 2023-07-14T11:43:42.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd6c41b0 0x560dbd6c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:43:42.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6cf0d9b700 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd6c41b0 0x560dbd6c6590 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca01b1bd429e85b server_cookie=919bf7fbe6fe03bf in_seq=0 out_seq=0 2023-07-14T11:43:42.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.140+0000 7f6ccbfff700 10 client.27761 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:42.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6ccbfff700 1 -- 172.21.15.161:0/555621363 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560dbd562050 con 0x560dbd6c41b0 2023-07-14T11:43:42.165 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6ccbfff700 10 client.27761 handle_command_reply: tid=0 2023-07-14T11:43:42.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 client.27761 shutdown 2023-07-14T11:43:42.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 msgr2=0x560dbd664b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.166 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd664b10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80994d0 tx=0x7f6cd8009310 comp rx=0 tx=0).stop 2023-07-14T11:43:42.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 shutdown_connections 2023-07-14T11:43:42.167 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x560dbd6c41b0 0x560dbd6c6590 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbd6542d0 0x560dbd666670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.168 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x560dbd65fad0 0x560dbd665140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 --2- 172.21.15.161:0/555621363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbd654be0 0x560dbd664b10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 >> 172.21.15.161:0/555621363 conn(0x560dbd64ada0 msgr2=0x560dbd6563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:42.169 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6ccbfff700 0 client.27761 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:43:42.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 shutdown_connections 2023-07-14T11:43:42.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 1 -- 172.21.15.161:0/555621363 wait complete. 2023-07-14T11:43:42.170 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.141+0000 7f6d068d7b80 20 client.27761 trim_cache size 0 max 16384 2023-07-14T11:43:42.171 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:43:42.172 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:43:42.172 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:43:42.173 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:43:42.173 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:43:42.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:43:42.173 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:43:42.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:43:42.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:43:42.174 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:43:42.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cdc0a12c0 msgr2=0x7f6cdc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cdc0a12c0 0x7f6cdc0a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009be0 tx=0x7f6ce0009260 comp rx=0 tx=0).stop 2023-07-14T11:43:42.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec100490 msgr2=0x7f6cec198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:43:42.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec100490 0x7f6cec198340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8005910 tx=0x7f6cd8005720 comp rx=0 tx=0).stop 2023-07-14T11:43:42.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 shutdown_connections 2023-07-14T11:43:42.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cdc0a12c0 0x7f6cdc0a3770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.178 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec0ffae0 0x7f6cec197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cec105af0 0x7f6cec19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.179 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 --2- 172.21.15.161:0/2307264517 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6cec100490 0x7f6cec198340 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:43:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 >> 172.21.15.161:0/2307264517 conn(0x7f6cec0fb5e0 msgr2=0x7f6cec10aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:43:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 shutdown_connections 2023-07-14T11:43:42.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:43:42.142+0000 7f6cf2fff700 1 -- 172.21.15.161:0/2307264517 wait complete. 2023-07-14T11:43:42.181 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:43:42.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:42 smithi175 ceph-mon[129081]: pgmap v676: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:42.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:42 smithi182 ceph-mon[129366]: pgmap v676: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:42.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:42 smithi161 ceph-mon[95526]: pgmap v676: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:44.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:44 smithi175 ceph-mon[129081]: pgmap v677: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:44.451 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:44 smithi182 ceph-mon[129366]: pgmap v677: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:44.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:44 smithi161 ceph-mon[95526]: pgmap v677: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:46.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:46 smithi175 ceph-mon[129081]: pgmap v678: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:46.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:46 smithi182 ceph-mon[129366]: pgmap v678: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:46.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:46 smithi161 ceph-mon[95526]: pgmap v678: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:48.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:48 smithi175 ceph-mon[129081]: pgmap v679: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:48.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:48 smithi182 ceph-mon[129366]: pgmap v679: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:48.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:48 smithi161 ceph-mon[95526]: pgmap v679: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:50.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:50 smithi175 ceph-mon[129081]: pgmap v680: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:50.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:50 smithi182 ceph-mon[129366]: pgmap v680: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:50.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:50 smithi161 ceph-mon[95526]: pgmap v680: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:52.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:52 smithi175 ceph-mon[129081]: pgmap v681: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:52.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:52 smithi175 ceph-mon[129081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:43:52.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:52 smithi182 ceph-mon[129366]: pgmap v681: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:52.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:52 smithi182 ceph-mon[129366]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:43:52.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:52 smithi161 ceph-mon[95526]: pgmap v681: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:43:52.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:52 smithi161 ceph-mon[95526]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T11:43:54.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:54 smithi175 ceph-mon[129081]: pgmap v682: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:54.451 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:54 smithi182 ceph-mon[129366]: pgmap v682: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:54.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:54 smithi161 ceph-mon[95526]: pgmap v682: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:43:56.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:56 smithi175 ceph-mon[129081]: pgmap v683: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:56.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:56 smithi182 ceph-mon[129366]: pgmap v683: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:56.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:56 smithi161 ceph-mon[95526]: pgmap v683: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:43:58.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:43:58 smithi175 ceph-mon[129081]: pgmap v684: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:58.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:43:58 smithi182 ceph-mon[129366]: pgmap v684: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:43:58.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:43:58 smithi161 ceph-mon[95526]: pgmap v684: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:00.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:00 smithi175 ceph-mon[129081]: pgmap v685: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:00.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:00 smithi182 ceph-mon[129366]: pgmap v685: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:00.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:00 smithi161 ceph-mon[95526]: pgmap v685: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:02.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:02 smithi175 ceph-mon[129081]: pgmap v686: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-14T11:44:02.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:02 smithi182 ceph-mon[129366]: pgmap v686: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-14T11:44:02.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:02 smithi161 ceph-mon[95526]: pgmap v686: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-14T11:44:04.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:04 smithi175 ceph-mon[129081]: pgmap v687: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:44:04.451 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:04 smithi182 ceph-mon[129366]: pgmap v687: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:44:04.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:04 smithi161 ceph-mon[95526]: pgmap v687: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:44:06.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:06 smithi175 ceph-mon[129081]: pgmap v688: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:44:06.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:06 smithi182 ceph-mon[129366]: pgmap v688: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:44:06.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:06 smithi161 ceph-mon[95526]: pgmap v688: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:44:08.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:08 smithi175 ceph-mon[129081]: pgmap v689: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-14T11:44:08.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:08 smithi182 ceph-mon[129366]: pgmap v689: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-14T11:44:08.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:08 smithi161 ceph-mon[95526]: pgmap v689: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-14T11:44:10.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:10 smithi175 ceph-mon[129081]: pgmap v690: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:44:11.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:10 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:44:11.715 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:10 smithi161 ceph-mon[95526]: pgmap v690: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:44:11.715 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:10 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:44:11.715 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:10 smithi182 ceph-mon[129366]: pgmap v690: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:44:11.716 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:10 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:44:11.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:11 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:11.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:11 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:11.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:11 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:44:11.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:11 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:44:11.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:11 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:12.158 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:11 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:12.158 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:11 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:12.158 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:11 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:44:12.159 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:11 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:44:12.159 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:11 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:12.182 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:44:12.201 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:11 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:12.201 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:11 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:12.202 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:11 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:44:12.202 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:11 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:44:12.202 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:11 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:44:12.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 -- 172.21.15.161:0/1765360708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5354105ae0 msgr2=0x7f5354105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 --2- 172.21.15.161:0/1765360708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5354105ae0 0x7f5354105eb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5344009990 tx=0x7f5344009ca0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 -- 172.21.15.161:0/1765360708 shutdown_connections 2023-07-14T11:44:12.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 --2- 172.21.15.161:0/1765360708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5354100480 0x7f535410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 --2- 172.21.15.161:0/1765360708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5354105ae0 0x7f5354105eb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 --2- 172.21.15.161:0/1765360708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53540ffad0 0x7f53540fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 -- 172.21.15.161:0/1765360708 >> 172.21.15.161:0/1765360708 conn(0x7f53540fb5f0 msgr2=0x7f53540fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:12.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 -- 172.21.15.161:0/1765360708 shutdown_connections 2023-07-14T11:44:12.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.397+0000 7f53597ff700 1 -- 172.21.15.161:0/1765360708 wait complete. 2023-07-14T11:44:12.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.398+0000 7f53597ff700 1 Processor -- start 2023-07-14T11:44:12.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.398+0000 7f53597ff700 1 -- start start 2023-07-14T11:44:12.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f53597ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53540ffad0 0x7f535419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f53597ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5354100480 0x7f535419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f53597ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5354105ae0 0x7f5354195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f53597ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354196100 con 0x7f5354105ae0 2023-07-14T11:44:12.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f53597ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354196270 con 0x7f53540ffad0 2023-07-14T11:44:12.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f53597ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354196430 con 0x7f5354100480 2023-07-14T11:44:12.393 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53540ffad0 0x7f535419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53540ffad0 0x7f535419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59778/0 (socket says 172.21.15.161:59778) 2023-07-14T11:44:12.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 -- 172.21.15.161:0/1628531499 learned_addr learned my addr 172.21.15.161:0/1628531499 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:12.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f534bfff700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5354100480 0x7f535419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f53537fe700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5354105ae0 0x7f5354195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 -- 172.21.15.161:0/1628531499 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5354100480 msgr2=0x7f535419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5354100480 0x7f535419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 -- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5354105ae0 msgr2=0x7f5354195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5354105ae0 0x7f5354195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.399+0000 7f5352ffd700 1 -- 172.21.15.161:0/1628531499 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009c40 con 0x7f53540ffad0 2023-07-14T11:44:12.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.400+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53540ffad0 0x7f535419ba20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5344005910 tx=0x7f5344004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:12.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.400+0000 7f5350ff9700 1 -- 172.21.15.161:0/1628531499 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534401d070 con 0x7f53540ffad0 2023-07-14T11:44:12.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.400+0000 7f5350ff9700 1 -- 172.21.15.161:0/1628531499 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5344022470 con 0x7f53540ffad0 2023-07-14T11:44:12.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.400+0000 7f5350ff9700 1 -- 172.21.15.161:0/1628531499 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344017650 con 0x7f53540ffad0 2023-07-14T11:44:12.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.400+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53440095f0 con 0x7f53540ffad0 2023-07-14T11:44:12.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.400+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53541968f0 con 0x7f53540ffad0 2023-07-14T11:44:12.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.403+0000 7f5350ff9700 1 -- 172.21.15.161:0/1628531499 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f53440225e0 con 0x7f53540ffad0 2023-07-14T11:44:12.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 Processor -- start 2023-07-14T11:44:12.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 -- start start 2023-07-14T11:44:12.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de5c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de6bee0 0x556d7de6e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7dd62cb0 con 0x556d7de6bee0 2023-07-14T11:44:12.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ddb5a40 con 0x556d7de5cbe0 2023-07-14T11:44:12.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.405+0000 7f536d1abb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ddaeed0 con 0x556d7de5c2d0 2023-07-14T11:44:12.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5350ff9700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f53400a12c0 0x7f53400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59788/0 (socket says 172.21.15.161:59788) 2023-07-14T11:44:12.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 -- 172.21.15.161:0/1139448940 learned_addr learned my addr 172.21.15.161:0/1139448940 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:12.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f53537fe700 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de5c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5350ff9700 1 -- 172.21.15.161:0/1628531499 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f53440ce400 con 0x7f53540ffad0 2023-07-14T11:44:12.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f534bfff700 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de6bee0 0x556d7de6e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f53537fe700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f53400a12c0 0x7f53400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 -- 172.21.15.161:0/1139448940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 msgr2=0x556d7de5c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de5c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 -- 172.21.15.161:0/1139448940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de6bee0 msgr2=0x556d7de6e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de6bee0 0x556d7de6e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 -- 172.21.15.161:0/1139448940 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7dbdcde0 con 0x556d7de5cbe0 2023-07-14T11:44:12.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f534bfff700 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de6bee0 0x556d7de6e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:44:12.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6b9a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f53440a2fb0 tx=0x7f5344000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90572ebb561fd44d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:12.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.406+0000 7f53497fa700 1 -- 172.21.15.161:0/1139448940 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534409b120 con 0x556d7de5cbe0 2023-07-14T11:44:12.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f53537fe700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f53400a12c0 0x7f53400a3770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f534c009c10 tx=0x7f534c00b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:12.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f53497fa700 1 -- 172.21.15.161:0/1139448940 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f534409f150 con 0x556d7de5cbe0 2023-07-14T11:44:12.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f53497fa700 1 -- 172.21.15.161:0/1139448940 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534409b5e0 con 0x556d7de5cbe0 2023-07-14T11:44:12.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 -- 172.21.15.161:0/1139448940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 msgr2=0x556d7de6b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6b9a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f53440a2fb0 tx=0x7f5344000bc0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 -- 172.21.15.161:0/1139448940 shutdown_connections 2023-07-14T11:44:12.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de6bee0 0x556d7de6e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de5c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1139448940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6b9a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 -- 172.21.15.161:0/1139448940 >> 172.21.15.161:0/1139448940 conn(0x556d7de52da0 msgr2=0x556d7de586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:12.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 -- 172.21.15.161:0/1139448940 shutdown_connections 2023-07-14T11:44:12.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.407+0000 7f536d1abb80 1 -- 172.21.15.161:0/1139448940 wait complete. 2023-07-14T11:44:12.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 -- start start 2023-07-14T11:44:12.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 Processor -- start 2023-07-14T11:44:12.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de6c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5cbe0 0x556d7de6c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7dd4eac0 con 0x556d7de5c2d0 2023-07-14T11:44:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7dd4e7c0 con 0x556d7de6bee0 2023-07-14T11:44:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.408+0000 7f536d1abb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7dc32920 con 0x556d7de5cbe0 2023-07-14T11:44:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f536d1abb80 10 client.0 _ll_register_callbacks cb 0x556d7dd4c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:44:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5cbe0 0x556d7de6c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f53537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5cbe0 0x556d7de6c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:43916/0 (socket says 172.21.15.161:43916) 2023-07-14T11:44:12.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 -- 172.21.15.161:0/1980577881 learned_addr learned my addr 172.21.15.161:0/1980577881 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:12.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de6c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 -- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 msgr2=0x556d7de6cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 -- 172.21.15.161:0/1980577881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 msgr2=0x556d7de6c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de6c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 -- 172.21.15.161:0/1980577881 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53541968f0 con 0x556d7de5cbe0 2023-07-14T11:44:12.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f53537fe700 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:44:12.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f534bfff700 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5cbe0 0x556d7de6c7f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f533c012740 tx=0x7f533c012a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a845285607a464a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:12.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.409+0000 7f533a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:44:12.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f533a7fc700 1 -- 172.21.15.161:0/1980577881 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c017070 con 0x556d7de5cbe0 2023-07-14T11:44:12.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d7dd084f0 con 0x556d7de5cbe0 2023-07-14T11:44:12.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f533a7fc700 1 -- 172.21.15.161:0/1980577881 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f533c004050 con 0x556d7de5cbe0 2023-07-14T11:44:12.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f533a7fc700 1 -- 172.21.15.161:0/1980577881 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f533c004210 con 0x556d7de5cbe0 2023-07-14T11:44:12.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 10 client.27817 fetch_fsmap learned FSMap version 46 2023-07-14T11:44:12.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556d7dbc5c60 con 0x556d7de5cbe0 2023-07-14T11:44:12.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f533a7fc700 1 -- 172.21.15.161:0/1980577881 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d7dd084f0 con 0x556d7de5cbe0 2023-07-14T11:44:12.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f533a7fc700 1 -- 172.21.15.161:0/1980577881 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f533c00f510 con 0x556d7de5cbe0 2023-07-14T11:44:12.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 10 client.27817 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:44:12.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 10 client.27817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:44:12.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 20 client.27817 populate_metadata read hostname 'smithi161' 2023-07-14T11:44:12.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7de7b5c0 0x556d7de7d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 4 client.27817 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:44:12.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.410+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d7dda7eb0 con 0x556d7de7b5c0 2023-07-14T11:44:12.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.411+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7de7b5c0 0x556d7de7d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.411+0000 7f533a7fc700 10 client.27817 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:12.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.411+0000 7f5352ffd700 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7de7b5c0 0x556d7de7d9d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63fcf326578c69ca server_cookie=3d7f4d2f6b4c609c in_seq=0 out_seq=0 2023-07-14T11:44:12.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f533a7fc700 1 -- 172.21.15.161:0/1980577881 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556d7dda7eb0 con 0x556d7de7b5c0 2023-07-14T11:44:12.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f533a7fc700 10 client.27817 handle_command_reply: tid=0 2023-07-14T11:44:12.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 client.27817 shutdown 2023-07-14T11:44:12.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5cbe0 msgr2=0x556d7de6c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.428 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5cbe0 0x556d7de6c7f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f533c012740 tx=0x7f533c012a50 comp rx=0 tx=0).stop 2023-07-14T11:44:12.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 shutdown_connections 2023-07-14T11:44:12.429 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7de7b5c0 0x556d7de7d9d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de6c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f533a7fc700 0 client.27817 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:12.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 --2- 172.21.15.161:0/1980577881 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5cbe0 0x556d7de6c7f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 >> 172.21.15.161:0/1980577881 conn(0x556d7de52da0 msgr2=0x556d7de58290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:12.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 shutdown_connections 2023-07-14T11:44:12.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 1 -- 172.21.15.161:0/1980577881 wait complete. 2023-07-14T11:44:12.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.412+0000 7f536d1abb80 20 client.27817 trim_cache size 0 max 16384 2023-07-14T11:44:12.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 Processor -- start 2023-07-14T11:44:12.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 -- start start 2023-07-14T11:44:12.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de65950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de6bee0 0x556d7de76a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7de52970 con 0x556d7de5c2d0 2023-07-14T11:44:12.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7de6b530 con 0x556d7de5cbe0 2023-07-14T11:44:12.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.430+0000 7f536d1abb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7de69480 con 0x556d7de6bee0 2023-07-14T11:44:12.436 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f534bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de6bee0 0x556d7de76a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de6bee0 0x556d7de76a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:43924/0 (socket says 172.21.15.161:43924) 2023-07-14T11:44:12.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de65950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f534bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59802/0 (socket says 172.21.15.161:59802) 2023-07-14T11:44:12.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 -- 172.21.15.161:0/701983783 learned_addr learned my addr 172.21.15.161:0/701983783 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:12.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 -- 172.21.15.161:0/701983783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 msgr2=0x556d7de6c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 --2- 172.21.15.161:0/701983783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 -- 172.21.15.161:0/701983783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 msgr2=0x556d7de65950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 --2- 172.21.15.161:0/701983783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de65950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 -- 172.21.15.161:0/701983783 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7dc03250 con 0x556d7de6bee0 2023-07-14T11:44:12.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.431+0000 7f53537fe700 1 --2- 172.21.15.161:0/701983783 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de6bee0 0x556d7de76a70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f534c009390 tx=0x7f534c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d268701043c6f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:12.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f5349ffb700 1 -- 172.21.15.161:0/701983783 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c00e040 con 0x556d7de6bee0 2023-07-14T11:44:12.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f5349ffb700 1 -- 172.21.15.161:0/701983783 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f534c004db0 con 0x556d7de6bee0 2023-07-14T11:44:12.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f5349ffb700 1 -- 172.21.15.161:0/701983783 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0048e0 con 0x556d7de6bee0 2023-07-14T11:44:12.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 -- 172.21.15.161:0/701983783 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de6bee0 msgr2=0x556d7de76a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 --2- 172.21.15.161:0/701983783 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de6bee0 0x556d7de76a70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f534c009390 tx=0x7f534c009920 comp rx=0 tx=0).stop 2023-07-14T11:44:12.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 -- 172.21.15.161:0/701983783 shutdown_connections 2023-07-14T11:44:12.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 --2- 172.21.15.161:0/701983783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5c2d0 0x556d7de65950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 --2- 172.21.15.161:0/701983783 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de6bee0 0x556d7de76a70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 --2- 172.21.15.161:0/701983783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de5cbe0 0x556d7de6c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 -- 172.21.15.161:0/701983783 >> 172.21.15.161:0/701983783 conn(0x556d7de52da0 msgr2=0x556d7de6c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:12.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 -- 172.21.15.161:0/701983783 shutdown_connections 2023-07-14T11:44:12.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.432+0000 7f536d1abb80 1 -- 172.21.15.161:0/701983783 wait complete. 2023-07-14T11:44:12.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.433+0000 7f536d1abb80 1 -- start start 2023-07-14T11:44:12.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.433+0000 7f536d1abb80 1 Processor -- start 2023-07-14T11:44:12.442 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de6d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5cbe0 0x556d7de6dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f536d1abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f536d1abb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7decf480 con 0x556d7de5cbe0 2023-07-14T11:44:12.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f536d1abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7decf5c0 con 0x556d7de6bee0 2023-07-14T11:44:12.443 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f536d1abb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7decf730 con 0x556d7de5c2d0 2023-07-14T11:44:12.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de6d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f536d1abb80 10 client.0 _ll_register_callbacks cb 0x7f533c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:44:12.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de6d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:43930/0 (socket says 172.21.15.161:43930) 2023-07-14T11:44:12.444 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 -- 172.21.15.161:0/467603280 learned_addr learned my addr 172.21.15.161:0/467603280 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:12.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f53537fe700 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f534bfff700 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5cbe0 0x556d7de6dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 -- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 msgr2=0x556d7de6e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.445 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 -- 172.21.15.161:0/467603280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5cbe0 msgr2=0x556d7de6dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:12.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5cbe0 0x556d7de6dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.434+0000 7f5352ffd700 1 -- 172.21.15.161:0/467603280 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c0090d0 con 0x556d7de5c2d0 2023-07-14T11:44:12.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f5352ffd700 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de6d980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f534409ae20 tx=0x7f5344009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53a80c2b2d07cfd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:12.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f533a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:44:12.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f533a7fc700 1 -- 172.21.15.161:0/467603280 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534409b120 con 0x556d7de5c2d0 2023-07-14T11:44:12.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f533a7fc700 1 -- 172.21.15.161:0/467603280 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f5344022900 con 0x556d7de5c2d0 2023-07-14T11:44:12.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f534409f150 con 0x556d7de5c2d0 2023-07-14T11:44:12.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f533a7fc700 1 -- 172.21.15.161:0/467603280 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53440b9240 con 0x556d7de5c2d0 2023-07-14T11:44:12.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 10 client.27823 fetch_fsmap learned FSMap version 46 2023-07-14T11:44:12.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556d7dbdb2b0 con 0x556d7de5c2d0 2023-07-14T11:44:12.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f533a7fc700 1 -- 172.21.15.161:0/467603280 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f534409f150 con 0x556d7de5c2d0 2023-07-14T11:44:12.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f533a7fc700 1 -- 172.21.15.161:0/467603280 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f534409b380 con 0x556d7de5c2d0 2023-07-14T11:44:12.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 10 client.27823 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:44:12.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 10 client.27823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:44:12.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 20 client.27823 populate_metadata read hostname 'smithi161' 2023-07-14T11:44:12.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7ded0a60 0x556d7ded2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:12.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 4 client.27823 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:44:12.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.435+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556d7dd6a050 con 0x556d7ded0a60 2023-07-14T11:44:12.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f534bfff700 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7ded0a60 0x556d7ded2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:12.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f533a7fc700 10 client.27823 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:12.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f534bfff700 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7ded0a60 0x556d7ded2e70 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89584df9e36fd0b1 server_cookie=74e83d09a25f4168 in_seq=0 out_seq=0 2023-07-14T11:44:12.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f533a7fc700 1 -- 172.21.15.161:0/467603280 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556d7dd6a050 con 0x556d7ded0a60 2023-07-14T11:44:12.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f533a7fc700 10 client.27823 handle_command_reply: tid=0 2023-07-14T11:44:12.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 client.27823 shutdown 2023-07-14T11:44:12.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 msgr2=0x556d7de6d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de6d980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f534409ae20 tx=0x7f5344009ca0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 shutdown_connections 2023-07-14T11:44:12.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x556d7ded0a60 0x556d7ded2e70 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d7de6bee0 0x556d7de6e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f533a7fc700 0 client.27823 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:12.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x556d7de5c2d0 0x556d7de6d980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 --2- 172.21.15.161:0/467603280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7de5cbe0 0x556d7de6dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.436+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 >> 172.21.15.161:0/467603280 conn(0x556d7de52da0 msgr2=0x556d7de69f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:12.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.437+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 shutdown_connections 2023-07-14T11:44:12.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.437+0000 7f536d1abb80 1 -- 172.21.15.161:0/467603280 wait complete. 2023-07-14T11:44:12.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.437+0000 7f536d1abb80 20 client.27823 trim_cache size 0 max 16384 2023-07-14T11:44:12.454 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:44:12.455 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:44:12.455 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:44:12.456 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:44:12.456 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:44:12.457 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:44:12.457 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:44:12.457 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:44:12.458 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:44:12.458 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:44:12.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.437+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f53400a12c0 msgr2=0x7f53400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.437+0000 7f53597ff700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f53400a12c0 0x7f53400a3770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f534c009c10 tx=0x7f534c00b040 comp rx=0 tx=0).stop 2023-07-14T11:44:12.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.437+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53540ffad0 msgr2=0x7f535419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:12.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53540ffad0 0x7f535419ba20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5344005910 tx=0x7f5344004710 comp rx=0 tx=0).stop 2023-07-14T11:44:12.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 shutdown_connections 2023-07-14T11:44:12.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f53400a12c0 0x7f53400a3770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5354105ae0 0x7f5354195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53540ffad0 0x7f535419ba20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 --2- 172.21.15.161:0/1628531499 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f5354100480 0x7f535419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:12.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 >> 172.21.15.161:0/1628531499 conn(0x7f53540fb5f0 msgr2=0x7f53540fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:12.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 shutdown_connections 2023-07-14T11:44:12.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:12.438+0000 7f53597ff700 1 -- 172.21.15.161:0/1628531499 wait complete. 2023-07-14T11:44:12.462 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:44:12.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:12 smithi161 ceph-mon[95526]: pgmap v691: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:44:12.985 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:12 smithi161 ceph-mon[95526]: osdmap e124: 12 total, 12 up, 12 in 2023-07-14T11:44:13.156 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:12 smithi175 ceph-mon[129081]: pgmap v691: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:44:13.156 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:12 smithi175 ceph-mon[129081]: osdmap e124: 12 total, 12 up, 12 in 2023-07-14T11:44:13.201 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:12 smithi182 ceph-mon[129366]: pgmap v691: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:44:13.201 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:12 smithi182 ceph-mon[129366]: osdmap e124: 12 total, 12 up, 12 in 2023-07-14T11:44:14.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:14 smithi175 ceph-mon[129081]: pgmap v693: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:14.451 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:14 smithi182 ceph-mon[129366]: pgmap v693: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:14.482 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:14 smithi161 ceph-mon[95526]: pgmap v693: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:16.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:16 smithi175 ceph-mon[129081]: pgmap v694: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:16.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:16 smithi182 ceph-mon[129366]: pgmap v694: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:16.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:16 smithi161 ceph-mon[95526]: pgmap v694: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:18.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:18 smithi175 ceph-mon[129081]: pgmap v695: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 7 op/s 2023-07-14T11:44:18.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:18 smithi175 ceph-mon[129081]: osdmap e125: 12 total, 12 up, 12 in 2023-07-14T11:44:18.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:18 smithi182 ceph-mon[129366]: pgmap v695: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 7 op/s 2023-07-14T11:44:18.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:18 smithi182 ceph-mon[129366]: osdmap e125: 12 total, 12 up, 12 in 2023-07-14T11:44:18.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:18 smithi161 ceph-mon[95526]: pgmap v695: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 7 op/s 2023-07-14T11:44:18.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:18 smithi161 ceph-mon[95526]: osdmap e125: 12 total, 12 up, 12 in 2023-07-14T11:44:20.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:20 smithi175 ceph-mon[129081]: pgmap v697: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 6 op/s 2023-07-14T11:44:20.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:20 smithi182 ceph-mon[129366]: pgmap v697: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 6 op/s 2023-07-14T11:44:20.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:20 smithi161 ceph-mon[95526]: pgmap v697: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 6 op/s 2023-07-14T11:44:22.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:22 smithi175 ceph-mon[129081]: pgmap v698: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-14T11:44:22.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:22 smithi182 ceph-mon[129366]: pgmap v698: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-14T11:44:22.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:22 smithi161 ceph-mon[95526]: pgmap v698: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-14T11:44:24.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:24 smithi175 ceph-mon[129081]: pgmap v699: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:24.451 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:24 smithi182 ceph-mon[129366]: pgmap v699: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:24.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:24 smithi161 ceph-mon[95526]: pgmap v699: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:26.910 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:26 smithi175 ceph-mon[129081]: pgmap v700: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:26.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:26 smithi182 ceph-mon[129366]: pgmap v700: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:26.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:26 smithi161 ceph-mon[95526]: pgmap v700: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:44:28.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:28 smithi175 ceph-mon[129081]: pgmap v701: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:44:28.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:28 smithi182 ceph-mon[129366]: pgmap v701: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:44:28.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:28 smithi161 ceph-mon[95526]: pgmap v701: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:44:30.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:30 smithi175 ceph-mon[129081]: pgmap v702: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:44:30.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:30 smithi182 ceph-mon[129366]: pgmap v702: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:44:30.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:30 smithi161 ceph-mon[95526]: pgmap v702: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:44:32.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:32 smithi175 ceph-mon[129081]: pgmap v703: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:32.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:32 smithi182 ceph-mon[129366]: pgmap v703: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:32.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:32 smithi161 ceph-mon[95526]: pgmap v703: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:34.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:34 smithi175 ceph-mon[129081]: pgmap v704: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:34.450 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:34 smithi182 ceph-mon[129366]: pgmap v704: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:34.482 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:34 smithi161 ceph-mon[95526]: pgmap v704: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:36.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:36 smithi175 ceph-mon[129081]: pgmap v705: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:36.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:36 smithi182 ceph-mon[129366]: pgmap v705: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:36.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:36 smithi161 ceph-mon[95526]: pgmap v705: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:38.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:38 smithi175 ceph-mon[129081]: pgmap v706: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:38.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:38 smithi182 ceph-mon[129366]: pgmap v706: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:38.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:38 smithi161 ceph-mon[95526]: pgmap v706: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:40.909 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:40 smithi175 ceph-mon[129081]: pgmap v707: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:40.951 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:40 smithi182 ceph-mon[129366]: pgmap v707: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:40.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:40 smithi161 ceph-mon[95526]: pgmap v707: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:42.463 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:44:42.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 -- 172.21.15.161:0/73386450 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce000728b0 msgr2=0x7fce00072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 --2- 172.21.15.161:0/73386450 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce000728b0 0x7fce00072c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009990 tx=0x7fcdf0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 -- 172.21.15.161:0/73386450 shutdown_connections 2023-07-14T11:44:42.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 --2- 172.21.15.161:0/73386450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce00110af0 0x7fce00112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 --2- 172.21.15.161:0/73386450 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce000728b0 0x7fce00072c80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 --2- 172.21.15.161:0/73386450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce000731c0 0x7fce001105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 -- 172.21.15.161:0/73386450 >> 172.21.15.161:0/73386450 conn(0x7fce000fb5e0 msgr2=0x7fce000fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:42.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 -- 172.21.15.161:0/73386450 shutdown_connections 2023-07-14T11:44:42.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.691+0000 7fce061ff700 1 -- 172.21.15.161:0/73386450 wait complete. 2023-07-14T11:44:42.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.692+0000 7fce061ff700 1 Processor -- start 2023-07-14T11:44:42.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.692+0000 7fce061ff700 1 -- start start 2023-07-14T11:44:42.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fce061ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce000728b0 0x7fce0019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fce061ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce000731c0 0x7fce0019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fce061ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce00110af0 0x7fce001a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fce061ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce001a1260 con 0x7fce000728b0 2023-07-14T11:44:42.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fce061ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce001a13d0 con 0x7fce000731c0 2023-07-14T11:44:42.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fce061ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce001a1590 con 0x7fce00110af0 2023-07-14T11:44:42.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce000731c0 0x7fce0019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce00110af0 0x7fce001a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce000731c0 0x7fce0019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57602/0 (socket says 172.21.15.161:57602) 2023-07-14T11:44:42.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdff7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce000728b0 0x7fce0019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 -- 172.21.15.161:0/1656235958 learned_addr learned my addr 172.21.15.161:0/1656235958 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:42.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 -- 172.21.15.161:0/1656235958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce00110af0 msgr2=0x7fce001a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce00110af0 0x7fce001a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 -- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce000728b0 msgr2=0x7fce0019c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce000728b0 0x7fce0019c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.693+0000 7fcdfeffd700 1 -- 172.21.15.161:0/1656235958 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4009520 con 0x7fce000731c0 2023-07-14T11:44:42.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fcdfffff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce00110af0 0x7fce001a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:44:42.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce000728b0 0x7fce0019c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:44:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce000731c0 0x7fce0019c7c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400eb30 tx=0x7fcdf400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fcdfcff9700 1 -- 172.21.15.161:0/1656235958 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4009880 con 0x7fce000731c0 2023-07-14T11:44:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fcdfcff9700 1 -- 172.21.15.161:0/1656235958 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf40044e0 con 0x7fce000731c0 2023-07-14T11:44:42.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf00095f0 con 0x7fce000731c0 2023-07-14T11:44:42.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fcdfcff9700 1 -- 172.21.15.161:0/1656235958 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4003d80 con 0x7fce000731c0 2023-07-14T11:44:42.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.694+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce001a8ed0 con 0x7fce000731c0 2023-07-14T11:44:42.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.697+0000 7fcdfcff9700 1 -- 172.21.15.161:0/1656235958 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcdf4003f20 con 0x7fce000731c0 2023-07-14T11:44:42.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fce19bebb80 1 Processor -- start 2023-07-14T11:44:42.706 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fce19bebb80 1 -- start start 2023-07-14T11:44:42.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fcdfcff9700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcde80a12c0 0x7fcde80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171c9c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.707 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171ca7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.708 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fce19bebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171ba2cb0 con 0x558171c9c2d0 2023-07-14T11:44:42.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.699+0000 7fce19bebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171bf5a40 con 0x558171ca7ad0 2023-07-14T11:44:42.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fce19bebb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171beeed0 con 0x558171c9cbe0 2023-07-14T11:44:42.709 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfcff9700 1 -- 172.21.15.161:0/1656235958 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fcdf4014070 con 0x7fce000731c0 2023-07-14T11:44:42.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfffff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcde80a12c0 0x7fcde80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdff7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171c9c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.710 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171ca7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.711 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171ca7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60030/0 (socket says 172.21.15.161:60030) 2023-07-14T11:44:42.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57616/0 (socket says 172.21.15.161:57616) 2023-07-14T11:44:42.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdff7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171c9c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47682/0 (socket says 172.21.15.161:47682) 2023-07-14T11:44:42.712 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 -- 172.21.15.161:0/2876449442 learned_addr learned my addr 172.21.15.161:0/2876449442 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 -- 172.21.15.161:0/2876449442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 msgr2=0x558171ca9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/2876449442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 -- 172.21.15.161:0/2876449442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 msgr2=0x558171c9c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/2876449442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171c9c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 -- 172.21.15.161:0/2876449442 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558171a1cde0 con 0x558171c9cbe0 2023-07-14T11:44:42.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/2876449442 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171ca7590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40044b0 tx=0x7fcdf409a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8018ee45c924bf87 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:42.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.700+0000 7fcdfffff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcde80a12c0 0x7fcde80a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009fd0 tx=0x7fcdf8009310 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:42.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fcde5ffb700 1 -- 172.21.15.161:0/2876449442 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40b1070 con 0x558171c9cbe0 2023-07-14T11:44:42.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fcde5ffb700 1 -- 172.21.15.161:0/2876449442 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf400cbd0 con 0x558171c9cbe0 2023-07-14T11:44:42.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fcde5ffb700 1 -- 172.21.15.161:0/2876449442 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40986d0 con 0x558171c9cbe0 2023-07-14T11:44:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 -- 172.21.15.161:0/2876449442 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 msgr2=0x558171ca7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 --2- 172.21.15.161:0/2876449442 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171ca7590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40044b0 tx=0x7fcdf409a230 comp rx=0 tx=0).stop 2023-07-14T11:44:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 -- 172.21.15.161:0/2876449442 shutdown_connections 2023-07-14T11:44:42.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 --2- 172.21.15.161:0/2876449442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171c9c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 --2- 172.21.15.161:0/2876449442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 --2- 172.21.15.161:0/2876449442 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171ca7590 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 -- 172.21.15.161:0/2876449442 >> 172.21.15.161:0/2876449442 conn(0x558171c92da0 msgr2=0x558171c986a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:42.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 -- 172.21.15.161:0/2876449442 shutdown_connections 2023-07-14T11:44:42.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.701+0000 7fce19bebb80 1 -- 172.21.15.161:0/2876449442 wait complete. 2023-07-14T11:44:42.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.702+0000 7fce19bebb80 1 -- start start 2023-07-14T11:44:42.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.702+0000 7fce19bebb80 1 Processor -- start 2023-07-14T11:44:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171ca4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171ca5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fce19bebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171b8eac0 con 0x558171c9c2d0 2023-07-14T11:44:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fce19bebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171b8e7c0 con 0x558171c9cbe0 2023-07-14T11:44:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fce19bebb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171a72920 con 0x558171ca7ad0 2023-07-14T11:44:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fce19bebb80 10 client.0 _ll_register_callbacks cb 0x558171b8c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:44:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171ca4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171ca5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171ca4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57628/0 (socket says 172.21.15.161:57628) 2023-07-14T11:44:42.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 -- 172.21.15.161:0/4056431048 learned_addr learned my addr 172.21.15.161:0/4056431048 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:42.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 -- 172.21.15.161:0/4056431048 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 msgr2=0x558171ca5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171ca5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 -- 172.21.15.161:0/4056431048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 msgr2=0x558171ca4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 -- 172.21.15.161:0/4056431048 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce001a8ed0 con 0x558171c9cbe0 2023-07-14T11:44:42.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfffff700 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171ca5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:44:42.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171ca4dc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400eb00 tx=0x7fcdf400cd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc26a34028b80055 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:42.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.703+0000 7fcddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:44:42.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fcddeffd700 1 -- 172.21.15.161:0/4056431048 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40b1030 con 0x558171c9cbe0 2023-07-14T11:44:42.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fcddeffd700 1 -- 172.21.15.161:0/4056431048 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fcdf40b2070 con 0x558171c9cbe0 2023-07-14T11:44:42.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558171b484f0 con 0x558171c9cbe0 2023-07-14T11:44:42.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fcddeffd700 1 -- 172.21.15.161:0/4056431048 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf409e360 con 0x558171c9cbe0 2023-07-14T11:44:42.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 10 client.27791 fetch_fsmap learned FSMap version 46 2023-07-14T11:44:42.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x558171a05c60 con 0x558171c9cbe0 2023-07-14T11:44:42.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fcddeffd700 1 -- 172.21.15.161:0/4056431048 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558171b484f0 con 0x558171c9cbe0 2023-07-14T11:44:42.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fcddeffd700 1 -- 172.21.15.161:0/4056431048 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcdf4099820 con 0x558171c9cbe0 2023-07-14T11:44:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 10 client.27791 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:44:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 10 client.27791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:44:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 20 client.27791 populate_metadata read hostname 'smithi161' 2023-07-14T11:44:42.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171cb70c0 0x558171cb94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 4 client.27791 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:44:42.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.704+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558171be7eb0 con 0x558171cb70c0 2023-07-14T11:44:42.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.705+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171cb70c0 0x558171cb94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.705+0000 7fcddeffd700 10 client.27791 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:42.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.705+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171cb70c0 0x558171cb94d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d45ab733ce12092 server_cookie=1dc9beda58fb5ade in_seq=0 out_seq=0 2023-07-14T11:44:42.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fcddeffd700 1 -- 172.21.15.161:0/4056431048 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558171be7eb0 con 0x558171cb70c0 2023-07-14T11:44:42.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fcddeffd700 10 client.27791 handle_command_reply: tid=0 2023-07-14T11:44:42.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 client.27791 shutdown 2023-07-14T11:44:42.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 msgr2=0x558171ca4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171ca4dc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400eb00 tx=0x7fcdf400cd40 comp rx=0 tx=0).stop 2023-07-14T11:44:42.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 shutdown_connections 2023-07-14T11:44:42.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171cb70c0 0x558171cb94d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fcddeffd700 0 client.27791 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:42.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171ca5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 --2- 172.21.15.161:0/4056431048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171ca4dc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 >> 172.21.15.161:0/4056431048 conn(0x558171c92da0 msgr2=0x558171c98290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:42.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 shutdown_connections 2023-07-14T11:44:42.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.706+0000 7fce19bebb80 1 -- 172.21.15.161:0/4056431048 wait complete. 2023-07-14T11:44:42.740 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.707+0000 7fce19bebb80 20 client.27791 trim_cache size 0 max 16384 2023-07-14T11:44:42.748 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.723+0000 7fce19bebb80 1 Processor -- start 2023-07-14T11:44:42.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fce19bebb80 1 -- start start 2023-07-14T11:44:42.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171cae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fce19bebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171c9d540 con 0x558171c9c2d0 2023-07-14T11:44:42.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fce19bebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171ca78c0 con 0x558171ca7ad0 2023-07-14T11:44:42.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fce19bebb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171cb3e60 con 0x558171c9cbe0 2023-07-14T11:44:42.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171cae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:57638/0 (socket says 172.21.15.161:57638) 2023-07-14T11:44:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171cae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60052/0 (socket says 172.21.15.161:60052) 2023-07-14T11:44:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfffff700 1 -- 172.21.15.161:0/1924770570 learned_addr learned my addr 172.21.15.161:0/1924770570 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:42.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfeffd700 1 -- 172.21.15.161:0/1924770570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 msgr2=0x558171ca1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.755 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfeffd700 1 -- 172.21.15.161:0/1924770570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 msgr2=0x558171ca3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.724+0000 7fcdfeffd700 1 -- 172.21.15.161:0/1924770570 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558171c945e0 con 0x558171c9cbe0 2023-07-14T11:44:42.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171cae4a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4096da0 tx=0x7fcdf40988e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=516b3136632c775e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:42.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fcde67fc700 1 -- 172.21.15.161:0/1924770570 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40a2390 con 0x558171c9cbe0 2023-07-14T11:44:42.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fcde67fc700 1 -- 172.21.15.161:0/1924770570 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf40a2580 con 0x558171c9cbe0 2023-07-14T11:44:42.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fcde67fc700 1 -- 172.21.15.161:0/1924770570 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf409a3c0 con 0x558171c9cbe0 2023-07-14T11:44:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fce19bebb80 1 -- 172.21.15.161:0/1924770570 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 msgr2=0x558171cae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fce19bebb80 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171cae4a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4096da0 tx=0x7fcdf40988e0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fce19bebb80 1 -- 172.21.15.161:0/1924770570 shutdown_connections 2023-07-14T11:44:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fce19bebb80 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171ca3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fce19bebb80 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171ca7ad0 0x558171ca1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fce19bebb80 1 --2- 172.21.15.161:0/1924770570 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171c9cbe0 0x558171cae4a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.725+0000 7fce19bebb80 1 -- 172.21.15.161:0/1924770570 >> 172.21.15.161:0/1924770570 conn(0x558171c92da0 msgr2=0x558171c9fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:42.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.726+0000 7fce19bebb80 1 -- 172.21.15.161:0/1924770570 shutdown_connections 2023-07-14T11:44:42.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.726+0000 7fce19bebb80 1 -- 172.21.15.161:0/1924770570 wait complete. 2023-07-14T11:44:42.763 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.726+0000 7fce19bebb80 1 -- start start 2023-07-14T11:44:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.726+0000 7fce19bebb80 1 Processor -- start 2023-07-14T11:44:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171cac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171cac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fce19bebb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171cacf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fce19bebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171d0a020 con 0x558171c9c2d0 2023-07-14T11:44:42.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fce19bebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171d0a160 con 0x558171c9cbe0 2023-07-14T11:44:42.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fce19bebb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558171d0a320 con 0x558171ca7ad0 2023-07-14T11:44:42.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fce19bebb80 10 client.0 _ll_register_callbacks cb 0x7fcdf4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:44:42.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171cacf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171cac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171cacf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:60058/0 (socket says 172.21.15.161:60058) 2023-07-14T11:44:42.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfffff700 1 -- 172.21.15.161:0/349472523 learned_addr learned my addr 172.21.15.161:0/349472523 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:44:42.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171cac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.766 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfeffd700 1 -- 172.21.15.161:0/349472523 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 msgr2=0x558171cacf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171cacf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfeffd700 1 -- 172.21.15.161:0/349472523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 msgr2=0x558171cac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:44:42.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171cac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfeffd700 1 -- 172.21.15.161:0/349472523 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558171a05c60 con 0x558171c9cbe0 2023-07-14T11:44:42.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.727+0000 7fcdfffff700 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171cacf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:44:42.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fcdfeffd700 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171cac9d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40a2940 tx=0x7fcdf409a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4acf60163e1e5bb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:44:42.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fcddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:44:42.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fcddeffd700 1 -- 172.21.15.161:0/349472523 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40b1070 con 0x558171c9cbe0 2023-07-14T11:44:42.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fcddeffd700 1 -- 172.21.15.161:0/349472523 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fcdf409e360 con 0x558171c9cbe0 2023-07-14T11:44:42.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdf400cbd0 con 0x558171c9cbe0 2023-07-14T11:44:42.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fcddeffd700 1 -- 172.21.15.161:0/349472523 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf40a2390 con 0x558171c9cbe0 2023-07-14T11:44:42.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 10 client.27797 fetch_fsmap learned FSMap version 46 2023-07-14T11:44:42.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x558171a43250 con 0x558171c9cbe0 2023-07-14T11:44:42.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fcddeffd700 1 -- 172.21.15.161:0/349472523 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf400cbd0 con 0x558171c9cbe0 2023-07-14T11:44:42.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fcddeffd700 1 -- 172.21.15.161:0/349472523 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcdf4099290 con 0x558171c9cbe0 2023-07-14T11:44:42.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 10 client.27797 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:44:42.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 10 client.27797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:44:42.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 20 client.27797 populate_metadata read hostname 'smithi161' 2023-07-14T11:44:42.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171d0b560 0x558171d0d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:44:42.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 4 client.27797 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:44:42.772 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.728+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558171baa050 con 0x558171d0b560 2023-07-14T11:44:42.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171d0b560 0x558171d0d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:44:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fcdff7fe700 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171d0b560 0x558171d0d940 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291e1a564a2e4c55 server_cookie=27c442d86cf0c82c in_seq=0 out_seq=0 2023-07-14T11:44:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fcddeffd700 10 client.27797 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fcddeffd700 1 -- 172.21.15.161:0/349472523 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558171baa050 con 0x558171d0b560 2023-07-14T11:44:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fcddeffd700 10 client.27797 handle_command_reply: tid=0 2023-07-14T11:44:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 client.27797 shutdown 2023-07-14T11:44:42.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 msgr2=0x558171cac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171cac9d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40a2940 tx=0x7fcdf409a230 comp rx=0 tx=0).stop 2023-07-14T11:44:42.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 shutdown_connections 2023-07-14T11:44:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x558171d0b560 0x558171d0d940 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558171c9c2d0 0x558171cac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558171ca7ad0 0x558171cacf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fcddeffd700 0 client.27797 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:44:42.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 --2- 172.21.15.161:0/349472523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558171c9cbe0 0x558171cac9d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.729+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 >> 172.21.15.161:0/349472523 conn(0x558171c92da0 msgr2=0x558171c9e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:42.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 shutdown_connections 2023-07-14T11:44:42.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce19bebb80 1 -- 172.21.15.161:0/349472523 wait complete. 2023-07-14T11:44:42.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce19bebb80 20 client.27797 trim_cache size 0 max 16384 2023-07-14T11:44:42.778 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:44:42.778 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:44:42.778 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:44:42.779 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:44:42.779 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:44:42.779 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:44:42.779 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:44:42.779 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:44:42.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:44:42.780 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:44:42.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcde80a12c0 msgr2=0x7fcde80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcde80a12c0 0x7fcde80a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009fd0 tx=0x7fcdf8009310 comp rx=0 tx=0).stop 2023-07-14T11:44:42.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce000731c0 msgr2=0x7fce0019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:44:42.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce000731c0 0x7fce0019c7c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400eb30 tx=0x7fcdf400ee40 comp rx=0 tx=0).stop 2023-07-14T11:44:42.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 shutdown_connections 2023-07-14T11:44:42.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcde80a12c0 0x7fcde80a3770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce000728b0 0x7fce0019c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fce00110af0 0x7fce001a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 --2- 172.21.15.161:0/1656235958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce000731c0 0x7fce0019c7c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:44:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 >> 172.21.15.161:0/1656235958 conn(0x7fce000fb5e0 msgr2=0x7fce000fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:44:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.730+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 shutdown_connections 2023-07-14T11:44:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:44:42.731+0000 7fce061ff700 1 -- 172.21.15.161:0/1656235958 wait complete. 2023-07-14T11:44:42.785 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:44:42.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:42 smithi175 ceph-mon[129081]: pgmap v708: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:42.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:42 smithi182 ceph-mon[129366]: pgmap v708: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:42.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:42 smithi161 ceph-mon[95526]: pgmap v708: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:44.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:44 smithi175 ceph-mon[129081]: pgmap v709: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:44.450 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:44 smithi182 ceph-mon[129366]: pgmap v709: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:44.483 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:44 smithi161 ceph-mon[95526]: pgmap v709: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:46.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:46 smithi175 ceph-mon[129081]: pgmap v710: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:46.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:46 smithi182 ceph-mon[129366]: pgmap v710: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:46.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:46 smithi161 ceph-mon[95526]: pgmap v710: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:48.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:48 smithi175 ceph-mon[129081]: pgmap v711: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:48.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:48 smithi182 ceph-mon[129366]: pgmap v711: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:48.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:48 smithi161 ceph-mon[95526]: pgmap v711: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:50.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:50 smithi175 ceph-mon[129081]: pgmap v712: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:50.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:50 smithi182 ceph-mon[129366]: pgmap v712: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:50.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:50 smithi161 ceph-mon[95526]: pgmap v712: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:52.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:52 smithi175 ceph-mon[129081]: pgmap v713: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:52.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:52 smithi182 ceph-mon[129366]: pgmap v713: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:52.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:52 smithi161 ceph-mon[95526]: pgmap v713: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:44:54.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:54 smithi175 ceph-mon[129081]: pgmap v714: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:54.451 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:54 smithi182 ceph-mon[129366]: pgmap v714: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:54.482 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:54 smithi161 ceph-mon[95526]: pgmap v714: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:44:56.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:56 smithi175 ceph-mon[129081]: pgmap v715: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:56.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:56 smithi182 ceph-mon[129366]: pgmap v715: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:56.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:56 smithi161 ceph-mon[95526]: pgmap v715: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:44:58.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:44:58 smithi175 ceph-mon[129081]: pgmap v716: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:58.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:44:58 smithi182 ceph-mon[129366]: pgmap v716: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:44:58.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:44:58 smithi161 ceph-mon[95526]: pgmap v716: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:00.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:00 smithi175 ceph-mon[129081]: pgmap v717: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:00.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:00 smithi182 ceph-mon[129366]: pgmap v717: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:00.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:00 smithi161 ceph-mon[95526]: pgmap v717: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:02.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:02 smithi175 ceph-mon[129081]: pgmap v718: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-14T11:45:02.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:02 smithi182 ceph-mon[129366]: pgmap v718: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-14T11:45:02.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:02 smithi161 ceph-mon[95526]: pgmap v718: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-14T11:45:04.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:04 smithi175 ceph-mon[129081]: pgmap v719: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:45:04.452 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:04 smithi182 ceph-mon[129366]: pgmap v719: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:45:04.482 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:04 smithi161 ceph-mon[95526]: pgmap v719: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-14T11:45:06.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:06 smithi175 ceph-mon[129081]: pgmap v720: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:45:06.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:06 smithi182 ceph-mon[129366]: pgmap v720: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:45:06.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:06 smithi161 ceph-mon[95526]: pgmap v720: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:45:08.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:08 smithi175 ceph-mon[129081]: pgmap v721: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-14T11:45:08.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:08 smithi182 ceph-mon[129366]: pgmap v721: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-14T11:45:08.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:08 smithi161 ceph-mon[95526]: pgmap v721: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-14T11:45:10.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:10 smithi175 ceph-mon[129081]: pgmap v722: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:45:10.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:10 smithi182 ceph-mon[129366]: pgmap v722: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:45:10.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:10 smithi161 ceph-mon[95526]: pgmap v722: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-14T11:45:12.786 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:45:12.847 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:12 smithi161 ceph-mon[95526]: pgmap v723: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:45:12.848 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:12 smithi161 ceph-mon[95526]: osdmap e126: 12 total, 12 up, 12 in 2023-07-14T11:45:12.848 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:12 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:45:12.848 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:12 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.848 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:12 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.849 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:12 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.849 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:12 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:12 smithi175 ceph-mon[129081]: pgmap v723: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:45:12.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:12 smithi175 ceph-mon[129081]: osdmap e126: 12 total, 12 up, 12 in 2023-07-14T11:45:12.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:12 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:45:12.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:12 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:12 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:12 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:12 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.952 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:12 smithi182 ceph-mon[129366]: pgmap v723: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:45:12.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:12 smithi182 ceph-mon[129366]: osdmap e126: 12 total, 12 up, 12 in 2023-07-14T11:45:12.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:12 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:45:12.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:12 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.953 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:12 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:12 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.954 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:12 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:12.999 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 -- 172.21.15.161:0/435148364 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940728b0 msgr2=0x7fc994072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.000 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 --2- 172.21.15.161:0/435148364 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940728b0 0x7fc994072c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc988009990 tx=0x7fc988009ca0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 -- 172.21.15.161:0/435148364 shutdown_connections 2023-07-14T11:45:13.001 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 --2- 172.21.15.161:0/435148364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99410c310 0x7fc99410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 --2- 172.21.15.161:0/435148364 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940728b0 0x7fc994072c80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 --2- 172.21.15.161:0/435148364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9940731c0 0x7fc99410bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.002 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 -- 172.21.15.161:0/435148364 >> 172.21.15.161:0/435148364 conn(0x7fc9940fb5f0 msgr2=0x7fc9940fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:13.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 -- 172.21.15.161:0/435148364 shutdown_connections 2023-07-14T11:45:13.003 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.010+0000 7fc99abff700 1 -- 172.21.15.161:0/435148364 wait complete. 2023-07-14T11:45:13.004 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 Processor -- start 2023-07-14T11:45:13.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 -- start start 2023-07-14T11:45:13.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9940728b0 0x7fc994197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940731c0 0x7fc994198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99410c310 0x7fc99419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.005 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99419ce30 con 0x7fc9940728b0 2023-07-14T11:45:13.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99419cfa0 con 0x7fc99410c310 2023-07-14T11:45:13.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.012+0000 7fc99abff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99419d160 con 0x7fc9940731c0 2023-07-14T11:45:13.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc99899b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9940728b0 0x7fc994197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940731c0 0x7fc994198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc99919c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99410c310 0x7fc99419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940731c0 0x7fc994198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:39096/0 (socket says 172.21.15.161:39096) 2023-07-14T11:45:13.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 -- 172.21.15.161:0/4209051654 learned_addr learned my addr 172.21.15.161:0/4209051654 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:13.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 -- 172.21.15.161:0/4209051654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99410c310 msgr2=0x7fc99419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.007 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99410c310 0x7fc99419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 -- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9940728b0 msgr2=0x7fc994197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9940728b0 0x7fc994197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.008 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 -- 172.21.15.161:0/4209051654 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880095f0 con 0x7fc9940731c0 2023-07-14T11:45:13.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc99919c700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99410c310 0x7fc99419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:45:13.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc993fff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940731c0 0x7fc994198390 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e9a0 tx=0x7fc98000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:13.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc991ffb700 1 -- 172.21.15.161:0/4209051654 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980009790 con 0x7fc9940731c0 2023-07-14T11:45:13.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc991ffb700 1 -- 172.21.15.161:0/4209051654 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc980010430 con 0x7fc9940731c0 2023-07-14T11:45:13.009 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.013+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99419d440 con 0x7fc9940731c0 2023-07-14T11:45:13.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.014+0000 7fc991ffb700 1 -- 172.21.15.161:0/4209051654 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980018680 con 0x7fc9940731c0 2023-07-14T11:45:13.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.014+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9941a4c80 con 0x7fc9940731c0 2023-07-14T11:45:13.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.017+0000 7fc991ffb700 1 -- 172.21.15.161:0/4209051654 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc9800105a0 con 0x7fc9940731c0 2023-07-14T11:45:13.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 Processor -- start 2023-07-14T11:45:13.015 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 -- start start 2023-07-14T11:45:13.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc991ffb700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9840a12c0 0x7fc9840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9ad32d0 0x55ece9ad36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.016 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc991ffb700 1 -- 172.21.15.161:0/4209051654 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fc980014070 con 0x7fc9940731c0 2023-07-14T11:45:13.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ade590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9adead0 0x55ece9ae0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece99d9cb0 con 0x55ece9adead0 2023-07-14T11:45:13.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece9a2ca40 con 0x55ece9ad32d0 2023-07-14T11:45:13.017 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.018+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece9a25ed0 con 0x55ece9ad3be0 2023-07-14T11:45:13.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9ad32d0 0x55ece9ad36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ade590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.018 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9ad32d0 0x55ece9ad36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49446/0 (socket says 172.21.15.161:49446) 2023-07-14T11:45:13.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 -- 172.21.15.161:0/513632138 learned_addr learned my addr 172.21.15.161:0/513632138 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:13.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99919c700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9840a12c0 0x7fc9840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99919c700 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9adead0 0x55ece9ae0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.019 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 -- 172.21.15.161:0/513632138 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 msgr2=0x55ece9ade590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ade590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.020 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 -- 172.21.15.161:0/513632138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9adead0 msgr2=0x55ece9ae0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9adead0 0x55ece9ae0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 -- 172.21.15.161:0/513632138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ece9853de0 con 0x55ece9ad32d0 2023-07-14T11:45:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.019+0000 7fc99899b700 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9ad32d0 0x55ece9ad36a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc988005ea0 tx=0x7fc988005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d45843728c10ea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.020+0000 7fc99919c700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9840a12c0 0x7fc9840a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009fd0 tx=0x7fc98c009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.020+0000 7fc97effd700 1 -- 172.21.15.161:0/513632138 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98801d070 con 0x55ece9ad32d0 2023-07-14T11:45:13.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.020+0000 7fc97effd700 1 -- 172.21.15.161:0/513632138 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc98801e040 con 0x55ece9ad32d0 2023-07-14T11:45:13.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.020+0000 7fc97effd700 1 -- 172.21.15.161:0/513632138 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988004990 con 0x55ece9ad32d0 2023-07-14T11:45:13.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.020+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/513632138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9ad32d0 msgr2=0x55ece9ad36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.022 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.020+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9ad32d0 0x55ece9ad36a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc988005ea0 tx=0x7fc988005ed0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/513632138 shutdown_connections 2023-07-14T11:45:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9adead0 0x55ece9ae0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9ad32d0 0x55ece9ad36a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/513632138 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ade590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/513632138 >> 172.21.15.161:0/513632138 conn(0x55ece9ac9da0 msgr2=0x55ece9acf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/513632138 shutdown_connections 2023-07-14T11:45:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/513632138 wait complete. 2023-07-14T11:45:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 -- start start 2023-07-14T11:45:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.021+0000 7fc9ae60bb80 1 Processor -- start 2023-07-14T11:45:13.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ade4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9adb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9adbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.025 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece99c5ac0 con 0x55ece9ad32d0 2023-07-14T11:45:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece99c57c0 con 0x55ece9adead0 2023-07-14T11:45:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece98a9920 con 0x55ece9ad3be0 2023-07-14T11:45:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc9ae60bb80 10 client.0 _ll_register_callbacks cb 0x55ece99c3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:45:13.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9adb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc99919c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9adbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.027 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9adb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:39122/0 (socket says 172.21.15.161:39122) 2023-07-14T11:45:13.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 -- 172.21.15.161:0/3413601003 learned_addr learned my addr 172.21.15.161:0/3413601003 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:13.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc99899b700 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ade4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.028 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 -- 172.21.15.161:0/3413601003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 msgr2=0x55ece9adbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9adbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 -- 172.21.15.161:0/3413601003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 msgr2=0x55ece9ade4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ade4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.022+0000 7fc993fff700 1 -- 172.21.15.161:0/3413601003 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99410f390 con 0x55ece9ad3be0 2023-07-14T11:45:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc993fff700 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9adb730 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc980096a20 tx=0x7fc98009a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d0d7aac0be464b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc973fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:45:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc973fff700 1 -- 172.21.15.161:0/3413601003 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800a2400 con 0x55ece9ad3be0 2023-07-14T11:45:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ece997f4f0 con 0x55ece9ad3be0 2023-07-14T11:45:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc973fff700 1 -- 172.21.15.161:0/3413601003 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fc9800b26a0 con 0x55ece9ad3be0 2023-07-14T11:45:13.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc973fff700 1 -- 172.21.15.161:0/3413601003 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9800994a0 con 0x55ece9ad3be0 2023-07-14T11:45:13.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc9ae60bb80 10 client.27853 fetch_fsmap learned FSMap version 46 2023-07-14T11:45:13.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ece983cc60 con 0x55ece9ad3be0 2023-07-14T11:45:13.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc973fff700 1 -- 172.21.15.161:0/3413601003 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ece997f4f0 con 0x55ece9ad3be0 2023-07-14T11:45:13.031 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.023+0000 7fc973fff700 1 -- 172.21.15.161:0/3413601003 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9800b22d0 con 0x55ece9ad3be0 2023-07-14T11:45:13.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc9ae60bb80 10 client.27853 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:45:13.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc9ae60bb80 10 client.27853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:45:13.032 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc9ae60bb80 20 client.27853 populate_metadata read hostname 'smithi161' 2023-07-14T11:45:13.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9aee340 0x55ece9af0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc9ae60bb80 4 client.27853 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:45:13.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ece9a1eeb0 con 0x55ece9aee340 2023-07-14T11:45:13.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc99899b700 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9aee340 0x55ece9af0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc973fff700 10 client.27853 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:13.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.024+0000 7fc99899b700 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9aee340 0x55ece9af0750 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d172de4652f5bb36 server_cookie=89cd8ec7057d736b in_seq=0 out_seq=0 2023-07-14T11:45:13.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc973fff700 1 -- 172.21.15.161:0/3413601003 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ece9a1eeb0 con 0x55ece9aee340 2023-07-14T11:45:13.034 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc973fff700 10 client.27853 handle_command_reply: tid=0 2023-07-14T11:45:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 client.27853 shutdown 2023-07-14T11:45:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 msgr2=0x55ece9adb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9adb730 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc980096a20 tx=0x7fc98009a0f0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 shutdown_connections 2023-07-14T11:45:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9aee340 0x55ece9af0750 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ade4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9adbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3413601003 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9adb730 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 >> 172.21.15.161:0/3413601003 conn(0x55ece9ac9da0 msgr2=0x55ece9acf290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc973fff700 0 client.27853 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:13.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 shutdown_connections 2023-07-14T11:45:13.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.025+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3413601003 wait complete. 2023-07-14T11:45:13.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.026+0000 7fc9ae60bb80 20 client.27853 trim_cache size 0 max 16384 2023-07-14T11:45:13.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 Processor -- start 2023-07-14T11:45:13.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 -- start start 2023-07-14T11:45:13.042 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ad61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ae54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9ad89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece9ad4e70 con 0x55ece9ad32d0 2023-07-14T11:45:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece997f4f0 con 0x55ece9adead0 2023-07-14T11:45:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece9acbec0 con 0x55ece9ad3be0 2023-07-14T11:45:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ae54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc99919c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9ad89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ae54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:39128/0 (socket says 172.21.15.161:39128) 2023-07-14T11:45:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc993fff700 1 -- 172.21.15.161:0/1491818879 learned_addr learned my addr 172.21.15.161:0/1491818879 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.043+0000 7fc99899b700 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ad61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc993fff700 1 -- 172.21.15.161:0/1491818879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 msgr2=0x55ece9ad89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc993fff700 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9ad89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc993fff700 1 -- 172.21.15.161:0/1491818879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 msgr2=0x55ece9ad61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc993fff700 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ad61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc993fff700 1 -- 172.21.15.161:0/1491818879 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ece9ace0e0 con 0x55ece9ad3be0 2023-07-14T11:45:13.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc993fff700 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ae54b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc980099cb0 tx=0x7fc98000c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0359f54600b0ddd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:13.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc97f7fe700 1 -- 172.21.15.161:0/1491818879 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800b8370 con 0x55ece9ad3be0 2023-07-14T11:45:13.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc97f7fe700 1 -- 172.21.15.161:0/1491818879 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9800cd130 con 0x55ece9ad3be0 2023-07-14T11:45:13.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.044+0000 7fc97f7fe700 1 -- 172.21.15.161:0/1491818879 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980099700 con 0x55ece9ad3be0 2023-07-14T11:45:13.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/1491818879 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 msgr2=0x55ece9ae54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ae54b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc980099cb0 tx=0x7fc98000c480 comp rx=0 tx=0).stop 2023-07-14T11:45:13.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/1491818879 shutdown_connections 2023-07-14T11:45:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9ad61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9ad89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/1491818879 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9ae54b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/1491818879 >> 172.21.15.161:0/1491818879 conn(0x55ece9ac9da0 msgr2=0x55ece9ad6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/1491818879 shutdown_connections 2023-07-14T11:45:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/1491818879 wait complete. 2023-07-14T11:45:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 -- start start 2023-07-14T11:45:13.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.045+0000 7fc9ae60bb80 1 Processor -- start 2023-07-14T11:45:13.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9b3aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9b3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc9ae60bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9b36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece9b368d0 con 0x55ece9ad32d0 2023-07-14T11:45:13.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece9b36a40 con 0x55ece9adead0 2023-07-14T11:45:13.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc9ae60bb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ece9b36c00 con 0x55ece9ad3be0 2023-07-14T11:45:13.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc9ae60bb80 10 client.0 _ll_register_callbacks cb 0x7fc980099950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:45:13.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9b3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc99919c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9b36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc99899b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9b3aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9b3af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:39130/0 (socket says 172.21.15.161:39130) 2023-07-14T11:45:13.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 -- 172.21.15.161:0/3623976854 learned_addr learned my addr 172.21.15.161:0/3623976854 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:13.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 -- 172.21.15.161:0/3623976854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 msgr2=0x55ece9b36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9b36190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 -- 172.21.15.161:0/3623976854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 msgr2=0x55ece9b3aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:13.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9b3aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.046+0000 7fc993fff700 1 -- 172.21.15.161:0/3623976854 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ece983cc60 con 0x55ece9ad3be0 2023-07-14T11:45:13.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc993fff700 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9b3af70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc9800a2220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43e04aab654550a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:13.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc973fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:45:13.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc973fff700 1 -- 172.21.15.161:0/3623976854 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980099d80 con 0x55ece9ad3be0 2023-07-14T11:45:13.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc973fff700 1 -- 172.21.15.161:0/3623976854 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fc980094bc0 con 0x55ece9ad3be0 2023-07-14T11:45:13.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc98801e040 con 0x55ece9ad3be0 2023-07-14T11:45:13.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc973fff700 1 -- 172.21.15.161:0/3623976854 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc980095da0 con 0x55ece9ad3be0 2023-07-14T11:45:13.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 10 client.27859 fetch_fsmap learned FSMap version 46 2023-07-14T11:45:13.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ece987a250 con 0x55ece9ad3be0 2023-07-14T11:45:13.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc973fff700 1 -- 172.21.15.161:0/3623976854 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc98801e040 con 0x55ece9ad3be0 2023-07-14T11:45:13.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc973fff700 1 -- 172.21.15.161:0/3623976854 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc98009ad90 con 0x55ece9ad3be0 2023-07-14T11:45:13.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 10 client.27859 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:45:13.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 10 client.27859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:45:13.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 20 client.27859 populate_metadata read hostname 'smithi161' 2023-07-14T11:45:13.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9b43450 0x55ece9b45830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:13.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 4 client.27859 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:45:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.047+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ece99e1050 con 0x55ece9b43450 2023-07-14T11:45:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc99899b700 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9b43450 0x55ece9b45830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:13.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc973fff700 10 client.27859 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:13.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc99899b700 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9b43450 0x55ece9b45830 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ab4dfd4a6d9ebc server_cookie=960f80b24f58f091 in_seq=0 out_seq=0 2023-07-14T11:45:13.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc973fff700 1 -- 172.21.15.161:0/3623976854 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ece99e1050 con 0x55ece9b43450 2023-07-14T11:45:13.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc973fff700 10 client.27859 handle_command_reply: tid=0 2023-07-14T11:45:13.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 client.27859 shutdown 2023-07-14T11:45:13.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 msgr2=0x55ece9b3af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9b3af70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc9800a2220 comp rx=0 tx=0).stop 2023-07-14T11:45:13.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 shutdown_connections 2023-07-14T11:45:13.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ece9b43450 0x55ece9b45830 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ece9ad32d0 0x55ece9b3aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ece9adead0 0x55ece9b36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc973fff700 0 client.27859 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:13.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 --2- 172.21.15.161:0/3623976854 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ece9ad3be0 0x55ece9b3af70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.048+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 >> 172.21.15.161:0/3623976854 conn(0x55ece9ac9da0 msgr2=0x55ece9add040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:13.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 shutdown_connections 2023-07-14T11:45:13.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc9ae60bb80 1 -- 172.21.15.161:0/3623976854 wait complete. 2023-07-14T11:45:13.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc9ae60bb80 20 client.27859 trim_cache size 0 max 16384 2023-07-14T11:45:13.071 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:45:13.071 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:45:13.071 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:45:13.072 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:45:13.072 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:45:13.073 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:45:13.073 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:45:13.073 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:45:13.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:45:13.074 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:45:13.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9840a12c0 msgr2=0x7fc9840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9840a12c0 0x7fc9840a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009fd0 tx=0x7fc98c009920 comp rx=0 tx=0).stop 2023-07-14T11:45:13.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940731c0 msgr2=0x7fc994198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:13.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940731c0 0x7fc994198390 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e9a0 tx=0x7fc98000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 shutdown_connections 2023-07-14T11:45:13.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc9840a12c0 0x7fc9840a3770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9940728b0 0x7fc994197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99410c310 0x7fc99419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 --2- 172.21.15.161:0/4209051654 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc9940731c0 0x7fc994198390 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:13.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 >> 172.21.15.161:0/4209051654 conn(0x7fc9940fb5f0 msgr2=0x7fc99410e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:13.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.049+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 shutdown_connections 2023-07-14T11:45:13.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:13.050+0000 7fc99abff700 1 -- 172.21.15.161:0/4209051654 wait complete. 2023-07-14T11:45:13.082 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:45:13.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:13 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:45:13.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:13 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:45:13.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:13 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:13.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:13 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:45:13.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:13 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:45:13.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:13 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:13.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:13 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:45:13.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:13 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:45:13.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:13 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:45:14.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:14 smithi175 ceph-mon[129081]: pgmap v725: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:14.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:14 smithi182 ceph-mon[129366]: pgmap v725: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:14.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:14 smithi161 ceph-mon[95526]: pgmap v725: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:16.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:16 smithi175 ceph-mon[129081]: pgmap v726: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:16.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:16 smithi182 ceph-mon[129366]: pgmap v726: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:16.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:16 smithi161 ceph-mon[95526]: pgmap v726: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:18.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:18 smithi175 ceph-mon[129081]: pgmap v727: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:18.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:18 smithi175 ceph-mon[129081]: osdmap e127: 12 total, 12 up, 12 in 2023-07-14T11:45:18.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:18 smithi182 ceph-mon[129366]: pgmap v727: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:18.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:18 smithi182 ceph-mon[129366]: osdmap e127: 12 total, 12 up, 12 in 2023-07-14T11:45:18.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:18 smithi161 ceph-mon[95526]: pgmap v727: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:45:18.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:18 smithi161 ceph-mon[95526]: osdmap e127: 12 total, 12 up, 12 in 2023-07-14T11:45:20.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:20 smithi175 ceph-mon[129081]: pgmap v729: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T11:45:20.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:20 smithi182 ceph-mon[129366]: pgmap v729: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T11:45:20.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:20 smithi161 ceph-mon[95526]: pgmap v729: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T11:45:22.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:22 smithi175 ceph-mon[129081]: pgmap v730: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2023-07-14T11:45:22.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:22 smithi182 ceph-mon[129366]: pgmap v730: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2023-07-14T11:45:22.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:22 smithi161 ceph-mon[95526]: pgmap v730: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2023-07-14T11:45:24.404 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:24 smithi175 ceph-mon[129081]: pgmap v731: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:45:24.450 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:24 smithi182 ceph-mon[129366]: pgmap v731: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:45:24.482 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:24 smithi161 ceph-mon[95526]: pgmap v731: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:45:26.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:26 smithi175 ceph-mon[129081]: pgmap v732: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:45:26.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:26 smithi182 ceph-mon[129366]: pgmap v732: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:45:26.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:26 smithi161 ceph-mon[95526]: pgmap v732: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:45:28.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:28 smithi175 ceph-mon[129081]: pgmap v733: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:45:28.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:28 smithi182 ceph-mon[129366]: pgmap v733: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:45:28.983 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:28 smithi161 ceph-mon[95526]: pgmap v733: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:45:30.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:30 smithi175 ceph-mon[129081]: pgmap v734: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:45:30.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:30 smithi182 ceph-mon[129366]: pgmap v734: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:45:30.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:30 smithi161 ceph-mon[95526]: pgmap v734: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:45:32.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:32 smithi175 ceph-mon[129081]: pgmap v735: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:32.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:32 smithi182 ceph-mon[129366]: pgmap v735: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:32.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:32 smithi161 ceph-mon[95526]: pgmap v735: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:34.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:34 smithi182 ceph-mon[129366]: pgmap v736: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:34.481 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:34 smithi161 ceph-mon[95526]: pgmap v736: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:34.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:34 smithi175 ceph-mon[129081]: pgmap v736: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:36.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:36 smithi175 ceph-mon[129081]: pgmap v737: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:36.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:36 smithi182 ceph-mon[129366]: pgmap v737: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:36.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:36 smithi161 ceph-mon[95526]: pgmap v737: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:38.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:38 smithi175 ceph-mon[129081]: pgmap v738: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:38.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:38 smithi182 ceph-mon[129366]: pgmap v738: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:38.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:38 smithi161 ceph-mon[95526]: pgmap v738: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:40.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:40 smithi175 ceph-mon[129081]: pgmap v739: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:40.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:40 smithi182 ceph-mon[129366]: pgmap v739: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:40.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:40 smithi161 ceph-mon[95526]: pgmap v739: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:42.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:42 smithi175 ceph-mon[129081]: pgmap v740: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:42.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:42 smithi182 ceph-mon[129366]: pgmap v740: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:42.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:42 smithi161 ceph-mon[95526]: pgmap v740: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:43.082 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:45:43.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 -- 172.21.15.161:0/2550904287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 msgr2=0x7fb70010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 --2- 172.21.15.161:0/2550904287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 0x7fb70010c940 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800b0a0 tx=0x7fb6f800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 -- 172.21.15.161:0/2550904287 shutdown_connections 2023-07-14T11:45:43.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 --2- 172.21.15.161:0/2550904287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700105b10 0x7fb700105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 --2- 172.21.15.161:0/2550904287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 0x7fb70010c940 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 --2- 172.21.15.161:0/2550904287 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 0x7fb7000fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 -- 172.21.15.161:0/2550904287 >> 172.21.15.161:0/2550904287 conn(0x7fb7000fb600 msgr2=0x7fb7000fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:43.297 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 -- 172.21.15.161:0/2550904287 shutdown_connections 2023-07-14T11:45:43.298 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.308+0000 7fb7073ff700 1 -- 172.21.15.161:0/2550904287 wait complete. 2023-07-14T11:45:43.301 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.309+0000 7fb7073ff700 1 Processor -- start 2023-07-14T11:45:43.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.309+0000 7fb7073ff700 1 -- start start 2023-07-14T11:45:43.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.309+0000 7fb7073ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 0x7fb700197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.309+0000 7fb7073ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 0x7fb700198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.302 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.309+0000 7fb7073ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700105b10 0x7fb70019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.309+0000 7fb7073ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019ced0 con 0x7fb700105b10 2023-07-14T11:45:43.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.309+0000 7fb7073ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019d040 con 0x7fb7001004b0 2023-07-14T11:45:43.303 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb7073ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019d200 con 0x7fb7000ffb00 2023-07-14T11:45:43.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 0x7fb700197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.304 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70499a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 0x7fb700198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 0x7fb700197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56106/0 (socket says 172.21.15.161:56106) 2023-07-14T11:45:43.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 -- 172.21.15.161:0/1765907545 learned_addr learned my addr 172.21.15.161:0/1765907545 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:43.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70599c700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700105b10 0x7fb70019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.306 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 -- 172.21.15.161:0/1765907545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 msgr2=0x7fb700198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 0x7fb700198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 -- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700105b10 msgr2=0x7fb70019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700105b10 0x7fb70019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 -- 172.21.15.161:0/1765907545 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f8009d00 con 0x7fb7000ffb00 2023-07-14T11:45:43.307 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70599c700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700105b10 0x7fb70019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:45:43.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb70519b700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 0x7fb700197ef0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e9a0 tx=0x7fb6fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:43.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.310+0000 7fb6f67fc700 1 -- 172.21.15.161:0/1765907545 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc009790 con 0x7fb7000ffb00 2023-07-14T11:45:43.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.311+0000 7fb6f67fc700 1 -- 172.21.15.161:0/1765907545 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc00cbd0 con 0x7fb7000ffb00 2023-07-14T11:45:43.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.311+0000 7fb6f67fc700 1 -- 172.21.15.161:0/1765907545 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc018680 con 0x7fb7000ffb00 2023-07-14T11:45:43.308 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.311+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70019d4e0 con 0x7fb7000ffb00 2023-07-14T11:45:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.311+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7001a4d20 con 0x7fb7000ffb00 2023-07-14T11:45:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.314+0000 7fb6f67fc700 1 -- 172.21.15.161:0/1765907545 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb6fc00cd40 con 0x7fb7000ffb00 2023-07-14T11:45:43.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.315+0000 7fb71aca0b80 1 Processor -- start 2023-07-14T11:45:43.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.315+0000 7fb6f67fc700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb6ec0a12c0 0x7fb6ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.315+0000 7fb71aca0b80 1 -- start start 2023-07-14T11:45:43.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.315+0000 7fb6f67fc700 1 -- 172.21.15.161:0/1765907545 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fb6fc014070 con 0x7fb7000ffb00 2023-07-14T11:45:43.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70599c700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb6ec0a12c0 0x7fb6ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d43590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d43ad0 0x55a581d45eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581c3ecb0 con 0x55a581d38be0 2023-07-14T11:45:43.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581c91a40 con 0x55a581d43ad0 2023-07-14T11:45:43.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581c8aed0 con 0x55a581d382d0 2023-07-14T11:45:43.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56118/0 (socket says 172.21.15.161:56118) 2023-07-14T11:45:43.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 -- 172.21.15.161:0/3664227220 learned_addr learned my addr 172.21.15.161:0/3664227220 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:43.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70599c700 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d43ad0 0x55a581d45eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70499a700 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d43590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 -- 172.21.15.161:0/3664227220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d43ad0 msgr2=0x55a581d45eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d43ad0 0x55a581d45eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 -- 172.21.15.161:0/3664227220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 msgr2=0x55a581d43590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d43590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 -- 172.21.15.161:0/3664227220 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a581ab8de0 con 0x55a581d382d0 2023-07-14T11:45:43.324 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70519b700 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d386a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc096a50 tx=0x7fb6fc099fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be37f3701dc1c054 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:43.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.316+0000 7fb70599c700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb6ec0a12c0 0x7fb6ec0a3770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009610 tx=0x7fb6f800bd70 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:43.325 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb6eb7fe700 1 -- 172.21.15.161:0/3664227220 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a2430 con 0x55a581d382d0 2023-07-14T11:45:43.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb6eb7fe700 1 -- 172.21.15.161:0/3664227220 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc09a240 con 0x55a581d382d0 2023-07-14T11:45:43.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb6eb7fe700 1 -- 172.21.15.161:0/3664227220 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a2200 con 0x55a581d382d0 2023-07-14T11:45:43.326 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3664227220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 msgr2=0x55a581d386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d386a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc096a50 tx=0x7fb6fc099fe0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3664227220 shutdown_connections 2023-07-14T11:45:43.327 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d43ad0 0x55a581d45eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d386a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.328 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3664227220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d43590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3664227220 >> 172.21.15.161:0/3664227220 conn(0x55a581d2eda0 msgr2=0x55a581d346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:43.329 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3664227220 shutdown_connections 2023-07-14T11:45:43.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.317+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3664227220 wait complete. 2023-07-14T11:45:43.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.318+0000 7fb71aca0b80 1 -- start start 2023-07-14T11:45:43.330 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.318+0000 7fb71aca0b80 1 Processor -- start 2023-07-14T11:45:43.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d382d0 0x55a581d40830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d40d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581d412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581c2aac0 con 0x55a581d382d0 2023-07-14T11:45:43.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581c2a7c0 con 0x55a581d38be0 2023-07-14T11:45:43.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581b0e920 con 0x55a581d43ad0 2023-07-14T11:45:43.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb71aca0b80 10 client.0 _ll_register_callbacks cb 0x55a581c28760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:45:43.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d40d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d40d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:42614/0 (socket says 172.21.15.161:42614) 2023-07-14T11:45:43.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 -- 172.21.15.161:0/3404698721 learned_addr learned my addr 172.21.15.161:0/3404698721 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:43.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70599c700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581d412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70519b700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d382d0 0x55a581d40830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 -- 172.21.15.161:0/3404698721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 msgr2=0x55a581d412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581d412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 -- 172.21.15.161:0/3404698721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d382d0 msgr2=0x55a581d40830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d382d0 0x55a581d40830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70499a700 1 -- 172.21.15.161:0/3404698721 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7001a4d20 con 0x55a581d38be0 2023-07-14T11:45:43.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.319+0000 7fb70599c700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581d412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:45:43.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb70499a700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d40d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0005c50 tx=0x7fb6f0012a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da86f4223a14bed server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:43.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb6e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:45:43.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/3404698721 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0017070 con 0x55a581d38be0 2023-07-14T11:45:43.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a581be44f0 con 0x55a581d38be0 2023-07-14T11:45:43.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/3404698721 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fb6f000f480 con 0x55a581d38be0 2023-07-14T11:45:43.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/3404698721 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f00122e0 con 0x55a581d38be0 2023-07-14T11:45:43.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb71aca0b80 10 client.27827 fetch_fsmap learned FSMap version 46 2023-07-14T11:45:43.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55a581aa1c60 con 0x55a581d38be0 2023-07-14T11:45:43.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.320+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/3404698721 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a581be44f0 con 0x55a581d38be0 2023-07-14T11:45:43.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/3404698721 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6f0010810 con 0x55a581d38be0 2023-07-14T11:45:43.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb71aca0b80 10 client.27827 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:45:43.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb71aca0b80 10 client.27827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:45:43.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb71aca0b80 20 client.27827 populate_metadata read hostname 'smithi161' 2023-07-14T11:45:43.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581d53530 0x55a581d55940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb71aca0b80 4 client.27827 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:45:43.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a581c83eb0 con 0x55a581d53530 2023-07-14T11:45:43.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb70519b700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581d53530 0x55a581d55940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb6e8ff9700 10 client.27827 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:43.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.321+0000 7fb70519b700 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581d53530 0x55a581d55940 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d68d504a54f8244 server_cookie=511b5799efbc7a4f in_seq=0 out_seq=0 2023-07-14T11:45:43.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.322+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/3404698721 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a581c83eb0 con 0x55a581d53530 2023-07-14T11:45:43.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.322+0000 7fb6e8ff9700 10 client.27827 handle_command_reply: tid=0 2023-07-14T11:45:43.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.322+0000 7fb71aca0b80 1 client.27827 shutdown 2023-07-14T11:45:43.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.322+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 msgr2=0x55a581d40d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.322+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d40d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0005c50 tx=0x7fb6f0012a20 comp rx=0 tx=0).stop 2023-07-14T11:45:43.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 shutdown_connections 2023-07-14T11:45:43.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581d53530 0x55a581d55940 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d382d0 0x55a581d40830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb6e8ff9700 0 client.27827 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:43.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581d412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3404698721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d40d70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 >> 172.21.15.161:0/3404698721 conn(0x55a581d2eda0 msgr2=0x55a581d34230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:43.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 shutdown_connections 2023-07-14T11:45:43.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3404698721 wait complete. 2023-07-14T11:45:43.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.323+0000 7fb71aca0b80 20 client.27827 trim_cache size 0 max 16384 2023-07-14T11:45:43.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 Processor -- start 2023-07-14T11:45:43.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 -- start start 2023-07-14T11:45:43.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d4a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d3dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d43ad0 0x55a581d3e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581d4fe40 con 0x55a581d43ad0 2023-07-14T11:45:43.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581d41b80 con 0x55a581d38be0 2023-07-14T11:45:43.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581d35ad0 con 0x55a581d382d0 2023-07-14T11:45:43.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb70519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d4a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb70519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d4a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56132/0 (socket says 172.21.15.161:56132) 2023-07-14T11:45:43.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb70499a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d3dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb70519b700 1 -- 172.21.15.161:0/3322182756 learned_addr learned my addr 172.21.15.161:0/3322182756 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:43.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.340+0000 7fb70599c700 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d43ad0 0x55a581d3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.354 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb70519b700 1 -- 172.21.15.161:0/3322182756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 msgr2=0x55a581d3dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb70519b700 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d3dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb70519b700 1 -- 172.21.15.161:0/3322182756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d43ad0 msgr2=0x55a581d3e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb70519b700 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d43ad0 0x55a581d3e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb70519b700 1 -- 172.21.15.161:0/3322182756 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a581d305e0 con 0x55a581d382d0 2023-07-14T11:45:43.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb70599c700 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d43ad0 0x55a581d3e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:45:43.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb70519b700 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d4a4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0cd280 tx=0x7fb6fc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddde9d0c8cba80d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:43.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb6ebfff700 1 -- 172.21.15.161:0/3322182756 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a24b0 con 0x55a581d382d0 2023-07-14T11:45:43.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb6ebfff700 1 -- 172.21.15.161:0/3322182756 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc09a3b0 con 0x55a581d382d0 2023-07-14T11:45:43.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.341+0000 7fb6ebfff700 1 -- 172.21.15.161:0/3322182756 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a1fa0 con 0x55a581d382d0 2023-07-14T11:45:43.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3322182756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 msgr2=0x55a581d4a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d4a4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0cd280 tx=0x7fb6fc00eef0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3322182756 shutdown_connections 2023-07-14T11:45:43.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d43ad0 0x55a581d3e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d382d0 0x55a581d4a4e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/3322182756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d38be0 0x55a581d3dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3322182756 >> 172.21.15.161:0/3322182756 conn(0x55a581d2eda0 msgr2=0x55a581d3bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:43.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3322182756 shutdown_connections 2023-07-14T11:45:43.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 -- 172.21.15.161:0/3322182756 wait complete. 2023-07-14T11:45:43.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 -- start start 2023-07-14T11:45:43.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.342+0000 7fb71aca0b80 1 Processor -- start 2023-07-14T11:45:43.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d382d0 0x55a581d484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb71aca0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581da63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581d491a0 con 0x55a581d38be0 2023-07-14T11:45:43.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581d49310 con 0x55a581d382d0 2023-07-14T11:45:43.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb71aca0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581da68f0 con 0x55a581d43ad0 2023-07-14T11:45:43.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb71aca0b80 10 client.0 _ll_register_callbacks cb 0x7fb6f0009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:45:43.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d382d0 0x55a581d484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d382d0 0x55a581d484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:42636/0 (socket says 172.21.15.161:42636) 2023-07-14T11:45:43.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 -- 172.21.15.161:0/439228112 learned_addr learned my addr 172.21.15.161:0/439228112 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:45:43.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70499a700 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70599c700 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581da63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 -- 172.21.15.161:0/439228112 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 msgr2=0x55a581da63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581da63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 -- 172.21.15.161:0/439228112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 msgr2=0x55a581d48a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:45:43.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d48a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.343+0000 7fb70519b700 1 -- 172.21.15.161:0/439228112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a581ab8de0 con 0x55a581d382d0 2023-07-14T11:45:43.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb70519b700 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d382d0 0x55a581d484d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0964e0 tx=0x7fb6fc099fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beac9e0ff2cab166 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:45:43.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb6e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:45:43.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/439228112 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0996d0 con 0x55a581d382d0 2023-07-14T11:45:43.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/439228112 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fb6fc0a1fa0 con 0x55a581d382d0 2023-07-14T11:45:43.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6fc09a240 con 0x55a581d382d0 2023-07-14T11:45:43.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/439228112 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6fc0b83d0 con 0x55a581d382d0 2023-07-14T11:45:43.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 10 client.27833 fetch_fsmap learned FSMap version 46 2023-07-14T11:45:43.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55a581adf250 con 0x55a581d382d0 2023-07-14T11:45:43.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/439228112 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc09a240 con 0x55a581d382d0 2023-07-14T11:45:43.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/439228112 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6fc098f40 con 0x55a581d382d0 2023-07-14T11:45:43.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 10 client.27833 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:45:43.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 10 client.27833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:45:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 20 client.27833 populate_metadata read hostname 'smithi161' 2023-07-14T11:45:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581da7b50 0x55a581da9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:45:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 4 client.27833 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:45:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.344+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a581c46050 con 0x55a581da7b50 2023-07-14T11:45:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb70499a700 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581da7b50 0x55a581da9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:45:43.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb6e8ff9700 10 client.27833 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:43.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb70499a700 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581da7b50 0x55a581da9f30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71dfd1acf2ffe00f server_cookie=7cd6f78b16d1b920 in_seq=0 out_seq=0 2023-07-14T11:45:43.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb6e8ff9700 1 -- 172.21.15.161:0/439228112 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a581c46050 con 0x55a581da7b50 2023-07-14T11:45:43.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb6e8ff9700 10 client.27833 handle_command_reply: tid=0 2023-07-14T11:45:43.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 client.27833 shutdown 2023-07-14T11:45:43.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d382d0 msgr2=0x55a581d484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d382d0 0x55a581d484d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0964e0 tx=0x7fb6fc099fe0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 shutdown_connections 2023-07-14T11:45:43.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55a581da7b50 0x55a581da9f30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55a581d43ad0 0x55a581da63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a581d382d0 0x55a581d484d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb6e8ff9700 0 client.27833 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:45:43.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 --2- 172.21.15.161:0/439228112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a581d38be0 0x55a581d48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.345+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 >> 172.21.15.161:0/439228112 conn(0x55a581d2eda0 msgr2=0x55a581d40780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:43.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 shutdown_connections 2023-07-14T11:45:43.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb71aca0b80 1 -- 172.21.15.161:0/439228112 wait complete. 2023-07-14T11:45:43.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb71aca0b80 20 client.27833 trim_cache size 0 max 16384 2023-07-14T11:45:43.371 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:45:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:45:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:45:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:45:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:45:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:45:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:45:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:45:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:45:43.374 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:45:43.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb6ec0a12c0 msgr2=0x7fb6ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb6ec0a12c0 0x7fb6ec0a3770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009610 tx=0x7fb6f800bd70 comp rx=0 tx=0).stop 2023-07-14T11:45:43.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 msgr2=0x7fb700197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:45:43.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 0x7fb700197ef0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e9a0 tx=0x7fb6fc00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 shutdown_connections 2023-07-14T11:45:43.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb6ec0a12c0 0x7fb6ec0a3770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700105b10 0x7fb70019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb7000ffb00 0x7fb700197ef0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 --2- 172.21.15.161:0/1765907545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7001004b0 0x7fb700198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:45:43.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 >> 172.21.15.161:0/1765907545 conn(0x7fb7000fb600 msgr2=0x7fb70010b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:45:43.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.346+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 shutdown_connections 2023-07-14T11:45:43.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:45:43.347+0000 7fb7073ff700 1 -- 172.21.15.161:0/1765907545 wait complete. 2023-07-14T11:45:43.378 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:45:44.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:44 smithi182 ceph-mon[129366]: pgmap v741: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:44.481 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:44 smithi161 ceph-mon[95526]: pgmap v741: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:44.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:44 smithi175 ceph-mon[129081]: pgmap v741: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:46.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:46 smithi175 ceph-mon[129081]: pgmap v742: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:46.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:46 smithi182 ceph-mon[129366]: pgmap v742: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:46.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:46 smithi161 ceph-mon[95526]: pgmap v742: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:48.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:48 smithi175 ceph-mon[129081]: pgmap v743: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:48.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:48 smithi182 ceph-mon[129366]: pgmap v743: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:48.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:48 smithi161 ceph-mon[95526]: pgmap v743: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:50.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:50 smithi175 ceph-mon[129081]: pgmap v744: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:50.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:50 smithi182 ceph-mon[129366]: pgmap v744: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:50.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:50 smithi161 ceph-mon[95526]: pgmap v744: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:52.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:52 smithi175 ceph-mon[129081]: pgmap v745: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:52.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:52 smithi182 ceph-mon[129366]: pgmap v745: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:52.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:52 smithi161 ceph-mon[95526]: pgmap v745: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:45:54.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:54 smithi182 ceph-mon[129366]: pgmap v746: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:54.481 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:54 smithi161 ceph-mon[95526]: pgmap v746: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:54.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:54 smithi175 ceph-mon[129081]: pgmap v746: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2023-07-14T11:45:56.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:56 smithi175 ceph-mon[129081]: pgmap v747: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:56.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:56 smithi182 ceph-mon[129366]: pgmap v747: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:56.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:56 smithi161 ceph-mon[95526]: pgmap v747: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:45:58.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:45:58 smithi175 ceph-mon[129081]: pgmap v748: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:58.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:45:58 smithi182 ceph-mon[129366]: pgmap v748: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:45:58.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:45:58 smithi161 ceph-mon[95526]: pgmap v748: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:00.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:00 smithi175 ceph-mon[129081]: pgmap v749: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:00.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:00 smithi182 ceph-mon[129366]: pgmap v749: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:00.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:00 smithi161 ceph-mon[95526]: pgmap v749: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:02.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:02 smithi175 ceph-mon[129081]: pgmap v750: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-14T11:46:02.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:02 smithi182 ceph-mon[129366]: pgmap v750: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-14T11:46:02.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:02 smithi161 ceph-mon[95526]: pgmap v750: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-14T11:46:04.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:04 smithi182 ceph-mon[129366]: pgmap v751: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:46:04.481 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:04 smithi161 ceph-mon[95526]: pgmap v751: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:46:04.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:04 smithi175 ceph-mon[129081]: pgmap v751: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:46:06.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:06 smithi175 ceph-mon[129081]: pgmap v752: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:46:06.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:06 smithi182 ceph-mon[129366]: pgmap v752: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:46:06.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:06 smithi161 ceph-mon[95526]: pgmap v752: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:46:08.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:08 smithi175 ceph-mon[129081]: pgmap v753: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:46:08.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:08 smithi182 ceph-mon[129366]: pgmap v753: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:46:08.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:08 smithi161 ceph-mon[95526]: pgmap v753: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:46:10.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:10 smithi175 ceph-mon[129081]: pgmap v754: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:46:10.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:10 smithi182 ceph-mon[129366]: pgmap v754: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:46:10.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:10 smithi161 ceph-mon[95526]: pgmap v754: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:46:12.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:12 smithi175 ceph-mon[129081]: pgmap v755: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:46:12.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:12 smithi175 ceph-mon[129081]: osdmap e128: 12 total, 12 up, 12 in 2023-07-14T11:46:12.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:12 smithi182 ceph-mon[129366]: pgmap v755: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:46:12.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:12 smithi182 ceph-mon[129366]: osdmap e128: 12 total, 12 up, 12 in 2023-07-14T11:46:12.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:12 smithi161 ceph-mon[95526]: pgmap v755: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-14T11:46:12.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:12 smithi161 ceph-mon[95526]: osdmap e128: 12 total, 12 up, 12 in 2023-07-14T11:46:13.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:46:13.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 -- 172.21.15.161:0/3584599062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e80ff9d0 msgr2=0x7f97e80ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 --2- 172.21.15.161:0/3584599062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e80ff9d0 0x7f97e80ffe40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009990 tx=0x7f97dc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 -- 172.21.15.161:0/3584599062 shutdown_connections 2023-07-14T11:46:13.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 --2- 172.21.15.161:0/3584599062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97e81059e0 0x7f97e8105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 --2- 172.21.15.161:0/3584599062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 0x7f97e810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 --2- 172.21.15.161:0/3584599062 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e80ff9d0 0x7f97e80ffe40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 -- 172.21.15.161:0/3584599062 >> 172.21.15.161:0/3584599062 conn(0x7f97e80fb4b0 msgr2=0x7f97e80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:13.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 -- 172.21.15.161:0/3584599062 shutdown_connections 2023-07-14T11:46:13.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.615+0000 7f97ec99d700 1 -- 172.21.15.161:0/3584599062 wait complete. 2023-07-14T11:46:13.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.616+0000 7f97ec99d700 1 Processor -- start 2023-07-14T11:46:13.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97ec99d700 1 -- start start 2023-07-14T11:46:13.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97ec99d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97e80ff9d0 0x7f97e8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97ec99d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 0x7f97e8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97ec99d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e81059e0 0x7f97e81a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97ec99d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8073580 con 0x7f97e80ff9d0 2023-07-14T11:46:13.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97ec99d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81a3fe0 con 0x7f97e8100380 2023-07-14T11:46:13.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97ec99d700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81a4120 con 0x7f97e81059e0 2023-07-14T11:46:13.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 0x7f97e8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97e80ff9d0 0x7f97e8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 0x7f97e8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52804/0 (socket says 172.21.15.161:52804) 2023-07-14T11:46:13.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97e6ffd700 1 -- 172.21.15.161:0/199369672 learned_addr learned my addr 172.21.15.161:0/199369672 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:13.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.617+0000 7f97e7fff700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e81059e0 0x7f97e81a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e6ffd700 1 -- 172.21.15.161:0/199369672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e81059e0 msgr2=0x7f97e81a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.610 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e81059e0 0x7f97e81a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e6ffd700 1 -- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97e80ff9d0 msgr2=0x7f97e8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97e80ff9d0 0x7f97e8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e6ffd700 1 -- 172.21.15.161:0/199369672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc0095f0 con 0x7f97e8100380 2023-07-14T11:46:13.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e77fe700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97e80ff9d0 0x7f97e8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:46:13.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e7fff700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e81059e0 0x7f97e81a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:46:13.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 0x7f97e8072d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005910 tx=0x7f97dc005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:13.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e4ff9700 1 -- 172.21.15.161:0/199369672 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc01d070 con 0x7f97e8100380 2023-07-14T11:46:13.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e4ff9700 1 -- 172.21.15.161:0/199369672 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc022470 con 0x7f97e8100380 2023-07-14T11:46:13.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e81a4320 con 0x7f97e8100380 2023-07-14T11:46:13.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.618+0000 7f97e4ff9700 1 -- 172.21.15.161:0/199369672 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc017650 con 0x7f97e8100380 2023-07-14T11:46:13.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.619+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e81a47a0 con 0x7f97e8100380 2023-07-14T11:46:13.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.623+0000 7f9801554b80 1 Processor -- start 2023-07-14T11:46:13.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.623+0000 7f97e4ff9700 1 -- 172.21.15.161:0/199369672 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f97dc0225e0 con 0x7f97e8100380 2023-07-14T11:46:13.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.623+0000 7f9801554b80 1 -- start start 2023-07-14T11:46:13.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f9801554b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f9801554b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f9801554b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9c5d30 0x557cbd9c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f9801554b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd8c0cb0 con 0x557cbd9bb4f0 2023-07-14T11:46:13.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f9801554b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd913a40 con 0x557cbd9bbe00 2023-07-14T11:46:13.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f9801554b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd90ced0 con 0x557cbd9c5d30 2023-07-14T11:46:13.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9c5d30 0x557cbd9c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f97e7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9c5d30 0x557cbd9c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:48964/0 (socket says 172.21.15.161:48964) 2023-07-14T11:46:13.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f97e6ffd700 1 -- 172.21.15.161:0/4029058460 learned_addr learned my addr 172.21.15.161:0/4029058460 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:13.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.624+0000 7f97e77fe700 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97e6ffd700 1 -- 172.21.15.161:0/4029058460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 msgr2=0x557cbd9c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97e6ffd700 1 -- 172.21.15.161:0/4029058460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 msgr2=0x557cbd9bb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9bb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97e6ffd700 1 -- 172.21.15.161:0/4029058460 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cbd73ade0 con 0x557cbd9c5d30 2023-07-14T11:46:13.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9c5d30 0x557cbd9c8110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0cb640 tx=0x7f97dc0d6670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74a9efb0aef1e58 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:13.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97e4ff9700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f97d00a13e0 0x7f97d00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97cdffb700 1 -- 172.21.15.161:0/4029058460 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0d02d0 con 0x557cbd9c5d30 2023-07-14T11:46:13.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97cdffb700 1 -- 172.21.15.161:0/4029058460 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc097420 con 0x557cbd9c5d30 2023-07-14T11:46:13.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.625+0000 7f97cdffb700 1 -- 172.21.15.161:0/4029058460 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0a92c0 con 0x557cbd9c5d30 2023-07-14T11:46:13.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.626+0000 7f97e4ff9700 1 -- 172.21.15.161:0/199369672 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f97dc0ce360 con 0x7f97e8100380 2023-07-14T11:46:13.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.626+0000 7f97e7fff700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f97d00a13e0 0x7f97d00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.626+0000 7f9801554b80 1 -- 172.21.15.161:0/4029058460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9c5d30 msgr2=0x557cbd9c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.626+0000 7f9801554b80 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9c5d30 0x557cbd9c8110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0cb640 tx=0x7f97dc0d6670 comp rx=0 tx=0).stop 2023-07-14T11:46:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.626+0000 7f97e7fff700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f97d00a13e0 0x7f97d00a3890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f97e8074920 tx=0x7f97e000b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 -- 172.21.15.161:0/4029058460 shutdown_connections 2023-07-14T11:46:13.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9bb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9c5d30 0x557cbd9c8110 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 --2- 172.21.15.161:0/4029058460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 -- 172.21.15.161:0/4029058460 >> 172.21.15.161:0/4029058460 conn(0x557cbd9b0ff0 msgr2=0x557cbd9b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:13.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 -- 172.21.15.161:0/4029058460 shutdown_connections 2023-07-14T11:46:13.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 -- 172.21.15.161:0/4029058460 wait complete. 2023-07-14T11:46:13.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.627+0000 7f9801554b80 1 -- start start 2023-07-14T11:46:13.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.628+0000 7f9801554b80 1 Processor -- start 2023-07-14T11:46:13.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.628+0000 7f9801554b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.628+0000 7f9801554b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.628+0000 7f9801554b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9c5d30 0x557cbd9c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.628+0000 7f9801554b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd8acac0 con 0x557cbd9c5d30 2023-07-14T11:46:13.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f9801554b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd8ac7c0 con 0x557cbd9bbe00 2023-07-14T11:46:13.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f9801554b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd790920 con 0x557cbd9bb4f0 2023-07-14T11:46:13.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f9801554b80 10 client.0 _ll_register_callbacks cb 0x557cbd8aa760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:46:13.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52836/0 (socket says 172.21.15.161:52836) 2023-07-14T11:46:13.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e6ffd700 1 -- 172.21.15.161:0/1645148180 learned_addr learned my addr 172.21.15.161:0/1645148180 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:13.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e7fff700 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9c5d30 0x557cbd9c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e77fe700 1 -- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 msgr2=0x557cbd9c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e77fe700 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e77fe700 1 -- 172.21.15.161:0/1645148180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9c5d30 msgr2=0x557cbd9c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e77fe700 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9c5d30 0x557cbd9c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e77fe700 1 -- 172.21.15.161:0/1645148180 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e81a47a0 con 0x557cbd9bb4f0 2023-07-14T11:46:13.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:46:13.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97e77fe700 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c6100 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f97d8012cd0 tx=0x7f97d8009d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52739331f9bc7da0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:13.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.629+0000 7f97c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:46:13.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.630+0000 7f97c6ffd700 1 -- 172.21.15.161:0/1645148180 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8017070 con 0x557cbd9bb4f0 2023-07-14T11:46:13.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.630+0000 7f97c6ffd700 1 -- 172.21.15.161:0/1645148180 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f97d8004020 con 0x557cbd9bb4f0 2023-07-14T11:46:13.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.630+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cbd8664f0 con 0x557cbd9bb4f0 2023-07-14T11:46:13.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.630+0000 7f97c6ffd700 1 -- 172.21.15.161:0/1645148180 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97d80041e0 con 0x557cbd9bb4f0 2023-07-14T11:46:13.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.630+0000 7f9801554b80 10 client.27889 fetch_fsmap learned FSMap version 46 2023-07-14T11:46:13.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.630+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x557cbd723c60 con 0x557cbd9bb4f0 2023-07-14T11:46:13.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f97c6ffd700 1 -- 172.21.15.161:0/1645148180 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557cbd8664f0 con 0x557cbd9bb4f0 2023-07-14T11:46:13.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f97c6ffd700 1 -- 172.21.15.161:0/1645148180 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f97d8003710 con 0x557cbd9bb4f0 2023-07-14T11:46:13.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f9801554b80 10 client.27889 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:46:13.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f9801554b80 10 client.27889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:46:13.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f9801554b80 20 client.27889 populate_metadata read hostname 'smithi161' 2023-07-14T11:46:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f9801554b80 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbd9d52d0 0x557cbd9d76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f9801554b80 4 client.27889 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:46:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cbd905eb0 con 0x557cbd9d52d0 2023-07-14T11:46:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.631+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbd9d52d0 0x557cbd9d76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.632+0000 7f97c6ffd700 10 client.27889 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:13.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.632+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbd9d52d0 0x557cbd9d76e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404ca56dced022b4 server_cookie=95619cef21880edb in_seq=0 out_seq=0 2023-07-14T11:46:13.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.632+0000 7f97c6ffd700 1 -- 172.21.15.161:0/1645148180 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557cbd905eb0 con 0x557cbd9d52d0 2023-07-14T11:46:13.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.632+0000 7f97c6ffd700 10 client.27889 handle_command_reply: tid=0 2023-07-14T11:46:13.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 client.27889 shutdown 2023-07-14T11:46:13.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 msgr2=0x557cbd9c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c6100 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f97d8012cd0 tx=0x7f97d8009d40 comp rx=0 tx=0).stop 2023-07-14T11:46:13.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 shutdown_connections 2023-07-14T11:46:13.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbd9d52d0 0x557cbd9d76e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9c5d30 0x557cbd9c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c6100 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f97c6ffd700 0 client.27889 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:13.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 --2- 172.21.15.161:0/1645148180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9bbe00 0x557cbd9c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 >> 172.21.15.161:0/1645148180 conn(0x557cbd9b0ff0 msgr2=0x557cbd9b64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:13.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 shutdown_connections 2023-07-14T11:46:13.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 1 -- 172.21.15.161:0/1645148180 wait complete. 2023-07-14T11:46:13.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.633+0000 7f9801554b80 20 client.27889 trim_cache size 0 max 16384 2023-07-14T11:46:13.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 Processor -- start 2023-07-14T11:46:13.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 -- start start 2023-07-14T11:46:13.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bbe00 0x557cbd9c4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd9d2a10 con 0x557cbd9bb4f0 2023-07-14T11:46:13.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd9c5360 con 0x557cbd9c5d30 2023-07-14T11:46:13.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.651+0000 7f9801554b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd9c3300 con 0x557cbd9bbe00 2023-07-14T11:46:13.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bbe00 0x557cbd9c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60318/0 (socket says 172.21.15.161:60318) 2023-07-14T11:46:13.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e77fe700 1 -- 172.21.15.161:0/683405025 learned_addr learned my addr 172.21.15.161:0/683405025 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:13.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e7fff700 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e6ffd700 1 -- 172.21.15.161:0/683405025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 msgr2=0x557cbd9b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e6ffd700 1 -- 172.21.15.161:0/683405025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 msgr2=0x557cbd9cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e6ffd700 1 -- 172.21.15.161:0/683405025 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cbd761250 con 0x557cbd9bbe00 2023-07-14T11:46:13.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bbe00 0x557cbd9c4560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f97dc097590 tx=0x7f97dc022f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17367c639790408 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:13.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97ce7fc700 1 -- 172.21.15.161:0/683405025 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0d1980 con 0x557cbd9bbe00 2023-07-14T11:46:13.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.652+0000 7f97ce7fc700 1 -- 172.21.15.161:0/683405025 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc0acd00 con 0x557cbd9bbe00 2023-07-14T11:46:13.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.653+0000 7f97ce7fc700 1 -- 172.21.15.161:0/683405025 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc09aeb0 con 0x557cbd9bbe00 2023-07-14T11:46:13.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.653+0000 7f9801554b80 1 -- 172.21.15.161:0/683405025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bbe00 msgr2=0x557cbd9c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.653+0000 7f9801554b80 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bbe00 0x557cbd9c4560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f97dc097590 tx=0x7f97dc022f90 comp rx=0 tx=0).stop 2023-07-14T11:46:13.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.653+0000 7f9801554b80 1 -- 172.21.15.161:0/683405025 shutdown_connections 2023-07-14T11:46:13.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bb4f0 0x557cbd9cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 --2- 172.21.15.161:0/683405025 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bbe00 0x557cbd9c4560 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 -- 172.21.15.161:0/683405025 >> 172.21.15.161:0/683405025 conn(0x557cbd9b0ff0 msgr2=0x557cbd9c0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:13.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 -- 172.21.15.161:0/683405025 shutdown_connections 2023-07-14T11:46:13.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 -- 172.21.15.161:0/683405025 wait complete. 2023-07-14T11:46:13.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 -- start start 2023-07-14T11:46:13.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.654+0000 7f9801554b80 1 Processor -- start 2023-07-14T11:46:13.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f9801554b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f9801554b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bbe00 0x557cbd9c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f9801554b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f9801554b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd9cae50 con 0x557cbd9bbe00 2023-07-14T11:46:13.657 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f9801554b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd9cafc0 con 0x557cbd9c5d30 2023-07-14T11:46:13.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f9801554b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbd9cb180 con 0x557cbd9bb4f0 2023-07-14T11:46:13.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f9801554b80 10 client.0 _ll_register_callbacks cb 0x7f97d80127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:46:13.658 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49006/0 (socket says 172.21.15.161:49006) 2023-07-14T11:46:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 -- 172.21.15.161:0/2897437378 learned_addr learned my addr 172.21.15.161:0/2897437378 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e7fff700 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.659 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bbe00 0x557cbd9c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 -- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 msgr2=0x557cbd9ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 -- 172.21.15.161:0/2897437378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bbe00 msgr2=0x557cbd9c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bbe00 0x557cbd9c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.655+0000 7f97e77fe700 1 -- 172.21.15.161:0/2897437378 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cbd73ade0 con 0x557cbd9bb4f0 2023-07-14T11:46:13.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f97e77fe700 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c9340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f97d8012420 tx=0x7f97d800fa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1d135df553107d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:13.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f97c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:46:13.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f97c6ffd700 1 -- 172.21.15.161:0/2897437378 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8017040 con 0x557cbd9bb4f0 2023-07-14T11:46:13.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97dc097420 con 0x557cbd9bb4f0 2023-07-14T11:46:13.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f97c6ffd700 1 -- 172.21.15.161:0/2897437378 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f97d8000e40 con 0x557cbd9bb4f0 2023-07-14T11:46:13.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f97c6ffd700 1 -- 172.21.15.161:0/2897437378 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97d800fe60 con 0x557cbd9bb4f0 2023-07-14T11:46:13.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f9801554b80 10 client.27895 fetch_fsmap learned FSMap version 46 2023-07-14T11:46:13.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x557cbd7392b0 con 0x557cbd9bb4f0 2023-07-14T11:46:13.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f97c6ffd700 1 -- 172.21.15.161:0/2897437378 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc097420 con 0x557cbd9bb4f0 2023-07-14T11:46:13.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.656+0000 7f97c6ffd700 1 -- 172.21.15.161:0/2897437378 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f97d801c500 con 0x557cbd9bb4f0 2023-07-14T11:46:13.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f9801554b80 10 client.27895 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:46:13.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f9801554b80 10 client.27895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:46:13.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f9801554b80 20 client.27895 populate_metadata read hostname 'smithi161' 2023-07-14T11:46:13.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f9801554b80 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbda2a3d0 0x557cbda2c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:13.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f9801554b80 4 client.27895 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:46:13.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557cbd8c8050 con 0x557cbda2a3d0 2023-07-14T11:46:13.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbda2a3d0 0x557cbda2c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:13.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f97e6ffd700 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbda2a3d0 0x557cbda2c7e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a0fddf44741c76 server_cookie=c76a088567062421 in_seq=0 out_seq=0 2023-07-14T11:46:13.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f97c6ffd700 10 client.27895 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:13.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f97c6ffd700 1 -- 172.21.15.161:0/2897437378 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557cbd8c8050 con 0x557cbda2a3d0 2023-07-14T11:46:13.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f97c6ffd700 10 client.27895 handle_command_reply: tid=0 2023-07-14T11:46:13.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.657+0000 7f9801554b80 1 client.27895 shutdown 2023-07-14T11:46:13.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 msgr2=0x557cbd9c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c9340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f97d8012420 tx=0x7f97d800fa60 comp rx=0 tx=0).stop 2023-07-14T11:46:13.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 shutdown_connections 2023-07-14T11:46:13.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x557cbda2a3d0 0x557cbda2c7e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cbd9c5d30 0x557cbd9ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x557cbd9bb4f0 0x557cbd9c9340 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f97c6ffd700 0 client.27895 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:13.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 --2- 172.21.15.161:0/2897437378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cbd9bbe00 0x557cbd9c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 >> 172.21.15.161:0/2897437378 conn(0x557cbd9b0ff0 msgr2=0x557cbd9bacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:13.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 shutdown_connections 2023-07-14T11:46:13.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 1 -- 172.21.15.161:0/2897437378 wait complete. 2023-07-14T11:46:13.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f9801554b80 20 client.27895 trim_cache size 0 max 16384 2023-07-14T11:46:13.670 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:46:13.671 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:46:13.671 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:46:13.671 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:46:13.671 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:46:13.672 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:46:13.672 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:46:13.672 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:46:13.672 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:46:13.672 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:46:13.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f97d00a13e0 msgr2=0x7f97d00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f97ec99d700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f97d00a13e0 0x7f97d00a3890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f97e8074920 tx=0x7f97e000b040 comp rx=0 tx=0).stop 2023-07-14T11:46:13.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 msgr2=0x7f97e8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:13.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.658+0000 7f97ec99d700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 0x7f97e8072d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005910 tx=0x7f97dc005720 comp rx=0 tx=0).stop 2023-07-14T11:46:13.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 shutdown_connections 2023-07-14T11:46:13.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f97d00a13e0 0x7f97d00a3890 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97e80ff9d0 0x7f97e8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f97e81059e0 0x7f97e81a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 --2- 172.21.15.161:0/199369672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e8100380 0x7f97e8072d50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:13.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 >> 172.21.15.161:0/199369672 conn(0x7f97e80fb4b0 msgr2=0x7f97e8103520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:13.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 shutdown_connections 2023-07-14T11:46:13.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:13.659+0000 7f97ec99d700 1 -- 172.21.15.161:0/199369672 wait complete. 2023-07-14T11:46:13.677 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:46:13.730 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:13 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:46:13.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:13 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:46:13.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:13 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:46:14.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:14 smithi175 ceph-mon[129081]: pgmap v757: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:14.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:14 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:46:14.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:14 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:46:14.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:14 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:46:14.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:14 smithi182 ceph-mon[129366]: pgmap v757: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:14.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:14 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:46:14.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:14 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:46:14.950 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:14 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:46:14.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:14 smithi161 ceph-mon[95526]: pgmap v757: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:14.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:14 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:46:14.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:14 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:46:14.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:14 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:46:16.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:16 smithi175 ceph-mon[129081]: pgmap v758: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:16.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:16 smithi182 ceph-mon[129366]: pgmap v758: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:16.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:16 smithi161 ceph-mon[95526]: pgmap v758: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:18.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:18 smithi175 ceph-mon[129081]: pgmap v759: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:18.905 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:18 smithi175 ceph-mon[129081]: osdmap e129: 12 total, 12 up, 12 in 2023-07-14T11:46:18.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:18 smithi182 ceph-mon[129366]: pgmap v759: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:18.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:18 smithi182 ceph-mon[129366]: osdmap e129: 12 total, 12 up, 12 in 2023-07-14T11:46:18.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:18 smithi161 ceph-mon[95526]: pgmap v759: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:46:18.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:18 smithi161 ceph-mon[95526]: osdmap e129: 12 total, 12 up, 12 in 2023-07-14T11:46:20.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:20 smithi175 ceph-mon[129081]: pgmap v761: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2023-07-14T11:46:20.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:20 smithi182 ceph-mon[129366]: pgmap v761: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2023-07-14T11:46:20.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:20 smithi161 ceph-mon[95526]: pgmap v761: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2023-07-14T11:46:22.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:22 smithi175 ceph-mon[129081]: pgmap v762: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:22.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:22 smithi182 ceph-mon[129366]: pgmap v762: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:22.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:22 smithi161 ceph-mon[95526]: pgmap v762: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:24.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:24 smithi182 ceph-mon[129366]: pgmap v763: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:24.481 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:24 smithi161 ceph-mon[95526]: pgmap v763: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:24.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:24 smithi175 ceph-mon[129081]: pgmap v763: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:26.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:26 smithi175 ceph-mon[129081]: pgmap v764: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:26.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:26 smithi182 ceph-mon[129366]: pgmap v764: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:26.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:26 smithi161 ceph-mon[95526]: pgmap v764: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:46:28.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:28 smithi175 ceph-mon[129081]: pgmap v765: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:46:28.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:28 smithi182 ceph-mon[129366]: pgmap v765: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:46:28.984 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:28 smithi161 ceph-mon[95526]: pgmap v765: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:46:30.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:30 smithi175 ceph-mon[129081]: pgmap v766: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:46:30.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:30 smithi182 ceph-mon[129366]: pgmap v766: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:46:30.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:30 smithi161 ceph-mon[95526]: pgmap v766: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:46:32.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:32 smithi175 ceph-mon[129081]: pgmap v767: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:32.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:32 smithi182 ceph-mon[129366]: pgmap v767: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:32.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:32 smithi161 ceph-mon[95526]: pgmap v767: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:34.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:34 smithi182 ceph-mon[129366]: pgmap v768: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:34.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:34 smithi161 ceph-mon[95526]: pgmap v768: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:34.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:34 smithi175 ceph-mon[129081]: pgmap v768: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:36.904 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:36 smithi175 ceph-mon[129081]: pgmap v769: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:36.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:36 smithi182 ceph-mon[129366]: pgmap v769: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:36.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:36 smithi161 ceph-mon[95526]: pgmap v769: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:38.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:38 smithi175 ceph-mon[129081]: pgmap v770: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:38.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:38 smithi182 ceph-mon[129366]: pgmap v770: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:38.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:38 smithi161 ceph-mon[95526]: pgmap v770: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:40.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:40 smithi175 ceph-mon[129081]: pgmap v771: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:40.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:40 smithi182 ceph-mon[129366]: pgmap v771: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:40.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:40 smithi161 ceph-mon[95526]: pgmap v771: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:42.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:42 smithi175 ceph-mon[129081]: pgmap v772: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:42.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:42 smithi182 ceph-mon[129366]: pgmap v772: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:42.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:42 smithi161 ceph-mon[95526]: pgmap v772: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:43.678 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:46:43.891 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 -- 172.21.15.161:0/1307198104 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 msgr2=0x7f50bc105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 --2- 172.21.15.161:0/1307198104 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 0x7f50bc105eb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009990 tx=0x7f50b8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 -- 172.21.15.161:0/1307198104 shutdown_connections 2023-07-14T11:46:43.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 --2- 172.21.15.161:0/1307198104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 0x7f50bc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.892 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 --2- 172.21.15.161:0/1307198104 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 0x7f50bc105eb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 --2- 172.21.15.161:0/1307198104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc0ffad0 0x7f50bc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 -- 172.21.15.161:0/1307198104 >> 172.21.15.161:0/1307198104 conn(0x7f50bc0fb5f0 msgr2=0x7f50bc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:43.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 -- 172.21.15.161:0/1307198104 shutdown_connections 2023-07-14T11:46:43.893 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.903+0000 7f50c37ff700 1 -- 172.21.15.161:0/1307198104 wait complete. 2023-07-14T11:46:43.895 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 Processor -- start 2023-07-14T11:46:43.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 -- start start 2023-07-14T11:46:43.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc0ffad0 0x7f50bc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.896 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 0x7f50bc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 0x7f50bc1a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc073580 con 0x7f50bc0ffad0 2023-07-14T11:46:43.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1a4310 con 0x7f50bc100480 2023-07-14T11:46:43.898 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c37ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1a4450 con 0x7f50bc105ae0 2023-07-14T11:46:43.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc0ffad0 0x7f50bc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 0x7f50bc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c159b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc0ffad0 0x7f50bc072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59106/0 (socket says 172.21.15.161:59106) 2023-07-14T11:46:43.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 0x7f50bc072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47592/0 (socket says 172.21.15.161:47592) 2023-07-14T11:46:43.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c159b700 1 -- 172.21.15.161:0/550656262 learned_addr learned my addr 172.21.15.161:0/550656262 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:43.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.905+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 0x7f50bc1a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.901 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c0d9a700 1 -- 172.21.15.161:0/550656262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 msgr2=0x7f50bc1a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 0x7f50bc1a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c0d9a700 1 -- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc0ffad0 msgr2=0x7f50bc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc0ffad0 0x7f50bc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c0d9a700 1 -- 172.21.15.161:0/550656262 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b80095f0 con 0x7f50bc100480 2023-07-14T11:46:43.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 0x7f50bc072d50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00e9f0 tx=0x7f50ac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:43.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50b27fc700 1 -- 172.21.15.161:0/550656262 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00c9c0 con 0x7f50bc100480 2023-07-14T11:46:43.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc1a46b0 con 0x7f50bc100480 2023-07-14T11:46:43.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50b27fc700 1 -- 172.21.15.161:0/550656262 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50ac010430 con 0x7f50bc100480 2023-07-14T11:46:43.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50b27fc700 1 -- 172.21.15.161:0/550656262 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac018680 con 0x7f50bc100480 2023-07-14T11:46:43.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.906+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc1a4b90 con 0x7f50bc100480 2023-07-14T11:46:43.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.910+0000 7f50b27fc700 1 -- 172.21.15.161:0/550656262 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f50ac010860 con 0x7f50bc100480 2023-07-14T11:46:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 Processor -- start 2023-07-14T11:46:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 -- start start 2023-07-14T11:46:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50b27fc700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f50a80a12c0 0x7f50a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e692d0 0x5559c5e696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e74590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e76eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5d6fcb0 con 0x5559c5e692d0 2023-07-14T11:46:43.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5dc2a40 con 0x5559c5e69be0 2023-07-14T11:46:43.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5dbbed0 con 0x5559c5e74ad0 2023-07-14T11:46:43.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50b27fc700 1 -- 172.21.15.161:0/550656262 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f50ac014070 con 0x7f50bc100480 2023-07-14T11:46:43.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.911+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f50a80a12c0 0x7f50a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.913 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e74590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e74590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47606/0 (socket says 172.21.15.161:47606) 2023-07-14T11:46:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3106485220 learned_addr learned my addr 172.21.15.161:0/3106485220 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e76eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c159b700 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e692d0 0x5559c5e696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3106485220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 msgr2=0x5559c5e76eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e76eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3106485220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e692d0 msgr2=0x5559c5e696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e692d0 0x5559c5e696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3106485220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c5be9de0 con 0x5559c5e69be0 2023-07-14T11:46:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e76eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:46:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e74590 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f50ac096980 tx=0x7f50ac0998f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f393c91d08a99e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f50a80a12c0 0x7f50a80a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009ec0 tx=0x7f50b4009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50a77fe700 1 -- 172.21.15.161:0/3106485220 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0a2360 con 0x5559c5e69be0 2023-07-14T11:46:43.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50a77fe700 1 -- 172.21.15.161:0/3106485220 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50ac0105a0 con 0x5559c5e69be0 2023-07-14T11:46:43.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.912+0000 7f50a77fe700 1 -- 172.21.15.161:0/3106485220 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0a1ed0 con 0x5559c5e69be0 2023-07-14T11:46:43.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3106485220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 msgr2=0x5559c5e74590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e74590 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f50ac096980 tx=0x7f50ac0998f0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3106485220 shutdown_connections 2023-07-14T11:46:43.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e692d0 0x5559c5e696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e76eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3106485220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e74590 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3106485220 >> 172.21.15.161:0/3106485220 conn(0x5559c5e5fda0 msgr2=0x5559c5e656a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:43.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3106485220 shutdown_connections 2023-07-14T11:46:43.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.913+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3106485220 wait complete. 2023-07-14T11:46:43.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 -- start start 2023-07-14T11:46:43.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 Processor -- start 2023-07-14T11:46:43.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e71820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e71d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5d5bac0 con 0x5559c5e74ad0 2023-07-14T11:46:43.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5d5b7c0 con 0x5559c5e69be0 2023-07-14T11:46:43.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5c3f920 con 0x5559c5e692d0 2023-07-14T11:46:43.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.914+0000 7f50d70ccb80 10 client.0 _ll_register_callbacks cb 0x5559c5d59760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:46:43.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c159b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e71820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e71d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e71d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47620/0 (socket says 172.21.15.161:47620) 2023-07-14T11:46:43.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 -- 172.21.15.161:0/612876194 learned_addr learned my addr 172.21.15.161:0/612876194 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:43.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 -- 172.21.15.161:0/612876194 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 msgr2=0x5559c5e71820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e71820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 -- 172.21.15.161:0/612876194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 msgr2=0x5559c5e722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 -- 172.21.15.161:0/612876194 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc1a4b90 con 0x5559c5e69be0 2023-07-14T11:46:43.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e71d60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0cab10 tx=0x7f50ac0996b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9144be842cbe10 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:43.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:46:43.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50a4ff9700 1 -- 172.21.15.161:0/612876194 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0a2360 con 0x5559c5e69be0 2023-07-14T11:46:43.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50a4ff9700 1 -- 172.21.15.161:0/612876194 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f50ac0a20e0 con 0x5559c5e69be0 2023-07-14T11:46:43.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.915+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559c5d154f0 con 0x5559c5e69be0 2023-07-14T11:46:43.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50a4ff9700 1 -- 172.21.15.161:0/612876194 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50ac0b8180 con 0x5559c5e69be0 2023-07-14T11:46:43.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 10 client.27863 fetch_fsmap learned FSMap version 46 2023-07-14T11:46:43.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5559c5bd2c60 con 0x5559c5e69be0 2023-07-14T11:46:43.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50a4ff9700 1 -- 172.21.15.161:0/612876194 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50ac063020 con 0x5559c5e69be0 2023-07-14T11:46:43.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50a4ff9700 1 -- 172.21.15.161:0/612876194 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50ac00cb90 con 0x5559c5e69be0 2023-07-14T11:46:43.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 10 client.27863 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:46:43.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 10 client.27863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:46:43.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 20 client.27863 populate_metadata read hostname 'smithi161' 2023-07-14T11:46:43.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5e84060 0x5559c5e86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 4 client.27863 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:46:43.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559c5db4eb0 con 0x5559c5e84060 2023-07-14T11:46:43.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.916+0000 7f50c159b700 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5e84060 0x5559c5e86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.917+0000 7f50a4ff9700 10 client.27863 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:43.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.917+0000 7f50c159b700 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5e84060 0x5559c5e86470 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9662335f38af1de8 server_cookie=e84d78a94b271c45 in_seq=0 out_seq=0 2023-07-14T11:46:43.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.917+0000 7f50a4ff9700 1 -- 172.21.15.161:0/612876194 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5559c5db4eb0 con 0x5559c5e84060 2023-07-14T11:46:43.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50a4ff9700 10 client.27863 handle_command_reply: tid=0 2023-07-14T11:46:43.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 client.27863 shutdown 2023-07-14T11:46:43.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 msgr2=0x5559c5e71d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e71d60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0cab10 tx=0x7f50ac0996b0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 shutdown_connections 2023-07-14T11:46:43.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5e84060 0x5559c5e86470 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e71820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/612876194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e71d60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50a4ff9700 0 client.27863 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:43.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 >> 172.21.15.161:0/612876194 conn(0x5559c5e5fda0 msgr2=0x5559c5e65290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:43.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 shutdown_connections 2023-07-14T11:46:43.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 1 -- 172.21.15.161:0/612876194 wait complete. 2023-07-14T11:46:43.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.918+0000 7f50d70ccb80 20 client.27863 trim_cache size 0 max 16384 2023-07-14T11:46:43.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.935+0000 7f50d70ccb80 1 Processor -- start 2023-07-14T11:46:43.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50d70ccb80 1 -- start start 2023-07-14T11:46:43.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e692d0 0x5559c5e7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e69be0 0x5559c5e6c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e71ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5e698f0 con 0x5559c5e69be0 2023-07-14T11:46:43.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5e69a30 con 0x5559c5e692d0 2023-07-14T11:46:43.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5e70dc0 con 0x5559c5e74ad0 2023-07-14T11:46:43.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50c159b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e692d0 0x5559c5e7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50c1d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e71ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50c159b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e692d0 0x5559c5e7ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47622/0 (socket says 172.21.15.161:47622) 2023-07-14T11:46:43.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50c159b700 1 -- 172.21.15.161:0/2779538258 learned_addr learned my addr 172.21.15.161:0/2779538258 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:43.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e69be0 0x5559c5e6c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50c159b700 1 -- 172.21.15.161:0/2779538258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 msgr2=0x5559c5e71ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.936+0000 7f50c159b700 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e71ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50c159b700 1 -- 172.21.15.161:0/2779538258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e69be0 msgr2=0x5559c5e6c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50c159b700 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e69be0 0x5559c5e6c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50c159b700 1 -- 172.21.15.161:0/2779538258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c5c10250 con 0x5559c5e692d0 2023-07-14T11:46:43.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e69be0 0x5559c5e6c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:46:43.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e71ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:46:43.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50c159b700 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e692d0 0x5559c5e7ff10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006010 tx=0x7f50b800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9fc3a53d4de4a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:43.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50a7fff700 1 -- 172.21.15.161:0/2779538258 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b801d040 con 0x5559c5e692d0 2023-07-14T11:46:43.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50a7fff700 1 -- 172.21.15.161:0/2779538258 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50b801e040 con 0x5559c5e692d0 2023-07-14T11:46:43.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50a7fff700 1 -- 172.21.15.161:0/2779538258 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80052a0 con 0x5559c5e692d0 2023-07-14T11:46:43.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.937+0000 7f50d70ccb80 1 -- 172.21.15.161:0/2779538258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e692d0 msgr2=0x5559c5e7ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e692d0 0x5559c5e7ff10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006010 tx=0x7f50b800b260 comp rx=0 tx=0).stop 2023-07-14T11:46:43.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 -- 172.21.15.161:0/2779538258 shutdown_connections 2023-07-14T11:46:43.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e74ad0 0x5559c5e71ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e692d0 0x5559c5e7ff10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/2779538258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e69be0 0x5559c5e6c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 -- 172.21.15.161:0/2779538258 >> 172.21.15.161:0/2779538258 conn(0x5559c5e5fda0 msgr2=0x5559c5e717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:43.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 -- 172.21.15.161:0/2779538258 shutdown_connections 2023-07-14T11:46:43.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 -- 172.21.15.161:0/2779538258 wait complete. 2023-07-14T11:46:43.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 -- start start 2023-07-14T11:46:43.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.938+0000 7f50d70ccb80 1 Processor -- start 2023-07-14T11:46:43.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e77af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50d70ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e78030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5ed7f30 con 0x5559c5e74ad0 2023-07-14T11:46:43.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5ed80a0 con 0x5559c5e69be0 2023-07-14T11:46:43.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50d70ccb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c5ed8260 con 0x5559c5e692d0 2023-07-14T11:46:43.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50d70ccb80 10 client.0 _ll_register_callbacks cb 0x7f50ac005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:46:43.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e77af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c159b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e77af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47634/0 (socket says 172.21.15.161:47634) 2023-07-14T11:46:43.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3030178633 learned_addr learned my addr 172.21.15.161:0/3030178633 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:46:43.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c1d9c700 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e78030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3030178633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 msgr2=0x5559c5e775b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e775b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3030178633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 msgr2=0x5559c5e78030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:46:43.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e78030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c0d9a700 1 -- 172.21.15.161:0/3030178633 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559c5bd2c60 con 0x5559c5e69be0 2023-07-14T11:46:43.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.939+0000 7f50c159b700 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:46:43.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50c0d9a700 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e77af0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c5b0 tx=0x7f50ac09a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad9b788cbdca365 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:46:43.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:46:43.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50a4ff9700 1 -- 172.21.15.161:0/3030178633 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac095e20 con 0x5559c5e69be0 2023-07-14T11:46:43.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50a4ff9700 1 -- 172.21.15.161:0/3030178633 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f50ac064080 con 0x5559c5e69be0 2023-07-14T11:46:43.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50ac0105a0 con 0x5559c5e69be0 2023-07-14T11:46:43.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50a4ff9700 1 -- 172.21.15.161:0/3030178633 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50ac099870 con 0x5559c5e69be0 2023-07-14T11:46:43.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 10 client.27869 fetch_fsmap learned FSMap version 46 2023-07-14T11:46:43.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f50b40090e0 con 0x5559c5e69be0 2023-07-14T11:46:43.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50a4ff9700 1 -- 172.21.15.161:0/3030178633 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50ac07c020 con 0x5559c5e69be0 2023-07-14T11:46:43.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50a4ff9700 1 -- 172.21.15.161:0/3030178633 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50ac098e70 con 0x5559c5e69be0 2023-07-14T11:46:43.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 10 client.27869 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:46:43.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 10 client.27869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:46:43.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 20 client.27869 populate_metadata read hostname 'smithi161' 2023-07-14T11:46:43.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5ed94f0 0x5559c5edb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:46:43.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 4 client.27869 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:46:43.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.940+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5559c5d77050 con 0x5559c5ed94f0 2023-07-14T11:46:43.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50c159b700 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5ed94f0 0x5559c5edb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:46:43.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50a4ff9700 10 client.27869 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:43.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50c159b700 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5ed94f0 0x5559c5edb8d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab6dc5aeba20633 server_cookie=db057cdc1e7e5e64 in_seq=0 out_seq=0 2023-07-14T11:46:43.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50a4ff9700 1 -- 172.21.15.161:0/3030178633 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5559c5d77050 con 0x5559c5ed94f0 2023-07-14T11:46:43.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50a4ff9700 10 client.27869 handle_command_reply: tid=0 2023-07-14T11:46:43.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 client.27869 shutdown 2023-07-14T11:46:43.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 msgr2=0x5559c5e77af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e77af0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c5b0 tx=0x7f50ac09a070 comp rx=0 tx=0).stop 2023-07-14T11:46:43.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 shutdown_connections 2023-07-14T11:46:43.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5559c5ed94f0 0x5559c5edb8d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559c5e74ad0 0x5559c5e78030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50a4ff9700 0 client.27869 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:46:43.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5559c5e692d0 0x5559c5e775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 --2- 172.21.15.161:0/3030178633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5559c5e69be0 0x5559c5e77af0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.941+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 >> 172.21.15.161:0/3030178633 conn(0x5559c5e5fda0 msgr2=0x5559c5e70610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:43.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 shutdown_connections 2023-07-14T11:46:43.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50d70ccb80 1 -- 172.21.15.161:0/3030178633 wait complete. 2023-07-14T11:46:43.962 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50d70ccb80 20 client.27869 trim_cache size 0 max 16384 2023-07-14T11:46:43.962 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:46:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:46:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:46:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:46:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:46:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:46:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:46:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:46:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:46:43.964 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:46:43.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f50a80a12c0 msgr2=0x7f50a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f50a80a12c0 0x7f50a80a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009ec0 tx=0x7f50b4009920 comp rx=0 tx=0).stop 2023-07-14T11:46:43.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 msgr2=0x7f50bc072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:46:43.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 0x7f50bc072d50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00e9f0 tx=0x7f50ac00ed00 comp rx=0 tx=0).stop 2023-07-14T11:46:43.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 shutdown_connections 2023-07-14T11:46:43.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f50a80a12c0 0x7f50a80a3770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc0ffad0 0x7f50bc072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f50bc105ae0 0x7f50bc1a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 --2- 172.21.15.161:0/550656262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50bc100480 0x7f50bc072d50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:46:43.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 >> 172.21.15.161:0/550656262 conn(0x7f50bc0fb5f0 msgr2=0x7f50bc0fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:46:43.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.942+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 shutdown_connections 2023-07-14T11:46:43.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:46:43.943+0000 7f50c37ff700 1 -- 172.21.15.161:0/550656262 wait complete. 2023-07-14T11:46:43.968 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:46:44.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:44 smithi182 ceph-mon[129366]: pgmap v773: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:44.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:44 smithi161 ceph-mon[95526]: pgmap v773: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:44.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:44 smithi175 ceph-mon[129081]: pgmap v773: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:46.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:46 smithi175 ceph-mon[129081]: pgmap v774: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:46.949 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:46 smithi182 ceph-mon[129366]: pgmap v774: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:46.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:46 smithi161 ceph-mon[95526]: pgmap v774: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:48.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:48 smithi175 ceph-mon[129081]: pgmap v775: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:48.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:48 smithi182 ceph-mon[129366]: pgmap v775: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:48.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:48 smithi161 ceph-mon[95526]: pgmap v775: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:50.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:50 smithi175 ceph-mon[129081]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:50.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:50 smithi182 ceph-mon[129366]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:50.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:50 smithi161 ceph-mon[95526]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:52.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:52 smithi175 ceph-mon[129081]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:52.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:52 smithi182 ceph-mon[129366]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:52.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:52 smithi161 ceph-mon[95526]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:46:54.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:54 smithi182 ceph-mon[129366]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:54.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:54 smithi161 ceph-mon[95526]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:54.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:54 smithi175 ceph-mon[129081]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:46:56.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:56 smithi175 ceph-mon[129081]: pgmap v779: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:56.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:56 smithi182 ceph-mon[129366]: pgmap v779: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:56.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:56 smithi161 ceph-mon[95526]: pgmap v779: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:46:58.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:46:58 smithi175 ceph-mon[129081]: pgmap v780: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:58.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:46:58 smithi182 ceph-mon[129366]: pgmap v780: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:46:58.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:46:58 smithi161 ceph-mon[95526]: pgmap v780: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:00.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:00 smithi175 ceph-mon[129081]: pgmap v781: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:00.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:00 smithi182 ceph-mon[129366]: pgmap v781: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:00.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:00 smithi161 ceph-mon[95526]: pgmap v781: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:02.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:02 smithi175 ceph-mon[129081]: pgmap v782: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:47:02.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:02 smithi182 ceph-mon[129366]: pgmap v782: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:47:02.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:02 smithi161 ceph-mon[95526]: pgmap v782: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:47:04.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:04 smithi182 ceph-mon[129366]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:47:04.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:04 smithi161 ceph-mon[95526]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:47:04.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:04 smithi175 ceph-mon[129081]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:47:06.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:06 smithi175 ceph-mon[129081]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:47:06.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:06 smithi182 ceph-mon[129366]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:47:06.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:06 smithi161 ceph-mon[95526]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:47:07.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:07 smithi175 ceph-mon[129081]: osdmap e130: 12 total, 12 up, 12 in 2023-07-14T11:47:07.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:07 smithi182 ceph-mon[129366]: osdmap e130: 12 total, 12 up, 12 in 2023-07-14T11:47:07.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:07 smithi161 ceph-mon[95526]: osdmap e130: 12 total, 12 up, 12 in 2023-07-14T11:47:08.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:08 smithi175 ceph-mon[129081]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:47:08.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:08 smithi182 ceph-mon[129366]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:47:08.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:08 smithi161 ceph-mon[95526]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:47:10.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:10 smithi175 ceph-mon[129081]: pgmap v787: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:47:10.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:10 smithi182 ceph-mon[129366]: pgmap v787: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:47:10.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:10 smithi161 ceph-mon[95526]: pgmap v787: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:47:12.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:12 smithi175 ceph-mon[129081]: pgmap v788: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:47:12.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:12 smithi182 ceph-mon[129366]: pgmap v788: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:47:12.981 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:12 smithi161 ceph-mon[95526]: pgmap v788: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:47:13.969 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:47:14.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 -- 172.21.15.161:0/24013193 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 msgr2=0x7fd634100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/24013193 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634100f50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd630009990 tx=0x7fd630009ca0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 -- 172.21.15.161:0/24013193 shutdown_connections 2023-07-14T11:47:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/24013193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd63410a0a0 0x7fd63410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/24013193 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634100f50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/24013193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd634101490 0x7fd634109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 -- 172.21.15.161:0/24013193 >> 172.21.15.161:0/24013193 conn(0x7fd6340fa4b0 msgr2=0x7fd6340fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 -- 172.21.15.161:0/24013193 shutdown_connections 2023-07-14T11:47:14.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.199+0000 7fd63c3ff700 1 -- 172.21.15.161:0/24013193 wait complete. 2023-07-14T11:47:14.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.200+0000 7fd63c3ff700 1 Processor -- start 2023-07-14T11:47:14.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.200+0000 7fd63c3ff700 1 -- start start 2023-07-14T11:47:14.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63c3ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63c3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634101490 0x7fd634198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63c3ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63410a0a0 0x7fd63419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63c3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419cee0 con 0x7fd634101490 2023-07-14T11:47:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63c3ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419d050 con 0x7fd63410a0a0 2023-07-14T11:47:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63c3ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419d210 con 0x7fd634100b80 2023-07-14T11:47:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:44668/0 (socket says 172.21.15.161:44668) 2023-07-14T11:47:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63a19b700 1 -- 172.21.15.161:0/1224239961 learned_addr learned my addr 172.21.15.161:0/1224239961 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63a99c700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63410a0a0 0x7fd63419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.201+0000 7fd63999a700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634101490 0x7fd634198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63a19b700 1 -- 172.21.15.161:0/1224239961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63410a0a0 msgr2=0x7fd63419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63a19b700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63410a0a0 0x7fd63419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63a19b700 1 -- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634101490 msgr2=0x7fd634198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63a19b700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634101490 0x7fd634198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63a19b700 1 -- 172.21.15.161:0/1224239961 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6300095f0 con 0x7fd634100b80 2023-07-14T11:47:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63999a700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634101490 0x7fd634198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:47:14.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63a19b700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634197f00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd630005910 tx=0x7fd630005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:14.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd62b7fe700 1 -- 172.21.15.161:0/1224239961 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63001d070 con 0x7fd634100b80 2023-07-14T11:47:14.198 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd62b7fe700 1 -- 172.21.15.161:0/1224239961 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd630022470 con 0x7fd634100b80 2023-07-14T11:47:14.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd62b7fe700 1 -- 172.21.15.161:0/1224239961 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630017650 con 0x7fd634100b80 2023-07-14T11:47:14.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.202+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63419d490 con 0x7fd634100b80 2023-07-14T11:47:14.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.203+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63419d710 con 0x7fd634100b80 2023-07-14T11:47:14.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.206+0000 7fd62b7fe700 1 -- 172.21.15.161:0/1224239961 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd63000fad0 con 0x7fd634100b80 2023-07-14T11:47:14.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.207+0000 7fd64fe05b80 1 Processor -- start 2023-07-14T11:47:14.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.207+0000 7fd64fe05b80 1 -- start start 2023-07-14T11:47:14.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.207 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d734590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd62b7fe700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd6200a12c0 0x7fd6200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d736eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d62fcb0 con 0x55ac7d734ad0 2023-07-14T11:47:14.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d682a40 con 0x55ac7d729be0 2023-07-14T11:47:14.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d67bed0 con 0x55ac7d7292d0 2023-07-14T11:47:14.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd62b7fe700 1 -- 172.21.15.161:0/1224239961 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd6300ce470 con 0x7fd634100b80 2023-07-14T11:47:14.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd63a99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd63a99c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7296a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:44674/0 (socket says 172.21.15.161:44674) 2023-07-14T11:47:14.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd63a99c700 1 -- 172.21.15.161:0/2502328073 learned_addr learned my addr 172.21.15.161:0/2502328073 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:14.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd63a99c700 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d736eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd63999a700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd6200a12c0 0x7fd6200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.208+0000 7fd63a19b700 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d734590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd63a99c700 1 -- 172.21.15.161:0/2502328073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 msgr2=0x55ac7d734590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd63a99c700 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d734590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd63a99c700 1 -- 172.21.15.161:0/2502328073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 msgr2=0x55ac7d736eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd63a99c700 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d736eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd63a99c700 1 -- 172.21.15.161:0/2502328073 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac7d4a9de0 con 0x55ac7d7292d0 2023-07-14T11:47:14.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd63999a700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd6200a12c0 0x7fd6200a3770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd634199510 tx=0x7fd624009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:14.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd63a99c700 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7296a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd62c004a60 tx=0x7fd62c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f122f3fa96f77f64 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:14.212 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd628bfe700 1 -- 172.21.15.161:0/2502328073 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c004bb0 con 0x55ac7d7292d0 2023-07-14T11:47:14.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd628bfe700 1 -- 172.21.15.161:0/2502328073 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c00f030 con 0x55ac7d7292d0 2023-07-14T11:47:14.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.209+0000 7fd628bfe700 1 -- 172.21.15.161:0/2502328073 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01aea0 con 0x55ac7d7292d0 2023-07-14T11:47:14.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2502328073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 msgr2=0x55ac7d7296a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7296a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd62c004a60 tx=0x7fd62c00b040 comp rx=0 tx=0).stop 2023-07-14T11:47:14.213 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2502328073 shutdown_connections 2023-07-14T11:47:14.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d736eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7296a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2502328073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d734590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2502328073 >> 172.21.15.161:0/2502328073 conn(0x55ac7d71fda0 msgr2=0x55ac7d7256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:14.214 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2502328073 shutdown_connections 2023-07-14T11:47:14.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.210+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2502328073 wait complete. 2023-07-14T11:47:14.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.211+0000 7fd64fe05b80 1 -- start start 2023-07-14T11:47:14.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.211+0000 7fd64fe05b80 1 Processor -- start 2023-07-14T11:47:14.215 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.211+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.211+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d731760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.216 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.211+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d731e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d61bac0 con 0x55ac7d734ad0 2023-07-14T11:47:14.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d61b7c0 con 0x55ac7d729be0 2023-07-14T11:47:14.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d4ff920 con 0x55ac7d7292d0 2023-07-14T11:47:14.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd64fe05b80 10 client.0 _ll_register_callbacks cb 0x55ac7d619760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:47:14.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63999a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d731760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7344e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:44684/0 (socket says 172.21.15.161:44684) 2023-07-14T11:47:14.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63999a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d731760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:50650/0 (socket says 172.21.15.161:50650) 2023-07-14T11:47:14.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 -- 172.21.15.161:0/3227156387 learned_addr learned my addr 172.21.15.161:0/3227156387 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:14.220 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a99c700 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d731e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 -- 172.21.15.161:0/3227156387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 msgr2=0x55ac7d731760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d731760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 -- 172.21.15.161:0/3227156387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 msgr2=0x55ac7d731e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d731e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 -- 172.21.15.161:0/3227156387 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c011040 con 0x55ac7d7292d0 2023-07-14T11:47:14.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a99c700 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d731e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:47:14.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd63a19b700 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7344e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd63009b1e0 tx=0x7fd63002a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c928a2cff112def4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:14.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.212+0000 7fd61dffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:47:14.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd61dffb700 1 -- 172.21.15.161:0/3227156387 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630079ec0 con 0x55ac7d7292d0 2023-07-14T11:47:14.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd61dffb700 1 -- 172.21.15.161:0/3227156387 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd63007c070 con 0x55ac7d7292d0 2023-07-14T11:47:14.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac7d5d54f0 con 0x55ac7d7292d0 2023-07-14T11:47:14.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd61dffb700 1 -- 172.21.15.161:0/3227156387 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd63009b250 con 0x55ac7d7292d0 2023-07-14T11:47:14.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 10 client.27925 fetch_fsmap learned FSMap version 46 2023-07-14T11:47:14.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ac7d492c60 con 0x55ac7d7292d0 2023-07-14T11:47:14.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd61dffb700 1 -- 172.21.15.161:0/3227156387 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac7d5d54f0 con 0x55ac7d7292d0 2023-07-14T11:47:14.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd61dffb700 1 -- 172.21.15.161:0/3227156387 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd6300ce110 con 0x55ac7d7292d0 2023-07-14T11:47:14.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 10 client.27925 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:47:14.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 10 client.27925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:47:14.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 20 client.27925 populate_metadata read hostname 'smithi161' 2023-07-14T11:47:14.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d7440c0 0x55ac7d7464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 4 client.27925 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:47:14.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.213+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac7d674eb0 con 0x55ac7d7440c0 2023-07-14T11:47:14.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.214+0000 7fd63999a700 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d7440c0 0x55ac7d7464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.214+0000 7fd61dffb700 10 client.27925 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:14.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.214+0000 7fd63999a700 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d7440c0 0x55ac7d7464d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c543aa4425b35e7d server_cookie=123f446aa64c9aa7 in_seq=0 out_seq=0 2023-07-14T11:47:14.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd61dffb700 1 -- 172.21.15.161:0/3227156387 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ac7d674eb0 con 0x55ac7d7440c0 2023-07-14T11:47:14.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd61dffb700 10 client.27925 handle_command_reply: tid=0 2023-07-14T11:47:14.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 client.27925 shutdown 2023-07-14T11:47:14.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 msgr2=0x55ac7d7344e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7344e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd63009b1e0 tx=0x7fd63002a000 comp rx=0 tx=0).stop 2023-07-14T11:47:14.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 shutdown_connections 2023-07-14T11:47:14.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d7440c0 0x55ac7d7464d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d734ad0 0x55ac7d731e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd61dffb700 0 client.27925 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:14.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d7344e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3227156387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d729be0 0x55ac7d731760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 >> 172.21.15.161:0/3227156387 conn(0x55ac7d71fda0 msgr2=0x55ac7d725230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:14.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.215+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 shutdown_connections 2023-07-14T11:47:14.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.216+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3227156387 wait complete. 2023-07-14T11:47:14.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.216+0000 7fd64fe05b80 20 client.27925 trim_cache size 0 max 16384 2023-07-14T11:47:14.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.234+0000 7fd64fe05b80 1 Processor -- start 2023-07-14T11:47:14.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.234+0000 7fd64fe05b80 1 -- start start 2023-07-14T11:47:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d734080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d730f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d734ad0 0x55ac7d731460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d732d40 con 0x55ac7d729be0 2023-07-14T11:47:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d72fcc0 con 0x55ac7d734ad0 2023-07-14T11:47:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d726ad0 con 0x55ac7d7292d0 2023-07-14T11:47:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d734080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd63999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d730f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d734080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:44688/0 (socket says 172.21.15.161:44688) 2023-07-14T11:47:14.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd63a99c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d734ad0 0x55ac7d731460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.235+0000 7fd63a19b700 1 -- 172.21.15.161:0/3233316829 learned_addr learned my addr 172.21.15.161:0/3233316829 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:14.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd63a19b700 1 -- 172.21.15.161:0/3233316829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d734ad0 msgr2=0x55ac7d731460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd63a19b700 1 --2- 172.21.15.161:0/3233316829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d734ad0 0x55ac7d731460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd63a19b700 1 -- 172.21.15.161:0/3233316829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 msgr2=0x55ac7d730f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd63a19b700 1 --2- 172.21.15.161:0/3233316829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d730f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd63a19b700 1 -- 172.21.15.161:0/3233316829 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac7d4d0250 con 0x55ac7d7292d0 2023-07-14T11:47:14.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd63a19b700 1 --2- 172.21.15.161:0/3233316829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d734080 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd6300ce440 tx=0x7fd630009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b7241bdc6ad53db server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:14.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd6293ff700 1 -- 172.21.15.161:0/3233316829 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63007c110 con 0x55ac7d7292d0 2023-07-14T11:47:14.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd6293ff700 1 -- 172.21.15.161:0/3233316829 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63009b500 con 0x55ac7d7292d0 2023-07-14T11:47:14.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.236+0000 7fd6293ff700 1 -- 172.21.15.161:0/3233316829 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63009b6f0 con 0x55ac7d7292d0 2023-07-14T11:47:14.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3233316829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 msgr2=0x55ac7d734080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3233316829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d734080 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd6300ce440 tx=0x7fd630009310 comp rx=0 tx=0).stop 2023-07-14T11:47:14.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3233316829 shutdown_connections 2023-07-14T11:47:14.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3233316829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d734ad0 0x55ac7d731460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3233316829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d7292d0 0x55ac7d734080 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/3233316829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d730f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3233316829 >> 172.21.15.161:0/3233316829 conn(0x55ac7d71fda0 msgr2=0x55ac7d72ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:14.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3233316829 shutdown_connections 2023-07-14T11:47:14.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 -- 172.21.15.161:0/3233316829 wait complete. 2023-07-14T11:47:14.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.237+0000 7fd64fe05b80 1 -- start start 2023-07-14T11:47:14.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 1 Processor -- start 2023-07-14T11:47:14.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d7292d0 0x55ac7d73b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d739630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d734ad0 0x55ac7d739b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d73a0b0 con 0x55ac7d729be0 2023-07-14T11:47:14.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d73a1f0 con 0x55ac7d7292d0 2023-07-14T11:47:14.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7d73a3b0 con 0x55ac7d734ad0 2023-07-14T11:47:14.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd64fe05b80 10 client.0 _ll_register_callbacks cb 0x7fd630097590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:47:14.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d7292d0 0x55ac7d73b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd63a19b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d7292d0 0x55ac7d73b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:50670/0 (socket says 172.21.15.161:50670) 2023-07-14T11:47:14.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.238+0000 7fd63a19b700 1 -- 172.21.15.161:0/2197970163 learned_addr learned my addr 172.21.15.161:0/2197970163 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:14.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63a99c700 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d734ad0 0x55ac7d739b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63999a700 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d739630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63a19b700 1 -- 172.21.15.161:0/2197970163 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d734ad0 msgr2=0x55ac7d739b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63a19b700 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d734ad0 0x55ac7d739b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63a19b700 1 -- 172.21.15.161:0/2197970163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 msgr2=0x55ac7d739630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:14.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63a19b700 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d739630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63a19b700 1 -- 172.21.15.161:0/2197970163 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac7d492c60 con 0x55ac7d7292d0 2023-07-14T11:47:14.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd63a19b700 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d7292d0 0x55ac7d73b370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd630005f50 tx=0x7fd630005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82502ed9b3292895 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:14.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd61dffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:47:14.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd61dffb700 1 -- 172.21.15.161:0/2197970163 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63007c110 con 0x55ac7d7292d0 2023-07-14T11:47:14.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd62c00f030 con 0x55ac7d7292d0 2023-07-14T11:47:14.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd61dffb700 1 -- 172.21.15.161:0/2197970163 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd63009b250 con 0x55ac7d7292d0 2023-07-14T11:47:14.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd61dffb700 1 -- 172.21.15.161:0/2197970163 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd63009f280 con 0x55ac7d7292d0 2023-07-14T11:47:14.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd64fe05b80 10 client.27887 fetch_fsmap learned FSMap version 46 2023-07-14T11:47:14.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.239+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ac7d4a82b0 con 0x55ac7d7292d0 2023-07-14T11:47:14.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd61dffb700 1 -- 172.21.15.161:0/2197970163 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c00f030 con 0x55ac7d7292d0 2023-07-14T11:47:14.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd61dffb700 1 -- 172.21.15.161:0/2197970163 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd6300b3c30 con 0x55ac7d7292d0 2023-07-14T11:47:14.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd64fe05b80 10 client.27887 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:47:14.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd64fe05b80 10 client.27887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:47:14.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd64fe05b80 20 client.27887 populate_metadata read hostname 'smithi161' 2023-07-14T11:47:14.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d798ef0 0x55ac7d79b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:14.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd64fe05b80 4 client.27887 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:47:14.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac7d637050 con 0x55ac7d798ef0 2023-07-14T11:47:14.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.240+0000 7fd63999a700 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d798ef0 0x55ac7d79b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:14.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd63999a700 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d798ef0 0x55ac7d79b2d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c6909895678e48 server_cookie=cfdf851477b95b24 in_seq=0 out_seq=0 2023-07-14T11:47:14.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd61dffb700 10 client.27887 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:14.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd61dffb700 1 -- 172.21.15.161:0/2197970163 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ac7d637050 con 0x55ac7d798ef0 2023-07-14T11:47:14.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd61dffb700 10 client.27887 handle_command_reply: tid=0 2023-07-14T11:47:14.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 client.27887 shutdown 2023-07-14T11:47:14.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d7292d0 msgr2=0x55ac7d73b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d7292d0 0x55ac7d73b370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd630005f50 tx=0x7fd630005fd0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 shutdown_connections 2023-07-14T11:47:14.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ac7d798ef0 0x55ac7d79b2d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ac7d734ad0 0x55ac7d739b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac7d7292d0 0x55ac7d73b370 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 --2- 172.21.15.161:0/2197970163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac7d729be0 0x55ac7d739630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 >> 172.21.15.161:0/2197970163 conn(0x55ac7d71fda0 msgr2=0x55ac7d72b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:14.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd61dffb700 0 client.27887 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:14.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 shutdown_connections 2023-07-14T11:47:14.249 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.241+0000 7fd64fe05b80 1 -- 172.21.15.161:0/2197970163 wait complete. 2023-07-14T11:47:14.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd64fe05b80 20 client.27887 trim_cache size 0 max 16384 2023-07-14T11:47:14.250 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:47:14.250 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:47:14.251 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:47:14.251 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:47:14.251 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:47:14.251 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:47:14.251 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:47:14.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:47:14.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:47:14.252 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:47:14.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd6200a12c0 msgr2=0x7fd6200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd6200a12c0 0x7fd6200a3770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd634199510 tx=0x7fd624009260 comp rx=0 tx=0).stop 2023-07-14T11:47:14.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 msgr2=0x7fd634197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:14.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634197f00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd630005910 tx=0x7fd630005740 comp rx=0 tx=0).stop 2023-07-14T11:47:14.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 shutdown_connections 2023-07-14T11:47:14.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd6200a12c0 0x7fd6200a3770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.254 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd63410a0a0 0x7fd63419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd634100b80 0x7fd634197f00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 --2- 172.21.15.161:0/1224239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634101490 0x7fd634198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:14.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 >> 172.21.15.161:0/1224239961 conn(0x7fd6340fa4b0 msgr2=0x7fd63410c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:14.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 shutdown_connections 2023-07-14T11:47:14.255 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:14.242+0000 7fd63c3ff700 1 -- 172.21.15.161:0/1224239961 wait complete. 2023-07-14T11:47:14.256 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:47:14.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:14 smithi182 ceph-mon[129366]: osdmap e131: 12 total, 12 up, 12 in 2023-07-14T11:47:14.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:14 smithi182 ceph-mon[129366]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:47:14.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:14 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:47:14.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:14 smithi161 ceph-mon[95526]: osdmap e131: 12 total, 12 up, 12 in 2023-07-14T11:47:14.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:14 smithi161 ceph-mon[95526]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:47:14.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:14 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:47:14.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:14 smithi175 ceph-mon[129081]: osdmap e131: 12 total, 12 up, 12 in 2023-07-14T11:47:14.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:14 smithi175 ceph-mon[129081]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:47:14.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:14 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:47:15.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:47:15.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:47:15.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:47:15.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:47:15.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:47:15.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:47:15.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:47:15.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:47:15.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:47:16.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:16 smithi161 ceph-mon[95526]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:47:16.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:16 smithi175 ceph-mon[129081]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:47:16.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:16 smithi182 ceph-mon[129366]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:47:18.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:18 smithi175 ceph-mon[129081]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:47:18.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:18 smithi182 ceph-mon[129366]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:47:18.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:18 smithi161 ceph-mon[95526]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:47:20.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:20 smithi175 ceph-mon[129081]: pgmap v793: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:47:20.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:20 smithi182 ceph-mon[129366]: pgmap v793: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:47:20.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:20 smithi161 ceph-mon[95526]: pgmap v793: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:47:22.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:22 smithi175 ceph-mon[129081]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:22.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:22 smithi182 ceph-mon[129366]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:22.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:22 smithi161 ceph-mon[95526]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:24.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:24 smithi161 ceph-mon[95526]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-07-14T11:47:24.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:24 smithi175 ceph-mon[129081]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-07-14T11:47:24.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:24 smithi182 ceph-mon[129366]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-07-14T11:47:26.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:26 smithi175 ceph-mon[129081]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T11:47:26.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:26 smithi182 ceph-mon[129366]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T11:47:26.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:26 smithi161 ceph-mon[95526]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T11:47:28.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:28 smithi175 ceph-mon[129081]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:28.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:28 smithi182 ceph-mon[129366]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:28.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:28 smithi161 ceph-mon[95526]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:30.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:30 smithi175 ceph-mon[129081]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T11:47:30.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:30 smithi182 ceph-mon[129366]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T11:47:30.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:30 smithi161 ceph-mon[95526]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T11:47:32.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:32 smithi175 ceph-mon[129081]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2023-07-14T11:47:32.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:32 smithi182 ceph-mon[129366]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2023-07-14T11:47:32.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:32 smithi161 ceph-mon[95526]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2023-07-14T11:47:34.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:34 smithi182 ceph-mon[129366]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:34.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:34 smithi161 ceph-mon[95526]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:34.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:34 smithi175 ceph-mon[129081]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:36.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:36 smithi175 ceph-mon[129081]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:36.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:36 smithi182 ceph-mon[129366]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:36.980 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:36 smithi161 ceph-mon[95526]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:38.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:38 smithi175 ceph-mon[129081]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:38.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:38 smithi182 ceph-mon[129366]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:38.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:38 smithi161 ceph-mon[95526]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:40.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:40 smithi175 ceph-mon[129081]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:40.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:40 smithi182 ceph-mon[129366]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:40.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:40 smithi161 ceph-mon[95526]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:42.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:42 smithi175 ceph-mon[129081]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:47:42.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:42 smithi182 ceph-mon[129366]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:47:42.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:42 smithi161 ceph-mon[95526]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:47:44.257 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:47:44.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:44 smithi182 ceph-mon[129366]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:44.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:44 smithi161 ceph-mon[95526]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:44.488 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 -- 172.21.15.161:0/1533196447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc100ca0 msgr2=0x7fabfc101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 --2- 172.21.15.161:0/1533196447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc100ca0 0x7fabfc101070 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009990 tx=0x7fabf0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 -- 172.21.15.161:0/1533196447 shutdown_connections 2023-07-14T11:47:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 --2- 172.21.15.161:0/1533196447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabfc10a070 0x7fabfc10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 --2- 172.21.15.161:0/1533196447 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc100ca0 0x7fabfc101070 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 --2- 172.21.15.161:0/1533196447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc1015b0 0x7fabfc109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 -- 172.21.15.161:0/1533196447 >> 172.21.15.161:0/1533196447 conn(0x7fabfc0fa510 msgr2=0x7fabfc0fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:44.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 -- 172.21.15.161:0/1533196447 shutdown_connections 2023-07-14T11:47:44.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.502+0000 7fac02dff700 1 -- 172.21.15.161:0/1533196447 wait complete. 2023-07-14T11:47:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 Processor -- start 2023-07-14T11:47:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 -- start start 2023-07-14T11:47:44.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc100ca0 0x7fabfc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc1015b0 0x7fabfc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabfc10a070 0x7fabfc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc19cdf0 con 0x7fabfc10a070 2023-07-14T11:47:44.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc19cf60 con 0x7fabfc100ca0 2023-07-14T11:47:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.503+0000 7fac02dff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc19d120 con 0x7fabfc1015b0 2023-07-14T11:47:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc100ca0 0x7fabfc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc100ca0 0x7fabfc197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49478/0 (socket says 172.21.15.161:49478) 2023-07-14T11:47:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 -- 172.21.15.161:0/2221524448 learned_addr learned my addr 172.21.15.161:0/2221524448 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:44.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fabfbfff700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc1015b0 0x7fabfc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac0139c700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabfc10a070 0x7fabfc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 -- 172.21.15.161:0/2221524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc1015b0 msgr2=0x7fabfc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc1015b0 0x7fabfc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 -- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabfc10a070 msgr2=0x7fabfc19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabfc10a070 0x7fabfc19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 -- 172.21.15.161:0/2221524448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009c40 con 0x7fabfc100ca0 2023-07-14T11:47:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac0139c700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabfc10a070 0x7fabfc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:47:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc100ca0 0x7fabfc197e10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005910 tx=0x7fabf0004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fabf9ffb700 1 -- 172.21.15.161:0/2221524448 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf001d070 con 0x7fabfc100ca0 2023-07-14T11:47:44.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fabf9ffb700 1 -- 172.21.15.161:0/2221524448 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabf0022470 con 0x7fabfc100ca0 2023-07-14T11:47:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf00095f0 con 0x7fabfc100ca0 2023-07-14T11:47:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.504+0000 7fabf9ffb700 1 -- 172.21.15.161:0/2221524448 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0017650 con 0x7fabfc100ca0 2023-07-14T11:47:44.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.505+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc1a4a60 con 0x7fabfc100ca0 2023-07-14T11:47:44.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.508+0000 7fabf9ffb700 1 -- 172.21.15.161:0/2221524448 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fabf00225e0 con 0x7fabfc100ca0 2023-07-14T11:47:44.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 Processor -- start 2023-07-14T11:47:44.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 -- start start 2023-07-14T11:47:44.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456aff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456b0aad0 0x55c456b0ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456a05cb0 con 0x55c456b0aad0 2023-07-14T11:47:44.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456a58a40 con 0x55c456affbe0 2023-07-14T11:47:44.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.509+0000 7fac16681b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456a51ed0 con 0x55c456aff2d0 2023-07-14T11:47:44.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fabf9ffb700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fabec0a12c0 0x7fabec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fabf9ffb700 1 -- 172.21.15.161:0/2221524448 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fabf00ce380 con 0x7fabfc100ca0 2023-07-14T11:47:44.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49492/0 (socket says 172.21.15.161:49492) 2023-07-14T11:47:44.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac0139c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456aff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 -- 172.21.15.161:0/2257026982 learned_addr learned my addr 172.21.15.161:0/2257026982 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:44.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fabfbfff700 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456b0aad0 0x55c456b0ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac0139c700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fabec0a12c0 0x7fabec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 -- 172.21.15.161:0/2257026982 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 msgr2=0x55c456aff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456aff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 -- 172.21.15.161:0/2257026982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456b0aad0 msgr2=0x55c456b0ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456b0aad0 0x55c456b0ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 -- 172.21.15.161:0/2257026982 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45687fde0 con 0x55c456affbe0 2023-07-14T11:47:44.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.510+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fabf009b0b0 tx=0x7fabf0066620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=195f438c951ac7ca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:44.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fabe6ffd700 1 -- 172.21.15.161:0/2257026982 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf009aa80 con 0x55c456affbe0 2023-07-14T11:47:44.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fabe6ffd700 1 -- 172.21.15.161:0/2257026982 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabf009b120 con 0x55c456affbe0 2023-07-14T11:47:44.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fabe6ffd700 1 -- 172.21.15.161:0/2257026982 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0029ea0 con 0x55c456affbe0 2023-07-14T11:47:44.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 -- 172.21.15.161:0/2257026982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 msgr2=0x55c456b0a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac0139c700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fabec0a12c0 0x7fabec0a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009c10 tx=0x7fabf400b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:44.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fabf009b0b0 tx=0x7fabf0066620 comp rx=0 tx=0).stop 2023-07-14T11:47:44.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 -- 172.21.15.161:0/2257026982 shutdown_connections 2023-07-14T11:47:44.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456b0aad0 0x55c456b0ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456aff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 --2- 172.21.15.161:0/2257026982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a590 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 -- 172.21.15.161:0/2257026982 >> 172.21.15.161:0/2257026982 conn(0x55c456af5da0 msgr2=0x55c456afb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:44.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.511+0000 7fac16681b80 1 -- 172.21.15.161:0/2257026982 shutdown_connections 2023-07-14T11:47:44.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.512+0000 7fac16681b80 1 -- 172.21.15.161:0/2257026982 wait complete. 2023-07-14T11:47:44.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.512+0000 7fac16681b80 1 -- start start 2023-07-14T11:47:44.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.512+0000 7fac16681b80 1 Processor -- start 2023-07-14T11:47:44.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac16681b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456aff2d0 0x55c456b09c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac16681b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac16681b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b0e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac16681b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4569f1ac0 con 0x55c456aff2d0 2023-07-14T11:47:44.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac16681b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4569f17c0 con 0x55c456affbe0 2023-07-14T11:47:44.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac16681b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4568d5920 con 0x55c456b0aad0 2023-07-14T11:47:44.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac16681b80 10 client.0 _ll_register_callbacks cb 0x55c4569ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:47:44.513 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49502/0 (socket says 172.21.15.161:49502) 2023-07-14T11:47:44.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 -- 172.21.15.161:0/4078782942 learned_addr learned my addr 172.21.15.161:0/4078782942 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:44.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac0139c700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b0e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac00b9b700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456aff2d0 0x55c456b09c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 -- 172.21.15.161:0/4078782942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 msgr2=0x55c456b0e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b0e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 -- 172.21.15.161:0/4078782942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456aff2d0 msgr2=0x55c456b09c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456aff2d0 0x55c456b09c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fabfbfff700 1 -- 172.21.15.161:0/4078782942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc1a4a60 con 0x55c456affbe0 2023-07-14T11:47:44.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.513+0000 7fac0139c700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b0e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:47:44.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fabfbfff700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a1d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fabe80127f0 tx=0x7fabe8012bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0d01304f5f9ce6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:44.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fabdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:47:44.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fabdbfff700 1 -- 172.21.15.161:0/4078782942 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8017070 con 0x55c456affbe0 2023-07-14T11:47:44.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4569ab4f0 con 0x55c456affbe0 2023-07-14T11:47:44.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fabdbfff700 1 -- 172.21.15.161:0/4078782942 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fabe800f640 con 0x55c456affbe0 2023-07-14T11:47:44.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fabdbfff700 1 -- 172.21.15.161:0/4078782942 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabe800f800 con 0x55c456affbe0 2023-07-14T11:47:44.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 10 client.27899 fetch_fsmap learned FSMap version 46 2023-07-14T11:47:44.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55c456868c60 con 0x55c456affbe0 2023-07-14T11:47:44.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fabdbfff700 1 -- 172.21.15.161:0/4078782942 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4569ab4f0 con 0x55c456affbe0 2023-07-14T11:47:44.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fabdbfff700 1 -- 172.21.15.161:0/4078782942 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fabe8010910 con 0x55c456affbe0 2023-07-14T11:47:44.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 10 client.27899 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:47:44.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 10 client.27899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:47:44.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 20 client.27899 populate_metadata read hostname 'smithi161' 2023-07-14T11:47:44.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b1a090 0x55c456b1c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 4 client.27899 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:47:44.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.514+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c456a4aeb0 con 0x55c456b1a090 2023-07-14T11:47:44.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.515+0000 7fac00b9b700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b1a090 0x55c456b1c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.515+0000 7fabdbfff700 10 client.27899 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:44.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.515+0000 7fac00b9b700 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b1a090 0x55c456b1c4a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d9a441d81729 server_cookie=343162b55bca21cd in_seq=0 out_seq=0 2023-07-14T11:47:44.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fabdbfff700 1 -- 172.21.15.161:0/4078782942 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c456a4aeb0 con 0x55c456b1a090 2023-07-14T11:47:44.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fabdbfff700 10 client.27899 handle_command_reply: tid=0 2023-07-14T11:47:44.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 client.27899 shutdown 2023-07-14T11:47:44.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 msgr2=0x55c456b0a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a1d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fabe80127f0 tx=0x7fabe8012bb0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 shutdown_connections 2023-07-14T11:47:44.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b1a090 0x55c456b1c4a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456aff2d0 0x55c456b09c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b0e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 --2- 172.21.15.161:0/4078782942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456affbe0 0x55c456b0a1d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fabdbfff700 0 client.27899 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:44.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 >> 172.21.15.161:0/4078782942 conn(0x55c456af5da0 msgr2=0x55c456afb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:44.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 shutdown_connections 2023-07-14T11:47:44.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 1 -- 172.21.15.161:0/4078782942 wait complete. 2023-07-14T11:47:44.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.516+0000 7fac16681b80 20 client.27899 trim_cache size 0 max 16384 2023-07-14T11:47:44.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 Processor -- start 2023-07-14T11:47:44.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 -- start start 2023-07-14T11:47:44.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456aff2d0 0x55c456b0a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b0e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b09240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456b0b450 con 0x55c456affbe0 2023-07-14T11:47:44.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456af5970 con 0x55c456aff2d0 2023-07-14T11:47:44.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.534+0000 7fac16681b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456afc4e0 con 0x55c456b0aad0 2023-07-14T11:47:44.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456aff2d0 0x55c456b0a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac0139c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b09240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456aff2d0 0x55c456b0a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49516/0 (socket says 172.21.15.161:49516) 2023-07-14T11:47:44.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 -- 172.21.15.161:0/2475577125 learned_addr learned my addr 172.21.15.161:0/2475577125 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:44.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac0139c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b09240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41914/0 (socket says 172.21.15.161:41914) 2023-07-14T11:47:44.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fabfbfff700 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b0e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 -- 172.21.15.161:0/2475577125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 msgr2=0x55c456b09240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b09240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 -- 172.21.15.161:0/2475577125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 msgr2=0x55c456b0e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b0e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 -- 172.21.15.161:0/2475577125 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4568a6250 con 0x55c456aff2d0 2023-07-14T11:47:44.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fac00b9b700 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456aff2d0 0x55c456b0a1b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009ce0 tx=0x7fabf009e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af68c67cd1de1840 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:44.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fabe77fe700 1 -- 172.21.15.161:0/2475577125 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf009aa80 con 0x55c456aff2d0 2023-07-14T11:47:44.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fabe77fe700 1 -- 172.21.15.161:0/2475577125 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabf0097460 con 0x55c456aff2d0 2023-07-14T11:47:44.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.535+0000 7fabe77fe700 1 -- 172.21.15.161:0/2475577125 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0066210 con 0x55c456aff2d0 2023-07-14T11:47:44.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 -- 172.21.15.161:0/2475577125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456aff2d0 msgr2=0x55c456b0a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456aff2d0 0x55c456b0a1b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009ce0 tx=0x7fabf009e040 comp rx=0 tx=0).stop 2023-07-14T11:47:44.530 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 -- 172.21.15.161:0/2475577125 shutdown_connections 2023-07-14T11:47:44.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456b0aad0 0x55c456b09240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456aff2d0 0x55c456b0a1b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 --2- 172.21.15.161:0/2475577125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b0e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 -- 172.21.15.161:0/2475577125 >> 172.21.15.161:0/2475577125 conn(0x55c456af5da0 msgr2=0x55c456b09aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:44.531 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 -- 172.21.15.161:0/2475577125 shutdown_connections 2023-07-14T11:47:44.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.536+0000 7fac16681b80 1 -- 172.21.15.161:0/2475577125 wait complete. 2023-07-14T11:47:44.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 -- start start 2023-07-14T11:47:44.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 Processor -- start 2023-07-14T11:47:44.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456b67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b67790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456b0aad0 0x55c456b12340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456b12910 con 0x55c456affbe0 2023-07-14T11:47:44.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456b12a80 con 0x55c456b0aad0 2023-07-14T11:47:44.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456b12c40 con 0x55c456aff2d0 2023-07-14T11:47:44.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac16681b80 10 client.0 _ll_register_callbacks cb 0x7fabe8009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:47:44.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456b67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac0139c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456b0aad0 0x55c456b12340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac00b9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456b67250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41930/0 (socket says 172.21.15.161:41930) 2023-07-14T11:47:44.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fac00b9b700 1 -- 172.21.15.161:0/3480313178 learned_addr learned my addr 172.21.15.161:0/3480313178 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:47:44.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.537+0000 7fabfbfff700 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b67790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac00b9b700 1 -- 172.21.15.161:0/3480313178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456b0aad0 msgr2=0x55c456b12340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac00b9b700 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456b0aad0 0x55c456b12340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac00b9b700 1 -- 172.21.15.161:0/3480313178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 msgr2=0x55c456b67790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:47:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac00b9b700 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b67790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac00b9b700 1 -- 172.21.15.161:0/3480313178 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45687fde0 con 0x55c456aff2d0 2023-07-14T11:47:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac00b9b700 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456b67250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabf00cd010 tx=0x7fabf009ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5768c17f5f4d8d9d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:47:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fabdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:47:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fabdbfff700 1 -- 172.21.15.161:0/3480313178 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0022900 con 0x55c456aff2d0 2023-07-14T11:47:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fabdbfff700 1 -- 172.21.15.161:0/3480313178 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fabf00cb990 con 0x55c456aff2d0 2023-07-14T11:47:44.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabf009b120 con 0x55c456aff2d0 2023-07-14T11:47:44.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fabdbfff700 1 -- 172.21.15.161:0/3480313178 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabf00cbb50 con 0x55c456aff2d0 2023-07-14T11:47:44.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac16681b80 10 client.27949 fetch_fsmap learned FSMap version 46 2023-07-14T11:47:44.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.538+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55c45687e2b0 con 0x55c456aff2d0 2023-07-14T11:47:44.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fabdbfff700 1 -- 172.21.15.161:0/3480313178 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabf009b120 con 0x55c456aff2d0 2023-07-14T11:47:44.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fabdbfff700 1 -- 172.21.15.161:0/3480313178 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fabf009a610 con 0x55c456aff2d0 2023-07-14T11:47:44.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fac16681b80 10 client.27949 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:47:44.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fac16681b80 10 client.27949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:47:44.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fac16681b80 20 client.27949 populate_metadata read hostname 'smithi161' 2023-07-14T11:47:44.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fac16681b80 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b6eb10 0x55c456b70ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:47:44.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fac16681b80 4 client.27949 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:47:44.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c456a0d050 con 0x55c456b6eb10 2023-07-14T11:47:44.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.539+0000 7fabfbfff700 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b6eb10 0x55c456b70ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:47:44.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fabdbfff700 10 client.27949 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:44.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fabfbfff700 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b6eb10 0x55c456b70ef0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae821865042441a server_cookie=1414b9d0e913ffaa in_seq=0 out_seq=0 2023-07-14T11:47:44.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fabdbfff700 1 -- 172.21.15.161:0/3480313178 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c456a0d050 con 0x55c456b6eb10 2023-07-14T11:47:44.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fabdbfff700 10 client.27949 handle_command_reply: tid=0 2023-07-14T11:47:44.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 client.27949 shutdown 2023-07-14T11:47:44.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 msgr2=0x55c456b67250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456b67250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabf00cd010 tx=0x7fabf009ad20 comp rx=0 tx=0).stop 2023-07-14T11:47:44.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 shutdown_connections 2023-07-14T11:47:44.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c456b6eb10 0x55c456b70ef0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c456b0aad0 0x55c456b12340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fabdbfff700 0 client.27949 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:47:44.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c456aff2d0 0x55c456b67250 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 --2- 172.21.15.161:0/3480313178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c456affbe0 0x55c456b67790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 >> 172.21.15.161:0/3480313178 conn(0x55c456af5da0 msgr2=0x55c456b01c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:44.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 shutdown_connections 2023-07-14T11:47:44.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 1 -- 172.21.15.161:0/3480313178 wait complete. 2023-07-14T11:47:44.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.540+0000 7fac16681b80 20 client.27949 trim_cache size 0 max 16384 2023-07-14T11:47:44.543 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:47:44.544 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:47:44.544 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:47:44.544 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:47:44.544 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:47:44.545 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:47:44.545 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:47:44.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:47:44.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:47:44.545 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:47:44.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fabec0a12c0 msgr2=0x7fabec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fabec0a12c0 0x7fabec0a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009c10 tx=0x7fabf400b040 comp rx=0 tx=0).stop 2023-07-14T11:47:44.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc100ca0 msgr2=0x7fabfc197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:47:44.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc100ca0 0x7fabfc197e10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005910 tx=0x7fabf0004710 comp rx=0 tx=0).stop 2023-07-14T11:47:44.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 shutdown_connections 2023-07-14T11:47:44.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fabec0a12c0 0x7fabec0a3770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabfc10a070 0x7fabfc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabfc100ca0 0x7fabfc197e10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 --2- 172.21.15.161:0/2221524448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fabfc1015b0 0x7fabfc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:47:44.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 >> 172.21.15.161:0/2221524448 conn(0x7fabfc0fa510 msgr2=0x7fabfc10c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:47:44.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 shutdown_connections 2023-07-14T11:47:44.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:47:44.541+0000 7fac02dff700 1 -- 172.21.15.161:0/2221524448 wait complete. 2023-07-14T11:47:44.549 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:47:44.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:44 smithi175 ceph-mon[129081]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:46.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:46 smithi175 ceph-mon[129081]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:46.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:46 smithi182 ceph-mon[129366]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:46.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:46 smithi161 ceph-mon[95526]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:48.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:48 smithi175 ceph-mon[129081]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:48.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:48 smithi182 ceph-mon[129366]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:48.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:48 smithi161 ceph-mon[95526]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:50.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:50 smithi175 ceph-mon[129081]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:50.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:50 smithi182 ceph-mon[129366]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:50.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:50 smithi161 ceph-mon[95526]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:52.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:52 smithi175 ceph-mon[129081]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:47:52.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:52 smithi182 ceph-mon[129366]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:47:52.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:52 smithi161 ceph-mon[95526]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:47:54.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:54 smithi161 ceph-mon[95526]: pgmap v810: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:54.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:54 smithi175 ceph-mon[129081]: pgmap v810: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:54.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:54 smithi182 ceph-mon[129366]: pgmap v810: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:47:56.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:56 smithi175 ceph-mon[129081]: pgmap v811: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:56.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:56 smithi182 ceph-mon[129366]: pgmap v811: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:56.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:56 smithi161 ceph-mon[95526]: pgmap v811: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:47:58.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:47:58 smithi175 ceph-mon[129081]: pgmap v812: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:58.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:47:58 smithi182 ceph-mon[129366]: pgmap v812: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:47:58.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:47:58 smithi161 ceph-mon[95526]: pgmap v812: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:00.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:00 smithi175 ceph-mon[129081]: pgmap v813: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:00.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:00 smithi182 ceph-mon[129366]: pgmap v813: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:00.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:00 smithi161 ceph-mon[95526]: pgmap v813: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:02.730 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:02 smithi161 ceph-mon[95526]: pgmap v814: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:48:02.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:02 smithi175 ceph-mon[129081]: pgmap v814: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:48:02.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:02 smithi182 ceph-mon[129366]: pgmap v814: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:48:04.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:04 smithi161 ceph-mon[95526]: pgmap v815: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:48:04.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:04 smithi175 ceph-mon[129081]: pgmap v815: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:48:04.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:04 smithi182 ceph-mon[129366]: pgmap v815: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:48:06.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:06 smithi175 ceph-mon[129081]: pgmap v816: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:48:06.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:06 smithi161 ceph-mon[95526]: pgmap v816: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:48:06.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:06 smithi182 ceph-mon[129366]: pgmap v816: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:48:07.729 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:07 smithi161 ceph-mon[95526]: osdmap e132: 12 total, 12 up, 12 in 2023-07-14T11:48:07.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:07 smithi175 ceph-mon[129081]: osdmap e132: 12 total, 12 up, 12 in 2023-07-14T11:48:07.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:07 smithi182 ceph-mon[129366]: osdmap e132: 12 total, 12 up, 12 in 2023-07-14T11:48:08.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:08 smithi175 ceph-mon[129081]: pgmap v818: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:48:08.948 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:08 smithi182 ceph-mon[129366]: pgmap v818: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:48:08.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:08 smithi161 ceph-mon[95526]: pgmap v818: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:48:10.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:10 smithi175 ceph-mon[129081]: pgmap v819: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:48:10.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:10 smithi182 ceph-mon[129366]: pgmap v819: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:48:10.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:10 smithi161 ceph-mon[95526]: pgmap v819: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:48:12.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:12 smithi175 ceph-mon[129081]: pgmap v820: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:48:12.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:12 smithi161 ceph-mon[95526]: pgmap v820: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:48:12.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:12 smithi182 ceph-mon[129366]: pgmap v820: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:48:14.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:14 smithi161 ceph-mon[95526]: osdmap e133: 12 total, 12 up, 12 in 2023-07-14T11:48:14.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:14 smithi161 ceph-mon[95526]: pgmap v822: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:48:14.551 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:48:14.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:14 smithi175 ceph-mon[129081]: osdmap e133: 12 total, 12 up, 12 in 2023-07-14T11:48:14.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:14 smithi175 ceph-mon[129081]: pgmap v822: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:48:14.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:14 smithi182 ceph-mon[129366]: osdmap e133: 12 total, 12 up, 12 in 2023-07-14T11:48:14.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:14 smithi182 ceph-mon[129366]: pgmap v822: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:48:14.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 -- 172.21.15.161:0/2067312188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 msgr2=0x7f04c4105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/2067312188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c4105ec0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098f0 tx=0x7f04b8009c00 comp rx=0 tx=0).stop 2023-07-14T11:48:14.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 -- 172.21.15.161:0/2067312188 shutdown_connections 2023-07-14T11:48:14.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/2067312188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c4100490 0x7f04c410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/2067312188 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c4105ec0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/2067312188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c40ffae0 0x7f04c40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 -- 172.21.15.161:0/2067312188 >> 172.21.15.161:0/2067312188 conn(0x7f04c40fb5e0 msgr2=0x7f04c40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:14.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.787+0000 7f04cb1ff700 1 -- 172.21.15.161:0/2067312188 shutdown_connections 2023-07-14T11:48:14.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.788+0000 7f04cb1ff700 1 -- 172.21.15.161:0/2067312188 wait complete. 2023-07-14T11:48:14.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.788+0000 7f04cb1ff700 1 Processor -- start 2023-07-14T11:48:14.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04cb1ff700 1 -- start start 2023-07-14T11:48:14.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04cb1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c40ffae0 0x7f04c4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04cb1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c4100490 0x7f04c4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04cb1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04cb1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c419ce00 con 0x7f04c40ffae0 2023-07-14T11:48:14.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04cb1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c419cf70 con 0x7f04c4100490 2023-07-14T11:48:14.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04cb1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c419d130 con 0x7f04c4105af0 2023-07-14T11:48:14.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c40ffae0 0x7f04c4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04c979c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c4100490 0x7f04c4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c40ffae0 0x7f04c4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41548/0 (socket says 172.21.15.161:41548) 2023-07-14T11:48:14.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04c8f9b700 1 -- 172.21.15.161:0/768428000 learned_addr learned my addr 172.21.15.161:0/768428000 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:14.779 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.789+0000 7f04c979c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:40422/0 (socket says 172.21.15.161:40422) 2023-07-14T11:48:14.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c8f9b700 1 -- 172.21.15.161:0/768428000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 msgr2=0x7f04c419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c8f9b700 1 -- 172.21.15.161:0/768428000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c4100490 msgr2=0x7f04c4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c4100490 0x7f04c4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.780 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c8f9b700 1 -- 172.21.15.161:0/768428000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b8009520 con 0x7f04c40ffae0 2023-07-14T11:48:14.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c979c700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:14.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c3fff700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c4100490 0x7f04c4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:14.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c40ffae0 0x7f04c4197e20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009810 tx=0x7f04b800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:14.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c1ffb700 1 -- 172.21.15.161:0/768428000 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b801c070 con 0x7f04c40ffae0 2023-07-14T11:48:14.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c419d3b0 con 0x7f04c40ffae0 2023-07-14T11:48:14.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c1ffb700 1 -- 172.21.15.161:0/768428000 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b800bbd0 con 0x7f04c40ffae0 2023-07-14T11:48:14.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04c1ffb700 1 -- 172.21.15.161:0/768428000 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8004930 con 0x7f04c40ffae0 2023-07-14T11:48:14.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.790+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c41a4c50 con 0x7f04c40ffae0 2023-07-14T11:48:14.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.793+0000 7f04c1ffb700 1 -- 172.21.15.161:0/768428000 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f04b8020070 con 0x7f04c40ffae0 2023-07-14T11:48:14.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 Processor -- start 2023-07-14T11:48:14.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04c1ffb700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f04b40a1340 0x7f04b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 -- start start 2023-07-14T11:48:14.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04c1ffb700 1 -- 172.21.15.161:0/768428000 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f04b80cdc70 con 0x7f04c40ffae0 2023-07-14T11:48:14.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04c979c700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f04b40a1340 0x7f04b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x55980939b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939bbe0 0x5598093a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598093a6ad0 0x5598093a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598092a1cb0 con 0x5598093a6ad0 2023-07-14T11:48:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598092f4a40 con 0x55980939b2d0 2023-07-14T11:48:14.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.795+0000 7f04debbcb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598092eded0 con 0x55980939bbe0 2023-07-14T11:48:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x55980939b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939bbe0 0x5598093a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x55980939b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60482/0 (socket says 172.21.15.161:60482) 2023-07-14T11:48:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 -- 172.21.15.161:0/3073034016 learned_addr learned my addr 172.21.15.161:0/3073034016 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:14.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c979c700 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598093a6ad0 0x5598093a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 -- 172.21.15.161:0/3073034016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939bbe0 msgr2=0x5598093a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939bbe0 0x5598093a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 -- 172.21.15.161:0/3073034016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598093a6ad0 msgr2=0x5598093a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598093a6ad0 0x5598093a8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 -- 172.21.15.161:0/3073034016 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980911bde0 con 0x55980939b2d0 2023-07-14T11:48:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c979c700 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598093a6ad0 0x5598093a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:48:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c979c700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f04b40a1340 0x7f04b40a37f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f04bc003e80 tx=0x7f04bc009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x55980939b6a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f04b8097480 tx=0x7f04b8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a3883067f79a45 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04aeffd700 1 -- 172.21.15.161:0/3073034016 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b809ea40 con 0x55980939b2d0 2023-07-14T11:48:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04aeffd700 1 -- 172.21.15.161:0/3073034016 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b80a2dd0 con 0x55980939b2d0 2023-07-14T11:48:14.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.796+0000 7f04aeffd700 1 -- 172.21.15.161:0/3073034016 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b809a160 con 0x55980939b2d0 2023-07-14T11:48:14.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 -- 172.21.15.161:0/3073034016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 msgr2=0x55980939b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x55980939b6a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f04b8097480 tx=0x7f04b8009260 comp rx=0 tx=0).stop 2023-07-14T11:48:14.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 -- 172.21.15.161:0/3073034016 shutdown_connections 2023-07-14T11:48:14.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598093a6ad0 0x5598093a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x55980939b6a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 --2- 172.21.15.161:0/3073034016 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939bbe0 0x5598093a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 -- 172.21.15.161:0/3073034016 >> 172.21.15.161:0/3073034016 conn(0x559809391da0 msgr2=0x5598093976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 -- 172.21.15.161:0/3073034016 shutdown_connections 2023-07-14T11:48:14.792 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.797+0000 7f04debbcb80 1 -- 172.21.15.161:0/3073034016 wait complete. 2023-07-14T11:48:14.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.798+0000 7f04debbcb80 1 -- start start 2023-07-14T11:48:14.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.798+0000 7f04debbcb80 1 Processor -- start 2023-07-14T11:48:14.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x5598093abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x5598093a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04debbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980928dac0 con 0x55980939bbe0 2023-07-14T11:48:14.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04debbcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980928d7c0 con 0x55980939b2d0 2023-07-14T11:48:14.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04debbcb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x559809171920 con 0x5598093a6ad0 2023-07-14T11:48:14.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04debbcb80 10 client.0 _ll_register_callbacks cb 0x55980928b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:48:14.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x5598093abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x5598093abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60498/0 (socket says 172.21.15.161:60498) 2023-07-14T11:48:14.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c8f9b700 1 -- 172.21.15.161:0/424133147 learned_addr learned my addr 172.21.15.161:0/424133147 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:14.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c979c700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x5598093a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c3fff700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c8f9b700 1 -- 172.21.15.161:0/424133147 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 msgr2=0x5598093a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x5598093a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c8f9b700 1 -- 172.21.15.161:0/424133147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 msgr2=0x5598093ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.799+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04c8f9b700 1 -- 172.21.15.161:0/424133147 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c41a4c50 con 0x55980939b2d0 2023-07-14T11:48:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04c3fff700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:48:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x5598093abb70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f04b80a28a0 tx=0x7f04b80caf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64d68c60f01078b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:48:14.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04a3fff700 1 -- 172.21.15.161:0/424133147 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b80b3120 con 0x55980939b2d0 2023-07-14T11:48:14.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598092474f0 con 0x55980939b2d0 2023-07-14T11:48:14.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04a3fff700 1 -- 172.21.15.161:0/424133147 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f04b80a2940 con 0x55980939b2d0 2023-07-14T11:48:14.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04a3fff700 1 -- 172.21.15.161:0/424133147 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b80a2b00 con 0x55980939b2d0 2023-07-14T11:48:14.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04debbcb80 10 client.27917 fetch_fsmap learned FSMap version 46 2023-07-14T11:48:14.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x559809104c60 con 0x55980939b2d0 2023-07-14T11:48:14.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04a3fff700 1 -- 172.21.15.161:0/424133147 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598092474f0 con 0x55980939b2d0 2023-07-14T11:48:14.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04a3fff700 1 -- 172.21.15.161:0/424133147 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04b8096900 con 0x55980939b2d0 2023-07-14T11:48:14.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04debbcb80 10 client.27917 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:48:14.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.800+0000 7f04debbcb80 10 client.27917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:48:14.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.801+0000 7f04debbcb80 20 client.27917 populate_metadata read hostname 'smithi161' 2023-07-14T11:48:14.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.801+0000 7f04debbcb80 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5598093b5af0 0x5598093b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.801+0000 7f04debbcb80 4 client.27917 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:48:14.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.801+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598092e6eb0 con 0x5598093b5af0 2023-07-14T11:48:14.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.801+0000 7f04c3fff700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5598093b5af0 0x5598093b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.801+0000 7f04a3fff700 10 client.27917 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:14.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.801+0000 7f04c3fff700 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5598093b5af0 0x5598093b7ed0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace2e068126cd497 server_cookie=265920b1317eb0cc in_seq=0 out_seq=0 2023-07-14T11:48:14.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.802+0000 7f04a3fff700 1 -- 172.21.15.161:0/424133147 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598092e6eb0 con 0x5598093b5af0 2023-07-14T11:48:14.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.802+0000 7f04a3fff700 10 client.27917 handle_command_reply: tid=0 2023-07-14T11:48:14.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.802+0000 7f04debbcb80 1 client.27917 shutdown 2023-07-14T11:48:14.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.802+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 msgr2=0x5598093abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.802+0000 7f04debbcb80 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x5598093abb70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f04b80a28a0 tx=0x7f04b80caf70 comp rx=0 tx=0).stop 2023-07-14T11:48:14.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 shutdown_connections 2023-07-14T11:48:14.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5598093b5af0 0x5598093b7ed0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x5598093a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04a3fff700 0 client.27917 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:14.804 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x5598093abb70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 --2- 172.21.15.161:0/424133147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 >> 172.21.15.161:0/424133147 conn(0x559809391da0 msgr2=0x559809397290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:14.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 shutdown_connections 2023-07-14T11:48:14.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 1 -- 172.21.15.161:0/424133147 wait complete. 2023-07-14T11:48:14.805 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.803+0000 7f04debbcb80 20 client.27917 trim_cache size 0 max 16384 2023-07-14T11:48:14.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.820+0000 7f04debbcb80 1 Processor -- start 2023-07-14T11:48:14.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.820+0000 7f04debbcb80 1 -- start start 2023-07-14T11:48:14.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939b2d0 0x5598093a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598093a6ad0 0x5598093ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04debbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598092474f0 con 0x55980939bbe0 2023-07-14T11:48:14.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04debbcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559809393ec0 con 0x5598093a6ad0 2023-07-14T11:48:14.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04debbcb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980939e250 con 0x55980939b2d0 2023-07-14T11:48:14.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939b2d0 0x5598093a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939b2d0 0x5598093a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:40442/0 (socket says 172.21.15.161:40442) 2023-07-14T11:48:14.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 -- 172.21.15.161:0/1366820019 learned_addr learned my addr 172.21.15.161:0/1366820019 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:14.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c979c700 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598093a6ad0 0x5598093ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c3fff700 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 -- 172.21.15.161:0/1366820019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598093a6ad0 msgr2=0x5598093ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598093a6ad0 0x5598093ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 -- 172.21.15.161:0/1366820019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 msgr2=0x5598093a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 -- 172.21.15.161:0/1366820019 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598093935e0 con 0x55980939b2d0 2023-07-14T11:48:14.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.821+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939b2d0 0x5598093a5bd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f04b8097480 tx=0x7f04b809a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f65256cf5655304 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:14.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04af7fe700 1 -- 172.21.15.161:0/1366820019 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8099b10 con 0x55980939b2d0 2023-07-14T11:48:14.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04af7fe700 1 -- 172.21.15.161:0/1366820019 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b8099d00 con 0x55980939b2d0 2023-07-14T11:48:14.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04af7fe700 1 -- 172.21.15.161:0/1366820019 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8096480 con 0x55980939b2d0 2023-07-14T11:48:14.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 -- 172.21.15.161:0/1366820019 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939b2d0 msgr2=0x5598093a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939b2d0 0x5598093a5bd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f04b8097480 tx=0x7f04b809a470 comp rx=0 tx=0).stop 2023-07-14T11:48:14.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 -- 172.21.15.161:0/1366820019 shutdown_connections 2023-07-14T11:48:14.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598093a6ad0 0x5598093ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55980939b2d0 0x5598093a5bd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 --2- 172.21.15.161:0/1366820019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598093a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 -- 172.21.15.161:0/1366820019 >> 172.21.15.161:0/1366820019 conn(0x559809391da0 msgr2=0x5598093a4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:14.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 -- 172.21.15.161:0/1366820019 shutdown_connections 2023-07-14T11:48:14.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.822+0000 7f04debbcb80 1 -- 172.21.15.161:0/1366820019 wait complete. 2023-07-14T11:48:14.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.823+0000 7f04debbcb80 1 -- start start 2023-07-14T11:48:14.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.823+0000 7f04debbcb80 1 Processor -- start 2023-07-14T11:48:14.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x559809402b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598094030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04debbcb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x559809401910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.817 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04debbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559809401e80 con 0x55980939bbe0 2023-07-14T11:48:14.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04debbcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559809401ff0 con 0x55980939b2d0 2023-07-14T11:48:14.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04debbcb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598094021b0 con 0x5598093a6ad0 2023-07-14T11:48:14.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04debbcb80 10 client.0 _ll_register_callbacks cb 0x7f04b0009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:48:14.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x559809402b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x559809402b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60528/0 (socket says 172.21.15.161:60528) 2023-07-14T11:48:14.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 -- 172.21.15.161:0/4217608353 learned_addr learned my addr 172.21.15.161:0/4217608353 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:14.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c979c700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x559809401910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c3fff700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598094030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 -- 172.21.15.161:0/4217608353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 msgr2=0x559809401910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x559809401910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 -- 172.21.15.161:0/4217608353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 msgr2=0x5598094030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:14.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598094030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.824+0000 7f04c8f9b700 1 -- 172.21.15.161:0/4217608353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559809104c60 con 0x55980939b2d0 2023-07-14T11:48:14.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04c979c700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x559809401910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:14.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04c8f9b700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x559809402b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f04b809a4e0 tx=0x7f04b809a800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e7496c1b6fe079 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:14.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:48:14.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04a3fff700 1 -- 172.21.15.161:0/4217608353 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b80a2b40 con 0x55980939b2d0 2023-07-14T11:48:14.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04a3fff700 1 -- 172.21.15.161:0/4217608353 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f04b809ea40 con 0x55980939b2d0 2023-07-14T11:48:14.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04b80a2dd0 con 0x55980939b2d0 2023-07-14T11:48:14.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04a3fff700 1 -- 172.21.15.161:0/4217608353 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b8096d50 con 0x55980939b2d0 2023-07-14T11:48:14.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 10 client.27923 fetch_fsmap learned FSMap version 46 2023-07-14T11:48:14.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x559809142250 con 0x55980939b2d0 2023-07-14T11:48:14.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04a3fff700 1 -- 172.21.15.161:0/4217608353 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b80a2dd0 con 0x55980939b2d0 2023-07-14T11:48:14.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04a3fff700 1 -- 172.21.15.161:0/4217608353 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04b8005040 con 0x55980939b2d0 2023-07-14T11:48:14.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 10 client.27923 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:48:14.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 10 client.27923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:48:14.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 20 client.27923 populate_metadata read hostname 'smithi161' 2023-07-14T11:48:14.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55980940a650 0x55980940ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:14.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 4 client.27923 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:48:14.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.825+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598092a9050 con 0x55980940a650 2023-07-14T11:48:14.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04c3fff700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55980940a650 0x55980940ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:14.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04a3fff700 10 client.27923 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:14.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04c3fff700 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55980940a650 0x55980940ca30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38abf1ca5ea3d37 server_cookie=1fbc2e9c783ff0f1 in_seq=0 out_seq=0 2023-07-14T11:48:14.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04a3fff700 1 -- 172.21.15.161:0/4217608353 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5598092a9050 con 0x55980940a650 2023-07-14T11:48:14.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04a3fff700 10 client.27923 handle_command_reply: tid=0 2023-07-14T11:48:14.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04debbcb80 1 client.27923 shutdown 2023-07-14T11:48:14.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 msgr2=0x559809402b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04debbcb80 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x559809402b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f04b809a4e0 tx=0x7f04b809a800 comp rx=0 tx=0).stop 2023-07-14T11:48:14.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 shutdown_connections 2023-07-14T11:48:14.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.826+0000 7f04debbcb80 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55980940a650 0x55980940ca30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04debbcb80 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5598093a6ad0 0x559809401910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04debbcb80 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55980939b2d0 0x559809402b80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04debbcb80 1 --2- 172.21.15.161:0/4217608353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980939bbe0 0x5598094030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04a3fff700 0 client.27923 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 >> 172.21.15.161:0/4217608353 conn(0x559809391da0 msgr2=0x5598093a5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 shutdown_connections 2023-07-14T11:48:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04debbcb80 1 -- 172.21.15.161:0/4217608353 wait complete. 2023-07-14T11:48:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04debbcb80 20 client.27923 trim_cache size 0 max 16384 2023-07-14T11:48:14.827 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:48:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:48:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:48:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:48:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:48:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:48:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:48:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:48:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:48:14.829 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:48:14.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f04b40a1340 msgr2=0x7f04b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f04b40a1340 0x7f04b40a37f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f04bc003e80 tx=0x7f04bc009920 comp rx=0 tx=0).stop 2023-07-14T11:48:14.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c40ffae0 msgr2=0x7f04c4197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:14.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c40ffae0 0x7f04c4197e20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009810 tx=0x7f04b800fa00 comp rx=0 tx=0).stop 2023-07-14T11:48:14.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 shutdown_connections 2023-07-14T11:48:14.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f04b40a1340 0x7f04b40a37f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c40ffae0 0x7f04c4197e20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f04c4105af0 0x7f04c419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 --2- 172.21.15.161:0/768428000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c4100490 0x7f04c4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:14.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.827+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 >> 172.21.15.161:0/768428000 conn(0x7f04c40fb5e0 msgr2=0x7f04c410ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:14.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.828+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 shutdown_connections 2023-07-14T11:48:14.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:14.828+0000 7f04cb1ff700 1 -- 172.21.15.161:0/768428000 wait complete. 2023-07-14T11:48:14.833 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:48:15.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:15 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:48:15.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:15 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:48:15.722 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:15 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:48:16.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:16 smithi161 ceph-mon[95526]: pgmap v823: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:48:16.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:16 smithi175 ceph-mon[129081]: pgmap v823: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:48:16.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:16 smithi182 ceph-mon[129366]: pgmap v823: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:48:18.867 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:18 smithi161 ceph-mon[95526]: pgmap v824: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:48:18.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:18 smithi175 ceph-mon[129081]: pgmap v824: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:48:18.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:18 smithi182 ceph-mon[129366]: pgmap v824: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:48:20.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:20 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:20 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:20 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:48:20.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:20 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:48:20.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:20 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.481 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:20 smithi161 ceph-mon[95526]: pgmap v825: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:48:20.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:20 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:20 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:20 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:48:20.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:20 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:48:20.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:20 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.654 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:20 smithi175 ceph-mon[129081]: pgmap v825: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:48:20.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:20 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:20 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:20 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:48:20.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:20 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:48:20.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:20 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:48:20.699 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:20 smithi182 ceph-mon[129366]: pgmap v825: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:48:22.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:22 smithi175 ceph-mon[129081]: pgmap v826: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:48:22.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:22 smithi182 ceph-mon[129366]: pgmap v826: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:48:22.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:22 smithi161 ceph-mon[95526]: pgmap v826: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:48:24.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:24 smithi161 ceph-mon[95526]: pgmap v827: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:24.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:24 smithi175 ceph-mon[129081]: pgmap v827: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:24.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:24 smithi182 ceph-mon[129366]: pgmap v827: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:26.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:26 smithi175 ceph-mon[129081]: pgmap v828: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:26.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:26 smithi182 ceph-mon[129366]: pgmap v828: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:26.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:26 smithi161 ceph-mon[95526]: pgmap v828: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:28.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:28 smithi175 ceph-mon[129081]: pgmap v829: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:28.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:28 smithi182 ceph-mon[129366]: pgmap v829: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:28.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:28 smithi161 ceph-mon[95526]: pgmap v829: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:30.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:30 smithi175 ceph-mon[129081]: pgmap v830: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:30.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:30 smithi182 ceph-mon[129366]: pgmap v830: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:30.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:30 smithi161 ceph-mon[95526]: pgmap v830: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:32.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:32 smithi175 ceph-mon[129081]: pgmap v831: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:32.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:32 smithi182 ceph-mon[129366]: pgmap v831: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:32.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:32 smithi161 ceph-mon[95526]: pgmap v831: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:34.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:34 smithi161 ceph-mon[95526]: pgmap v832: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:34.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:34 smithi175 ceph-mon[129081]: pgmap v832: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:34.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:34 smithi182 ceph-mon[129366]: pgmap v832: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:36.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:36 smithi175 ceph-mon[129081]: pgmap v833: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:36.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:36 smithi182 ceph-mon[129366]: pgmap v833: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:36.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:36 smithi161 ceph-mon[95526]: pgmap v833: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:38.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:38 smithi175 ceph-mon[129081]: pgmap v834: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:38.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:38 smithi182 ceph-mon[129366]: pgmap v834: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:38.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:38 smithi161 ceph-mon[95526]: pgmap v834: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:40.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:40 smithi175 ceph-mon[129081]: pgmap v835: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:40.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:40 smithi182 ceph-mon[129366]: pgmap v835: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:40.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:40 smithi161 ceph-mon[95526]: pgmap v835: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:42.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:42 smithi175 ceph-mon[129081]: pgmap v836: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:42.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:42 smithi182 ceph-mon[129366]: pgmap v836: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:42.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:42 smithi161 ceph-mon[95526]: pgmap v836: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:44.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:44 smithi161 ceph-mon[95526]: pgmap v837: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:44.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:44 smithi175 ceph-mon[129081]: pgmap v837: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:44.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:44 smithi182 ceph-mon[129366]: pgmap v837: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:44.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:48:45.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.066+0000 7fed7f9ff700 1 -- 172.21.15.161:0/865262554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed78105af0 msgr2=0x7fed78105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.066+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/865262554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed78105af0 0x7fed78105ec0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fed740098f0 tx=0x7fed74009c00 comp rx=0 tx=0).stop 2023-07-14T11:48:45.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.067+0000 7fed7f9ff700 1 -- 172.21.15.161:0/865262554 shutdown_connections 2023-07-14T11:48:45.055 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.067+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/865262554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed78105af0 0x7fed78105ec0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.067+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/865262554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed78100490 0x7fed7810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.067+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/865262554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed780ffae0 0x7fed780fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.067+0000 7fed7f9ff700 1 -- 172.21.15.161:0/865262554 >> 172.21.15.161:0/865262554 conn(0x7fed780fb5e0 msgr2=0x7fed780fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:45.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.067+0000 7fed7f9ff700 1 -- 172.21.15.161:0/865262554 shutdown_connections 2023-07-14T11:48:45.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.067+0000 7fed7f9ff700 1 -- 172.21.15.161:0/865262554 wait complete. 2023-07-14T11:48:45.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 Processor -- start 2023-07-14T11:48:45.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 -- start start 2023-07-14T11:48:45.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed780ffae0 0x7fed78072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed78100490 0x7fed78072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed78105af0 0x7fed781a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed78073580 con 0x7fed780ffae0 2023-07-14T11:48:45.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a4380 con 0x7fed78100490 2023-07-14T11:48:45.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.068+0000 7fed7f9ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a44c0 con 0x7fed78105af0 2023-07-14T11:48:45.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7cf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed78100490 0x7fed78072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed780ffae0 0x7fed78072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7cf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed78100490 0x7fed78072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60960/0 (socket says 172.21.15.161:60960) 2023-07-14T11:48:45.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7df9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed78105af0 0x7fed781a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7cf9a700 1 -- 172.21.15.161:0/2357978799 learned_addr learned my addr 172.21.15.161:0/2357978799 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:45.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7d79b700 1 -- 172.21.15.161:0/2357978799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed78105af0 msgr2=0x7fed781a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed78105af0 0x7fed781a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7d79b700 1 -- 172.21.15.161:0/2357978799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed78100490 msgr2=0x7fed78072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed78100490 0x7fed78072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7d79b700 1 -- 172.21.15.161:0/2357978799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed74009520 con 0x7fed780ffae0 2023-07-14T11:48:45.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed78100490 0x7fed78072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:45.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7df9c700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed78105af0 0x7fed781a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:45.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed780ffae0 0x7fed78072810 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fed74009810 tx=0x7fed7400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:45.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed6e7fc700 1 -- 172.21.15.161:0/2357978799 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7401c070 con 0x7fed780ffae0 2023-07-14T11:48:45.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed781a46c0 con 0x7fed780ffae0 2023-07-14T11:48:45.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.069+0000 7fed6e7fc700 1 -- 172.21.15.161:0/2357978799 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7400bba0 con 0x7fed780ffae0 2023-07-14T11:48:45.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.070+0000 7fed6e7fc700 1 -- 172.21.15.161:0/2357978799 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74003970 con 0x7fed780ffae0 2023-07-14T11:48:45.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.070+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed781a4b40 con 0x7fed780ffae0 2023-07-14T11:48:45.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.073+0000 7fed6e7fc700 1 -- 172.21.15.161:0/2357978799 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fed74020030 con 0x7fed780ffae0 2023-07-14T11:48:45.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.074+0000 7fed932f9b80 1 Processor -- start 2023-07-14T11:48:45.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.074+0000 7fed6e7fc700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fed640a12c0 0x7fed640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.074+0000 7fed932f9b80 1 -- start start 2023-07-14T11:48:45.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.074+0000 7fed6e7fc700 1 -- 172.21.15.161:0/2357978799 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fed740cdea0 con 0x7fed780ffae0 2023-07-14T11:48:45.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.074+0000 7fed7df9c700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fed640a12c0 0x7fed640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113f2d0 0x55bbc113f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc113fbe0 0x55bbc114a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc114aad0 0x55bbc114ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed932f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc1045cb0 con 0x55bbc113fbe0 2023-07-14T11:48:45.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed932f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc1098a40 con 0x55bbc114aad0 2023-07-14T11:48:45.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed932f9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc1091ed0 con 0x55bbc113f2d0 2023-07-14T11:48:45.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113f2d0 0x55bbc113f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113f2d0 0x55bbc113f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36096/0 (socket says 172.21.15.161:36096) 2023-07-14T11:48:45.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 -- 172.21.15.161:0/1057890060 learned_addr learned my addr 172.21.15.161:0/1057890060 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:45.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7df9c700 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc114aad0 0x55bbc114ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc113fbe0 0x55bbc114a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 -- 172.21.15.161:0/1057890060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc114aad0 msgr2=0x55bbc114ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc114aad0 0x55bbc114ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 -- 172.21.15.161:0/1057890060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc113fbe0 msgr2=0x55bbc114a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc113fbe0 0x55bbc114a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 -- 172.21.15.161:0/1057890060 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbc0ebfde0 con 0x55bbc113f2d0 2023-07-14T11:48:45.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7df9c700 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc114aad0 0x55bbc114ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:48:45.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7df9c700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fed640a12c0 0x7fed640a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed78074920 tx=0x7fed7000b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:45.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed7d79b700 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113f2d0 0x55bbc113f6a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fed740976b0 tx=0x7fed74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458b2b84d180afbb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:45.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.075+0000 7fed637fe700 1 -- 172.21.15.161:0/1057890060 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7409ec70 con 0x55bbc113f2d0 2023-07-14T11:48:45.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed637fe700 1 -- 172.21.15.161:0/1057890060 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7409e030 con 0x55bbc113f2d0 2023-07-14T11:48:45.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed637fe700 1 -- 172.21.15.161:0/1057890060 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed740cb4b0 con 0x55bbc113f2d0 2023-07-14T11:48:45.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 -- 172.21.15.161:0/1057890060 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113f2d0 msgr2=0x55bbc113f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113f2d0 0x55bbc113f6a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fed740976b0 tx=0x7fed74009c00 comp rx=0 tx=0).stop 2023-07-14T11:48:45.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 -- 172.21.15.161:0/1057890060 shutdown_connections 2023-07-14T11:48:45.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc114aad0 0x55bbc114ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113f2d0 0x55bbc113f6a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 --2- 172.21.15.161:0/1057890060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc113fbe0 0x55bbc114a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 -- 172.21.15.161:0/1057890060 >> 172.21.15.161:0/1057890060 conn(0x55bbc1135da0 msgr2=0x55bbc113b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:45.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 -- 172.21.15.161:0/1057890060 shutdown_connections 2023-07-14T11:48:45.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.076+0000 7fed932f9b80 1 -- 172.21.15.161:0/1057890060 wait complete. 2023-07-14T11:48:45.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.077+0000 7fed932f9b80 1 -- start start 2023-07-14T11:48:45.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.077+0000 7fed932f9b80 1 Processor -- start 2023-07-14T11:48:45.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1149c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc114aad0 0x55bbc114e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed932f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc1031ac0 con 0x55bbc114aad0 2023-07-14T11:48:45.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed932f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc10317c0 con 0x55bbc113f2d0 2023-07-14T11:48:45.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed932f9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc0f15920 con 0x55bbc113fbe0 2023-07-14T11:48:45.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed932f9b80 10 client.0 _ll_register_callbacks cb 0x55bbc102f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:48:45.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7cf9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7cf9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36106/0 (socket says 172.21.15.161:36106) 2023-07-14T11:48:45.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:60988/0 (socket says 172.21.15.161:60988) 2023-07-14T11:48:45.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7cf9a700 1 -- 172.21.15.161:0/2789888782 learned_addr learned my addr 172.21.15.161:0/2789888782 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:45.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7df9c700 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc114aad0 0x55bbc114e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 -- 172.21.15.161:0/2789888782 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 msgr2=0x55bbc114a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 -- 172.21.15.161:0/2789888782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc114aad0 msgr2=0x55bbc114e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc114aad0 0x55bbc114e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 -- 172.21.15.161:0/2789888782 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed781a4b40 con 0x55bbc113f2d0 2023-07-14T11:48:45.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:45.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.078+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1149c00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fed740a2ad0 tx=0x7fed74097010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=222bb4fc3e3feb31 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:45.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:48:45.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed60ff9700 1 -- 172.21.15.161:0/2789888782 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7409ec70 con 0x55bbc113f2d0 2023-07-14T11:48:45.084 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed60ff9700 1 -- 172.21.15.161:0/2789888782 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fed74097350 con 0x55bbc113f2d0 2023-07-14T11:48:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbc0feb4f0 con 0x55bbc113f2d0 2023-07-14T11:48:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed60ff9700 1 -- 172.21.15.161:0/2789888782 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed74097510 con 0x55bbc113f2d0 2023-07-14T11:48:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 10 client.27935 fetch_fsmap learned FSMap version 46 2023-07-14T11:48:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55bbc0ea8c60 con 0x55bbc113f2d0 2023-07-14T11:48:45.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed60ff9700 1 -- 172.21.15.161:0/2789888782 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed74064020 con 0x55bbc113f2d0 2023-07-14T11:48:45.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed60ff9700 1 -- 172.21.15.161:0/2789888782 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed7409aea0 con 0x55bbc113f2d0 2023-07-14T11:48:45.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 10 client.27935 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:48:45.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 10 client.27935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:48:45.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 20 client.27935 populate_metadata read hostname 'smithi161' 2023-07-14T11:48:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc1159c10 0x55bbc115c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 4 client.27935 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:48:45.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.079+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbc108aeb0 con 0x55bbc1159c10 2023-07-14T11:48:45.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.080+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc1159c10 0x55bbc115c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.080+0000 7fed60ff9700 10 client.27935 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:45.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.080+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc1159c10 0x55bbc115c020 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=632dd871210411a4 server_cookie=600aabed3911a444 in_seq=0 out_seq=0 2023-07-14T11:48:45.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed60ff9700 1 -- 172.21.15.161:0/2789888782 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bbc108aeb0 con 0x55bbc1159c10 2023-07-14T11:48:45.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed60ff9700 10 client.27935 handle_command_reply: tid=0 2023-07-14T11:48:45.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 client.27935 shutdown 2023-07-14T11:48:45.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 msgr2=0x55bbc1149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1149c00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fed740a2ad0 tx=0x7fed74097010 comp rx=0 tx=0).stop 2023-07-14T11:48:45.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 shutdown_connections 2023-07-14T11:48:45.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc1159c10 0x55bbc115c020 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc114aad0 0x55bbc114e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1149c00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2789888782 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed60ff9700 0 client.27935 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:45.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 >> 172.21.15.161:0/2789888782 conn(0x55bbc1135da0 msgr2=0x55bbc113b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:45.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 shutdown_connections 2023-07-14T11:48:45.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.081+0000 7fed932f9b80 1 -- 172.21.15.161:0/2789888782 wait complete. 2023-07-14T11:48:45.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.082+0000 7fed932f9b80 20 client.27935 trim_cache size 0 max 16384 2023-07-14T11:48:45.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.098+0000 7fed932f9b80 1 Processor -- start 2023-07-14T11:48:45.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed932f9b80 1 -- start start 2023-07-14T11:48:45.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc113b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1144400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed932f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc1140e30 con 0x55bbc1135da0 2023-07-14T11:48:45.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed932f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc114a940 con 0x55bbc113f2d0 2023-07-14T11:48:45.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed932f9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc113c330 con 0x55bbc113fbe0 2023-07-14T11:48:45.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc113b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed7df9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed7cf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc113b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46768/0 (socket says 172.21.15.161:46768) 2023-07-14T11:48:45.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed7df9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36120/0 (socket says 172.21.15.161:36120) 2023-07-14T11:48:45.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.099+0000 7fed7d79b700 1 -- 172.21.15.161:0/3173523173 learned_addr learned my addr 172.21.15.161:0/3173523173 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:45.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed7df9c700 1 -- 172.21.15.161:0/3173523173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 msgr2=0x55bbc1144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed7df9c700 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed7df9c700 1 -- 172.21.15.161:0/3173523173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 msgr2=0x55bbc113b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed7df9c700 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc113b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed7df9c700 1 -- 172.21.15.161:0/3173523173 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbc11375e0 con 0x55bbc113fbe0 2023-07-14T11:48:45.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed7d79b700 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc113b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:45.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed7df9c700 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114de10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fed700046b0 tx=0x7fed7001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac847f83abc683d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:45.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed63fff700 1 -- 172.21.15.161:0/3173523173 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7002b660 con 0x55bbc113fbe0 2023-07-14T11:48:45.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed63fff700 1 -- 172.21.15.161:0/3173523173 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7000f040 con 0x55bbc113fbe0 2023-07-14T11:48:45.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.100+0000 7fed63fff700 1 -- 172.21.15.161:0/3173523173 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70017bd0 con 0x55bbc113fbe0 2023-07-14T11:48:45.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 -- 172.21.15.161:0/3173523173 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 msgr2=0x55bbc114de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114de10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fed700046b0 tx=0x7fed7001c000 comp rx=0 tx=0).stop 2023-07-14T11:48:45.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 -- 172.21.15.161:0/3173523173 shutdown_connections 2023-07-14T11:48:45.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc113b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc114de10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 --2- 172.21.15.161:0/3173523173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 -- 172.21.15.161:0/3173523173 >> 172.21.15.161:0/3173523173 conn(0x55bbc113a170 msgr2=0x55bbc114d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:45.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 -- 172.21.15.161:0/3173523173 shutdown_connections 2023-07-14T11:48:45.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 -- 172.21.15.161:0/3173523173 wait complete. 2023-07-14T11:48:45.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 -- start start 2023-07-14T11:48:45.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.101+0000 7fed932f9b80 1 Processor -- start 2023-07-14T11:48:45.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc1156640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1154970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed932f9b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc1154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed932f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc11556d0 con 0x55bbc1135da0 2023-07-14T11:48:45.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed932f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc1155840 con 0x55bbc113f2d0 2023-07-14T11:48:45.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed932f9b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc11adb20 con 0x55bbc113fbe0 2023-07-14T11:48:45.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed932f9b80 10 client.0 _ll_register_callbacks cb 0x7fed7409aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:48:45.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed7cf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed7d79b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc1156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed7df9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc1154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed7cf9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:32774/0 (socket says 172.21.15.161:32774) 2023-07-14T11:48:45.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed7cf9a700 1 -- 172.21.15.161:0/2875119649 learned_addr learned my addr 172.21.15.161:0/2875119649 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:48:45.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed7cf9a700 1 -- 172.21.15.161:0/2875119649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 msgr2=0x55bbc1154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.102+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc1154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed7cf9a700 1 -- 172.21.15.161:0/2875119649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 msgr2=0x55bbc1156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:48:45.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc1156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed7cf9a700 1 -- 172.21.15.161:0/2875119649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbc0ea8c60 con 0x55bbc113f2d0 2023-07-14T11:48:45.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed7df9c700 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc1154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:45.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc1156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:48:45.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed7cf9a700 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1154970 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fed68004790 tx=0x7fed68013f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3733278ca2071008 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:48:45.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:48:45.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed60ff9700 1 -- 172.21.15.161:0/2875119649 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68017070 con 0x55bbc113f2d0 2023-07-14T11:48:45.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed60ff9700 1 -- 172.21.15.161:0/2875119649 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fed68021c50 con 0x55bbc113f2d0 2023-07-14T11:48:45.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed7409e030 con 0x55bbc113f2d0 2023-07-14T11:48:45.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed60ff9700 1 -- 172.21.15.161:0/2875119649 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed680045a0 con 0x55bbc113f2d0 2023-07-14T11:48:45.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed932f9b80 10 client.27941 fetch_fsmap learned FSMap version 46 2023-07-14T11:48:45.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.103+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fed700048c0 con 0x55bbc113f2d0 2023-07-14T11:48:45.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed60ff9700 1 -- 172.21.15.161:0/2875119649 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7409e030 con 0x55bbc113f2d0 2023-07-14T11:48:45.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed60ff9700 1 -- 172.21.15.161:0/2875119649 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed68003820 con 0x55bbc113f2d0 2023-07-14T11:48:45.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed932f9b80 10 client.27941 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:48:45.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed932f9b80 10 client.27941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:48:45.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed932f9b80 20 client.27941 populate_metadata read hostname 'smithi161' 2023-07-14T11:48:45.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc114aad0 0x55bbc11b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:48:45.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed932f9b80 4 client.27941 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:48:45.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bbc104d050 con 0x55bbc114aad0 2023-07-14T11:48:45.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.104+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc114aad0 0x55bbc11b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:48:45.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed7d79b700 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc114aad0 0x55bbc11b1370 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9081a61e1db3570 server_cookie=5556f9e7ad075333 in_seq=0 out_seq=0 2023-07-14T11:48:45.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed60ff9700 10 client.27941 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:45.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed60ff9700 1 -- 172.21.15.161:0/2875119649 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bbc104d050 con 0x55bbc114aad0 2023-07-14T11:48:45.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed60ff9700 10 client.27941 handle_command_reply: tid=0 2023-07-14T11:48:45.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 client.27941 shutdown 2023-07-14T11:48:45.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 msgr2=0x55bbc1154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1154970 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fed68004790 tx=0x7fed68013f20 comp rx=0 tx=0).stop 2023-07-14T11:48:45.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 shutdown_connections 2023-07-14T11:48:45.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55bbc114aad0 0x55bbc11b1370 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbc1135da0 0x55bbc1156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55bbc113fbe0 0x55bbc1154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed60ff9700 0 client.27941 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:48:45.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 --2- 172.21.15.161:0/2875119649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bbc113f2d0 0x55bbc1154970 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 >> 172.21.15.161:0/2875119649 conn(0x55bbc113a170 msgr2=0x55bbc1143c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:45.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 shutdown_connections 2023-07-14T11:48:45.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 1 -- 172.21.15.161:0/2875119649 wait complete. 2023-07-14T11:48:45.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.105+0000 7fed932f9b80 20 client.27941 trim_cache size 0 max 16384 2023-07-14T11:48:45.116 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:48:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:48:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:48:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:48:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:48:45.118 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:48:45.118 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:48:45.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:48:45.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:48:45.118 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:48:45.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fed640a12c0 msgr2=0x7fed640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fed640a12c0 0x7fed640a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed78074920 tx=0x7fed7000b040 comp rx=0 tx=0).stop 2023-07-14T11:48:45.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed780ffae0 msgr2=0x7fed78072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:48:45.120 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed780ffae0 0x7fed78072810 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fed74009810 tx=0x7fed7400fa00 comp rx=0 tx=0).stop 2023-07-14T11:48:45.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 shutdown_connections 2023-07-14T11:48:45.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fed640a12c0 0x7fed640a3770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed780ffae0 0x7fed78072810 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fed78105af0 0x7fed781a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.121 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 --2- 172.21.15.161:0/2357978799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed78100490 0x7fed78072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:48:45.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 >> 172.21.15.161:0/2357978799 conn(0x7fed780fb5e0 msgr2=0x7fed780fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:48:45.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 shutdown_connections 2023-07-14T11:48:45.122 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:48:45.106+0000 7fed7f9ff700 1 -- 172.21.15.161:0/2357978799 wait complete. 2023-07-14T11:48:45.122 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:48:46.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:46 smithi175 ceph-mon[129081]: pgmap v838: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:46.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:46 smithi182 ceph-mon[129366]: pgmap v838: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:46.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:46 smithi161 ceph-mon[95526]: pgmap v838: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:48.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:48 smithi175 ceph-mon[129081]: pgmap v839: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:48.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:48 smithi182 ceph-mon[129366]: pgmap v839: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:48.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:48 smithi161 ceph-mon[95526]: pgmap v839: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:50.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:50 smithi175 ceph-mon[129081]: pgmap v840: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:50.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:50 smithi182 ceph-mon[129366]: pgmap v840: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:50.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:50 smithi161 ceph-mon[95526]: pgmap v840: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:52.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:52 smithi175 ceph-mon[129081]: pgmap v841: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:52.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:52 smithi182 ceph-mon[129366]: pgmap v841: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:52.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:52 smithi161 ceph-mon[95526]: pgmap v841: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:48:54.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:54 smithi161 ceph-mon[95526]: pgmap v842: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:54.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:54 smithi175 ceph-mon[129081]: pgmap v842: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:54.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:54 smithi182 ceph-mon[129366]: pgmap v842: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:48:56.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:56 smithi175 ceph-mon[129081]: pgmap v843: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:56.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:56 smithi182 ceph-mon[129366]: pgmap v843: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:56.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:56 smithi161 ceph-mon[95526]: pgmap v843: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:48:58.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:48:58 smithi175 ceph-mon[129081]: pgmap v844: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:58.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:48:58 smithi182 ceph-mon[129366]: pgmap v844: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:48:58.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:48:58 smithi161 ceph-mon[95526]: pgmap v844: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:00.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:00 smithi175 ceph-mon[129081]: pgmap v845: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:00.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:00 smithi182 ceph-mon[129366]: pgmap v845: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:00.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:00 smithi161 ceph-mon[95526]: pgmap v845: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:02.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:02 smithi175 ceph-mon[129081]: pgmap v846: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:49:02.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:02 smithi182 ceph-mon[129366]: pgmap v846: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:49:02.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:02 smithi161 ceph-mon[95526]: pgmap v846: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:49:04.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:04 smithi161 ceph-mon[95526]: pgmap v847: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:49:04.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:04 smithi175 ceph-mon[129081]: pgmap v847: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:49:04.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:04 smithi182 ceph-mon[129366]: pgmap v847: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:49:06.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:06 smithi175 ceph-mon[129081]: pgmap v848: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:49:06.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:06 smithi182 ceph-mon[129366]: pgmap v848: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:49:06.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:06 smithi161 ceph-mon[95526]: pgmap v848: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:49:07.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:07 smithi175 ceph-mon[129081]: osdmap e134: 12 total, 12 up, 12 in 2023-07-14T11:49:07.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:07 smithi182 ceph-mon[129366]: osdmap e134: 12 total, 12 up, 12 in 2023-07-14T11:49:07.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:07 smithi161 ceph-mon[95526]: osdmap e134: 12 total, 12 up, 12 in 2023-07-14T11:49:08.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:08 smithi175 ceph-mon[129081]: pgmap v850: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:49:08.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:08 smithi182 ceph-mon[129366]: pgmap v850: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:49:08.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:08 smithi161 ceph-mon[95526]: pgmap v850: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:49:10.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:10 smithi175 ceph-mon[129081]: pgmap v851: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:49:10.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:10 smithi182 ceph-mon[129366]: pgmap v851: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:49:10.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:10 smithi161 ceph-mon[95526]: pgmap v851: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:49:12.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:12 smithi175 ceph-mon[129081]: pgmap v852: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:49:12.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:12 smithi182 ceph-mon[129366]: pgmap v852: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:49:12.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:12 smithi161 ceph-mon[95526]: pgmap v852: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:49:14.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:14 smithi161 ceph-mon[95526]: osdmap e135: 12 total, 12 up, 12 in 2023-07-14T11:49:14.480 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:14 smithi161 ceph-mon[95526]: pgmap v854: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:49:14.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:14 smithi175 ceph-mon[129081]: osdmap e135: 12 total, 12 up, 12 in 2023-07-14T11:49:14.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:14 smithi175 ceph-mon[129081]: pgmap v854: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:49:14.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:14 smithi182 ceph-mon[129366]: osdmap e135: 12 total, 12 up, 12 in 2023-07-14T11:49:14.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:14 smithi182 ceph-mon[129366]: pgmap v854: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:49:15.124 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:49:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 -- 172.21.15.161:0/650209531 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe134105af0 msgr2=0x7fe134105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/650209531 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe134105af0 0x7fe134105ec0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe130009990 tx=0x7fe130009ca0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 -- 172.21.15.161:0/650209531 shutdown_connections 2023-07-14T11:49:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/650209531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe134100490 0x7fe13410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/650209531 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe134105af0 0x7fe134105ec0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/650209531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1340ffae0 0x7fe1340fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 -- 172.21.15.161:0/650209531 >> 172.21.15.161:0/650209531 conn(0x7fe1340fb5e0 msgr2=0x7fe1340fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:15.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 -- 172.21.15.161:0/650209531 shutdown_connections 2023-07-14T11:49:15.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.353+0000 7fe13b7ff700 1 -- 172.21.15.161:0/650209531 wait complete. 2023-07-14T11:49:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.354+0000 7fe13b7ff700 1 Processor -- start 2023-07-14T11:49:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.354+0000 7fe13b7ff700 1 -- start start 2023-07-14T11:49:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.354+0000 7fe13b7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe1340ffae0 0x7fe134197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.354+0000 7fe13b7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe134100490 0x7fe134198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13b7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe134105af0 0x7fe13419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13b7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13419ce20 con 0x7fe134105af0 2023-07-14T11:49:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13b7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13419cf90 con 0x7fe134100490 2023-07-14T11:49:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13b7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13419d150 con 0x7fe1340ffae0 2023-07-14T11:49:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe1340ffae0 0x7fe134197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe1340ffae0 0x7fe134197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59112/0 (socket says 172.21.15.161:59112) 2023-07-14T11:49:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 -- 172.21.15.161:0/3408189558 learned_addr learned my addr 172.21.15.161:0/3408189558 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe138d9a700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe134100490 0x7fe134198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe139d9c700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe134105af0 0x7fe13419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 -- 172.21.15.161:0/3408189558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe134100490 msgr2=0x7fe134198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe134100490 0x7fe134198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 -- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe134105af0 msgr2=0x7fe13419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe134105af0 0x7fe13419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 -- 172.21.15.161:0/3408189558 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1300095f0 con 0x7fe1340ffae0 2023-07-14T11:49:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.355+0000 7fe13959b700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe1340ffae0 0x7fe134197e40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe130005910 tx=0x7fe130004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.356+0000 7fe12a7fc700 1 -- 172.21.15.161:0/3408189558 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13001d070 con 0x7fe1340ffae0 2023-07-14T11:49:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.356+0000 7fe12a7fc700 1 -- 172.21.15.161:0/3408189558 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe130022470 con 0x7fe1340ffae0 2023-07-14T11:49:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.356+0000 7fe12a7fc700 1 -- 172.21.15.161:0/3408189558 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130017650 con 0x7fe1340ffae0 2023-07-14T11:49:15.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.356+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13419d3d0 con 0x7fe1340ffae0 2023-07-14T11:49:15.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.356+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13419d650 con 0x7fe1340ffae0 2023-07-14T11:49:15.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.359+0000 7fe12a7fc700 1 -- 172.21.15.161:0/3408189558 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe1300225e0 con 0x7fe1340ffae0 2023-07-14T11:49:15.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.360+0000 7fe14f16eb80 1 Processor -- start 2023-07-14T11:49:15.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.360+0000 7fe14f16eb80 1 -- start start 2023-07-14T11:49:15.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.360+0000 7fe12a7fc700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe1200a9b50 0x7fe1200ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe12a7fc700 1 -- 172.21.15.161:0/3408189558 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fe1300ce0f0 con 0x7fe1340ffae0 2023-07-14T11:49:15.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5c2d0 0x55b69ff5c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff67590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff69eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69fe62cb0 con 0x55b69ff67ad0 2023-07-14T11:49:15.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69feb5a40 con 0x55b69ff5cbe0 2023-07-14T11:49:15.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69feaeed0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe139d9c700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe1200a9b50 0x7fe1200ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5c2d0 0x55b69ff5c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe138d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff67590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5c2d0 0x55b69ff5c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59124/0 (socket says 172.21.15.161:59124) 2023-07-14T11:49:15.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 -- 172.21.15.161:0/3825329824 learned_addr learned my addr 172.21.15.161:0/3825329824 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:15.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe139d9c700 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff69eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 -- 172.21.15.161:0/3825329824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 msgr2=0x55b69ff67590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.367 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff67590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 -- 172.21.15.161:0/3825329824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 msgr2=0x55b69ff69eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff69eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe13959b700 1 -- 172.21.15.161:0/3825329824 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b69fcdcde0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.361+0000 7fe139d9c700 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff69eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:49:15.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.362+0000 7fe13959b700 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5c2d0 0x55b69ff5c6a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe13009ae20 tx=0x7fe13009aa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff872df009993144 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:15.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.362+0000 7fe139d9c700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe1200a9b50 0x7fe1200ac000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00f6e0 tx=0x7fe12c009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:15.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.362+0000 7fe11f7fe700 1 -- 172.21.15.161:0/3825329824 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13009eec0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.362+0000 7fe11f7fe700 1 -- 172.21.15.161:0/3825329824 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1300a3250 con 0x55b69ff5c2d0 2023-07-14T11:49:15.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.362+0000 7fe11f7fe700 1 -- 172.21.15.161:0/3825329824 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13009a5e0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.362+0000 7fe14f16eb80 1 -- 172.21.15.161:0/3825329824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5c2d0 msgr2=0x55b69ff5c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5c2d0 0x55b69ff5c6a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe13009ae20 tx=0x7fe13009aa50 comp rx=0 tx=0).stop 2023-07-14T11:49:15.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 -- 172.21.15.161:0/3825329824 shutdown_connections 2023-07-14T11:49:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff69eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5c2d0 0x55b69ff5c6a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/3825329824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff67590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 -- 172.21.15.161:0/3825329824 >> 172.21.15.161:0/3825329824 conn(0x55b69ff52da0 msgr2=0x55b69ff586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:15.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 -- 172.21.15.161:0/3825329824 shutdown_connections 2023-07-14T11:49:15.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 -- 172.21.15.161:0/3825329824 wait complete. 2023-07-14T11:49:15.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 -- start start 2023-07-14T11:49:15.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.363+0000 7fe14f16eb80 1 Processor -- start 2023-07-14T11:49:15.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5c2d0 0x55b69ff67ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff68920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69fe4eac0 con 0x55b69ff67ad0 2023-07-14T11:49:15.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69fe4e7c0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69fd32920 con 0x55b69ff5cbe0 2023-07-14T11:49:15.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe14f16eb80 10 client.0 _ll_register_callbacks cb 0x55b69fe4c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:49:15.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff683e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59130/0 (socket says 172.21.15.161:59130) 2023-07-14T11:49:15.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 -- 172.21.15.161:0/1930531511 learned_addr learned my addr 172.21.15.161:0/1930531511 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:15.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe13959b700 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5c2d0 0x55b69ff67ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe139d9c700 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff68920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 -- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5c2d0 msgr2=0x55b69ff67ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5c2d0 0x55b69ff67ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 -- 172.21.15.161:0/1930531511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 msgr2=0x55b69ff68920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff68920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.364+0000 7fe138d9a700 1 -- 172.21.15.161:0/1930531511 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13419d3d0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe138d9a700 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff683e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe1240127f0 tx=0x7fe124012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c919ae75c5b5ed3b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:15.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe11cff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:49:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe11cff9700 1 -- 172.21.15.161:0/1930531511 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124017070 con 0x55b69ff5cbe0 2023-07-14T11:49:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe11cff9700 1 -- 172.21.15.161:0/1930531511 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fe12400f5c0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b69fe084f0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe11cff9700 1 -- 172.21.15.161:0/1930531511 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe12400f780 con 0x55b69ff5cbe0 2023-07-14T11:49:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe14f16eb80 10 client.27997 fetch_fsmap learned FSMap version 46 2023-07-14T11:49:15.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55b69fcc5c60 con 0x55b69ff5cbe0 2023-07-14T11:49:15.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.365+0000 7fe11cff9700 1 -- 172.21.15.161:0/1930531511 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b69fe084f0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe11cff9700 1 -- 172.21.15.161:0/1930531511 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe1240108c0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe14f16eb80 10 client.27997 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:49:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe14f16eb80 10 client.27997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:49:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe14f16eb80 20 client.27997 populate_metadata read hostname 'smithi161' 2023-07-14T11:49:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ff77130 0x55b69ff79540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe14f16eb80 4 client.27997 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:49:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b69fea7eb0 con 0x55b69ff77130 2023-07-14T11:49:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe13959b700 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ff77130 0x55b69ff79540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe11cff9700 10 client.27997 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.366+0000 7fe13959b700 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ff77130 0x55b69ff79540 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7276a5524ce6bf01 server_cookie=ad64c05bad912bff in_seq=0 out_seq=0 2023-07-14T11:49:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe11cff9700 1 -- 172.21.15.161:0/1930531511 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b69fea7eb0 con 0x55b69ff77130 2023-07-14T11:49:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe11cff9700 10 client.27997 handle_command_reply: tid=0 2023-07-14T11:49:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 client.27997 shutdown 2023-07-14T11:49:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 msgr2=0x55b69ff683e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff683e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe1240127f0 tx=0x7fe124012b00 comp rx=0 tx=0).stop 2023-07-14T11:49:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 shutdown_connections 2023-07-14T11:49:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ff77130 0x55b69ff79540 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff67ad0 0x55b69ff68920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe11cff9700 0 client.27997 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5c2d0 0x55b69ff67ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1930531511 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff683e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 >> 172.21.15.161:0/1930531511 conn(0x55b69ff52da0 msgr2=0x55b69ff58290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:15.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 shutdown_connections 2023-07-14T11:49:15.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.367+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1930531511 wait complete. 2023-07-14T11:49:15.392 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.368+0000 7fe14f16eb80 20 client.27997 trim_cache size 0 max 16384 2023-07-14T11:49:15.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 Processor -- start 2023-07-14T11:49:15.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 -- start start 2023-07-14T11:49:15.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff60460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff67ad0 0x55b69ff609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69fea7eb0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69fe084f0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69ff529c0 con 0x55b69ff67ad0 2023-07-14T11:49:15.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe13959b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe139d9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff67ad0 0x55b69ff609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe138d9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff60460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe13959b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff662f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47010/0 (socket says 172.21.15.161:47010) 2023-07-14T11:49:15.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.386+0000 7fe13959b700 1 -- 172.21.15.161:0/91782378 learned_addr learned my addr 172.21.15.161:0/91782378 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:15.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe13959b700 1 -- 172.21.15.161:0/91782378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff67ad0 msgr2=0x55b69ff609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe13959b700 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff67ad0 0x55b69ff609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.400 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe13959b700 1 -- 172.21.15.161:0/91782378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 msgr2=0x55b69ff60460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe13959b700 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff60460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe13959b700 1 -- 172.21.15.161:0/91782378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b69ff545e0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe138d9a700 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff60460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:49:15.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe13959b700 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff662f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe1300991e0 tx=0x7fe130009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be458f991899df3c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:15.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe11ffff700 1 -- 172.21.15.161:0/91782378 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130022900 con 0x55b69ff5c2d0 2023-07-14T11:49:15.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe11ffff700 1 -- 172.21.15.161:0/91782378 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe13009a380 con 0x55b69ff5c2d0 2023-07-14T11:49:15.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.387+0000 7fe11ffff700 1 -- 172.21.15.161:0/91782378 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300cbbd0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 -- 172.21.15.161:0/91782378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 msgr2=0x55b69ff662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff662f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe1300991e0 tx=0x7fe130009ca0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 -- 172.21.15.161:0/91782378 shutdown_connections 2023-07-14T11:49:15.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff662f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff67ad0 0x55b69ff609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/91782378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff5cbe0 0x55b69ff60460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 -- 172.21.15.161:0/91782378 >> 172.21.15.161:0/91782378 conn(0x55b69ff52da0 msgr2=0x55b69ff61ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:15.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 -- 172.21.15.161:0/91782378 shutdown_connections 2023-07-14T11:49:15.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 -- 172.21.15.161:0/91782378 wait complete. 2023-07-14T11:49:15.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 -- start start 2023-07-14T11:49:15.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.388+0000 7fe14f16eb80 1 Processor -- start 2023-07-14T11:49:15.405 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff6b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff6bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe14f16eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff67ad0 0x55b69ff69dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69ff6a3a0 con 0x55b69ff5c2d0 2023-07-14T11:49:15.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69ff6a510 con 0x55b69ff67ad0 2023-07-14T11:49:15.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe14f16eb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69ff6a6d0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe14f16eb80 10 client.0 _ll_register_callbacks cb 0x7fe124009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:49:15.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe138d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff6bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe13959b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff6b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe139d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff67ad0 0x55b69ff69dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe138d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff6bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:59138/0 (socket says 172.21.15.161:59138) 2023-07-14T11:49:15.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.389+0000 7fe138d9a700 1 -- 172.21.15.161:0/1170187483 learned_addr learned my addr 172.21.15.161:0/1170187483 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:15.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe138d9a700 1 -- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff67ad0 msgr2=0x55b69ff69dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe138d9a700 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff67ad0 0x55b69ff69dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe138d9a700 1 -- 172.21.15.161:0/1170187483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 msgr2=0x55b69ff6b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:15.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe138d9a700 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff6b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe138d9a700 1 -- 172.21.15.161:0/1170187483 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b69fcc5c60 con 0x55b69ff5cbe0 2023-07-14T11:49:15.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe139d9c700 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff67ad0 0x55b69ff69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:49:15.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe138d9a700 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff6bde0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55b69ff67060 tx=0x7fe124010b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aec06486f713750 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:15.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe11cff9700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:49:15.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe11cff9700 1 -- 172.21.15.161:0/1170187483 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124017070 con 0x55b69ff5cbe0 2023-07-14T11:49:15.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe11cff9700 1 -- 172.21.15.161:0/1170187483 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fe124010e40 con 0x55b69ff5cbe0 2023-07-14T11:49:15.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1300a3250 con 0x55b69ff5cbe0 2023-07-14T11:49:15.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe11cff9700 1 -- 172.21.15.161:0/1170187483 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe12401c420 con 0x55b69ff5cbe0 2023-07-14T11:49:15.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe14f16eb80 10 client.28003 fetch_fsmap learned FSMap version 46 2023-07-14T11:49:15.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.390+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55b69fcdcde0 con 0x55b69ff5cbe0 2023-07-14T11:49:15.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe11cff9700 1 -- 172.21.15.161:0/1170187483 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1300a3250 con 0x55b69ff5cbe0 2023-07-14T11:49:15.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe11cff9700 1 -- 172.21.15.161:0/1170187483 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe124005690 con 0x55b69ff5cbe0 2023-07-14T11:49:15.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe14f16eb80 10 client.28003 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:49:15.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe14f16eb80 10 client.28003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:49:15.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe14f16eb80 20 client.28003 populate_metadata read hostname 'smithi161' 2023-07-14T11:49:15.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ffcc150 0x55b69ffce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:15.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe14f16eb80 4 client.28003 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:49:15.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b69fe6a050 con 0x55b69ffcc150 2023-07-14T11:49:15.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe13959b700 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ffcc150 0x55b69ffce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:15.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe11cff9700 10 client.28003 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:15.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.391+0000 7fe13959b700 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ffcc150 0x55b69ffce560 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63feb9ee40c8379c server_cookie=c2f01b66377075ee in_seq=0 out_seq=0 2023-07-14T11:49:15.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe11cff9700 1 -- 172.21.15.161:0/1170187483 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b69fe6a050 con 0x55b69ffcc150 2023-07-14T11:49:15.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe11cff9700 10 client.28003 handle_command_reply: tid=0 2023-07-14T11:49:15.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 client.28003 shutdown 2023-07-14T11:49:15.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 msgr2=0x55b69ff6bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff6bde0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55b69ff67060 tx=0x7fe124010b60 comp rx=0 tx=0).stop 2023-07-14T11:49:15.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 shutdown_connections 2023-07-14T11:49:15.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55b69ffcc150 0x55b69ffce560 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b69ff5c2d0 0x55b69ff6b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe11cff9700 0 client.28003 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:15.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b69ff67ad0 0x55b69ff69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 --2- 172.21.15.161:0/1170187483 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55b69ff5cbe0 0x55b69ff6bde0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 >> 172.21.15.161:0/1170187483 conn(0x55b69ff52da0 msgr2=0x55b69ff5e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:15.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 shutdown_connections 2023-07-14T11:49:15.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 1 -- 172.21.15.161:0/1170187483 wait complete. 2023-07-14T11:49:15.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.392+0000 7fe14f16eb80 20 client.28003 trim_cache size 0 max 16384 2023-07-14T11:49:15.417 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:49:15.417 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:49:15.418 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:49:15.418 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:49:15.418 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:49:15.418 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:49:15.418 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:49:15.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:49:15.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:49:15.419 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:49:15.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe1200a9b50 msgr2=0x7fe1200ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe1200a9b50 0x7fe1200ac000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00f6e0 tx=0x7fe12c009920 comp rx=0 tx=0).stop 2023-07-14T11:49:15.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe1340ffae0 msgr2=0x7fe134197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:15.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe1340ffae0 0x7fe134197e40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe130005910 tx=0x7fe130004710 comp rx=0 tx=0).stop 2023-07-14T11:49:15.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 shutdown_connections 2023-07-14T11:49:15.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fe1200a9b50 0x7fe1200ac000 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe134105af0 0x7fe13419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fe1340ffae0 0x7fe134197e40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 --2- 172.21.15.161:0/3408189558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe134100490 0x7fe134198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:15.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 >> 172.21.15.161:0/3408189558 conn(0x7fe1340fb5e0 msgr2=0x7fe13410ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:15.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 shutdown_connections 2023-07-14T11:49:15.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:15.393+0000 7fe13b7ff700 1 -- 172.21.15.161:0/3408189558 wait complete. 2023-07-14T11:49:15.423 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:49:16.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:16 smithi175 ceph-mon[129081]: pgmap v855: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:49:16.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:16 smithi182 ceph-mon[129366]: pgmap v855: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:49:16.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:16 smithi161 ceph-mon[95526]: pgmap v855: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:49:18.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:18 smithi175 ceph-mon[129081]: pgmap v856: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:49:18.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:18 smithi182 ceph-mon[129366]: pgmap v856: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:49:18.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:18 smithi161 ceph-mon[95526]: pgmap v856: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:49:19.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:19 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:49:19.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:19 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:49:19.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:19 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:49:20.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:20 smithi175 ceph-mon[129081]: pgmap v857: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:49:20.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:20 smithi182 ceph-mon[129366]: pgmap v857: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:49:20.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:20 smithi161 ceph-mon[95526]: pgmap v857: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:49:22.894 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:22 smithi175 ceph-mon[129081]: pgmap v858: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:49:22.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:22 smithi182 ceph-mon[129366]: pgmap v858: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:49:22.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:22 smithi161 ceph-mon[95526]: pgmap v858: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:49:24.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:49:24.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:49:24.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.653 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:24 smithi175 ceph-mon[129081]: pgmap v859: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:24.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.698 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.699 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.699 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.700 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:49:24.701 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:49:24.701 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.701 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:24 smithi182 ceph-mon[129366]: pgmap v859: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:24.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.729 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.729 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.729 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.730 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:49:24.730 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:49:24.730 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:49:24.731 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:24 smithi161 ceph-mon[95526]: pgmap v859: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:26.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:26 smithi175 ceph-mon[129081]: pgmap v860: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:26.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:26 smithi182 ceph-mon[129366]: pgmap v860: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:26.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:26 smithi161 ceph-mon[95526]: pgmap v860: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:28.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:28 smithi175 ceph-mon[129081]: pgmap v861: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:28.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:28 smithi182 ceph-mon[129366]: pgmap v861: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:28.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:28 smithi161 ceph-mon[95526]: pgmap v861: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:30.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:30 smithi175 ceph-mon[129081]: pgmap v862: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:30.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:30 smithi182 ceph-mon[129366]: pgmap v862: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:30.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:30 smithi161 ceph-mon[95526]: pgmap v862: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:32.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:32 smithi175 ceph-mon[129081]: pgmap v863: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:32.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:32 smithi182 ceph-mon[129366]: pgmap v863: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:32.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:32 smithi161 ceph-mon[95526]: pgmap v863: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:34.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:34 smithi175 ceph-mon[129081]: pgmap v864: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:34.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:34 smithi182 ceph-mon[129366]: pgmap v864: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:34.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:34 smithi161 ceph-mon[95526]: pgmap v864: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:36.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:36 smithi175 ceph-mon[129081]: pgmap v865: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:36.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:36 smithi182 ceph-mon[129366]: pgmap v865: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:36.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:36 smithi161 ceph-mon[95526]: pgmap v865: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:38.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:38 smithi175 ceph-mon[129081]: pgmap v866: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:38.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:38 smithi182 ceph-mon[129366]: pgmap v866: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:38.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:38 smithi161 ceph-mon[95526]: pgmap v866: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:40.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:40 smithi175 ceph-mon[129081]: pgmap v867: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:40.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:40 smithi182 ceph-mon[129366]: pgmap v867: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:40.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:40 smithi161 ceph-mon[95526]: pgmap v867: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:42.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:42 smithi175 ceph-mon[129081]: pgmap v868: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:42.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:42 smithi182 ceph-mon[129366]: pgmap v868: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:42.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:42 smithi161 ceph-mon[95526]: pgmap v868: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:44.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:44 smithi175 ceph-mon[129081]: pgmap v869: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:44.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:44 smithi182 ceph-mon[129366]: pgmap v869: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:44.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:44 smithi161 ceph-mon[95526]: pgmap v869: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:45.425 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:49:45.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 -- 172.21.15.161:0/3683949532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 msgr2=0x7f418410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 --2- 172.21.15.161:0/3683949532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 0x7f418410c4b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f417c00b0a0 tx=0x7f417c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 -- 172.21.15.161:0/3683949532 shutdown_connections 2023-07-14T11:49:45.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 --2- 172.21.15.161:0/3683949532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4184100ca0 0x7f4184101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 --2- 172.21.15.161:0/3683949532 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 0x7f418410c4b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 --2- 172.21.15.161:0/3683949532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41841015b0 0x7f4184109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 -- 172.21.15.161:0/3683949532 >> 172.21.15.161:0/3683949532 conn(0x7f41840fa4b0 msgr2=0x7f41840fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:45.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 -- 172.21.15.161:0/3683949532 shutdown_connections 2023-07-14T11:49:45.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.654+0000 7f41891ff700 1 -- 172.21.15.161:0/3683949532 wait complete. 2023-07-14T11:49:45.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 Processor -- start 2023-07-14T11:49:45.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 -- start start 2023-07-14T11:49:45.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4184100ca0 0x7f4184197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41841015b0 0x7f41841984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 0x7f418419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418419cf60 con 0x7f41841015b0 2023-07-14T11:49:45.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418419d0d0 con 0x7f4184100ca0 2023-07-14T11:49:45.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.655+0000 7f41891ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418419d290 con 0x7f418410a070 2023-07-14T11:49:45.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4184100ca0 0x7f4184197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4184100ca0 0x7f4184197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51388/0 (socket says 172.21.15.161:51388) 2023-07-14T11:49:45.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 -- 172.21.15.161:0/3887559376 learned_addr learned my addr 172.21.15.161:0/3887559376 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:45.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f418359e700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 0x7f418419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f418259c700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41841015b0 0x7f41841984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 -- 172.21.15.161:0/3887559376 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 msgr2=0x7f418419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 0x7f418419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 -- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41841015b0 msgr2=0x7f41841984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41841015b0 0x7f41841984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 -- 172.21.15.161:0/3887559376 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c009d00 con 0x7f4184100ca0 2023-07-14T11:49:45.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f418359e700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 0x7f418419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:49:45.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f4182d9d700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4184100ca0 0x7f4184197f80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f417400e9a0 tx=0x7f417400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:45.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f416bfff700 1 -- 172.21.15.161:0/3887559376 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174009790 con 0x7f4184100ca0 2023-07-14T11:49:45.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418419d570 con 0x7f4184100ca0 2023-07-14T11:49:45.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.656+0000 7f416bfff700 1 -- 172.21.15.161:0/3887559376 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4174010430 con 0x7f4184100ca0 2023-07-14T11:49:45.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.657+0000 7f416bfff700 1 -- 172.21.15.161:0/3887559376 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174018680 con 0x7f4184100ca0 2023-07-14T11:49:45.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.657+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41841a4db0 con 0x7f4184100ca0 2023-07-14T11:49:45.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.660+0000 7f416bfff700 1 -- 172.21.15.161:0/3887559376 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f41740105a0 con 0x7f4184100ca0 2023-07-14T11:49:45.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 Processor -- start 2023-07-14T11:49:45.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 -- start start 2023-07-14T11:49:45.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f416bfff700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f416c0a12c0 0x7f416c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa71590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008fa73eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008f96ccb0 con 0x56008fa71ad0 2023-07-14T11:49:45.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008f9bfa40 con 0x56008fa66be0 2023-07-14T11:49:45.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.661+0000 7f419cb56b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008f9b8ed0 con 0x56008fa662d0 2023-07-14T11:49:45.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f416bfff700 1 -- 172.21.15.161:0/3887559376 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4174014070 con 0x7f4184100ca0 2023-07-14T11:49:45.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa71590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41962/0 (socket says 172.21.15.161:41962) 2023-07-14T11:49:45.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f4182d9d700 1 -- 172.21.15.161:0/3644590779 learned_addr learned my addr 172.21.15.161:0/3644590779 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:45.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418359e700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f416c0a12c0 0x7f416c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418359e700 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008fa73eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.655 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418259c700 1 -- 172.21.15.161:0/3644590779 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 msgr2=0x56008fa666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418259c700 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418259c700 1 -- 172.21.15.161:0/3644590779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 msgr2=0x56008fa73eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418259c700 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008fa73eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418259c700 1 -- 172.21.15.161:0/3644590779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56008f7e6de0 con 0x56008fa66be0 2023-07-14T11:49:45.656 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418359e700 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008fa73eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:49:45.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.662+0000 7f418259c700 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa71590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4178009c80 tx=0x7f4178009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7584da5274211423 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:45.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.663+0000 7f41697fa700 1 -- 172.21.15.161:0/3644590779 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178011cb0 con 0x56008fa66be0 2023-07-14T11:49:45.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.663+0000 7f41697fa700 1 -- 172.21.15.161:0/3644590779 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4178011e10 con 0x56008fa66be0 2023-07-14T11:49:45.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.663+0000 7f41697fa700 1 -- 172.21.15.161:0/3644590779 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178005280 con 0x56008fa66be0 2023-07-14T11:49:45.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.663+0000 7f419cb56b80 1 -- 172.21.15.161:0/3644590779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 msgr2=0x56008fa71590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.663+0000 7f418359e700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f416c0a12c0 0x7f416c0a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f417c00b780 tx=0x7f417c0093a0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:45.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.663+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa71590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4178009c80 tx=0x7f4178009f90 comp rx=0 tx=0).stop 2023-07-14T11:49:45.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 -- 172.21.15.161:0/3644590779 shutdown_connections 2023-07-14T11:49:45.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008fa73eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3644590779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa71590 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 -- 172.21.15.161:0/3644590779 >> 172.21.15.161:0/3644590779 conn(0x56008fa5cda0 msgr2=0x56008fa626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:45.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 -- 172.21.15.161:0/3644590779 shutdown_connections 2023-07-14T11:49:45.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 -- 172.21.15.161:0/3644590779 wait complete. 2023-07-14T11:49:45.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 -- start start 2023-07-14T11:49:45.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.664+0000 7f419cb56b80 1 Processor -- start 2023-07-14T11:49:45.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa662d0 0x56008fa71510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa66be0 0x56008fa6e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa71ad0 0x56008fa6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f419cb56b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008f958ac0 con 0x56008fa66be0 2023-07-14T11:49:45.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f419cb56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008f9587c0 con 0x56008fa662d0 2023-07-14T11:49:45.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f419cb56b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008f83c920 con 0x56008fa71ad0 2023-07-14T11:49:45.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f419cb56b80 10 client.0 _ll_register_callbacks cb 0x56008f956760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:49:45.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa662d0 0x56008fa71510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa662d0 0x56008fa71510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51400/0 (socket says 172.21.15.161:51400) 2023-07-14T11:49:45.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f4182d9d700 1 -- 172.21.15.161:0/3796919181 learned_addr learned my addr 172.21.15.161:0/3796919181 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:45.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f418359e700 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa71ad0 0x56008fa6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f418259c700 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa66be0 0x56008fa6e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f4182d9d700 1 -- 172.21.15.161:0/3796919181 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa71ad0 msgr2=0x56008fa6eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f4182d9d700 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa71ad0 0x56008fa6eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f4182d9d700 1 -- 172.21.15.161:0/3796919181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa66be0 msgr2=0x56008fa6e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.665+0000 7f4182d9d700 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa66be0 0x56008fa6e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4182d9d700 1 -- 172.21.15.161:0/3796919181 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41780098e0 con 0x56008fa662d0 2023-07-14T11:49:45.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4182d9d700 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa662d0 0x56008fa71510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4174099ca0 tx=0x7f41740999f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d36d951833afaa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:45.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4166ff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:49:45.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4166ff5700 1 -- 172.21.15.161:0/3796919181 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41740a23f0 con 0x56008fa662d0 2023-07-14T11:49:45.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4166ff5700 1 -- 172.21.15.161:0/3796919181 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f41740ca8a0 con 0x56008fa662d0 2023-07-14T11:49:45.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56008f9124f0 con 0x56008fa662d0 2023-07-14T11:49:45.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4166ff5700 1 -- 172.21.15.161:0/3796919181 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41740caa60 con 0x56008fa662d0 2023-07-14T11:49:45.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f419cb56b80 10 client.27971 fetch_fsmap learned FSMap version 46 2023-07-14T11:49:45.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56008f7cfc60 con 0x56008fa662d0 2023-07-14T11:49:45.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4166ff5700 1 -- 172.21.15.161:0/3796919181 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56008f9124f0 con 0x56008fa662d0 2023-07-14T11:49:45.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f4166ff5700 1 -- 172.21.15.161:0/3796919181 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f417409a200 con 0x56008fa662d0 2023-07-14T11:49:45.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f419cb56b80 10 client.27971 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:49:45.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.666+0000 7f419cb56b80 10 client.27971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:49:45.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.667+0000 7f419cb56b80 20 client.27971 populate_metadata read hostname 'smithi161' 2023-07-14T11:49:45.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.667+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fa813e0 0x56008fa837c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.667+0000 7f419cb56b80 4 client.27971 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:49:45.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.667+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56008f9b1eb0 con 0x56008fa813e0 2023-07-14T11:49:45.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.667+0000 7f418259c700 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fa813e0 0x56008fa837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.667+0000 7f418259c700 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fa813e0 0x56008fa837c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35704f416b06278c server_cookie=9cbbf0c3fa8a50f8 in_seq=0 out_seq=0 2023-07-14T11:49:45.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.667+0000 7f4166ff5700 10 client.27971 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:45.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f4166ff5700 1 -- 172.21.15.161:0/3796919181 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56008f9b1eb0 con 0x56008fa813e0 2023-07-14T11:49:45.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f4166ff5700 10 client.27971 handle_command_reply: tid=0 2023-07-14T11:49:45.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 client.27971 shutdown 2023-07-14T11:49:45.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa662d0 msgr2=0x56008fa71510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa662d0 0x56008fa71510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4174099ca0 tx=0x7f41740999f0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 shutdown_connections 2023-07-14T11:49:45.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fa813e0 0x56008fa837c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa71ad0 0x56008fa6eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa662d0 0x56008fa71510 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 --2- 172.21.15.161:0/3796919181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa66be0 0x56008fa6e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 >> 172.21.15.161:0/3796919181 conn(0x56008fa5cda0 msgr2=0x56008fa62230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:45.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f4166ff5700 0 client.27971 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:45.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.668+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 shutdown_connections 2023-07-14T11:49:45.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.669+0000 7f419cb56b80 1 -- 172.21.15.161:0/3796919181 wait complete. 2023-07-14T11:49:45.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.669+0000 7f419cb56b80 20 client.27971 trim_cache size 0 max 16384 2023-07-14T11:49:45.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 Processor -- start 2023-07-14T11:49:45.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 -- start start 2023-07-14T11:49:45.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa662d0 0x56008fa710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa66be0 0x56008fa6df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa71ad0 0x56008fa6e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.678 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008fa811c0 con 0x56008fa662d0 2023-07-14T11:49:45.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008fa5c970 con 0x56008fa71ad0 2023-07-14T11:49:45.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f419cb56b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008fa7dde0 con 0x56008fa66be0 2023-07-14T11:49:45.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f418259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa66be0 0x56008fa6df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f418359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa71ad0 0x56008fa6e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.679 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f418259c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa66be0 0x56008fa6df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41988/0 (socket says 172.21.15.161:41988) 2023-07-14T11:49:45.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f418259c700 1 -- 172.21.15.161:0/1806443323 learned_addr learned my addr 172.21.15.161:0/1806443323 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:45.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.686+0000 7f4182d9d700 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa662d0 0x56008fa710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f418259c700 1 -- 172.21.15.161:0/1806443323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa71ad0 msgr2=0x56008fa6e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f418259c700 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa71ad0 0x56008fa6e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.680 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f418259c700 1 -- 172.21.15.161:0/1806443323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa662d0 msgr2=0x56008fa710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f418259c700 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa662d0 0x56008fa710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f418259c700 1 -- 172.21.15.161:0/1806443323 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56008f80d250 con 0x56008fa66be0 2023-07-14T11:49:45.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f418259c700 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa66be0 0x56008fa6df50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4178019000 tx=0x7f41780117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907efae03f17c628 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:45.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f4169ffb700 1 -- 172.21.15.161:0/1806443323 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178004870 con 0x56008fa66be0 2023-07-14T11:49:45.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f4169ffb700 1 -- 172.21.15.161:0/1806443323 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4178011b30 con 0x56008fa66be0 2023-07-14T11:49:45.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.687+0000 7f4169ffb700 1 -- 172.21.15.161:0/1806443323 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178023670 con 0x56008fa66be0 2023-07-14T11:49:45.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 -- 172.21.15.161:0/1806443323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa66be0 msgr2=0x56008fa6df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa66be0 0x56008fa6df50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4178019000 tx=0x7f41780117d0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 -- 172.21.15.161:0/1806443323 shutdown_connections 2023-07-14T11:49:45.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa662d0 0x56008fa710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa71ad0 0x56008fa6e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1806443323 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa66be0 0x56008fa6df50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 -- 172.21.15.161:0/1806443323 >> 172.21.15.161:0/1806443323 conn(0x56008fa5cda0 msgr2=0x56008fa69a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:45.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 -- 172.21.15.161:0/1806443323 shutdown_connections 2023-07-14T11:49:45.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 -- 172.21.15.161:0/1806443323 wait complete. 2023-07-14T11:49:45.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.688+0000 7f419cb56b80 1 -- start start 2023-07-14T11:49:45.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 1 Processor -- start 2023-07-14T11:49:45.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa76c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008face210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008face7c0 con 0x56008fa71ad0 2023-07-14T11:49:45.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008face930 con 0x56008fa66be0 2023-07-14T11:49:45.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008faceaf0 con 0x56008fa662d0 2023-07-14T11:49:45.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f419cb56b80 10 client.0 _ll_register_callbacks cb 0x7f4174099ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:49:45.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f418359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008face210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f418259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f4182d9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa766c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41990/0 (socket says 172.21.15.161:41990) 2023-07-14T11:49:45.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.689+0000 7f4182d9d700 1 -- 172.21.15.161:0/1650129972 learned_addr learned my addr 172.21.15.161:0/1650129972 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:49:45.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f418259c700 1 -- 172.21.15.161:0/1650129972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 msgr2=0x56008fa766c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f418259c700 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa766c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f418259c700 1 -- 172.21.15.161:0/1650129972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 msgr2=0x56008face210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:49:45.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f418259c700 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008face210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f418259c700 1 -- 172.21.15.161:0/1650129972 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417801d040 con 0x56008fa66be0 2023-07-14T11:49:45.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f4182d9d700 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:49:45.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f418359e700 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008face210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:49:45.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f418259c700 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa76c00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4178009c80 tx=0x7f4178004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35d5adee67c60cf6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:49:45.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f4166ff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:49:45.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f4166ff5700 1 -- 172.21.15.161:0/1650129972 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178004250 con 0x56008fa66be0 2023-07-14T11:49:45.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f4166ff5700 1 -- 172.21.15.161:0/1650129972 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f41780043b0 con 0x56008fa66be0 2023-07-14T11:49:45.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4178011e10 con 0x56008fa66be0 2023-07-14T11:49:45.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f4166ff5700 1 -- 172.21.15.161:0/1650129972 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4178004570 con 0x56008fa66be0 2023-07-14T11:49:45.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f419cb56b80 10 client.27977 fetch_fsmap learned FSMap version 46 2023-07-14T11:49:45.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.690+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56008f7e52b0 con 0x56008fa66be0 2023-07-14T11:49:45.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f4166ff5700 1 -- 172.21.15.161:0/1650129972 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4178011e10 con 0x56008fa66be0 2023-07-14T11:49:45.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f4166ff5700 1 -- 172.21.15.161:0/1650129972 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4178003ac0 con 0x56008fa66be0 2023-07-14T11:49:45.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f419cb56b80 10 client.27977 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:49:45.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f419cb56b80 10 client.27977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:49:45.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f419cb56b80 20 client.27977 populate_metadata read hostname 'smithi161' 2023-07-14T11:49:45.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fad5f70 0x56008fad8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:49:45.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f419cb56b80 4 client.27977 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:49:45.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56008f974050 con 0x56008fad5f70 2023-07-14T11:49:45.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f4182d9d700 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fad5f70 0x56008fad8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:49:45.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f4182d9d700 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fad5f70 0x56008fad8350 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27bacb90122dcad server_cookie=6666eedac5fc8bf1 in_seq=0 out_seq=0 2023-07-14T11:49:45.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.691+0000 7f4166ff5700 10 client.27977 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:45.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f4166ff5700 1 -- 172.21.15.161:0/1650129972 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56008f974050 con 0x56008fad5f70 2023-07-14T11:49:45.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f4166ff5700 10 client.27977 handle_command_reply: tid=0 2023-07-14T11:49:45.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 client.27977 shutdown 2023-07-14T11:49:45.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 msgr2=0x56008fa76c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa76c00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4178009c80 tx=0x7f4178004a90 comp rx=0 tx=0).stop 2023-07-14T11:49:45.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 shutdown_connections 2023-07-14T11:49:45.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x56008fad5f70 0x56008fad8350 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56008fa71ad0 0x56008face210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f4166ff5700 0 client.27977 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:49:45.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x56008fa662d0 0x56008fa766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 --2- 172.21.15.161:0/1650129972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56008fa66be0 0x56008fa76c00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 >> 172.21.15.161:0/1650129972 conn(0x56008fa5cda0 msgr2=0x56008fa618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:45.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 shutdown_connections 2023-07-14T11:49:45.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 1 -- 172.21.15.161:0/1650129972 wait complete. 2023-07-14T11:49:45.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.692+0000 7f419cb56b80 20 client.27977 trim_cache size 0 max 16384 2023-07-14T11:49:45.695 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:49:45.695 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:49:45.695 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:49:45.695 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:49:45.696 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:49:45.696 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:49:45.696 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:49:45.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:49:45.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:49:45.697 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:49:45.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f416c0a12c0 msgr2=0x7f416c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f416c0a12c0 0x7f416c0a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f417c00b780 tx=0x7f417c0093a0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4184100ca0 msgr2=0x7f4184197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:49:45.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4184100ca0 0x7f4184197f80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f417400e9a0 tx=0x7f417400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 shutdown_connections 2023-07-14T11:49:45.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f416c0a12c0 0x7f416c0a3770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f418410a070 0x7f418419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4184100ca0 0x7f4184197f80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.699 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 --2- 172.21.15.161:0/3887559376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41841015b0 0x7f41841984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:49:45.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 >> 172.21.15.161:0/3887559376 conn(0x7f41840fa4b0 msgr2=0x7f41840fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:49:45.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 shutdown_connections 2023-07-14T11:49:45.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:49:45.693+0000 7f41891ff700 1 -- 172.21.15.161:0/3887559376 wait complete. 2023-07-14T11:49:45.700 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:49:46.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:46 smithi175 ceph-mon[129081]: pgmap v870: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:46.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:46 smithi182 ceph-mon[129366]: pgmap v870: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:46.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:46 smithi161 ceph-mon[95526]: pgmap v870: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:48.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:48 smithi175 ceph-mon[129081]: pgmap v871: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:48.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:48 smithi182 ceph-mon[129366]: pgmap v871: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:48.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:48 smithi161 ceph-mon[95526]: pgmap v871: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:50.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:50 smithi175 ceph-mon[129081]: pgmap v872: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:50.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:50 smithi182 ceph-mon[129366]: pgmap v872: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:50.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:50 smithi161 ceph-mon[95526]: pgmap v872: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:52.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:52 smithi175 ceph-mon[129081]: pgmap v873: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:52.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:52 smithi182 ceph-mon[129366]: pgmap v873: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:52.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:52 smithi161 ceph-mon[95526]: pgmap v873: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:49:54.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:54 smithi175 ceph-mon[129081]: pgmap v874: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:54.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:54 smithi182 ceph-mon[129366]: pgmap v874: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:54.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:54 smithi161 ceph-mon[95526]: pgmap v874: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:49:56.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:56 smithi175 ceph-mon[129081]: pgmap v875: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:56.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:56 smithi182 ceph-mon[129366]: pgmap v875: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:56.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:56 smithi161 ceph-mon[95526]: pgmap v875: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:49:58.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:49:58 smithi175 ceph-mon[129081]: pgmap v876: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:58.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:49:58 smithi182 ceph-mon[129366]: pgmap v876: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:49:58.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:49:58 smithi161 ceph-mon[95526]: pgmap v876: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:00.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:00 smithi175 ceph-mon[129081]: pgmap v877: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:00.902 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:00 smithi175 ceph-mon[129081]: overall HEALTH_OK 2023-07-14T11:50:00.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:00 smithi182 ceph-mon[129366]: pgmap v877: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:00.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:00 smithi182 ceph-mon[129366]: overall HEALTH_OK 2023-07-14T11:50:00.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:00 smithi161 ceph-mon[95526]: pgmap v877: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:00.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:00 smithi161 ceph-mon[95526]: overall HEALTH_OK 2023-07-14T11:50:02.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:02 smithi175 ceph-mon[129081]: pgmap v878: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:50:02.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:02 smithi182 ceph-mon[129366]: pgmap v878: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:50:02.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:02 smithi161 ceph-mon[95526]: pgmap v878: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:50:04.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:04 smithi175 ceph-mon[129081]: pgmap v879: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:50:04.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:04 smithi182 ceph-mon[129366]: pgmap v879: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:50:04.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:04 smithi161 ceph-mon[95526]: pgmap v879: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:50:06.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:06 smithi175 ceph-mon[129081]: pgmap v880: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:50:06.947 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:06 smithi182 ceph-mon[129366]: pgmap v880: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:50:06.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:06 smithi161 ceph-mon[95526]: pgmap v880: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:50:07.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:07 smithi182 ceph-mon[129366]: osdmap e136: 12 total, 12 up, 12 in 2023-07-14T11:50:07.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:07 smithi161 ceph-mon[95526]: osdmap e136: 12 total, 12 up, 12 in 2023-07-14T11:50:08.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:07 smithi175 ceph-mon[129081]: osdmap e136: 12 total, 12 up, 12 in 2023-07-14T11:50:08.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:08 smithi182 ceph-mon[129366]: pgmap v882: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:50:08.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:08 smithi161 ceph-mon[95526]: pgmap v882: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:50:09.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:08 smithi175 ceph-mon[129081]: pgmap v882: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:50:10.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:10 smithi182 ceph-mon[129366]: pgmap v883: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:50:10.979 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:10 smithi161 ceph-mon[95526]: pgmap v883: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:50:11.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:10 smithi175 ceph-mon[129081]: pgmap v883: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:50:12.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:12 smithi182 ceph-mon[129366]: pgmap v884: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:50:12.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:12 smithi161 ceph-mon[95526]: pgmap v884: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:50:13.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:12 smithi175 ceph-mon[129081]: pgmap v884: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:50:14.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:14 smithi175 ceph-mon[129081]: osdmap e137: 12 total, 12 up, 12 in 2023-07-14T11:50:14.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:14 smithi175 ceph-mon[129081]: pgmap v886: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:50:14.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:14 smithi182 ceph-mon[129366]: osdmap e137: 12 total, 12 up, 12 in 2023-07-14T11:50:14.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:14 smithi182 ceph-mon[129366]: pgmap v886: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:50:14.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:14 smithi161 ceph-mon[95526]: osdmap e137: 12 total, 12 up, 12 in 2023-07-14T11:50:14.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:14 smithi161 ceph-mon[95526]: pgmap v886: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T11:50:15.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:50:15.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 -- 172.21.15.161:0/58818288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 msgr2=0x7f87d8109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 --2- 172.21.15.161:0/58818288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d8109aa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009990 tx=0x7f87cc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 -- 172.21.15.161:0/58818288 shutdown_connections 2023-07-14T11:50:15.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 --2- 172.21.15.161:0/58818288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87d8100ca0 0x7f87d8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 --2- 172.21.15.161:0/58818288 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87d810a070 0x7f87d810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 --2- 172.21.15.161:0/58818288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d8109aa0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 -- 172.21.15.161:0/58818288 >> 172.21.15.161:0/58818288 conn(0x7f87d80fa4b0 msgr2=0x7f87d80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:15.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 -- 172.21.15.161:0/58818288 shutdown_connections 2023-07-14T11:50:15.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.936+0000 7f87de7ff700 1 -- 172.21.15.161:0/58818288 wait complete. 2023-07-14T11:50:15.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.937+0000 7f87de7ff700 1 Processor -- start 2023-07-14T11:50:15.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.937+0000 7f87de7ff700 1 -- start start 2023-07-14T11:50:15.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87de7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87d8100ca0 0x7f87d819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87de7ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87de7ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87d810a070 0x7f87d8195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87de7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d81960f0 con 0x7f87d8100ca0 2023-07-14T11:50:15.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87de7ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8196260 con 0x7f87d81015b0 2023-07-14T11:50:15.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87de7ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8196420 con 0x7f87d810a070 2023-07-14T11:50:15.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87d8100ca0 0x7f87d819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45406/0 (socket says 172.21.15.161:45406) 2023-07-14T11:50:15.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 -- 172.21.15.161:0/1454219336 learned_addr learned my addr 172.21.15.161:0/1454219336 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:15.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87d810a070 0x7f87d8195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 -- 172.21.15.161:0/1454219336 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87d810a070 msgr2=0x7f87d8195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87d810a070 0x7f87d8195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 -- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87d8100ca0 msgr2=0x7f87d819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87d8100ca0 0x7f87d819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d77fe700 1 -- 172.21.15.161:0/1454219336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0009c40 con 0x7f87d81015b0 2023-07-14T11:50:15.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.938+0000 7f87d7fff700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87d8100ca0 0x7f87d819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:50:15.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.939+0000 7f87d77fe700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d819bf50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f87cc005910 tx=0x7f87cc005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:15.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.939+0000 7f87d57fa700 1 -- 172.21.15.161:0/1454219336 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc01d070 con 0x7f87d81015b0 2023-07-14T11:50:15.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.939+0000 7f87d57fa700 1 -- 172.21.15.161:0/1454219336 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87cc022470 con 0x7f87d81015b0 2023-07-14T11:50:15.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.939+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc0095f0 con 0x7f87d81015b0 2023-07-14T11:50:15.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.939+0000 7f87d57fa700 1 -- 172.21.15.161:0/1454219336 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc017650 con 0x7f87d81015b0 2023-07-14T11:50:15.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.939+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d81a2820 con 0x7f87d81015b0 2023-07-14T11:50:15.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.942+0000 7f87d57fa700 1 -- 172.21.15.161:0/1454219336 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f87cc0225e0 con 0x7f87d81015b0 2023-07-14T11:50:15.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.943+0000 7f87f2193b80 1 Processor -- start 2023-07-14T11:50:15.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87f2193b80 1 -- start start 2023-07-14T11:50:15.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d57fa700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f87c00a1340 0x7f87c00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.938 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d57fa700 1 -- 172.21.15.161:0/1454219336 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f87cc0ce350 con 0x7f87d81015b0 2023-07-14T11:50:15.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87f2193b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d80becb0 con 0x55c6d81b8be0 2023-07-14T11:50:15.939 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87f2193b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d8111a40 con 0x55c6d81c7ee0 2023-07-14T11:50:15.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87f2193b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d810aed0 con 0x55c6d81b82d0 2023-07-14T11:50:15.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f87c00a1340 0x7f87c00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81b86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41100/0 (socket says 172.21.15.161:41100) 2023-07-14T11:50:15.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d7fff700 1 -- 172.21.15.161:0/820968446 learned_addr learned my addr 172.21.15.161:0/820968446 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:15.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d77fe700 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d7fff700 1 -- 172.21.15.161:0/820968446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 msgr2=0x55c6d81ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d7fff700 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d7fff700 1 -- 172.21.15.161:0/820968446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 msgr2=0x55c6d81c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.944+0000 7f87d7fff700 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.945+0000 7f87d7fff700 1 -- 172.21.15.161:0/820968446 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6d7f38de0 con 0x55c6d81b82d0 2023-07-14T11:50:15.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.945+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:50:15.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.945+0000 7f87d7fff700 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81b86a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f87c80115f0 tx=0x7f87c8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c07634541f15c1f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:15.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.945+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f87c00a1340 0x7f87c00a37f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87d819c710 tx=0x7f87d0009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:15.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.945+0000 7f87be7fc700 1 -- 172.21.15.161:0/820968446 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8019410 con 0x55c6d81b82d0 2023-07-14T11:50:15.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.945+0000 7f87be7fc700 1 -- 172.21.15.161:0/820968446 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c8019a30 con 0x55c6d81b82d0 2023-07-14T11:50:15.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.945+0000 7f87be7fc700 1 -- 172.21.15.161:0/820968446 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c80189f0 con 0x55c6d81b82d0 2023-07-14T11:50:15.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 -- 172.21.15.161:0/820968446 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 msgr2=0x55c6d81b86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81b86a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f87c80115f0 tx=0x7f87c8011900 comp rx=0 tx=0).stop 2023-07-14T11:50:15.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 -- 172.21.15.161:0/820968446 shutdown_connections 2023-07-14T11:50:15.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81b86a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 --2- 172.21.15.161:0/820968446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 -- 172.21.15.161:0/820968446 >> 172.21.15.161:0/820968446 conn(0x55c6d81aeda0 msgr2=0x55c6d81b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:15.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 -- 172.21.15.161:0/820968446 shutdown_connections 2023-07-14T11:50:15.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 -- 172.21.15.161:0/820968446 wait complete. 2023-07-14T11:50:15.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.946+0000 7f87f2193b80 1 -- start start 2023-07-14T11:50:15.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 1 Processor -- start 2023-07-14T11:50:15.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b82d0 0x55c6d81c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81c7ee0 0x55c6d81c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d80aaac0 con 0x55c6d81b8be0 2023-07-14T11:50:15.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d80aa7c0 con 0x55c6d81b82d0 2023-07-14T11:50:15.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d7f8e920 con 0x55c6d81c7ee0 2023-07-14T11:50:15.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87f2193b80 10 client.0 _ll_register_callbacks cb 0x55c6d80a8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:50:15.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87d7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b82d0 0x55c6d81c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87dcd9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81c7ee0 0x55c6d81c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87dcd9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81c7ee0 0x55c6d81c5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41102/0 (socket says 172.21.15.161:41102) 2023-07-14T11:50:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.947+0000 7f87dcd9c700 1 -- 172.21.15.161:0/3378660134 learned_addr learned my addr 172.21.15.161:0/3378660134 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87dcd9c700 1 -- 172.21.15.161:0/3378660134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b82d0 msgr2=0x55c6d81c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b82d0 0x55c6d81c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87dcd9c700 1 -- 172.21.15.161:0/3378660134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 msgr2=0x55c6d81c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87dcd9c700 1 -- 172.21.15.161:0/3378660134 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c8009dd0 con 0x55c6d81c7ee0 2023-07-14T11:50:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87d77fe700 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:50:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81c7ee0 0x55c6d81c5110 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f87d0019000 tx=0x7f87d000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80c66c9ac2f0c3cb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:50:15.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87b77fe700 1 -- 172.21.15.161:0/3378660134 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0021070 con 0x55c6d81c7ee0 2023-07-14T11:50:15.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6d80644f0 con 0x55c6d81c7ee0 2023-07-14T11:50:15.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.948+0000 7f87b77fe700 1 -- 172.21.15.161:0/3378660134 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f87d0004bb0 con 0x55c6d81c7ee0 2023-07-14T11:50:15.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87b77fe700 1 -- 172.21.15.161:0/3378660134 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d000e040 con 0x55c6d81c7ee0 2023-07-14T11:50:15.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 10 client.28033 fetch_fsmap learned FSMap version 46 2023-07-14T11:50:15.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55c6d7f21c60 con 0x55c6d81c7ee0 2023-07-14T11:50:15.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87b77fe700 1 -- 172.21.15.161:0/3378660134 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6d80644f0 con 0x55c6d81c7ee0 2023-07-14T11:50:15.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87b77fe700 1 -- 172.21.15.161:0/3378660134 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87d001a940 con 0x55c6d81c7ee0 2023-07-14T11:50:15.955 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 10 client.28033 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:50:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 10 client.28033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:50:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 20 client.28033 populate_metadata read hostname 'smithi161' 2023-07-14T11:50:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d81d77c0 0x55c6d81d9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 4 client.28033 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:50:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6d8103eb0 con 0x55c6d81d77c0 2023-07-14T11:50:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.949+0000 7f87d7fff700 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d81d77c0 0x55c6d81d9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.950+0000 7f87d7fff700 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d81d77c0 0x55c6d81d9bd0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ad6b058fe826a1 server_cookie=231819eb7a3ba7df in_seq=0 out_seq=0 2023-07-14T11:50:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.950+0000 7f87b77fe700 10 client.28033 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:15.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.950+0000 7f87b77fe700 1 -- 172.21.15.161:0/3378660134 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6d8103eb0 con 0x55c6d81d77c0 2023-07-14T11:50:15.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87b77fe700 10 client.28033 handle_command_reply: tid=0 2023-07-14T11:50:15.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 client.28033 shutdown 2023-07-14T11:50:15.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81c7ee0 msgr2=0x55c6d81c5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81c7ee0 0x55c6d81c5110 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f87d0019000 tx=0x7f87d000b040 comp rx=0 tx=0).stop 2023-07-14T11:50:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 shutdown_connections 2023-07-14T11:50:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d81d77c0 0x55c6d81d9bd0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81c7ee0 0x55c6d81c5110 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87b77fe700 0 client.28033 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b82d0 0x55c6d81c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 --2- 172.21.15.161:0/3378660134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 >> 172.21.15.161:0/3378660134 conn(0x55c6d81aeda0 msgr2=0x55c6d81b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:15.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 shutdown_connections 2023-07-14T11:50:15.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 1 -- 172.21.15.161:0/3378660134 wait complete. 2023-07-14T11:50:15.961 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.951+0000 7f87f2193b80 20 client.28033 trim_cache size 0 max 16384 2023-07-14T11:50:15.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.968+0000 7f87f2193b80 1 Processor -- start 2023-07-14T11:50:15.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87f2193b80 1 -- start start 2023-07-14T11:50:15.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.966 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87f2193b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d81d75f0 con 0x55c6d81b8be0 2023-07-14T11:50:15.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87f2193b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d81b9e30 con 0x55c6d81c7ee0 2023-07-14T11:50:15.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87f2193b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d81b5d20 con 0x55c6d81b82d0 2023-07-14T11:50:15.967 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:41108/0 (socket says 172.21.15.161:41108) 2023-07-14T11:50:15.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 -- 172.21.15.161:0/4019591554 learned_addr learned my addr 172.21.15.161:0/4019591554 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:15.968 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 -- 172.21.15.161:0/4019591554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 msgr2=0x55c6d81b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 -- 172.21.15.161:0/4019591554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 msgr2=0x55c6d81d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87d7fff700 1 -- 172.21.15.161:0/4019591554 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6d7f5f250 con 0x55c6d81b82d0 2023-07-14T11:50:15.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.969+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:50:15.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.970+0000 7f87d7fff700 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f87c801afd0 tx=0x7f87c80185d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e1fd5d5ffce163 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:15.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.970+0000 7f87beffd700 1 -- 172.21.15.161:0/4019591554 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8029b90 con 0x55c6d81b82d0 2023-07-14T11:50:15.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.970+0000 7f87beffd700 1 -- 172.21.15.161:0/4019591554 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c8029cf0 con 0x55c6d81b82d0 2023-07-14T11:50:15.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.970+0000 7f87beffd700 1 -- 172.21.15.161:0/4019591554 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c80050c0 con 0x55c6d81b82d0 2023-07-14T11:50:15.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.970+0000 7f87f2193b80 1 -- 172.21.15.161:0/4019591554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 msgr2=0x55c6d81ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.970+0000 7f87f2193b80 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f87c801afd0 tx=0x7f87c80185d0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 -- 172.21.15.161:0/4019591554 shutdown_connections 2023-07-14T11:50:15.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81c7ee0 0x55c6d81b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce960 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 --2- 172.21.15.161:0/4019591554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81b8be0 0x55c6d81d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 -- 172.21.15.161:0/4019591554 >> 172.21.15.161:0/4019591554 conn(0x55c6d81aeda0 msgr2=0x55c6d81d3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:15.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 -- 172.21.15.161:0/4019591554 shutdown_connections 2023-07-14T11:50:15.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 -- 172.21.15.161:0/4019591554 wait complete. 2023-07-14T11:50:15.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 -- start start 2023-07-14T11:50:15.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.971+0000 7f87f2193b80 1 Processor -- start 2023-07-14T11:50:15.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b8be0 0x55c6d81ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87f2193b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81c7ee0 0x55c6d81cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87f2193b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d81cd600 con 0x55c6d81c7ee0 2023-07-14T11:50:15.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87f2193b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d81cd770 con 0x55c6d81b8be0 2023-07-14T11:50:15.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87f2193b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d81cd930 con 0x55c6d81b82d0 2023-07-14T11:50:15.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87f2193b80 10 client.0 _ll_register_callbacks cb 0x7f87d000fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:50:15.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b8be0 0x55c6d81ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b8be0 0x55c6d81ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45440/0 (socket says 172.21.15.161:45440) 2023-07-14T11:50:15.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 -- 172.21.15.161:0/349866210 learned_addr learned my addr 172.21.15.161:0/349866210 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:15.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87dcd9c700 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81c7ee0 0x55c6d81cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 -- 172.21.15.161:0/349866210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 msgr2=0x55c6d81ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 -- 172.21.15.161:0/349866210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81c7ee0 msgr2=0x55c6d81cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:15.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81c7ee0 0x55c6d81cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.972+0000 7f87d77fe700 1 -- 172.21.15.161:0/349866210 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6d7f38de0 con 0x55c6d81b8be0 2023-07-14T11:50:15.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87d77fe700 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b8be0 0x55c6d81ccb40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0cb740 tx=0x7f87cc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1c1c01967685b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:15.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:50:15.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87b77fe700 1 -- 172.21.15.161:0/349866210 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc09ac30 con 0x55c6d81b8be0 2023-07-14T11:50:15.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87c8019a30 con 0x55c6d81b8be0 2023-07-14T11:50:15.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87b77fe700 1 -- 172.21.15.161:0/349866210 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f87cc09b0f0 con 0x55c6d81b8be0 2023-07-14T11:50:15.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87b77fe700 1 -- 172.21.15.161:0/349866210 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87cc09aa50 con 0x55c6d81b8be0 2023-07-14T11:50:15.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 10 client.27995 fetch_fsmap learned FSMap version 46 2023-07-14T11:50:15.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55c6d7f372b0 con 0x55c6d81b8be0 2023-07-14T11:50:15.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87b77fe700 1 -- 172.21.15.161:0/349866210 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87cc0cb080 con 0x55c6d81b8be0 2023-07-14T11:50:15.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87b77fe700 1 -- 172.21.15.161:0/349866210 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87cc0cb1f0 con 0x55c6d81b8be0 2023-07-14T11:50:15.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 10 client.27995 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:50:15.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 10 client.27995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:50:15.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 20 client.27995 populate_metadata read hostname 'smithi161' 2023-07-14T11:50:15.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d822c8b0 0x55c6d822ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:15.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 4 client.27995 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:50:15.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.973+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6d80c6050 con 0x55c6d822c8b0 2023-07-14T11:50:15.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87d7fff700 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d822c8b0 0x55c6d822ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:15.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87d7fff700 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d822c8b0 0x55c6d822ec90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a305aa79093bf3 server_cookie=a7bc28e3c552f581 in_seq=0 out_seq=0 2023-07-14T11:50:15.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87b77fe700 10 client.27995 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:15.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87b77fe700 1 -- 172.21.15.161:0/349866210 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c6d80c6050 con 0x55c6d822c8b0 2023-07-14T11:50:15.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87b77fe700 10 client.27995 handle_command_reply: tid=0 2023-07-14T11:50:15.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 client.27995 shutdown 2023-07-14T11:50:15.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b8be0 msgr2=0x55c6d81ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.986 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b8be0 0x55c6d81ccb40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0cb740 tx=0x7f87cc009310 comp rx=0 tx=0).stop 2023-07-14T11:50:15.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 shutdown_connections 2023-07-14T11:50:15.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55c6d822c8b0 0x55c6d822ec90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6d81c7ee0 0x55c6d81cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87b77fe700 0 client.27995 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:15.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55c6d81b82d0 0x55c6d81ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 --2- 172.21.15.161:0/349866210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6d81b8be0 0x55c6d81ccb40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.974+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 >> 172.21.15.161:0/349866210 conn(0x55c6d81aeda0 msgr2=0x55c6d81b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:15.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 shutdown_connections 2023-07-14T11:50:15.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87f2193b80 1 -- 172.21.15.161:0/349866210 wait complete. 2023-07-14T11:50:15.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87f2193b80 20 client.27995 trim_cache size 0 max 16384 2023-07-14T11:50:15.989 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:50:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:50:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:50:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:50:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:50:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:50:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:50:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:50:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:50:15.991 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:50:15.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f87c00a1340 msgr2=0x7f87c00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f87c00a1340 0x7f87c00a37f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87d819c710 tx=0x7f87d0009920 comp rx=0 tx=0).stop 2023-07-14T11:50:15.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 msgr2=0x7f87d819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:15.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d819bf50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f87cc005910 tx=0x7f87cc005720 comp rx=0 tx=0).stop 2023-07-14T11:50:15.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 shutdown_connections 2023-07-14T11:50:15.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f87c00a1340 0x7f87c00a37f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87d8100ca0 0x7f87d819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.994 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f87d810a070 0x7f87d8195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 --2- 172.21.15.161:0/1454219336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81015b0 0x7f87d819bf50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:15.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 >> 172.21.15.161:0/1454219336 conn(0x7f87d80fa4b0 msgr2=0x7f87d810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:15.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 shutdown_connections 2023-07-14T11:50:15.995 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:15.975+0000 7f87de7ff700 1 -- 172.21.15.161:0/1454219336 wait complete. 2023-07-14T11:50:15.995 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:50:16.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:16 smithi175 ceph-mon[129081]: pgmap v887: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:50:16.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:16 smithi182 ceph-mon[129366]: pgmap v887: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:50:16.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:16 smithi161 ceph-mon[95526]: pgmap v887: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 114 B/s wr, 5 op/s 2023-07-14T11:50:18.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:18 smithi175 ceph-mon[129081]: pgmap v888: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:50:18.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:18 smithi182 ceph-mon[129366]: pgmap v888: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:50:18.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:18 smithi161 ceph-mon[95526]: pgmap v888: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:50:20.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:20 smithi175 ceph-mon[129081]: pgmap v889: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:50:20.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:20 smithi182 ceph-mon[129366]: pgmap v889: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:50:20.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:20 smithi161 ceph-mon[95526]: pgmap v889: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:50:22.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:22 smithi182 ceph-mon[129366]: pgmap v890: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:50:22.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:22 smithi161 ceph-mon[95526]: pgmap v890: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:50:23.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:22 smithi175 ceph-mon[129081]: pgmap v890: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:50:24.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:24 smithi175 ceph-mon[129081]: pgmap v891: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:24.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:24 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:50:24.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:24 smithi182 ceph-mon[129366]: pgmap v891: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:24.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:24 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:50:24.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:24 smithi161 ceph-mon[95526]: pgmap v891: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:24.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:24 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:50:25.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:25 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:50:25.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:25 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:50:25.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:25 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:50:25.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:25 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:50:25.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:25 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:50:25.697 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:25 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:50:25.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:25 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:50:25.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:25 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:50:25.729 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:25 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:50:26.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:26 smithi175 ceph-mon[129081]: pgmap v892: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:26.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:26 smithi182 ceph-mon[129366]: pgmap v892: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:26.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:26 smithi161 ceph-mon[95526]: pgmap v892: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:28.901 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:28 smithi175 ceph-mon[129081]: pgmap v893: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:28.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:28 smithi182 ceph-mon[129366]: pgmap v893: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:28.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:28 smithi161 ceph-mon[95526]: pgmap v893: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:30.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:30 smithi182 ceph-mon[129366]: pgmap v894: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:30.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:30 smithi161 ceph-mon[95526]: pgmap v894: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:31.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:30 smithi175 ceph-mon[129081]: pgmap v894: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:32.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:32 smithi182 ceph-mon[129366]: pgmap v895: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:32.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:32 smithi161 ceph-mon[95526]: pgmap v895: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:33.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:32 smithi175 ceph-mon[129081]: pgmap v895: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:34.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:34 smithi175 ceph-mon[129081]: pgmap v896: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:34.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:34 smithi182 ceph-mon[129366]: pgmap v896: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:34.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:34 smithi161 ceph-mon[95526]: pgmap v896: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:36.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:36 smithi182 ceph-mon[129366]: pgmap v897: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:36.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:36 smithi161 ceph-mon[95526]: pgmap v897: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:37.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:36 smithi175 ceph-mon[129081]: pgmap v897: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:38.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:38 smithi182 ceph-mon[129366]: pgmap v898: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:38.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:38 smithi161 ceph-mon[95526]: pgmap v898: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:39.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:38 smithi175 ceph-mon[129081]: pgmap v898: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:40.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:40 smithi182 ceph-mon[129366]: pgmap v899: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:40.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:40 smithi161 ceph-mon[95526]: pgmap v899: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:41.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:40 smithi175 ceph-mon[129081]: pgmap v899: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:42.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:42 smithi182 ceph-mon[129366]: pgmap v900: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:42.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:42 smithi161 ceph-mon[95526]: pgmap v900: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:43.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:42 smithi175 ceph-mon[129081]: pgmap v900: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:44.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:44 smithi175 ceph-mon[129081]: pgmap v901: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:44.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:44 smithi182 ceph-mon[129366]: pgmap v901: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:44.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:44 smithi161 ceph-mon[95526]: pgmap v901: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:45.996 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:50:46.217 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 -- 172.21.15.161:0/1553473207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e80ffae0 msgr2=0x7f06e80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 --2- 172.21.15.161:0/1553473207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e80ffae0 0x7f06e80fff50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009990 tx=0x7f06d4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 -- 172.21.15.161:0/1553473207 shutdown_connections 2023-07-14T11:50:46.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 --2- 172.21.15.161:0/1553473207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e8100490 0x7f06e810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 --2- 172.21.15.161:0/1553473207 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f06e8105af0 0x7f06e8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 --2- 172.21.15.161:0/1553473207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e80ffae0 0x7f06e80fff50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 -- 172.21.15.161:0/1553473207 >> 172.21.15.161:0/1553473207 conn(0x7f06e80fb5e0 msgr2=0x7f06e80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:46.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 -- 172.21.15.161:0/1553473207 shutdown_connections 2023-07-14T11:50:46.219 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.233+0000 7f06eebff700 1 -- 172.21.15.161:0/1553473207 wait complete. 2023-07-14T11:50:46.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.234+0000 7f06eebff700 1 Processor -- start 2023-07-14T11:50:46.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.234+0000 7f06eebff700 1 -- start start 2023-07-14T11:50:46.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06eebff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e80ffae0 0x7f06e8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06eebff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e8100490 0x7f06e8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06eebff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f06e8105af0 0x7f06e81a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06eebff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e8073580 con 0x7f06e80ffae0 2023-07-14T11:50:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06eebff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81a4340 con 0x7f06e8100490 2023-07-14T11:50:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06eebff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81a4480 con 0x7f06e8105af0 2023-07-14T11:50:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e8100490 0x7f06e8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e8100490 0x7f06e8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35376/0 (socket says 172.21.15.161:35376) 2023-07-14T11:50:46.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 -- 172.21.15.161:0/3308270126 learned_addr learned my addr 172.21.15.161:0/3308270126 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:46.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06ec99b700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e80ffae0 0x7f06e8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06ed19c700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f06e8105af0 0x7f06e81a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 -- 172.21.15.161:0/3308270126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f06e8105af0 msgr2=0x7f06e81a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f06e8105af0 0x7f06e81a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 -- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e80ffae0 msgr2=0x7f06e8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e80ffae0 0x7f06e8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 -- 172.21.15.161:0/3308270126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40095f0 con 0x7f06e8100490 2023-07-14T11:50:46.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.235+0000 7f06e7fff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e8100490 0x7f06e8072d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f06d4005910 tx=0x7f06d4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:46.224 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.236+0000 7f06e5ffb700 1 -- 172.21.15.161:0/3308270126 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401d070 con 0x7f06e8100490 2023-07-14T11:50:46.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.236+0000 7f06e5ffb700 1 -- 172.21.15.161:0/3308270126 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d4022470 con 0x7f06e8100490 2023-07-14T11:50:46.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.236+0000 7f06e5ffb700 1 -- 172.21.15.161:0/3308270126 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4017650 con 0x7f06e8100490 2023-07-14T11:50:46.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.236+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e81a4680 con 0x7f06e8100490 2023-07-14T11:50:46.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.236+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e81a4aa0 con 0x7f06e8100490 2023-07-14T11:50:46.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.239+0000 7f06e5ffb700 1 -- 172.21.15.161:0/3308270126 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f06d40225e0 con 0x7f06e8100490 2023-07-14T11:50:46.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.240+0000 7f0702594b80 1 Processor -- start 2023-07-14T11:50:46.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f0702594b80 1 -- start start 2023-07-14T11:50:46.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06e5ffb700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06d80a1340 0x7f06d80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06e5ffb700 1 -- 172.21.15.161:0/3308270126 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f06d40ce260 con 0x7f06e8100490 2023-07-14T11:50:46.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f0702594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46baa56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06ed19c700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06d80a1340 0x7f06d80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f0702594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bab0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f0702594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bab2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f0702594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46b9abcb0 con 0x55f46baa52d0 2023-07-14T11:50:46.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f0702594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46b9fea40 con 0x55f46bab0ad0 2023-07-14T11:50:46.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f0702594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46b9f7ed0 con 0x55f46baa5be0 2023-07-14T11:50:46.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06ec99b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46baa56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06ed19c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bab2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bab0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bab0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56098/0 (socket says 172.21.15.161:56098) 2023-07-14T11:50:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06e7fff700 1 -- 172.21.15.161:0/2082298708 learned_addr learned my addr 172.21.15.161:0/2082298708 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.241+0000 7f06ed19c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bab2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35380/0 (socket says 172.21.15.161:35380) 2023-07-14T11:50:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06e7fff700 1 -- 172.21.15.161:0/2082298708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 msgr2=0x55f46bab2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06e7fff700 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bab2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06e7fff700 1 -- 172.21.15.161:0/2082298708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 msgr2=0x55f46baa56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06e7fff700 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46baa56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06e7fff700 1 -- 172.21.15.161:0/2082298708 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f46b825de0 con 0x55f46baa5be0 2023-07-14T11:50:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06ec99b700 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46baa56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:50:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06ed19c700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06d80a1340 0x7f06d80a37f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f06e8074920 tx=0x7f06e000fc00 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06e7fff700 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bab0590 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f06d4024000 tx=0x7f06d409add0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae93d37b1bf5d2d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06d2ffd700 1 -- 172.21.15.161:0/2082298708 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d409b000 con 0x55f46baa5be0 2023-07-14T11:50:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06d2ffd700 1 -- 172.21.15.161:0/2082298708 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d409f030 con 0x55f46baa5be0 2023-07-14T11:50:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.242+0000 7f06d2ffd700 1 -- 172.21.15.161:0/2082298708 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d409b460 con 0x55f46baa5be0 2023-07-14T11:50:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 -- 172.21.15.161:0/2082298708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 msgr2=0x55f46bab0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bab0590 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f06d4024000 tx=0x7f06d409add0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 -- 172.21.15.161:0/2082298708 shutdown_connections 2023-07-14T11:50:46.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46baa56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bab2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 --2- 172.21.15.161:0/2082298708 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bab0590 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 -- 172.21.15.161:0/2082298708 >> 172.21.15.161:0/2082298708 conn(0x55f46ba9bda0 msgr2=0x55f46baa16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:46.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 -- 172.21.15.161:0/2082298708 shutdown_connections 2023-07-14T11:50:46.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 -- 172.21.15.161:0/2082298708 wait complete. 2023-07-14T11:50:46.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.243+0000 7f0702594b80 1 -- start start 2023-07-14T11:50:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 1 Processor -- start 2023-07-14T11:50:46.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bab0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46bab13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46bab0ad0 0x55f46bab1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46b997ac0 con 0x55f46baa52d0 2023-07-14T11:50:46.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46b9977c0 con 0x55f46baa5be0 2023-07-14T11:50:46.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46b87b920 con 0x55f46bab0ad0 2023-07-14T11:50:46.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f0702594b80 10 client.0 _ll_register_callbacks cb 0x55f46b995760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:50:46.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46bab13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46bab13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:35382/0 (socket says 172.21.15.161:35382) 2023-07-14T11:50:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f06e7fff700 1 -- 172.21.15.161:0/3406267488 learned_addr learned my addr 172.21.15.161:0/3406267488 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f06ed19c700 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46bab0ad0 0x55f46bab1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.244+0000 7f06ec99b700 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bab0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06e7fff700 1 -- 172.21.15.161:0/3406267488 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46bab0ad0 msgr2=0x55f46bab1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06e7fff700 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46bab0ad0 0x55f46bab1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06e7fff700 1 -- 172.21.15.161:0/3406267488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 msgr2=0x55f46bab0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06e7fff700 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bab0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06e7fff700 1 -- 172.21.15.161:0/3406267488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc0098e0 con 0x55f46baa5be0 2023-07-14T11:50:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06e7fff700 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46bab13e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f06d409ad00 tx=0x7f06d40cb610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4cb10c84c4c8c29 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:50:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06c7fff700 1 -- 172.21.15.161:0/3406267488 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d409b000 con 0x55f46baa5be0 2023-07-14T11:50:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06c7fff700 1 -- 172.21.15.161:0/3406267488 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f06d409b260 con 0x55f46baa5be0 2023-07-14T11:50:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f46b9514f0 con 0x55f46baa5be0 2023-07-14T11:50:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06c7fff700 1 -- 172.21.15.161:0/3406267488 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d409b420 con 0x55f46baa5be0 2023-07-14T11:50:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f0702594b80 10 client.28007 fetch_fsmap learned FSMap version 46 2023-07-14T11:50:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f06e81a4aa0 con 0x55f46baa5be0 2023-07-14T11:50:46.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.245+0000 7f06c7fff700 1 -- 172.21.15.161:0/3406267488 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d40cb080 con 0x55f46baa5be0 2023-07-14T11:50:46.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f06c7fff700 1 -- 172.21.15.161:0/3406267488 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06d40a2fc0 con 0x55f46baa5be0 2023-07-14T11:50:46.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f0702594b80 10 client.28007 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:50:46.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f0702594b80 10 client.28007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:50:46.243 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f0702594b80 20 client.28007 populate_metadata read hostname 'smithi161' 2023-07-14T11:50:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f0702594b80 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46babfc20 0x55f46bac2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f0702594b80 4 client.28007 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:50:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f46b9f0eb0 con 0x55f46babfc20 2023-07-14T11:50:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.246+0000 7f06ec99b700 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46babfc20 0x55f46bac2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f06ec99b700 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46babfc20 0x55f46bac2000 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febd24946c6b7bb server_cookie=2634866c6bf90b5a in_seq=0 out_seq=0 2023-07-14T11:50:46.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f06c7fff700 10 client.28007 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:46.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f06c7fff700 1 -- 172.21.15.161:0/3406267488 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f46b9f0eb0 con 0x55f46babfc20 2023-07-14T11:50:46.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f06c7fff700 10 client.28007 handle_command_reply: tid=0 2023-07-14T11:50:46.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 client.28007 shutdown 2023-07-14T11:50:46.245 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 msgr2=0x55f46bab13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46bab13e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f06d409ad00 tx=0x7f06d40cb610 comp rx=0 tx=0).stop 2023-07-14T11:50:46.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 shutdown_connections 2023-07-14T11:50:46.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46babfc20 0x55f46bac2000 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bab0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.246 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46bab0ad0 0x55f46bab1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 --2- 172.21.15.161:0/3406267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46bab13e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f06c7fff700 0 client.28007 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:46.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.247+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 >> 172.21.15.161:0/3406267488 conn(0x55f46ba9bda0 msgr2=0x55f46baa1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:46.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.248+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 shutdown_connections 2023-07-14T11:50:46.247 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.248+0000 7f0702594b80 1 -- 172.21.15.161:0/3406267488 wait complete. 2023-07-14T11:50:46.248 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.248+0000 7f0702594b80 20 client.28007 trim_cache size 0 max 16384 2023-07-14T11:50:46.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 Processor -- start 2023-07-14T11:50:46.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 -- start start 2023-07-14T11:50:46.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa52d0 0x55f46bab4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.250 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46baaf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46bab0ad0 0x55f46babc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46baa6580 con 0x55f46bab0ad0 2023-07-14T11:50:46.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46baa2310 con 0x55f46baa5be0 2023-07-14T11:50:46.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f0702594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46baa71a0 con 0x55f46baa52d0 2023-07-14T11:50:46.251 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f06ec99b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa52d0 0x55f46bab4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f06ec99b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa52d0 0x55f46bab4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56118/0 (socket says 172.21.15.161:56118) 2023-07-14T11:50:46.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f06ec99b700 1 -- 172.21.15.161:0/3715759153 learned_addr learned my addr 172.21.15.161:0/3715759153 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:46.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.265+0000 7f06e7fff700 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46baaf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ed19c700 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46bab0ad0 0x55f46babc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.252 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ec99b700 1 -- 172.21.15.161:0/3715759153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 msgr2=0x55f46baaf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ec99b700 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46baaf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ec99b700 1 -- 172.21.15.161:0/3715759153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46bab0ad0 msgr2=0x55f46babc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ec99b700 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46bab0ad0 0x55f46babc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ec99b700 1 -- 172.21.15.161:0/3715759153 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f46b80ec60 con 0x55f46baa52d0 2023-07-14T11:50:46.253 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ed19c700 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46bab0ad0 0x55f46babc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:50:46.256 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06ec99b700 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa52d0 0x55f46bab4ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009e00 tx=0x7f06dc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a87be1001817a44 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:46.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06d37fe700 1 -- 172.21.15.161:0/3715759153 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc01c5b0 con 0x55f46baa52d0 2023-07-14T11:50:46.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06d37fe700 1 -- 172.21.15.161:0/3715759153 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06dc017040 con 0x55f46baa52d0 2023-07-14T11:50:46.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.266+0000 7f06d37fe700 1 -- 172.21.15.161:0/3715759153 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc004b90 con 0x55f46baa52d0 2023-07-14T11:50:46.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 -- 172.21.15.161:0/3715759153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa52d0 msgr2=0x55f46bab4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.257 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa52d0 0x55f46bab4ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009e00 tx=0x7f06dc009260 comp rx=0 tx=0).stop 2023-07-14T11:50:46.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 -- 172.21.15.161:0/3715759153 shutdown_connections 2023-07-14T11:50:46.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46bab0ad0 0x55f46babc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa52d0 0x55f46bab4ed0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 --2- 172.21.15.161:0/3715759153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46baa5be0 0x55f46baaf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.258 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 -- 172.21.15.161:0/3715759153 >> 172.21.15.161:0/3715759153 conn(0x55f46ba9bda0 msgr2=0x55f46ba9fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:46.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 -- 172.21.15.161:0/3715759153 shutdown_connections 2023-07-14T11:50:46.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 -- 172.21.15.161:0/3715759153 wait complete. 2023-07-14T11:50:46.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.267+0000 7f0702594b80 1 -- start start 2023-07-14T11:50:46.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 1 Processor -- start 2023-07-14T11:50:46.259 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bb0d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bb0d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bb07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46bb08020 con 0x55f46baa52d0 2023-07-14T11:50:46.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46bb08190 con 0x55f46bab0ad0 2023-07-14T11:50:46.260 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46bb08350 con 0x55f46baa5be0 2023-07-14T11:50:46.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f0702594b80 10 client.0 _ll_register_callbacks cb 0x7f06dc009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:50:46.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f06ec99b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bb0d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bb0d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f06ec99b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bb0d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42346/0 (socket says 172.21.15.161:42346) 2023-07-14T11:50:46.261 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f06ec99b700 1 -- 172.21.15.161:0/222095240 learned_addr learned my addr 172.21.15.161:0/222095240 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:50:46.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f06e7fff700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bb0d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:56130/0 (socket says 172.21.15.161:56130) 2023-07-14T11:50:46.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.268+0000 7f06ed19c700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bb07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06e7fff700 1 -- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 msgr2=0x55f46bb07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06e7fff700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bb07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.262 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06e7fff700 1 -- 172.21.15.161:0/222095240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 msgr2=0x55f46bb0d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:50:46.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06e7fff700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bb0d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06e7fff700 1 -- 172.21.15.161:0/222095240 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc009c70 con 0x55f46baa5be0 2023-07-14T11:50:46.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06ed19c700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bb07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:50:46.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06ec99b700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bb0d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:50:46.263 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06e7fff700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bb0d930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f06d40a3580 tx=0x7f06d40cbb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463bdc126eec637d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:50:46.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:50:46.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06c7fff700 1 -- 172.21.15.161:0/222095240 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d409b460 con 0x55f46baa5be0 2023-07-14T11:50:46.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06c7fff700 1 -- 172.21.15.161:0/222095240 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f06d409b000 con 0x55f46baa5be0 2023-07-14T11:50:46.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06d409f030 con 0x55f46baa5be0 2023-07-14T11:50:46.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06c7fff700 1 -- 172.21.15.161:0/222095240 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d4096df0 con 0x55f46baa5be0 2023-07-14T11:50:46.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f0702594b80 10 client.28057 fetch_fsmap learned FSMap version 46 2023-07-14T11:50:46.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55f46b84c250 con 0x55f46baa5be0 2023-07-14T11:50:46.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.269+0000 7f06c7fff700 1 -- 172.21.15.161:0/222095240 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d409f030 con 0x55f46baa5be0 2023-07-14T11:50:46.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f06c7fff700 1 -- 172.21.15.161:0/222095240 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06d4099f60 con 0x55f46baa5be0 2023-07-14T11:50:46.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f0702594b80 10 client.28057 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:50:46.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f0702594b80 10 client.28057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:50:46.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f0702594b80 20 client.28057 populate_metadata read hostname 'smithi161' 2023-07-14T11:50:46.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f0702594b80 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46bb14d20 0x55f46bb17100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:50:46.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f0702594b80 4 client.28057 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:50:46.266 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f46b9b3050 con 0x55f46bb14d20 2023-07-14T11:50:46.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f06ec99b700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46bb14d20 0x55f46bb17100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:50:46.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f06ec99b700 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46bb14d20 0x55f46bb17100 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad9dc33deaaff79 server_cookie=b1fc3c4e05c81c4e in_seq=0 out_seq=0 2023-07-14T11:50:46.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f06c7fff700 10 client.28057 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:46.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f06c7fff700 1 -- 172.21.15.161:0/222095240 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f46b9b3050 con 0x55f46bb14d20 2023-07-14T11:50:46.267 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f06c7fff700 10 client.28057 handle_command_reply: tid=0 2023-07-14T11:50:46.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.270+0000 7f0702594b80 1 client.28057 shutdown 2023-07-14T11:50:46.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 msgr2=0x55f46bb0d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bb0d930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f06d40a3580 tx=0x7f06d40cbb50 comp rx=0 tx=0).stop 2023-07-14T11:50:46.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 shutdown_connections 2023-07-14T11:50:46.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55f46bb14d20 0x55f46bb17100 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f46baa52d0 0x55f46bb0d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f46bab0ad0 0x55f46bb07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 --2- 172.21.15.161:0/222095240 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55f46baa5be0 0x55f46bb0d930 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f06c7fff700 0 client.28057 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:50:46.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 >> 172.21.15.161:0/222095240 conn(0x55f46ba9bda0 msgr2=0x55f46baaf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:46.269 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 shutdown_connections 2023-07-14T11:50:46.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 1 -- 172.21.15.161:0/222095240 wait complete. 2023-07-14T11:50:46.270 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f0702594b80 20 client.28057 trim_cache size 0 max 16384 2023-07-14T11:50:46.270 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:50:46.271 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:50:46.271 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:50:46.271 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:50:46.271 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:50:46.271 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:50:46.272 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:50:46.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:50:46.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:50:46.272 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:50:46.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06d80a1340 msgr2=0x7f06d80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f06eebff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06d80a1340 0x7f06d80a37f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f06e8074920 tx=0x7f06e000fc00 comp rx=0 tx=0).stop 2023-07-14T11:50:46.273 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e8100490 msgr2=0x7f06e8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:50:46.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.271+0000 7f06eebff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e8100490 0x7f06e8072d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f06d4005910 tx=0x7f06d4004710 comp rx=0 tx=0).stop 2023-07-14T11:50:46.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 shutdown_connections 2023-07-14T11:50:46.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f06d80a1340 0x7f06d80a37f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e80ffae0 0x7f06e8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.274 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f06e8105af0 0x7f06e81a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 --2- 172.21.15.161:0/3308270126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06e8100490 0x7f06e8072d50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:50:46.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 >> 172.21.15.161:0/3308270126 conn(0x7f06e80fb5e0 msgr2=0x7f06e81035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:50:46.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 shutdown_connections 2023-07-14T11:50:46.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:50:46.272+0000 7f06eebff700 1 -- 172.21.15.161:0/3308270126 wait complete. 2023-07-14T11:50:46.276 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:50:46.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:46 smithi182 ceph-mon[129366]: pgmap v902: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:46.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:46 smithi161 ceph-mon[95526]: pgmap v902: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:47.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:46 smithi175 ceph-mon[129081]: pgmap v902: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:48.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:48 smithi182 ceph-mon[129366]: pgmap v903: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:48.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:48 smithi161 ceph-mon[95526]: pgmap v903: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:49.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:48 smithi175 ceph-mon[129081]: pgmap v903: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:50.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:50 smithi182 ceph-mon[129366]: pgmap v904: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:50.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:50 smithi161 ceph-mon[95526]: pgmap v904: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:51.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:50 smithi175 ceph-mon[129081]: pgmap v904: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:52.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:52 smithi182 ceph-mon[129366]: pgmap v905: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:52.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:52 smithi161 ceph-mon[95526]: pgmap v905: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:53.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:52 smithi175 ceph-mon[129081]: pgmap v905: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:50:54.652 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:54 smithi175 ceph-mon[129081]: pgmap v906: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:54.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:54 smithi182 ceph-mon[129366]: pgmap v906: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:54.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:54 smithi161 ceph-mon[95526]: pgmap v906: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:50:56.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:56 smithi182 ceph-mon[129366]: pgmap v907: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:56.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:56 smithi161 ceph-mon[95526]: pgmap v907: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:57.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:56 smithi175 ceph-mon[129081]: pgmap v907: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:50:58.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:50:58 smithi182 ceph-mon[129366]: pgmap v908: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:58.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:50:58 smithi161 ceph-mon[95526]: pgmap v908: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:50:59.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:50:58 smithi175 ceph-mon[129081]: pgmap v908: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:00.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:00 smithi182 ceph-mon[129366]: pgmap v909: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:00.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:00 smithi161 ceph-mon[95526]: pgmap v909: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:01.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:00 smithi175 ceph-mon[129081]: pgmap v909: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:02.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:02 smithi182 ceph-mon[129366]: pgmap v910: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:51:02.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:02 smithi161 ceph-mon[95526]: pgmap v910: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:51:03.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:02 smithi175 ceph-mon[129081]: pgmap v910: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:51:04.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:04 smithi175 ceph-mon[129081]: pgmap v911: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:51:04.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:04 smithi182 ceph-mon[129366]: pgmap v911: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:51:04.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:04 smithi161 ceph-mon[95526]: pgmap v911: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:51:06.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:06 smithi182 ceph-mon[129366]: pgmap v912: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:51:06.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:06 smithi161 ceph-mon[95526]: pgmap v912: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:51:07.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:06 smithi175 ceph-mon[129081]: pgmap v912: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:51:07.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:07 smithi182 ceph-mon[129366]: osdmap e138: 12 total, 12 up, 12 in 2023-07-14T11:51:07.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:07 smithi161 ceph-mon[95526]: osdmap e138: 12 total, 12 up, 12 in 2023-07-14T11:51:08.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:07 smithi175 ceph-mon[129081]: osdmap e138: 12 total, 12 up, 12 in 2023-07-14T11:51:08.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:08 smithi182 ceph-mon[129366]: pgmap v914: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:51:08.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:08 smithi161 ceph-mon[95526]: pgmap v914: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:51:09.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:08 smithi175 ceph-mon[129081]: pgmap v914: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:10 smithi182 ceph-mon[129366]: pgmap v915: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:51:10.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:10 smithi161 ceph-mon[95526]: pgmap v915: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:51:11.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:10 smithi175 ceph-mon[129081]: pgmap v915: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-07-14T11:51:12.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:12 smithi182 ceph-mon[129366]: pgmap v916: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:51:12.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:12 smithi161 ceph-mon[95526]: pgmap v916: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:51:13.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:12 smithi175 ceph-mon[129081]: pgmap v916: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-14T11:51:14.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:14 smithi175 ceph-mon[129081]: pgmap v917: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:51:14.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:14 smithi182 ceph-mon[129366]: pgmap v917: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:51:14.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:14 smithi161 ceph-mon[95526]: pgmap v917: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:51:16.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:51:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 -- 172.21.15.161:0/4231782321 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 msgr2=0x7fb504072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/4231782321 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504072c00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009990 tx=0x7fb4f8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 -- 172.21.15.161:0/4231782321 shutdown_connections 2023-07-14T11:51:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/4231782321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50410c3a0 0x7fb50410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/4231782321 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504072c00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/4231782321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb504073140 0x7fb50410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 -- 172.21.15.161:0/4231782321 >> 172.21.15.161:0/4231782321 conn(0x7fb5040fb5e0 msgr2=0x7fb5040fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:16.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 -- 172.21.15.161:0/4231782321 shutdown_connections 2023-07-14T11:51:16.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.506+0000 7fb50b1ff700 1 -- 172.21.15.161:0/4231782321 wait complete. 2023-07-14T11:51:16.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 Processor -- start 2023-07-14T11:51:16.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 -- start start 2023-07-14T11:51:16.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb504073140 0x7fb5041982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50410c3a0 0x7fb50419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50419cd90 con 0x7fb50410c3a0 2023-07-14T11:51:16.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50419cf00 con 0x7fb504073140 2023-07-14T11:51:16.495 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.508+0000 7fb50b1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50419d0c0 con 0x7fb504072830 2023-07-14T11:51:16.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47216/0 (socket says 172.21.15.161:47216) 2023-07-14T11:51:16.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 -- 172.21.15.161:0/1828871293 learned_addr learned my addr 172.21.15.161:0/1828871293 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:16.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb50979c700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50410c3a0 0x7fb50419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb503fff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb504073140 0x7fb5041982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 -- 172.21.15.161:0/1828871293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb504073140 msgr2=0x7fb5041982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb504073140 0x7fb5041982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 -- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50410c3a0 msgr2=0x7fb50419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50410c3a0 0x7fb50419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 -- 172.21.15.161:0/1828871293 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009520 con 0x7fb504072830 2023-07-14T11:51:16.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb50979c700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50410c3a0 0x7fb50419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:51:16.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb503fff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb504073140 0x7fb5041982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:51:16.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb508f9b700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504197db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8000c00 tx=0x7fb4f8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:16.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb501ffb700 1 -- 172.21.15.161:0/1828871293 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f801d070 con 0x7fb504072830 2023-07-14T11:51:16.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb501ffb700 1 -- 172.21.15.161:0/1828871293 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f800fbc0 con 0x7fb504072830 2023-07-14T11:51:16.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb501ffb700 1 -- 172.21.15.161:0/1828871293 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8017650 con 0x7fb504072830 2023-07-14T11:51:16.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.509+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f80095f0 con 0x7fb504072830 2023-07-14T11:51:16.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.510+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5041a4a00 con 0x7fb504072830 2023-07-14T11:51:16.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.513+0000 7fb501ffb700 1 -- 172.21.15.161:0/1828871293 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb4f8021410 con 0x7fb504072830 2023-07-14T11:51:16.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 Processor -- start 2023-07-14T11:51:16.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 -- start start 2023-07-14T11:51:16.502 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca62d0 0x55e3b7ca66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7baccb0 con 0x55e3b7cb1ad0 2023-07-14T11:51:16.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7bffa40 con 0x55e3b7ca62d0 2023-07-14T11:51:16.503 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7bf8ed0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.514+0000 7fb501ffb700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb4f40a12c0 0x7fb4f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb508f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb508f9b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47232/0 (socket says 172.21.15.161:47232) 2023-07-14T11:51:16.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb508f9b700 1 -- 172.21.15.161:0/3084777097 learned_addr learned my addr 172.21.15.161:0/3084777097 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:16.504 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca62d0 0x55e3b7ca66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb503fff700 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb4f40a12c0 0x7fb4f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb501ffb700 1 -- 172.21.15.161:0/1828871293 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fb4f80ce5d0 con 0x7fb504072830 2023-07-14T11:51:16.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 -- 172.21.15.161:0/3084777097 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 msgr2=0x55e3b7cb1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.505 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 -- 172.21.15.161:0/3084777097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7cb1ad0 msgr2=0x55e3b7cb3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 -- 172.21.15.161:0/3084777097 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b7a26de0 con 0x55e3b7ca62d0 2023-07-14T11:51:16.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb503fff700 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:51:16.506 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.515+0000 7fb50979c700 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca62d0 0x55e3b7ca66a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc014c00 tx=0x7fb4fc014f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc71145ef61fe2a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:16.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb4eeffd700 1 -- 172.21.15.161:0/3084777097 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00e040 con 0x55e3b7ca62d0 2023-07-14T11:51:16.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb50979c700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb4f40a12c0 0x7fb4f40a3770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc010000 tx=0x7fb4fc01e000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:16.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb4eeffd700 1 -- 172.21.15.161:0/3084777097 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4fc0092d0 con 0x55e3b7ca62d0 2023-07-14T11:51:16.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb4eeffd700 1 -- 172.21.15.161:0/3084777097 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0077f0 con 0x55e3b7ca62d0 2023-07-14T11:51:16.507 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3084777097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca62d0 msgr2=0x55e3b7ca66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca62d0 0x55e3b7ca66a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc014c00 tx=0x7fb4fc014f10 comp rx=0 tx=0).stop 2023-07-14T11:51:16.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3084777097 shutdown_connections 2023-07-14T11:51:16.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.508 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca62d0 0x55e3b7ca66a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3084777097 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.516+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3084777097 >> 172.21.15.161:0/3084777097 conn(0x55e3b7c9cda0 msgr2=0x55e3b7ca26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:16.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.517+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3084777097 shutdown_connections 2023-07-14T11:51:16.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.517+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3084777097 wait complete. 2023-07-14T11:51:16.509 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.517+0000 7fb51eaeab80 1 -- start start 2023-07-14T11:51:16.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.517+0000 7fb51eaeab80 1 Processor -- start 2023-07-14T11:51:16.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.510 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7caf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7b98ac0 con 0x55e3b7ca62d0 2023-07-14T11:51:16.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7b987c0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7a7c920 con 0x55e3b7cb1ad0 2023-07-14T11:51:16.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb51eaeab80 10 client.0 _ll_register_callbacks cb 0x55e3b7b96760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:51:16.511 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb50979c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7caf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59738/0 (socket says 172.21.15.161:59738) 2023-07-14T11:51:16.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 -- 172.21.15.161:0/2167911802 learned_addr learned my addr 172.21.15.161:0/2167911802 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:16.512 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb508f9b700 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.514 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 -- 172.21.15.161:0/2167911802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7cb1ad0 msgr2=0x55e3b7caf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7caf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 -- 172.21.15.161:0/2167911802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 msgr2=0x55e3b7cae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.515 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.518+0000 7fb503fff700 1 -- 172.21.15.161:0/2167911802 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc0147b0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb503fff700 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caed90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0013a10 tx=0x7fb4f0013d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7fb8a50c681f38 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:16.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb4e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:51:16.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb4e3fff700 1 -- 172.21.15.161:0/2167911802 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011c80 con 0x55e3b7ca6be0 2023-07-14T11:51:16.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3b7b524f0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.516 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb4e3fff700 1 -- 172.21.15.161:0/2167911802 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fb4f0013e40 con 0x55e3b7ca6be0 2023-07-14T11:51:16.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb4e3fff700 1 -- 172.21.15.161:0/2167911802 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f0011de0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb51eaeab80 10 client.28025 fetch_fsmap learned FSMap version 46 2023-07-14T11:51:16.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.519+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55e3b7a0fc60 con 0x55e3b7ca6be0 2023-07-14T11:51:16.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb4e3fff700 1 -- 172.21.15.161:0/2167911802 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3b7b524f0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb4e3fff700 1 -- 172.21.15.161:0/2167911802 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb4f001b510 con 0x55e3b7ca6be0 2023-07-14T11:51:16.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb51eaeab80 10 client.28025 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:51:16.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb51eaeab80 10 client.28025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:51:16.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb51eaeab80 20 client.28025 populate_metadata read hostname 'smithi161' 2023-07-14T11:51:16.518 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7cc1450 0x55e3b7cc3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb51eaeab80 4 client.28025 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:51:16.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3b7bf1eb0 con 0x55e3b7cc1450 2023-07-14T11:51:16.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb508f9b700 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7cc1450 0x55e3b7cc3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb4e3fff700 10 client.28025 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:16.519 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.520+0000 7fb508f9b700 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7cc1450 0x55e3b7cc3860 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e138c00f24b2ea7e server_cookie=7c70cdb3b1e059ec in_seq=0 out_seq=0 2023-07-14T11:51:16.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb4e3fff700 1 -- 172.21.15.161:0/2167911802 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e3b7bf1eb0 con 0x55e3b7cc1450 2023-07-14T11:51:16.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb4e3fff700 10 client.28025 handle_command_reply: tid=0 2023-07-14T11:51:16.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 client.28025 shutdown 2023-07-14T11:51:16.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca6be0 msgr2=0x55e3b7caed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.520 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caed90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0013a10 tx=0x7fb4f0013d20 comp rx=0 tx=0).stop 2023-07-14T11:51:16.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 shutdown_connections 2023-07-14T11:51:16.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7cc1450 0x55e3b7cc3860 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7caf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.521 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb4e3fff700 0 client.28025 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:16.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/2167911802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caed90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 >> 172.21.15.161:0/2167911802 conn(0x55e3b7c9cda0 msgr2=0x55e3b7ca2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:16.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.521+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 shutdown_connections 2023-07-14T11:51:16.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.522+0000 7fb51eaeab80 1 -- 172.21.15.161:0/2167911802 wait complete. 2023-07-14T11:51:16.522 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.522+0000 7fb51eaeab80 20 client.28025 trim_cache size 0 max 16384 2023-07-14T11:51:16.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 Processor -- start 2023-07-14T11:51:16.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 -- start start 2023-07-14T11:51:16.524 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cbcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7ca88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7c9eec0 con 0x55e3b7ca62d0 2023-07-14T11:51:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7ca70d0 con 0x55e3b7cb1ad0 2023-07-14T11:51:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7ca7540 con 0x55e3b7ca6be0 2023-07-14T11:51:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb503fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb50979c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7ca88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb50979c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7ca88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:59754/0 (socket says 172.21.15.161:59754) 2023-07-14T11:51:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb503fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47248/0 (socket says 172.21.15.161:47248) 2023-07-14T11:51:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb50979c700 1 -- 172.21.15.161:0/802748962 learned_addr learned my addr 172.21.15.161:0/802748962 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.539+0000 7fb508f9b700 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cbcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb503fff700 1 -- 172.21.15.161:0/802748962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 msgr2=0x55e3b7ca88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb503fff700 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7ca88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb503fff700 1 -- 172.21.15.161:0/802748962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 msgr2=0x55e3b7cbcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb503fff700 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cbcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb503fff700 1 -- 172.21.15.161:0/802748962 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b7a4d250 con 0x55e3b7ca6be0 2023-07-14T11:51:16.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb503fff700 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caf100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000c00 tx=0x7fb4f00116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f1a4763102205d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:16.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb4ef7fe700 1 -- 172.21.15.161:0/802748962 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0030070 con 0x55e3b7ca6be0 2023-07-14T11:51:16.528 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb4ef7fe700 1 -- 172.21.15.161:0/802748962 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f0005780 con 0x55e3b7ca6be0 2023-07-14T11:51:16.529 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.540+0000 7fb4ef7fe700 1 -- 172.21.15.161:0/802748962 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0004280 con 0x55e3b7ca6be0 2023-07-14T11:51:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 -- 172.21.15.161:0/802748962 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 msgr2=0x55e3b7caf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caf100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000c00 tx=0x7fb4f00116c0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 -- 172.21.15.161:0/802748962 shutdown_connections 2023-07-14T11:51:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cbcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7ca88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/802748962 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7caf100 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.533 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 -- 172.21.15.161:0/802748962 >> 172.21.15.161:0/802748962 conn(0x55e3b7c9cda0 msgr2=0x55e3b7ca81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 -- 172.21.15.161:0/802748962 shutdown_connections 2023-07-14T11:51:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 -- 172.21.15.161:0/802748962 wait complete. 2023-07-14T11:51:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 -- start start 2023-07-14T11:51:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.541+0000 7fb51eaeab80 1 Processor -- start 2023-07-14T11:51:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cb43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb51eaeab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7cb6490 con 0x55e3b7ca62d0 2023-07-14T11:51:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7d15320 con 0x55e3b7cb1ad0 2023-07-14T11:51:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb51eaeab80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b7d15460 con 0x55e3b7ca6be0 2023-07-14T11:51:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb51eaeab80 10 client.0 _ll_register_callbacks cb 0x7fb4f0013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:51:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb50979c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb503fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb508f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cb43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb503fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47256/0 (socket says 172.21.15.161:47256) 2023-07-14T11:51:16.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb503fff700 1 -- 172.21.15.161:0/3486703817 learned_addr learned my addr 172.21.15.161:0/3486703817 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:16.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb503fff700 1 -- 172.21.15.161:0/3486703817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 msgr2=0x55e3b7cb4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.542+0000 7fb503fff700 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb503fff700 1 -- 172.21.15.161:0/3486703817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 msgr2=0x55e3b7cb43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:16.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb503fff700 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cb43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb503fff700 1 -- 172.21.15.161:0/3486703817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3b7a0fc60 con 0x55e3b7ca6be0 2023-07-14T11:51:16.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb503fff700 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb4910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb4f001b8c0 tx=0x7fb4f0013510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c08e90989d48dd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:16.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb4e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:51:16.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb4e3fff700 1 -- 172.21.15.161:0/3486703817 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0030040 con 0x55e3b7ca6be0 2023-07-14T11:51:16.538 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4fc0092d0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb4e3fff700 1 -- 172.21.15.161:0/3486703817 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9512+0+0 (secure 0 0 0) 0x7fb4f0000e40 con 0x55e3b7ca6be0 2023-07-14T11:51:16.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb4e3fff700 1 -- 172.21.15.161:0/3486703817 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f001bea0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb51eaeab80 10 client.28075 fetch_fsmap learned FSMap version 46 2023-07-14T11:51:16.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55e3b7a252b0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.543+0000 7fb4e3fff700 1 -- 172.21.15.161:0/3486703817 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4fc0092d0 con 0x55e3b7ca6be0 2023-07-14T11:51:16.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb4e3fff700 1 -- 172.21.15.161:0/3486703817 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb4f0023050 con 0x55e3b7ca6be0 2023-07-14T11:51:16.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb51eaeab80 10 client.28075 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:51:16.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb51eaeab80 10 client.28075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:51:16.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb51eaeab80 20 client.28075 populate_metadata read hostname 'smithi161' 2023-07-14T11:51:16.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7d168b0 0x55e3b7d18cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:16.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb51eaeab80 4 client.28075 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:51:16.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e3b7bb4050 con 0x55e3b7d168b0 2023-07-14T11:51:16.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb508f9b700 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7d168b0 0x55e3b7d18cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:16.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb508f9b700 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7d168b0 0x55e3b7d18cc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c918ed05e28aac9 server_cookie=a151a947eb3d37d4 in_seq=0 out_seq=0 2023-07-14T11:51:16.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.544+0000 7fb4e3fff700 10 client.28075 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:16.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb4e3fff700 1 -- 172.21.15.161:0/3486703817 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e3b7bb4050 con 0x55e3b7d168b0 2023-07-14T11:51:16.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb4e3fff700 10 client.28075 handle_command_reply: tid=0 2023-07-14T11:51:16.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 client.28075 shutdown 2023-07-14T11:51:16.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 msgr2=0x55e3b7cb4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb4910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb4f001b8c0 tx=0x7fb4f0013510 comp rx=0 tx=0).stop 2023-07-14T11:51:16.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 shutdown_connections 2023-07-14T11:51:16.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55e3b7d168b0 0x55e3b7d18cc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3b7ca62d0 0x55e3b7cb43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3b7cb1ad0 0x55e3b7cb4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 --2- 172.21.15.161:0/3486703817 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55e3b7ca6be0 0x55e3b7cb4910 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb4e3fff700 0 client.28075 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:16.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 >> 172.21.15.161:0/3486703817 conn(0x55e3b7c9cda0 msgr2=0x55e3b7ca8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:16.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 shutdown_connections 2023-07-14T11:51:16.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 1 -- 172.21.15.161:0/3486703817 wait complete. 2023-07-14T11:51:16.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.545+0000 7fb51eaeab80 20 client.28075 trim_cache size 0 max 16384 2023-07-14T11:51:16.546 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:51:16.546 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:51:16.546 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:51:16.546 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:51:16.547 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:51:16.547 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:51:16.547 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:51:16.547 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:51:16.547 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:51:16.547 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:51:16.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb4f40a12c0 msgr2=0x7fb4f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb4f40a12c0 0x7fb4f40a3770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc010000 tx=0x7fb4fc01e000 comp rx=0 tx=0).stop 2023-07-14T11:51:16.549 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 msgr2=0x7fb504197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:16.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504197db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8000c00 tx=0x7fb4f8005ef0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 shutdown_connections 2023-07-14T11:51:16.550 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb4f40a12c0 0x7fb4f40a3770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50410c3a0 0x7fb50419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb504072830 0x7fb504197db0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 --2- 172.21.15.161:0/1828871293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb504073140 0x7fb5041982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 >> 172.21.15.161:0/1828871293 conn(0x7fb5040fb5e0 msgr2=0x7fb50410e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:16.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 shutdown_connections 2023-07-14T11:51:16.552 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:16.546+0000 7fb50b1ff700 1 -- 172.21.15.161:0/1828871293 wait complete. 2023-07-14T11:51:16.552 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:51:16.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:16 smithi182 ceph-mon[129366]: pgmap v918: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:51:16.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:16 smithi161 ceph-mon[95526]: pgmap v918: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:51:17.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:16 smithi175 ceph-mon[129081]: pgmap v918: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:51:18.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:18 smithi182 ceph-mon[129366]: pgmap v919: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 93 B/s wr, 7 op/s 2023-07-14T11:51:18.946 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:18 smithi182 ceph-mon[129366]: osdmap e139: 12 total, 12 up, 12 in 2023-07-14T11:51:18.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:18 smithi161 ceph-mon[95526]: pgmap v919: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 93 B/s wr, 7 op/s 2023-07-14T11:51:18.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:18 smithi161 ceph-mon[95526]: osdmap e139: 12 total, 12 up, 12 in 2023-07-14T11:51:19.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:18 smithi175 ceph-mon[129081]: pgmap v919: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 93 B/s wr, 7 op/s 2023-07-14T11:51:19.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:18 smithi175 ceph-mon[129081]: osdmap e139: 12 total, 12 up, 12 in 2023-07-14T11:51:20.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:20 smithi182 ceph-mon[129366]: pgmap v921: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:51:20.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:20 smithi161 ceph-mon[95526]: pgmap v921: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:51:21.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:20 smithi175 ceph-mon[129081]: pgmap v921: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:51:22.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:22 smithi182 ceph-mon[129366]: pgmap v922: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:22.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:22 smithi161 ceph-mon[95526]: pgmap v922: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:23.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:22 smithi175 ceph-mon[129081]: pgmap v922: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:24.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:24 smithi175 ceph-mon[129081]: pgmap v923: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:24.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:24 smithi182 ceph-mon[129366]: pgmap v923: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:24.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:24 smithi161 ceph-mon[95526]: pgmap v923: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:25.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:25 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:51:25.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:25 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:51:25.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:25 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:51:26.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:51:26.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:51:26.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:51:26.651 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:26 smithi175 ceph-mon[129081]: pgmap v924: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:26.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:51:26.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:51:26.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:51:26.696 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:26 smithi182 ceph-mon[129366]: pgmap v924: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:26.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:51:26.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:51:26.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:51:26.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:26 smithi161 ceph-mon[95526]: pgmap v924: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:51:28.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:28 smithi182 ceph-mon[129366]: pgmap v925: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:51:28.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:28 smithi161 ceph-mon[95526]: pgmap v925: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:51:29.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:28 smithi175 ceph-mon[129081]: pgmap v925: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:51:30.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:30 smithi182 ceph-mon[129366]: pgmap v926: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:51:30.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:30 smithi161 ceph-mon[95526]: pgmap v926: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:51:31.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:30 smithi175 ceph-mon[129081]: pgmap v926: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:51:32.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:32 smithi182 ceph-mon[129366]: pgmap v927: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:32.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:32 smithi161 ceph-mon[95526]: pgmap v927: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:33.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:32 smithi175 ceph-mon[129081]: pgmap v927: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:34.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:34 smithi175 ceph-mon[129081]: pgmap v928: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:34.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:34 smithi182 ceph-mon[129366]: pgmap v928: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:34.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:34 smithi161 ceph-mon[95526]: pgmap v928: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:36.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:36 smithi182 ceph-mon[129366]: pgmap v929: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:36.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:36 smithi161 ceph-mon[95526]: pgmap v929: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:37.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:36 smithi175 ceph-mon[129081]: pgmap v929: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:38.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:38 smithi182 ceph-mon[129366]: pgmap v930: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:38.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:38 smithi161 ceph-mon[95526]: pgmap v930: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:39.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:38 smithi175 ceph-mon[129081]: pgmap v930: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:40.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:40 smithi182 ceph-mon[129366]: pgmap v931: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:40.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:40 smithi161 ceph-mon[95526]: pgmap v931: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:41.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:40 smithi175 ceph-mon[129081]: pgmap v931: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:42.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:42 smithi182 ceph-mon[129366]: pgmap v932: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:42.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:42 smithi161 ceph-mon[95526]: pgmap v932: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:43.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:42 smithi175 ceph-mon[129081]: pgmap v932: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:44.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:44 smithi175 ceph-mon[129081]: pgmap v933: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:44.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:44 smithi182 ceph-mon[129366]: pgmap v933: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:44.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:44 smithi161 ceph-mon[95526]: pgmap v933: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:46.554 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:51:46.768 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 -- 172.21.15.161:0/880095252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604810a070 msgr2=0x7f604810c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 --2- 172.21.15.161:0/880095252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604810a070 0x7f604810c4b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f604000b0a0 tx=0x7f604000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.769 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 -- 172.21.15.161:0/880095252 shutdown_connections 2023-07-14T11:51:46.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 --2- 172.21.15.161:0/880095252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604810a070 0x7f604810c4b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 --2- 172.21.15.161:0/880095252 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6048100ca0 0x7f6048101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.770 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 --2- 172.21.15.161:0/880095252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60481015b0 0x7f6048109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 -- 172.21.15.161:0/880095252 >> 172.21.15.161:0/880095252 conn(0x7f60480fa4b0 msgr2=0x7f60480fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:46.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 -- 172.21.15.161:0/880095252 shutdown_connections 2023-07-14T11:51:46.771 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.784+0000 7f604e1ff700 1 -- 172.21.15.161:0/880095252 wait complete. 2023-07-14T11:51:46.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 Processor -- start 2023-07-14T11:51:46.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 -- start start 2023-07-14T11:51:46.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6048100ca0 0x7f6048197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.773 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60481015b0 0x7f60481984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604810a070 0x7f604819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604819cf90 con 0x7f604810a070 2023-07-14T11:51:46.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604819d100 con 0x7f6048100ca0 2023-07-14T11:51:46.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.785+0000 7f604e1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604819d2c0 con 0x7f60481015b0 2023-07-14T11:51:46.774 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6048100ca0 0x7f6048197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6048100ca0 0x7f6048197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56432/0 (socket says 172.21.15.161:56432) 2023-07-14T11:51:46.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 -- 172.21.15.161:0/1870465727 learned_addr learned my addr 172.21.15.161:0/1870465727 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:46.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f6047fff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604810a070 0x7f604819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f6046ffd700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60481015b0 0x7f60481984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.775 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 -- 172.21.15.161:0/1870465727 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60481015b0 msgr2=0x7f60481984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60481015b0 0x7f60481984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 -- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604810a070 msgr2=0x7f604819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604810a070 0x7f604819c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 -- 172.21.15.161:0/1870465727 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038009520 con 0x7f6048100ca0 2023-07-14T11:51:46.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f60477fe700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6048100ca0 0x7f6048197fb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f603800eb30 tx=0x7f603800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:46.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f6044ff9700 1 -- 172.21.15.161:0/1870465727 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038009880 con 0x7f6048100ca0 2023-07-14T11:51:46.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f6044ff9700 1 -- 172.21.15.161:0/1870465727 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60380044e0 con 0x7f6048100ca0 2023-07-14T11:51:46.777 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6040009d00 con 0x7f6048100ca0 2023-07-14T11:51:46.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.786+0000 7f6044ff9700 1 -- 172.21.15.161:0/1870465727 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038003d80 con 0x7f6048100ca0 2023-07-14T11:51:46.778 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.787+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604819d790 con 0x7f6048100ca0 2023-07-14T11:51:46.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.791+0000 7f6044ff9700 1 -- 172.21.15.161:0/1870465727 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6038003f20 con 0x7f6048100ca0 2023-07-14T11:51:46.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.792+0000 7f6061ac1b80 1 Processor -- start 2023-07-14T11:51:46.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.792+0000 7f6061ac1b80 1 -- start start 2023-07-14T11:51:46.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.792+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x5580369006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.792+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.792+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x55803690deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558036806cb0 con 0x55803690bad0 2023-07-14T11:51:46.782 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558036859a40 con 0x5580369002d0 2023-07-14T11:51:46.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558036852ed0 con 0x558036900be0 2023-07-14T11:51:46.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x5580369006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x5580369006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56440/0 (socket says 172.21.15.161:56440) 2023-07-14T11:51:46.783 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 -- 172.21.15.161:0/3792441255 learned_addr learned my addr 172.21.15.161:0/3792441255 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:46.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f60477fe700 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6044ff9700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60300a1340 0x7f60300a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6046ffd700 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x55803690deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6044ff9700 1 -- 172.21.15.161:0/1870465727 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f6038014070 con 0x7f6048100ca0 2023-07-14T11:51:46.784 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 -- 172.21.15.161:0/3792441255 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 msgr2=0x55803690b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 -- 172.21.15.161:0/3792441255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 msgr2=0x55803690deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x55803690deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 -- 172.21.15.161:0/3792441255 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558036680de0 con 0x5580369002d0 2023-07-14T11:51:46.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f60477fe700 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:51:46.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.793+0000 7f6047fff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60300a1340 0x7f60300a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.794+0000 7f6047fff700 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x5580369006a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6040015000 tx=0x7f6040009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfd864082fc015ef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:46.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.794+0000 7f602dffb700 1 -- 172.21.15.161:0/3792441255 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604000e040 con 0x5580369002d0 2023-07-14T11:51:46.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.794+0000 7f602dffb700 1 -- 172.21.15.161:0/3792441255 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604001e460 con 0x5580369002d0 2023-07-14T11:51:46.786 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.794+0000 7f602dffb700 1 -- 172.21.15.161:0/3792441255 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400079a0 con 0x5580369002d0 2023-07-14T11:51:46.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 -- 172.21.15.161:0/3792441255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 msgr2=0x5580369006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x5580369006a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6040015000 tx=0x7f6040009360 comp rx=0 tx=0).stop 2023-07-14T11:51:46.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6047fff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60300a1340 0x7f60300a37f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f604002a000 tx=0x7f6040027000 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:46.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 -- 172.21.15.161:0/3792441255 shutdown_connections 2023-07-14T11:51:46.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x55803690deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x5580369006a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/3792441255 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 -- 172.21.15.161:0/3792441255 >> 172.21.15.161:0/3792441255 conn(0x5580368f6da0 msgr2=0x5580368fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:46.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 -- 172.21.15.161:0/3792441255 shutdown_connections 2023-07-14T11:51:46.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 -- 172.21.15.161:0/3792441255 wait complete. 2023-07-14T11:51:46.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.795+0000 7f6061ac1b80 1 -- start start 2023-07-14T11:51:46.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 1 Processor -- start 2023-07-14T11:51:46.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x55803690b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036908d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580367f2ac0 con 0x5580369002d0 2023-07-14T11:51:46.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580367f27c0 con 0x55803690bad0 2023-07-14T11:51:46.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580366d6920 con 0x558036900be0 2023-07-14T11:51:46.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6061ac1b80 10 client.0 _ll_register_callbacks cb 0x5580367f0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:51:46.790 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6047fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036908d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6046ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f60477fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x55803690b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6047fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036908d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:56448/0 (socket says 172.21.15.161:56448) 2023-07-14T11:51:46.791 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.796+0000 7f6047fff700 1 -- 172.21.15.161:0/2315744634 learned_addr learned my addr 172.21.15.161:0/2315744634 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:46.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6046ffd700 1 -- 172.21.15.161:0/2315744634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 msgr2=0x558036908d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.793 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6046ffd700 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036908d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6046ffd700 1 -- 172.21.15.161:0/2315744634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 msgr2=0x55803690b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6046ffd700 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x55803690b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6046ffd700 1 -- 172.21.15.161:0/2315744634 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604819d790 con 0x558036900be0 2023-07-14T11:51:46.794 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6046ffd700 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369086d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f603c012680 tx=0x7f603c012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a9c753ee16005e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:46.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6026ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:51:46.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6026ffd700 1 -- 172.21.15.161:0/2315744634 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c017070 con 0x558036900be0 2023-07-14T11:51:46.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580367ac4f0 con 0x558036900be0 2023-07-14T11:51:46.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6026ffd700 1 -- 172.21.15.161:0/2315744634 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f603c010430 con 0x558036900be0 2023-07-14T11:51:46.795 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6026ffd700 1 -- 172.21.15.161:0/2315744634 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f603c021630 con 0x558036900be0 2023-07-14T11:51:46.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6061ac1b80 10 client.28087 fetch_fsmap learned FSMap version 46 2023-07-14T11:51:46.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f604810d170 con 0x558036900be0 2023-07-14T11:51:46.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.797+0000 7f6026ffd700 1 -- 172.21.15.161:0/2315744634 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580367ac4f0 con 0x558036900be0 2023-07-14T11:51:46.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f6026ffd700 1 -- 172.21.15.161:0/2315744634 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f603c010680 con 0x558036900be0 2023-07-14T11:51:46.796 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f6061ac1b80 10 client.28087 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:51:46.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f6061ac1b80 10 client.28087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:51:46.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f6061ac1b80 20 client.28087 populate_metadata read hostname 'smithi161' 2023-07-14T11:51:46.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55803691b390 0x55803691d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f6061ac1b80 4 client.28087 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:51:46.797 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55803684beb0 con 0x55803691b390 2023-07-14T11:51:46.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.798+0000 7f60477fe700 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55803691b390 0x55803691d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6026ffd700 10 client.28087 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:46.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f60477fe700 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55803691b390 0x55803691d7a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700e6c0dc578213c server_cookie=ead476891a16219d in_seq=0 out_seq=0 2023-07-14T11:51:46.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6026ffd700 1 -- 172.21.15.161:0/2315744634 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55803684beb0 con 0x55803691b390 2023-07-14T11:51:46.798 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6026ffd700 10 client.28087 handle_command_reply: tid=0 2023-07-14T11:51:46.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 client.28087 shutdown 2023-07-14T11:51:46.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 msgr2=0x5580369086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369086d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f603c012680 tx=0x7f603c012990 comp rx=0 tx=0).stop 2023-07-14T11:51:46.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 shutdown_connections 2023-07-14T11:51:46.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55803691b390 0x55803691d7a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.799 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x55803690b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6026ffd700 0 client.28087 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:46.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036908d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.799+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2315744634 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369086d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.800 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.800+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 >> 172.21.15.161:0/2315744634 conn(0x5580368f6da0 msgr2=0x5580368fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:46.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.800+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 shutdown_connections 2023-07-14T11:51:46.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.800+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2315744634 wait complete. 2023-07-14T11:51:46.801 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.800+0000 7f6061ac1b80 20 client.28087 trim_cache size 0 max 16384 2023-07-14T11:51:46.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 Processor -- start 2023-07-14T11:51:46.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 -- start start 2023-07-14T11:51:46.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x55803690deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.802 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x5580369039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580369031b0 con 0x55803690bad0 2023-07-14T11:51:46.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803691d9c0 con 0x5580369002d0 2023-07-14T11:51:46.803 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.817+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803691db80 con 0x558036900be0 2023-07-14T11:51:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f60477fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x55803690deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:36852/0 (socket says 172.21.15.161:36852) 2023-07-14T11:51:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 -- 172.21.15.161:0/2111686726 learned_addr learned my addr 172.21.15.161:0/2111686726 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6047fff700 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x5580369039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 -- 172.21.15.161:0/2111686726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 msgr2=0x55803690deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x55803690deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 -- 172.21.15.161:0/2111686726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 msgr2=0x5580369039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x5580369039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 -- 172.21.15.161:0/2111686726 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558036669c60 con 0x558036900be0 2023-07-14T11:51:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f60477fe700 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x55803690deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:51:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6047fff700 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x5580369039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:51:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f6046ffd700 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690aff0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f603c000c00 tx=0x7f603c009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a58f393825b0e933 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f602e7fc700 1 -- 172.21.15.161:0/2111686726 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c017040 con 0x558036900be0 2023-07-14T11:51:46.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f602e7fc700 1 -- 172.21.15.161:0/2111686726 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603c010630 con 0x558036900be0 2023-07-14T11:51:46.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.818+0000 7f602e7fc700 1 -- 172.21.15.161:0/2111686726 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c003c60 con 0x558036900be0 2023-07-14T11:51:46.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2111686726 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 msgr2=0x55803690aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690aff0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f603c000c00 tx=0x7f603c009770 comp rx=0 tx=0).stop 2023-07-14T11:51:46.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2111686726 shutdown_connections 2023-07-14T11:51:46.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55803690bad0 0x5580369039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580369002d0 0x55803690deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/2111686726 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x55803690aff0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2111686726 >> 172.21.15.161:0/2111686726 conn(0x5580368f6da0 msgr2=0x558036922100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:46.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2111686726 shutdown_connections 2023-07-14T11:51:46.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 -- 172.21.15.161:0/2111686726 wait complete. 2023-07-14T11:51:46.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.819+0000 7f6061ac1b80 1 -- start start 2023-07-14T11:51:46.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 1 Processor -- start 2023-07-14T11:51:46.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x558036912200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.811 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036910a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558036912b10 con 0x5580369002d0 2023-07-14T11:51:46.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558036910f70 con 0x55803690bad0 2023-07-14T11:51:46.812 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x558036911130 con 0x558036900be0 2023-07-14T11:51:46.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6061ac1b80 10 client.0 _ll_register_callbacks cb 0x7f603c009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:51:46.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f60477fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x558036912200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6047fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036910a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f60477fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x558036912200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48074/0 (socket says 172.21.15.161:48074) 2023-07-14T11:51:46.813 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f60477fe700 1 -- 172.21.15.161:0/849824440 learned_addr learned my addr 172.21.15.161:0/849824440 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:51:46.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.820+0000 7f6046ffd700 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6046ffd700 1 -- 172.21.15.161:0/849824440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 msgr2=0x558036910a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6046ffd700 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036910a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6046ffd700 1 -- 172.21.15.161:0/849824440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 msgr2=0x558036912200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:51:46.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6046ffd700 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x558036912200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.814 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6046ffd700 1 -- 172.21.15.161:0/849824440 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6040009d00 con 0x558036900be0 2023-07-14T11:51:46.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6046ffd700 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369104c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5580368fdd90 tx=0x7f603c004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3228735332b585bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:51:46.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6026ffd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:51:46.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6026ffd700 1 -- 172.21.15.161:0/849824440 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c017040 con 0x558036900be0 2023-07-14T11:51:46.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6026ffd700 1 -- 172.21.15.161:0/849824440 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f603c0107a0 con 0x558036900be0 2023-07-14T11:51:46.815 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f604001e460 con 0x558036900be0 2023-07-14T11:51:46.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6026ffd700 1 -- 172.21.15.161:0/849824440 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f603c0057d0 con 0x558036900be0 2023-07-14T11:51:46.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6061ac1b80 10 client.28093 fetch_fsmap learned FSMap version 46 2023-07-14T11:51:46.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.821+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f604810d170 con 0x558036900be0 2023-07-14T11:51:46.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6026ffd700 1 -- 172.21.15.161:0/849824440 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604001e460 con 0x558036900be0 2023-07-14T11:51:46.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6026ffd700 1 -- 172.21.15.161:0/849824440 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f603c01cba0 con 0x558036900be0 2023-07-14T11:51:46.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6061ac1b80 10 client.28093 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:51:46.818 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6061ac1b80 10 client.28093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:51:46.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6061ac1b80 20 client.28093 populate_metadata read hostname 'smithi161' 2023-07-14T11:51:46.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5580369701a0 0x558036972580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:51:46.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6061ac1b80 4 client.28093 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:51:46.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55803680e050 con 0x5580369701a0 2023-07-14T11:51:46.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f60477fe700 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5580369701a0 0x558036972580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:51:46.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f6026ffd700 10 client.28093 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:46.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.822+0000 7f60477fe700 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5580369701a0 0x558036972580 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46565276815eeaf server_cookie=6f0e0e904b85e16e in_seq=0 out_seq=0 2023-07-14T11:51:46.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6026ffd700 1 -- 172.21.15.161:0/849824440 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55803680e050 con 0x5580369701a0 2023-07-14T11:51:46.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6026ffd700 10 client.28093 handle_command_reply: tid=0 2023-07-14T11:51:46.820 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 client.28093 shutdown 2023-07-14T11:51:46.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 msgr2=0x5580369104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369104c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5580368fdd90 tx=0x7f603c004a00 comp rx=0 tx=0).stop 2023-07-14T11:51:46.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 shutdown_connections 2023-07-14T11:51:46.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5580369701a0 0x558036972580 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.821 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580369002d0 0x558036912200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55803690bad0 0x558036910a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 --2- 172.21.15.161:0/849824440 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x558036900be0 0x5580369104c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6026ffd700 0 client.28093 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:51:46.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 >> 172.21.15.161:0/849824440 conn(0x5580368f6da0 msgr2=0x5580369039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:46.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 shutdown_connections 2023-07-14T11:51:46.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 1 -- 172.21.15.161:0/849824440 wait complete. 2023-07-14T11:51:46.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.823+0000 7f6061ac1b80 20 client.28093 trim_cache size 0 max 16384 2023-07-14T11:51:46.823 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:51:46.823 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:51:46.824 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:51:46.824 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:51:46.824 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:51:46.824 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:51:46.825 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:51:46.825 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:51:46.825 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:51:46.825 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:51:46.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60300a1340 msgr2=0x7f60300a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60300a1340 0x7f60300a37f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f604002a000 tx=0x7f6040027000 comp rx=0 tx=0).stop 2023-07-14T11:51:46.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6048100ca0 msgr2=0x7f6048197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:51:46.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6048100ca0 0x7f6048197fb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f603800eb30 tx=0x7f603800ee40 comp rx=0 tx=0).stop 2023-07-14T11:51:46.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 shutdown_connections 2023-07-14T11:51:46.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f60300a1340 0x7f60300a37f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604810a070 0x7f604819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6048100ca0 0x7f6048197fb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 --2- 172.21.15.161:0/1870465727 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f60481015b0 0x7f60481984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:51:46.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 >> 172.21.15.161:0/1870465727 conn(0x7f60480fa4b0 msgr2=0x7f60480fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:51:46.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 shutdown_connections 2023-07-14T11:51:46.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:51:46.824+0000 7f604e1ff700 1 -- 172.21.15.161:0/1870465727 wait complete. 2023-07-14T11:51:46.830 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:51:46.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:46 smithi182 ceph-mon[129366]: pgmap v934: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:46.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:46 smithi161 ceph-mon[95526]: pgmap v934: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:47.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:46 smithi175 ceph-mon[129081]: pgmap v934: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:48.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:48 smithi182 ceph-mon[129366]: pgmap v935: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:48.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:48 smithi161 ceph-mon[95526]: pgmap v935: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:49.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:48 smithi175 ceph-mon[129081]: pgmap v935: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:50.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:50 smithi182 ceph-mon[129366]: pgmap v936: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:50.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:50 smithi161 ceph-mon[95526]: pgmap v936: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:51.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:50 smithi175 ceph-mon[129081]: pgmap v936: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:52.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:52 smithi182 ceph-mon[129366]: pgmap v937: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:52.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:52 smithi161 ceph-mon[95526]: pgmap v937: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:53.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:52 smithi175 ceph-mon[129081]: pgmap v937: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T11:51:54.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:54 smithi175 ceph-mon[129081]: pgmap v938: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:54.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:54 smithi182 ceph-mon[129366]: pgmap v938: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:54.728 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:54 smithi161 ceph-mon[95526]: pgmap v938: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T11:51:56.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:56 smithi182 ceph-mon[129366]: pgmap v939: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:56.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:56 smithi161 ceph-mon[95526]: pgmap v939: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:57.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:56 smithi175 ceph-mon[129081]: pgmap v939: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:51:58.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:51:58 smithi182 ceph-mon[129366]: pgmap v940: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:58.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:51:58 smithi161 ceph-mon[95526]: pgmap v940: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:51:59.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:51:58 smithi175 ceph-mon[129081]: pgmap v940: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:00.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:00 smithi182 ceph-mon[129366]: pgmap v941: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:00.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:00 smithi161 ceph-mon[95526]: pgmap v941: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:01.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:00 smithi175 ceph-mon[129081]: pgmap v941: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:02.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:02 smithi182 ceph-mon[129366]: pgmap v942: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:52:02.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:02 smithi161 ceph-mon[95526]: pgmap v942: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:52:03.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:02 smithi175 ceph-mon[129081]: pgmap v942: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-07-14T11:52:04.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:04 smithi175 ceph-mon[129081]: pgmap v943: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:52:04.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:04 smithi182 ceph-mon[129366]: pgmap v943: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:52:04.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:04 smithi161 ceph-mon[95526]: pgmap v943: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-14T11:52:06.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:06 smithi182 ceph-mon[129366]: pgmap v944: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:52:06.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:06 smithi161 ceph-mon[95526]: pgmap v944: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:52:07.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:06 smithi175 ceph-mon[129081]: pgmap v944: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:52:08.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:08 smithi182 ceph-mon[129366]: pgmap v945: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:52:08.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:08 smithi161 ceph-mon[95526]: pgmap v945: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:52:09.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:08 smithi175 ceph-mon[129081]: pgmap v945: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:52:10.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:10 smithi182 ceph-mon[129366]: pgmap v946: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:52:10.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:10 smithi161 ceph-mon[95526]: pgmap v946: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:52:11.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:10 smithi175 ceph-mon[129081]: pgmap v946: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:52:12.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:12 smithi182 ceph-mon[129366]: pgmap v947: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-14T11:52:12.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:12 smithi182 ceph-mon[129366]: osdmap e140: 12 total, 12 up, 12 in 2023-07-14T11:52:12.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:12 smithi161 ceph-mon[95526]: pgmap v947: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-14T11:52:12.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:12 smithi161 ceph-mon[95526]: osdmap e140: 12 total, 12 up, 12 in 2023-07-14T11:52:13.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:12 smithi175 ceph-mon[129081]: pgmap v947: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-14T11:52:13.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:12 smithi175 ceph-mon[129081]: osdmap e140: 12 total, 12 up, 12 in 2023-07-14T11:52:14.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:14 smithi175 ceph-mon[129081]: pgmap v949: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:52:14.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:14 smithi182 ceph-mon[129366]: pgmap v949: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:52:14.727 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:14 smithi161 ceph-mon[95526]: pgmap v949: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:52:16.832 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:52:16.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:16 smithi182 ceph-mon[129366]: pgmap v950: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:52:16.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:16 smithi161 ceph-mon[95526]: pgmap v950: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:52:17.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 -- 172.21.15.161:0/2381193472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8105ae0 msgr2=0x7f6ce8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/2381193472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8105ae0 0x7f6ce8105eb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40098f0 tx=0x7f6ce4009c00 comp rx=0 tx=0).stop 2023-07-14T11:52:17.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 -- 172.21.15.161:0/2381193472 shutdown_connections 2023-07-14T11:52:17.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/2381193472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8105ae0 0x7f6ce8105eb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/2381193472 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ce8100480 0x7f6ce810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/2381193472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 0x7f6ce80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.047 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 -- 172.21.15.161:0/2381193472 >> 172.21.15.161:0/2381193472 conn(0x7f6ce80fb5f0 msgr2=0x7f6ce80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:17.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 -- 172.21.15.161:0/2381193472 shutdown_connections 2023-07-14T11:52:17.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.062+0000 7f6cf07ff700 1 -- 172.21.15.161:0/2381193472 wait complete. 2023-07-14T11:52:17.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.063+0000 7f6cf07ff700 1 Processor -- start 2023-07-14T11:52:17.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cf07ff700 1 -- start start 2023-07-14T11:52:17.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cf07ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 0x7f6ce8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cf07ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8100480 0x7f6ce8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cf07ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ce8105ae0 0x7f6ce819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cf07ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce819ce00 con 0x7f6ce8100480 2023-07-14T11:52:17.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cf07ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce819cf70 con 0x7f6ce80ffad0 2023-07-14T11:52:17.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cf07ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce819d130 con 0x7f6ce8105ae0 2023-07-14T11:52:17.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cee59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 0x7f6ce8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cedd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8100480 0x7f6ce8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6ceed9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ce8105ae0 0x7f6ce819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cee59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 0x7f6ce8197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45558/0 (socket says 172.21.15.161:45558) 2023-07-14T11:52:17.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cee59b700 1 -- 172.21.15.161:0/612440872 learned_addr learned my addr 172.21.15.161:0/612440872 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:17.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cee59b700 1 -- 172.21.15.161:0/612440872 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ce8105ae0 msgr2=0x7f6ce819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cee59b700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ce8105ae0 0x7f6ce819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cee59b700 1 -- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8100480 msgr2=0x7f6ce8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.064+0000 7f6cee59b700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8100480 0x7f6ce8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.065+0000 7f6cee59b700 1 -- 172.21.15.161:0/612440872 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80095f0 con 0x7f6ce80ffad0 2023-07-14T11:52:17.053 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.065+0000 7f6cee59b700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 0x7f6ce8197e20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009810 tx=0x7f6ce400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:17.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.065+0000 7f6cdf7fe700 1 -- 172.21.15.161:0/612440872 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce401c070 con 0x7f6ce80ffad0 2023-07-14T11:52:17.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.065+0000 7f6cdf7fe700 1 -- 172.21.15.161:0/612440872 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce400bbd0 con 0x7f6ce80ffad0 2023-07-14T11:52:17.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.065+0000 7f6cdf7fe700 1 -- 172.21.15.161:0/612440872 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4004930 con 0x7f6ce80ffad0 2023-07-14T11:52:17.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.065+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce0009c40 con 0x7f6ce80ffad0 2023-07-14T11:52:17.054 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.065+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce4009520 con 0x7f6ce80ffad0 2023-07-14T11:52:17.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.068+0000 7f6cdf7fe700 1 -- 172.21.15.161:0/612440872 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6ce4020070 con 0x7f6ce80ffad0 2023-07-14T11:52:17.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 Processor -- start 2023-07-14T11:52:17.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 -- start start 2023-07-14T11:52:17.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6cdf7fe700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cd40a1340 0x7f6cd40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6cdf7fe700 1 -- 172.21.15.161:0/612440872 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f6ce40cdcf0 con 0x7f6ce80ffad0 2023-07-14T11:52:17.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380a9b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9bbe0 0x565380aa6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380aa8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653809a1cb0 con 0x565380a9b2d0 2023-07-14T11:52:17.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653809f4a40 con 0x565380a9bbe0 2023-07-14T11:52:17.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6d04051b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653809eded0 con 0x565380aa6ad0 2023-07-14T11:52:17.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6ceed9c700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cd40a1340 0x7f6cd40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6cedd9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9bbe0 0x565380aa6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.070+0000 7f6cee59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380a9b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9bbe0 0x565380aa6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45564/0 (socket says 172.21.15.161:45564) 2023-07-14T11:52:17.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cee59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380a9b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46314/0 (socket says 172.21.15.161:46314) 2023-07-14T11:52:17.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1945385151 learned_addr learned my addr 172.21.15.161:0/1945385151 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:17.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6ceed9c700 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380aa8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.061 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1945385151 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 msgr2=0x565380aa8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380aa8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1945385151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 msgr2=0x565380a9b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380a9b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1945385151 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56538081bde0 con 0x565380a9bbe0 2023-07-14T11:52:17.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9bbe0 0x565380aa6590 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009f90 tx=0x7f6cd8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6545dae5853666f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:17.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6ceed9c700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cd40a1340 0x7f6cd40a37f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009fd0 tx=0x7f6ce000fdd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:17.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cdcff9700 1 -- 172.21.15.161:0/1945385151 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8004310 con 0x565380a9bbe0 2023-07-14T11:52:17.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cdcff9700 1 -- 172.21.15.161:0/1945385151 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd8004470 con 0x565380a9bbe0 2023-07-14T11:52:17.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.071+0000 7f6cdcff9700 1 -- 172.21.15.161:0/1945385151 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8019940 con 0x565380a9bbe0 2023-07-14T11:52:17.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 -- 172.21.15.161:0/1945385151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9bbe0 msgr2=0x565380aa6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9bbe0 0x565380aa6590 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009f90 tx=0x7f6cd8011720 comp rx=0 tx=0).stop 2023-07-14T11:52:17.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 -- 172.21.15.161:0/1945385151 shutdown_connections 2023-07-14T11:52:17.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380a9b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380aa8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1945385151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9bbe0 0x565380aa6590 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 -- 172.21.15.161:0/1945385151 >> 172.21.15.161:0/1945385151 conn(0x565380a91da0 msgr2=0x565380a976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:17.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 -- 172.21.15.161:0/1945385151 shutdown_connections 2023-07-14T11:52:17.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.072+0000 7f6d04051b80 1 -- 172.21.15.161:0/1945385151 wait complete. 2023-07-14T11:52:17.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 -- start start 2023-07-14T11:52:17.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 Processor -- start 2023-07-14T11:52:17.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380aa6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aa3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380aa6ad0 0x565380aa3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538098dac0 con 0x565380a9b2d0 2023-07-14T11:52:17.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538098d7c0 con 0x565380aa6ad0 2023-07-14T11:52:17.066 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x565380871920 con 0x565380a9bbe0 2023-07-14T11:52:17.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.073+0000 7f6d04051b80 10 client.0 _ll_register_callbacks cb 0x56538098b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:52:17.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aa3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aa3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:43672/0 (socket says 172.21.15.161:43672) 2023-07-14T11:52:17.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1128612427 learned_addr learned my addr 172.21.15.161:0/1128612427 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:17.067 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6ceed9c700 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380aa6ad0 0x565380aa3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cee59b700 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380aa6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380aa6ad0 msgr2=0x565380aa3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380aa6ad0 0x565380aa3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1128612427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 msgr2=0x565380aa6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.068 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380aa6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 -- 172.21.15.161:0/1128612427 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd8009bc0 con 0x565380a9bbe0 2023-07-14T11:52:17.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aa3790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8005950 tx=0x7f6cd80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f649801e4dd27315 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:17.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:52:17.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/1128612427 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8003890 con 0x565380a9bbe0 2023-07-14T11:52:17.069 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/1128612427 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f6cd80039f0 con 0x565380a9bbe0 2023-07-14T11:52:17.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.074+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653809474f0 con 0x565380a9bbe0 2023-07-14T11:52:17.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/1128612427 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd8003bb0 con 0x565380a9bbe0 2023-07-14T11:52:17.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 10 client.28105 fetch_fsmap learned FSMap version 46 2023-07-14T11:52:17.070 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x565380804c60 con 0x565380a9bbe0 2023-07-14T11:52:17.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/1128612427 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653809474f0 con 0x565380a9bbe0 2023-07-14T11:52:17.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/1128612427 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6cd801a560 con 0x565380a9bbe0 2023-07-14T11:52:17.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 10 client.28105 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:52:17.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 10 client.28105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:52:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 20 client.28105 populate_metadata read hostname 'smithi161' 2023-07-14T11:52:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380ab63c0 0x565380ab87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 4 client.28105 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:52:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653809e6eb0 con 0x565380ab63c0 2023-07-14T11:52:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.075+0000 7f6cee59b700 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380ab63c0 0x565380ab87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.076+0000 7f6cd1ffb700 10 client.28105 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:17.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.076+0000 7f6cee59b700 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380ab63c0 0x565380ab87d0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebc353c51a8196a server_cookie=c94188ecb605f8ca in_seq=0 out_seq=0 2023-07-14T11:52:17.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.076+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/1128612427 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5653809e6eb0 con 0x565380ab63c0 2023-07-14T11:52:17.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.076+0000 7f6cd1ffb700 10 client.28105 handle_command_reply: tid=0 2023-07-14T11:52:17.074 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 client.28105 shutdown 2023-07-14T11:52:17.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 msgr2=0x565380aa3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aa3790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8005950 tx=0x7f6cd80057f0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 shutdown_connections 2023-07-14T11:52:17.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380ab63c0 0x565380ab87d0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.075 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9b2d0 0x565380aa6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380aa6ad0 0x565380aa3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6cd1ffb700 0 client.28105 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 --2- 172.21.15.161:0/1128612427 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aa3790 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 >> 172.21.15.161:0/1128612427 conn(0x565380a91da0 msgr2=0x565380a97230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 shutdown_connections 2023-07-14T11:52:17.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 1 -- 172.21.15.161:0/1128612427 wait complete. 2023-07-14T11:52:17.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.077+0000 7f6d04051b80 20 client.28105 trim_cache size 0 max 16384 2023-07-14T11:52:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 Processor -- start 2023-07-14T11:52:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 -- start start 2023-07-14T11:52:17.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9bbe0 0x565380a9da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380a9dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565380a91970 con 0x565380a9bbe0 2023-07-14T11:52:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565380a9cdd0 con 0x565380a9b2d0 2023-07-14T11:52:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.094+0000 7f6d04051b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x565380aa1cf0 con 0x565380aa6ad0 2023-07-14T11:52:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6ceed9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380a9dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:45574/0 (socket says 172.21.15.161:45574) 2023-07-14T11:52:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6ceed9c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380a9dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:43682/0 (socket says 172.21.15.161:43682) 2023-07-14T11:52:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 -- 172.21.15.161:0/4255010997 learned_addr learned my addr 172.21.15.161:0/4255010997 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9bbe0 0x565380a9da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 -- 172.21.15.161:0/4255010997 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 msgr2=0x565380a9dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380a9dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 -- 172.21.15.161:0/4255010997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9bbe0 msgr2=0x565380a9da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9bbe0 0x565380a9da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 -- 172.21.15.161:0/4255010997 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565380842250 con 0x565380a9b2d0 2023-07-14T11:52:17.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cee59b700 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad4f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4005bf0 tx=0x7f6ce40a0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0edb89c6a32877a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:17.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cdd7fa700 1 -- 172.21.15.161:0/4255010997 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40b3180 con 0x565380a9b2d0 2023-07-14T11:52:17.085 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cdd7fa700 1 -- 172.21.15.161:0/4255010997 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce40a2dd0 con 0x565380a9b2d0 2023-07-14T11:52:17.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.095+0000 7f6cdd7fa700 1 -- 172.21.15.161:0/4255010997 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce409ac70 con 0x565380a9b2d0 2023-07-14T11:52:17.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 -- 172.21.15.161:0/4255010997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 msgr2=0x565380aad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad4f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4005bf0 tx=0x7f6ce40a0040 comp rx=0 tx=0).stop 2023-07-14T11:52:17.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 -- 172.21.15.161:0/4255010997 shutdown_connections 2023-07-14T11:52:17.086 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380aa6ad0 0x565380a9dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad4f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 --2- 172.21.15.161:0/4255010997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380a9bbe0 0x565380a9da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 -- 172.21.15.161:0/4255010997 >> 172.21.15.161:0/4255010997 conn(0x565380a91da0 msgr2=0x565380a9d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:17.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 -- 172.21.15.161:0/4255010997 shutdown_connections 2023-07-14T11:52:17.087 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.096+0000 7f6d04051b80 1 -- 172.21.15.161:0/4255010997 wait complete. 2023-07-14T11:52:17.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 -- start start 2023-07-14T11:52:17.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 Processor -- start 2023-07-14T11:52:17.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.088 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380aa6ad0 0x565380aac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565380aac910 con 0x565380aa6ad0 2023-07-14T11:52:17.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x565380b0a270 con 0x565380a9b2d0 2023-07-14T11:52:17.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x565380b0a3b0 con 0x565380a9bbe0 2023-07-14T11:52:17.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6d04051b80 10 client.0 _ll_register_callbacks cb 0x7f6ce4005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:52:17.089 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6cedd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6cee59b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6cedd9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aabb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:43690/0 (socket says 172.21.15.161:43690) 2023-07-14T11:52:17.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.097+0000 7f6cedd9a700 1 -- 172.21.15.161:0/2666980154 learned_addr learned my addr 172.21.15.161:0/2666980154 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:17.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6ceed9c700 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380aa6ad0 0x565380aac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cedd9a700 1 -- 172.21.15.161:0/2666980154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 msgr2=0x565380aad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cedd9a700 1 -- 172.21.15.161:0/2666980154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380aa6ad0 msgr2=0x565380aac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:17.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380aa6ad0 0x565380aac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cedd9a700 1 -- 172.21.15.161:0/2666980154 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce409be70 con 0x565380a9bbe0 2023-07-14T11:52:17.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cedd9a700 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aabb10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8004440 tx=0x7f6cd801a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf08ee176a91e35 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:17.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:52:17.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/2666980154 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8009ee0 con 0x565380a9bbe0 2023-07-14T11:52:17.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cd8004470 con 0x565380a9bbe0 2023-07-14T11:52:17.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/2666980154 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f6cd8000e40 con 0x565380a9bbe0 2023-07-14T11:52:17.092 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.098+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/2666980154 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd801abd0 con 0x565380a9bbe0 2023-07-14T11:52:17.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 10 client.28111 fetch_fsmap learned FSMap version 46 2023-07-14T11:52:17.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56538081a2b0 con 0x565380a9bbe0 2023-07-14T11:52:17.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/2666980154 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd8004470 con 0x565380a9bbe0 2023-07-14T11:52:17.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/2666980154 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6cd80045e0 con 0x565380a9bbe0 2023-07-14T11:52:17.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 10 client.28111 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:52:17.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 10 client.28111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:52:17.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 20 client.28111 populate_metadata read hostname 'smithi161' 2023-07-14T11:52:17.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380b0b700 0x565380b0dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:17.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 4 client.28111 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:52:17.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5653809a9050 con 0x565380b0b700 2023-07-14T11:52:17.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.099+0000 7f6cee59b700 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380b0b700 0x565380b0dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:17.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6cd1ffb700 10 client.28111 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:17.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6cee59b700 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380b0b700 0x565380b0dae0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4858c924d89922db server_cookie=4870d054ab38c6d4 in_seq=0 out_seq=0 2023-07-14T11:52:17.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6cd1ffb700 1 -- 172.21.15.161:0/2666980154 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5653809a9050 con 0x565380b0b700 2023-07-14T11:52:17.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6cd1ffb700 10 client.28111 handle_command_reply: tid=0 2023-07-14T11:52:17.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 client.28111 shutdown 2023-07-14T11:52:17.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 msgr2=0x565380aabb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aabb10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8004440 tx=0x7f6cd801a6f0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 shutdown_connections 2023-07-14T11:52:17.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x565380b0b700 0x565380b0dae0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565380aa6ad0 0x565380aac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565380a9b2d0 0x565380aad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 --2- 172.21.15.161:0/2666980154 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x565380a9bbe0 0x565380aabb10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6cd1ffb700 0 client.28111 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:17.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 >> 172.21.15.161:0/2666980154 conn(0x565380a91da0 msgr2=0x565380a9dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:17.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 shutdown_connections 2023-07-14T11:52:17.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 1 -- 172.21.15.161:0/2666980154 wait complete. 2023-07-14T11:52:17.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.100+0000 7f6d04051b80 20 client.28111 trim_cache size 0 max 16384 2023-07-14T11:52:17.098 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:52:17.099 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:52:17.099 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:52:17.099 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:52:17.099 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:52:17.099 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:52:17.100 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:52:17.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:52:17.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:52:17.100 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:52:17.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cd40a1340 msgr2=0x7f6cd40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cd40a1340 0x7f6cd40a37f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009fd0 tx=0x7f6ce000fdd0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 msgr2=0x7f6ce8197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:17.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 0x7f6ce8197e20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009810 tx=0x7f6ce400fa00 comp rx=0 tx=0).stop 2023-07-14T11:52:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 shutdown_connections 2023-07-14T11:52:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f6cd40a1340 0x7f6cd40a37f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f6ce8105ae0 0x7f6ce819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80ffad0 0x7f6ce8197e20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 --2- 172.21.15.161:0/612440872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ce8100480 0x7f6ce8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 >> 172.21.15.161:0/612440872 conn(0x7f6ce80fb5f0 msgr2=0x7f6ce810adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 shutdown_connections 2023-07-14T11:52:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:17.101+0000 7f6cf07ff700 1 -- 172.21.15.161:0/612440872 wait complete. 2023-07-14T11:52:17.104 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:52:17.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:16 smithi175 ceph-mon[129081]: pgmap v950: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:52:18.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:18 smithi182 ceph-mon[129366]: pgmap v951: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 7 op/s 2023-07-14T11:52:18.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:18 smithi182 ceph-mon[129366]: osdmap e141: 12 total, 12 up, 12 in 2023-07-14T11:52:18.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:18 smithi161 ceph-mon[95526]: pgmap v951: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 7 op/s 2023-07-14T11:52:18.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:18 smithi161 ceph-mon[95526]: osdmap e141: 12 total, 12 up, 12 in 2023-07-14T11:52:19.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:18 smithi175 ceph-mon[129081]: pgmap v951: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 7 op/s 2023-07-14T11:52:19.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:18 smithi175 ceph-mon[129081]: osdmap e141: 12 total, 12 up, 12 in 2023-07-14T11:52:20.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:20 smithi182 ceph-mon[129366]: pgmap v953: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:52:20.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:20 smithi161 ceph-mon[95526]: pgmap v953: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:52:21.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:20 smithi175 ceph-mon[129081]: pgmap v953: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T11:52:22.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:22 smithi182 ceph-mon[129366]: pgmap v954: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 206 B/s wr, 7 op/s 2023-07-14T11:52:22.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:22 smithi161 ceph-mon[95526]: pgmap v954: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 206 B/s wr, 7 op/s 2023-07-14T11:52:23.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:22 smithi175 ceph-mon[129081]: pgmap v954: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 206 B/s wr, 7 op/s 2023-07-14T11:52:24.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:24 smithi175 ceph-mon[129081]: pgmap v955: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-14T11:52:24.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:24 smithi182 ceph-mon[129366]: pgmap v955: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-14T11:52:24.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:24 smithi161 ceph-mon[95526]: pgmap v955: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-14T11:52:25.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:25 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:26.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:25 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:26.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:25 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:26.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:26 smithi161 ceph-mon[95526]: pgmap v956: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-14T11:52:26.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:26.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:26.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:26.978 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:26 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:26 smithi175 ceph-mon[129081]: pgmap v956: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-14T11:52:27.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:26 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:26 smithi182 ceph-mon[129366]: pgmap v956: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-14T11:52:27.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:27.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:26 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:28 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:28 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:28 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:52:28.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:28 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:52:28.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:28 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:28 smithi175 ceph-mon[129081]: pgmap v957: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:52:28.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:28 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:28 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:28 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:52:28.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:28 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:52:28.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:28 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:28 smithi182 ceph-mon[129366]: pgmap v957: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:52:28.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:28 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:28 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:28 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:52:28.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:28 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:52:28.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:28 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:52:28.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:28 smithi161 ceph-mon[95526]: pgmap v957: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:52:30.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:30 smithi182 ceph-mon[129366]: pgmap v958: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-14T11:52:30.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:30 smithi161 ceph-mon[95526]: pgmap v958: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-14T11:52:31.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:30 smithi175 ceph-mon[129081]: pgmap v958: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-14T11:52:32.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:32 smithi182 ceph-mon[129366]: pgmap v959: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:32.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:32 smithi161 ceph-mon[95526]: pgmap v959: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:33.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:32 smithi175 ceph-mon[129081]: pgmap v959: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:34.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:34 smithi175 ceph-mon[129081]: pgmap v960: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:34.695 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:34 smithi182 ceph-mon[129366]: pgmap v960: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:34.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:34 smithi161 ceph-mon[95526]: pgmap v960: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:36.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:36 smithi182 ceph-mon[129366]: pgmap v961: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:36.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:36 smithi161 ceph-mon[95526]: pgmap v961: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:37.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:36 smithi175 ceph-mon[129081]: pgmap v961: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:38.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:38 smithi182 ceph-mon[129366]: pgmap v962: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:38.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:38 smithi161 ceph-mon[95526]: pgmap v962: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:39.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:38 smithi175 ceph-mon[129081]: pgmap v962: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:40.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:40 smithi161 ceph-mon[95526]: pgmap v963: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:41.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:40 smithi175 ceph-mon[129081]: pgmap v963: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:41.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:40 smithi182 ceph-mon[129366]: pgmap v963: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:42.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:42 smithi161 ceph-mon[95526]: pgmap v964: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:43.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:42 smithi175 ceph-mon[129081]: pgmap v964: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:43.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:42 smithi182 ceph-mon[129366]: pgmap v964: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:44.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:44 smithi175 ceph-mon[129081]: pgmap v965: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:44.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:44 smithi182 ceph-mon[129366]: pgmap v965: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:44.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:44 smithi161 ceph-mon[95526]: pgmap v965: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:46.945 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:46 smithi182 ceph-mon[129366]: pgmap v966: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:46.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:46 smithi161 ceph-mon[95526]: pgmap v966: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:47.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:52:47.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:46 smithi175 ceph-mon[129081]: pgmap v966: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:47.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 -- 172.21.15.161:0/1272579077 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 msgr2=0x7f2ecc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/1272579077 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 0x7f2ecc072c80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009990 tx=0x7f2ebc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 -- 172.21.15.161:0/1272579077 shutdown_connections 2023-07-14T11:52:47.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/1272579077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ecc10c310 0x7f2ecc10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/1272579077 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 0x7f2ecc072c80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/1272579077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 0x7f2ecc10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 -- 172.21.15.161:0/1272579077 >> 172.21.15.161:0/1272579077 conn(0x7f2ecc0fb5f0 msgr2=0x7f2ecc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 -- 172.21.15.161:0/1272579077 shutdown_connections 2023-07-14T11:52:47.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.351+0000 7f2ed0fff700 1 -- 172.21.15.161:0/1272579077 wait complete. 2023-07-14T11:52:47.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 Processor -- start 2023-07-14T11:52:47.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 -- start start 2023-07-14T11:52:47.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 0x7f2ecc197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 0x7f2ecc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ecc10c310 0x7f2ecc19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc19ce30 con 0x7f2ecc10c310 2023-07-14T11:52:47.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc19cfa0 con 0x7f2ecc0731c0 2023-07-14T11:52:47.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.352+0000 7f2ed0fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc19d160 con 0x7f2ecc0728b0 2023-07-14T11:52:47.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 0x7f2ecc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 0x7f2ecc197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 0x7f2ecc198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47764/0 (socket says 172.21.15.161:47764) 2023-07-14T11:52:47.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3388690521 learned_addr learned my addr 172.21.15.161:0/3388690521 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:47.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ecc10c310 0x7f2ecc19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3388690521 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 msgr2=0x7f2ecc197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 0x7f2ecc197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ecc10c310 msgr2=0x7f2ecc19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ecc10c310 0x7f2ecc19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3388690521 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0095f0 con 0x7f2ecc0731c0 2023-07-14T11:52:47.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.353+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 0x7f2ecc198390 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400e9a0 tx=0x7f2eb400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.354+0000 7f2ec37fe700 1 -- 172.21.15.161:0/3388690521 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4009790 con 0x7f2ecc0731c0 2023-07-14T11:52:47.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.354+0000 7f2ec37fe700 1 -- 172.21.15.161:0/3388690521 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eb4010430 con 0x7f2ecc0731c0 2023-07-14T11:52:47.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.354+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc19d440 con 0x7f2ecc0731c0 2023-07-14T11:52:47.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.354+0000 7f2ec37fe700 1 -- 172.21.15.161:0/3388690521 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4018680 con 0x7f2ecc0731c0 2023-07-14T11:52:47.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.354+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc1a4c80 con 0x7f2ecc0731c0 2023-07-14T11:52:47.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.357+0000 7f2ec37fe700 1 -- 172.21.15.161:0/3388690521 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2eb40105a0 con 0x7f2ecc0731c0 2023-07-14T11:52:47.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 Processor -- start 2023-07-14T11:52:47.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 -- start start 2023-07-14T11:52:47.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ec37fe700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2eb80a12c0 0x7f2eb80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba145316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba1453c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.345 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba1453eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14437cb0 con 0x55ba1453cad0 2023-07-14T11:52:47.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1448aa40 con 0x55ba14531be0 2023-07-14T11:52:47.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.358+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14483ed0 con 0x55ba145312d0 2023-07-14T11:52:47.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2ec37fe700 1 -- 172.21.15.161:0/3388690521 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f2eb4014070 con 0x7f2ecc0731c0 2023-07-14T11:52:47.346 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba145316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2eb80a12c0 0x7f2eb80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba145316a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:37278/0 (socket says 172.21.15.161:37278) 2023-07-14T11:52:47.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 -- 172.21.15.161:0/4174321961 learned_addr learned my addr 172.21.15.161:0/4174321961 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:47.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2ec3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba1453c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba1453eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.347 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 -- 172.21.15.161:0/4174321961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 msgr2=0x55ba1453c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba1453c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 -- 172.21.15.161:0/4174321961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 msgr2=0x55ba1453eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba1453eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 -- 172.21.15.161:0/4174321961 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba142b1de0 con 0x55ba145312d0 2023-07-14T11:52:47.348 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba1453c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:52:47.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2eb80a12c0 0x7f2eb80a3770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009ec0 tx=0x7f2ec4009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.349 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.359+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba145316a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc005ea0 tx=0x7f2ebc005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d88fd668b44f23fb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ec0ff9700 1 -- 172.21.15.161:0/4174321961 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc01d070 con 0x55ba145312d0 2023-07-14T11:52:47.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ec0ff9700 1 -- 172.21.15.161:0/4174321961 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ebc01e040 con 0x55ba145312d0 2023-07-14T11:52:47.350 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ec0ff9700 1 -- 172.21.15.161:0/4174321961 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc004930 con 0x55ba145312d0 2023-07-14T11:52:47.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4174321961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 msgr2=0x55ba145316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba145316a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc005ea0 tx=0x7f2ebc005ed0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4174321961 shutdown_connections 2023-07-14T11:52:47.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba1453eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.351 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba145316a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4174321961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba1453c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.360+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4174321961 >> 172.21.15.161:0/4174321961 conn(0x55ba14527da0 msgr2=0x55ba1452d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.361+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4174321961 shutdown_connections 2023-07-14T11:52:47.352 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.361+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4174321961 wait complete. 2023-07-14T11:52:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.361+0000 7f2ee486cb80 1 -- start start 2023-07-14T11:52:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.361+0000 7f2ee486cb80 1 Processor -- start 2023-07-14T11:52:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14539740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14423ac0 con 0x55ba1453cad0 2023-07-14T11:52:47.356 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba144237c0 con 0x55ba14531be0 2023-07-14T11:52:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14307920 con 0x55ba145312d0 2023-07-14T11:52:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ee486cb80 10 client.0 _ll_register_callbacks cb 0x55ba14421760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:52:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:37290/0 (socket says 172.21.15.161:37290) 2023-07-14T11:52:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 -- 172.21.15.161:0/4242459998 learned_addr learned my addr 172.21.15.161:0/4242459998 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:47.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14539740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 -- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 msgr2=0x55ba14539740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14539740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 -- 172.21.15.161:0/4242459998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 msgr2=0x55ba14539d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.362+0000 7f2eca59c700 1 -- 172.21.15.161:0/4242459998 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc1a4c80 con 0x55ba145312d0 2023-07-14T11:52:47.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453c450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00bb30 tx=0x7f2ebc022b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af841bc7567255d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2eb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:52:47.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4242459998 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc01d030 con 0x55ba145312d0 2023-07-14T11:52:47.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba143dd4f0 con 0x55ba145312d0 2023-07-14T11:52:47.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4242459998 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f2ebc00f460 con 0x55ba145312d0 2023-07-14T11:52:47.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4242459998 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ebc0056b0 con 0x55ba145312d0 2023-07-14T11:52:47.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2ee486cb80 10 client.28123 fetch_fsmap learned FSMap version 46 2023-07-14T11:52:47.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f2ecc10f390 con 0x55ba145312d0 2023-07-14T11:52:47.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.363+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4242459998 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba143dd4f0 con 0x55ba145312d0 2023-07-14T11:52:47.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4242459998 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ebc02c400 con 0x55ba145312d0 2023-07-14T11:52:47.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ee486cb80 10 client.28123 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:52:47.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ee486cb80 10 client.28123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:52:47.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ee486cb80 20 client.28123 populate_metadata read hostname 'smithi161' 2023-07-14T11:52:47.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba1454bfd0 0x55ba1454e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ee486cb80 4 client.28123 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:52:47.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba1447ceb0 con 0x55ba1454bfd0 2023-07-14T11:52:47.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba1454bfd0 0x55ba1454e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2eb1ffb700 10 client.28123 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:47.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.364+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba1454bfd0 0x55ba1454e3e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada58dd36970f8f2 server_cookie=882cc6abc69d874c in_seq=0 out_seq=0 2023-07-14T11:52:47.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4242459998 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba1447ceb0 con 0x55ba1454bfd0 2023-07-14T11:52:47.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2eb1ffb700 10 client.28123 handle_command_reply: tid=0 2023-07-14T11:52:47.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 client.28123 shutdown 2023-07-14T11:52:47.363 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 msgr2=0x55ba1453c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453c450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00bb30 tx=0x7f2ebc022b00 comp rx=0 tx=0).stop 2023-07-14T11:52:47.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 shutdown_connections 2023-07-14T11:52:47.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba1454bfd0 0x55ba1454e3e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2eb1ffb700 0 client.28123 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:47.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453c450 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.365+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4242459998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14539740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.366+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 >> 172.21.15.161:0/4242459998 conn(0x55ba14527da0 msgr2=0x55ba1452d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.366+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 shutdown_connections 2023-07-14T11:52:47.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.366+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4242459998 wait complete. 2023-07-14T11:52:47.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.366+0000 7f2ee486cb80 20 client.28123 trim_cache size 0 max 16384 2023-07-14T11:52:47.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 Processor -- start 2023-07-14T11:52:47.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 -- start start 2023-07-14T11:52:47.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14538f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14532e70 con 0x55ba1453cad0 2023-07-14T11:52:47.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1452ead0 con 0x55ba14531be0 2023-07-14T11:52:47.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14532540 con 0x55ba145312d0 2023-07-14T11:52:47.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ec3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14538f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ec3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14538f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47794/0 (socket says 172.21.15.161:47794) 2023-07-14T11:52:47.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.383+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3529362411 learned_addr learned my addr 172.21.15.161:0/3529362411 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:47.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3529362411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 msgr2=0x55ba1453bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3529362411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 msgr2=0x55ba14539440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec3fff700 1 -- 172.21.15.161:0/3529362411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba1429ac60 con 0x55ba14531be0 2023-07-14T11:52:47.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:52:47.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14538f00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40964b0 tx=0x7f2eb400c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f32d6f228b0abaa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec17fa700 1 -- 172.21.15.161:0/3529362411 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4099fb0 con 0x55ba14531be0 2023-07-14T11:52:47.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec17fa700 1 -- 172.21.15.161:0/3529362411 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eb409dfe0 con 0x55ba14531be0 2023-07-14T11:52:47.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.384+0000 7f2ec17fa700 1 -- 172.21.15.161:0/3529362411 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb409a210 con 0x55ba14531be0 2023-07-14T11:52:47.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 -- 172.21.15.161:0/3529362411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 msgr2=0x55ba14538f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14538f00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40964b0 tx=0x7f2eb400c480 comp rx=0 tx=0).stop 2023-07-14T11:52:47.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 -- 172.21.15.161:0/3529362411 shutdown_connections 2023-07-14T11:52:47.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba1453cad0 0x55ba14539440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba145312d0 0x55ba1453bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/3529362411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba14531be0 0x55ba14538f00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 -- 172.21.15.161:0/3529362411 >> 172.21.15.161:0/3529362411 conn(0x55ba14527da0 msgr2=0x55ba14534a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 -- 172.21.15.161:0/3529362411 shutdown_connections 2023-07-14T11:52:47.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 -- 172.21.15.161:0/3529362411 wait complete. 2023-07-14T11:52:47.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 -- start start 2023-07-14T11:52:47.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.385+0000 7f2ee486cb80 1 Processor -- start 2023-07-14T11:52:47.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba145312d0 0x55ba14543380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba14531be0 0x55ba145416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ee486cb80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba1453cad0 0x55ba14541bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14542170 con 0x55ba14531be0 2023-07-14T11:52:47.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba145422b0 con 0x55ba145312d0 2023-07-14T11:52:47.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ee486cb80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba14542440 con 0x55ba1453cad0 2023-07-14T11:52:47.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ee486cb80 10 client.0 _ll_register_callbacks cb 0x7f2ec4008750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:52:47.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba145312d0 0x55ba14543380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ecad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba1453cad0 0x55ba14541bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ecad9d700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba1453cad0 0x55ba14541bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:37304/0 (socket says 172.21.15.161:37304) 2023-07-14T11:52:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ecad9d700 1 -- 172.21.15.161:0/4158647729 learned_addr learned my addr 172.21.15.161:0/4158647729 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:52:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2eca59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba145312d0 0x55ba14543380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:47798/0 (socket says 172.21.15.161:47798) 2023-07-14T11:52:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba14531be0 0x55ba145416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.386+0000 7f2eca59c700 1 -- 172.21.15.161:0/4158647729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba1453cad0 msgr2=0x55ba14541bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba1453cad0 0x55ba14541bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eca59c700 1 -- 172.21.15.161:0/4158647729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba14531be0 msgr2=0x55ba145416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba14531be0 0x55ba145416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eca59c700 1 -- 172.21.15.161:0/4158647729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc19d440 con 0x55ba145312d0 2023-07-14T11:52:47.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba14531be0 0x55ba145416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:52:47.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2ecad9d700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba1453cad0 0x55ba14541bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:52:47.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eca59c700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba145312d0 0x55ba14543380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc02c7e0 tx=0x7f2ebc02c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01bee685315779b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:52:47.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4158647729 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc01d040 con 0x55ba145312d0 2023-07-14T11:52:47.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ebc01e040 con 0x55ba145312d0 2023-07-14T11:52:47.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4158647729 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f2ebc02cad0 con 0x55ba145312d0 2023-07-14T11:52:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4158647729 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ebc022b70 con 0x55ba145312d0 2023-07-14T11:52:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2ee486cb80 10 client.28085 fetch_fsmap learned FSMap version 46 2023-07-14T11:52:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ba142d8250 con 0x55ba145312d0 2023-07-14T11:52:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4158647729 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ebc01e040 con 0x55ba145312d0 2023-07-14T11:52:47.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.387+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4158647729 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ebc0046c0 con 0x55ba145312d0 2023-07-14T11:52:47.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ee486cb80 10 client.28085 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:52:47.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ee486cb80 10 client.28085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:52:47.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ee486cb80 20 client.28085 populate_metadata read hostname 'smithi161' 2023-07-14T11:52:47.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba145a0e40 0x55ba145a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ee486cb80 4 client.28085 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:52:47.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba1443f050 con 0x55ba145a0e40 2023-07-14T11:52:47.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba145a0e40 0x55ba145a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2eb1ffb700 10 client.28085 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:47.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.388+0000 7f2ec3fff700 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba145a0e40 0x55ba145a3220 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e2539aa43598da server_cookie=d16ffb3b57f17905 in_seq=0 out_seq=0 2023-07-14T11:52:47.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2eb1ffb700 1 -- 172.21.15.161:0/4158647729 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ba1443f050 con 0x55ba145a0e40 2023-07-14T11:52:47.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2eb1ffb700 10 client.28085 handle_command_reply: tid=0 2023-07-14T11:52:47.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 client.28085 shutdown 2023-07-14T11:52:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba145312d0 msgr2=0x55ba14543380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba145312d0 0x55ba14543380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc02c7e0 tx=0x7f2ebc02c690 comp rx=0 tx=0).stop 2023-07-14T11:52:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 shutdown_connections 2023-07-14T11:52:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x55ba145a0e40 0x55ba145a3220 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x55ba1453cad0 0x55ba14541bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba145312d0 0x55ba14543380 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 --2- 172.21.15.161:0/4158647729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba14531be0 0x55ba145416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2eb1ffb700 0 client.28085 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:52:47.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 >> 172.21.15.161:0/4158647729 conn(0x55ba14527da0 msgr2=0x55ba145333e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.390 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 shutdown_connections 2023-07-14T11:52:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 1 -- 172.21.15.161:0/4158647729 wait complete. 2023-07-14T11:52:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ee486cb80 20 client.28085 trim_cache size 0 max 16384 2023-07-14T11:52:47.391 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:52:47.392 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:52:47.392 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:52:47.392 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:52:47.392 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:52:47.392 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:52:47.393 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:52:47.393 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:52:47.393 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:52:47.393 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:52:47.394 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2eb80a12c0 msgr2=0x7f2eb80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.389+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2eb80a12c0 0x7f2eb80a3770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009ec0 tx=0x7f2ec4009920 comp rx=0 tx=0).stop 2023-07-14T11:52:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 msgr2=0x7f2ecc198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 0x7f2ecc198390 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400e9a0 tx=0x7f2eb400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 shutdown_connections 2023-07-14T11:52:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2eb80a12c0 0x7f2eb80a3770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ecc10c310 0x7f2ecc19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2ecc0728b0 0x7f2ecc197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 --2- 172.21.15.161:0/3388690521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ecc0731c0 0x7f2ecc198390 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 >> 172.21.15.161:0/3388690521 conn(0x7f2ecc0fb5f0 msgr2=0x7f2ecc10e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 shutdown_connections 2023-07-14T11:52:47.397 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:52:47.390+0000 7f2ed0fff700 1 -- 172.21.15.161:0/3388690521 wait complete. 2023-07-14T11:52:47.397 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:52:48.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:48 smithi161 ceph-mon[95526]: pgmap v967: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:49.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:48 smithi175 ceph-mon[129081]: pgmap v967: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:49.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:48 smithi182 ceph-mon[129366]: pgmap v967: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:50.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:50 smithi161 ceph-mon[95526]: pgmap v968: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:51.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:50 smithi175 ceph-mon[129081]: pgmap v968: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:51.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:50 smithi182 ceph-mon[129366]: pgmap v968: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:52.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:52 smithi161 ceph-mon[95526]: pgmap v969: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:53.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:52 smithi175 ceph-mon[129081]: pgmap v969: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:53.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:52 smithi182 ceph-mon[129366]: pgmap v969: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:54.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:54 smithi175 ceph-mon[129081]: pgmap v970: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:54.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:54 smithi182 ceph-mon[129366]: pgmap v970: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:54.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:54 smithi161 ceph-mon[95526]: pgmap v970: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:56.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:56 smithi161 ceph-mon[95526]: pgmap v971: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:57.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:56 smithi175 ceph-mon[129081]: pgmap v971: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:57.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:56 smithi182 ceph-mon[129366]: pgmap v971: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:52:58.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:52:58 smithi161 ceph-mon[95526]: pgmap v972: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:59.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:52:58 smithi175 ceph-mon[129081]: pgmap v972: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:52:59.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:52:58 smithi182 ceph-mon[129366]: pgmap v972: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:00.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:00 smithi161 ceph-mon[95526]: pgmap v973: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:01.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:00 smithi175 ceph-mon[129081]: pgmap v973: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:01.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:00 smithi182 ceph-mon[129366]: pgmap v973: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:02.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:02 smithi161 ceph-mon[95526]: pgmap v974: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:03.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:02 smithi175 ceph-mon[129081]: pgmap v974: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:03.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:02 smithi182 ceph-mon[129366]: pgmap v974: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:04.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:04 smithi175 ceph-mon[129081]: pgmap v975: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:04.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:04 smithi182 ceph-mon[129366]: pgmap v975: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:04.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:04 smithi161 ceph-mon[95526]: pgmap v975: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:06.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:06 smithi161 ceph-mon[95526]: pgmap v976: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:07.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:06 smithi175 ceph-mon[129081]: pgmap v976: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:07.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:06 smithi182 ceph-mon[129366]: pgmap v976: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:08.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:08 smithi161 ceph-mon[95526]: pgmap v977: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:09.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:08 smithi175 ceph-mon[129081]: pgmap v977: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:09.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:08 smithi182 ceph-mon[129366]: pgmap v977: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:10.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:10 smithi161 ceph-mon[95526]: pgmap v978: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:11.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:10 smithi175 ceph-mon[129081]: pgmap v978: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:11.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:10 smithi182 ceph-mon[129366]: pgmap v978: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:53:12.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:12 smithi161 ceph-mon[95526]: pgmap v979: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:12.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:12 smithi161 ceph-mon[95526]: osdmap e142: 12 total, 12 up, 12 in 2023-07-14T11:53:13.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:12 smithi175 ceph-mon[129081]: pgmap v979: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:13.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:12 smithi175 ceph-mon[129081]: osdmap e142: 12 total, 12 up, 12 in 2023-07-14T11:53:13.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:12 smithi182 ceph-mon[129366]: pgmap v979: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:53:13.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:12 smithi182 ceph-mon[129366]: osdmap e142: 12 total, 12 up, 12 in 2023-07-14T11:53:14.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:14 smithi175 ceph-mon[129081]: pgmap v981: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:14.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:14 smithi182 ceph-mon[129366]: pgmap v981: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:14.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:14 smithi161 ceph-mon[95526]: pgmap v981: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:16.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:16 smithi161 ceph-mon[95526]: pgmap v982: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:17.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:16 smithi175 ceph-mon[129081]: pgmap v982: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:17.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:16 smithi182 ceph-mon[129366]: pgmap v982: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:17.398 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:53:17.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 -- 172.21.15.161:0/833392494 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c105af0 msgr2=0x7f1c6c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.611 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 --2- 172.21.15.161:0/833392494 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c105af0 0x7f1c6c105ec0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009990 tx=0x7f1c5c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 -- 172.21.15.161:0/833392494 shutdown_connections 2023-07-14T11:53:17.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 --2- 172.21.15.161:0/833392494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6c100490 0x7f1c6c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 --2- 172.21.15.161:0/833392494 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c105af0 0x7f1c6c105ec0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 --2- 172.21.15.161:0/833392494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c0ffae0 0x7f1c6c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 -- 172.21.15.161:0/833392494 >> 172.21.15.161:0/833392494 conn(0x7f1c6c0fb5e0 msgr2=0x7f1c6c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:17.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 -- 172.21.15.161:0/833392494 shutdown_connections 2023-07-14T11:53:17.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.627+0000 7f1c71bff700 1 -- 172.21.15.161:0/833392494 wait complete. 2023-07-14T11:53:17.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.628+0000 7f1c71bff700 1 Processor -- start 2023-07-14T11:53:17.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.628+0000 7f1c71bff700 1 -- start start 2023-07-14T11:53:17.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.628+0000 7f1c71bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c0ffae0 0x7f1c6c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.628+0000 7f1c71bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c100490 0x7f1c6c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c71bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6c105af0 0x7f1c6c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c71bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c19cde0 con 0x7f1c6c105af0 2023-07-14T11:53:17.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c71bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c19cf50 con 0x7f1c6c100490 2023-07-14T11:53:17.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c71bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c19d110 con 0x7f1c6c0ffae0 2023-07-14T11:53:17.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c100490 0x7f1c6c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c0ffae0 0x7f1c6c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c100490 0x7f1c6c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49416/0 (socket says 172.21.15.161:49416) 2023-07-14T11:53:17.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 -- 172.21.15.161:0/2503462747 learned_addr learned my addr 172.21.15.161:0/2503462747 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:17.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6bfff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6c105af0 0x7f1c6c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 -- 172.21.15.161:0/2503462747 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c0ffae0 msgr2=0x7f1c6c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c0ffae0 0x7f1c6c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 -- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6c105af0 msgr2=0x7f1c6c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6c105af0 0x7f1c6c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 -- 172.21.15.161:0/2503462747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0095f0 con 0x7f1c6c100490 2023-07-14T11:53:17.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6bfff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6c105af0 0x7f1c6c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:17.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.629+0000 7f1c6affd700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c100490 0x7f1c6c198340 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000e9a0 tx=0x7f1c6000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:17.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.630+0000 7f1c68ff9700 1 -- 172.21.15.161:0/2503462747 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60009790 con 0x7f1c6c100490 2023-07-14T11:53:17.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.630+0000 7f1c68ff9700 1 -- 172.21.15.161:0/2503462747 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c6000cbd0 con 0x7f1c6c100490 2023-07-14T11:53:17.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.630+0000 7f1c68ff9700 1 -- 172.21.15.161:0/2503462747 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60018680 con 0x7f1c6c100490 2023-07-14T11:53:17.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.630+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c19d3f0 con 0x7f1c6c100490 2023-07-14T11:53:17.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.630+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c1a4c30 con 0x7f1c6c100490 2023-07-14T11:53:17.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.633+0000 7f1c68ff9700 1 -- 172.21.15.161:0/2503462747 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1c6000cd40 con 0x7f1c6c100490 2023-07-14T11:53:17.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 Processor -- start 2023-07-14T11:53:17.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 -- start start 2023-07-14T11:53:17.630 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c68ff9700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c540a12c0 0x7f1c540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd76b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd776590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd776ad0 0x5649fd778eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c68ff9700 1 -- 172.21.15.161:0/2503462747 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f1c60014070 con 0x7f1c6c100490 2023-07-14T11:53:17.631 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd671cb0 con 0x5649fd76b2d0 2023-07-14T11:53:17.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd6c4a40 con 0x5649fd76bbe0 2023-07-14T11:53:17.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.634+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd6bded0 con 0x5649fd776ad0 2023-07-14T11:53:17.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6bfff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c540a12c0 0x7f1c540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd776590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd76b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd776590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:49422/0 (socket says 172.21.15.161:49422) 2023-07-14T11:53:17.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 -- 172.21.15.161:0/252787938 learned_addr learned my addr 172.21.15.161:0/252787938 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:17.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6bfff700 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd776ad0 0x5649fd778eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 -- 172.21.15.161:0/252787938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd776ad0 msgr2=0x5649fd778eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd776ad0 0x5649fd778eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 -- 172.21.15.161:0/252787938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 msgr2=0x5649fd76b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd76b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 -- 172.21.15.161:0/252787938 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649fd4ebde0 con 0x5649fd76bbe0 2023-07-14T11:53:17.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6affd700 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd776590 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1c60096ab0 tx=0x7f1c60099a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9fc6914ecbeeb09 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:17.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c6bfff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c540a12c0 0x7f1c540a3770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c64007a70 tx=0x7f1c64009c50 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:17.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c51ffb700 1 -- 172.21.15.161:0/252787938 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6009e070 con 0x5649fd76bbe0 2023-07-14T11:53:17.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c51ffb700 1 -- 172.21.15.161:0/252787938 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c600a2490 con 0x5649fd76bbe0 2023-07-14T11:53:17.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.635+0000 7f1c51ffb700 1 -- 172.21.15.161:0/252787938 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600a2000 con 0x5649fd76bbe0 2023-07-14T11:53:17.635 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 -- 172.21.15.161:0/252787938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 msgr2=0x5649fd776590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd776590 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1c60096ab0 tx=0x7f1c60099a20 comp rx=0 tx=0).stop 2023-07-14T11:53:17.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 -- 172.21.15.161:0/252787938 shutdown_connections 2023-07-14T11:53:17.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd76b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.636 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd776ad0 0x5649fd778eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/252787938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd776590 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 -- 172.21.15.161:0/252787938 >> 172.21.15.161:0/252787938 conn(0x5649fd761da0 msgr2=0x5649fd7676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:17.637 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 -- 172.21.15.161:0/252787938 shutdown_connections 2023-07-14T11:53:17.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.636+0000 7f1c854d0b80 1 -- 172.21.15.161:0/252787938 wait complete. 2023-07-14T11:53:17.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.637+0000 7f1c854d0b80 1 -- start start 2023-07-14T11:53:17.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.637+0000 7f1c854d0b80 1 Processor -- start 2023-07-14T11:53:17.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76b2d0 0x5649fd776580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd773870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd776ad0 0x5649fd773fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd65dac0 con 0x5649fd776ad0 2023-07-14T11:53:17.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd65d7c0 con 0x5649fd76bbe0 2023-07-14T11:53:17.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd541920 con 0x5649fd76b2d0 2023-07-14T11:53:17.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c854d0b80 10 client.0 _ll_register_callbacks cb 0x5649fd65b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:17.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76b2d0 0x5649fd776580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76b2d0 0x5649fd776580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47916/0 (socket says 172.21.15.161:47916) 2023-07-14T11:53:17.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd773870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 -- 172.21.15.161:0/3781790810 learned_addr learned my addr 172.21.15.161:0/3781790810 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:17.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6bfff700 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd776ad0 0x5649fd773fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 -- 172.21.15.161:0/3781790810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 msgr2=0x5649fd773870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd773870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 -- 172.21.15.161:0/3781790810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd776ad0 msgr2=0x5649fd773fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd776ad0 0x5649fd773fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.638+0000 7f1c6b7fe700 1 -- 172.21.15.161:0/3781790810 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c1a4c30 con 0x5649fd76b2d0 2023-07-14T11:53:17.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c6bfff700 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd776ad0 0x5649fd773fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:17.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c6b7fe700 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76b2d0 0x5649fd776580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c010 tx=0x7f1c5c00b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f048a4822131dc2f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:17.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c4affd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:53:17.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c4affd700 1 -- 172.21.15.161:0/3781790810 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c01d030 con 0x5649fd76b2d0 2023-07-14T11:53:17.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649fd6174f0 con 0x5649fd76b2d0 2023-07-14T11:53:17.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c4affd700 1 -- 172.21.15.161:0/3781790810 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f1c5c004700 con 0x5649fd76b2d0 2023-07-14T11:53:17.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c4affd700 1 -- 172.21.15.161:0/3781790810 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5c005720 con 0x5649fd76b2d0 2023-07-14T11:53:17.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c854d0b80 10 client.28141 fetch_fsmap learned FSMap version 46 2023-07-14T11:53:17.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.639+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5649fd4d4c60 con 0x5649fd76b2d0 2023-07-14T11:53:17.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c4affd700 1 -- 172.21.15.161:0/3781790810 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649fd6174f0 con 0x5649fd76b2d0 2023-07-14T11:53:17.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c4affd700 1 -- 172.21.15.161:0/3781790810 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c5c00fa90 con 0x5649fd76b2d0 2023-07-14T11:53:17.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c854d0b80 10 client.28141 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:53:17.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c854d0b80 10 client.28141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:53:17.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c854d0b80 20 client.28141 populate_metadata read hostname 'smithi161' 2023-07-14T11:53:17.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7864a0 0x5649fd7888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c854d0b80 4 client.28141 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:53:17.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649fd6b6eb0 con 0x5649fd7864a0 2023-07-14T11:53:17.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.640+0000 7f1c6affd700 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7864a0 0x5649fd7888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.641+0000 7f1c4affd700 10 client.28141 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:17.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.641+0000 7f1c6affd700 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7864a0 0x5649fd7888b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a00923e4f009de4 server_cookie=88f69d4849c7a6d8 in_seq=0 out_seq=0 2023-07-14T11:53:17.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.641+0000 7f1c4affd700 1 -- 172.21.15.161:0/3781790810 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649fd6b6eb0 con 0x5649fd7864a0 2023-07-14T11:53:17.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.641+0000 7f1c4affd700 10 client.28141 handle_command_reply: tid=0 2023-07-14T11:53:17.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.641+0000 7f1c854d0b80 1 client.28141 shutdown 2023-07-14T11:53:17.649 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76b2d0 msgr2=0x5649fd776580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76b2d0 0x5649fd776580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c010 tx=0x7f1c5c00b650 comp rx=0 tx=0).stop 2023-07-14T11:53:17.650 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 shutdown_connections 2023-07-14T11:53:17.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7864a0 0x5649fd7888b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd776ad0 0x5649fd773fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.651 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76b2d0 0x5649fd776580 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/3781790810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd76bbe0 0x5649fd773870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c4affd700 0 client.28141 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:17.652 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 >> 172.21.15.161:0/3781790810 conn(0x5649fd761da0 msgr2=0x5649fd767230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:17.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 shutdown_connections 2023-07-14T11:53:17.653 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 1 -- 172.21.15.161:0/3781790810 wait complete. 2023-07-14T11:53:17.654 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.642+0000 7f1c854d0b80 20 client.28141 trim_cache size 0 max 16384 2023-07-14T11:53:17.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.659+0000 7f1c854d0b80 1 Processor -- start 2023-07-14T11:53:17.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.659+0000 7f1c854d0b80 1 -- start start 2023-07-14T11:53:17.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd776120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.660 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd773090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd7735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd761970 con 0x5649fd76b2d0 2023-07-14T11:53:17.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd76b8f0 con 0x5649fd776ad0 2023-07-14T11:53:17.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd773b10 con 0x5649fd76bbe0 2023-07-14T11:53:17.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd773090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd776120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd7735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd773090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47924/0 (socket says 172.21.15.161:47924) 2023-07-14T11:53:17.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 -- 172.21.15.161:0/327721348 learned_addr learned my addr 172.21.15.161:0/327721348 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:17.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 -- 172.21.15.161:0/327721348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 msgr2=0x5649fd7735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 --2- 172.21.15.161:0/327721348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd7735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 -- 172.21.15.161:0/327721348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 msgr2=0x5649fd776120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 --2- 172.21.15.161:0/327721348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd776120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.660+0000 7f1c6affd700 1 -- 172.21.15.161:0/327721348 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649fd512250 con 0x5649fd76bbe0 2023-07-14T11:53:17.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c6affd700 1 --2- 172.21.15.161:0/327721348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd773090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c5b0 tx=0x7f1c6001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4276ba91f7f4f6c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:17.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c527fc700 1 -- 172.21.15.161:0/327721348 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6009a3c0 con 0x5649fd76bbe0 2023-07-14T11:53:17.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c527fc700 1 -- 172.21.15.161:0/327721348 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c6009a520 con 0x5649fd76bbe0 2023-07-14T11:53:17.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c527fc700 1 -- 172.21.15.161:0/327721348 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600b8420 con 0x5649fd76bbe0 2023-07-14T11:53:17.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 -- 172.21.15.161:0/327721348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 msgr2=0x5649fd773090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/327721348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd773090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c5b0 tx=0x7f1c6001f000 comp rx=0 tx=0).stop 2023-07-14T11:53:17.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 -- 172.21.15.161:0/327721348 shutdown_connections 2023-07-14T11:53:17.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/327721348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd776120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/327721348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd7735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/327721348 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd773090 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 -- 172.21.15.161:0/327721348 >> 172.21.15.161:0/327721348 conn(0x5649fd761da0 msgr2=0x5649fd76ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:17.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 -- 172.21.15.161:0/327721348 shutdown_connections 2023-07-14T11:53:17.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.661+0000 7f1c854d0b80 1 -- 172.21.15.161:0/327721348 wait complete. 2023-07-14T11:53:17.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 -- start start 2023-07-14T11:53:17.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 Processor -- start 2023-07-14T11:53:17.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd7d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd7d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd77b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd77ba50 con 0x5649fd76b2d0 2023-07-14T11:53:17.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd77bbc0 con 0x5649fd776ad0 2023-07-14T11:53:17.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fd77bd80 con 0x5649fd76bbe0 2023-07-14T11:53:17.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.662+0000 7f1c854d0b80 10 client.0 _ll_register_callbacks cb 0x5649fd65b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:17.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd7d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd77b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd7d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd7d3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:47928/0 (socket says 172.21.15.161:47928) 2023-07-14T11:53:17.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 -- 172.21.15.161:0/455478438 learned_addr learned my addr 172.21.15.161:0/455478438 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:17.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 -- 172.21.15.161:0/455478438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 msgr2=0x5649fd77b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd77b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 -- 172.21.15.161:0/455478438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 msgr2=0x5649fd7d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:17.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd7d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 -- 172.21.15.161:0/455478438 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649fd4ebde0 con 0x5649fd76bbe0 2023-07-14T11:53:17.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c6affd700 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd7d3ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1c6009a200 tx=0x7f1c600cd2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67cdb1b843a2702 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:17.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c4affd700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:53:17.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c4affd700 1 -- 172.21.15.161:0/455478438 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600b81c0 con 0x5649fd76bbe0 2023-07-14T11:53:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c600a2490 con 0x5649fd76bbe0 2023-07-14T11:53:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.663+0000 7f1c4affd700 1 -- 172.21.15.161:0/455478438 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9512+0+0 (secure 0 0 0) 0x7f1c600b8320 con 0x5649fd76bbe0 2023-07-14T11:53:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c4affd700 1 -- 172.21.15.161:0/455478438 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c600a2000 con 0x5649fd76bbe0 2023-07-14T11:53:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 10 client.28147 fetch_fsmap learned FSMap version 46 2023-07-14T11:53:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5649fd4ea2b0 con 0x5649fd76bbe0 2023-07-14T11:53:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c4affd700 1 -- 172.21.15.161:0/455478438 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c600a2490 con 0x5649fd76bbe0 2023-07-14T11:53:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c4affd700 1 -- 172.21.15.161:0/455478438 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c600b2a20 con 0x5649fd76bbe0 2023-07-14T11:53:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 10 client.28147 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:53:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 10 client.28147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:53:17.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 20 client.28147 populate_metadata read hostname 'smithi161' 2023-07-14T11:53:17.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7daf60 0x5649fd7dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:17.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 4 client.28147 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:53:17.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5649fd679050 con 0x5649fd7daf60 2023-07-14T11:53:17.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.664+0000 7f1c6b7fe700 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7daf60 0x5649fd7dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:17.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c6b7fe700 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7daf60 0x5649fd7dd340 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fc12320630d31 server_cookie=73300e7de35068d2 in_seq=0 out_seq=0 2023-07-14T11:53:17.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c4affd700 10 client.28147 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:17.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c4affd700 1 -- 172.21.15.161:0/455478438 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5649fd679050 con 0x5649fd7daf60 2023-07-14T11:53:17.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c4affd700 10 client.28147 handle_command_reply: tid=0 2023-07-14T11:53:17.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 client.28147 shutdown 2023-07-14T11:53:17.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 msgr2=0x5649fd7d3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd7d3ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1c6009a200 tx=0x7f1c600cd2e0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 shutdown_connections 2023-07-14T11:53:17.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x5649fd7daf60 0x5649fd7dd340 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649fd76b2d0 0x5649fd7d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649fd776ad0 0x5649fd77b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c4affd700 0 client.28147 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:17.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 --2- 172.21.15.161:0/455478438 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x5649fd76bbe0 0x5649fd7d3ba0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 >> 172.21.15.161:0/455478438 conn(0x5649fd761da0 msgr2=0x5649fd7668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:17.676 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 shutdown_connections 2023-07-14T11:53:17.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 1 -- 172.21.15.161:0/455478438 wait complete. 2023-07-14T11:53:17.677 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.665+0000 7f1c854d0b80 20 client.28147 trim_cache size 0 max 16384 2023-07-14T11:53:17.677 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:53:17.678 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:53:17.678 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:53:17.678 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:53:17.678 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:53:17.678 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:53:17.679 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:53:17.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:17.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:17.679 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:53:17.681 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c540a12c0 msgr2=0x7f1c540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c540a12c0 0x7f1c540a3770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c64007a70 tx=0x7f1c64009c50 comp rx=0 tx=0).stop 2023-07-14T11:53:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c100490 msgr2=0x7f1c6c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c100490 0x7f1c6c198340 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000e9a0 tx=0x7f1c6000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 shutdown_connections 2023-07-14T11:53:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1c540a12c0 0x7f1c540a3770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6c105af0 0x7f1c6c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1c6c0ffae0 0x7f1c6c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 --2- 172.21.15.161:0/2503462747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c6c100490 0x7f1c6c198340 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 >> 172.21.15.161:0/2503462747 conn(0x7f1c6c0fb5e0 msgr2=0x7f1c6c10b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 shutdown_connections 2023-07-14T11:53:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:17.666+0000 7f1c71bff700 1 -- 172.21.15.161:0/2503462747 wait complete. 2023-07-14T11:53:17.686 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:53:18.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:18 smithi161 ceph-mon[95526]: pgmap v983: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:18.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:18 smithi161 ceph-mon[95526]: osdmap e143: 12 total, 12 up, 12 in 2023-07-14T11:53:19.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:18 smithi175 ceph-mon[129081]: pgmap v983: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:19.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:18 smithi175 ceph-mon[129081]: osdmap e143: 12 total, 12 up, 12 in 2023-07-14T11:53:19.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:18 smithi182 ceph-mon[129366]: pgmap v983: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:53:19.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:18 smithi182 ceph-mon[129366]: osdmap e143: 12 total, 12 up, 12 in 2023-07-14T11:53:20.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:20 smithi161 ceph-mon[95526]: pgmap v985: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2023-07-14T11:53:21.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:20 smithi175 ceph-mon[129081]: pgmap v985: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2023-07-14T11:53:21.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:20 smithi182 ceph-mon[129366]: pgmap v985: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2023-07-14T11:53:22.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:22 smithi161 ceph-mon[95526]: pgmap v986: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 103 B/s wr, 7 op/s 2023-07-14T11:53:23.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:22 smithi175 ceph-mon[129081]: pgmap v986: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 103 B/s wr, 7 op/s 2023-07-14T11:53:23.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:22 smithi182 ceph-mon[129366]: pgmap v986: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 103 B/s wr, 7 op/s 2023-07-14T11:53:24.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:24 smithi175 ceph-mon[129081]: pgmap v987: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:53:24.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:24 smithi182 ceph-mon[129366]: pgmap v987: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:53:24.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:24 smithi161 ceph-mon[95526]: pgmap v987: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:53:26.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:26 smithi161 ceph-mon[95526]: pgmap v988: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:53:27.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:26 smithi175 ceph-mon[129081]: pgmap v988: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:53:27.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:26 smithi182 ceph-mon[129366]: pgmap v988: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:53:27.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:27 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:28.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:27 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:28.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:27 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:28.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:28 smithi161 ceph-mon[95526]: pgmap v989: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:53:29.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:28 smithi175 ceph-mon[129081]: pgmap v989: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:53:29.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:28 smithi182 ceph-mon[129366]: pgmap v989: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2023-07-14T11:53:30.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:30 smithi161 ceph-mon[95526]: pgmap v990: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2023-07-14T11:53:30.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:30 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:30.977 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:30 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:31.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:30 smithi175 ceph-mon[129081]: pgmap v990: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2023-07-14T11:53:31.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:30 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:31.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:30 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:31.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:30 smithi182 ceph-mon[129366]: pgmap v990: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2023-07-14T11:53:31.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:30 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:31.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:30 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: pgmap v991: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2023-07-14T11:53:32.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.152 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:53:32.152 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:53:32.152 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.152 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.152 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.153 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.153 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:53:32.153 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:31 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:53:32.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: pgmap v991: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2023-07-14T11:53:32.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.197 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:53:32.197 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:53:32.197 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.197 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.197 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.198 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.198 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:53:32.198 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:31 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:53:32.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:32.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: pgmap v991: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2023-07-14T11:53:32.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.228 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi175 to 196.2M 2023-07-14T11:53:32.228 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi175 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T11:53:32.228 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.228 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:32.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi161 to 220.9M 2023-07-14T11:53:32.229 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:31 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi161 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T11:53:33.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:53:33.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:53:33.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:33.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:33.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:33.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:53:33.195 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:53:33.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:33.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:33.196 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:33.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: Adjusting osd_memory_target on smithi182 to 196.2M 2023-07-14T11:53:33.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: Unable to set osd_memory_target on smithi182 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-07-14T11:53:33.227 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:33.228 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:33.228 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:53:34.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:34 smithi175 ceph-mon[129081]: pgmap v992: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:34.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:34 smithi182 ceph-mon[129366]: pgmap v992: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:34.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:34 smithi161 ceph-mon[95526]: pgmap v992: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:36.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:36 smithi161 ceph-mon[95526]: pgmap v993: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:37.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:36 smithi175 ceph-mon[129081]: pgmap v993: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:37.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:36 smithi182 ceph-mon[129366]: pgmap v993: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:38.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:38 smithi161 ceph-mon[95526]: pgmap v994: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:39.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:38 smithi175 ceph-mon[129081]: pgmap v994: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:39.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:38 smithi182 ceph-mon[129366]: pgmap v994: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:40.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:40 smithi161 ceph-mon[95526]: pgmap v995: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:41.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:40 smithi175 ceph-mon[129081]: pgmap v995: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:41.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:40 smithi182 ceph-mon[129366]: pgmap v995: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:42.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:42 smithi161 ceph-mon[95526]: pgmap v996: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:43.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:42 smithi175 ceph-mon[129081]: pgmap v996: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:43.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:42 smithi182 ceph-mon[129366]: pgmap v996: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:44.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:44 smithi175 ceph-mon[129081]: pgmap v997: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:44.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:44 smithi182 ceph-mon[129366]: pgmap v997: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:44.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:44 smithi161 ceph-mon[95526]: pgmap v997: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:46.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:46 smithi161 ceph-mon[95526]: pgmap v998: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:47.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:46 smithi175 ceph-mon[129081]: pgmap v998: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:47.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:46 smithi182 ceph-mon[129366]: pgmap v998: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:47.686 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T11:53:47.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.920+0000 7fa2fabff700 1 -- 172.21.15.161:0/3742160520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 msgr2=0x7fa2f4105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.920+0000 7fa2fabff700 1 --2- 172.21.15.161:0/3742160520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 0x7fa2f4105ec0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009990 tx=0x7fa2e8009ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.921+0000 7fa2fabff700 1 -- 172.21.15.161:0/3742160520 shutdown_connections 2023-07-14T11:53:47.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.921+0000 7fa2fabff700 1 --2- 172.21.15.161:0/3742160520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 0x7fa2f410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.921+0000 7fa2fabff700 1 --2- 172.21.15.161:0/3742160520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 0x7fa2f4105ec0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.921+0000 7fa2fabff700 1 --2- 172.21.15.161:0/3742160520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f40ffae0 0x7fa2f40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.921+0000 7fa2fabff700 1 -- 172.21.15.161:0/3742160520 >> 172.21.15.161:0/3742160520 conn(0x7fa2f40fb5e0 msgr2=0x7fa2f40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.921+0000 7fa2fabff700 1 -- 172.21.15.161:0/3742160520 shutdown_connections 2023-07-14T11:53:47.905 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.921+0000 7fa2fabff700 1 -- 172.21.15.161:0/3742160520 wait complete. 2023-07-14T11:53:47.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.922+0000 7fa2fabff700 1 Processor -- start 2023-07-14T11:53:47.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.922+0000 7fa2fabff700 1 -- start start 2023-07-14T11:53:47.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2fabff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f40ffae0 0x7fa2f4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2fabff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 0x7fa2f4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2fabff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 0x7fa2f41a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2fabff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f4073580 con 0x7fa2f40ffae0 2023-07-14T11:53:47.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2fabff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41a4300 con 0x7fa2f4105af0 2023-07-14T11:53:47.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2fabff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41a4440 con 0x7fa2f4100490 2023-07-14T11:53:47.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 0x7fa2f4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 0x7fa2f4072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52122/0 (socket says 172.21.15.161:52122) 2023-07-14T11:53:47.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2f3fff700 1 -- 172.21.15.161:0/2380430952 learned_addr learned my addr 172.21.15.161:0/2380430952 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:47.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2f919c700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 0x7fa2f41a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2f899b700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f40ffae0 0x7fa2f4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2f3fff700 1 -- 172.21.15.161:0/2380430952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 msgr2=0x7fa2f41a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.923+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 0x7fa2f41a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2f3fff700 1 -- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f40ffae0 msgr2=0x7fa2f4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f40ffae0 0x7fa2f4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2f3fff700 1 -- 172.21.15.161:0/2380430952 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80095f0 con 0x7fa2f4100490 2023-07-14T11:53:47.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 0x7fa2f4072d50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000e9a0 tx=0x7fa2e000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2f1ffb700 1 -- 172.21.15.161:0/2380430952 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0009790 con 0x7fa2f4100490 2023-07-14T11:53:47.911 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2f1ffb700 1 -- 172.21.15.161:0/2380430952 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e0010430 con 0x7fa2f4100490 2023-07-14T11:53:47.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f41a46a0 con 0x7fa2f4100490 2023-07-14T11:53:47.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2f1ffb700 1 -- 172.21.15.161:0/2380430952 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0018680 con 0x7fa2f4100490 2023-07-14T11:53:47.912 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.924+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f41a4b80 con 0x7fa2f4100490 2023-07-14T11:53:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.927+0000 7fa2f1ffb700 1 -- 172.21.15.161:0/2380430952 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa2e00105a0 con 0x7fa2f4100490 2023-07-14T11:53:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.928+0000 7fa30e553b80 1 Processor -- start 2023-07-14T11:53:47.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa30e553b80 1 -- start start 2023-07-14T11:53:47.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa2f1ffb700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2e40a12c0 0x7fa2e40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27ae2d0 0x561ed27ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.915 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27b9ad0 0x561ed27bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa30e553b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed26b4cb0 con 0x561ed27b9ad0 2023-07-14T11:53:47.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa30e553b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed2707a40 con 0x561ed27ae2d0 2023-07-14T11:53:47.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa30e553b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed2700ed0 con 0x561ed27aebe0 2023-07-14T11:53:47.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa2f1ffb700 1 -- 172.21.15.161:0/2380430952 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa2e0014070 con 0x7fa2f4100490 2023-07-14T11:53:47.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa2f899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27ae2d0 0x561ed27ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa2f899b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27ae2d0 0x561ed27ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55382/0 (socket says 172.21.15.161:55382) 2023-07-14T11:53:47.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa2f899b700 1 -- 172.21.15.161:0/3983466361 learned_addr learned my addr 172.21.15.161:0/3983466361 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:47.917 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.929+0000 7fa2f919c700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2e40a12c0 0x7fa2e40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f3fff700 1 -- 172.21.15.161:0/3983466361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27ae2d0 msgr2=0x561ed27ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27ae2d0 0x561ed27ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f919c700 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27b9ad0 0x561ed27bbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f3fff700 1 -- 172.21.15.161:0/3983466361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27b9ad0 msgr2=0x561ed27bbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27b9ad0 0x561ed27bbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f3fff700 1 -- 172.21.15.161:0/3983466361 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed252ede0 con 0x561ed27aebe0 2023-07-14T11:53:47.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f919c700 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27b9ad0 0x561ed27bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:47.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b9590 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00ca630 tx=0x7fa2e0099eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bab5f8e316fe8ba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2deffd700 1 -- 172.21.15.161:0/3983466361 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00071c0 con 0x561ed27aebe0 2023-07-14T11:53:47.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2deffd700 1 -- 172.21.15.161:0/3983466361 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e0099fb0 con 0x561ed27aebe0 2023-07-14T11:53:47.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.930+0000 7fa2deffd700 1 -- 172.21.15.161:0/3983466361 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000a250 con 0x561ed27aebe0 2023-07-14T11:53:47.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 -- 172.21.15.161:0/3983466361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 msgr2=0x561ed27b9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b9590 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00ca630 tx=0x7fa2e0099eb0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa2f919c700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2e40a12c0 0x7fa2e40a3770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009ec0 tx=0x7fa2ec009920 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 -- 172.21.15.161:0/3983466361 shutdown_connections 2023-07-14T11:53:47.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27b9ad0 0x561ed27bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27ae2d0 0x561ed27ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3983466361 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b9590 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 -- 172.21.15.161:0/3983466361 >> 172.21.15.161:0/3983466361 conn(0x561ed27a4da0 msgr2=0x561ed27aa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 -- 172.21.15.161:0/3983466361 shutdown_connections 2023-07-14T11:53:47.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.931+0000 7fa30e553b80 1 -- 172.21.15.161:0/3983466361 wait complete. 2023-07-14T11:53:47.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 -- start start 2023-07-14T11:53:47.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 Processor -- start 2023-07-14T11:53:47.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27ae2d0 0x561ed27b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27aebe0 0x561ed27b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed26a0ac0 con 0x561ed27aebe0 2023-07-14T11:53:47.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed26a07c0 con 0x561ed27b9ad0 2023-07-14T11:53:47.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed2584920 con 0x561ed27ae2d0 2023-07-14T11:53:47.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.932+0000 7fa30e553b80 10 client.0 _ll_register_callbacks cb 0x561ed269e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:47.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27ae2d0 0x561ed27b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f919c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27ae2d0 0x561ed27b9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52138/0 (socket says 172.21.15.161:52138) 2023-07-14T11:53:47.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 -- 172.21.15.161:0/2435375862 learned_addr learned my addr 172.21.15.161:0/2435375862 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:47.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27aebe0 0x561ed27b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 -- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 msgr2=0x561ed27b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 -- 172.21.15.161:0/2435375862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27aebe0 msgr2=0x561ed27b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27aebe0 0x561ed27b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 -- 172.21.15.161:0/2435375862 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f41a4b80 con 0x561ed27ae2d0 2023-07-14T11:53:47.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f919c700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:47.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27aebe0 0x561ed27b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:47.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2f899b700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27ae2d0 0x561ed27b9560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009990 tx=0x7fa2e800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e2f2d883e3ab2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.933+0000 7fa2d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.182:3300/0 2023-07-14T11:53:47.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa2d3fff700 1 -- 172.21.15.161:0/2435375862 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e801d070 con 0x561ed27ae2d0 2023-07-14T11:53:47.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa2d3fff700 1 -- 172.21.15.161:0/2435375862 <== mon.1 v2:172.21.15.182:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa2e8004500 con 0x561ed27ae2d0 2023-07-14T11:53:47.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ed265a4f0 con 0x561ed27ae2d0 2023-07-14T11:53:47.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa2d3fff700 1 -- 172.21.15.161:0/2435375862 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e8004700 con 0x561ed27ae2d0 2023-07-14T11:53:47.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 10 client.28159 fetch_fsmap learned FSMap version 46 2023-07-14T11:53:47.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561ed2517c60 con 0x561ed27ae2d0 2023-07-14T11:53:47.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa2d3fff700 1 -- 172.21.15.161:0/2435375862 <== mon.1 v2:172.21.15.182:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ed265a4f0 con 0x561ed27ae2d0 2023-07-14T11:53:47.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa2d3fff700 1 -- 172.21.15.161:0/2435375862 <== mon.1 v2:172.21.15.182:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2e800f920 con 0x561ed27ae2d0 2023-07-14T11:53:47.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 10 client.28159 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:53:47.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 10 client.28159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:53:47.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 20 client.28159 populate_metadata read hostname 'smithi161' 2023-07-14T11:53:47.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed27c9440 0x561ed27cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 4 client.28159 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:53:47.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.934+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ed26f9eb0 con 0x561ed27c9440 2023-07-14T11:53:47.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.935+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed27c9440 0x561ed27cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.935+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed27c9440 0x561ed27cb850 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92187dd34379fb0a server_cookie=125f760da9015ce6 in_seq=0 out_seq=0 2023-07-14T11:53:47.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.935+0000 7fa2d3fff700 10 client.28159 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:47.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa2d3fff700 1 -- 172.21.15.161:0/2435375862 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561ed26f9eb0 con 0x561ed27c9440 2023-07-14T11:53:47.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa2d3fff700 10 client.28159 handle_command_reply: tid=0 2023-07-14T11:53:47.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 client.28159 shutdown 2023-07-14T11:53:47.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27ae2d0 msgr2=0x561ed27b9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27ae2d0 0x561ed27b9560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009990 tx=0x7fa2e800b610 comp rx=0 tx=0).stop 2023-07-14T11:53:47.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 shutdown_connections 2023-07-14T11:53:47.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed27c9440 0x561ed27cb850 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa2d3fff700 0 client.28159 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:47.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27ae2d0 0x561ed27b9560 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2435375862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27aebe0 0x561ed27b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.936 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 >> 172.21.15.161:0/2435375862 conn(0x561ed27a4da0 msgr2=0x561ed27aa230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 shutdown_connections 2023-07-14T11:53:47.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 1 -- 172.21.15.161:0/2435375862 wait complete. 2023-07-14T11:53:47.937 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.936+0000 7fa30e553b80 20 client.28159 trim_cache size 0 max 16384 2023-07-14T11:53:47.940 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 Processor -- start 2023-07-14T11:53:47.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 -- start start 2023-07-14T11:53:47.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.941 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed27b7dc0 con 0x561ed27ae2d0 2023-07-14T11:53:47.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed27a1fa0 con 0x561ed27b9ad0 2023-07-14T11:53:47.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.953+0000 7fa30e553b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed27a4970 con 0x561ed27aebe0 2023-07-14T11:53:47.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.942 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f899b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55408/0 (socket says 172.21.15.161:55408) 2023-07-14T11:53:47.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 -- 172.21.15.161:0/2778934622 learned_addr learned my addr 172.21.15.161:0/2778934622 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:47.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 -- 172.21.15.161:0/2778934622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 msgr2=0x561ed27b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.943 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 -- 172.21.15.161:0/2778934622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 msgr2=0x561ed27b9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27b9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 -- 172.21.15.161:0/2778934622 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed2555250 con 0x561ed27b9ad0 2023-07-14T11:53:47.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:47.944 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2f919c700 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6500 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec017a60 tx=0x7fa2ec015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b57a12d1ef72e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.954+0000 7fa2df7fe700 1 -- 172.21.15.161:0/2778934622 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00e040 con 0x561ed27b9ad0 2023-07-14T11:53:47.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa2df7fe700 1 -- 172.21.15.161:0/2778934622 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2ec00f040 con 0x561ed27b9ad0 2023-07-14T11:53:47.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa2df7fe700 1 -- 172.21.15.161:0/2778934622 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec024c70 con 0x561ed27b9ad0 2023-07-14T11:53:47.945 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 -- 172.21.15.161:0/2778934622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 msgr2=0x561ed27b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6500 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec017a60 tx=0x7fa2ec015000 comp rx=0 tx=0).stop 2023-07-14T11:53:47.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 -- 172.21.15.161:0/2778934622 shutdown_connections 2023-07-14T11:53:47.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27b9ad0 0x561ed27b6500 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 --2- 172.21.15.161:0/2778934622 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27aebe0 0x561ed27b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 -- 172.21.15.161:0/2778934622 >> 172.21.15.161:0/2778934622 conn(0x561ed27a4da0 msgr2=0x561ed27b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 -- 172.21.15.161:0/2778934622 shutdown_connections 2023-07-14T11:53:47.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.955+0000 7fa30e553b80 1 -- 172.21.15.161:0/2778934622 wait complete. 2023-07-14T11:53:47.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 -- start start 2023-07-14T11:53:47.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 Processor -- start 2023-07-14T11:53:47.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27aebe0 0x561ed27be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27b9ad0 0x561ed27beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed27beff0 con 0x561ed27ae2d0 2023-07-14T11:53:47.948 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed27bf160 con 0x561ed27aebe0 2023-07-14T11:53:47.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed27bf320 con 0x561ed27b9ad0 2023-07-14T11:53:47.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.956+0000 7fa30e553b80 10 client.0 _ll_register_callbacks cb 0x7fa2e800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:47.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27aebe0 0x561ed27be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27aebe0 0x561ed27be570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55424/0 (socket says 172.21.15.161:55424) 2023-07-14T11:53:47.949 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f919c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27b9ad0 0x561ed27beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 -- 172.21.15.161:0/3265167661 learned_addr learned my addr 172.21.15.161:0/3265167661 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:47.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f899b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 -- 172.21.15.161:0/3265167661 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27b9ad0 msgr2=0x561ed27beab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.950 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27b9ad0 0x561ed27beab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 -- 172.21.15.161:0/3265167661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 msgr2=0x561ed27c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 -- 172.21.15.161:0/3265167661 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ed252ede0 con 0x561ed27aebe0 2023-07-14T11:53:47.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2f3fff700 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27aebe0 0x561ed27be570 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0017040 tx=0x7fa2e0099eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c38a2d783f0af5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.951 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2023-07-14T11:53:47.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2d3fff700 1 -- 172.21.15.161:0/3265167661 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00caab0 con 0x561ed27aebe0 2023-07-14T11:53:47.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2e0099fb0 con 0x561ed27aebe0 2023-07-14T11:53:47.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2d3fff700 1 -- 172.21.15.161:0/3265167661 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fa2e009a210 con 0x561ed27aebe0 2023-07-14T11:53:47.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa2d3fff700 1 -- 172.21.15.161:0/3265167661 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e009a3d0 con 0x561ed27aebe0 2023-07-14T11:53:47.952 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa30e553b80 10 client.28121 fetch_fsmap learned FSMap version 46 2023-07-14T11:53:47.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.957+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fa2ec017770 con 0x561ed27aebe0 2023-07-14T11:53:47.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa2d3fff700 1 -- 172.21.15.161:0/3265167661 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e0099fb0 con 0x561ed27aebe0 2023-07-14T11:53:47.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa2d3fff700 1 -- 172.21.15.161:0/3265167661 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2e00a1f70 con 0x561ed27aebe0 2023-07-14T11:53:47.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa30e553b80 10 client.28121 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T11:53:47.953 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa30e553b80 10 client.28121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T11:53:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa30e553b80 20 client.28121 populate_metadata read hostname 'smithi161' 2023-07-14T11:53:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed281e530 0x561ed2820940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa30e553b80 4 client.28121 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-14T11:53:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 --> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ed26bc050 con 0x561ed281e530 2023-07-14T11:53:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa2f899b700 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed281e530 0x561ed2820940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa2d3fff700 10 client.28121 ms_handle_connect on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:47.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.958+0000 7fa2f899b700 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed281e530 0x561ed2820940 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae82077efb924b7 server_cookie=ec55ceaeb994ce7f in_seq=0 out_seq=0 2023-07-14T11:53:47.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa2d3fff700 1 -- 172.21.15.161:0/3265167661 <== mds.0 v2:172.21.15.175:6838/2144664422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561ed26bc050 con 0x561ed281e530 2023-07-14T11:53:47.956 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa2d3fff700 10 client.28121 handle_command_reply: tid=0 2023-07-14T11:53:47.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 client.28121 shutdown 2023-07-14T11:53:47.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27aebe0 msgr2=0x561ed27be570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27aebe0 0x561ed27be570 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0017040 tx=0x7fa2e0099eb0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.957 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 shutdown_connections 2023-07-14T11:53:47.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:6838/2144664422,v1:172.21.15.175:6839/2144664422] conn(0x561ed281e530 0x561ed2820940 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ed27ae2d0 0x561ed27c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa2d3fff700 0 client.28121 ms_handle_reset on v2:172.21.15.175:6838/2144664422 2023-07-14T11:53:47.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x561ed27b9ad0 0x561ed27beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.958 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 --2- 172.21.15.161:0/3265167661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed27aebe0 0x561ed27be570 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 >> 172.21.15.161:0/3265167661 conn(0x561ed27a4da0 msgr2=0x561ed27b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 shutdown_connections 2023-07-14T11:53:47.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 1 -- 172.21.15.161:0/3265167661 wait complete. 2023-07-14T11:53:47.959 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.959+0000 7fa30e553b80 20 client.28121 trim_cache size 0 max 16384 2023-07-14T11:53:47.960 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:53:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T11:53:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "scrubs": { 2023-07-14T11:53:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "b3742157-5095-4515-9f5d-963568dbe011": { 2023-07-14T11:53:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "path": "/", 2023-07-14T11:53:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "tag": "b3742157-5095-4515-9f5d-963568dbe011", 2023-07-14T11:53:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "options": "recursive,force" 2023-07-14T11:53:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:47.962 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:53:47.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2e40a12c0 msgr2=0x7fa2e40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2e40a12c0 0x7fa2e40a3770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009ec0 tx=0x7fa2ec009920 comp rx=0 tx=0).stop 2023-07-14T11:53:47.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 msgr2=0x7fa2f4072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.963 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 0x7fa2f4072d50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000e9a0 tx=0x7fa2e000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 shutdown_connections 2023-07-14T11:53:47.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fa2e40a12c0 0x7fa2e40a3770 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f40ffae0 0x7fa2f4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.964 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f4105af0 0x7fa2f41a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 --2- 172.21.15.161:0/2380430952 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fa2f4100490 0x7fa2f4072d50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 >> 172.21.15.161:0/2380430952 conn(0x7fa2f40fb5e0 msgr2=0x7fa2f40fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 shutdown_connections 2023-07-14T11:53:47.965 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:47.960+0000 7fa2fabff700 1 -- 172.21.15.161:0/2380430952 wait complete. 2023-07-14T11:53:47.966 INFO:tasks.cephfs.filesystem:scrub status for tag:b3742157-5095-4515-9f5d-963568dbe011 - {'path': '/', 'tag': 'b3742157-5095-4515-9f5d-963568dbe011', 'options': 'recursive,force'} 2023-07-14T11:53:47.966 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs/filesystem.py", line 1684, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-07-14T11:53:47.974 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-14T11:53:47.978 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-14T11:53:47.989 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:48.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 -- 172.21.15.161:0/1317022500 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 msgr2=0x7fca5c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 --2- 172.21.15.161:0/1317022500 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 0x7fca5c109b00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fca500098f0 tx=0x7fca50009c00 comp rx=0 tx=0).stop 2023-07-14T11:53:48.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 -- 172.21.15.161:0/1317022500 shutdown_connections 2023-07-14T11:53:48.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 --2- 172.21.15.161:0/1317022500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca5c10a040 0x7fca5c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 --2- 172.21.15.161:0/1317022500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 0x7fca5c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 --2- 172.21.15.161:0/1317022500 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 0x7fca5c109b00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 -- 172.21.15.161:0/1317022500 >> 172.21.15.161:0/1317022500 conn(0x7fca5c0fa490 msgr2=0x7fca5c0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 -- 172.21.15.161:0/1317022500 shutdown_connections 2023-07-14T11:53:48.223 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.238+0000 7fca615ff700 1 -- 172.21.15.161:0/1317022500 wait complete. 2023-07-14T11:53:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.240+0000 7fca615ff700 1 Processor -- start 2023-07-14T11:53:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.240+0000 7fca615ff700 1 -- start start 2023-07-14T11:53:48.225 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.240+0000 7fca615ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 0x7fca5c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca615ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 0x7fca5c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca615ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca5c10a040 0x7fca5c195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca615ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c196100 con 0x7fca5c10a040 2023-07-14T11:53:48.226 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca615ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c196270 con 0x7fca5c100d00 2023-07-14T11:53:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca615ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c196430 con 0x7fca5c101610 2023-07-14T11:53:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 0x7fca5c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 0x7fca5c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55448/0 (socket says 172.21.15.161:55448) 2023-07-14T11:53:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 -- 172.21.15.161:0/1396208504 learned_addr learned my addr 172.21.15.161:0/1396208504 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5a7fc700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 0x7fca5c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5b7fe700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca5c10a040 0x7fca5c195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 -- 172.21.15.161:0/1396208504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 msgr2=0x7fca5c19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 0x7fca5c19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 -- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca5c10a040 msgr2=0x7fca5c195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca5c10a040 0x7fca5c195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 -- 172.21.15.161:0/1396208504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca50009520 con 0x7fca5c100d00 2023-07-14T11:53:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca5affd700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 0x7fca5c19ba20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9c0 tx=0x7fca4c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.241+0000 7fca43fff700 1 -- 172.21.15.161:0/1396208504 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00cbd0 con 0x7fca5c100d00 2023-07-14T11:53:48.229 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.242+0000 7fca43fff700 1 -- 172.21.15.161:0/1396208504 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4c00eeb0 con 0x7fca5c100d00 2023-07-14T11:53:48.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.242+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c196710 con 0x7fca5c100d00 2023-07-14T11:53:48.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.242+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c1a2a10 con 0x7fca5c100d00 2023-07-14T11:53:48.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.242+0000 7fca43fff700 1 -- 172.21.15.161:0/1396208504 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c005580 con 0x7fca5c100d00 2023-07-14T11:53:48.230 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.243+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c067300 con 0x7fca5c100d00 2023-07-14T11:53:48.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.245+0000 7fca43fff700 1 -- 172.21.15.161:0/1396208504 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fca4c004750 con 0x7fca5c100d00 2023-07-14T11:53:48.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.246+0000 7fca43fff700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fca440a1300 0x7fca440a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.231 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.246+0000 7fca43fff700 1 -- 172.21.15.161:0/1396208504 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fca4c014070 con 0x7fca5c100d00 2023-07-14T11:53:48.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.247+0000 7fca5a7fc700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fca440a1300 0x7fca440a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.247+0000 7fca5a7fc700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fca440a1300 0x7fca440a37b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca50005e90 tx=0x7fca50019040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.249+0000 7fca43fff700 1 -- 172.21.15.161:0/1396208504 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fca4c096b20 con 0x7fca5c100d00 2023-07-14T11:53:48.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.450+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca5c1a2c60 con 0x7fca5c100d00 2023-07-14T11:53:48.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.452+0000 7fca43fff700 1 -- 172.21.15.161:0/1396208504 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+25182 (secure 0 0 0) 0x7fca4c09a140 con 0x7fca5c100d00 2023-07-14T11:53:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:53:48.435 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":143,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:53:18.308612+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":57,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":57,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/142996677":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/3308966068":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/389110447":"2023-07-15T11:22:15.015135+0000","172.21.15.161:6801/2004305703":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/2004305703":"2023-07-15T11:22:15.015135+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/182250477":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":44,"length":1},{"begin":57,"length":1}]},{"pool":4,"snaps":[{"begin":44,"length":1},{"begin":57,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fca440a1300 msgr2=0x7fca440a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fca440a1300 0x7fca440a37b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca50005e90 tx=0x7fca50019040 comp rx=0 tx=0).stop 2023-07-14T11:53:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 msgr2=0x7fca5c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 0x7fca5c19ba20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9c0 tx=0x7fca4c00ecd0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 shutdown_connections 2023-07-14T11:53:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fca440a1300 0x7fca440a37b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca5c10a040 0x7fca5c195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5c100d00 0x7fca5c19ba20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 --2- 172.21.15.161:0/1396208504 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fca5c101610 0x7fca5c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.454+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 >> 172.21.15.161:0/1396208504 conn(0x7fca5c0fa490 msgr2=0x7fca5c10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.455+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 shutdown_connections 2023-07-14T11:53:48.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.455+0000 7fca615ff700 1 -- 172.21.15.161:0/1396208504 wait complete. 2023-07-14T11:53:48.456 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 57, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 57, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:53:48.457 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:53:48.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 -- 172.21.15.161:0/1505204939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260105ae0 msgr2=0x7f8260105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.661 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 --2- 172.21.15.161:0/1505204939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260105ae0 0x7f8260105eb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8250009990 tx=0x7f8250009ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 -- 172.21.15.161:0/1505204939 shutdown_connections 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 --2- 172.21.15.161:0/1505204939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8260100480 0x7f826010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 --2- 172.21.15.161:0/1505204939 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260105ae0 0x7f8260105eb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 --2- 172.21.15.161:0/1505204939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 0x7f82600fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 -- 172.21.15.161:0/1505204939 >> 172.21.15.161:0/1505204939 conn(0x7f82600fb5f0 msgr2=0x7f82600fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 -- 172.21.15.161:0/1505204939 shutdown_connections 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.678+0000 7f82659ff700 1 -- 172.21.15.161:0/1505204939 wait complete. 2023-07-14T11:53:48.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.679+0000 7f82659ff700 1 Processor -- start 2023-07-14T11:53:48.664 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.680+0000 7f82659ff700 1 -- start start 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.680+0000 7f82659ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 0x7f826019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.680+0000 7f82659ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260100480 0x7f826019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.680+0000 7f82659ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8260105ae0 0x7f8260195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.680+0000 7f82659ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260196080 con 0x7f8260105ae0 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.680+0000 7f82659ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82601961f0 con 0x7f82600ffad0 2023-07-14T11:53:48.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.680+0000 7f82659ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82601963b0 con 0x7f8260100480 2023-07-14T11:53:48.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260100480 0x7f826019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 0x7f826019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 0x7f826019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55474/0 (socket says 172.21.15.161:55474) 2023-07-14T11:53:48.666 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825e7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260100480 0x7f826019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52184/0 (socket says 172.21.15.161:52184) 2023-07-14T11:53:48.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 -- 172.21.15.161:0/792460998 learned_addr learned my addr 172.21.15.161:0/792460998 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:48.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825f7fe700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8260105ae0 0x7f8260195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 -- 172.21.15.161:0/792460998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260100480 msgr2=0x7f826019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260100480 0x7f826019bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.667 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 -- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8260105ae0 msgr2=0x7f8260195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8260105ae0 0x7f8260195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 -- 172.21.15.161:0/792460998 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82500095f0 con 0x7f82600ffad0 2023-07-14T11:53:48.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825f7fe700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8260105ae0 0x7f8260195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:48.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825e7fc700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260100480 0x7f826019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:48.668 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f825effd700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 0x7f826019b9a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8250005910 tx=0x7f8250005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f82649fd700 1 -- 172.21.15.161:0/792460998 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825001d070 con 0x7f82600ffad0 2023-07-14T11:53:48.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8260196630 con 0x7f82600ffad0 2023-07-14T11:53:48.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f82649fd700 1 -- 172.21.15.161:0/792460998 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8250022470 con 0x7f82600ffad0 2023-07-14T11:53:48.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.681+0000 7f82649fd700 1 -- 172.21.15.161:0/792460998 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250017650 con 0x7f82600ffad0 2023-07-14T11:53:48.669 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.682+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8260072920 con 0x7f82600ffad0 2023-07-14T11:53:48.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.682+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8260067290 con 0x7f82600ffad0 2023-07-14T11:53:48.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.685+0000 7f82649fd700 1 -- 172.21.15.161:0/792460998 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f82500225e0 con 0x7f82600ffad0 2023-07-14T11:53:48.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.686+0000 7f82649fd700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f82480a12c0 0x7f82480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.686+0000 7f82649fd700 1 -- 172.21.15.161:0/792460998 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f82500ce5d0 con 0x7f82600ffad0 2023-07-14T11:53:48.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.686+0000 7f825e7fc700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f82480a12c0 0x7f82480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.687+0000 7f825e7fc700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f82480a12c0 0x7f82480a3770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8260105200 tx=0x7f8254009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.689+0000 7f82649fd700 1 -- 172.21.15.161:0/792460998 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f82500976f0 con 0x7f82600ffad0 2023-07-14T11:53:48.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.891+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8260196f20 con 0x7f82600ffad0 2023-07-14T11:53:48.874 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.892+0000 7f82649fd700 1 -- 172.21.15.161:0/792460998 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7f825009ad10 con 0x7f82600ffad0 2023-07-14T11:53:48.875 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:53:48.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f82480a12c0 msgr2=0x7f82480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f82480a12c0 0x7f82480a3770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8260105200 tx=0x7f8254009260 comp rx=0 tx=0).stop 2023-07-14T11:53:48.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 msgr2=0x7f826019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.877 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 0x7f826019b9a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8250005910 tx=0x7f8250005720 comp rx=0 tx=0).stop 2023-07-14T11:53:48.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 shutdown_connections 2023-07-14T11:53:48.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f82480a12c0 0x7f82480a3770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8260105ae0 0x7f8260195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82600ffad0 0x7f826019b9a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.878 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 --2- 172.21.15.161:0/792460998 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8260100480 0x7f826019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 >> 172.21.15.161:0/792460998 conn(0x7f82600fb5f0 msgr2=0x7f82600fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 shutdown_connections 2023-07-14T11:53:48.879 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:48.894+0000 7f82659ff700 1 -- 172.21.15.161:0/792460998 wait complete. 2023-07-14T11:53:48.895 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:53:48.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:48 smithi161 ceph-mon[95526]: pgmap v999: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:48.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:48 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1396208504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:53:49.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 -- 172.21.15.161:0/4269466187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c073160 msgr2=0x7f2c7c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 --2- 172.21.15.161:0/4269466187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c073160 0x7f2c7c10be40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009990 tx=0x7f2c68009ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 -- 172.21.15.161:0/4269466187 shutdown_connections 2023-07-14T11:53:49.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 --2- 172.21.15.161:0/4269466187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7c072850 0x7f2c7c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 --2- 172.21.15.161:0/4269466187 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c10c380 0x7f2c7c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 --2- 172.21.15.161:0/4269466187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c073160 0x7f2c7c10be40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.100 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 -- 172.21.15.161:0/4269466187 >> 172.21.15.161:0/4269466187 conn(0x7f2c7c0fb5e0 msgr2=0x7f2c7c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 -- 172.21.15.161:0/4269466187 shutdown_connections 2023-07-14T11:53:49.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.116+0000 7f2c82bff700 1 -- 172.21.15.161:0/4269466187 wait complete. 2023-07-14T11:53:49.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.117+0000 7f2c82bff700 1 Processor -- start 2023-07-14T11:53:49.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.117+0000 7f2c82bff700 1 -- start start 2023-07-14T11:53:49.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c82bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c072850 0x7f2c7c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c82bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7c073160 0x7f2c7c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c82bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c10c380 0x7f2c7c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c82bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c19cdf0 con 0x7f2c7c073160 2023-07-14T11:53:49.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c82bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c19cf60 con 0x7f2c7c10c380 2023-07-14T11:53:49.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c82bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c19d120 con 0x7f2c7c072850 2023-07-14T11:53:49.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c072850 0x7f2c7c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c072850 0x7f2c7c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52200/0 (socket says 172.21.15.161:52200) 2023-07-14T11:53:49.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7c073160 0x7f2c7c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 -- 172.21.15.161:0/2025693867 learned_addr learned my addr 172.21.15.161:0/2025693867 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:49.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8119c700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c10c380 0x7f2c7c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 -- 172.21.15.161:0/2025693867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c10c380 msgr2=0x7f2c7c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c10c380 0x7f2c7c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 -- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7c073160 msgr2=0x7f2c7c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7c073160 0x7f2c7c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.118+0000 7f2c8099b700 1 -- 172.21.15.161:0/2025693867 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680095f0 con 0x7f2c7c072850 2023-07-14T11:53:49.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.119+0000 7f2c8099b700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c072850 0x7f2c7c197e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000e9a0 tx=0x7f2c7000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.119+0000 7f2c79ffb700 1 -- 172.21.15.161:0/2025693867 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70009790 con 0x7f2c7c072850 2023-07-14T11:53:49.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.119+0000 7f2c79ffb700 1 -- 172.21.15.161:0/2025693867 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c70010430 con 0x7f2c7c072850 2023-07-14T11:53:49.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.119+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c19d400 con 0x7f2c7c072850 2023-07-14T11:53:49.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.119+0000 7f2c79ffb700 1 -- 172.21.15.161:0/2025693867 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70018680 con 0x7f2c7c072850 2023-07-14T11:53:49.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.119+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c1a4c40 con 0x7f2c7c072850 2023-07-14T11:53:49.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.120+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c106ca0 con 0x7f2c7c072850 2023-07-14T11:53:49.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.122+0000 7f2c79ffb700 1 -- 172.21.15.161:0/2025693867 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2c700105a0 con 0x7f2c7c072850 2023-07-14T11:53:49.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.123+0000 7f2c79ffb700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2c6c0a12c0 0x7f2c6c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.123+0000 7f2c79ffb700 1 -- 172.21.15.161:0/2025693867 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f2c70014070 con 0x7f2c7c072850 2023-07-14T11:53:49.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.124+0000 7f2c7bfff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2c6c0a12c0 0x7f2c6c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.124+0000 7f2c7bfff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2c6c0a12c0 0x7f2c6c0a3770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c199420 tx=0x7f2c68005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.125+0000 7f2c79ffb700 1 -- 172.21.15.161:0/2025693867 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c70096290 con 0x7f2c7c072850 2023-07-14T11:53:49.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:48 smithi175 ceph-mon[129081]: pgmap v999: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:49.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:48 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1396208504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:53:49.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:48 smithi182 ceph-mon[129366]: pgmap v999: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:49.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:48 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1396208504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:53:49.310 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.328+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c7c02c850 con 0x7f2c7c072850 2023-07-14T11:53:49.311 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.328+0000 7f2c79ffb700 1 -- 172.21.15.161:0/2025693867 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c700998b0 con 0x7f2c7c072850 2023-07-14T11:53:49.311 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:53:49.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2c6c0a12c0 msgr2=0x7f2c6c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.313 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2c6c0a12c0 0x7f2c6c0a3770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c199420 tx=0x7f2c68005fd0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c072850 msgr2=0x7f2c7c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c072850 0x7f2c7c197e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000e9a0 tx=0x7f2c7000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 shutdown_connections 2023-07-14T11:53:49.314 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f2c6c0a12c0 0x7f2c6c0a3770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7c10c380 0x7f2c7c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f2c7c072850 0x7f2c7c197e10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 --2- 172.21.15.161:0/2025693867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7c073160 0x7f2c7c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 >> 172.21.15.161:0/2025693867 conn(0x7f2c7c0fb5e0 msgr2=0x7f2c7c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 shutdown_connections 2023-07-14T11:53:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.331+0000 7f2c82bff700 1 -- 172.21.15.161:0/2025693867 wait complete. 2023-07-14T11:53:49.332 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:53:49.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 -- 172.21.15.161:0/1795567987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f74140ffad0 msgr2=0x7f74140fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 --2- 172.21.15.161:0/1795567987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f74140ffad0 0x7f74140fff40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7400009990 tx=0x7f7400009ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 -- 172.21.15.161:0/1795567987 shutdown_connections 2023-07-14T11:53:49.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 --2- 172.21.15.161:0/1795567987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7414100480 0x7f741410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 --2- 172.21.15.161:0/1795567987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7414105ae0 0x7f7414105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 --2- 172.21.15.161:0/1795567987 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f74140ffad0 0x7f74140fff40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 -- 172.21.15.161:0/1795567987 >> 172.21.15.161:0/1795567987 conn(0x7f74140fb5f0 msgr2=0x7f74140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 -- 172.21.15.161:0/1795567987 shutdown_connections 2023-07-14T11:53:49.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.552+0000 7f741adff700 1 -- 172.21.15.161:0/1795567987 wait complete. 2023-07-14T11:53:49.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.553+0000 7f741adff700 1 Processor -- start 2023-07-14T11:53:49.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.554+0000 7f741adff700 1 -- start start 2023-07-14T11:53:49.539 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.554+0000 7f741adff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74140ffad0 0x7f741419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.554+0000 7f741adff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7414100480 0x7f741419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.554+0000 7f741adff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7414105ae0 0x7f7414195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.554+0000 7f741adff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7414196070 con 0x7f74140ffad0 2023-07-14T11:53:49.540 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.554+0000 7f741adff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74141961e0 con 0x7f7414100480 2023-07-14T11:53:49.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.554+0000 7f741adff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74141963a0 con 0x7f7414105ae0 2023-07-14T11:53:49.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74140ffad0 0x7f741419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74140ffad0 0x7f741419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57286/0 (socket says 172.21.15.161:57286) 2023-07-14T11:53:49.541 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7413fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7414100480 0x7f741419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 -- 172.21.15.161:0/2160674357 learned_addr learned my addr 172.21.15.161:0/2160674357 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:49.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f741939c700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7414105ae0 0x7f7414195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 -- 172.21.15.161:0/2160674357 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7414105ae0 msgr2=0x7f7414195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.542 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7414105ae0 0x7f7414195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 -- 172.21.15.161:0/2160674357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7414100480 msgr2=0x7f741419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7414100480 0x7f741419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 -- 172.21.15.161:0/2160674357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74000095f0 con 0x7f74140ffad0 2023-07-14T11:53:49.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7413fff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7414100480 0x7f741419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:49.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f741939c700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7414105ae0 0x7f7414195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:49.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7418b9b700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74140ffad0 0x7f741419b990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f740800e9a0 tx=0x7f740800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f7411ffb700 1 -- 172.21.15.161:0/2160674357 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408009790 con 0x7f74140ffad0 2023-07-14T11:53:49.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.555+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7414196680 con 0x7f74140ffad0 2023-07-14T11:53:49.544 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.556+0000 7f7411ffb700 1 -- 172.21.15.161:0/2160674357 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7408010430 con 0x7f74140ffad0 2023-07-14T11:53:49.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.556+0000 7f7411ffb700 1 -- 172.21.15.161:0/2160674357 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408018680 con 0x7f74140ffad0 2023-07-14T11:53:49.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.556+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74141a2980 con 0x7f74140ffad0 2023-07-14T11:53:49.545 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.557+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7414067290 con 0x7f74140ffad0 2023-07-14T11:53:49.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.559+0000 7f7411ffb700 1 -- 172.21.15.161:0/2160674357 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f74080105a0 con 0x7f74140ffad0 2023-07-14T11:53:49.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.560+0000 7f7411ffb700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f74040a12c0 0x7f74040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.546 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.560+0000 7f7411ffb700 1 -- 172.21.15.161:0/2160674357 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f7408014070 con 0x7f74140ffad0 2023-07-14T11:53:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.560+0000 7f7413fff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f74040a12c0 0x7f74040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.561+0000 7f7413fff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f74040a12c0 0x7f74040a3770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f74141051b0 tx=0x7f7400005fb0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.564+0000 7f7411ffb700 1 -- 172.21.15.161:0/2160674357 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7408051790 con 0x7f74140ffad0 2023-07-14T11:53:49.745 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.763+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7414196f10 con 0x7f74140ffad0 2023-07-14T11:53:49.746 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.763+0000 7f7411ffb700 1 -- 172.21.15.161:0/2160674357 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7414196f10 con 0x7f74140ffad0 2023-07-14T11:53:49.746 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:53:49.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f74040a12c0 msgr2=0x7f74040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f74040a12c0 0x7f74040a3770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f74141051b0 tx=0x7f7400005fb0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.749 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74140ffad0 msgr2=0x7f741419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74140ffad0 0x7f741419b990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f740800e9a0 tx=0x7f740800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 shutdown_connections 2023-07-14T11:53:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f74040a12c0 0x7f74040a3770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.750 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74140ffad0 0x7f741419b990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f7414105ae0 0x7f7414195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 --2- 172.21.15.161:0/2160674357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7414100480 0x7f741419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 >> 172.21.15.161:0/2160674357 conn(0x7f74140fb5f0 msgr2=0x7f74141035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.751 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 shutdown_connections 2023-07-14T11:53:49.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.766+0000 7f741adff700 1 -- 172.21.15.161:0/2160674357 wait complete. 2023-07-14T11:53:49.767 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:53:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 -- 172.21.15.161:0/1144592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 msgr2=0x7f1670105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 --2- 172.21.15.161:0/1144592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f1670105ec0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f166c009990 tx=0x7f166c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 -- 172.21.15.161:0/1144592425 shutdown_connections 2023-07-14T11:53:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 --2- 172.21.15.161:0/1144592425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1670100490 0x7f167010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 --2- 172.21.15.161:0/1144592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f1670105ec0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 --2- 172.21.15.161:0/1144592425 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16700ffae0 0x7f16700fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 -- 172.21.15.161:0/1144592425 >> 172.21.15.161:0/1144592425 conn(0x7f16700fb5e0 msgr2=0x7f16700fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 -- 172.21.15.161:0/1144592425 shutdown_connections 2023-07-14T11:53:49.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.988+0000 7f16787ff700 1 -- 172.21.15.161:0/1144592425 wait complete. 2023-07-14T11:53:49.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.989+0000 7f16787ff700 1 Processor -- start 2023-07-14T11:53:49.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.989+0000 7f16787ff700 1 -- start start 2023-07-14T11:53:49.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.989+0000 7f16787ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16700ffae0 0x7f1670197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f16787ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1670100490 0x7f16701983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f16787ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f167019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f16787ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167019ce40 con 0x7f16700ffae0 2023-07-14T11:53:49.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f16787ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167019cfb0 con 0x7f1670105af0 2023-07-14T11:53:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f16787ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167019d170 con 0x7f1670100490 2023-07-14T11:53:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1670100490 0x7f16701983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1676d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f167019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f167659b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16700ffae0 0x7f1670197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1676d9c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f167019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:55518/0 (socket says 172.21.15.161:55518) 2023-07-14T11:53:49.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f167659b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16700ffae0 0x7f1670197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57302/0 (socket says 172.21.15.161:57302) 2023-07-14T11:53:49.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1670100490 0x7f16701983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52230/0 (socket says 172.21.15.161:52230) 2023-07-14T11:53:49.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1676d9c700 1 -- 172.21.15.161:0/1160688015 learned_addr learned my addr 172.21.15.161:0/1160688015 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:49.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 -- 172.21.15.161:0/1160688015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 msgr2=0x7f167019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f167019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 -- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16700ffae0 msgr2=0x7f1670197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16700ffae0 0x7f1670197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 -- 172.21.15.161:0/1160688015 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1660009520 con 0x7f1670100490 2023-07-14T11:53:49.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1676d9c700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f167019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f167659b700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16700ffae0 0x7f1670197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.990+0000 7f1675d9a700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1670100490 0x7f16701983a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f166000e9f0 tx=0x7f166000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.991+0000 7f16677fe700 1 -- 172.21.15.161:0/1160688015 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000cb70 con 0x7f1670100490 2023-07-14T11:53:49.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.991+0000 7f16677fe700 1 -- 172.21.15.161:0/1160688015 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f166000ee90 con 0x7f1670100490 2023-07-14T11:53:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.991+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c0095f0 con 0x7f1670100490 2023-07-14T11:53:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.991+0000 7f16677fe700 1 -- 172.21.15.161:0/1160688015 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660003cf0 con 0x7f1670100490 2023-07-14T11:53:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.991+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16701a4ab0 con 0x7f1670100490 2023-07-14T11:53:49.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.992+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1670067290 con 0x7f1670100490 2023-07-14T11:53:49.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/792460998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:53:49.982 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:49 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2025693867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:53:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.994+0000 7f16677fe700 1 -- 172.21.15.161:0/1160688015 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f166000ccd0 con 0x7f1670100490 2023-07-14T11:53:49.983 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.995+0000 7f16677fe700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f165c0a12c0 0x7f165c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.996+0000 7f16677fe700 1 -- 172.21.15.161:0/1160688015 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f1660014070 con 0x7f1670100490 2023-07-14T11:53:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.996+0000 7f167659b700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f165c0a12c0 0x7f165c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.996+0000 7f167659b700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f165c0a12c0 0x7f165c0a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b2e0 tx=0x7f166c005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:49.998+0000 7f16677fe700 1 -- 172.21.15.161:0/1160688015 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1660096ac0 con 0x7f1670100490 2023-07-14T11:53:50.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:49 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/792460998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:53:50.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:49 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2025693867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:53:50.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.201+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f16701a4da0 con 0x7f1670100490 2023-07-14T11:53:50.184 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.201+0000 7f16677fe700 1 -- 172.21.15.161:0/1160688015 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v143) v1 ==== 103+0+11 (secure 0 0 0) 0x7f166009a0e0 con 0x7f1670100490 2023-07-14T11:53:50.185 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:53:50.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f165c0a12c0 msgr2=0x7f165c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.186 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f165c0a12c0 0x7f165c0a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b2e0 tx=0x7f166c005fd0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1670100490 msgr2=0x7f16701983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1670100490 0x7f16701983a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f166000e9f0 tx=0x7f166000ed00 comp rx=0 tx=0).stop 2023-07-14T11:53:50.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 shutdown_connections 2023-07-14T11:53:50.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f165c0a12c0 0x7f165c0a3770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16700ffae0 0x7f1670197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1670105af0 0x7f167019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 --2- 172.21.15.161:0/1160688015 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1670100490 0x7f16701983a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 >> 172.21.15.161:0/1160688015 conn(0x7f16700fb5e0 msgr2=0x7f16700fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 shutdown_connections 2023-07-14T11:53:50.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.204+0000 7f16787ff700 1 -- 172.21.15.161:0/1160688015 wait complete. 2023-07-14T11:53:50.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:49 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/792460998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:53:50.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:49 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2025693867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:53:50.205 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-14T11:53:50.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 -- 172.21.15.161:0/1683394581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 msgr2=0x7fb32810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 --2- 172.21.15.161:0/1683394581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 0x7fb32810be50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310009ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 -- 172.21.15.161:0/1683394581 shutdown_connections 2023-07-14T11:53:50.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 --2- 172.21.15.161:0/1683394581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb328072850 0x7fb328072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 --2- 172.21.15.161:0/1683394581 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb32810c390 0x7fb32810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 --2- 172.21.15.161:0/1683394581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 0x7fb32810be50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 -- 172.21.15.161:0/1683394581 >> 172.21.15.161:0/1683394581 conn(0x7fb3280fb630 msgr2=0x7fb3280fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 -- 172.21.15.161:0/1683394581 shutdown_connections 2023-07-14T11:53:50.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.424+0000 7fb32cdff700 1 -- 172.21.15.161:0/1683394581 wait complete. 2023-07-14T11:53:50.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.425+0000 7fb32cdff700 1 Processor -- start 2023-07-14T11:53:50.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.425+0000 7fb32cdff700 1 -- start start 2023-07-14T11:53:50.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.425+0000 7fb32cdff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb328072850 0x7fb328197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.425+0000 7fb32cdff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 0x7fb328198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32cdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb32810c390 0x7fb32819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32cdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819cdb0 con 0x7fb32810c390 2023-07-14T11:53:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32cdff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819cf20 con 0x7fb328073160 2023-07-14T11:53:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32cdff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819d0e0 con 0x7fb328072850 2023-07-14T11:53:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb328072850 0x7fb328197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb31ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 0x7fb328198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb328072850 0x7fb328197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:52250/0 (socket says 172.21.15.161:52250) 2023-07-14T11:53:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 -- 172.21.15.161:0/2289891367 learned_addr learned my addr 172.21.15.161:0/2289891367 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:53:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb326d9d700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb32810c390 0x7fb32819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 -- 172.21.15.161:0/2289891367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 msgr2=0x7fb328198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 0x7fb328198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 -- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb32810c390 msgr2=0x7fb32819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb32810c390 0x7fb32819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 -- 172.21.15.161:0/2289891367 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb318009520 con 0x7fb328072850 2023-07-14T11:53:50.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb326d9d700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb32810c390 0x7fb32819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:50.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb31ffff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 0x7fb328198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:50.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.426+0000 7fb32659c700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb328072850 0x7fb328197dd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb31800eb30 tx=0x7fb31800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.427+0000 7fb31f7fe700 1 -- 172.21.15.161:0/2289891367 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318009880 con 0x7fb328072850 2023-07-14T11:53:50.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.427+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3100095f0 con 0x7fb328072850 2023-07-14T11:53:50.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.427+0000 7fb31f7fe700 1 -- 172.21.15.161:0/2289891367 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3180044e0 con 0x7fb328072850 2023-07-14T11:53:50.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.427+0000 7fb31f7fe700 1 -- 172.21.15.161:0/2289891367 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318003e90 con 0x7fb328072850 2023-07-14T11:53:50.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.427+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32819d5b0 con 0x7fb328072850 2023-07-14T11:53:50.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.428+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb328106cb0 con 0x7fb328072850 2023-07-14T11:53:50.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.430+0000 7fb31f7fe700 1 -- 172.21.15.161:0/2289891367 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb318004000 con 0x7fb328072850 2023-07-14T11:53:50.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.431+0000 7fb31f7fe700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb3140a1300 0x7fb3140a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.431+0000 7fb31f7fe700 1 -- 172.21.15.161:0/2289891367 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fb318014070 con 0x7fb328072850 2023-07-14T11:53:50.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.431+0000 7fb31ffff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb3140a1300 0x7fb3140a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.432+0000 7fb31ffff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb3140a1300 0x7fb3140a37b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3281993e0 tx=0x7fb310005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.433+0000 7fb31f7fe700 1 -- 172.21.15.161:0/2289891367 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb318096c80 con 0x7fb328072850 2023-07-14T11:53:50.673 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.690+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fb328067300 con 0x7fb328072850 2023-07-14T11:53:50.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.702+0000 7fb31f7fe700 1 -- 172.21.15.161:0/2289891367 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733783 (secure 0 0 0) 0x7fb31809a2a0 con 0x7fb328072850 2023-07-14T11:53:50.692 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:53:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:53:50.693 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 25, 2023-07-14T11:53:50.693 INFO:teuthology.orchestra.run.smithi161.stdout: "active_gid": 15543, 2023-07-14T11:53:50.693 INFO:teuthology.orchestra.run.smithi161.stdout: "active_name": "x", 2023-07-14T11:53:50.694 INFO:teuthology.orchestra.run.smithi161.stdout: "active_addrs": { 2023-07-14T11:53:50.694 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2023-07-14T11:53:50.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.694 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2023-07-14T11:53:50.694 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2023-07-14T11:53:50.695 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4046273551 2023-07-14T11:53:50.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2023-07-14T11:53:50.695 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2023-07-14T11:53:50.696 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4046273551 2023-07-14T11:53:50.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.696 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:50.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.696 INFO:teuthology.orchestra.run.smithi161.stdout: "active_addr": "172.21.15.161:6801/4046273551", 2023-07-14T11:53:50.697 INFO:teuthology.orchestra.run.smithi161.stdout: "active_change": "2023-07-14T11:22:15.015376+0000", 2023-07-14T11:53:50.697 INFO:teuthology.orchestra.run.smithi161.stdout: "active_mgr_features": 4540138322906710015, 2023-07-14T11:53:50.697 INFO:teuthology.orchestra.run.smithi161.stdout: "available": true, 2023-07-14T11:53:50.697 INFO:teuthology.orchestra.run.smithi161.stdout: "standbys": [ 2023-07-14T11:53:50.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.698 INFO:teuthology.orchestra.run.smithi161.stdout: "gid": 25574, 2023-07-14T11:53:50.698 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "y", 2023-07-14T11:53:50.698 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr_features": 4540138322906710015, 2023-07-14T11:53:50.698 INFO:teuthology.orchestra.run.smithi161.stdout: "available_modules": [ 2023-07-14T11:53:50.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.698 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alerts", 2023-07-14T11:53:50.699 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:50.699 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:50.699 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:50.699 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:50.700 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:50.700 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.700 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.700 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.700 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:50.700 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.701 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.701 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.701 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T11:53:50.701 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.701 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.702 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.702 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:50.702 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:50.702 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.703 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.703 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.703 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.703 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.703 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.704 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.704 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.704 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.704 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.704 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.705 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.705 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.705 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.705 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.705 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.706 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.706 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.706 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:50.706 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:50.707 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.707 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.707 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.707 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.707 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.707 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.708 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.708 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.708 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.708 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.709 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.709 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:50.709 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:50.709 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.710 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.710 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.710 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:50.710 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.710 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.711 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.711 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.711 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.711 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.711 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.712 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.712 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.712 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.712 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.712 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.713 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.713 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.713 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:50.713 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:50.714 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.714 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.714 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.714 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.714 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.715 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.715 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.715 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.715 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.715 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.716 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.716 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_destination": { 2023-07-14T11:53:50.716 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_destination", 2023-07-14T11:53:50.716 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.717 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.717 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.717 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.717 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.717 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.718 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.718 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Email address to send alerts to", 2023-07-14T11:53:50.718 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.718 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.718 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.719 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_from_name": { 2023-07-14T11:53:50.719 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_from_name", 2023-07-14T11:53:50.719 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.719 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.719 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.720 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "Ceph", 2023-07-14T11:53:50.720 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.720 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.720 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.720 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Email From: name", 2023-07-14T11:53:50.721 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.721 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.721 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.721 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_host": { 2023-07-14T11:53:50.722 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_host", 2023-07-14T11:53:50.722 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.722 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.722 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.722 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.723 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.723 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.723 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.723 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP server", 2023-07-14T11:53:50.723 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.723 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.724 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.724 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_password": { 2023-07-14T11:53:50.724 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_password", 2023-07-14T11:53:50.724 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.725 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.725 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.725 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.725 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.725 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.726 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.726 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Password to authenticate with", 2023-07-14T11:53:50.726 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.726 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.726 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.727 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_port": { 2023-07-14T11:53:50.727 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_port", 2023-07-14T11:53:50.727 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.727 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.727 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.728 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "465", 2023-07-14T11:53:50.728 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.728 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.728 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.729 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP port", 2023-07-14T11:53:50.729 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.729 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.729 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.729 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_sender": { 2023-07-14T11:53:50.730 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_sender", 2023-07-14T11:53:50.730 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.730 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.730 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.730 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.731 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.731 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.731 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.731 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP envelope sender", 2023-07-14T11:53:50.731 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.732 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.732 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.732 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_ssl": { 2023-07-14T11:53:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_ssl", 2023-07-14T11:53:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T11:53:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.735 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.735 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_user": { 2023-07-14T11:53:50.735 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_user", 2023-07-14T11:53:50.735 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.736 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.736 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.736 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.736 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.736 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.737 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.737 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "User to authenticate as", 2023-07-14T11:53:50.737 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.737 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.737 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.738 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.738 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.738 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "balancer", 2023-07-14T11:53:50.739 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:50.739 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:50.739 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:50.739 INFO:teuthology.orchestra.run.smithi161.stdout: "active": { 2023-07-14T11:53:50.739 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "active", 2023-07-14T11:53:50.740 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.740 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.740 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.740 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.740 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.740 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.741 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.741 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T11:53:50.741 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.741 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.741 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.742 INFO:teuthology.orchestra.run.smithi161.stdout: "begin_time": { 2023-07-14T11:53:50.742 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "begin_time", 2023-07-14T11:53:50.742 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.742 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.743 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.743 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0000", 2023-07-14T11:53:50.743 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.743 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.743 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.744 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T11:53:50.744 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T11:53:50.744 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.744 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.745 INFO:teuthology.orchestra.run.smithi161.stdout: "begin_weekday": { 2023-07-14T11:53:50.745 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "begin_weekday", 2023-07-14T11:53:50.745 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:50.745 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.745 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.745 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:50.746 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0", 2023-07-14T11:53:50.746 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "6", 2023-07-14T11:53:50.746 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.746 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T11:53:50.746 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T11:53:50.747 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.747 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.747 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_max_iterations": { 2023-07-14T11:53:50.747 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_max_iterations", 2023-07-14T11:53:50.748 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:50.748 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.748 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.748 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "25", 2023-07-14T11:53:50.748 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:50.749 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "250", 2023-07-14T11:53:50.749 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.749 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T11:53:50.749 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.749 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.749 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.750 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_metrics": { 2023-07-14T11:53:50.750 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_metrics", 2023-07-14T11:53:50.750 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.750 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.751 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.751 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T11:53:50.751 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.751 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.751 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.752 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T11:53:50.752 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T11:53:50.752 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.752 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.753 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_step": { 2023-07-14T11:53:50.753 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_step", 2023-07-14T11:53:50.753 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:50.753 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.753 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.754 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0.5", 2023-07-14T11:53:50.754 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0.001", 2023-07-14T11:53:50.754 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "0.999", 2023-07-14T11:53:50.754 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.754 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "aggressiveness of optimization", 2023-07-14T11:53:50.754 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T11:53:50.755 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.755 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.755 INFO:teuthology.orchestra.run.smithi161.stdout: "end_time": { 2023-07-14T11:53:50.755 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "end_time", 2023-07-14T11:53:50.756 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.756 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.756 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.756 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "2359", 2023-07-14T11:53:50.756 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.757 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.757 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.757 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T11:53:50.757 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T11:53:50.757 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.758 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.758 INFO:teuthology.orchestra.run.smithi161.stdout: "end_weekday": { 2023-07-14T11:53:50.758 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "end_weekday", 2023-07-14T11:53:50.758 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:50.759 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.759 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.759 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:50.759 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0", 2023-07-14T11:53:50.759 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "6", 2023-07-14T11:53:50.759 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.760 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T11:53:50.760 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T11:53:50.760 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.760 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.761 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:50.762 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:50.762 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.762 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.762 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.762 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.763 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.763 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.763 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.763 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.763 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.764 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.764 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.764 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.764 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.764 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.765 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.765 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.765 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.765 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.766 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:50.766 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:50.766 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.766 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.766 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.766 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.767 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.767 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.767 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.767 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.768 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.768 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.768 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.768 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:50.769 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:50.769 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.769 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.769 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.769 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:50.770 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.770 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.770 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.770 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.770 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.770 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.771 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.771 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.771 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.771 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.771 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.772 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.772 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.772 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.772 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:50.773 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:50.773 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.773 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.773 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.773 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.774 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.774 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.774 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.774 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.774 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.774 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.775 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.775 INFO:teuthology.orchestra.run.smithi161.stdout: "min_score": { 2023-07-14T11:53:50.775 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "min_score", 2023-07-14T11:53:50.775 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:50.776 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.776 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.776 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:50.776 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.776 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.777 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.777 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T11:53:50.777 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.777 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.777 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.778 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": { 2023-07-14T11:53:50.778 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mode", 2023-07-14T11:53:50.778 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.778 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.779 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.779 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "upmap", 2023-07-14T11:53:50.779 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.779 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.779 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.780 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-compat", 2023-07-14T11:53:50.780 INFO:teuthology.orchestra.run.smithi161.stdout: "none", 2023-07-14T11:53:50.780 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap" 2023-07-14T11:53:50.780 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.780 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Balancer mode", 2023-07-14T11:53:50.780 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.781 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.781 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.781 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_ids": { 2023-07-14T11:53:50.781 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pool_ids", 2023-07-14T11:53:50.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.782 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.782 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.782 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.782 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.783 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.783 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.783 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T11:53:50.783 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.783 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.784 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.784 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:50.784 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:50.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.785 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.785 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.785 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:50.785 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.785 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.786 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.786 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T11:53:50.786 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.786 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.786 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.787 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap_max_deviation": { 2023-07-14T11:53:50.787 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "upmap_max_deviation", 2023-07-14T11:53:50.787 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.787 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.788 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.788 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:50.788 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:50.788 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.788 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.789 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T11:53:50.789 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T11:53:50.789 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.789 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.790 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap_max_optimizations": { 2023-07-14T11:53:50.790 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "upmap_max_optimizations", 2023-07-14T11:53:50.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:50.790 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.790 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.790 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:50.791 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.791 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.791 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.791 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T11:53:50.791 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.792 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.792 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.792 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.792 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.793 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cephadm", 2023-07-14T11:53:50.793 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:50.793 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:50.793 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:50.794 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_down_multiplier": { 2023-07-14T11:53:50.794 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_down_multiplier", 2023-07-14T11:53:50.794 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:50.794 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.794 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.795 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3.0", 2023-07-14T11:53:50.795 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.795 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.795 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.795 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T11:53:50.796 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.796 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.796 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.796 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_refresh_rate": { 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_refresh_rate", 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "20", 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.799 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.799 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.799 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_starting_port": { 2023-07-14T11:53:50.799 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_starting_port", 2023-07-14T11:53:50.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.800 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.800 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.800 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "4721", 2023-07-14T11:53:50.800 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T11:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "alertmanager_web_password": { 2023-07-14T11:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alertmanager_web_password", 2023-07-14T11:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Alertmanager web password", 2023-07-14T11:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "alertmanager_web_user": { 2023-07-14T11:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alertmanager_web_user", 2023-07-14T11:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Alertmanager web user", 2023-07-14T11:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_ptrace": { 2023-07-14T11:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_ptrace", 2023-07-14T11:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T11:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T11:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "autotune_interval": { 2023-07-14T11:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "autotune_interval", 2023-07-14T11:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T11:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "autotune_memory_target_ratio": { 2023-07-14T11:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T11:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0.7", 2023-07-14T11:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T11:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "cgroups_split": { 2023-07-14T11:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cgroups_split", 2023-07-14T11:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T11:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "config_checks_enabled": { 2023-07-14T11:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_checks_enabled", 2023-07-14T11:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T11:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "config_dashboard": { 2023-07-14T11:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_dashboard", 2023-07-14T11:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T11:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_alertmanager": { 2023-07-14T11:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_alertmanager", 2023-07-14T11:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T11:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_base": { 2023-07-14T11:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_base", 2023-07-14T11:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T11:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Container image name, without the tag", 2023-07-14T11:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_elasticsearch": { 2023-07-14T11:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_elasticsearch", 2023-07-14T11:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T11:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "elasticsearch container image", 2023-07-14T11:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_grafana": { 2023-07-14T11:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_grafana", 2023-07-14T11:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T11:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_haproxy": { 2023-07-14T11:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_haproxy", 2023-07-14T11:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T11:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "HAproxy container image", 2023-07-14T11:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_agent": { 2023-07-14T11:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_agent", 2023-07-14T11:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T11:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger agent container image", 2023-07-14T11:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_collector": { 2023-07-14T11:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_collector", 2023-07-14T11:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T11:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger collector container image", 2023-07-14T11:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_query": { 2023-07-14T11:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_query", 2023-07-14T11:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T11:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger query container image", 2023-07-14T11:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_keepalived": { 2023-07-14T11:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_keepalived", 2023-07-14T11:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T11:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Keepalived container image", 2023-07-14T11:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_loki": { 2023-07-14T11:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_loki", 2023-07-14T11:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T11:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Loki container image", 2023-07-14T11:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_node_exporter": { 2023-07-14T11:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_node_exporter", 2023-07-14T11:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T11:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_prometheus": { 2023-07-14T11:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_prometheus", 2023-07-14T11:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T11:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_promtail": { 2023-07-14T11:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_promtail", 2023-07-14T11:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T11:53:50.858 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.858 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.858 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.858 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Promtail container image", 2023-07-14T11:53:50.858 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.859 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.859 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.859 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_snmp_gateway": { 2023-07-14T11:53:50.859 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_snmp_gateway", 2023-07-14T11:53:50.860 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.860 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.860 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.860 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T11:53:50.860 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.861 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.861 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.861 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SNMP Gateway container image", 2023-07-14T11:53:50.861 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.861 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.861 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.862 INFO:teuthology.orchestra.run.smithi161.stdout: "container_init": { 2023-07-14T11:53:50.862 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_init", 2023-07-14T11:53:50.862 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.862 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.863 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.863 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.863 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.863 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.863 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.864 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T11:53:50.864 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.864 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.864 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.865 INFO:teuthology.orchestra.run.smithi161.stdout: "daemon_cache_timeout": { 2023-07-14T11:53:50.865 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "daemon_cache_timeout", 2023-07-14T11:53:50.865 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.865 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.865 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.866 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:50.866 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.866 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.866 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.866 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T11:53:50.867 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.867 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.868 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.868 INFO:teuthology.orchestra.run.smithi161.stdout: "default_cephadm_command_timeout": { 2023-07-14T11:53:50.869 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T11:53:50.869 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.869 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.869 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.869 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "900", 2023-07-14T11:53:50.870 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.870 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.870 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.870 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T11:53:50.870 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.871 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.871 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.871 INFO:teuthology.orchestra.run.smithi161.stdout: "default_registry": { 2023-07-14T11:53:50.871 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "default_registry", 2023-07-14T11:53:50.872 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.872 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.872 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.872 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io", 2023-07-14T11:53:50.872 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.873 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.873 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.873 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T11:53:50.873 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.873 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.874 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.874 INFO:teuthology.orchestra.run.smithi161.stdout: "device_cache_timeout": { 2023-07-14T11:53:50.874 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_cache_timeout", 2023-07-14T11:53:50.874 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.875 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.875 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.875 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1800", 2023-07-14T11:53:50.875 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.875 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.875 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.876 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache device inventory", 2023-07-14T11:53:50.876 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.876 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.876 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.877 INFO:teuthology.orchestra.run.smithi161.stdout: "device_enhanced_scan": { 2023-07-14T11:53:50.877 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_enhanced_scan", 2023-07-14T11:53:50.877 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.877 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.877 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.878 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.878 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.878 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.878 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.878 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T11:53:50.879 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.879 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.879 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.879 INFO:teuthology.orchestra.run.smithi161.stdout: "facts_cache_timeout": { 2023-07-14T11:53:50.880 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "facts_cache_timeout", 2023-07-14T11:53:50.880 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.880 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.880 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.880 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:50.881 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.881 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.881 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.881 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache host facts data", 2023-07-14T11:53:50.881 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.882 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.882 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.882 INFO:teuthology.orchestra.run.smithi161.stdout: "host_check_interval": { 2023-07-14T11:53:50.882 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "host_check_interval", 2023-07-14T11:53:50.882 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.883 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.883 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.883 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:50.883 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.883 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.884 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.884 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to perform a host check", 2023-07-14T11:53:50.884 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.884 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.884 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.885 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:50.885 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:50.885 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.885 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.886 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.886 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.886 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.886 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.886 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.886 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.887 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.887 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.887 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.887 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.887 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.888 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.888 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.888 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.888 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.888 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.889 INFO:teuthology.orchestra.run.smithi161.stdout: "log_refresh_metadata": { 2023-07-14T11:53:50.889 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_refresh_metadata", 2023-07-14T11:53:50.889 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.889 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.890 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.890 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.890 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.890 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.890 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.891 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T11:53:50.891 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.891 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.891 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.891 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:50.892 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:50.892 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.892 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.892 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.892 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.893 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.893 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.893 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.893 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T11:53:50.893 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:50.895 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.895 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.895 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.895 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:50.895 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.896 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.896 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.896 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.896 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.896 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.897 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.897 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.897 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.897 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.897 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.898 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.898 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.898 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.898 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:50.899 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:50.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.899 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.899 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.899 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.899 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.900 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.900 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.900 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.900 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.901 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.901 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.901 INFO:teuthology.orchestra.run.smithi161.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T11:53:50.901 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T11:53:50.902 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.902 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.902 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.902 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.902 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.903 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.903 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.903 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T11:53:50.903 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.904 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.904 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.904 INFO:teuthology.orchestra.run.smithi161.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T11:53:50.904 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T11:53:50.905 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.905 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.905 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.905 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "*", 2023-07-14T11:53:50.905 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.906 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.906 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.906 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T11:53:50.906 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.906 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.907 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.907 INFO:teuthology.orchestra.run.smithi161.stdout: "max_count_per_host": { 2023-07-14T11:53:50.907 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_count_per_host", 2023-07-14T11:53:50.907 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.908 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.908 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.908 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:50.908 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.908 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.909 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.909 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "max number of daemons per service per host", 2023-07-14T11:53:50.909 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.909 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.909 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.910 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd_draining_count": { 2023-07-14T11:53:50.910 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_osd_draining_count", 2023-07-14T11:53:50.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.910 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.911 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.911 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:50.911 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.911 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.911 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.912 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T11:53:50.912 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.912 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.912 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.913 INFO:teuthology.orchestra.run.smithi161.stdout: "migration_current": { 2023-07-14T11:53:50.913 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "migration_current", 2023-07-14T11:53:50.913 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.913 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.913 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.914 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.914 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.914 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.914 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.914 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "internal - do not modify", 2023-07-14T11:53:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": { 2023-07-14T11:53:50.916 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mode", 2023-07-14T11:53:50.916 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.916 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.916 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.916 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "root", 2023-07-14T11:53:50.917 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.917 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.917 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.917 INFO:teuthology.orchestra.run.smithi161.stdout: "cephadm-package", 2023-07-14T11:53:50.917 INFO:teuthology.orchestra.run.smithi161.stdout: "root" 2023-07-14T11:53:50.918 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.918 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T11:53:50.918 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.918 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.918 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.919 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_alerts_path": { 2023-07-14T11:53:50.919 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_alerts_path", 2023-07-14T11:53:50.919 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.919 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.919 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.920 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T11:53:50.920 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.920 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.920 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.920 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T11:53:50.921 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.921 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.921 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.921 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_web_password": { 2023-07-14T11:53:50.922 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_web_password", 2023-07-14T11:53:50.922 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.922 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.922 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.922 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:50.923 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.923 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.923 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.923 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus web password", 2023-07-14T11:53:50.923 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.923 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.924 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.924 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_web_user": { 2023-07-14T11:53:50.924 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_web_user", 2023-07-14T11:53:50.924 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.925 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.925 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.925 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:50.925 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.925 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.926 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.926 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus web user", 2023-07-14T11:53:50.926 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.926 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.926 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.927 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_insecure": { 2023-07-14T11:53:50.927 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_insecure", 2023-07-14T11:53:50.927 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.927 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.927 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.928 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.928 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.928 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.928 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.928 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T11:53:50.929 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.929 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.929 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.929 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_password": { 2023-07-14T11:53:50.930 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_password", 2023-07-14T11:53:50.930 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.930 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.930 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.930 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.931 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.931 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.931 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.931 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T11:53:50.931 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.932 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.932 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.932 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_url": { 2023-07-14T11:53:50.932 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_url", 2023-07-14T11:53:50.933 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.933 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.933 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.933 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.933 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.933 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.934 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.934 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T11:53:50.934 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.934 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.934 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.935 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_username": { 2023-07-14T11:53:50.935 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_username", 2023-07-14T11:53:50.935 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.935 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.936 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.936 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.936 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.936 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.936 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.937 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T11:53:50.937 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.937 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.937 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.938 INFO:teuthology.orchestra.run.smithi161.stdout: "secure_monitoring_stack": { 2023-07-14T11:53:50.938 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "secure_monitoring_stack", 2023-07-14T11:53:50.938 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.938 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.938 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.939 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.939 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.939 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.939 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.939 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T11:53:50.939 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.940 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.940 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.940 INFO:teuthology.orchestra.run.smithi161.stdout: "service_discovery_port": { 2023-07-14T11:53:50.940 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "service_discovery_port", 2023-07-14T11:53:50.941 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.941 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.941 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.941 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8765", 2023-07-14T11:53:50.941 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.942 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.942 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.942 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "cephadm service discovery port", 2023-07-14T11:53:50.942 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.942 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.943 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.943 INFO:teuthology.orchestra.run.smithi161.stdout: "ssh_config_file": { 2023-07-14T11:53:50.943 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssh_config_file", 2023-07-14T11:53:50.943 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.944 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.944 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.944 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.944 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.944 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.945 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.945 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T11:53:50.945 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.945 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.945 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.946 INFO:teuthology.orchestra.run.smithi161.stdout: "use_agent": { 2023-07-14T11:53:50.946 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "use_agent", 2023-07-14T11:53:50.946 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.946 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.946 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.947 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.947 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.947 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.947 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.947 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T11:53:50.948 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.948 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.948 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.948 INFO:teuthology.orchestra.run.smithi161.stdout: "use_repo_digest": { 2023-07-14T11:53:50.949 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "use_repo_digest", 2023-07-14T11:53:50.949 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.949 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.949 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.949 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.949 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.950 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.950 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.950 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T11:53:50.950 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.950 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.951 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.951 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_failed_host_check": { 2023-07-14T11:53:50.951 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_failed_host_check", 2023-07-14T11:53:50.951 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.952 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.952 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.952 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.952 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.952 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.953 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.953 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T11:53:50.953 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.953 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.953 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.954 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_stray_daemons": { 2023-07-14T11:53:50.954 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_stray_daemons", 2023-07-14T11:53:50.954 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.954 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.954 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.955 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.955 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.955 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.955 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.955 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T11:53:50.956 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.956 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.956 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.956 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_stray_hosts": { 2023-07-14T11:53:50.957 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_stray_hosts", 2023-07-14T11:53:50.957 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.957 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.957 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.957 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.957 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.958 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.958 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.958 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T11:53:50.958 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.958 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.959 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.959 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.959 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.960 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crash", 2023-07-14T11:53:50.960 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:50.960 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:50.960 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:50.960 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:50.961 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:50.961 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.961 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.961 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.962 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.962 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.962 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.962 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.962 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.962 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.963 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.963 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.963 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.963 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.963 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.964 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.964 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.964 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.964 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:50.965 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:50.965 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.965 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.965 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.965 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.966 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.966 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.966 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.966 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.966 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.967 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.967 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.967 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:50.967 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:50.967 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.968 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.968 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:50.968 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.968 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.969 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:50.969 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:50.969 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:50.969 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:50.969 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:50.970 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:50.970 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:50.970 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:50.970 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.970 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.971 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.971 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.971 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:50.971 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:50.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.972 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.972 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.972 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.972 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.973 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.973 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.973 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.973 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.973 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.974 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.974 INFO:teuthology.orchestra.run.smithi161.stdout: "retain_interval": { 2023-07-14T11:53:50.974 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "retain_interval", 2023-07-14T11:53:50.974 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.974 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.975 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.975 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "31536000", 2023-07-14T11:53:50.975 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.975 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.975 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.976 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T11:53:50.976 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.976 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.976 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.977 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_recent_interval": { 2023-07-14T11:53:50.977 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_recent_interval", 2023-07-14T11:53:50.977 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:50.977 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.977 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:50.978 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1209600", 2023-07-14T11:53:50.978 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.978 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.978 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.978 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T11:53:50.979 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.979 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.979 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.979 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.979 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:50.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:50.980 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "dashboard", 2023-07-14T11:53:50.980 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:50.980 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:50.980 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:50.981 INFO:teuthology.orchestra.run.smithi161.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T11:53:50.981 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T11:53:50.981 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:50.981 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.981 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.982 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:50.982 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.982 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.982 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.983 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.983 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.983 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.983 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.984 INFO:teuthology.orchestra.run.smithi161.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T11:53:50.984 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T11:53:50.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:50.984 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.984 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.984 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:50.985 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.985 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.985 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.985 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.985 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.986 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.986 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.986 INFO:teuthology.orchestra.run.smithi161.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T11:53:50.986 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T11:53:50.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.987 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.987 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.987 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.987 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.988 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.988 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.988 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.988 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.988 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.989 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.989 INFO:teuthology.orchestra.run.smithi161.stdout: "AUDIT_API_ENABLED": { 2023-07-14T11:53:50.989 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T11:53:50.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.990 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.990 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.990 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:50.990 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.990 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.991 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.991 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.991 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.991 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.991 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi161.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.993 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.993 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.993 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.993 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.993 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi161.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.997 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:50.997 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:50.997 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:50.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:50.997 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.000 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.000 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.000 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.006 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.006 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T11:53:51.006 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T11:53:51.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.007 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.007 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.007 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.007 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.008 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.008 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.008 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.008 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.009 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.009 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.009 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T11:53:51.009 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T11:53:51.009 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.010 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.010 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.010 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.010 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.010 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.011 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.011 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.011 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.011 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.012 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.012 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T11:53:51.012 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T11:53:51.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.012 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.013 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.013 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.013 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.013 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.013 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.014 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.014 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.014 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.014 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.015 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T11:53:51.015 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T11:53:51.015 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.015 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.016 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.016 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.016 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.016 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.016 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.017 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.017 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.017 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.017 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.018 INFO:teuthology.orchestra.run.smithi161.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T11:53:51.018 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T11:53:51.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.018 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.018 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.020 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.020 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.020 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T11:53:51.020 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T11:53:51.021 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.021 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.021 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.021 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:51.021 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.022 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.022 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.022 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.022 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.022 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.023 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.023 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T11:53:51.023 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T11:53:51.023 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.023 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.024 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.024 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.024 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.024 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.024 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.025 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.025 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.025 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.025 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.026 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_URL": { 2023-07-14T11:53:51.026 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_URL", 2023-07-14T11:53:51.026 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.026 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.026 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.027 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.027 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.027 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.027 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.027 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.028 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.028 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.028 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.028 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T11:53:51.029 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T11:53:51.029 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.029 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.029 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.029 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:51.030 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.030 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.030 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.030 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.030 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.030 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.031 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.031 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T11:53:51.031 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T11:53:51.031 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.032 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.032 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.032 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.032 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.032 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.033 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.033 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.033 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.033 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.033 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.034 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T11:53:51.034 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T11:53:51.034 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.034 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.035 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.035 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.035 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.035 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.035 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.035 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.036 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.036 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.036 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.036 INFO:teuthology.orchestra.run.smithi161.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T11:53:51.037 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T11:53:51.037 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.037 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.037 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.037 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.038 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.038 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.038 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.038 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.038 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.039 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.039 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.039 INFO:teuthology.orchestra.run.smithi161.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T11:53:51.039 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T11:53:51.039 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.040 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.040 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.040 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.040 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.040 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.041 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.041 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.041 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.041 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.041 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.042 INFO:teuthology.orchestra.run.smithi161.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T11:53:51.042 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T11:53:51.042 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.042 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.042 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.043 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.043 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.043 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.043 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.043 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.044 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.044 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.044 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.044 INFO:teuthology.orchestra.run.smithi161.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T11:53:51.045 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T11:53:51.045 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.045 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.045 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.046 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.046 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.046 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.046 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.046 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.046 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.047 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.047 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.047 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T11:53:51.047 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T11:53:51.048 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.048 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.048 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.048 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.048 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.049 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.049 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.049 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.049 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.049 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.050 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.050 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T11:53:51.050 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T11:53:51.050 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.050 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.051 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.051 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.051 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.051 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.051 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.052 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.052 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.052 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.052 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.053 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T11:53:51.053 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T11:53:51.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.053 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.053 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.054 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.054 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.054 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.054 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.054 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.055 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.055 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.055 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.055 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T11:53:51.055 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T11:53:51.056 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.056 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.056 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.056 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.056 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.057 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.057 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.057 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.057 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.057 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.058 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.058 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T11:53:51.058 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T11:53:51.058 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.059 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.059 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.059 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.059 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.059 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.060 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.060 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.060 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.060 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.060 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.061 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T11:53:51.061 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T11:53:51.061 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.061 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.061 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.062 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.062 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.062 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.062 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.062 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.063 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.063 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.063 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.063 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T11:53:51.064 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T11:53:51.064 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.064 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.064 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.064 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.065 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.065 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.065 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.065 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.065 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.065 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.066 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.066 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T11:53:51.066 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T11:53:51.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.067 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.067 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.067 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.067 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.067 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.068 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.068 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.068 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.068 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.068 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.069 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T11:53:51.069 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T11:53:51.069 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.069 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.069 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.070 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T11:53:51.070 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.070 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.070 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.070 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.071 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.071 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.071 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.071 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T11:53:51.072 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T11:53:51.072 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.072 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.072 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.072 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:51.073 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.073 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.073 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.073 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.073 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.074 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.074 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.074 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T11:53:51.074 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T11:53:51.074 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.075 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.075 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.075 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8", 2023-07-14T11:53:51.075 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.075 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.076 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.076 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.076 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.076 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.076 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.077 INFO:teuthology.orchestra.run.smithi161.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T11:53:51.077 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T11:53:51.077 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.077 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.078 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.078 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "45", 2023-07-14T11:53:51.078 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.078 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.078 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.078 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.079 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.079 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.079 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.080 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T11:53:51.080 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T11:53:51.080 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.080 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.080 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.081 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.081 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.081 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.081 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.081 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.081 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.082 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.082 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.082 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T11:53:51.082 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T11:53:51.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.083 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.083 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.083 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:51.083 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.084 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.084 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.084 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.084 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.085 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.085 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T11:53:51.085 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T11:53:51.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.086 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.086 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.086 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.086 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.086 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.086 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.087 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.087 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.087 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.087 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.088 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T11:53:51.088 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T11:53:51.088 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.088 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.089 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.089 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.089 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.089 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.089 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.090 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.090 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.090 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.090 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T11:53:51.091 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T11:53:51.091 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.091 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.091 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.091 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:51.091 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.092 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.092 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.092 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.092 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.092 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.093 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.093 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T11:53:51.093 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T11:53:51.093 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.094 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.094 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.094 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:51.094 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.094 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.095 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.095 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.095 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.095 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.095 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.096 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T11:53:51.096 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T11:53:51.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.096 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.096 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.097 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:51.097 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.097 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.097 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.097 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.098 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.098 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.098 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.098 INFO:teuthology.orchestra.run.smithi161.stdout: "cross_origin_url": { 2023-07-14T11:53:51.099 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cross_origin_url", 2023-07-14T11:53:51.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.099 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.099 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.099 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.100 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.100 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.100 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.100 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.100 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.101 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.101 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.101 INFO:teuthology.orchestra.run.smithi161.stdout: "crt_file": { 2023-07-14T11:53:51.101 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crt_file", 2023-07-14T11:53:51.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.102 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.102 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.102 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.102 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.102 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.103 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.103 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.103 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.103 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.103 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.104 INFO:teuthology.orchestra.run.smithi161.stdout: "debug": { 2023-07-14T11:53:51.104 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "debug", 2023-07-14T11:53:51.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.104 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.105 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.105 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.105 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.105 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.105 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.105 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable/disable debug options", 2023-07-14T11:53:51.106 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.106 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.106 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.106 INFO:teuthology.orchestra.run.smithi161.stdout: "jwt_token_ttl": { 2023-07-14T11:53:51.107 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "jwt_token_ttl", 2023-07-14T11:53:51.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.107 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.107 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.107 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "28800", 2023-07-14T11:53:51.108 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.108 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.108 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.108 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.108 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.109 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.109 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.109 INFO:teuthology.orchestra.run.smithi161.stdout: "key_file": { 2023-07-14T11:53:51.109 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "key_file", 2023-07-14T11:53:51.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.110 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.110 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.110 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.110 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.110 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.111 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.111 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.111 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.111 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.111 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.112 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.112 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.112 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.112 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.113 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.113 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.113 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.113 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.113 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.114 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.114 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.114 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.114 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.114 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.115 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.115 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.115 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.115 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.115 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.116 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.116 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.116 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.116 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.117 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.117 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.117 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.117 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.117 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.118 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.118 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.118 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.118 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.118 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.119 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.119 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.119 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.119 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.120 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.120 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.120 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.120 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.120 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.121 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.121 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.121 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.121 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.121 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.121 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.122 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.122 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.122 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.122 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.122 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.123 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.123 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.123 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.124 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.124 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.124 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.124 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.124 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.125 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.125 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.125 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.125 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.126 INFO:teuthology.orchestra.run.smithi161.stdout: "motd": { 2023-07-14T11:53:51.126 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "motd", 2023-07-14T11:53:51.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.126 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.126 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.127 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.127 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.127 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.127 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.127 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "The message of the day", 2023-07-14T11:53:51.128 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.128 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.128 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.128 INFO:teuthology.orchestra.run.smithi161.stdout: "redirect_resolve_ip_addr": { 2023-07-14T11:53:51.129 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T11:53:51.129 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.129 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.129 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.129 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.130 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.130 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.130 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.130 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.130 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.130 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.131 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.131 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:51.131 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:51.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.132 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.132 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.132 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "::", 2023-07-14T11:53:51.132 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.132 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.133 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.133 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.133 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.133 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.133 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.134 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:51.134 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:51.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.134 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.134 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.135 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8080", 2023-07-14T11:53:51.135 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.135 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.135 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.135 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.136 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.136 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.136 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.136 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl": { 2023-07-14T11:53:51.137 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl", 2023-07-14T11:53:51.137 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.137 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.137 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.137 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.138 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.138 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.138 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.138 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.138 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.138 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.139 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.139 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl_server_port": { 2023-07-14T11:53:51.139 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl_server_port", 2023-07-14T11:53:51.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.140 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.140 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.140 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8443", 2023-07-14T11:53:51.140 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.140 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.141 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.141 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.141 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.141 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.141 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.142 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_behaviour": { 2023-07-14T11:53:51.142 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_behaviour", 2023-07-14T11:53:51.142 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.142 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.142 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.143 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "redirect", 2023-07-14T11:53:51.143 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.143 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.143 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.143 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.144 INFO:teuthology.orchestra.run.smithi161.stdout: "redirect" 2023-07-14T11:53:51.144 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.144 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.144 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.145 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.145 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_error_status_code": { 2023-07-14T11:53:51.145 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_error_status_code", 2023-07-14T11:53:51.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.146 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.146 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.146 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "500", 2023-07-14T11:53:51.146 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "400", 2023-07-14T11:53:51.146 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "599", 2023-07-14T11:53:51.147 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.147 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.147 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.147 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.147 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.148 INFO:teuthology.orchestra.run.smithi161.stdout: "url_prefix": { 2023-07-14T11:53:51.148 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "url_prefix", 2023-07-14T11:53:51.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.148 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.149 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.149 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.149 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.149 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.149 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.149 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.150 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.150 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.150 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.150 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.150 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.151 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "devicehealth", 2023-07-14T11:53:51.151 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.151 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.152 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.152 INFO:teuthology.orchestra.run.smithi161.stdout: "enable_monitoring": { 2023-07-14T11:53:51.152 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enable_monitoring", 2023-07-14T11:53:51.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.152 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.153 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.153 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.153 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.153 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.154 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "monitor device health metrics", 2023-07-14T11:53:51.154 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.154 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.154 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.155 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.155 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.155 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.155 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.155 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.156 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.156 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.156 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.156 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.156 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.157 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.157 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.157 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.157 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.157 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.158 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.158 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.158 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.158 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.158 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.159 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.159 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.159 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.159 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.160 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.160 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.160 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.160 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.160 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.160 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.161 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.161 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.161 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.161 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.162 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.162 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.162 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.162 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.162 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.163 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.163 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.163 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.163 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.163 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.164 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.164 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.164 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.164 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.164 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.164 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.165 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.165 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.165 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.165 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.166 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.166 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.166 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.166 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.166 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.167 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.167 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.167 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.167 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.167 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.167 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.168 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.168 INFO:teuthology.orchestra.run.smithi161.stdout: "mark_out_threshold": { 2023-07-14T11:53:51.168 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mark_out_threshold", 2023-07-14T11:53:51.168 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.169 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.169 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "2419200", 2023-07-14T11:53:51.169 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.169 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.170 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.170 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T11:53:51.170 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.170 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.170 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.171 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": { 2023-07-14T11:53:51.171 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pool_name", 2023-07-14T11:53:51.171 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.171 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.171 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.172 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "device_health_metrics", 2023-07-14T11:53:51.172 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.172 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.172 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.172 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T11:53:51.173 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.173 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.173 INFO:teuthology.orchestra.run.smithi161.stdout: "retention_period": { 2023-07-14T11:53:51.174 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "retention_period", 2023-07-14T11:53:51.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.174 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.174 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.174 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15552000", 2023-07-14T11:53:51.175 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.175 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.175 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.175 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long to retain device health metrics", 2023-07-14T11:53:51.175 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.175 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.176 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.176 INFO:teuthology.orchestra.run.smithi161.stdout: "scrape_frequency": { 2023-07-14T11:53:51.176 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "scrape_frequency", 2023-07-14T11:53:51.176 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.177 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.177 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.177 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "86400", 2023-07-14T11:53:51.177 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.177 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.178 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.178 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T11:53:51.178 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.178 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.178 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.179 INFO:teuthology.orchestra.run.smithi161.stdout: "self_heal": { 2023-07-14T11:53:51.179 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "self_heal", 2023-07-14T11:53:51.179 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.179 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.179 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.180 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.180 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.180 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.180 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.181 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T11:53:51.181 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.181 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.181 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.182 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:51.182 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:51.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.182 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.182 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.182 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:51.183 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.183 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.183 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.183 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T11:53:51.183 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.184 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.184 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_threshold": { 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_threshold", 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "7257600", 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.186 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.186 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.186 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T11:53:51.186 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.186 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.187 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.188 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "diskprediction_local", 2023-07-14T11:53:51.188 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.188 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.188 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.189 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.189 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.189 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.189 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.189 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.189 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.190 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.190 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.190 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.190 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.190 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.191 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.191 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.191 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.191 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.191 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.192 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.192 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.192 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.193 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.193 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.193 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.193 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.193 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.194 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.194 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.194 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.194 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.194 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.194 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.195 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.195 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.195 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.195 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.196 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.196 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.196 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.196 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.197 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.197 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.197 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.197 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.197 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.198 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.198 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.198 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.198 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.198 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.199 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.199 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.199 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.199 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.200 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.200 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.200 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.200 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.201 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.201 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.201 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.201 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.201 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.201 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.202 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.202 INFO:teuthology.orchestra.run.smithi161.stdout: "predict_interval": { 2023-07-14T11:53:51.202 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "predict_interval", 2023-07-14T11:53:51.202 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.203 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.203 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.203 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "86400", 2023-07-14T11:53:51.203 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.203 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.204 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.204 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.204 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.204 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.204 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.205 INFO:teuthology.orchestra.run.smithi161.stdout: "predictor_model": { 2023-07-14T11:53:51.205 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "predictor_model", 2023-07-14T11:53:51.205 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.205 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.205 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.206 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "prophetstor", 2023-07-14T11:53:51.206 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.206 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.206 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.206 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.207 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.207 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.207 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.207 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:51.208 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:51.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.208 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.208 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.208 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:51.209 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.209 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.209 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.209 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.209 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.210 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.210 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.211 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "influx", 2023-07-14T11:53:51.211 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": false, 2023-07-14T11:53:51.211 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "influxdb python module not found", 2023-07-14T11:53:51.211 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.212 INFO:teuthology.orchestra.run.smithi161.stdout: "batch_size": { 2023-07-14T11:53:51.212 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "batch_size", 2023-07-14T11:53:51.212 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.212 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.212 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.213 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5000", 2023-07-14T11:53:51.213 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.213 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.213 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.213 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T11:53:51.214 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.214 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.214 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.214 INFO:teuthology.orchestra.run.smithi161.stdout: "database": { 2023-07-14T11:53:51.215 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "database", 2023-07-14T11:53:51.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.215 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.215 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.215 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "ceph", 2023-07-14T11:53:51.216 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.216 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.216 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.216 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T11:53:51.216 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.217 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.217 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.217 INFO:teuthology.orchestra.run.smithi161.stdout: "hostname": { 2023-07-14T11:53:51.217 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "hostname", 2023-07-14T11:53:51.218 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.218 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.218 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.218 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.218 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.219 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.219 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.219 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB server hostname", 2023-07-14T11:53:51.219 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.219 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.220 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.220 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:51.220 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:51.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.221 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.221 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.221 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "30", 2023-07-14T11:53:51.221 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "5", 2023-07-14T11:53:51.221 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.221 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.222 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T11:53:51.222 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.222 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.222 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.223 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.223 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.223 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.223 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.224 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.224 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.224 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.224 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.224 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.225 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.225 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.225 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.225 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.226 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.226 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.226 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.226 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.226 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.227 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.227 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.227 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.227 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.228 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.228 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.228 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.228 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.228 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.228 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.229 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.229 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.229 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.229 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.230 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.230 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.230 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.230 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.230 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.231 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.231 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.231 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.231 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.231 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.232 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.232 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.232 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.232 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.232 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.233 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.233 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.233 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.233 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.233 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.234 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.234 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.234 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.234 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.234 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.235 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.235 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.235 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.235 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.235 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.236 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.236 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.236 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.236 INFO:teuthology.orchestra.run.smithi161.stdout: "password": { 2023-07-14T11:53:51.237 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "password", 2023-07-14T11:53:51.237 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.237 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.237 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.237 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.238 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.238 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.238 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.238 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "password of InfluxDB server user", 2023-07-14T11:53:51.238 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.238 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.239 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.239 INFO:teuthology.orchestra.run.smithi161.stdout: "port": { 2023-07-14T11:53:51.239 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "port", 2023-07-14T11:53:51.239 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.240 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.240 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.240 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8086", 2023-07-14T11:53:51.240 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.240 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.241 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.241 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB server port", 2023-07-14T11:53:51.241 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.241 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.241 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.242 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl": { 2023-07-14T11:53:51.242 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl", 2023-07-14T11:53:51.242 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.242 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.242 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.243 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "false", 2023-07-14T11:53:51.243 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.243 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.243 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.243 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T11:53:51.244 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.244 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.244 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.244 INFO:teuthology.orchestra.run.smithi161.stdout: "threads": { 2023-07-14T11:53:51.245 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "threads", 2023-07-14T11:53:51.245 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.245 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.245 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.245 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:51.246 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:51.246 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "32", 2023-07-14T11:53:51.246 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.246 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T11:53:51.246 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.246 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.247 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.247 INFO:teuthology.orchestra.run.smithi161.stdout: "username": { 2023-07-14T11:53:51.247 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "username", 2023-07-14T11:53:51.247 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.248 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.248 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.248 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.248 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.248 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.249 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.249 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "username of InfluxDB server user", 2023-07-14T11:53:51.249 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.249 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.249 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.250 INFO:teuthology.orchestra.run.smithi161.stdout: "verify_ssl": { 2023-07-14T11:53:51.250 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "verify_ssl", 2023-07-14T11:53:51.250 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.250 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.250 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.251 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "true", 2023-07-14T11:53:51.251 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.251 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.251 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.251 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T11:53:51.252 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.252 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.252 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.253 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "insights", 2023-07-14T11:53:51.253 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.253 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.254 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.254 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.254 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.254 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.254 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.255 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.255 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.255 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.255 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.255 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.256 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.256 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.256 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.256 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.256 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.257 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.257 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.257 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.257 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.257 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.258 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.258 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.258 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.258 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.259 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.259 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.259 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.259 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.259 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.260 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.260 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.260 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.260 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.260 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.261 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.261 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.261 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.261 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.262 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.262 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.262 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.262 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.262 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.263 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.263 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.263 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.263 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.263 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.263 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.264 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.264 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.264 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.264 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.264 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.265 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.265 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.265 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.265 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.266 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.266 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.266 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.266 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.266 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.267 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.267 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.267 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.267 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.267 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.267 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.268 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.268 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "iostat", 2023-07-14T11:53:51.268 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.268 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.269 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.269 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.269 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.269 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.269 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.270 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.270 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.270 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.270 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.270 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.271 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.271 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.271 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.271 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.272 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.272 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.272 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.272 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.272 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.273 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.273 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.273 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.273 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.273 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.274 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.274 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.274 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.274 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.274 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.275 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.275 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.275 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.275 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.275 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.276 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.276 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.276 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.276 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.277 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.277 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.277 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.277 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.277 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.277 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.278 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.278 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.278 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.278 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.278 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.279 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.279 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.279 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.279 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.279 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.280 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.280 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.280 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.280 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.282 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.282 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.282 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.282 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.282 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.283 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "k8sevents", 2023-07-14T11:53:51.283 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.283 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.284 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.284 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_event_retention_days": { 2023-07-14T11:53:51.284 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ceph_event_retention_days", 2023-07-14T11:53:51.284 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.284 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.285 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.285 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "7", 2023-07-14T11:53:51.285 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.285 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.285 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.285 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T11:53:51.286 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.286 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.286 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.286 INFO:teuthology.orchestra.run.smithi161.stdout: "config_check_secs": { 2023-07-14T11:53:51.287 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_check_secs", 2023-07-14T11:53:51.287 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.287 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.287 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.287 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:51.288 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "10", 2023-07-14T11:53:51.288 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.288 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.288 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T11:53:51.288 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.289 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.289 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.289 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.289 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.290 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.290 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.290 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.290 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.290 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.291 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.291 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.291 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.291 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.291 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.291 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.292 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.292 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.292 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.292 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.292 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.293 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.293 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.293 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.293 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.294 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.294 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.294 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.294 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.294 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.295 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.295 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.295 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.295 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.295 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.295 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.296 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.296 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.296 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.296 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.297 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.297 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.297 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.297 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.297 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.298 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.298 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.298 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.298 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.298 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.299 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.299 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.299 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.299 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.300 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.300 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.300 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.300 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.301 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.301 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.301 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.301 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.301 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.302 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.302 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.302 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.302 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.302 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.303 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.303 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.303 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.303 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.304 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "localpool", 2023-07-14T11:53:51.304 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.304 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.304 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.304 INFO:teuthology.orchestra.run.smithi161.stdout: "failure_domain": { 2023-07-14T11:53:51.305 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "failure_domain", 2023-07-14T11:53:51.305 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.305 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.305 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.305 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "host", 2023-07-14T11:53:51.306 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.306 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.306 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.306 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "failure domain for any created local pool", 2023-07-14T11:53:51.306 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T11:53:51.306 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.307 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.307 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.307 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.307 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.308 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.308 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.308 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.308 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.308 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.309 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.309 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.309 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.309 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.309 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.310 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.310 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.310 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.310 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.310 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.311 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.311 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.311 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.311 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.311 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.312 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.312 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.312 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.312 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.313 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.313 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.313 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.313 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.313 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.313 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.314 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.314 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.314 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.314 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.315 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.315 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.315 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.315 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.316 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.316 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.316 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.316 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.316 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.317 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.317 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.317 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.317 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.317 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.318 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.318 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.318 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.318 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.319 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.319 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.319 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.319 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.319 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.319 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.320 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.320 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.320 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.320 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.320 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.321 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": { 2023-07-14T11:53:51.321 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "min_size", 2023-07-14T11:53:51.321 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.321 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.322 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.322 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.322 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.322 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.322 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.323 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default min_size for any created local pool", 2023-07-14T11:53:51.323 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T11:53:51.323 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.323 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.324 INFO:teuthology.orchestra.run.smithi161.stdout: "num_rep": { 2023-07-14T11:53:51.324 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "num_rep", 2023-07-14T11:53:51.324 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.324 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.324 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.324 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3", 2023-07-14T11:53:51.325 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.325 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.325 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.325 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default replica count for any created local pool", 2023-07-14T11:53:51.325 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.326 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.326 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.326 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": { 2023-07-14T11:53:51.326 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pg_num", 2023-07-14T11:53:51.327 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.327 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.327 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.327 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "128", 2023-07-14T11:53:51.327 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.328 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.328 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.328 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T11:53:51.328 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.328 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.328 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.329 INFO:teuthology.orchestra.run.smithi161.stdout: "prefix": { 2023-07-14T11:53:51.329 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prefix", 2023-07-14T11:53:51.329 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.329 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.330 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.330 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.330 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.330 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.330 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.331 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "name prefix for any created local pool", 2023-07-14T11:53:51.331 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.331 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.331 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.332 INFO:teuthology.orchestra.run.smithi161.stdout: "subtree": { 2023-07-14T11:53:51.332 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "subtree", 2023-07-14T11:53:51.332 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.332 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.332 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.333 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "rack", 2023-07-14T11:53:51.333 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.333 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.333 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.333 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T11:53:51.333 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T11:53:51.334 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.334 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.334 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.334 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.335 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mds_autoscaler", 2023-07-14T11:53:51.335 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.335 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.335 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.336 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.336 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.336 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.336 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.336 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.337 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.337 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.337 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.337 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.337 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.338 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.338 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.338 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.338 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.338 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.339 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.339 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.339 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.339 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.339 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.340 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.340 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.340 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.340 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.341 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.341 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.341 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.341 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.341 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.341 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.342 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.342 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.342 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.342 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.343 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.343 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.343 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.343 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.343 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.344 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.344 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.344 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.344 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.344 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.345 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.345 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.345 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.345 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.345 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.347 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.347 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.347 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.347 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.347 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.348 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.348 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.348 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.348 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.348 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.349 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.349 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.350 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mirroring", 2023-07-14T11:53:51.350 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.350 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.350 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.350 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.351 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.351 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.351 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.351 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.351 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.352 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.352 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.352 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.352 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.352 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.353 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.353 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.353 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.353 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.354 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.354 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.354 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.354 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.354 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.355 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.355 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.355 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.355 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.355 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.356 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.356 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.356 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.356 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.356 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.357 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.357 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.357 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.358 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.358 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.358 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.358 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.358 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.359 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.359 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.359 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.359 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.359 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.360 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.360 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.360 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.360 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.360 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.361 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.361 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.361 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.361 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.361 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.362 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.362 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.362 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.362 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.363 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.363 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.363 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.363 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.363 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.364 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.364 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.364 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.364 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.364 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.365 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.365 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "nfs", 2023-07-14T11:53:51.365 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.365 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.366 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.366 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.366 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.366 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.366 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.367 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.367 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.367 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.367 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.367 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.368 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.368 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.368 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.368 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.368 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.369 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.369 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.369 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.369 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.369 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.370 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.370 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.370 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.370 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.370 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.371 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.371 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.371 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.371 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.371 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.372 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.372 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.372 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.372 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.373 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.373 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.373 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.373 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.373 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.374 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.374 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.374 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.374 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.374 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.375 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.375 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.375 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.375 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.375 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.376 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.376 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.376 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.376 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.376 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.377 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.377 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.377 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.377 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.379 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.379 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.379 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.379 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.379 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "orchestrator", 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi161.stdout: "fail_fs": { 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "fail_fs", 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.384 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.384 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.384 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.384 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.385 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.385 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.385 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.385 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.385 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.386 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.386 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.386 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.386 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.386 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.388 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.388 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.388 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.391 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.391 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.391 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.391 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.392 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.392 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.392 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.392 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.392 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.393 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.393 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.393 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.393 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.393 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.395 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.395 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.395 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.396 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.396 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.396 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.396 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.396 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.397 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.397 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.397 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.397 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.397 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.398 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.398 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.398 INFO:teuthology.orchestra.run.smithi161.stdout: "orchestrator": { 2023-07-14T11:53:51.398 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "orchestrator", 2023-07-14T11:53:51.399 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.399 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.399 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.399 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.399 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.400 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.400 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.400 INFO:teuthology.orchestra.run.smithi161.stdout: "cephadm", 2023-07-14T11:53:51.400 INFO:teuthology.orchestra.run.smithi161.stdout: "rook", 2023-07-14T11:53:51.400 INFO:teuthology.orchestra.run.smithi161.stdout: "test_orchestrator" 2023-07-14T11:53:51.400 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.401 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Orchestrator backend", 2023-07-14T11:53:51.401 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.401 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.401 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.401 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.402 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.402 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "osd_perf_query", 2023-07-14T11:53:51.402 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.403 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.403 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.403 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.403 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.403 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.403 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.404 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.404 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.404 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.404 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.404 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.405 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.405 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.405 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.405 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.405 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.406 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.406 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.406 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.406 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.406 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.407 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.407 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.407 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.407 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.408 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.408 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.408 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.408 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.409 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.409 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.409 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.409 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.409 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.410 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.410 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.410 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.410 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.411 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.411 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.411 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.411 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.411 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.412 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.412 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.412 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.412 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.412 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.413 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.413 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.413 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.413 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.413 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.414 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.414 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.414 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.415 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.415 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.415 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.415 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.415 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.416 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.416 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.416 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.416 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.417 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.418 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "osd_support", 2023-07-14T11:53:51.418 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.418 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.418 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.418 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.418 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.419 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.419 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.419 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.419 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.420 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.420 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.420 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.420 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.420 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.421 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.421 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.421 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.421 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.421 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.422 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.422 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.422 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.422 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.423 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.423 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.423 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.423 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.423 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.423 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.424 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.424 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.424 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.424 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.425 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.425 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.425 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.425 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.425 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.426 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.426 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.426 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.426 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.426 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.427 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.427 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.427 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.427 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.427 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.428 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.428 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.428 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.428 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.428 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.429 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.429 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.429 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.429 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.430 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.430 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.430 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.430 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.430 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.431 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.431 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.431 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.431 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.431 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.431 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.432 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pg_autoscaler", 2023-07-14T11:53:51.433 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.433 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.433 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.433 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.433 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.434 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.434 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.434 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.434 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.434 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.435 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.435 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.435 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.435 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.435 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.435 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.436 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.436 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.436 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.436 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.436 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.437 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.437 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.437 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.437 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.438 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.438 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.438 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.438 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.438 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.440 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.440 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.440 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.440 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.441 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.441 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.441 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.441 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.441 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.442 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.442 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.442 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.442 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.442 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.443 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.443 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.443 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.443 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.443 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.443 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.444 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.444 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.444 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.444 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.445 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.445 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.445 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.445 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.445 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.446 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.446 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.446 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.446 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.446 INFO:teuthology.orchestra.run.smithi161.stdout: "noautoscale": { 2023-07-14T11:53:51.447 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "noautoscale", 2023-07-14T11:53:51.447 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.447 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.447 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.447 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.448 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.448 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.448 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.448 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "global autoscale flag", 2023-07-14T11:53:51.448 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T11:53:51.449 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.449 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.449 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:51.449 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:51.450 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.450 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.450 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.450 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:51.450 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.451 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.451 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.451 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.451 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.451 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.452 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.452 INFO:teuthology.orchestra.run.smithi161.stdout: "threshold": { 2023-07-14T11:53:51.452 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "threshold", 2023-07-14T11:53:51.452 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.453 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.453 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.453 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3.0", 2023-07-14T11:53:51.453 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1.0", 2023-07-14T11:53:51.453 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.454 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.454 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "scaling threshold", 2023-07-14T11:53:51.454 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T11:53:51.454 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.454 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.454 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.455 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "progress", 2023-07-14T11:53:51.455 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.456 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.456 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.456 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_pg_recovery_event": { 2023-07-14T11:53:51.456 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_pg_recovery_event", 2023-07-14T11:53:51.456 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.457 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.457 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.457 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.457 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.457 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.458 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.458 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T11:53:51.458 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.458 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.458 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.459 INFO:teuthology.orchestra.run.smithi161.stdout: "enabled": { 2023-07-14T11:53:51.459 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enabled", 2023-07-14T11:53:51.459 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.459 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.459 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.460 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.460 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.460 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.460 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.460 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.461 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.461 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.461 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.461 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.462 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.462 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.462 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.462 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.462 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.463 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.463 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.463 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.463 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.463 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.463 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.464 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.464 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.464 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.464 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.464 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.465 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.465 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.465 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.465 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.466 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.466 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.466 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.466 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.466 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.467 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.467 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.467 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.467 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.467 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.467 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.468 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.468 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.468 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.468 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.469 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.469 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.469 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.469 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.469 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.470 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.470 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.470 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.470 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.470 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.471 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.471 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.471 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.471 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.471 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.472 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.472 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.472 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.472 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.473 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.473 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.473 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.473 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.473 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.474 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.474 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.474 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.474 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.474 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.475 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.475 INFO:teuthology.orchestra.run.smithi161.stdout: "max_completed_events": { 2023-07-14T11:53:51.475 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_completed_events", 2023-07-14T11:53:51.475 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.476 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.476 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.476 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "50", 2023-07-14T11:53:51.476 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.476 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.477 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.477 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "number of past completed events to remember", 2023-07-14T11:53:51.477 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.477 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.477 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.478 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:51.478 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:51.478 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.478 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.478 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.479 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:51.479 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.479 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.479 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.479 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long the module is going to sleep", 2023-07-14T11:53:51.480 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.480 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.480 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.480 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.480 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.481 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus", 2023-07-14T11:53:51.481 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.481 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.482 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.482 INFO:teuthology.orchestra.run.smithi161.stdout: "cache": { 2023-07-14T11:53:51.482 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cache", 2023-07-14T11:53:51.482 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.482 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.482 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.483 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.483 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.483 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.483 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.483 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: "exclude_perf_counters": { 2023-07-14T11:53:51.485 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "exclude_perf_counters", 2023-07-14T11:53:51.485 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.485 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.485 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.485 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.486 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.486 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.486 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.486 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T11:53:51.486 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T11:53:51.487 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.487 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.487 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.487 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.488 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.488 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.488 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.488 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.488 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.489 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.489 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.489 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.489 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.489 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.490 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.490 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.490 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.490 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.490 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.490 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.491 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.491 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.491 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.491 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.492 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.492 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.492 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.492 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.492 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.493 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.493 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.493 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.493 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.493 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.494 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.494 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.494 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.494 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.495 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.495 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.495 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.495 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.495 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.495 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.496 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.496 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.496 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.496 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.496 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.497 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.497 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.497 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.497 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.497 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.498 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.498 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.498 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.498 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.499 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.499 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.499 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.499 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.499 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.499 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.500 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.500 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.500 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.500 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.501 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_stats_pools": { 2023-07-14T11:53:51.501 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_stats_pools", 2023-07-14T11:53:51.501 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.501 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.501 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.502 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.502 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.502 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.502 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.502 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.503 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.503 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.503 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.503 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T11:53:51.503 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T11:53:51.504 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.504 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.504 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.504 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "300", 2023-07-14T11:53:51.504 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.505 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.505 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.505 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.505 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.505 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.506 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.506 INFO:teuthology.orchestra.run.smithi161.stdout: "scrape_interval": { 2023-07-14T11:53:51.506 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "scrape_interval", 2023-07-14T11:53:51.506 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.507 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.507 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.507 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15.0", 2023-07-14T11:53:51.507 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.507 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.507 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.508 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.508 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.508 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.508 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.509 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:51.509 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:51.509 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.509 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.509 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.510 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "::", 2023-07-14T11:53:51.510 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.510 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.510 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.510 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T11:53:51.511 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.511 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.511 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.511 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:51.512 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:51.512 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.512 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.512 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.512 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "9283", 2023-07-14T11:53:51.513 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.513 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.513 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.513 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T11:53:51.513 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.513 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.514 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.514 INFO:teuthology.orchestra.run.smithi161.stdout: "stale_cache_strategy": { 2023-07-14T11:53:51.514 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "stale_cache_strategy", 2023-07-14T11:53:51.515 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.515 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.515 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.515 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "log", 2023-07-14T11:53:51.515 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.516 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.516 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.516 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.516 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.516 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.516 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.517 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_behaviour": { 2023-07-14T11:53:51.517 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_behaviour", 2023-07-14T11:53:51.517 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.517 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.518 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.518 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "default", 2023-07-14T11:53:51.518 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.518 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.518 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.519 INFO:teuthology.orchestra.run.smithi161.stdout: "default", 2023-07-14T11:53:51.519 INFO:teuthology.orchestra.run.smithi161.stdout: "error" 2023-07-14T11:53:51.519 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.519 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.519 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.520 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.520 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.520 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_error_status_code": { 2023-07-14T11:53:51.520 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_error_status_code", 2023-07-14T11:53:51.521 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.521 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.521 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.521 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "500", 2023-07-14T11:53:51.521 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "400", 2023-07-14T11:53:51.521 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "599", 2023-07-14T11:53:51.522 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.522 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.522 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.522 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.522 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.523 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.523 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.523 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_support", 2023-07-14T11:53:51.524 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.524 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.524 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.524 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.524 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.525 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.525 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.525 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.525 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.525 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.526 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.526 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.526 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.526 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.526 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.527 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.527 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.527 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.527 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.527 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.528 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.528 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.528 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.528 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.529 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.529 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.529 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.529 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.529 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.529 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.530 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.530 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.530 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.530 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.530 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.531 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.531 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.531 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.531 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.532 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.532 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.532 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.532 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.532 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.533 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.533 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.533 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.533 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.533 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.534 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.534 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.534 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.534 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.534 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.535 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.535 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.535 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.535 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.535 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.536 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.536 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.536 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.536 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.536 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.537 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.537 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.537 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.537 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.537 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.538 INFO:teuthology.orchestra.run.smithi161.stdout: "max_concurrent_snap_create": { 2023-07-14T11:53:51.538 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_concurrent_snap_create", 2023-07-14T11:53:51.538 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.538 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.539 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.539 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:51.539 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.539 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.539 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.539 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.540 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.540 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.540 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.540 INFO:teuthology.orchestra.run.smithi161.stdout: "mirror_snapshot_schedule": { 2023-07-14T11:53:51.541 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T11:53:51.541 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.541 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.541 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.541 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.542 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.542 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.542 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.542 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.542 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.543 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.543 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.543 INFO:teuthology.orchestra.run.smithi161.stdout: "trash_purge_schedule": { 2023-07-14T11:53:51.543 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "trash_purge_schedule", 2023-07-14T11:53:51.544 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.544 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.544 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.544 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.544 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.544 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.545 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.545 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.545 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.545 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.545 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.546 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "restful", 2023-07-14T11:53:51.547 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.547 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.547 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.547 INFO:teuthology.orchestra.run.smithi161.stdout: "enable_auth": { 2023-07-14T11:53:51.547 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enable_auth", 2023-07-14T11:53:51.548 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.548 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.548 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.548 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.548 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.549 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.549 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.549 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.549 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.549 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.550 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.550 INFO:teuthology.orchestra.run.smithi161.stdout: "key_file": { 2023-07-14T11:53:51.550 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "key_file", 2023-07-14T11:53:51.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.551 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.551 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.551 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.551 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.551 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.552 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.552 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.552 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.552 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.552 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.553 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.553 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.553 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.553 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.554 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.554 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.554 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.554 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.554 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.555 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.555 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.555 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.555 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.555 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.555 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.556 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.556 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.556 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.556 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.556 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.557 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.557 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.557 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.557 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.558 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.558 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.558 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.558 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.558 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.559 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.559 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.559 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.559 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.559 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.560 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.560 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.560 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.560 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.561 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.561 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.561 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.561 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.562 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.562 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.562 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.562 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.562 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.563 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.563 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.563 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.563 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.563 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.564 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.564 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.564 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.565 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.565 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.565 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.565 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.565 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.566 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.566 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.566 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.566 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.567 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:51.567 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:51.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.567 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.568 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.568 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.568 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.568 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.568 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.569 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.569 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.569 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.569 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:51.570 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:51.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.570 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.570 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.570 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.570 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.571 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.571 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.571 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.571 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.571 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.572 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.573 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rgw", 2023-07-14T11:53:51.573 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.573 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.573 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.573 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.574 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.574 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.574 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.574 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.575 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.575 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.575 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.575 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.575 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.576 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.576 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.576 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.576 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.576 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.577 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.577 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.577 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.577 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.578 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.578 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.578 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.578 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.578 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.579 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.579 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.579 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.579 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.579 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.579 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.580 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.580 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.580 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.581 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.581 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.581 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.581 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.581 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.582 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.582 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.582 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.582 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.582 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.583 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.583 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.583 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.583 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.583 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.583 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.584 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.584 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.584 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.585 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.585 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.585 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.585 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.585 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.585 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.586 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.586 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.586 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.586 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.586 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.587 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.588 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rook", 2023-07-14T11:53:51.588 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.588 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.588 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.588 INFO:teuthology.orchestra.run.smithi161.stdout: "drive_group_interval": { 2023-07-14T11:53:51.589 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "drive_group_interval", 2023-07-14T11:53:51.589 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.589 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.589 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.589 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "300.0", 2023-07-14T11:53:51.589 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.590 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.590 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.590 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T11:53:51.590 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.590 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.591 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.591 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.591 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.591 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.592 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.592 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.592 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.592 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.592 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.593 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.593 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.593 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.593 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.593 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.594 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.594 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.594 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.594 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.594 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.594 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.595 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.595 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.595 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "storage_class": { 2023-07-14T11:53:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "storage_class", 2023-07-14T11:53:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "local", 2023-07-14T11:53:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T11:53:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "selftest", 2023-07-14T11:53:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "roption1": { 2023-07-14T11:53:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "roption1", 2023-07-14T11:53:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "roption2": { 2023-07-14T11:53:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "roption2", 2023-07-14T11:53:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "xyz", 2023-07-14T11:53:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption1": { 2023-07-14T11:53:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption1", 2023-07-14T11:53:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption2": { 2023-07-14T11:53:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption2", 2023-07-14T11:53:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption3": { 2023-07-14T11:53:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption3", 2023-07-14T11:53:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption4": { 2023-07-14T11:53:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption4", 2023-07-14T11:53:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption5": { 2023-07-14T11:53:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption5", 2023-07-14T11:53:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption6": { 2023-07-14T11:53:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption6", 2023-07-14T11:53:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption7": { 2023-07-14T11:53:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption7", 2023-07-14T11:53:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "42", 2023-07-14T11:53:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "testkey": { 2023-07-14T11:53:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testkey", 2023-07-14T11:53:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "testlkey": { 2023-07-14T11:53:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testlkey", 2023-07-14T11:53:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "testnewline": { 2023-07-14T11:53:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testnewline", 2023-07-14T11:53:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snap_schedule", 2023-07-14T11:53:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_m_granularity": { 2023-07-14T11:53:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_m_granularity", 2023-07-14T11:53:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T11:53:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "dump_on_update": { 2023-07-14T11:53:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "dump_on_update", 2023-07-14T11:53:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "dump database to debug log on update", 2023-07-14T11:53:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "stats", 2023-07-14T11:53:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.688 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.688 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.688 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.688 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.689 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.689 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.689 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.689 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.689 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.690 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.690 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.690 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.690 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.690 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.691 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "status", 2023-07-14T11:53:51.691 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.692 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.692 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.692 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.692 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.692 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.693 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.693 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.693 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.693 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.693 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.694 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.694 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.694 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.694 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.694 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.695 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.695 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.695 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.695 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.695 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.695 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.696 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.696 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.696 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.696 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.697 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.697 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.697 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.697 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.697 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.698 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.698 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.698 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.698 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.698 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.699 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.699 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.699 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.699 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.700 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.700 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.700 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.700 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.700 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.701 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.701 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.701 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.701 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.701 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.702 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.702 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.702 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.702 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.702 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.703 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.703 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.703 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.703 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.704 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.704 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.704 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.704 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.704 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.705 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.705 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.705 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.705 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.705 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.706 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "telegraf", 2023-07-14T11:53:51.706 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.707 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.707 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.707 INFO:teuthology.orchestra.run.smithi161.stdout: "address": { 2023-07-14T11:53:51.707 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "address", 2023-07-14T11:53:51.707 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.708 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.708 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.708 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T11:53:51.708 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.708 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.709 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.709 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.709 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.709 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.709 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.710 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:51.710 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:51.710 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.710 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.711 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.711 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15", 2023-07-14T11:53:51.711 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.711 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.711 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.711 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.712 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.712 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.712 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.712 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.713 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.713 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.713 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.713 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.713 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.714 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.714 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.714 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.714 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.714 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.715 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.715 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.715 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.715 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.715 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.716 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.716 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.716 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.716 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.717 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.717 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.717 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.717 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.717 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.717 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.718 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.718 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.718 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.718 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.718 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.719 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.719 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.719 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.719 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.720 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.720 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.720 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.720 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.720 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.721 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.721 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.721 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.721 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.721 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.722 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.722 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.722 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.722 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.722 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.722 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.723 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.723 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.723 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.723 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.724 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.724 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.724 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.724 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.724 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.725 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.725 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.725 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.725 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.725 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.726 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.726 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.726 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.727 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "telemetry", 2023-07-14T11:53:51.727 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.727 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.727 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.727 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_basic": { 2023-07-14T11:53:51.727 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_basic", 2023-07-14T11:53:51.728 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.728 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.728 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.728 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.728 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.729 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.729 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.729 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T11:53:51.729 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.729 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.730 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.730 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_crash": { 2023-07-14T11:53:51.730 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_crash", 2023-07-14T11:53:51.730 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.730 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.731 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.731 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.731 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.731 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.731 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.732 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T11:53:51.732 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.732 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.732 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.733 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_device": { 2023-07-14T11:53:51.733 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_device", 2023-07-14T11:53:51.733 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.733 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.733 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.734 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.734 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.734 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.734 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.734 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T11:53:51.735 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.735 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.735 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.735 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_ident": { 2023-07-14T11:53:51.735 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_ident", 2023-07-14T11:53:51.736 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.736 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.736 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.736 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.736 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.737 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.737 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.737 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T11:53:51.737 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.737 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.738 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.738 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_perf": { 2023-07-14T11:53:51.738 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_perf", 2023-07-14T11:53:51.738 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.739 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.739 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.739 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.739 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.739 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.740 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.740 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T11:53:51.740 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.740 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.740 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.741 INFO:teuthology.orchestra.run.smithi161.stdout: "contact": { 2023-07-14T11:53:51.741 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "contact", 2023-07-14T11:53:51.741 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.741 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.741 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.742 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.742 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.742 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.742 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.743 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.743 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.743 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.744 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.744 INFO:teuthology.orchestra.run.smithi161.stdout: "description": { 2023-07-14T11:53:51.744 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "description", 2023-07-14T11:53:51.745 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.745 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.745 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.745 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.745 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.746 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.746 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.746 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.746 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.746 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.747 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.747 INFO:teuthology.orchestra.run.smithi161.stdout: "device_url": { 2023-07-14T11:53:51.747 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_url", 2023-07-14T11:53:51.747 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.747 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.748 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.748 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T11:53:51.748 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.748 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.748 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.749 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.749 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.749 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.749 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.750 INFO:teuthology.orchestra.run.smithi161.stdout: "enabled": { 2023-07-14T11:53:51.750 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enabled", 2023-07-14T11:53:51.750 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.750 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.750 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.751 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.751 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.751 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.751 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.751 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.752 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.752 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.752 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.752 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:51.753 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:51.753 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.753 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.753 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.753 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "24", 2023-07-14T11:53:51.754 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "8", 2023-07-14T11:53:51.754 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.754 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.754 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.754 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.754 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.755 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.755 INFO:teuthology.orchestra.run.smithi161.stdout: "last_opt_revision": { 2023-07-14T11:53:51.755 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "last_opt_revision", 2023-07-14T11:53:51.755 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.756 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.756 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.756 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1", 2023-07-14T11:53:51.756 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.756 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.757 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.757 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.757 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.757 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.757 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.758 INFO:teuthology.orchestra.run.smithi161.stdout: "leaderboard": { 2023-07-14T11:53:51.758 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "leaderboard", 2023-07-14T11:53:51.758 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.758 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.758 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.759 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.759 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.759 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.759 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.759 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.760 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.760 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.760 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.760 INFO:teuthology.orchestra.run.smithi161.stdout: "leaderboard_description": { 2023-07-14T11:53:51.761 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "leaderboard_description", 2023-07-14T11:53:51.761 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.761 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.761 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.761 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.762 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.762 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.762 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.762 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.762 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.762 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.763 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.763 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.763 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.763 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.764 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.764 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.764 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.764 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.764 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.765 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.765 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.765 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.765 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.765 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.766 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.766 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.766 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.766 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.766 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.766 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.767 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.767 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.767 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.768 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.768 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.768 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.768 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.768 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.769 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.769 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.769 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.769 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.769 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.770 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.770 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.770 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.770 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.770 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.771 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.771 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.771 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.771 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.771 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.772 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.772 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.772 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.772 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.772 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.773 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.773 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.773 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.773 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.773 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.774 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.774 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.774 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.774 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.774 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.775 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.775 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.775 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.775 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.775 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.776 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.776 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.776 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.776 INFO:teuthology.orchestra.run.smithi161.stdout: "organization": { 2023-07-14T11:53:51.777 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "organization", 2023-07-14T11:53:51.777 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.777 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.777 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.777 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.777 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.779 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.779 INFO:teuthology.orchestra.run.smithi161.stdout: "proxy": { 2023-07-14T11:53:51.779 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "proxy", 2023-07-14T11:53:51.779 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.780 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.780 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.780 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.780 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.780 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.781 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.781 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.781 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.781 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.781 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.782 INFO:teuthology.orchestra.run.smithi161.stdout: "url": { 2023-07-14T11:53:51.782 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "url", 2023-07-14T11:53:51.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.782 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.782 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.783 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T11:53:51.783 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.783 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.783 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.783 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.784 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.784 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.784 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.785 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "test_orchestrator", 2023-07-14T11:53:51.785 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.785 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.785 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.786 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.786 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.786 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.786 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.787 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.787 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.787 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.787 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.787 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.788 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.788 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.788 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.788 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.788 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.789 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.789 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.789 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.789 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.789 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.790 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.790 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.790 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.791 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.791 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.791 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.791 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.791 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.792 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.792 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.792 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.792 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.793 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.793 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.793 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.793 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.793 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.794 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.794 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.794 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.794 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.794 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.794 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.795 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.795 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.795 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.795 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.795 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.796 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.796 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.796 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.796 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.797 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.797 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.797 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.797 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.798 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.798 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.798 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.798 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.798 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.799 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.799 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.799 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.799 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.799 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.800 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "volumes", 2023-07-14T11:53:51.800 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.800 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.801 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.801 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.801 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "max_concurrent_clones": { 2023-07-14T11:53:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_concurrent_clones", 2023-07-14T11:53:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "4", 2023-07-14T11:53:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "snapshot_clone_delay": { 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snapshot_clone_delay", 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix", 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "discovery_interval": { 2023-07-14T11:53:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "discovery_interval", 2023-07-14T11:53:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "100", 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "identifier": { 2023-07-14T11:53:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "identifier", 2023-07-14T11:53:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_host": { 2023-07-14T11:53:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_host", 2023-07-14T11:53:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_port": { 2023-07-14T11:53:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_port", 2023-07-14T11:53:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10051", 2023-07-14T11:53:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.848 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.848 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.848 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.848 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.848 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.849 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_sender": { 2023-07-14T11:53:51.849 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_sender", 2023-07-14T11:53:51.849 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.849 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.849 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.850 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T11:53:51.850 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.850 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.850 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.850 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.851 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.851 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.851 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.851 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.851 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.852 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.852 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:51.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.852 INFO:teuthology.orchestra.run.smithi161.stdout: "gid": 25582, 2023-07-14T11:53:51.853 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "z", 2023-07-14T11:53:51.853 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr_features": 4540138322906710015, 2023-07-14T11:53:51.853 INFO:teuthology.orchestra.run.smithi161.stdout: "available_modules": [ 2023-07-14T11:53:51.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.853 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alerts", 2023-07-14T11:53:51.853 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.854 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.854 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.854 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:51.854 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:51.854 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.855 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.855 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.855 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:51.855 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.855 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.856 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.856 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T11:53:51.856 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.856 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.856 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.857 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.857 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.857 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.857 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.857 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.858 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.858 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.858 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.858 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.858 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.859 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.859 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.859 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.859 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.859 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.860 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.860 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.860 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.860 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.860 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.861 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.861 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.861 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.861 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.861 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.862 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.862 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.862 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.862 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.862 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.863 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.863 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.863 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.863 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.864 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.864 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.864 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.864 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.864 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.865 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.865 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.865 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.865 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.865 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.866 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.866 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.866 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.866 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.866 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.867 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.867 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.867 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.867 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.868 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.868 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.868 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.868 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.869 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.869 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.869 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.869 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.869 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.870 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.870 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.870 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.870 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.870 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_destination": { 2023-07-14T11:53:51.871 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_destination", 2023-07-14T11:53:51.871 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.871 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.871 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.871 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.872 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.872 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.872 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.872 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Email address to send alerts to", 2023-07-14T11:53:51.872 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.873 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.873 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.873 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_from_name": { 2023-07-14T11:53:51.873 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_from_name", 2023-07-14T11:53:51.874 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.874 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.874 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.874 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "Ceph", 2023-07-14T11:53:51.874 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.875 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.875 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.875 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Email From: name", 2023-07-14T11:53:51.875 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.875 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.876 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.876 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_host": { 2023-07-14T11:53:51.876 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_host", 2023-07-14T11:53:51.876 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.877 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.877 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.877 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.877 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.877 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.877 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.878 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP server", 2023-07-14T11:53:51.878 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.878 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.878 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.879 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_password": { 2023-07-14T11:53:51.879 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_password", 2023-07-14T11:53:51.879 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.879 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.879 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.880 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.880 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.880 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.880 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.880 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Password to authenticate with", 2023-07-14T11:53:51.881 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.881 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.881 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.881 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_port": { 2023-07-14T11:53:51.882 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_port", 2023-07-14T11:53:51.882 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.882 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.882 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.882 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "465", 2023-07-14T11:53:51.882 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.883 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.883 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.883 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP port", 2023-07-14T11:53:51.883 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.883 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.884 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.884 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_sender": { 2023-07-14T11:53:51.884 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_sender", 2023-07-14T11:53:51.884 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.885 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.885 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.885 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.885 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.885 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.886 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.886 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP envelope sender", 2023-07-14T11:53:51.886 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.886 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.886 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.887 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_ssl": { 2023-07-14T11:53:51.887 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_ssl", 2023-07-14T11:53:51.887 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.887 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.887 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.888 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.888 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.888 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.888 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.888 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T11:53:51.889 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.889 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.889 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.889 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_user": { 2023-07-14T11:53:51.890 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_user", 2023-07-14T11:53:51.890 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.890 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.890 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.890 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.891 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.891 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.891 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.891 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "User to authenticate as", 2023-07-14T11:53:51.891 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.892 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.892 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.893 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "balancer", 2023-07-14T11:53:51.893 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.893 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.893 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.893 INFO:teuthology.orchestra.run.smithi161.stdout: "active": { 2023-07-14T11:53:51.894 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "active", 2023-07-14T11:53:51.894 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.894 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.894 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.894 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.895 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.895 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.895 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.895 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T11:53:51.895 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.896 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.896 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.896 INFO:teuthology.orchestra.run.smithi161.stdout: "begin_time": { 2023-07-14T11:53:51.896 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "begin_time", 2023-07-14T11:53:51.897 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.897 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.897 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.897 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0000", 2023-07-14T11:53:51.897 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.897 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.898 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.898 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T11:53:51.898 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T11:53:51.898 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.898 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.899 INFO:teuthology.orchestra.run.smithi161.stdout: "begin_weekday": { 2023-07-14T11:53:51.899 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "begin_weekday", 2023-07-14T11:53:51.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:51.900 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.900 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.900 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:51.900 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0", 2023-07-14T11:53:51.900 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "6", 2023-07-14T11:53:51.901 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.901 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T11:53:51.901 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T11:53:51.901 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.901 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.902 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_max_iterations": { 2023-07-14T11:53:51.902 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_max_iterations", 2023-07-14T11:53:51.902 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:51.902 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.903 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.903 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "25", 2023-07-14T11:53:51.903 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:51.903 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "250", 2023-07-14T11:53:51.903 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.904 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T11:53:51.904 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.904 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.904 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.905 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_metrics": { 2023-07-14T11:53:51.905 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_metrics", 2023-07-14T11:53:51.905 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.905 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.905 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.906 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T11:53:51.906 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.906 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.906 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.906 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T11:53:51.907 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T11:53:51.907 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.907 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.907 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_step": { 2023-07-14T11:53:51.907 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_step", 2023-07-14T11:53:51.908 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.908 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.908 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.908 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0.5", 2023-07-14T11:53:51.908 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0.001", 2023-07-14T11:53:51.909 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "0.999", 2023-07-14T11:53:51.909 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.909 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "aggressiveness of optimization", 2023-07-14T11:53:51.909 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T11:53:51.909 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.910 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.910 INFO:teuthology.orchestra.run.smithi161.stdout: "end_time": { 2023-07-14T11:53:51.910 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "end_time", 2023-07-14T11:53:51.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.911 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.911 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.911 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "2359", 2023-07-14T11:53:51.911 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.911 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.912 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.912 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T11:53:51.912 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T11:53:51.912 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.912 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.913 INFO:teuthology.orchestra.run.smithi161.stdout: "end_weekday": { 2023-07-14T11:53:51.913 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "end_weekday", 2023-07-14T11:53:51.913 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:51.913 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.913 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.914 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:51.914 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0", 2023-07-14T11:53:51.914 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "6", 2023-07-14T11:53:51.914 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.914 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T11:53:51.915 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T11:53:51.915 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.915 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.915 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:51.916 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:51.916 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.916 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.916 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.916 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.917 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.917 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.917 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.917 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.917 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.917 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.918 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.918 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.918 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.918 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.918 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.919 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.919 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.919 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.919 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:51.920 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:51.920 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.920 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.920 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.920 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.921 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.921 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.921 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.921 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.921 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.921 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.922 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.922 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:51.922 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:51.922 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.923 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.923 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.923 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:51.923 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.923 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.924 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.924 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:51.924 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:51.924 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:51.924 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:51.925 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:51.925 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:51.925 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.925 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.925 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.925 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.926 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.926 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:51.926 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:51.926 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.927 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.927 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.927 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.927 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.927 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.928 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.928 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:51.928 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.928 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.928 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.929 INFO:teuthology.orchestra.run.smithi161.stdout: "min_score": { 2023-07-14T11:53:51.929 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "min_score", 2023-07-14T11:53:51.929 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.929 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T11:53:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": { 2023-07-14T11:53:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mode", 2023-07-14T11:53:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "upmap", 2023-07-14T11:53:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-compat", 2023-07-14T11:53:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "none", 2023-07-14T11:53:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap" 2023-07-14T11:53:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Balancer mode", 2023-07-14T11:53:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_ids": { 2023-07-14T11:53:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pool_ids", 2023-07-14T11:53:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T11:53:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T11:53:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap_max_deviation": { 2023-07-14T11:53:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "upmap_max_deviation", 2023-07-14T11:53:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T11:53:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T11:53:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap_max_optimizations": { 2023-07-14T11:53:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "upmap_max_optimizations", 2023-07-14T11:53:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T11:53:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cephadm", 2023-07-14T11:53:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_down_multiplier": { 2023-07-14T11:53:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_down_multiplier", 2023-07-14T11:53:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3.0", 2023-07-14T11:53:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T11:53:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_refresh_rate": { 2023-07-14T11:53:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_refresh_rate", 2023-07-14T11:53:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "20", 2023-07-14T11:53:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T11:53:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_starting_port": { 2023-07-14T11:53:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_starting_port", 2023-07-14T11:53:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "4721", 2023-07-14T11:53:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T11:53:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "alertmanager_web_password": { 2023-07-14T11:53:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alertmanager_web_password", 2023-07-14T11:53:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Alertmanager web password", 2023-07-14T11:53:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "alertmanager_web_user": { 2023-07-14T11:53:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alertmanager_web_user", 2023-07-14T11:53:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Alertmanager web user", 2023-07-14T11:53:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_ptrace": { 2023-07-14T11:53:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_ptrace", 2023-07-14T11:53:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T11:53:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T11:53:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "autotune_interval": { 2023-07-14T11:53:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "autotune_interval", 2023-07-14T11:53:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T11:53:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "autotune_memory_target_ratio": { 2023-07-14T11:53:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T11:53:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0.7", 2023-07-14T11:53:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T11:53:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "cgroups_split": { 2023-07-14T11:53:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cgroups_split", 2023-07-14T11:53:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T11:53:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "config_checks_enabled": { 2023-07-14T11:53:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_checks_enabled", 2023-07-14T11:53:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T11:53:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "config_dashboard": { 2023-07-14T11:53:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_dashboard", 2023-07-14T11:53:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T11:53:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_alertmanager": { 2023-07-14T11:53:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_alertmanager", 2023-07-14T11:53:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T11:53:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_base": { 2023-07-14T11:53:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_base", 2023-07-14T11:53:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T11:53:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Container image name, without the tag", 2023-07-14T11:53:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_elasticsearch": { 2023-07-14T11:53:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_elasticsearch", 2023-07-14T11:53:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T11:53:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "elasticsearch container image", 2023-07-14T11:53:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_grafana": { 2023-07-14T11:53:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_grafana", 2023-07-14T11:53:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T11:53:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_haproxy": { 2023-07-14T11:53:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_haproxy", 2023-07-14T11:53:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T11:53:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "HAproxy container image", 2023-07-14T11:53:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_agent": { 2023-07-14T11:53:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_agent", 2023-07-14T11:53:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T11:53:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger agent container image", 2023-07-14T11:53:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_collector": { 2023-07-14T11:53:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_collector", 2023-07-14T11:53:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T11:53:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger collector container image", 2023-07-14T11:53:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_query": { 2023-07-14T11:53:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_query", 2023-07-14T11:53:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T11:53:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger query container image", 2023-07-14T11:53:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_keepalived": { 2023-07-14T11:53:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_keepalived", 2023-07-14T11:53:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T11:53:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Keepalived container image", 2023-07-14T11:53:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_loki": { 2023-07-14T11:53:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_loki", 2023-07-14T11:53:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T11:53:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Loki container image", 2023-07-14T11:53:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_node_exporter": { 2023-07-14T11:53:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_node_exporter", 2023-07-14T11:53:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T11:53:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_prometheus": { 2023-07-14T11:53:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_prometheus", 2023-07-14T11:53:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T11:53:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_promtail": { 2023-07-14T11:53:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_promtail", 2023-07-14T11:53:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Promtail container image", 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_snmp_gateway": { 2023-07-14T11:53:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_snmp_gateway", 2023-07-14T11:53:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T11:53:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SNMP Gateway container image", 2023-07-14T11:53:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "container_init": { 2023-07-14T11:53:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_init", 2023-07-14T11:53:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T11:53:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: "daemon_cache_timeout": { 2023-07-14T11:53:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "daemon_cache_timeout", 2023-07-14T11:53:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.019 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.019 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.019 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:52.019 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.019 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.020 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.020 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T11:53:52.020 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.020 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.020 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.021 INFO:teuthology.orchestra.run.smithi161.stdout: "default_cephadm_command_timeout": { 2023-07-14T11:53:52.021 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T11:53:52.021 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.021 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.022 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.022 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "900", 2023-07-14T11:53:52.022 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.022 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.022 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.022 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T11:53:52.023 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.023 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.023 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.023 INFO:teuthology.orchestra.run.smithi161.stdout: "default_registry": { 2023-07-14T11:53:52.024 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "default_registry", 2023-07-14T11:53:52.024 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.024 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.024 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.024 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io", 2023-07-14T11:53:52.025 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.025 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.025 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.025 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T11:53:52.025 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.026 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.026 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.026 INFO:teuthology.orchestra.run.smithi161.stdout: "device_cache_timeout": { 2023-07-14T11:53:52.026 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_cache_timeout", 2023-07-14T11:53:52.027 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.027 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.027 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.027 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1800", 2023-07-14T11:53:52.027 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.028 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.028 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.028 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache device inventory", 2023-07-14T11:53:52.028 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.028 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.029 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.029 INFO:teuthology.orchestra.run.smithi161.stdout: "device_enhanced_scan": { 2023-07-14T11:53:52.029 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_enhanced_scan", 2023-07-14T11:53:52.029 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.030 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.030 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.030 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.030 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.030 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.031 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.031 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T11:53:52.031 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.031 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.031 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.032 INFO:teuthology.orchestra.run.smithi161.stdout: "facts_cache_timeout": { 2023-07-14T11:53:52.032 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "facts_cache_timeout", 2023-07-14T11:53:52.032 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.032 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.033 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.033 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:52.033 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.033 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.033 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.034 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache host facts data", 2023-07-14T11:53:52.034 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.034 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.034 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.035 INFO:teuthology.orchestra.run.smithi161.stdout: "host_check_interval": { 2023-07-14T11:53:52.035 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "host_check_interval", 2023-07-14T11:53:52.035 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.035 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.035 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.035 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:52.036 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.036 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.036 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.036 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to perform a host check", 2023-07-14T11:53:52.036 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.037 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.037 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.037 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.037 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.038 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.038 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.038 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.038 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.039 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.039 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.039 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.039 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.039 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.040 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.040 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.040 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.040 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.040 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.041 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.041 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.041 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.041 INFO:teuthology.orchestra.run.smithi161.stdout: "log_refresh_metadata": { 2023-07-14T11:53:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_refresh_metadata", 2023-07-14T11:53:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.043 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.043 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.043 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T11:53:52.043 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.043 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.044 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.044 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.044 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.044 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.045 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.045 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.045 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.045 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.045 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.046 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.046 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T11:53:52.046 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.046 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.046 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.047 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.047 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.047 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.047 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.048 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.048 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.048 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.048 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.048 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.049 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.049 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.049 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.049 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.049 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.050 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.050 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.050 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.050 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.050 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.051 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.051 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.051 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.051 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.051 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.052 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.052 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.052 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.052 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.053 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.053 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.053 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.053 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.053 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.054 INFO:teuthology.orchestra.run.smithi161.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T11:53:52.054 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T11:53:52.054 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.054 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.054 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.055 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.055 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.055 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.055 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.055 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T11:53:52.056 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.056 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.056 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.056 INFO:teuthology.orchestra.run.smithi161.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T11:53:52.057 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T11:53:52.057 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.057 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.057 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.057 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "*", 2023-07-14T11:53:52.058 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.058 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.058 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.058 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T11:53:52.058 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.059 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.059 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.059 INFO:teuthology.orchestra.run.smithi161.stdout: "max_count_per_host": { 2023-07-14T11:53:52.059 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_count_per_host", 2023-07-14T11:53:52.059 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.060 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.060 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.060 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:52.060 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.060 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "max number of daemons per service per host", 2023-07-14T11:53:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.061 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.062 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd_draining_count": { 2023-07-14T11:53:52.062 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_osd_draining_count", 2023-07-14T11:53:52.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.062 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.063 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.063 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:52.063 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.063 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.063 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.064 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T11:53:52.064 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.064 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.064 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.065 INFO:teuthology.orchestra.run.smithi161.stdout: "migration_current": { 2023-07-14T11:53:52.065 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "migration_current", 2023-07-14T11:53:52.065 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.065 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.065 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.066 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.066 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.066 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.066 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.066 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "internal - do not modify", 2023-07-14T11:53:52.066 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.067 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.067 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.067 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": { 2023-07-14T11:53:52.068 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mode", 2023-07-14T11:53:52.068 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.068 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.068 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.068 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "root", 2023-07-14T11:53:52.069 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.069 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.069 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.069 INFO:teuthology.orchestra.run.smithi161.stdout: "cephadm-package", 2023-07-14T11:53:52.069 INFO:teuthology.orchestra.run.smithi161.stdout: "root" 2023-07-14T11:53:52.070 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.070 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T11:53:52.070 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.070 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.070 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.071 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_alerts_path": { 2023-07-14T11:53:52.071 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_alerts_path", 2023-07-14T11:53:52.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.071 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.071 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.072 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T11:53:52.072 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.072 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.072 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.072 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T11:53:52.073 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.073 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.073 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.073 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_web_password": { 2023-07-14T11:53:52.074 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_web_password", 2023-07-14T11:53:52.074 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.074 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.074 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.074 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:52.075 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.075 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.075 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.075 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus web password", 2023-07-14T11:53:52.075 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.076 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.076 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.076 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_web_user": { 2023-07-14T11:53:52.076 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_web_user", 2023-07-14T11:53:52.077 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.077 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.077 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.077 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:52.077 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.077 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.078 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.078 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus web user", 2023-07-14T11:53:52.078 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.078 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.078 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.079 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_insecure": { 2023-07-14T11:53:52.079 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_insecure", 2023-07-14T11:53:52.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.079 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.080 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.080 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.080 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.080 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.080 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.081 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T11:53:52.081 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.081 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.081 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.082 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_password": { 2023-07-14T11:53:52.082 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_password", 2023-07-14T11:53:52.082 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.082 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.082 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.082 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.083 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.083 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.083 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.083 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T11:53:52.083 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.084 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.084 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_url": { 2023-07-14T11:53:52.084 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_url", 2023-07-14T11:53:52.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.085 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.085 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.085 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.085 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.086 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.086 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.086 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T11:53:52.086 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.086 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.087 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.087 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_username": { 2023-07-14T11:53:52.087 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_username", 2023-07-14T11:53:52.087 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.088 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.088 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.088 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.088 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.089 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.089 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T11:53:52.089 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.089 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.089 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.090 INFO:teuthology.orchestra.run.smithi161.stdout: "secure_monitoring_stack": { 2023-07-14T11:53:52.090 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "secure_monitoring_stack", 2023-07-14T11:53:52.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.090 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.090 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.091 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.091 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.091 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.091 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.091 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T11:53:52.092 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.092 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.092 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.092 INFO:teuthology.orchestra.run.smithi161.stdout: "service_discovery_port": { 2023-07-14T11:53:52.093 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "service_discovery_port", 2023-07-14T11:53:52.093 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.093 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.093 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.093 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8765", 2023-07-14T11:53:52.094 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.094 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.094 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.094 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "cephadm service discovery port", 2023-07-14T11:53:52.094 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.095 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.095 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.095 INFO:teuthology.orchestra.run.smithi161.stdout: "ssh_config_file": { 2023-07-14T11:53:52.095 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssh_config_file", 2023-07-14T11:53:52.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.096 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.096 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.096 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.096 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.097 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.097 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.097 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T11:53:52.097 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.097 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.098 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.098 INFO:teuthology.orchestra.run.smithi161.stdout: "use_agent": { 2023-07-14T11:53:52.098 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "use_agent", 2023-07-14T11:53:52.098 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.099 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.099 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.099 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.099 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.099 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.100 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.100 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T11:53:52.100 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.100 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.100 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.101 INFO:teuthology.orchestra.run.smithi161.stdout: "use_repo_digest": { 2023-07-14T11:53:52.101 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "use_repo_digest", 2023-07-14T11:53:52.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.101 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.102 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.102 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.102 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.102 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.102 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.103 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T11:53:52.103 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.103 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.103 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.103 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_failed_host_check": { 2023-07-14T11:53:52.104 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_failed_host_check", 2023-07-14T11:53:52.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.104 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.104 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.104 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.105 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.105 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.105 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.105 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T11:53:52.105 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.106 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.106 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.106 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_stray_daemons": { 2023-07-14T11:53:52.106 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_stray_daemons", 2023-07-14T11:53:52.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.107 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.107 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.107 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.107 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.107 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.108 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.108 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T11:53:52.108 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.108 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.108 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.109 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_stray_hosts": { 2023-07-14T11:53:52.109 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_stray_hosts", 2023-07-14T11:53:52.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.109 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.110 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.110 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.110 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.110 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.110 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.111 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T11:53:52.111 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.111 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.111 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.112 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crash", 2023-07-14T11:53:52.112 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.112 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.113 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.113 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.113 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.113 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.113 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.114 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.114 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.114 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.114 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.114 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.115 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.115 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.115 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.115 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.115 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.116 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.116 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.116 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.116 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.116 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.117 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.117 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.117 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.117 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.118 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.118 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.118 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.118 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.118 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.118 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.119 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.119 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.119 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.119 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.120 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.120 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.120 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.120 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.121 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.121 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.121 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.121 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.121 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.122 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.122 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.122 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.122 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.122 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.123 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.123 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.123 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.123 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.123 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.124 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.124 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.124 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.124 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.124 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.125 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.125 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.125 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.125 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.125 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.126 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.126 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.126 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.126 INFO:teuthology.orchestra.run.smithi161.stdout: "retain_interval": { 2023-07-14T11:53:52.127 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "retain_interval", 2023-07-14T11:53:52.127 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.127 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.127 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.127 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "31536000", 2023-07-14T11:53:52.127 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.128 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.128 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.128 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T11:53:52.128 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.128 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.129 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.129 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_recent_interval": { 2023-07-14T11:53:52.129 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_recent_interval", 2023-07-14T11:53:52.129 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.130 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.130 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.130 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1209600", 2023-07-14T11:53:52.130 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.130 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.131 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.131 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T11:53:52.131 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.131 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.131 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.132 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.132 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.132 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "dashboard", 2023-07-14T11:53:52.132 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.133 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.133 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.133 INFO:teuthology.orchestra.run.smithi161.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T11:53:52.133 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T11:53:52.133 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.134 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.134 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.134 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:52.134 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.134 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.135 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.135 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.135 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.135 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.135 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.136 INFO:teuthology.orchestra.run.smithi161.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T11:53:52.136 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T11:53:52.136 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.136 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.137 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.137 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.137 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.137 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.137 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.138 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.138 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.138 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.138 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.138 INFO:teuthology.orchestra.run.smithi161.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T11:53:52.139 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T11:53:52.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.139 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.139 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.140 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.140 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.140 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.140 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.140 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.141 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.141 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.141 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.141 INFO:teuthology.orchestra.run.smithi161.stdout: "AUDIT_API_ENABLED": { 2023-07-14T11:53:52.141 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T11:53:52.142 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.142 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.142 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.142 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.142 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.143 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.143 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.143 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.143 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.143 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.144 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.144 INFO:teuthology.orchestra.run.smithi161.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T11:53:52.144 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T11:53:52.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.145 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.145 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.145 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.145 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.145 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.145 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.148 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.148 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.149 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.149 INFO:teuthology.orchestra.run.smithi161.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T11:53:52.149 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T11:53:52.149 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.150 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.150 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.150 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.150 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.150 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.151 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.151 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.151 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.151 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.151 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.152 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T11:53:52.152 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T11:53:52.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.152 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.153 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.153 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.153 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.153 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.154 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.154 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.154 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.154 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.155 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T11:53:52.155 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T11:53:52.155 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.155 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.155 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.155 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.156 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.156 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.156 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.156 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.156 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.157 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.157 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T11:53:52.157 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T11:53:52.158 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.158 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.158 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.158 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.158 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.159 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.159 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.159 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.159 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.159 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.160 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.160 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T11:53:52.160 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T11:53:52.160 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.160 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.161 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.161 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.161 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.161 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.161 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.162 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.162 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.162 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.162 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.163 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T11:53:52.163 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T11:53:52.163 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.163 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.163 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.164 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.164 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.164 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.164 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.164 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.165 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.165 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.165 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.165 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T11:53:52.165 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T11:53:52.166 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.166 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.166 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.166 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.166 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.167 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.167 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.167 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.167 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.167 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.168 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.168 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T11:53:52.168 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T11:53:52.168 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.169 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.169 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.169 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.169 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.170 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.170 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.170 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.170 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.170 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.171 INFO:teuthology.orchestra.run.smithi161.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T11:53:52.171 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T11:53:52.171 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.171 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.171 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.172 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.172 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.172 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.172 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.172 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.173 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.173 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.173 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T11:53:52.174 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T11:53:52.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.174 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.174 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.174 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:52.175 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.175 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.175 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.175 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.175 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.176 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.176 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.176 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T11:53:52.176 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T11:53:52.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.177 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.177 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.177 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.177 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.178 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.178 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.178 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.178 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.178 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.179 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.179 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_URL": { 2023-07-14T11:53:52.179 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_URL", 2023-07-14T11:53:52.179 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.180 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.180 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.180 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.180 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.180 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.181 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.181 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.181 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.181 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.181 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.182 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T11:53:52.182 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T11:53:52.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.182 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.182 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.183 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:52.183 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.183 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.183 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.183 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.184 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.184 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.184 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T11:53:52.185 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T11:53:52.185 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.185 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.185 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.185 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.186 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.186 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.186 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.186 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.186 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.186 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.187 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.187 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T11:53:52.187 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T11:53:52.187 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.188 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.188 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.188 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.188 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.189 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.189 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.189 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.189 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.189 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.190 INFO:teuthology.orchestra.run.smithi161.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T11:53:52.190 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T11:53:52.190 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.190 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.191 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.191 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.191 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.191 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.191 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.192 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.192 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.192 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.193 INFO:teuthology.orchestra.run.smithi161.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T11:53:52.193 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T11:53:52.193 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.193 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.193 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.194 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.194 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.194 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.194 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.194 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.195 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.195 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.195 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.195 INFO:teuthology.orchestra.run.smithi161.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T11:53:52.195 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T11:53:52.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.196 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.196 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.196 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.196 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.197 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.197 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.197 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.197 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.197 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.198 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.198 INFO:teuthology.orchestra.run.smithi161.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T11:53:52.198 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T11:53:52.198 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.199 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.199 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.199 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.199 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.199 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.200 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.200 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.200 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.200 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.201 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T11:53:52.201 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T11:53:52.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.201 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.201 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.202 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.202 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.202 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.202 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.202 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.203 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.203 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.203 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.203 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T11:53:52.204 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T11:53:52.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.204 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.204 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.204 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.205 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.205 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.205 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.205 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.205 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.206 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.206 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.206 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T11:53:52.206 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T11:53:52.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.207 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.207 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.207 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.207 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.207 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.208 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.208 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.208 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.208 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.208 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.209 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T11:53:52.209 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T11:53:52.209 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.209 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.210 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.210 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.210 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.210 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.210 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.211 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.211 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.211 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.211 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.212 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T11:53:52.212 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T11:53:52.212 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.212 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.212 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.213 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.213 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.213 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.213 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.213 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.214 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.214 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.214 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.214 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T11:53:52.214 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T11:53:52.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.215 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.215 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.215 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.215 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.216 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.216 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.216 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.216 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.216 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.217 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.217 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T11:53:52.217 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T11:53:52.217 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.218 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.218 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.218 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.218 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.218 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.218 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.220 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T11:53:52.220 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T11:53:52.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.220 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.220 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.222 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.222 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.222 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.222 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.225 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.225 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.225 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T11:53:52.225 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8", 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi161.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "45", 2023-07-14T11:53:52.232 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.232 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.232 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.232 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.232 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.233 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.233 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.233 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T11:53:52.233 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T11:53:52.234 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.234 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.234 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.234 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.234 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.235 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.235 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.235 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.235 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.235 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.236 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.236 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T11:53:52.236 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T11:53:52.236 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.237 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.237 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.237 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:52.237 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.237 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.238 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.238 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.238 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.238 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.238 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.239 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T11:53:52.239 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T11:53:52.239 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.240 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.240 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.240 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.240 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.241 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.241 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.241 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.241 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.241 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.242 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.242 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T11:53:52.242 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T11:53:52.242 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.242 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.243 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.243 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.243 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.243 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.243 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.244 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.244 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.244 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.244 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.245 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T11:53:52.245 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T11:53:52.245 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.245 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.245 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.246 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:52.246 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.246 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.246 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.246 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.247 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.247 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.247 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.247 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T11:53:52.248 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T11:53:52.248 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.248 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.248 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.248 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:52.249 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.249 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.249 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.249 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.249 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.250 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.250 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.250 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T11:53:52.250 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T11:53:52.250 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.251 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.251 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.251 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:52.251 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.251 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.252 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.252 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.252 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.252 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.252 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.253 INFO:teuthology.orchestra.run.smithi161.stdout: "cross_origin_url": { 2023-07-14T11:53:52.253 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cross_origin_url", 2023-07-14T11:53:52.253 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.254 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.254 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.254 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.254 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.254 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.255 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.255 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.255 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.255 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.255 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.256 INFO:teuthology.orchestra.run.smithi161.stdout: "crt_file": { 2023-07-14T11:53:52.256 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crt_file", 2023-07-14T11:53:52.256 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.256 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.256 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.257 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.257 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.257 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.257 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.257 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.258 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.258 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.258 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.258 INFO:teuthology.orchestra.run.smithi161.stdout: "debug": { 2023-07-14T11:53:52.259 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "debug", 2023-07-14T11:53:52.259 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.259 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.259 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.259 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.260 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.260 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.260 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.260 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable/disable debug options", 2023-07-14T11:53:52.260 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.261 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.261 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.261 INFO:teuthology.orchestra.run.smithi161.stdout: "jwt_token_ttl": { 2023-07-14T11:53:52.261 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "jwt_token_ttl", 2023-07-14T11:53:52.262 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.262 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.262 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.262 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "28800", 2023-07-14T11:53:52.262 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.263 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.263 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.263 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.263 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.263 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.264 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.264 INFO:teuthology.orchestra.run.smithi161.stdout: "key_file": { 2023-07-14T11:53:52.264 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "key_file", 2023-07-14T11:53:52.264 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.264 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.265 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.265 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.265 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.265 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.265 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.266 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.266 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.266 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.266 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.267 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.267 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.267 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.267 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.267 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.268 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.268 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.268 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.268 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.269 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.269 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.269 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.269 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.269 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.269 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.270 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.270 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.270 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.270 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.270 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.274 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.274 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.274 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.274 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.274 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.274 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.275 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.275 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.275 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.275 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.275 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.276 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.276 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.276 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.276 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.276 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.277 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.277 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.277 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.277 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.278 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.278 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.278 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.278 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.278 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.278 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.279 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.279 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.279 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.279 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.279 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: "motd": { 2023-07-14T11:53:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "motd", 2023-07-14T11:53:52.281 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.281 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.281 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.281 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.281 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.282 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.282 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.282 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "The message of the day", 2023-07-14T11:53:52.282 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.282 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.283 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.283 INFO:teuthology.orchestra.run.smithi161.stdout: "redirect_resolve_ip_addr": { 2023-07-14T11:53:52.283 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T11:53:52.283 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.284 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.284 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.284 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.284 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.284 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.285 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.285 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.285 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.285 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.285 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.286 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:52.286 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:52.286 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.287 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.287 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.287 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "::", 2023-07-14T11:53:52.287 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.287 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.288 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.288 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.288 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.288 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.288 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.289 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:52.289 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:52.289 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.289 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.290 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.290 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8080", 2023-07-14T11:53:52.290 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.290 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.290 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.291 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.291 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.291 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.291 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.292 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl": { 2023-07-14T11:53:52.292 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl", 2023-07-14T11:53:52.292 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.292 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.292 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.293 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.293 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.293 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.293 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.293 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.294 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.294 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.294 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.294 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl_server_port": { 2023-07-14T11:53:52.294 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl_server_port", 2023-07-14T11:53:52.295 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.295 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.295 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.295 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8443", 2023-07-14T11:53:52.295 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.296 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.296 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.296 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.296 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.296 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.297 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.297 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_behaviour": { 2023-07-14T11:53:52.297 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_behaviour", 2023-07-14T11:53:52.297 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.298 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.298 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.298 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "redirect", 2023-07-14T11:53:52.298 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.298 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.299 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.299 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.299 INFO:teuthology.orchestra.run.smithi161.stdout: "redirect" 2023-07-14T11:53:52.299 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.299 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.300 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.300 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.300 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.300 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_error_status_code": { 2023-07-14T11:53:52.301 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_error_status_code", 2023-07-14T11:53:52.301 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.301 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.301 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.301 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "500", 2023-07-14T11:53:52.301 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "400", 2023-07-14T11:53:52.302 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "599", 2023-07-14T11:53:52.302 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.302 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.302 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.302 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.303 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.303 INFO:teuthology.orchestra.run.smithi161.stdout: "url_prefix": { 2023-07-14T11:53:52.303 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "url_prefix", 2023-07-14T11:53:52.303 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.304 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.304 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.304 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.304 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.304 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.305 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.305 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.305 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.305 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.305 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.305 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.306 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.306 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "devicehealth", 2023-07-14T11:53:52.306 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.307 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.307 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.307 INFO:teuthology.orchestra.run.smithi161.stdout: "enable_monitoring": { 2023-07-14T11:53:52.307 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enable_monitoring", 2023-07-14T11:53:52.307 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.308 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.308 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.308 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.308 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.308 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.309 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.309 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "monitor device health metrics", 2023-07-14T11:53:52.309 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.309 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.309 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.310 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.310 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.310 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.310 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.310 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.311 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.311 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.311 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.311 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.311 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.312 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.312 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.312 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.312 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.312 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.313 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.313 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.313 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.313 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.313 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.314 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.314 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.314 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.314 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.315 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.315 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.315 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.315 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.315 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.315 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.316 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.316 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.316 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.316 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.317 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.317 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.317 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.317 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.317 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.318 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.318 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.318 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.318 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.318 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.319 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.319 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.319 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.319 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.319 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.319 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.320 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.320 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.320 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.320 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.321 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.321 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.321 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.321 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.322 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.322 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.322 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.322 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.322 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.322 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.323 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.323 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.323 INFO:teuthology.orchestra.run.smithi161.stdout: "mark_out_threshold": { 2023-07-14T11:53:52.323 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mark_out_threshold", 2023-07-14T11:53:52.324 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.324 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.324 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.324 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "2419200", 2023-07-14T11:53:52.324 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.325 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.325 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.325 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T11:53:52.325 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.325 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.326 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.326 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": { 2023-07-14T11:53:52.326 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pool_name", 2023-07-14T11:53:52.326 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.327 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.327 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.327 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "device_health_metrics", 2023-07-14T11:53:52.327 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.327 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.328 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.328 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T11:53:52.328 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.328 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.328 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.329 INFO:teuthology.orchestra.run.smithi161.stdout: "retention_period": { 2023-07-14T11:53:52.329 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "retention_period", 2023-07-14T11:53:52.329 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.329 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.329 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.330 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15552000", 2023-07-14T11:53:52.330 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.330 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.330 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.330 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long to retain device health metrics", 2023-07-14T11:53:52.331 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.331 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.331 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.331 INFO:teuthology.orchestra.run.smithi161.stdout: "scrape_frequency": { 2023-07-14T11:53:52.332 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "scrape_frequency", 2023-07-14T11:53:52.332 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.332 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.332 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.332 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "86400", 2023-07-14T11:53:52.333 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.333 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.333 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.333 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T11:53:52.333 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.333 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.334 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.334 INFO:teuthology.orchestra.run.smithi161.stdout: "self_heal": { 2023-07-14T11:53:52.334 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "self_heal", 2023-07-14T11:53:52.334 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.335 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.335 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.335 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.335 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.335 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.336 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.336 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T11:53:52.336 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.336 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.336 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.337 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:52.337 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:52.337 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.337 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.337 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.338 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:52.338 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.338 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.338 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.338 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T11:53:52.339 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.339 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.339 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.339 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.339 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_threshold": { 2023-07-14T11:53:52.340 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_threshold", 2023-07-14T11:53:52.340 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.340 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.340 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.340 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "7257600", 2023-07-14T11:53:52.341 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.341 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.341 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.341 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T11:53:52.341 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.342 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.342 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.342 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.342 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.343 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "diskprediction_local", 2023-07-14T11:53:52.343 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.343 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.343 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.343 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.344 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.344 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.344 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.344 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.344 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.345 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.345 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.345 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.345 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.345 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.346 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.346 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.346 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.346 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.346 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.347 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.347 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.347 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.347 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.347 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.348 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.348 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.348 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.348 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.348 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.349 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.349 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.349 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.349 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.349 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.350 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.350 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.350 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.350 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.351 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.351 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.351 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.351 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.351 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.351 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.352 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.352 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.352 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.352 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.352 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.353 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.353 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.353 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.353 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.353 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.354 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.354 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.354 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.354 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.355 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.355 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.355 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.355 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.355 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.356 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.356 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.356 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.356 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.356 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.357 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.357 INFO:teuthology.orchestra.run.smithi161.stdout: "predict_interval": { 2023-07-14T11:53:52.357 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "predict_interval", 2023-07-14T11:53:52.357 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.358 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.358 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.358 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "86400", 2023-07-14T11:53:52.358 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.358 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.359 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.359 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.359 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.359 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.359 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.360 INFO:teuthology.orchestra.run.smithi161.stdout: "predictor_model": { 2023-07-14T11:53:52.360 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "predictor_model", 2023-07-14T11:53:52.360 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.360 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.360 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.361 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "prophetstor", 2023-07-14T11:53:52.361 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.361 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.361 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.361 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.362 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.362 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.362 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.362 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:52.363 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:52.363 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.363 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.363 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.363 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:52.364 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.364 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.364 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.364 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.364 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.364 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.365 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.365 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.365 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.366 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "influx", 2023-07-14T11:53:52.366 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": false, 2023-07-14T11:53:52.366 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "influxdb python module not found", 2023-07-14T11:53:52.366 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.366 INFO:teuthology.orchestra.run.smithi161.stdout: "batch_size": { 2023-07-14T11:53:52.367 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "batch_size", 2023-07-14T11:53:52.367 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.367 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.367 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.367 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5000", 2023-07-14T11:53:52.368 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.368 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.368 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.368 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T11:53:52.368 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.369 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.369 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.369 INFO:teuthology.orchestra.run.smithi161.stdout: "database": { 2023-07-14T11:53:52.369 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "database", 2023-07-14T11:53:52.369 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.370 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.370 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.370 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "ceph", 2023-07-14T11:53:52.370 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.370 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.371 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.371 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T11:53:52.371 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.371 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.371 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.372 INFO:teuthology.orchestra.run.smithi161.stdout: "hostname": { 2023-07-14T11:53:52.372 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "hostname", 2023-07-14T11:53:52.372 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.372 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.373 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.373 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.373 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.373 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.373 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.373 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB server hostname", 2023-07-14T11:53:52.374 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.374 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.374 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.374 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:52.375 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:52.375 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.375 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.375 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.375 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "30", 2023-07-14T11:53:52.376 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "5", 2023-07-14T11:53:52.376 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.376 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.376 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T11:53:52.376 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.377 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.377 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.377 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.377 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.378 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.378 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.378 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.378 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.378 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.379 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.379 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.379 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.379 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.379 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.380 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.380 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.380 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.380 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.381 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.381 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.381 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.381 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.381 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.382 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.382 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.382 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.382 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.382 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.383 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.383 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.383 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.383 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.383 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.384 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.384 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.384 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.384 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.385 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.385 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.385 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.385 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.385 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.385 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.386 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.386 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.386 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.386 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.386 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.387 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.387 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.387 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.387 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.387 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.388 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.388 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.388 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.388 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.389 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.389 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.389 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.389 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.390 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.390 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.390 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.390 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.390 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.391 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.391 INFO:teuthology.orchestra.run.smithi161.stdout: "password": { 2023-07-14T11:53:52.391 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "password", 2023-07-14T11:53:52.391 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.391 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.392 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.392 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.392 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.392 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.392 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.393 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "password of InfluxDB server user", 2023-07-14T11:53:52.393 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.393 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.393 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.394 INFO:teuthology.orchestra.run.smithi161.stdout: "port": { 2023-07-14T11:53:52.394 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "port", 2023-07-14T11:53:52.394 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.394 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.394 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.395 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8086", 2023-07-14T11:53:52.395 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.395 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.395 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.395 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB server port", 2023-07-14T11:53:52.396 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.396 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.396 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.396 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl": { 2023-07-14T11:53:52.397 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl", 2023-07-14T11:53:52.397 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.397 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.397 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.397 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "false", 2023-07-14T11:53:52.398 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.398 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.398 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.398 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T11:53:52.398 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: "threads": { 2023-07-14T11:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "threads", 2023-07-14T11:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "32", 2023-07-14T11:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T11:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "username": { 2023-07-14T11:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "username", 2023-07-14T11:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "username of InfluxDB server user", 2023-07-14T11:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "verify_ssl": { 2023-07-14T11:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "verify_ssl", 2023-07-14T11:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "true", 2023-07-14T11:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T11:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "insights", 2023-07-14T11:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "iostat", 2023-07-14T11:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "k8sevents", 2023-07-14T11:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_event_retention_days": { 2023-07-14T11:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ceph_event_retention_days", 2023-07-14T11:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "7", 2023-07-14T11:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T11:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "config_check_secs": { 2023-07-14T11:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_check_secs", 2023-07-14T11:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "10", 2023-07-14T11:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T11:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "localpool", 2023-07-14T11:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "failure_domain": { 2023-07-14T11:53:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "failure_domain", 2023-07-14T11:53:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "host", 2023-07-14T11:53:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "failure domain for any created local pool", 2023-07-14T11:53:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T11:53:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": { 2023-07-14T11:53:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "min_size", 2023-07-14T11:53:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default min_size for any created local pool", 2023-07-14T11:53:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T11:53:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "num_rep": { 2023-07-14T11:53:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "num_rep", 2023-07-14T11:53:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3", 2023-07-14T11:53:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default replica count for any created local pool", 2023-07-14T11:53:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": { 2023-07-14T11:53:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pg_num", 2023-07-14T11:53:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "128", 2023-07-14T11:53:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T11:53:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "prefix": { 2023-07-14T11:53:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prefix", 2023-07-14T11:53:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "name prefix for any created local pool", 2023-07-14T11:53:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "subtree": { 2023-07-14T11:53:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "subtree", 2023-07-14T11:53:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "rack", 2023-07-14T11:53:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T11:53:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T11:53:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mds_autoscaler", 2023-07-14T11:53:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.497 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.497 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.497 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.497 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.497 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.498 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.498 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.498 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.498 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.499 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.499 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.499 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.499 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.499 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.499 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.500 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.500 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.500 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.500 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.500 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.501 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.501 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.501 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.501 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.501 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.504 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.504 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.504 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.504 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.505 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.505 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.505 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.506 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mirroring", 2023-07-14T11:53:52.506 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.506 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.506 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.506 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.507 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.507 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.507 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.507 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.507 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.508 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.508 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.508 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.508 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.508 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.509 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.509 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.509 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.509 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.509 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.510 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.510 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.510 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.510 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.511 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.511 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.511 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.511 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.511 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.511 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.512 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.512 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.512 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.512 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.512 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.513 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.513 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.513 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.513 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.514 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.514 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.514 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.514 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.515 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.515 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.515 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.515 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.515 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.515 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.516 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.516 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.516 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.516 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.516 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.517 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.517 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.517 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.517 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.518 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.518 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.518 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.518 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.518 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.519 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.519 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.519 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.519 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.519 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.520 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.520 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.520 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.520 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.521 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "nfs", 2023-07-14T11:53:52.521 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.521 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.521 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.522 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.522 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.522 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.522 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.522 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.523 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.523 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.523 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.523 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.523 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.524 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.524 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.524 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.524 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.524 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.524 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.525 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.525 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.525 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.525 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.526 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.526 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.526 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.526 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.526 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.527 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.527 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.527 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.527 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.527 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.528 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.528 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.528 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.528 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.529 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.529 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.529 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.529 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.529 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.529 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.530 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.530 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.530 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.530 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.530 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.531 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.531 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.531 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.531 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.531 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.532 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.532 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.532 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.532 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.533 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.533 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.533 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.533 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.533 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.534 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.534 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.534 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.534 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.534 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.535 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.535 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.535 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.535 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.536 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "orchestrator", 2023-07-14T11:53:52.536 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.536 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.537 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.537 INFO:teuthology.orchestra.run.smithi161.stdout: "fail_fs": { 2023-07-14T11:53:52.537 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "fail_fs", 2023-07-14T11:53:52.537 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.538 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.538 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.538 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.538 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.538 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.539 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.539 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T11:53:52.539 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.539 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.539 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.540 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.540 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.540 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.540 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.540 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.541 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.541 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.541 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.541 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.542 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.542 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.542 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.542 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.543 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.543 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.543 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.543 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.543 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.543 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.544 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.544 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.545 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.545 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.545 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.546 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.546 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.546 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.546 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.546 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.546 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.547 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.547 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.547 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.548 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.548 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.548 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.548 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.548 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.549 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.549 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.549 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.549 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.549 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.550 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.550 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.550 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.550 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.550 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.551 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.551 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.551 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.551 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.552 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.552 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.552 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.552 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.553 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.553 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.553 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.553 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.553 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.554 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.554 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.554 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.554 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.555 INFO:teuthology.orchestra.run.smithi161.stdout: "orchestrator": { 2023-07-14T11:53:52.555 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "orchestrator", 2023-07-14T11:53:52.555 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.555 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.555 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.556 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.556 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.556 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.556 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.556 INFO:teuthology.orchestra.run.smithi161.stdout: "cephadm", 2023-07-14T11:53:52.557 INFO:teuthology.orchestra.run.smithi161.stdout: "rook", 2023-07-14T11:53:52.557 INFO:teuthology.orchestra.run.smithi161.stdout: "test_orchestrator" 2023-07-14T11:53:52.557 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.557 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Orchestrator backend", 2023-07-14T11:53:52.557 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.558 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.558 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.559 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "osd_perf_query", 2023-07-14T11:53:52.559 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.559 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.559 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.560 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.560 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.560 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.560 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.560 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.561 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.561 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.561 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.561 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.561 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.562 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.562 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.562 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.562 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.562 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.563 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.563 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.563 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.563 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.564 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.564 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.564 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.565 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.565 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.565 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.565 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.565 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.566 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.566 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.566 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.566 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.567 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.567 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.567 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.568 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.568 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.568 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.568 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.568 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.569 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.569 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.569 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.569 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.569 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.570 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.570 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.570 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.570 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.570 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.571 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.571 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.571 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.572 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.572 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.572 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.572 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.572 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.573 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.573 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.573 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.573 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.573 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.573 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.574 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "osd_support", 2023-07-14T11:53:52.574 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.575 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.575 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.575 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.575 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.575 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.576 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.576 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.576 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.576 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.576 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.577 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.577 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.577 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.577 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.577 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.577 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.578 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.578 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.578 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.578 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.578 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.579 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.579 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.579 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.579 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.580 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.580 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.580 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.580 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.580 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.581 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.581 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.581 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.581 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.581 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.582 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.582 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.582 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.582 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.583 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.583 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.583 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.583 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.583 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.584 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.584 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.584 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.584 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.584 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.585 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.585 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.585 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.585 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.585 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.586 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.586 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.586 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.586 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.586 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.587 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.587 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.587 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.587 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.589 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pg_autoscaler", 2023-07-14T11:53:52.590 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.590 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.590 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.590 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.591 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.591 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.591 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.591 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.591 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.592 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.593 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.593 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.593 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.593 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.593 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.594 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.594 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.594 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.594 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.595 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.595 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.595 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.596 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.596 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.596 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.596 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.597 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.597 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.597 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.597 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.597 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.598 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.598 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.598 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.598 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.599 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.599 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.599 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.599 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.599 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.600 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.600 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.600 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.600 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.600 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.601 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.601 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.601 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.601 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.601 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.602 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.602 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.602 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.602 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.602 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.604 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.604 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.604 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.604 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.605 INFO:teuthology.orchestra.run.smithi161.stdout: "noautoscale": { 2023-07-14T11:53:52.605 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "noautoscale", 2023-07-14T11:53:52.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.605 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.606 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.606 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.606 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.606 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.606 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.606 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "global autoscale flag", 2023-07-14T11:53:52.607 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T11:53:52.607 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.607 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.607 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:52.608 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:52.608 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.608 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.608 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.608 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:52.609 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.609 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.609 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.609 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.609 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.610 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.610 INFO:teuthology.orchestra.run.smithi161.stdout: "threshold": { 2023-07-14T11:53:52.610 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "threshold", 2023-07-14T11:53:52.611 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:52.611 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.611 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.611 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3.0", 2023-07-14T11:53:52.611 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1.0", 2023-07-14T11:53:52.611 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.612 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.612 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "scaling threshold", 2023-07-14T11:53:52.612 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T11:53:52.612 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.613 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.614 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "progress", 2023-07-14T11:53:52.614 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.614 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.614 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.614 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_pg_recovery_event": { 2023-07-14T11:53:52.615 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_pg_recovery_event", 2023-07-14T11:53:52.615 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.615 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.615 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.615 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.615 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.616 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.616 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.616 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T11:53:52.616 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.617 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.617 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.617 INFO:teuthology.orchestra.run.smithi161.stdout: "enabled": { 2023-07-14T11:53:52.617 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enabled", 2023-07-14T11:53:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.623 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.623 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.623 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.625 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.625 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.625 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.625 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.625 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.626 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.626 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.626 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.626 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.633 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.633 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.633 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.633 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.633 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.634 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.634 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.634 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.634 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.635 INFO:teuthology.orchestra.run.smithi161.stdout: "max_completed_events": { 2023-07-14T11:53:52.635 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_completed_events", 2023-07-14T11:53:52.635 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.635 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.635 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.636 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "50", 2023-07-14T11:53:52.636 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.636 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.636 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.636 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "number of past completed events to remember", 2023-07-14T11:53:52.637 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.637 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.637 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:52.637 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:52.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.638 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.638 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.638 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:52.638 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.639 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.639 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.639 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long the module is going to sleep", 2023-07-14T11:53:52.639 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.639 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.640 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.641 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus", 2023-07-14T11:53:52.641 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.641 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.641 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.641 INFO:teuthology.orchestra.run.smithi161.stdout: "cache": { 2023-07-14T11:53:52.642 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cache", 2023-07-14T11:53:52.642 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.642 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.642 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.642 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.642 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.643 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.643 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.643 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.643 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.643 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.644 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.644 INFO:teuthology.orchestra.run.smithi161.stdout: "exclude_perf_counters": { 2023-07-14T11:53:52.644 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "exclude_perf_counters", 2023-07-14T11:53:52.644 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.645 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.645 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.645 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.645 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.645 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.646 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.646 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T11:53:52.646 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T11:53:52.646 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.646 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.647 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.647 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.647 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.647 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.647 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.648 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.648 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.648 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.648 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.648 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.649 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.649 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.649 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.649 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.649 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.650 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.650 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.650 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.650 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.650 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.651 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.651 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.651 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.652 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.652 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.652 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.652 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.652 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.653 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.653 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.653 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.653 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.654 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.654 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.654 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.654 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.654 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.654 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.655 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.655 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.655 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.655 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.655 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.656 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.656 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.656 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.656 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.657 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.657 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.657 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.657 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.657 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.658 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.658 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.658 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.658 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.659 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.659 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.659 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.659 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.659 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.660 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.660 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.660 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.660 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.661 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_stats_pools": { 2023-07-14T11:53:52.661 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_stats_pools", 2023-07-14T11:53:52.661 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.661 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.661 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.662 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.662 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.662 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.662 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.662 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.662 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.663 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.663 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.663 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T11:53:52.663 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T11:53:52.664 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.664 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.664 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.664 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "300", 2023-07-14T11:53:52.664 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.665 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.665 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.665 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.665 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.665 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.666 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.666 INFO:teuthology.orchestra.run.smithi161.stdout: "scrape_interval": { 2023-07-14T11:53:52.666 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "scrape_interval", 2023-07-14T11:53:52.666 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:52.667 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.667 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.667 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15.0", 2023-07-14T11:53:52.667 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.667 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.667 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.668 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.668 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.668 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.668 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.669 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:52.669 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:52.669 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.669 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.669 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.670 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "::", 2023-07-14T11:53:52.670 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.670 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.670 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.670 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T11:53:52.671 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.671 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.671 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.671 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:52.672 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:52.672 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.672 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.672 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.672 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "9283", 2023-07-14T11:53:52.673 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.673 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.673 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.673 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T11:53:52.673 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.673 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.674 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.674 INFO:teuthology.orchestra.run.smithi161.stdout: "stale_cache_strategy": { 2023-07-14T11:53:52.674 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "stale_cache_strategy", 2023-07-14T11:53:52.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.675 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.675 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.675 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "log", 2023-07-14T11:53:52.675 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.675 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.676 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.676 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.676 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.676 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.676 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.677 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_behaviour": { 2023-07-14T11:53:52.677 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_behaviour", 2023-07-14T11:53:52.677 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.677 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.678 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "default", 2023-07-14T11:53:52.678 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.678 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.678 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.679 INFO:teuthology.orchestra.run.smithi161.stdout: "default", 2023-07-14T11:53:52.679 INFO:teuthology.orchestra.run.smithi161.stdout: "error" 2023-07-14T11:53:52.679 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.679 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.679 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.680 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.680 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.680 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_error_status_code": { 2023-07-14T11:53:52.680 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_error_status_code", 2023-07-14T11:53:52.681 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.681 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.681 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.681 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "500", 2023-07-14T11:53:52.681 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "400", 2023-07-14T11:53:52.682 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "599", 2023-07-14T11:53:52.682 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.682 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.682 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.682 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.682 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.683 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_support", 2023-07-14T11:53:52.684 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.684 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.684 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.684 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.684 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.685 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.685 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.685 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.685 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.686 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.686 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.686 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.686 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.686 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.687 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.687 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.687 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.687 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.687 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.688 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.688 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.688 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.688 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.689 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.689 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.689 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.689 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.689 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.689 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.690 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.690 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.690 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.690 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.690 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.691 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.691 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.691 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.691 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.692 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.692 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.692 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.692 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.692 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.693 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.693 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.693 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.693 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.693 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.693 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.694 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.694 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.694 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.694 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.694 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.695 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.695 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.695 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.696 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.696 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.696 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.696 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.696 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.697 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.697 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.697 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.697 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.697 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.698 INFO:teuthology.orchestra.run.smithi161.stdout: "max_concurrent_snap_create": { 2023-07-14T11:53:52.698 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_concurrent_snap_create", 2023-07-14T11:53:52.698 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.698 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.699 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.699 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:52.699 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.699 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.699 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.700 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.700 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.700 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.700 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.701 INFO:teuthology.orchestra.run.smithi161.stdout: "mirror_snapshot_schedule": { 2023-07-14T11:53:52.701 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T11:53:52.701 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.701 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.702 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.702 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.702 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.702 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.702 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.703 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.703 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.703 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.703 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.704 INFO:teuthology.orchestra.run.smithi161.stdout: "trash_purge_schedule": { 2023-07-14T11:53:52.704 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "trash_purge_schedule", 2023-07-14T11:53:52.704 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.704 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.704 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.705 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.705 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.705 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.705 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.705 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.706 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.706 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.706 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.707 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "restful", 2023-07-14T11:53:52.707 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.707 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.707 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.708 INFO:teuthology.orchestra.run.smithi161.stdout: "enable_auth": { 2023-07-14T11:53:52.708 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enable_auth", 2023-07-14T11:53:52.708 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.708 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.708 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.709 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.709 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.709 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.709 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.709 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: "key_file": { 2023-07-14T11:53:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "key_file", 2023-07-14T11:53:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.711 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.712 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.712 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.712 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.712 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.713 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.713 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.713 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.713 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.713 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.714 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.714 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.714 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.714 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.714 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.715 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.715 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.715 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.715 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.715 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.716 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.716 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.716 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.716 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.716 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.717 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.717 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.717 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.717 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.718 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.718 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.718 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.718 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.718 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.719 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.719 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.719 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.719 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.719 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.719 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.720 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.720 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.720 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.720 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.721 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.721 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.721 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.721 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.721 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.722 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.722 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.722 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.722 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.722 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.723 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.723 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.723 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.723 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.723 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.724 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.724 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.724 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.724 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.725 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.725 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.725 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.725 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.725 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.725 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.726 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.726 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.726 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.726 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.726 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.727 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:52.727 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:52.727 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.727 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.729 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.729 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.729 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.729 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.729 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:52.730 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:52.730 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.730 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.730 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.730 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.731 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.731 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.731 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.731 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.731 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.732 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.732 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.732 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.732 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.733 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rgw", 2023-07-14T11:53:52.733 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.733 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.733 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.733 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.734 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.734 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.734 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.734 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.734 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.735 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.735 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.735 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.735 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.735 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.736 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.736 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.736 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.736 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.736 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.737 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.737 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.737 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.737 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.738 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.738 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.738 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.738 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.738 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.739 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.739 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.739 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.739 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.739 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.740 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.740 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.740 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.740 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.741 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.741 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.741 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.741 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.741 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.741 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.742 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.742 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.742 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.742 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.742 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.743 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.743 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.743 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.743 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.743 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.744 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.744 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.744 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.744 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.745 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.745 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.745 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.745 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.745 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.746 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.746 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.746 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.746 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.746 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.747 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.747 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.747 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.747 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.748 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rook", 2023-07-14T11:53:52.748 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.748 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.748 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.748 INFO:teuthology.orchestra.run.smithi161.stdout: "drive_group_interval": { 2023-07-14T11:53:52.749 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "drive_group_interval", 2023-07-14T11:53:52.749 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:52.749 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.749 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.749 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "300.0", 2023-07-14T11:53:52.750 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.750 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.750 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.750 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T11:53:52.750 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.751 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.751 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.751 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.751 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.752 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.752 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.752 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.752 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.752 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.753 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.753 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.753 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.753 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.753 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.754 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.754 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.754 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.754 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.754 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.754 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.755 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.755 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.755 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.755 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.756 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.756 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.756 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.756 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.756 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.757 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.757 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.757 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.757 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.757 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.758 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.758 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.758 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.758 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.759 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.759 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.759 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.759 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.759 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.760 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.760 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.760 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.760 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.761 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.761 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.761 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.761 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.761 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.762 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.762 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.762 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.762 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.763 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.763 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.763 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.763 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.763 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.764 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.764 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.764 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.764 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.764 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.765 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.765 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.765 INFO:teuthology.orchestra.run.smithi161.stdout: "storage_class": { 2023-07-14T11:53:52.765 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "storage_class", 2023-07-14T11:53:52.766 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.766 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.766 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.766 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "local", 2023-07-14T11:53:52.766 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.766 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.767 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.767 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T11:53:52.767 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.767 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.767 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.768 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.768 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.768 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "selftest", 2023-07-14T11:53:52.769 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.769 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.769 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.769 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.769 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.770 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.770 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.770 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.770 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.770 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.771 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.771 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.771 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.771 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.771 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.772 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.772 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.772 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.772 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.772 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.773 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.773 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.773 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.773 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.774 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.774 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.774 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.774 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.774 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.775 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.775 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.775 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.775 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.775 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.776 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.776 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.776 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.776 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.776 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.777 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.777 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.777 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.777 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.777 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.778 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.778 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.778 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.778 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.778 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.779 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.779 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.779 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.779 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.779 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.780 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.780 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.780 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.780 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.781 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.781 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.781 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.781 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.781 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.782 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.782 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.782 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.782 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.782 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.782 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.783 INFO:teuthology.orchestra.run.smithi161.stdout: "roption1": { 2023-07-14T11:53:52.783 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "roption1", 2023-07-14T11:53:52.783 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.783 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.784 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.784 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.784 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.784 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.784 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.785 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.785 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.785 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.785 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.786 INFO:teuthology.orchestra.run.smithi161.stdout: "roption2": { 2023-07-14T11:53:52.786 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "roption2", 2023-07-14T11:53:52.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.786 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.786 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.787 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "xyz", 2023-07-14T11:53:52.787 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.787 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.787 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.787 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.787 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.788 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.788 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.788 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption1": { 2023-07-14T11:53:52.788 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption1", 2023-07-14T11:53:52.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.789 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.789 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.789 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.789 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.790 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.790 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.790 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.790 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.790 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.791 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.791 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption2": { 2023-07-14T11:53:52.791 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption2", 2023-07-14T11:53:52.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.791 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.792 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.792 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.792 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.792 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.792 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.793 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.793 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.793 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.793 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption3": { 2023-07-14T11:53:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption3", 2023-07-14T11:53:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.795 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.795 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.795 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.795 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.795 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.795 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.796 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.796 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.796 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption4": { 2023-07-14T11:53:52.796 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption4", 2023-07-14T11:53:52.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.797 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.797 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.797 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.797 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.798 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.798 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.798 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.798 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.798 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.799 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.799 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption5": { 2023-07-14T11:53:52.799 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption5", 2023-07-14T11:53:52.799 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.799 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.800 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.800 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.800 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.800 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.800 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.801 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.801 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.801 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.801 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.802 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption6": { 2023-07-14T11:53:52.802 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption6", 2023-07-14T11:53:52.802 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.802 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.802 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.803 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.803 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.803 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.803 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.803 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.803 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.804 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.804 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.804 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption7": { 2023-07-14T11:53:52.804 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption7", 2023-07-14T11:53:52.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.805 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.805 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.805 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.805 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:52.806 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "42", 2023-07-14T11:53:52.806 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.806 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.806 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.806 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.807 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.807 INFO:teuthology.orchestra.run.smithi161.stdout: "testkey": { 2023-07-14T11:53:52.807 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testkey", 2023-07-14T11:53:52.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.808 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.808 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.808 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.808 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.808 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.809 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.809 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.809 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.809 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.809 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.810 INFO:teuthology.orchestra.run.smithi161.stdout: "testlkey": { 2023-07-14T11:53:52.810 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testlkey", 2023-07-14T11:53:52.810 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.810 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.811 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.811 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.811 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.811 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.811 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi161.stdout: "testnewline": { 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testnewline", 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.815 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.815 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.815 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.815 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.815 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.816 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.816 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snap_schedule", 2023-07-14T11:53:52.816 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.817 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.817 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.817 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_m_granularity": { 2023-07-14T11:53:52.817 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_m_granularity", 2023-07-14T11:53:52.817 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.818 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.818 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.818 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.818 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.818 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.819 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.819 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T11:53:52.819 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.819 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.819 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.820 INFO:teuthology.orchestra.run.smithi161.stdout: "dump_on_update": { 2023-07-14T11:53:52.820 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "dump_on_update", 2023-07-14T11:53:52.820 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.820 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.821 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.821 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.821 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.821 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.821 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.822 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "dump database to debug log on update", 2023-07-14T11:53:52.822 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.822 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.822 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.823 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.823 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.823 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.823 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.823 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.824 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.824 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.824 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.824 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.824 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.825 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.825 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.825 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.825 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.825 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.826 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.826 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.826 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.826 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.826 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.827 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.827 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.827 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.827 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.828 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.828 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.828 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.828 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.828 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.829 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.829 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.829 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.829 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.830 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.830 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.830 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.830 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.831 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.831 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.831 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.831 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.831 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.831 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.832 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.832 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.832 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.832 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.832 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.833 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.833 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.833 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.833 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.834 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.834 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.834 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.834 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.834 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.835 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.835 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.835 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.835 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.835 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.836 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.836 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.837 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "stats", 2023-07-14T11:53:52.837 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.837 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.837 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.838 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.838 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.838 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.838 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.838 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.839 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.839 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.839 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.839 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.839 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.840 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.840 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.840 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.840 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.840 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.840 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.841 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.841 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.841 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.841 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.842 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.842 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.842 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.842 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.842 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.843 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.843 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.843 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.843 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.843 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.844 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.844 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.844 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.844 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.845 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.845 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.845 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.845 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.845 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.846 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.846 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.846 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.846 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.846 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.846 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.847 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.847 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.847 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.847 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.847 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.848 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.848 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.848 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.848 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.849 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.849 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.849 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.849 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.849 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.850 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.850 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.850 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.850 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.850 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.851 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.851 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.851 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.851 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.852 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "status", 2023-07-14T11:53:52.852 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.852 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.852 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.853 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.853 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.853 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.853 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.853 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.853 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.854 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.854 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.854 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.854 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.854 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.855 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.855 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.855 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.855 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.855 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.856 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.856 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.856 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.856 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.857 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.857 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.857 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.857 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.857 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.858 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.858 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.858 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.858 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.858 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.859 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.859 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.859 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.859 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.859 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.860 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.860 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.860 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.860 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.860 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.861 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.861 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.861 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.861 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.861 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.862 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.862 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.862 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.862 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.862 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.864 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.864 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.864 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.864 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.864 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.865 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.865 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.865 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.865 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.865 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.866 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.866 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.866 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.866 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.867 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "telegraf", 2023-07-14T11:53:52.867 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.867 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.867 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.867 INFO:teuthology.orchestra.run.smithi161.stdout: "address": { 2023-07-14T11:53:52.867 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "address", 2023-07-14T11:53:52.868 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.868 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.868 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.868 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T11:53:52.868 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.869 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.869 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.869 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.869 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.869 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.870 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.870 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:52.870 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:52.870 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.871 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.871 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.871 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15", 2023-07-14T11:53:52.871 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.871 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.871 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.872 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.872 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.872 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.872 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.873 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.873 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.873 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.873 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.873 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.874 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.874 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.874 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.874 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.874 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.875 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.875 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.875 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.875 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.875 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.875 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.876 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.876 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.876 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.876 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.877 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.877 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.877 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.877 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.877 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.878 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.878 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.878 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.878 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.878 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.879 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.879 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.879 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.879 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.880 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.880 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.880 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.880 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.880 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.881 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.881 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.881 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.881 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.881 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.882 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.882 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.882 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.882 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.882 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.882 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.883 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.883 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.883 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.883 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.884 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.884 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.884 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.884 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.884 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.885 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.885 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.885 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.885 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.885 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.886 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.886 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.886 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.886 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.887 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "telemetry", 2023-07-14T11:53:52.887 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.887 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.887 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.888 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_basic": { 2023-07-14T11:53:52.888 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_basic", 2023-07-14T11:53:52.888 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.888 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.888 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.889 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.889 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.889 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.889 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.889 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T11:53:52.890 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.890 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.890 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.890 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_crash": { 2023-07-14T11:53:52.891 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_crash", 2023-07-14T11:53:52.891 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.891 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.891 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.891 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.891 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.892 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.892 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.892 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T11:53:52.892 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.892 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.893 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.893 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_device": { 2023-07-14T11:53:52.893 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_device", 2023-07-14T11:53:52.893 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.894 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.894 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.894 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:52.894 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.894 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.895 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.895 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T11:53:52.895 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.895 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.895 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.896 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_ident": { 2023-07-14T11:53:52.896 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_ident", 2023-07-14T11:53:52.896 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.896 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.896 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.897 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.897 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.897 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.897 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.897 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T11:53:52.898 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.898 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.898 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.898 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_perf": { 2023-07-14T11:53:52.899 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_perf", 2023-07-14T11:53:52.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.899 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.899 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.899 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.900 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.900 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.900 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.900 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T11:53:52.900 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.901 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.901 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.901 INFO:teuthology.orchestra.run.smithi161.stdout: "contact": { 2023-07-14T11:53:52.901 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "contact", 2023-07-14T11:53:52.901 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.902 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.902 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.902 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.902 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.902 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.903 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.903 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.903 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.903 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.903 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.904 INFO:teuthology.orchestra.run.smithi161.stdout: "description": { 2023-07-14T11:53:52.904 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "description", 2023-07-14T11:53:52.904 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.904 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.905 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.905 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.905 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.905 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.905 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.905 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.906 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.906 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.906 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.907 INFO:teuthology.orchestra.run.smithi161.stdout: "device_url": { 2023-07-14T11:53:52.907 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_url", 2023-07-14T11:53:52.907 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.907 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.907 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.907 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T11:53:52.908 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.908 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.908 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.908 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.908 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.909 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.909 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.909 INFO:teuthology.orchestra.run.smithi161.stdout: "enabled": { 2023-07-14T11:53:52.909 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enabled", 2023-07-14T11:53:52.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.910 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.910 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.910 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.910 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.911 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.911 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.911 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.911 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.911 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.912 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.912 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:52.912 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:52.913 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.913 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.913 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.913 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "24", 2023-07-14T11:53:52.913 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "8", 2023-07-14T11:53:52.914 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.914 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.914 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.914 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.914 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.915 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.915 INFO:teuthology.orchestra.run.smithi161.stdout: "last_opt_revision": { 2023-07-14T11:53:52.915 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "last_opt_revision", 2023-07-14T11:53:52.915 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.916 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.916 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.916 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1", 2023-07-14T11:53:52.916 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.916 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.916 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.917 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.917 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.917 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.917 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.918 INFO:teuthology.orchestra.run.smithi161.stdout: "leaderboard": { 2023-07-14T11:53:52.918 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "leaderboard", 2023-07-14T11:53:52.918 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.918 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.918 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.919 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.919 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.919 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.919 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.919 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.920 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.920 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.920 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.920 INFO:teuthology.orchestra.run.smithi161.stdout: "leaderboard_description": { 2023-07-14T11:53:52.920 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "leaderboard_description", 2023-07-14T11:53:52.921 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.921 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.921 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.921 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.921 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.922 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.922 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.922 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.922 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.922 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.923 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.923 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.923 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.923 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.924 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.924 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.924 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.924 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.924 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.925 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.925 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.925 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.925 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.925 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.926 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.926 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.926 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.926 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.926 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.926 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.927 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.927 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.927 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.927 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.928 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.928 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.928 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.928 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.928 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.929 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.929 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.929 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.929 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.929 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.930 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.930 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.930 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.930 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.930 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "organization": { 2023-07-14T11:53:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "organization", 2023-07-14T11:53:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "proxy": { 2023-07-14T11:53:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "proxy", 2023-07-14T11:53:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "url": { 2023-07-14T11:53:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "url", 2023-07-14T11:53:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T11:53:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "test_orchestrator", 2023-07-14T11:53:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "volumes", 2023-07-14T11:53:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "max_concurrent_clones": { 2023-07-14T11:53:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_concurrent_clones", 2023-07-14T11:53:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "4", 2023-07-14T11:53:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T11:53:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "snapshot_clone_delay": { 2023-07-14T11:53:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snapshot_clone_delay", 2023-07-14T11:53:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T11:53:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix", 2023-07-14T11:53:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "discovery_interval": { 2023-07-14T11:53:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "discovery_interval", 2023-07-14T11:53:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "100", 2023-07-14T11:53:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "identifier": { 2023-07-14T11:53:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "identifier", 2023-07-14T11:53:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_host": { 2023-07-14T11:53:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_host", 2023-07-14T11:53:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_port": { 2023-07-14T11:53:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_port", 2023-07-14T11:53:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10051", 2023-07-14T11:53:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_sender": { 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_sender", 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "modules": [ 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "cephadm", 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "dashboard", 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "iostat", 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "nfs", 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "restful", 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_schedule" 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "available_modules": [ 2023-07-14T11:53:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alerts", 2023-07-14T11:53:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.027 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.027 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.027 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.027 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.027 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.028 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.028 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.028 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.028 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.029 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.029 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.029 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.029 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.029 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.030 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.030 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.030 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.030 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.030 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.030 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.031 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.031 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_destination": { 2023-07-14T11:53:53.031 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_destination", 2023-07-14T11:53:53.031 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.032 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.032 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.032 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.032 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.032 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.033 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.033 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Email address to send alerts to", 2023-07-14T11:53:53.033 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.033 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.033 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.034 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_from_name": { 2023-07-14T11:53:53.034 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_from_name", 2023-07-14T11:53:53.034 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.034 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.035 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.035 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "Ceph", 2023-07-14T11:53:53.035 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.035 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.035 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.036 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Email From: name", 2023-07-14T11:53:53.036 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.036 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.036 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.037 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_host": { 2023-07-14T11:53:53.037 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_host", 2023-07-14T11:53:53.037 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.037 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.037 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.037 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.038 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.038 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.038 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.038 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP server", 2023-07-14T11:53:53.038 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.039 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.039 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.039 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_password": { 2023-07-14T11:53:53.039 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_password", 2023-07-14T11:53:53.040 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.040 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.040 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.040 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.040 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.041 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.041 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.041 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Password to authenticate with", 2023-07-14T11:53:53.041 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.041 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.041 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.042 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_port": { 2023-07-14T11:53:53.042 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_port", 2023-07-14T11:53:53.042 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.043 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.043 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.043 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "465", 2023-07-14T11:53:53.043 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.043 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.043 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.044 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP port", 2023-07-14T11:53:53.044 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.044 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.044 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.045 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_sender": { 2023-07-14T11:53:53.045 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_sender", 2023-07-14T11:53:53.045 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.045 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.045 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.046 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.046 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.046 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.046 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.046 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SMTP envelope sender", 2023-07-14T11:53:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_ssl": { 2023-07-14T11:53:53.047 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_ssl", 2023-07-14T11:53:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.048 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T11:53:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.049 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.050 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.050 INFO:teuthology.orchestra.run.smithi161.stdout: "smtp_user": { 2023-07-14T11:53:53.050 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "smtp_user", 2023-07-14T11:53:53.050 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.051 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.051 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.051 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.051 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.051 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.052 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.052 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "User to authenticate as", 2023-07-14T11:53:53.052 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.052 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.052 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.052 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.053 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.053 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "balancer", 2023-07-14T11:53:53.053 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.054 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.054 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.054 INFO:teuthology.orchestra.run.smithi161.stdout: "active": { 2023-07-14T11:53:53.054 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "active", 2023-07-14T11:53:53.054 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.055 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.055 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.055 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.055 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.055 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T11:53:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.056 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: "begin_time": { 2023-07-14T11:53:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "begin_time", 2023-07-14T11:53:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.057 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.058 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.058 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0000", 2023-07-14T11:53:53.058 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.058 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.058 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.058 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T11:53:53.059 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T11:53:53.059 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.059 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.059 INFO:teuthology.orchestra.run.smithi161.stdout: "begin_weekday": { 2023-07-14T11:53:53.060 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "begin_weekday", 2023-07-14T11:53:53.060 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:53.060 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.060 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.060 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:53.061 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0", 2023-07-14T11:53:53.061 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "6", 2023-07-14T11:53:53.061 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.061 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T11:53:53.061 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T11:53:53.062 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.062 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.062 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_max_iterations": { 2023-07-14T11:53:53.062 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_max_iterations", 2023-07-14T11:53:53.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:53.063 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.063 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.063 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "25", 2023-07-14T11:53:53.063 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:53.063 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "250", 2023-07-14T11:53:53.064 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.064 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T11:53:53.064 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.064 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.064 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.065 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_metrics": { 2023-07-14T11:53:53.065 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_metrics", 2023-07-14T11:53:53.065 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.065 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.066 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.066 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T11:53:53.066 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.066 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.066 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.067 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T11:53:53.067 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T11:53:53.067 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.067 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.068 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_compat_step": { 2023-07-14T11:53:53.068 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crush_compat_step", 2023-07-14T11:53:53.068 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.068 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.068 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.069 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0.5", 2023-07-14T11:53:53.069 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0.001", 2023-07-14T11:53:53.069 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "0.999", 2023-07-14T11:53:53.069 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.069 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "aggressiveness of optimization", 2023-07-14T11:53:53.069 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T11:53:53.070 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.070 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.070 INFO:teuthology.orchestra.run.smithi161.stdout: "end_time": { 2023-07-14T11:53:53.070 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "end_time", 2023-07-14T11:53:53.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.071 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.071 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.071 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "2359", 2023-07-14T11:53:53.071 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.072 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.072 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.072 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T11:53:53.072 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T11:53:53.072 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.073 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.073 INFO:teuthology.orchestra.run.smithi161.stdout: "end_weekday": { 2023-07-14T11:53:53.073 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "end_weekday", 2023-07-14T11:53:53.073 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:53.074 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.074 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.074 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:53.074 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "0", 2023-07-14T11:53:53.074 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "6", 2023-07-14T11:53:53.075 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.075 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T11:53:53.075 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T11:53:53.075 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.075 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.076 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.076 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.076 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.076 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.076 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.077 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.077 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.077 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.077 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.077 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.078 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.078 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.078 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.078 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.078 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.079 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.079 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.079 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.079 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.079 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.080 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.080 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.080 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.080 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.080 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.081 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.081 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.081 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.081 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.081 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.082 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.082 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.082 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.082 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.083 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.083 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.083 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.083 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.084 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.084 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.084 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.084 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.084 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.085 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.085 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.085 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.085 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.085 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.086 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.086 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.086 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.086 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.087 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.087 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.087 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.087 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.087 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.087 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.088 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.088 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.088 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.088 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.088 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.089 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.089 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.089 INFO:teuthology.orchestra.run.smithi161.stdout: "min_score": { 2023-07-14T11:53:53.089 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "min_score", 2023-07-14T11:53:53.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.090 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.090 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.090 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:53.090 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.091 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.091 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.091 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T11:53:53.091 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.091 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.092 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.092 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": { 2023-07-14T11:53:53.092 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mode", 2023-07-14T11:53:53.092 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "upmap", 2023-07-14T11:53:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-compat", 2023-07-14T11:53:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "none", 2023-07-14T11:53:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap" 2023-07-14T11:53:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Balancer mode", 2023-07-14T11:53:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_ids": { 2023-07-14T11:53:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pool_ids", 2023-07-14T11:53:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T11:53:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T11:53:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap_max_deviation": { 2023-07-14T11:53:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "upmap_max_deviation", 2023-07-14T11:53:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T11:53:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T11:53:53.105 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.105 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.105 INFO:teuthology.orchestra.run.smithi161.stdout: "upmap_max_optimizations": { 2023-07-14T11:53:53.105 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "upmap_max_optimizations", 2023-07-14T11:53:53.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:53.106 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.106 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.106 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.106 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.106 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.107 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.107 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T11:53:53.107 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.107 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.107 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.108 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cephadm", 2023-07-14T11:53:53.109 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.109 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.109 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.109 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_down_multiplier": { 2023-07-14T11:53:53.109 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_down_multiplier", 2023-07-14T11:53:53.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.110 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.110 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.110 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3.0", 2023-07-14T11:53:53.110 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.110 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.111 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.111 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T11:53:53.111 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.111 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.111 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.112 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_refresh_rate": { 2023-07-14T11:53:53.112 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_refresh_rate", 2023-07-14T11:53:53.112 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.112 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.113 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.113 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "20", 2023-07-14T11:53:53.113 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.113 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.113 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.114 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T11:53:53.114 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.114 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.114 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.115 INFO:teuthology.orchestra.run.smithi161.stdout: "agent_starting_port": { 2023-07-14T11:53:53.115 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "agent_starting_port", 2023-07-14T11:53:53.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.115 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.115 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.115 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "4721", 2023-07-14T11:53:53.116 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.116 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.116 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.116 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T11:53:53.116 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.117 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.117 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.117 INFO:teuthology.orchestra.run.smithi161.stdout: "alertmanager_web_password": { 2023-07-14T11:53:53.117 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alertmanager_web_password", 2023-07-14T11:53:53.118 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.118 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.118 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.118 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:53.118 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.118 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.119 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.119 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Alertmanager web password", 2023-07-14T11:53:53.119 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.119 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.119 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.120 INFO:teuthology.orchestra.run.smithi161.stdout: "alertmanager_web_user": { 2023-07-14T11:53:53.120 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "alertmanager_web_user", 2023-07-14T11:53:53.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.120 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.121 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.121 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:53.121 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.121 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.121 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.122 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Alertmanager web user", 2023-07-14T11:53:53.122 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.122 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.122 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.123 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_ptrace": { 2023-07-14T11:53:53.123 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_ptrace", 2023-07-14T11:53:53.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.123 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.123 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.123 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.124 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.124 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.124 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.124 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T11:53:53.124 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T11:53:53.125 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.125 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.125 INFO:teuthology.orchestra.run.smithi161.stdout: "autotune_interval": { 2023-07-14T11:53:53.125 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "autotune_interval", 2023-07-14T11:53:53.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.126 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.126 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.126 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:53.126 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.127 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.127 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.127 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T11:53:53.127 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.127 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.128 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.128 INFO:teuthology.orchestra.run.smithi161.stdout: "autotune_memory_target_ratio": { 2023-07-14T11:53:53.128 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T11:53:53.128 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.129 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.129 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.129 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0.7", 2023-07-14T11:53:53.129 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.129 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.130 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.130 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T11:53:53.130 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.130 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.130 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.131 INFO:teuthology.orchestra.run.smithi161.stdout: "cgroups_split": { 2023-07-14T11:53:53.131 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cgroups_split", 2023-07-14T11:53:53.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.131 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.132 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.132 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.132 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.132 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.132 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.133 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T11:53:53.133 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.133 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.133 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.134 INFO:teuthology.orchestra.run.smithi161.stdout: "config_checks_enabled": { 2023-07-14T11:53:53.134 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_checks_enabled", 2023-07-14T11:53:53.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.134 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.134 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.135 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.135 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.135 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.135 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.135 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T11:53:53.136 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.136 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.136 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.136 INFO:teuthology.orchestra.run.smithi161.stdout: "config_dashboard": { 2023-07-14T11:53:53.137 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_dashboard", 2023-07-14T11:53:53.137 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.137 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.137 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.137 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.138 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.138 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.138 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.138 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T11:53:53.138 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.139 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.139 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.139 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_alertmanager": { 2023-07-14T11:53:53.139 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_alertmanager", 2023-07-14T11:53:53.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.140 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.140 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.140 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T11:53:53.140 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.141 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.141 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.141 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:53.141 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.141 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.142 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.142 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_base": { 2023-07-14T11:53:53.142 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_base", 2023-07-14T11:53:53.142 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.143 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.143 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.143 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T11:53:53.143 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.143 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.144 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.144 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Container image name, without the tag", 2023-07-14T11:53:53.144 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.145 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.145 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_elasticsearch": { 2023-07-14T11:53:53.145 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_elasticsearch", 2023-07-14T11:53:53.146 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.146 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.146 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.146 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T11:53:53.147 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.147 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.147 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.147 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "elasticsearch container image", 2023-07-14T11:53:53.148 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.148 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.149 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_grafana": { 2023-07-14T11:53:53.149 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_grafana", 2023-07-14T11:53:53.149 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.149 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.149 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.150 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T11:53:53.150 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.150 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.150 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.151 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:53.151 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.151 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.151 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.152 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_haproxy": { 2023-07-14T11:53:53.152 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_haproxy", 2023-07-14T11:53:53.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.152 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.153 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T11:53:53.153 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.153 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.153 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.153 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "HAproxy container image", 2023-07-14T11:53:53.154 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.154 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.154 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.154 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_agent": { 2023-07-14T11:53:53.155 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_agent", 2023-07-14T11:53:53.155 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.155 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.155 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.155 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T11:53:53.156 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.156 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.156 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.156 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger agent container image", 2023-07-14T11:53:53.156 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.157 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.157 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_collector": { 2023-07-14T11:53:53.157 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_collector", 2023-07-14T11:53:53.158 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.158 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.158 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.158 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T11:53:53.158 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.159 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.159 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.159 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger collector container image", 2023-07-14T11:53:53.159 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.159 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.160 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.160 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_jaeger_query": { 2023-07-14T11:53:53.160 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_jaeger_query", 2023-07-14T11:53:53.160 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.161 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.161 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.161 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T11:53:53.161 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.161 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.162 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.162 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Jaeger query container image", 2023-07-14T11:53:53.162 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.162 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.162 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.163 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_keepalived": { 2023-07-14T11:53:53.163 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_keepalived", 2023-07-14T11:53:53.163 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.163 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.164 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.164 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T11:53:53.164 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.164 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.164 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.165 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Keepalived container image", 2023-07-14T11:53:53.165 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.165 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.165 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.166 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_loki": { 2023-07-14T11:53:53.166 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_loki", 2023-07-14T11:53:53.166 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.166 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.166 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.166 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T11:53:53.167 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.167 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.167 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.167 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Loki container image", 2023-07-14T11:53:53.168 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.168 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.168 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.168 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_node_exporter": { 2023-07-14T11:53:53.169 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_node_exporter", 2023-07-14T11:53:53.169 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.169 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.169 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T11:53:53.170 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.170 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.170 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.170 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:53.170 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.170 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_prometheus": { 2023-07-14T11:53:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_prometheus", 2023-07-14T11:53:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.172 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.172 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.172 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T11:53:53.172 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.172 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.173 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.173 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus container image", 2023-07-14T11:53:53.173 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.173 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.174 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_promtail": { 2023-07-14T11:53:53.174 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_promtail", 2023-07-14T11:53:53.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.174 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.175 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.175 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T11:53:53.175 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.175 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.175 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.176 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Promtail container image", 2023-07-14T11:53:53.176 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.176 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.176 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.177 INFO:teuthology.orchestra.run.smithi161.stdout: "container_image_snmp_gateway": { 2023-07-14T11:53:53.177 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_image_snmp_gateway", 2023-07-14T11:53:53.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.177 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.177 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.178 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T11:53:53.178 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.178 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.178 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.178 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "SNMP Gateway container image", 2023-07-14T11:53:53.178 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.179 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.179 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.179 INFO:teuthology.orchestra.run.smithi161.stdout: "container_init": { 2023-07-14T11:53:53.179 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "container_init", 2023-07-14T11:53:53.180 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.180 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.180 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.180 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.180 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.181 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.181 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.181 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T11:53:53.181 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.181 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.182 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.182 INFO:teuthology.orchestra.run.smithi161.stdout: "daemon_cache_timeout": { 2023-07-14T11:53:53.182 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "daemon_cache_timeout", 2023-07-14T11:53:53.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.183 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.183 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:53.183 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.183 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.183 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.184 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T11:53:53.184 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.184 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.185 INFO:teuthology.orchestra.run.smithi161.stdout: "default_cephadm_command_timeout": { 2023-07-14T11:53:53.185 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T11:53:53.185 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.185 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.185 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.186 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "900", 2023-07-14T11:53:53.186 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.186 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.186 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.186 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T11:53:53.187 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.187 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.187 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.187 INFO:teuthology.orchestra.run.smithi161.stdout: "default_registry": { 2023-07-14T11:53:53.188 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "default_registry", 2023-07-14T11:53:53.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.188 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.188 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "docker.io", 2023-07-14T11:53:53.189 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.189 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.189 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.189 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T11:53:53.189 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.190 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.190 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.190 INFO:teuthology.orchestra.run.smithi161.stdout: "device_cache_timeout": { 2023-07-14T11:53:53.190 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_cache_timeout", 2023-07-14T11:53:53.190 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.191 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.191 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.191 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1800", 2023-07-14T11:53:53.191 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.191 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.192 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.192 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache device inventory", 2023-07-14T11:53:53.192 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.192 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.193 INFO:teuthology.orchestra.run.smithi161.stdout: "device_enhanced_scan": { 2023-07-14T11:53:53.193 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_enhanced_scan", 2023-07-14T11:53:53.193 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.193 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.194 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.194 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.194 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.194 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.194 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.195 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T11:53:53.195 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.195 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.195 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.195 INFO:teuthology.orchestra.run.smithi161.stdout: "facts_cache_timeout": { 2023-07-14T11:53:53.196 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "facts_cache_timeout", 2023-07-14T11:53:53.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.196 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.196 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.196 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:53.197 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.197 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.197 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.197 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "seconds to cache host facts data", 2023-07-14T11:53:53.197 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.198 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.198 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.198 INFO:teuthology.orchestra.run.smithi161.stdout: "host_check_interval": { 2023-07-14T11:53:53.198 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "host_check_interval", 2023-07-14T11:53:53.199 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.199 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.199 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.199 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:53.199 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.200 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.200 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.200 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to perform a host check", 2023-07-14T11:53:53.200 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.201 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.201 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.201 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.204 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.204 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.204 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.204 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.204 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.205 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.205 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.205 INFO:teuthology.orchestra.run.smithi161.stdout: "log_refresh_metadata": { 2023-07-14T11:53:53.205 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_refresh_metadata", 2023-07-14T11:53:53.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.206 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.206 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.206 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.206 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.206 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.207 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.207 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T11:53:53.207 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.207 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.208 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.208 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.208 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.209 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.209 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.209 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.209 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.209 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.209 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.210 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T11:53:53.210 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.210 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.210 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.211 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.211 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.211 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.211 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.211 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.212 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.212 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.212 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.212 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.212 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.213 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.213 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.213 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.213 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.213 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.214 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.214 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.214 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.214 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.214 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.215 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.215 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.215 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.215 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.216 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.216 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.216 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.216 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.216 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.217 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.217 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.217 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.217 INFO:teuthology.orchestra.run.smithi161.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T11:53:53.218 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T11:53:53.218 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.218 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.218 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.218 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.219 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.219 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.219 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.219 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T11:53:53.219 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.219 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.220 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.220 INFO:teuthology.orchestra.run.smithi161.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T11:53:53.220 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T11:53:53.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.221 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.221 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.221 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "*", 2023-07-14T11:53:53.221 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.221 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.222 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.222 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T11:53:53.222 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.222 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.222 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.223 INFO:teuthology.orchestra.run.smithi161.stdout: "max_count_per_host": { 2023-07-14T11:53:53.223 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_count_per_host", 2023-07-14T11:53:53.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.223 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.224 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.224 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.224 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.224 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.224 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.225 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "max number of daemons per service per host", 2023-07-14T11:53:53.225 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.225 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.225 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.225 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd_draining_count": { 2023-07-14T11:53:53.226 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_osd_draining_count", 2023-07-14T11:53:53.226 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.226 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.226 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.226 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.227 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.227 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.227 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.227 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T11:53:53.227 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.228 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.228 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.228 INFO:teuthology.orchestra.run.smithi161.stdout: "migration_current": { 2023-07-14T11:53:53.228 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "migration_current", 2023-07-14T11:53:53.229 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.229 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.229 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.229 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.229 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.230 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.230 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.230 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "internal - do not modify", 2023-07-14T11:53:53.230 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.230 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.231 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.231 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": { 2023-07-14T11:53:53.231 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mode", 2023-07-14T11:53:53.231 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.231 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.232 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.232 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "root", 2023-07-14T11:53:53.232 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.232 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.232 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.233 INFO:teuthology.orchestra.run.smithi161.stdout: "cephadm-package", 2023-07-14T11:53:53.233 INFO:teuthology.orchestra.run.smithi161.stdout: "root" 2023-07-14T11:53:53.233 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.233 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T11:53:53.233 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.234 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.234 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.234 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_alerts_path": { 2023-07-14T11:53:53.234 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_alerts_path", 2023-07-14T11:53:53.235 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.235 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.235 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.235 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T11:53:53.235 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.236 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.236 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.236 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T11:53:53.236 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.236 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.237 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.237 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_web_password": { 2023-07-14T11:53:53.237 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_web_password", 2023-07-14T11:53:53.237 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.237 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.238 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.238 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:53.238 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.238 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.238 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.239 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus web password", 2023-07-14T11:53:53.239 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.239 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.239 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.240 INFO:teuthology.orchestra.run.smithi161.stdout: "prometheus_web_user": { 2023-07-14T11:53:53.240 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus_web_user", 2023-07-14T11:53:53.240 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.240 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.240 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.241 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:53.241 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.241 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.241 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.241 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Prometheus web user", 2023-07-14T11:53:53.241 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.242 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.242 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.242 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_insecure": { 2023-07-14T11:53:53.242 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_insecure", 2023-07-14T11:53:53.243 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.243 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.243 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.243 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.243 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.244 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.244 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.244 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T11:53:53.244 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.244 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.245 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.245 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_password": { 2023-07-14T11:53:53.245 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_password", 2023-07-14T11:53:53.245 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.245 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.246 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.246 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.246 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.246 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.246 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.247 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T11:53:53.247 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.247 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.247 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.248 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_url": { 2023-07-14T11:53:53.248 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_url", 2023-07-14T11:53:53.248 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.248 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.248 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.249 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.249 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.249 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.249 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.249 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T11:53:53.250 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.250 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.250 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.250 INFO:teuthology.orchestra.run.smithi161.stdout: "registry_username": { 2023-07-14T11:53:53.251 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "registry_username", 2023-07-14T11:53:53.251 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.251 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.251 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.251 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.252 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.252 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.252 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.252 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T11:53:53.252 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.252 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.253 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.253 INFO:teuthology.orchestra.run.smithi161.stdout: "secure_monitoring_stack": { 2023-07-14T11:53:53.253 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "secure_monitoring_stack", 2023-07-14T11:53:53.253 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.254 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.254 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.254 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.254 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.254 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.255 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.255 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T11:53:53.255 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.255 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.255 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.256 INFO:teuthology.orchestra.run.smithi161.stdout: "service_discovery_port": { 2023-07-14T11:53:53.256 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "service_discovery_port", 2023-07-14T11:53:53.256 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.256 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.257 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.257 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8765", 2023-07-14T11:53:53.257 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.257 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.257 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.258 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "cephadm service discovery port", 2023-07-14T11:53:53.258 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.258 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.258 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.259 INFO:teuthology.orchestra.run.smithi161.stdout: "ssh_config_file": { 2023-07-14T11:53:53.259 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssh_config_file", 2023-07-14T11:53:53.259 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.259 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.259 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.259 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.260 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.260 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.260 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.260 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T11:53:53.260 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.261 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.261 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.261 INFO:teuthology.orchestra.run.smithi161.stdout: "use_agent": { 2023-07-14T11:53:53.261 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "use_agent", 2023-07-14T11:53:53.262 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.262 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.262 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.262 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.262 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.263 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.263 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.263 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T11:53:53.263 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.263 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.263 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.264 INFO:teuthology.orchestra.run.smithi161.stdout: "use_repo_digest": { 2023-07-14T11:53:53.264 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "use_repo_digest", 2023-07-14T11:53:53.264 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.264 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.265 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.265 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.265 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.265 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.265 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.266 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T11:53:53.266 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.266 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.266 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.267 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_failed_host_check": { 2023-07-14T11:53:53.267 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_failed_host_check", 2023-07-14T11:53:53.267 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.267 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.267 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.268 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.268 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.268 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.268 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.268 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T11:53:53.269 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.269 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.269 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.269 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.269 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_stray_daemons": { 2023-07-14T11:53:53.270 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_stray_daemons", 2023-07-14T11:53:53.270 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.270 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.270 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.270 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.271 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.271 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.271 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.271 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T11:53:53.271 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.271 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.272 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.272 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_on_stray_hosts": { 2023-07-14T11:53:53.272 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_on_stray_hosts", 2023-07-14T11:53:53.272 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.273 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.273 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.273 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.273 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.273 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.274 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.274 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T11:53:53.274 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.274 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.274 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.275 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.275 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.275 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crash", 2023-07-14T11:53:53.276 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.276 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.276 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.276 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.276 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.277 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.277 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.277 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.277 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.277 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.277 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.278 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.278 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.278 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.278 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.278 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.279 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.279 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.279 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.279 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.279 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.280 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.280 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.280 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.280 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.281 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.281 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.281 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.281 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.281 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.282 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.282 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.282 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.282 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.282 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.283 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.283 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.283 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.284 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.284 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.284 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.285 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.285 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.285 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.285 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.285 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.286 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.286 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.286 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.286 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.286 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.287 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.287 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.287 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.287 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.287 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.288 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.288 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.288 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.288 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.288 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.289 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.289 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.289 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.289 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.289 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.290 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.290 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.290 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.290 INFO:teuthology.orchestra.run.smithi161.stdout: "retain_interval": { 2023-07-14T11:53:53.291 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "retain_interval", 2023-07-14T11:53:53.291 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.291 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.291 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.291 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "31536000", 2023-07-14T11:53:53.292 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.292 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.292 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.292 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T11:53:53.293 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.293 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.293 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.293 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_recent_interval": { 2023-07-14T11:53:53.294 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_recent_interval", 2023-07-14T11:53:53.294 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.294 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.294 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.294 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1209600", 2023-07-14T11:53:53.295 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.295 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.295 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.295 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T11:53:53.295 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.296 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.296 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.296 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.296 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.297 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "dashboard", 2023-07-14T11:53:53.297 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.297 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.297 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.298 INFO:teuthology.orchestra.run.smithi161.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T11:53:53.298 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T11:53:53.298 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.298 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.298 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.299 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.299 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.299 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.299 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.299 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.300 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.300 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.300 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.300 INFO:teuthology.orchestra.run.smithi161.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T11:53:53.301 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T11:53:53.301 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.301 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.301 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.301 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.302 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.302 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.302 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.302 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.302 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.303 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.303 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.303 INFO:teuthology.orchestra.run.smithi161.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T11:53:53.303 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T11:53:53.304 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.304 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.304 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.304 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.304 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.305 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.305 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.305 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.305 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.305 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.306 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.306 INFO:teuthology.orchestra.run.smithi161.stdout: "AUDIT_API_ENABLED": { 2023-07-14T11:53:53.306 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T11:53:53.306 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.306 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.307 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.307 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.307 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.307 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.307 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.308 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.308 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.308 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.308 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.309 INFO:teuthology.orchestra.run.smithi161.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T11:53:53.309 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T11:53:53.309 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.309 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.309 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.310 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.310 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.310 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.310 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.310 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.311 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.311 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.311 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.311 INFO:teuthology.orchestra.run.smithi161.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T11:53:53.311 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T11:53:53.312 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.312 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.312 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.312 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.312 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.313 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.313 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.313 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.313 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.313 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.314 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.314 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T11:53:53.314 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T11:53:53.314 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.315 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.315 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.315 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.315 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.315 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.316 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.316 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.316 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.316 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.316 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.317 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T11:53:53.317 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T11:53:53.317 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.317 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.317 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.318 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.318 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.318 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.318 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.318 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.319 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.319 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.319 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.319 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T11:53:53.320 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T11:53:53.320 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.320 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.320 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.320 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.321 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.321 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.321 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.321 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.321 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.322 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.322 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.322 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T11:53:53.322 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T11:53:53.323 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.323 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.323 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.323 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.323 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.324 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.324 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.324 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.324 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.324 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.325 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.325 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T11:53:53.325 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T11:53:53.325 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.325 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T11:53:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T11:53:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T11:53:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T11:53:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T11:53:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T11:53:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T11:53:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T11:53:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T11:53:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T11:53:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_URL": { 2023-07-14T11:53:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_URL", 2023-07-14T11:53:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T11:53:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T11:53:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T11:53:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T11:53:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T11:53:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T11:53:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T11:53:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T11:53:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T11:53:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T11:53:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T11:53:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T11:53:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T11:53:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T11:53:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T11:53:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T11:53:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T11:53:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T11:53:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T11:53:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T11:53:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T11:53:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T11:53:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T11:53:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T11:53:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T11:53:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T11:53:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T11:53:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T11:53:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T11:53:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T11:53:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.385 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.385 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.385 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.385 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.385 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.386 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.386 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T11:53:53.386 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T11:53:53.386 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.386 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.387 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.387 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T11:53:53.387 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.387 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.387 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T11:53:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T11:53:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.391 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.391 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.391 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.391 INFO:teuthology.orchestra.run.smithi161.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T11:53:53.392 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T11:53:53.392 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.392 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.392 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.392 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8", 2023-07-14T11:53:53.393 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.393 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.393 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.393 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.393 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.394 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.394 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.394 INFO:teuthology.orchestra.run.smithi161.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T11:53:53.394 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T11:53:53.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.395 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.395 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.395 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "45", 2023-07-14T11:53:53.395 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.396 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.396 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.396 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.396 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.396 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.397 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.397 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T11:53:53.397 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T11:53:53.397 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.398 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.398 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.398 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.398 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.398 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.399 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.399 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.399 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.399 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.399 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.400 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T11:53:53.400 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T11:53:53.400 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.400 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.401 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.401 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "admin", 2023-07-14T11:53:53.401 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.401 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.402 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.402 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.402 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.402 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.402 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi161.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.408 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.408 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.408 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.408 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T11:53:53.409 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T11:53:53.409 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.409 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.409 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.409 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:53.409 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.410 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.410 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.410 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.410 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.410 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.411 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.411 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T11:53:53.411 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T11:53:53.411 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.413 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.413 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.413 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.413 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.413 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.414 INFO:teuthology.orchestra.run.smithi161.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T11:53:53.414 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T11:53:53.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.414 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.416 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.416 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.416 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.416 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi161.stdout: "cross_origin_url": { 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cross_origin_url", 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi161.stdout: "crt_file": { 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "crt_file", 2023-07-14T11:53:53.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.420 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.420 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.420 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.420 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.421 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.421 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.421 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.421 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.421 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.422 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.422 INFO:teuthology.orchestra.run.smithi161.stdout: "debug": { 2023-07-14T11:53:53.422 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "debug", 2023-07-14T11:53:53.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.422 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.423 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.423 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.423 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.423 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.423 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.424 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Enable/disable debug options", 2023-07-14T11:53:53.424 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.424 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.424 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.425 INFO:teuthology.orchestra.run.smithi161.stdout: "jwt_token_ttl": { 2023-07-14T11:53:53.425 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "jwt_token_ttl", 2023-07-14T11:53:53.425 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.425 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.425 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.426 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "28800", 2023-07-14T11:53:53.426 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.426 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.426 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.426 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.427 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.427 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.427 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.427 INFO:teuthology.orchestra.run.smithi161.stdout: "key_file": { 2023-07-14T11:53:53.428 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "key_file", 2023-07-14T11:53:53.428 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.428 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.428 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.428 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.429 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.429 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.429 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.429 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.429 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.430 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.430 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.430 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.430 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.431 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.431 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.431 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.431 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.431 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.431 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.432 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.432 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.432 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.432 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.432 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.433 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.433 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.433 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.433 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.433 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.434 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.434 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.434 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.434 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.435 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.435 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.435 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.435 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.435 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.436 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.436 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.436 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.436 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.436 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.437 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.437 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.437 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.437 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.437 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "motd": { 2023-07-14T11:53:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "motd", 2023-07-14T11:53:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "The message of the day", 2023-07-14T11:53:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "redirect_resolve_ip_addr": { 2023-07-14T11:53:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T11:53:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "::", 2023-07-14T11:53:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8080", 2023-07-14T11:53:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl": { 2023-07-14T11:53:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl", 2023-07-14T11:53:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl_server_port": { 2023-07-14T11:53:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl_server_port", 2023-07-14T11:53:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8443", 2023-07-14T11:53:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_behaviour": { 2023-07-14T11:53:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_behaviour", 2023-07-14T11:53:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "redirect", 2023-07-14T11:53:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "redirect" 2023-07-14T11:53:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_error_status_code": { 2023-07-14T11:53:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_error_status_code", 2023-07-14T11:53:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "500", 2023-07-14T11:53:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "400", 2023-07-14T11:53:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "599", 2023-07-14T11:53:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "url_prefix": { 2023-07-14T11:53:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "url_prefix", 2023-07-14T11:53:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "devicehealth", 2023-07-14T11:53:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "enable_monitoring": { 2023-07-14T11:53:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enable_monitoring", 2023-07-14T11:53:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "monitor device health metrics", 2023-07-14T11:53:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "mark_out_threshold": { 2023-07-14T11:53:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mark_out_threshold", 2023-07-14T11:53:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "2419200", 2023-07-14T11:53:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T11:53:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": { 2023-07-14T11:53:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pool_name", 2023-07-14T11:53:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "device_health_metrics", 2023-07-14T11:53:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T11:53:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "retention_period": { 2023-07-14T11:53:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "retention_period", 2023-07-14T11:53:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15552000", 2023-07-14T11:53:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long to retain device health metrics", 2023-07-14T11:53:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "scrape_frequency": { 2023-07-14T11:53:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "scrape_frequency", 2023-07-14T11:53:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "86400", 2023-07-14T11:53:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T11:53:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "self_heal": { 2023-07-14T11:53:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "self_heal", 2023-07-14T11:53:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T11:53:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T11:53:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "warn_threshold": { 2023-07-14T11:53:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "warn_threshold", 2023-07-14T11:53:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "7257600", 2023-07-14T11:53:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T11:53:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "diskprediction_local", 2023-07-14T11:53:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "predict_interval": { 2023-07-14T11:53:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "predict_interval", 2023-07-14T11:53:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "86400", 2023-07-14T11:53:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "predictor_model": { 2023-07-14T11:53:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "predictor_model", 2023-07-14T11:53:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "prophetstor", 2023-07-14T11:53:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "600", 2023-07-14T11:53:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "influx", 2023-07-14T11:53:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": false, 2023-07-14T11:53:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "influxdb python module not found", 2023-07-14T11:53:53.531 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.531 INFO:teuthology.orchestra.run.smithi161.stdout: "batch_size": { 2023-07-14T11:53:53.531 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "batch_size", 2023-07-14T11:53:53.531 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.531 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.532 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.532 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5000", 2023-07-14T11:53:53.532 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.532 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.532 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.533 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T11:53:53.533 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.533 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.533 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.534 INFO:teuthology.orchestra.run.smithi161.stdout: "database": { 2023-07-14T11:53:53.534 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "database", 2023-07-14T11:53:53.534 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.534 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.534 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.535 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "ceph", 2023-07-14T11:53:53.535 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.535 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.535 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.535 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T11:53:53.535 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.536 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.536 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.536 INFO:teuthology.orchestra.run.smithi161.stdout: "hostname": { 2023-07-14T11:53:53.536 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "hostname", 2023-07-14T11:53:53.537 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.537 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.537 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.537 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.537 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.538 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.538 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.538 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB server hostname", 2023-07-14T11:53:53.538 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.538 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.540 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.540 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "30", 2023-07-14T11:53:53.540 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "5", 2023-07-14T11:53:53.540 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.540 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.541 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T11:53:53.541 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.541 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.541 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.542 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.542 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.542 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.542 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.542 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.543 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.543 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.543 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.543 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.543 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.543 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.544 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.544 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.544 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.544 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.544 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.545 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.545 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.545 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.545 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.546 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.546 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.546 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.546 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.546 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.547 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.547 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.547 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.547 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.547 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.547 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.548 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.548 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.548 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.548 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.549 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.549 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.549 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.549 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.550 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.550 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.550 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.550 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.550 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.551 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.551 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.551 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.551 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.551 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.552 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.552 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.552 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.552 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.552 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.553 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.553 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.553 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.553 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.553 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.554 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.554 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.554 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.554 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.554 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.555 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.555 INFO:teuthology.orchestra.run.smithi161.stdout: "password": { 2023-07-14T11:53:53.555 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "password", 2023-07-14T11:53:53.555 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.556 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.556 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.556 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.556 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.556 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.557 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.557 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "password of InfluxDB server user", 2023-07-14T11:53:53.557 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.557 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.557 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.558 INFO:teuthology.orchestra.run.smithi161.stdout: "port": { 2023-07-14T11:53:53.558 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "port", 2023-07-14T11:53:53.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.558 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.559 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.559 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "8086", 2023-07-14T11:53:53.559 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.560 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.560 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.560 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "InfluxDB server port", 2023-07-14T11:53:53.560 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.560 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.561 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.561 INFO:teuthology.orchestra.run.smithi161.stdout: "ssl": { 2023-07-14T11:53:53.561 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ssl", 2023-07-14T11:53:53.561 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.562 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.562 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.562 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "false", 2023-07-14T11:53:53.562 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.562 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.563 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.563 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T11:53:53.563 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.563 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.564 INFO:teuthology.orchestra.run.smithi161.stdout: "threads": { 2023-07-14T11:53:53.564 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "threads", 2023-07-14T11:53:53.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.564 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.564 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.565 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:53.565 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:53.565 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "32", 2023-07-14T11:53:53.565 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.565 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T11:53:53.566 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.566 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.566 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.566 INFO:teuthology.orchestra.run.smithi161.stdout: "username": { 2023-07-14T11:53:53.567 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "username", 2023-07-14T11:53:53.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.567 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.567 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.568 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.568 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.568 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.568 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "username of InfluxDB server user", 2023-07-14T11:53:53.568 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.569 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.569 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.569 INFO:teuthology.orchestra.run.smithi161.stdout: "verify_ssl": { 2023-07-14T11:53:53.569 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "verify_ssl", 2023-07-14T11:53:53.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.570 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.570 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.570 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "true", 2023-07-14T11:53:53.570 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.571 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.571 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.571 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T11:53:53.571 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.571 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.571 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.572 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "insights", 2023-07-14T11:53:53.573 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.573 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.573 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.573 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.573 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.574 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.574 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.574 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.574 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.575 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.575 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.575 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.575 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.575 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.575 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.576 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.576 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.576 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.576 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.576 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.577 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.577 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.577 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.577 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.578 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.578 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.578 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.578 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.578 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.579 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.579 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.579 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.579 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.579 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.580 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.580 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.580 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.581 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.581 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.581 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.581 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.581 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.581 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.582 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.582 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.582 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.582 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.582 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.583 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.583 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.583 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.583 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.583 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.584 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.584 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.584 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.585 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.585 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.585 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.585 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.585 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.586 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.586 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.586 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.586 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.586 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.586 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.587 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "iostat", 2023-07-14T11:53:53.587 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.588 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.588 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.588 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.588 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.589 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.589 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.589 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.589 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.589 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.590 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.590 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.590 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.590 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.590 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.590 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.591 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.591 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.591 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.591 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.592 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.592 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.592 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.592 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.593 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.593 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.593 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.593 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.593 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.593 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.594 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.594 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.594 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.594 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.594 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.595 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.595 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.595 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.596 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.596 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.596 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.596 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.596 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.597 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.597 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.597 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.597 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.597 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.598 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.598 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.598 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.598 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.598 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.599 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.599 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.599 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.599 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.599 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.600 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.600 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.600 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.600 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.600 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "k8sevents", 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_event_retention_days": { 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "ceph_event_retention_days", 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.604 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.604 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.604 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "7", 2023-07-14T11:53:53.604 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.604 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.604 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.605 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T11:53:53.605 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.605 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.605 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.606 INFO:teuthology.orchestra.run.smithi161.stdout: "config_check_secs": { 2023-07-14T11:53:53.606 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "config_check_secs", 2023-07-14T11:53:53.606 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.606 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.606 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.607 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.607 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "10", 2023-07-14T11:53:53.607 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.607 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.607 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T11:53:53.608 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.608 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.608 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.608 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.609 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.609 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.609 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.609 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.610 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.610 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.610 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.610 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.610 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.611 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.611 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.611 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.611 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.611 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.611 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.612 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.612 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.612 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.612 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.613 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.613 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.613 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.613 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.613 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.614 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.614 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.614 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.614 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.614 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.615 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.615 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.615 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.615 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.616 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.616 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.616 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.616 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.616 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.617 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.617 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.617 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.617 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.617 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.618 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.618 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.618 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.618 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.618 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.619 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.619 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.619 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.619 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.620 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.620 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.620 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.620 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.620 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.621 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.621 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.621 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.621 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.621 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.622 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.622 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.623 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "localpool", 2023-07-14T11:53:53.623 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.623 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.623 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.623 INFO:teuthology.orchestra.run.smithi161.stdout: "failure_domain": { 2023-07-14T11:53:53.624 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "failure_domain", 2023-07-14T11:53:53.624 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.624 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.624 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.624 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "host", 2023-07-14T11:53:53.625 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.625 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.625 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.625 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "failure domain for any created local pool", 2023-07-14T11:53:53.625 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T11:53:53.626 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.626 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.626 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.626 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.627 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.627 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.627 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.627 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.627 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.628 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.628 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.628 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.628 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.628 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.629 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.629 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.629 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.629 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.629 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.629 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.630 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.630 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.630 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.630 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.631 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.631 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.631 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.631 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.631 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.632 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.632 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.632 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.632 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.632 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.633 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.633 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.633 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.633 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.634 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.634 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.634 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.634 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.634 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.635 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.635 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.635 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.635 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.635 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.636 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.636 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.636 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.636 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.636 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.637 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.637 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.637 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.637 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.638 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.638 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.638 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.638 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.638 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.639 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.639 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.639 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.639 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.639 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.640 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": { 2023-07-14T11:53:53.640 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "min_size", 2023-07-14T11:53:53.640 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.640 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.641 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.641 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.641 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.641 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.642 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default min_size for any created local pool", 2023-07-14T11:53:53.642 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T11:53:53.642 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.642 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.643 INFO:teuthology.orchestra.run.smithi161.stdout: "num_rep": { 2023-07-14T11:53:53.643 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "num_rep", 2023-07-14T11:53:53.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.643 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.644 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3", 2023-07-14T11:53:53.644 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.644 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.644 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.644 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default replica count for any created local pool", 2023-07-14T11:53:53.645 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.645 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.645 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.645 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": { 2023-07-14T11:53:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pg_num", 2023-07-14T11:53:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "128", 2023-07-14T11:53:53.647 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.647 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.647 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.647 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T11:53:53.647 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.647 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.648 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.648 INFO:teuthology.orchestra.run.smithi161.stdout: "prefix": { 2023-07-14T11:53:53.648 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prefix", 2023-07-14T11:53:53.648 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.649 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.649 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.649 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.649 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.649 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.650 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.650 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "name prefix for any created local pool", 2023-07-14T11:53:53.650 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.650 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.650 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.651 INFO:teuthology.orchestra.run.smithi161.stdout: "subtree": { 2023-07-14T11:53:53.651 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "subtree", 2023-07-14T11:53:53.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.651 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.652 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.652 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "rack", 2023-07-14T11:53:53.652 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.652 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.652 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.653 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T11:53:53.653 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T11:53:53.653 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.653 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.653 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.654 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mds_autoscaler", 2023-07-14T11:53:53.654 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.654 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.655 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.655 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.655 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.655 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.655 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.656 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.656 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.656 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.656 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.656 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.657 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.657 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.657 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.657 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.657 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.658 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.658 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.658 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.658 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.658 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.659 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.659 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.659 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.659 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.660 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.660 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.660 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.660 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.660 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.661 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.661 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.661 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.661 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.662 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.662 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.662 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.662 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.662 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.663 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.663 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.663 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.663 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.663 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.664 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.664 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.664 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.664 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.664 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.665 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.665 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.665 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.665 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.665 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.666 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.666 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.666 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.666 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.667 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.667 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.667 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.667 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.667 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.668 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.668 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.668 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.668 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.669 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mirroring", 2023-07-14T11:53:53.669 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.669 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.670 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.670 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.670 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.670 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.671 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.671 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.671 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.671 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.671 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.671 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.672 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.672 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.672 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.672 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.672 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.673 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.673 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.673 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.673 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.673 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.674 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.674 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.674 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.675 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.675 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.675 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.675 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.676 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.676 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.676 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.676 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.676 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.677 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.677 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.677 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.677 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.678 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.678 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.678 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.678 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.678 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.679 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.679 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.679 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.679 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.679 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.680 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.680 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.680 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.680 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.680 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.681 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.681 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.681 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.681 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.682 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.682 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.682 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.682 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.682 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.683 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.683 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.683 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.683 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.684 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.684 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "nfs", 2023-07-14T11:53:53.684 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.685 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.685 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.685 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.685 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.685 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.686 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.686 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.686 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.686 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.686 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.687 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.687 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.687 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.687 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.687 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.688 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.688 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.688 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.688 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.688 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.689 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.689 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.689 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.689 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.689 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.690 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.690 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.690 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.690 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.690 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.691 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.691 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.691 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.691 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.692 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.692 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.692 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.692 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.692 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.693 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.693 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.693 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.693 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.693 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.693 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.694 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.694 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.694 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.694 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.694 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.695 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.695 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.695 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.695 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.696 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.696 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.696 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.696 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.696 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.697 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.697 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.697 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.697 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.697 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.698 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.698 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.698 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.698 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.698 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.699 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "orchestrator", 2023-07-14T11:53:53.699 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.699 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.699 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.700 INFO:teuthology.orchestra.run.smithi161.stdout: "fail_fs": { 2023-07-14T11:53:53.700 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "fail_fs", 2023-07-14T11:53:53.700 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.700 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.700 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.701 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.701 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.701 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.701 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.701 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T11:53:53.702 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.702 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.702 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.702 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.702 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.703 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.703 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.703 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.703 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.703 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.704 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.704 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.704 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.704 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.704 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.705 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.705 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.705 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.705 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.705 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.706 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.706 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.706 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.706 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.706 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.707 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.707 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.707 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.707 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.707 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.708 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.708 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.708 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.708 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.708 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.709 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.709 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.709 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.709 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.710 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.710 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.710 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.710 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.710 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.711 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.711 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.711 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.711 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.711 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.711 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.712 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.712 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.712 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.712 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.712 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.713 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.713 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.713 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.713 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.714 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.714 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.714 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.714 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.714 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.715 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.715 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.715 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.715 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.715 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.716 INFO:teuthology.orchestra.run.smithi161.stdout: "orchestrator": { 2023-07-14T11:53:53.716 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "orchestrator", 2023-07-14T11:53:53.716 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.716 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.717 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.717 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.717 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.717 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.717 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.718 INFO:teuthology.orchestra.run.smithi161.stdout: "cephadm", 2023-07-14T11:53:53.718 INFO:teuthology.orchestra.run.smithi161.stdout: "rook", 2023-07-14T11:53:53.718 INFO:teuthology.orchestra.run.smithi161.stdout: "test_orchestrator" 2023-07-14T11:53:53.718 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.718 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Orchestrator backend", 2023-07-14T11:53:53.719 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.719 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.719 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.719 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.719 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.720 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "osd_perf_query", 2023-07-14T11:53:53.720 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.720 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.721 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.721 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.721 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.721 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.721 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.722 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.722 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.722 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.722 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.722 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.723 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.723 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.723 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.723 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.723 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.723 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.724 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.724 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.724 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.724 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.724 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.725 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.725 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.725 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.725 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.726 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.726 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.726 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.726 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.726 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.727 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.727 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.727 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.727 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.728 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.728 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.728 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.728 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.728 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.728 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.729 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.729 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.729 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.729 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.729 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.730 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.730 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.730 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.730 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.730 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.731 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.731 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.731 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.731 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.732 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.732 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.732 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.732 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.732 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.733 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.733 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.733 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.733 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.733 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.734 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.734 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.734 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.734 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.734 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.735 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "osd_support", 2023-07-14T11:53:53.735 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.735 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.735 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.736 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.736 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.736 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.736 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.736 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.737 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.737 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.737 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.737 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.738 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.738 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.738 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.738 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.738 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.738 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.739 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.739 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.739 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.739 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.739 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.740 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.740 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.740 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.740 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.741 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.741 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.741 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.741 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.741 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.742 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.742 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.742 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.742 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.743 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.743 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.743 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.743 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.743 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.744 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.744 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.744 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.744 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.744 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.745 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.745 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.745 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.745 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.745 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.746 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.746 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.746 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.746 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.746 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.747 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.747 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.747 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.747 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.747 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.748 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.748 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.748 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.748 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.748 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.749 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.749 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.749 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.749 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.749 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.750 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "pg_autoscaler", 2023-07-14T11:53:53.750 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.750 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.751 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.751 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.751 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.751 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.751 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.751 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.752 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.752 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.752 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.752 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.752 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.753 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.753 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.753 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.753 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.753 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.754 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.754 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.754 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.754 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.754 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.755 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.755 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.755 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.755 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.755 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.756 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.756 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.756 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.756 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.756 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.757 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.757 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.757 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.757 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.758 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.758 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.758 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.758 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.758 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.759 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.759 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.759 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.759 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.759 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.760 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.760 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.760 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.760 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.760 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.761 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.761 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.761 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.761 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.762 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.762 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.762 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.762 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.762 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.762 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.763 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.763 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.763 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.763 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.763 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.764 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.764 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.764 INFO:teuthology.orchestra.run.smithi161.stdout: "noautoscale": { 2023-07-14T11:53:53.764 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "noautoscale", 2023-07-14T11:53:53.765 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.765 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.765 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.765 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.765 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.766 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.766 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.766 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "global autoscale flag", 2023-07-14T11:53:53.766 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T11:53:53.766 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.767 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.767 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:53.767 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:53.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.767 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.768 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.768 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:53.768 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.768 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.768 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.769 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.769 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.769 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.769 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.770 INFO:teuthology.orchestra.run.smithi161.stdout: "threshold": { 2023-07-14T11:53:53.770 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "threshold", 2023-07-14T11:53:53.770 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.770 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.770 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.771 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "3.0", 2023-07-14T11:53:53.771 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1.0", 2023-07-14T11:53:53.771 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.771 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.771 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "scaling threshold", 2023-07-14T11:53:53.771 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T11:53:53.772 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.772 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.772 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.772 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.773 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "progress", 2023-07-14T11:53:53.773 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.773 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.773 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.774 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_pg_recovery_event": { 2023-07-14T11:53:53.774 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_pg_recovery_event", 2023-07-14T11:53:53.774 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.774 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.774 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.775 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.775 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.775 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.775 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.775 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T11:53:53.775 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.776 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.776 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.776 INFO:teuthology.orchestra.run.smithi161.stdout: "enabled": { 2023-07-14T11:53:53.776 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enabled", 2023-07-14T11:53:53.777 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.777 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.777 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.777 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.778 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.778 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.778 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.778 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.779 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.779 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.779 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.779 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.779 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.780 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.780 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.780 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.780 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.780 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.781 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.781 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.781 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.781 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.781 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.782 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.782 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.782 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.782 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.782 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.783 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.783 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.783 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.783 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.784 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.784 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.784 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.784 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.785 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.785 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.785 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.785 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.785 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.786 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.786 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.786 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.786 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.787 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.787 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.787 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.787 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.787 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.788 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.788 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.788 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.788 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.788 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.789 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.789 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.789 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.789 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.789 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.789 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.790 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.790 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.790 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.790 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.791 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.791 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.791 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.791 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.792 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.792 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.792 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.792 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.792 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.793 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.793 INFO:teuthology.orchestra.run.smithi161.stdout: "max_completed_events": { 2023-07-14T11:53:53.793 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_completed_events", 2023-07-14T11:53:53.793 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.794 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.794 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.794 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "50", 2023-07-14T11:53:53.794 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.794 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.794 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.795 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "number of past completed events to remember", 2023-07-14T11:53:53.795 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.795 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.795 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.796 INFO:teuthology.orchestra.run.smithi161.stdout: "sleep_interval": { 2023-07-14T11:53:53.796 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "sleep_interval", 2023-07-14T11:53:53.796 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:53.796 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.796 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.797 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "5", 2023-07-14T11:53:53.797 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.797 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.797 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.797 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "how long the module is going to sleep", 2023-07-14T11:53:53.798 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.798 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.798 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.799 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "prometheus", 2023-07-14T11:53:53.799 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.799 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.799 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.800 INFO:teuthology.orchestra.run.smithi161.stdout: "cache": { 2023-07-14T11:53:53.800 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cache", 2023-07-14T11:53:53.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.800 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.800 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.801 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.801 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.801 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.801 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.801 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.802 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.802 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.802 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.802 INFO:teuthology.orchestra.run.smithi161.stdout: "exclude_perf_counters": { 2023-07-14T11:53:53.803 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "exclude_perf_counters", 2023-07-14T11:53:53.803 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.803 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.803 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.803 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.804 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.804 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.804 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.804 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T11:53:53.805 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T11:53:53.805 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.805 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.805 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.805 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.806 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.806 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.806 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.806 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.807 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.807 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.807 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.807 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.807 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.808 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.808 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.808 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.808 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.808 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.810 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.810 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.810 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.810 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.810 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.815 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.815 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.815 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.815 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.815 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.816 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.816 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.816 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.816 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.816 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.817 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.817 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.817 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.817 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.817 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.818 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.818 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.818 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.818 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.818 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_stats_pools": { 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_stats_pools", 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "300", 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi161.stdout: "scrape_interval": { 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "scrape_interval", 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15.0", 2023-07-14T11:53:53.826 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.826 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.826 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.826 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.826 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.827 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.827 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.827 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:53.827 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "::", 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.831 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.831 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "9283", 2023-07-14T11:53:53.831 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.831 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.831 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.832 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T11:53:53.832 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.832 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.832 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.833 INFO:teuthology.orchestra.run.smithi161.stdout: "stale_cache_strategy": { 2023-07-14T11:53:53.833 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "stale_cache_strategy", 2023-07-14T11:53:53.833 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.833 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.833 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.834 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "log", 2023-07-14T11:53:53.834 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.834 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.834 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.834 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.835 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.835 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.835 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.835 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_behaviour": { 2023-07-14T11:53:53.836 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_behaviour", 2023-07-14T11:53:53.836 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.836 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.836 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.836 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "default", 2023-07-14T11:53:53.837 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.837 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.837 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.837 INFO:teuthology.orchestra.run.smithi161.stdout: "default", 2023-07-14T11:53:53.837 INFO:teuthology.orchestra.run.smithi161.stdout: "error" 2023-07-14T11:53:53.838 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.838 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.838 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.838 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.838 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.839 INFO:teuthology.orchestra.run.smithi161.stdout: "standby_error_status_code": { 2023-07-14T11:53:53.839 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "standby_error_status_code", 2023-07-14T11:53:53.839 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.839 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.839 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.840 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "500", 2023-07-14T11:53:53.840 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "400", 2023-07-14T11:53:53.840 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "599", 2023-07-14T11:53:53.840 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.840 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.841 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.841 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.841 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.841 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.841 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.842 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_support", 2023-07-14T11:53:53.842 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.842 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.843 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.843 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.843 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.843 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.843 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.843 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.844 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.844 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.844 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.844 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.844 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.845 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.845 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.845 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.845 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.845 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.846 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.846 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.846 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.846 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.846 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.847 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.847 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.847 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.847 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.848 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.848 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.848 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.848 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.848 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.849 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.849 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.849 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.849 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.849 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.850 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.850 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.850 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.850 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.850 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.851 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.851 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.851 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.851 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.851 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.852 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.852 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.852 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.852 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.852 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.853 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.853 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.853 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.853 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.853 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.854 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.854 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.854 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.854 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.854 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.855 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.855 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.855 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.855 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.855 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.856 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.856 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.856 INFO:teuthology.orchestra.run.smithi161.stdout: "max_concurrent_snap_create": { 2023-07-14T11:53:53.856 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_concurrent_snap_create", 2023-07-14T11:53:53.857 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.857 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.857 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.857 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10", 2023-07-14T11:53:53.857 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.858 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.858 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.858 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.858 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.858 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.859 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.859 INFO:teuthology.orchestra.run.smithi161.stdout: "mirror_snapshot_schedule": { 2023-07-14T11:53:53.859 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T11:53:53.859 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.859 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.860 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.860 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.860 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.860 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.860 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.861 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.861 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.861 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.861 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.862 INFO:teuthology.orchestra.run.smithi161.stdout: "trash_purge_schedule": { 2023-07-14T11:53:53.862 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "trash_purge_schedule", 2023-07-14T11:53:53.862 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.862 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.862 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.863 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.863 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.863 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.863 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.863 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.863 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.864 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.864 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.864 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.864 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.865 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "restful", 2023-07-14T11:53:53.865 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.865 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.865 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.866 INFO:teuthology.orchestra.run.smithi161.stdout: "enable_auth": { 2023-07-14T11:53:53.866 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enable_auth", 2023-07-14T11:53:53.866 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.866 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.866 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.867 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.867 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.867 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.867 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.867 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.868 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.868 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.868 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.868 INFO:teuthology.orchestra.run.smithi161.stdout: "key_file": { 2023-07-14T11:53:53.869 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "key_file", 2023-07-14T11:53:53.869 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.869 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.869 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.869 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.870 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.870 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.870 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.870 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.870 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.871 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.871 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.871 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.871 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.871 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.872 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.872 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.872 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.872 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.872 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.873 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.873 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.873 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.873 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.873 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.874 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.874 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.874 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.874 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.874 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.875 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.875 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.875 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.875 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.875 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.876 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.876 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.876 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.876 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.876 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.877 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.877 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.877 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.877 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.877 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.878 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.878 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.878 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.878 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.879 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.879 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.879 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.879 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.879 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.880 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.880 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.880 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.880 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.881 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.881 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.881 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.881 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.881 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.881 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.882 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.882 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.882 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.882 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.883 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.883 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.883 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.883 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.883 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.884 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.884 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.884 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.884 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.884 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.885 INFO:teuthology.orchestra.run.smithi161.stdout: "server_addr": { 2023-07-14T11:53:53.885 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_addr", 2023-07-14T11:53:53.885 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.885 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.886 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.886 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.886 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.886 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.886 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.887 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.887 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.887 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.887 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.888 INFO:teuthology.orchestra.run.smithi161.stdout: "server_port": { 2023-07-14T11:53:53.888 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "server_port", 2023-07-14T11:53:53.888 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.888 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.888 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.889 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.889 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.889 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.889 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.889 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.889 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.890 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.890 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.890 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.890 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.891 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rgw", 2023-07-14T11:53:53.891 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.891 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.891 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.892 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.892 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.892 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.892 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.892 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.893 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.893 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.893 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.893 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.893 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.893 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.894 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.894 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.894 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.894 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.894 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.895 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.895 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.895 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.895 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.896 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.896 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.896 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.896 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.896 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.897 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.897 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.897 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.897 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.897 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.898 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.898 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.898 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.898 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.898 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.899 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.899 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.899 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.899 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.900 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.900 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.900 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.900 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.900 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.901 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.901 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.901 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.901 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.901 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.902 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.902 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.902 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.902 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.903 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.903 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.903 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.903 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.903 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.904 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.904 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.904 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.904 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.904 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.905 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.905 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.906 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rook", 2023-07-14T11:53:53.906 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.906 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.906 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.906 INFO:teuthology.orchestra.run.smithi161.stdout: "drive_group_interval": { 2023-07-14T11:53:53.907 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "drive_group_interval", 2023-07-14T11:53:53.907 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.907 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.907 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.907 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "300.0", 2023-07-14T11:53:53.908 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.908 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.908 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.908 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T11:53:53.908 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.909 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.909 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.909 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.909 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.910 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.910 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.911 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.911 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.912 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.912 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.912 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.913 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.913 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.913 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.914 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.914 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.915 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.915 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.916 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.916 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.916 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.917 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.918 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.918 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.918 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.919 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.919 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.920 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.920 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.921 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.921 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.921 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.922 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.922 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.923 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.923 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.924 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.924 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.925 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.925 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.926 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.926 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.926 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.927 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.927 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.928 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.928 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.928 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.929 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.929 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.930 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.930 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.930 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.931 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.932 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.932 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.933 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.933 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.933 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.934 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.934 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.935 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.935 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.935 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.936 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.936 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.937 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.937 INFO:teuthology.orchestra.run.smithi161.stdout: "storage_class": { 2023-07-14T11:53:53.938 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "storage_class", 2023-07-14T11:53:53.938 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.939 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.939 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.939 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "local", 2023-07-14T11:53:53.940 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.940 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.941 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.941 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T11:53:53.941 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.942 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.942 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.942 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.943 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.944 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "selftest", 2023-07-14T11:53:53.944 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.945 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:53.945 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:53.945 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:53.946 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:53.946 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.946 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.947 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.947 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.948 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.948 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.948 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.949 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.949 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.950 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.950 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.950 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.951 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.951 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.951 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.952 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.952 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.953 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.954 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:53.954 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:53.954 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.955 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.955 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.955 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.955 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.955 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.956 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.956 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.956 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.956 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.956 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.957 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:53.957 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:53.957 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.957 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.958 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.958 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:53.958 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.958 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.963 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.963 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.963 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.963 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.963 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.964 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.964 INFO:teuthology.orchestra.run.smithi161.stdout: "roption1": { 2023-07-14T11:53:53.964 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "roption1", 2023-07-14T11:53:53.965 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.965 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.965 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.965 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.965 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.966 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.966 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.966 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.966 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.966 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.967 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.967 INFO:teuthology.orchestra.run.smithi161.stdout: "roption2": { 2023-07-14T11:53:53.967 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "roption2", 2023-07-14T11:53:53.967 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.968 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.968 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "xyz", 2023-07-14T11:53:53.968 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.968 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.969 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.969 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.969 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.969 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.969 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.970 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption1": { 2023-07-14T11:53:53.970 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption1", 2023-07-14T11:53:53.970 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.970 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.971 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.971 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.971 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.971 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.971 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.972 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.972 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.972 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.972 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.973 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption2": { 2023-07-14T11:53:53.973 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption2", 2023-07-14T11:53:53.973 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.973 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.973 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.974 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.974 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.974 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.974 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.974 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.975 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.975 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.975 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.976 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption3": { 2023-07-14T11:53:53.976 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption3", 2023-07-14T11:53:53.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "float", 2023-07-14T11:53:53.976 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.976 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.977 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.977 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.977 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.977 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.977 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.978 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.978 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.978 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.978 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption4": { 2023-07-14T11:53:53.979 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption4", 2023-07-14T11:53:53.979 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.979 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.979 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.979 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.980 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.980 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.980 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.980 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.980 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.981 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.981 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.981 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption5": { 2023-07-14T11:53:53.981 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption5", 2023-07-14T11:53:53.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.982 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.982 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.982 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.982 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.983 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.983 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.983 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.983 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.983 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.984 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.984 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption6": { 2023-07-14T11:53:53.984 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption6", 2023-07-14T11:53:53.985 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:53.985 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.985 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.985 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:53.985 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.986 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.986 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.986 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.986 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.986 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.987 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.987 INFO:teuthology.orchestra.run.smithi161.stdout: "rwoption7": { 2023-07-14T11:53:53.987 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rwoption7", 2023-07-14T11:53:53.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:53.988 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.988 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.988 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.988 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "1", 2023-07-14T11:53:53.988 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "42", 2023-07-14T11:53:53.989 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.989 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.989 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.989 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.989 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.990 INFO:teuthology.orchestra.run.smithi161.stdout: "testkey": { 2023-07-14T11:53:53.990 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testkey", 2023-07-14T11:53:53.990 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.990 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.991 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.991 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.991 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.991 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.991 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.992 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.992 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.992 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.992 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.993 INFO:teuthology.orchestra.run.smithi161.stdout: "testlkey": { 2023-07-14T11:53:53.993 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testlkey", 2023-07-14T11:53:53.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.993 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.994 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.994 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.994 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.994 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.994 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.995 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.995 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.995 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.995 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.996 INFO:teuthology.orchestra.run.smithi161.stdout: "testnewline": { 2023-07-14T11:53:53.996 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "testnewline", 2023-07-14T11:53:53.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:53.996 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:53.996 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:53.997 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:53.997 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:53.997 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:53.997 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:53.997 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:53.998 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:53.998 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:53.998 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:53.998 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.998 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:53.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:53.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:53.999 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snap_schedule", 2023-07-14T11:53:53.999 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:53.999 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_m_granularity": { 2023-07-14T11:53:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "allow_m_granularity", 2023-07-14T11:53:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.001 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.001 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.001 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.001 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.001 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.002 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.002 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T11:53:54.002 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.002 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.002 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "dump_on_update": { 2023-07-14T11:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "dump_on_update", 2023-07-14T11:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "dump database to debug log on update", 2023-07-14T11:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.017 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.017 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.017 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.017 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.018 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.018 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.018 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.018 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.019 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.019 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.019 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.019 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.020 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.020 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.021 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "stats", 2023-07-14T11:53:54.021 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:54.021 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.021 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.022 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.022 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.022 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.022 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.022 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.023 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.023 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.023 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.023 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.024 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.024 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.024 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.024 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.024 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.025 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.025 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.025 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.025 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.025 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.026 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.026 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.026 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.026 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.027 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.027 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.027 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.027 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.027 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.028 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.028 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.028 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.028 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.029 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.029 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.029 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.029 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.030 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.030 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.030 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.030 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.030 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.031 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.031 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.031 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.031 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.031 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.032 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.032 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.032 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.032 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.032 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.033 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.033 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.033 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.033 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.034 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.034 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.034 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.034 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.035 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.035 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.035 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.035 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.035 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.036 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.036 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.036 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.036 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.037 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "status", 2023-07-14T11:53:54.037 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:54.037 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.037 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.038 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.038 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.038 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.038 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.039 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.039 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.039 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.039 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.039 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.040 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.040 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.040 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.040 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.041 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.041 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.041 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.041 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.041 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.042 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.042 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.042 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.043 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.043 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.043 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.043 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.043 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.044 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.044 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.044 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.044 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.045 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.045 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.045 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.045 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.046 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.046 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.046 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.046 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.046 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.047 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.047 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.047 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.047 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.047 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.048 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.048 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.048 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.048 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.048 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.049 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.049 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.049 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.050 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.050 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.050 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.050 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.050 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.051 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.051 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.051 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.051 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.051 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.052 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.052 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.052 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.052 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.052 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.053 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "telegraf", 2023-07-14T11:53:54.053 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:54.053 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.054 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.054 INFO:teuthology.orchestra.run.smithi161.stdout: "address": { 2023-07-14T11:53:54.054 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "address", 2023-07-14T11:53:54.054 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.054 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.055 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.055 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T11:53:54.055 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.055 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.056 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.056 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.056 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.056 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.056 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.057 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:54.057 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:54.057 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:54.057 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.058 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.058 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "15", 2023-07-14T11:53:54.058 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.058 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.058 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.059 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.059 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.059 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.059 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.060 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.060 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.060 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.060 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.060 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.061 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.061 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.061 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.061 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.061 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.062 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.062 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.062 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.062 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.062 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.063 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.063 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.063 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.063 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.063 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.064 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.064 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.064 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.065 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.065 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.065 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.065 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.065 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.066 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.066 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.066 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.066 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.066 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.067 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.067 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.067 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.067 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.068 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.068 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.068 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.068 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.068 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.069 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.069 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.069 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.069 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.069 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.070 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.070 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.070 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.070 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.070 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.071 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.071 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.071 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.072 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.072 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.072 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.072 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.073 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.073 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.073 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.073 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.073 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.074 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.075 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "telemetry", 2023-07-14T11:53:54.075 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:54.075 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.075 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.075 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_basic": { 2023-07-14T11:53:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_basic", 2023-07-14T11:53:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.077 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:54.077 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.077 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.077 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.077 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T11:53:54.078 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.078 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.078 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.078 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_crash": { 2023-07-14T11:53:54.079 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_crash", 2023-07-14T11:53:54.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.079 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.079 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:54.080 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.080 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.080 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.080 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T11:53:54.080 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.081 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.081 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.081 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_device": { 2023-07-14T11:53:54.081 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_device", 2023-07-14T11:53:54.082 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.082 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.082 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.082 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "True", 2023-07-14T11:53:54.082 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.083 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.083 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.083 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T11:53:54.083 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.084 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.084 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_ident": { 2023-07-14T11:53:54.084 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_ident", 2023-07-14T11:53:54.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.085 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.085 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.085 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.085 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.086 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.086 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.086 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T11:53:54.086 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.086 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.087 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.087 INFO:teuthology.orchestra.run.smithi161.stdout: "channel_perf": { 2023-07-14T11:53:54.087 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "channel_perf", 2023-07-14T11:53:54.087 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.088 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.088 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.088 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.089 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.089 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.089 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T11:53:54.089 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.089 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.090 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.090 INFO:teuthology.orchestra.run.smithi161.stdout: "contact": { 2023-07-14T11:53:54.090 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "contact", 2023-07-14T11:53:54.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.091 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.091 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.091 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.091 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.091 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.092 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.092 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.092 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.092 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.092 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.093 INFO:teuthology.orchestra.run.smithi161.stdout: "description": { 2023-07-14T11:53:54.093 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "description", 2023-07-14T11:53:54.093 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.093 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.094 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.094 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.094 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.094 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.095 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.095 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.095 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.095 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.095 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.096 INFO:teuthology.orchestra.run.smithi161.stdout: "device_url": { 2023-07-14T11:53:54.096 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "device_url", 2023-07-14T11:53:54.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.096 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.097 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.097 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T11:53:54.097 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.097 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.097 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.098 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.098 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.098 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.098 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.099 INFO:teuthology.orchestra.run.smithi161.stdout: "enabled": { 2023-07-14T11:53:54.099 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "enabled", 2023-07-14T11:53:54.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.099 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.099 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.100 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.100 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.100 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.100 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.101 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.101 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.101 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.101 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.102 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:54.102 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:54.102 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:54.102 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.102 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.103 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "24", 2023-07-14T11:53:54.103 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "8", 2023-07-14T11:53:54.103 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.103 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.103 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.104 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.104 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.104 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.104 INFO:teuthology.orchestra.run.smithi161.stdout: "last_opt_revision": { 2023-07-14T11:53:54.105 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "last_opt_revision", 2023-07-14T11:53:54.105 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:54.105 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.105 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.105 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "1", 2023-07-14T11:53:54.106 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.106 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.106 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.106 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.106 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.107 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.107 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.107 INFO:teuthology.orchestra.run.smithi161.stdout: "leaderboard": { 2023-07-14T11:53:54.107 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "leaderboard", 2023-07-14T11:53:54.108 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.108 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.108 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.108 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.109 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.109 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.109 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.109 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.109 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.110 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.110 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.110 INFO:teuthology.orchestra.run.smithi161.stdout: "leaderboard_description": { 2023-07-14T11:53:54.110 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "leaderboard_description", 2023-07-14T11:53:54.111 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.111 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.111 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.111 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.111 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.112 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.112 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.112 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.112 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.112 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.113 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.113 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.113 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.113 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.114 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.114 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.114 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.114 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.114 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.115 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.115 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.115 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.115 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.115 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.116 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.116 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.116 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.116 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.116 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.117 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.117 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.117 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.117 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.118 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.118 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.118 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.118 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.118 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.119 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.119 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.119 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.119 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.120 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.120 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.120 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.120 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.121 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.121 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.121 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.121 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.121 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.122 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.122 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.122 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.122 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.122 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.123 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.123 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.123 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.123 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.123 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.124 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.124 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.124 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.124 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.125 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.125 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.125 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.125 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.125 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.126 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.126 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.126 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.126 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.127 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.127 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.127 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.127 INFO:teuthology.orchestra.run.smithi161.stdout: "organization": { 2023-07-14T11:53:54.128 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "organization", 2023-07-14T11:53:54.128 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.128 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.128 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.128 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.129 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.129 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.129 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.129 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.129 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.130 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.130 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.130 INFO:teuthology.orchestra.run.smithi161.stdout: "proxy": { 2023-07-14T11:53:54.130 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "proxy", 2023-07-14T11:53:54.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.131 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.131 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.131 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.132 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.132 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.132 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.132 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.132 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.133 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.133 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.133 INFO:teuthology.orchestra.run.smithi161.stdout: "url": { 2023-07-14T11:53:54.133 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "url", 2023-07-14T11:53:54.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.134 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.134 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.134 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T11:53:54.134 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.135 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.135 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.135 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.135 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.135 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.136 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.136 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.136 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.137 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "test_orchestrator", 2023-07-14T11:53:54.137 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:54.137 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.137 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.137 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.138 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.138 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.138 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.138 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.138 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.139 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.139 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.139 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.139 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.139 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.140 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.140 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.140 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.140 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.141 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.141 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.141 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.141 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.141 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.142 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.142 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.142 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.142 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.143 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.143 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.143 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.143 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.143 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.144 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.144 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.144 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.145 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.145 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.145 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.145 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.146 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.146 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.146 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.146 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.147 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.147 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.147 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.147 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.147 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.148 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.148 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.148 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.148 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.149 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.149 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.149 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.149 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.150 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.150 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.150 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.150 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.150 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.151 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.151 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.151 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.151 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.152 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.153 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "volumes", 2023-07-14T11:53:54.153 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:54.153 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.153 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.153 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.154 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.154 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.154 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.154 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.154 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.155 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.155 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.155 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.155 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.155 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.156 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.156 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.156 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.156 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.156 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.157 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.157 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.157 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.158 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.158 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.158 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.158 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.158 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.159 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.159 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.159 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.159 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.159 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.160 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.160 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.160 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.164 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.164 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.164 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.165 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.165 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.165 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.165 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.166 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.166 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.166 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.166 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.166 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.167 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.167 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.167 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.167 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.168 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.168 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.168 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.168 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.169 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.169 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.169 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.169 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.170 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.170 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.170 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.170 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.170 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.171 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.171 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.171 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.172 INFO:teuthology.orchestra.run.smithi161.stdout: "max_concurrent_clones": { 2023-07-14T11:53:54.172 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "max_concurrent_clones", 2023-07-14T11:53:54.172 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:54.173 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.173 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.173 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "4", 2023-07-14T11:53:54.173 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.173 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.174 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.174 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T11:53:54.174 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.174 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.174 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.175 INFO:teuthology.orchestra.run.smithi161.stdout: "snapshot_clone_delay": { 2023-07-14T11:53:54.175 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snapshot_clone_delay", 2023-07-14T11:53:54.175 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:54.175 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.176 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.176 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "0", 2023-07-14T11:53:54.176 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.176 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.176 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.177 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T11:53:54.177 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.177 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.177 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.178 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix", 2023-07-14T11:53:54.178 INFO:teuthology.orchestra.run.smithi161.stdout: "can_run": true, 2023-07-14T11:53:54.179 INFO:teuthology.orchestra.run.smithi161.stdout: "error_string": "", 2023-07-14T11:53:54.179 INFO:teuthology.orchestra.run.smithi161.stdout: "module_options": { 2023-07-14T11:53:54.179 INFO:teuthology.orchestra.run.smithi161.stdout: "discovery_interval": { 2023-07-14T11:53:54.179 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "discovery_interval", 2023-07-14T11:53:54.179 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "uint", 2023-07-14T11:53:54.180 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.180 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.180 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "100", 2023-07-14T11:53:54.180 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.181 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.181 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.181 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.181 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.181 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.182 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.182 INFO:teuthology.orchestra.run.smithi161.stdout: "identifier": { 2023-07-14T11:53:54.182 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "identifier", 2023-07-14T11:53:54.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.183 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.183 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.183 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.183 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.184 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.184 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.184 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.185 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.185 INFO:teuthology.orchestra.run.smithi161.stdout: "interval": { 2023-07-14T11:53:54.185 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "interval", 2023-07-14T11:53:54.185 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "secs", 2023-07-14T11:53:54.186 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.186 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.186 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "60", 2023-07-14T11:53:54.186 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.186 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.187 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.187 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.187 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.187 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.187 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.188 INFO:teuthology.orchestra.run.smithi161.stdout: "log_level": { 2023-07-14T11:53:54.188 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_level", 2023-07-14T11:53:54.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.189 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.189 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.189 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.189 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.189 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.189 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.190 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.190 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.190 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.190 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.190 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.191 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.191 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.191 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.191 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.191 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.192 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.192 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster": { 2023-07-14T11:53:54.192 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster", 2023-07-14T11:53:54.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.193 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.193 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.193 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.193 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.193 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.194 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.194 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.194 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.194 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.194 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.195 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_cluster_level": { 2023-07-14T11:53:54.195 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_cluster_level", 2023-07-14T11:53:54.195 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.195 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.195 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.196 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "info", 2023-07-14T11:53:54.196 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.196 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.196 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [ 2023-07-14T11:53:54.196 INFO:teuthology.orchestra.run.smithi161.stdout: "", 2023-07-14T11:53:54.197 INFO:teuthology.orchestra.run.smithi161.stdout: "critical", 2023-07-14T11:53:54.197 INFO:teuthology.orchestra.run.smithi161.stdout: "debug", 2023-07-14T11:53:54.197 INFO:teuthology.orchestra.run.smithi161.stdout: "error", 2023-07-14T11:53:54.197 INFO:teuthology.orchestra.run.smithi161.stdout: "info", 2023-07-14T11:53:54.197 INFO:teuthology.orchestra.run.smithi161.stdout: "warning" 2023-07-14T11:53:54.198 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.198 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.198 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.198 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.198 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.199 INFO:teuthology.orchestra.run.smithi161.stdout: "log_to_file": { 2023-07-14T11:53:54.199 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "log_to_file", 2023-07-14T11:53:54.199 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "bool", 2023-07-14T11:53:54.199 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.200 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2023-07-14T11:53:54.200 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "False", 2023-07-14T11:53:54.200 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.200 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.200 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.201 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.201 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.201 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.201 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.202 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_host": { 2023-07-14T11:53:54.202 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_host", 2023-07-14T11:53:54.202 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.202 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.202 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.202 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "", 2023-07-14T11:53:54.203 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.203 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.203 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.203 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.203 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.204 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.204 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.204 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_port": { 2023-07-14T11:53:54.204 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_port", 2023-07-14T11:53:54.205 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "int", 2023-07-14T11:53:54.205 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.205 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.205 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "10051", 2023-07-14T11:53:54.205 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.206 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.206 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.206 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.206 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.206 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.207 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.207 INFO:teuthology.orchestra.run.smithi161.stdout: "zabbix_sender": { 2023-07-14T11:53:54.207 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "zabbix_sender", 2023-07-14T11:53:54.207 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "str", 2023-07-14T11:53:54.208 INFO:teuthology.orchestra.run.smithi161.stdout: "level": "advanced", 2023-07-14T11:53:54.208 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 0, 2023-07-14T11:53:54.208 INFO:teuthology.orchestra.run.smithi161.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T11:53:54.208 INFO:teuthology.orchestra.run.smithi161.stdout: "min": "", 2023-07-14T11:53:54.208 INFO:teuthology.orchestra.run.smithi161.stdout: "max": "", 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi161.stdout: "enum_allowed": [], 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi161.stdout: "desc": "", 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi161.stdout: "long_desc": "", 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi161.stdout: "tags": [], 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi161.stdout: "see_also": [] 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi161.stdout: "services": { 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi161.stdout: "dashboard": "https://172.21.15.161:8443/" 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi161.stdout: "always_on_modules": { 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi161.stdout: "octopus": [ 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi161.stdout: "balancer", 2023-07-14T11:53:54.212 INFO:teuthology.orchestra.run.smithi161.stdout: "crash", 2023-07-14T11:53:54.212 INFO:teuthology.orchestra.run.smithi161.stdout: "devicehealth", 2023-07-14T11:53:54.212 INFO:teuthology.orchestra.run.smithi161.stdout: "orchestrator", 2023-07-14T11:53:54.212 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscaler", 2023-07-14T11:53:54.212 INFO:teuthology.orchestra.run.smithi161.stdout: "progress", 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_support", 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi161.stdout: "status", 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi161.stdout: "telemetry", 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi161.stdout: "volumes" 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi161.stdout: "pacific": [ 2023-07-14T11:53:54.214 INFO:teuthology.orchestra.run.smithi161.stdout: "balancer", 2023-07-14T11:53:54.214 INFO:teuthology.orchestra.run.smithi161.stdout: "crash", 2023-07-14T11:53:54.214 INFO:teuthology.orchestra.run.smithi161.stdout: "devicehealth", 2023-07-14T11:53:54.214 INFO:teuthology.orchestra.run.smithi161.stdout: "orchestrator", 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscaler", 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi161.stdout: "progress", 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_support", 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi161.stdout: "status", 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi161.stdout: "telemetry", 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi161.stdout: "volumes" 2023-07-14T11:53:54.216 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.216 INFO:teuthology.orchestra.run.smithi161.stdout: "quincy": [ 2023-07-14T11:53:54.216 INFO:teuthology.orchestra.run.smithi161.stdout: "balancer", 2023-07-14T11:53:54.216 INFO:teuthology.orchestra.run.smithi161.stdout: "crash", 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi161.stdout: "devicehealth", 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi161.stdout: "orchestrator", 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscaler", 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi161.stdout: "progress", 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_support", 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi161.stdout: "status", 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi161.stdout: "telemetry", 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi161.stdout: "volumes" 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi161.stdout: "reef": [ 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi161.stdout: "balancer", 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi161.stdout: "crash", 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi161.stdout: "devicehealth", 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi161.stdout: "orchestrator", 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscaler", 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi161.stdout: "progress", 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi161.stdout: "rbd_support", 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi161.stdout: "status", 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi161.stdout: "telemetry", 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi161.stdout: "volumes" 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi161.stdout: "last_failure_osd_epoch": 93, 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi161.stdout: "active_clients": [ 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "devicehealth", 2023-07-14T11:53:54.222 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2023-07-14T11:53:54.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.222 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2023-07-14T11:53:54.222 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:53:54.222 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 1197174849 2023-07-14T11:53:54.223 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.223 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.223 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "libcephsqlite", 2023-07-14T11:53:54.223 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2023-07-14T11:53:54.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.224 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2023-07-14T11:53:54.224 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:53:54.224 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4250475172 2023-07-14T11:53:54.224 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.225 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "rbd_support", 2023-07-14T11:53:54.225 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2023-07-14T11:53:54.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.226 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2023-07-14T11:53:54.226 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:53:54.226 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 105704933 2023-07-14T11:53:54.226 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.227 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.227 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snap_schedule", 2023-07-14T11:53:54.227 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2023-07-14T11:53:54.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.228 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2023-07-14T11:53:54.228 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:53:54.228 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 39311932 2023-07-14T11:53:54.228 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.229 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.229 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "snap_schedule", 2023-07-14T11:53:54.229 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2023-07-14T11:53:54.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.230 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2023-07-14T11:53:54.230 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:53:54.230 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 1382754340 2023-07-14T11:53:54.230 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.230 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:54.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.231 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "volumes", 2023-07-14T11:53:54.231 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2023-07-14T11:53:54.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2023-07-14T11:53:54.232 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2023-07-14T11:53:54.232 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:0", 2023-07-14T11:53:54.232 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4089363807 2023-07-14T11:53:54.232 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.232 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.233 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:54.233 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T11:53:54.233 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:53:54.235 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:50 smithi182 ceph-mon[129366]: pgmap v1000: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:54.235 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:50 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2160674357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:53:54.235 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:50 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1160688015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:53:54.235 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:50 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2289891367' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T11:53:54.236 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:52 smithi182 ceph-mon[129366]: pgmap v1001: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:54.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:50 smithi175 ceph-mon[129081]: pgmap v1000: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:54.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:50 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2160674357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:53:54.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:50 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1160688015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:53:54.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:50 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2289891367' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T11:53:54.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:52 smithi175 ceph-mon[129081]: pgmap v1001: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:54.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb3140a1300 msgr2=0x7fb3140a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb3140a1300 0x7fb3140a37b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3281993e0 tx=0x7fb310005fd0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb328072850 msgr2=0x7fb328197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb328072850 0x7fb328197dd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb31800eb30 tx=0x7fb31800ee40 comp rx=0 tx=0).stop 2023-07-14T11:53:54.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 shutdown_connections 2023-07-14T11:53:54.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fb3140a1300 0x7fb3140a37b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb32810c390 0x7fb32819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fb328072850 0x7fb328197dd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 --2- 172.21.15.161:0/2289891367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328073160 0x7fb328198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.239 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 >> 172.21.15.161:0/2289891367 conn(0x7fb3280fb630 msgr2=0x7fb328103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:54.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 shutdown_connections 2023-07-14T11:53:54.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:53:50.709+0000 7fb32cdff700 1 -- 172.21.15.161:0/2289891367 wait complete. 2023-07-14T11:53:54.248 DEBUG:tasks.check_counter:Getting stats from a 2023-07-14T11:53:54.248 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-14T11:53:54.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:50 smithi161 ceph-mon[95526]: pgmap v1000: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:54.249 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:50 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2160674357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:53:54.250 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:50 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1160688015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:53:54.250 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:50 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2289891367' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T11:53:54.250 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:52 smithi161 ceph-mon[95526]: pgmap v1001: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:54.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:54 smithi175 ceph-mon[129081]: pgmap v1002: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:54.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:54 smithi182 ceph-mon[129366]: pgmap v1002: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:54.725 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:54 smithi161 ceph-mon[95526]: pgmap v1002: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:53:56.396 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:53:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:53:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2023-07-14T11:53:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2023-07-14T11:53:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2023-07-14T11:53:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2023-07-14T11:53:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T11:53:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2023-07-14T11:53:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.003908855, 2023-07-14T11:53:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T11:53:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000729067, 2023-07-14T11:53:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:53:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:53:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:53:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T11:53:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T11:53:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:53:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 8, 2023-07-14T11:53:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 421, 2023-07-14T11:53:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3584, 2023-07-14T11:53:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 377344, 2023-07-14T11:53:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2023-07-14T11:53:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T11:53:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.043142878, 2023-07-14T11:53:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.031790003, 2023-07-14T11:53:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.004239947, 2023-07-14T11:53:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:53:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:53:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 419, 2023-07-14T11:53:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.052810363, 2023-07-14T11:53:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000126039 2023-07-14T11:53:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:53:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3584, 2023-07-14T11:53:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 377344 2023-07-14T11:53:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:53:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 556, 2023-07-14T11:53:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 546, 2023-07-14T11:53:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1518288, 2023-07-14T11:53:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 313424, 2023-07-14T11:53:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2023-07-14T11:53:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2023-07-14T11:53:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.103570126, 2023-07-14T11:53:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.044741834, 2023-07-14T11:53:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.048263208, 2023-07-14T11:53:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.012367601, 2023-07-14T11:53:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:53:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 545, 2023-07-14T11:53:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.047998376, 2023-07-14T11:53:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000088070 2023-07-14T11:53:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:53:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1518288, 2023-07-14T11:53:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 313424 2023-07-14T11:53:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T11:53:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2023-07-14T11:53:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T11:53:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T11:53:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:53:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T11:53:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:53:56.412 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:53:56.412 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:53:56.412 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:53:56.412 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:53:56.412 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:53:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:53:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:53:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:53:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:53:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:53:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:53:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:53:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T11:53:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:53:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T11:53:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:53:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:53:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:53:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:53:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:53:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:53:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:53:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T11:53:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:53:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:53:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:53:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T11:53:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T11:53:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T11:53:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:53:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:53:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:53:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:53:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148714, 2023-07-14T11:53:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2023-07-14T11:53:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:53:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T11:53:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T11:53:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T11:53:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:53:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T11:53:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:53:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T11:53:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2023-07-14T11:53:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2023-07-14T11:53:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:53:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:53:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T11:53:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T11:53:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2023-07-14T11:53:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2023-07-14T11:53:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T11:53:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T11:53:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T11:53:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T11:53:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:53:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T11:53:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T11:53:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T11:53:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 560, 2023-07-14T11:53:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1439705, 2023-07-14T11:53:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T11:53:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 560, 2023-07-14T11:53:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T11:53:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T11:53:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 560, 2023-07-14T11:53:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1439705, 2023-07-14T11:53:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T11:53:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:56.426 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:53:56.871 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-14T11:53:56.872 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-14T11:53:56.872 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-07-14T11:53:56.872 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-07-14T11:53:56.873 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-07-14T11:53:56.873 DEBUG:tasks.check_counter:Getting stats from d 2023-07-14T11:53:56.873 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-14T11:53:56.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:56 smithi161 ceph-mon[95526]: pgmap v1003: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:57.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:56 smithi175 ceph-mon[129081]: pgmap v1003: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:57.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:56 smithi182 ceph-mon[129366]: pgmap v1003: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:53:57.600 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:53:58.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:53:58 smithi161 ceph-mon[95526]: pgmap v1004: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:59.121 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:53:59.122 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:53:59.122 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2023-07-14T11:53:59.123 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2023-07-14T11:53:59.123 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2023-07-14T11:53:59.123 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2023-07-14T11:53:59.124 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T11:53:59.124 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2023-07-14T11:53:59.125 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.003793319, 2023-07-14T11:53:59.125 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T11:53:59.125 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000866396, 2023-07-14T11:53:59.126 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:53:59.126 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:53:59.127 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:59.127 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:59.127 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:59.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.128 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:53:59.128 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:59.129 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:59.129 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:59.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.130 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T11:53:59.130 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T11:53:59.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.131 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:53:59.131 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2023-07-14T11:53:59.132 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 422, 2023-07-14T11:53:59.132 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 640, 2023-07-14T11:53:59.132 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 377424, 2023-07-14T11:53:59.133 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2023-07-14T11:53:59.133 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T11:53:59.134 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.042807312, 2023-07-14T11:53:59.134 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.031954294, 2023-07-14T11:53:59.134 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.004100915, 2023-07-14T11:53:59.135 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:53:59.135 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:53:59.135 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 419, 2023-07-14T11:53:59.136 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.051175258, 2023-07-14T11:53:59.136 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000122136 2023-07-14T11:53:59.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.137 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:53:59.137 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:59.138 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:59.138 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:59.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-14T11:53:59.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 377424 2023-07-14T11:53:59.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.140 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:53:59.140 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 565, 2023-07-14T11:53:59.141 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 553, 2023-07-14T11:53:59.143 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1522592, 2023-07-14T11:53:59.143 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 316832, 2023-07-14T11:53:59.143 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2023-07-14T11:53:59.143 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2023-07-14T11:53:59.143 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.095356634, 2023-07-14T11:53:59.144 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.033146439, 2023-07-14T11:53:59.144 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.048343644, 2023-07-14T11:53:59.144 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.014789750, 2023-07-14T11:53:59.144 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:53:59.144 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 553, 2023-07-14T11:53:59.145 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.045727417, 2023-07-14T11:53:59.145 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000082689 2023-07-14T11:53:59.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.145 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:53:59.145 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:59.146 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:59.146 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:59.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.146 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1522592, 2023-07-14T11:53:59.146 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 316832 2023-07-14T11:53:59.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.147 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T11:53:59.147 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2023-07-14T11:53:59.147 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T11:53:59.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.148 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T11:53:59.148 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:53:59.148 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T11:53:59.148 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:53:59.148 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:53:59.148 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:53:59.149 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:53:59.149 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:53:59.149 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:53:59.149 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:53:59.149 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:53:59.150 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:53:59.150 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:53:59.150 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:53:59.150 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:53:59.150 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:53:59.151 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T11:53:59.151 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:53:59.151 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T11:53:59.151 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:53:59.151 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:53:59.152 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:53:59.152 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:53:59.152 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:53:59.152 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:53:59.152 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:53:59.153 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T11:53:59.153 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:53:59.153 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:53:59.153 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:53:59.153 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T11:53:59.153 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T11:53:59.154 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T11:53:59.154 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:53:59.154 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:53:59.154 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:53:59.154 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:53:59.155 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148714, 2023-07-14T11:53:59.155 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2023-07-14T11:53:59.155 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:53:59.155 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T11:53:59.155 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T11:53:59.156 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T11:53:59.156 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:53:59.156 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T11:53:59.156 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:53:59.156 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T11:53:59.157 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2023-07-14T11:53:59.157 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2023-07-14T11:53:59.157 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:53:59.157 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:53:59.157 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T11:53:59.158 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T11:53:59.158 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2023-07-14T11:53:59.158 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2023-07-14T11:53:59.158 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T11:53:59.158 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T11:53:59.159 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T11:53:59.159 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T11:53:59.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:53:59.159 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:53:59.159 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 565, 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1440355, 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 565, 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 565, 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1440355, 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:53:59.163 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:53:58 smithi175 ceph-mon[129081]: pgmap v1004: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:59.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:53:58 smithi182 ceph-mon[129366]: pgmap v1004: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:53:59.594 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-14T11:53:59.594 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-14T11:53:59.594 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-07-14T11:53:59.595 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-07-14T11:53:59.595 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-07-14T11:53:59.595 DEBUG:tasks.check_counter:Getting stats from g 2023-07-14T11:53:59.595 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-14T11:54:00.327 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:54:00.918 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:00 smithi161 ceph-mon[95526]: pgmap v1005: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:01.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:00 smithi175 ceph-mon[129081]: pgmap v1005: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:01.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:00 smithi182 ceph-mon[129366]: pgmap v1005: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:01.846 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:54:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 562, 2023-07-14T11:54:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 550, 2023-07-14T11:54:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1521776, 2023-07-14T11:54:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 315056, 2023-07-14T11:54:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T11:54:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2023-07-14T11:54:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.107209272, 2023-07-14T11:54:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.045631326, 2023-07-14T11:54:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.047810920, 2023-07-14T11:54:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.012794626, 2023-07-14T11:54:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 550, 2023-07-14T11:54:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.041441723, 2023-07-14T11:54:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000075348 2023-07-14T11:54:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1521776, 2023-07-14T11:54:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 315056 2023-07-14T11:54:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2023-07-14T11:54:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 421, 2023-07-14T11:54:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 640, 2023-07-14T11:54:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 377248, 2023-07-14T11:54:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2023-07-14T11:54:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T11:54:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.042796795, 2023-07-14T11:54:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.031634203, 2023-07-14T11:54:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.003780721, 2023-07-14T11:54:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 418, 2023-07-14T11:54:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.048522466, 2023-07-14T11:54:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000116082 2023-07-14T11:54:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-14T11:54:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 377248 2023-07-14T11:54:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2023-07-14T11:54:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2023-07-14T11:54:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2023-07-14T11:54:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2023-07-14T11:54:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2023-07-14T11:54:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2023-07-14T11:54:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.000856442, 2023-07-14T11:54:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T11:54:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000717674, 2023-07-14T11:54:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T11:54:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T11:54:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T11:54:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2023-07-14T11:54:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T11:54:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T11:54:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:01.864 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:01.864 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:01.864 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:01.864 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:01.864 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:01.865 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:01.865 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:01.865 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:01.865 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:01.865 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:01.865 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:01.866 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:01.866 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:01.866 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:01.866 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:01.866 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:01.867 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:01.867 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:01.867 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:01.867 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T11:54:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:01.869 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148714, 2023-07-14T11:54:01.869 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2023-07-14T11:54:01.869 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:54:01.869 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T11:54:01.870 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T11:54:01.870 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T11:54:01.870 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:01.870 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:01.870 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:01.871 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:01.871 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2023-07-14T11:54:01.871 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2023-07-14T11:54:01.871 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:01.871 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:01.872 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:01.872 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T11:54:01.872 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2023-07-14T11:54:01.872 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2023-07-14T11:54:01.872 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:01.873 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T11:54:01.873 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:01.873 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T11:54:01.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:01.873 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:01.874 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T11:54:01.874 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T11:54:01.874 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T11:54:01.874 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 562, 2023-07-14T11:54:01.874 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1439965, 2023-07-14T11:54:01.874 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:01.875 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 562, 2023-07-14T11:54:01.875 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T11:54:01.875 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T11:54:01.875 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 562, 2023-07-14T11:54:01.875 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1439965, 2023-07-14T11:54:01.876 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T11:54:01.876 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:01.876 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:01.876 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:01.876 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:54:01.877 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:54:01.878 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:54:02.214 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-14T11:54:02.215 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-14T11:54:02.215 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-07-14T11:54:02.215 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-07-14T11:54:02.216 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-07-14T11:54:02.216 DEBUG:tasks.check_counter:Getting stats from j 2023-07-14T11:54:02.216 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-14T11:54:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.a/config 2023-07-14T11:54:02.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:02 smithi161 ceph-mon[95526]: pgmap v1006: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 39 KiB/s wr, 22 op/s 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:02 smithi175 ceph-mon[129081]: pgmap v1006: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 39 KiB/s wr, 22 op/s 2023-07-14T11:54:03.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:02 smithi182 ceph-mon[129366]: pgmap v1006: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 39 KiB/s wr, 22 op/s 2023-07-14T11:54:04.462 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T11:54:04.462 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:04.462 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2023-07-14T11:54:04.463 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2023-07-14T11:54:04.463 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2023-07-14T11:54:04.463 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2023-07-14T11:54:04.463 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T11:54:04.464 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2023-07-14T11:54:04.464 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.003286341, 2023-07-14T11:54:04.464 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T11:54:04.464 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000670670, 2023-07-14T11:54:04.464 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:04.465 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:04.465 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:04.465 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:04.465 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:04.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.465 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:04.466 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:04.466 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:04.466 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:04.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.466 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T11:54:04.467 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T11:54:04.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.467 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:04.467 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 563, 2023-07-14T11:54:04.467 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 553, 2023-07-14T11:54:04.468 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1520192, 2023-07-14T11:54:04.468 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 317568, 2023-07-14T11:54:04.468 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2023-07-14T11:54:04.468 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2023-07-14T11:54:04.468 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.103795723, 2023-07-14T11:54:04.469 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.045395372, 2023-07-14T11:54:04.469 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.047557584, 2023-07-14T11:54:04.469 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.012533159, 2023-07-14T11:54:04.469 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:04.469 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 552, 2023-07-14T11:54:04.469 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.037749964, 2023-07-14T11:54:04.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000068387 2023-07-14T11:54:04.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.470 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:04.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:04.470 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:04.471 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:04.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.471 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1520192, 2023-07-14T11:54:04.471 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 317568 2023-07-14T11:54:04.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.471 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:04.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 8, 2023-07-14T11:54:04.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 426, 2023-07-14T11:54:04.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3584, 2023-07-14T11:54:04.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 378224, 2023-07-14T11:54:04.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2023-07-14T11:54:04.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T11:54:04.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.044121702, 2023-07-14T11:54:04.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.032667788, 2023-07-14T11:54:04.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.004818120, 2023-07-14T11:54:04.474 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:04.474 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:04.474 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 424, 2023-07-14T11:54:04.474 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.074091691, 2023-07-14T11:54:04.474 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000174744 2023-07-14T11:54:04.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:04.475 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:04.475 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:04.475 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:04.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.476 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3584, 2023-07-14T11:54:04.476 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 378224 2023-07-14T11:54:04.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.476 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T11:54:04.477 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2023-07-14T11:54:04.477 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T11:54:04.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.477 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T11:54:04.477 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:04.478 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:04.478 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:04.478 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:04.478 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:04.478 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:04.478 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:04.479 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:04.479 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:04.479 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:04.479 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:04.479 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:04.480 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:04.480 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:04.480 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:04.480 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:04.480 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:04.481 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:04.481 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:04.481 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:04.481 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:04.481 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:04.481 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:04.482 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:04.482 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:04.482 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:04.482 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:04.482 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:04.483 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:04.483 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:04.483 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:04.483 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T11:54:04.483 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:04.484 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:04.484 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:04.484 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:04.484 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 147930, 2023-07-14T11:54:04.484 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 28, 2023-07-14T11:54:04.485 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:54:04.485 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T11:54:04.485 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T11:54:04.485 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T11:54:04.485 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:04.486 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:04.486 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:04.486 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:04.486 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2023-07-14T11:54:04.486 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2023-07-14T11:54:04.486 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:04.487 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:04.487 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:04.487 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T11:54:04.487 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2023-07-14T11:54:04.487 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2023-07-14T11:54:04.488 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:04.488 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T11:54:04.488 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:04.488 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T11:54:04.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T11:54:04.489 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:04.489 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T11:54:04.489 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T11:54:04.489 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T11:54:04.489 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 567, 2023-07-14T11:54:04.490 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1440615, 2023-07-14T11:54:04.490 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:04.490 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 567, 2023-07-14T11:54:04.490 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T11:54:04.490 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T11:54:04.490 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 567, 2023-07-14T11:54:04.491 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1440615, 2023-07-14T11:54:04.491 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T11:54:04.491 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T11:54:04.491 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T11:54:04.491 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T11:54:04.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:54:04.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T11:54:04.492 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T11:54:04.493 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:04 smithi161 ceph-mon[95526]: pgmap v1007: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-07-14T11:54:04.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:04 smithi175 ceph-mon[129081]: pgmap v1007: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-07-14T11:54:04.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:04 smithi182 ceph-mon[129366]: pgmap v1007: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-07-14T11:54:04.939 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-14T11:54:04.940 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-14T11:54:04.940 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-07-14T11:54:04.940 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-07-14T11:54:04.941 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-07-14T11:54:04.941 DEBUG:tasks.check_counter:Getting stats from b 2023-07-14T11:54:04.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-14T11:54:06.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:06 smithi161 ceph-mon[95526]: pgmap v1008: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-07-14T11:54:07.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:06 smithi175 ceph-mon[129081]: pgmap v1008: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-07-14T11:54:07.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:06 smithi182 ceph-mon[129366]: pgmap v1008: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-07-14T11:54:07.870 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:54:08.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:08 smithi161 ceph-mon[95526]: pgmap v1009: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:54:09.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:08 smithi175 ceph-mon[129081]: pgmap v1009: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:54:09.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:08 smithi182 ceph-mon[129366]: pgmap v1009: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:54:10.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:10 smithi175 ceph-mon[129081]: pgmap v1010: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:54:11.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:10 smithi182 ceph-mon[129366]: pgmap v1010: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:54:11.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:10 smithi161 ceph-mon[95526]: pgmap v1010: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-07-14T11:54:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2023-07-14T11:54:11.658 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:11.658 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 426051, 2023-07-14T11:54:11.658 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 639863, 2023-07-14T11:54:11.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 145393565, 2023-07-14T11:54:11.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1056300350, 2023-07-14T11:54:11.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 27, 2023-07-14T11:54:11.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2023-07-14T11:54:11.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 27.992681756, 2023-07-14T11:54:11.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 12.550366465, 2023-07-14T11:54:11.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 16.192116779, 2023-07-14T11:54:11.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 2.709549054, 2023-07-14T11:54:11.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:11.660 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 639860, 2023-07-14T11:54:11.661 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 99.477930611, 2023-07-14T11:54:11.661 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000155468 2023-07-14T11:54:11.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.661 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:11.661 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 729050, 2023-07-14T11:54:11.662 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.127721411, 2023-07-14T11:54:11.662 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000175 2023-07-14T11:54:11.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.662 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 128053312, 2023-07-14T11:54:11.662 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 428182256 2023-07-14T11:54:11.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.663 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:11.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 241797, 2023-07-14T11:54:11.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 252634, 2023-07-14T11:54:11.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 66999651, 2023-07-14T11:54:11.664 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 498755898, 2023-07-14T11:54:11.664 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 16, 2023-07-14T11:54:11.664 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2023-07-14T11:54:11.664 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 15.243994565, 2023-07-14T11:54:11.664 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 6.186226344, 2023-07-14T11:54:11.665 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 7.811495188, 2023-07-14T11:54:11.665 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 2.496659685, 2023-07-14T11:54:11.665 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:11.665 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 252635, 2023-07-14T11:54:11.665 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 21.431646107, 2023-07-14T11:54:11.665 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000084832 2023-07-14T11:54:11.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.666 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:11.666 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 371773, 2023-07-14T11:54:11.666 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.056655881, 2023-07-14T11:54:11.666 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000152 2023-07-14T11:54:11.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.667 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2725264, 2023-07-14T11:54:11.667 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 707632 2023-07-14T11:54:11.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.667 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:11.668 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 3091767, 2023-07-14T11:54:11.668 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 3127074, 2023-07-14T11:54:11.668 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 507530865, 2023-07-14T11:54:11.668 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 794283385, 2023-07-14T11:54:11.668 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 218, 2023-07-14T11:54:11.668 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2023-07-14T11:54:11.669 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 87.870175430, 2023-07-14T11:54:11.669 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 34.025052826, 2023-07-14T11:54:11.669 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 58.264373475, 2023-07-14T11:54:11.669 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 1.770993597, 2023-07-14T11:54:11.669 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:11.670 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3127071, 2023-07-14T11:54:11.670 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 53.436164074, 2023-07-14T11:54:11.670 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000017088 2023-07-14T11:54:11.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.670 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:11.671 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6116379, 2023-07-14T11:54:11.671 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.618612301, 2023-07-14T11:54:11.671 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000101 2023-07-14T11:54:11.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.671 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 42528, 2023-07-14T11:54:11.671 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 263360 2023-07-14T11:54:11.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.672 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2023-07-14T11:54:11.672 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2023-07-14T11:54:11.672 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2023-07-14T11:54:11.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.673 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2023-07-14T11:54:11.673 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2023-07-14T11:54:11.673 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2023-07-14T11:54:11.673 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 175513, 2023-07-14T11:54:11.673 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 166.102438385, 2023-07-14T11:54:11.674 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000946382 2023-07-14T11:54:11.674 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.674 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2023-07-14T11:54:11.674 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2023-07-14T11:54:11.675 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2023-07-14T11:54:11.675 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 61428, 2023-07-14T11:54:11.675 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 15.024795045, 2023-07-14T11:54:11.675 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000244591 2023-07-14T11:54:11.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.676 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2023-07-14T11:54:11.676 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 246083, 2023-07-14T11:54:11.676 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 224178, 2023-07-14T11:54:11.676 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2023-07-14T11:54:11.676 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 224178, 2023-07-14T11:54:11.677 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1328.218969391, 2023-07-14T11:54:11.677 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.005924840 2023-07-14T11:54:11.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.677 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2023-07-14T11:54:11.677 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 21905, 2023-07-14T11:54:11.678 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 5, 2023-07-14T11:54:11.678 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 2, 2023-07-14T11:54:11.678 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 245, 2023-07-14T11:54:11.678 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 18, 2023-07-14T11:54:11.678 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 25, 2023-07-14T11:54:11.679 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 2643, 2023-07-14T11:54:11.679 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 1470, 2023-07-14T11:54:11.679 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 1136, 2023-07-14T11:54:11.679 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 37, 2023-07-14T11:54:11.679 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 542, 2023-07-14T11:54:11.680 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 2688, 2023-07-14T11:54:11.680 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 225, 2023-07-14T11:54:11.680 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 225, 2023-07-14T11:54:11.680 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 23, 2023-07-14T11:54:11.680 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 338366, 2023-07-14T11:54:11.680 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 235168, 2023-07-14T11:54:11.681 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 21873, 2023-07-14T11:54:11.681 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 26, 2023-07-14T11:54:11.681 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2023-07-14T11:54:11.681 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2023-07-14T11:54:11.681 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 1525, 2023-07-14T11:54:11.682 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2023-07-14T11:54:11.682 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2023-07-14T11:54:11.682 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 29, 2023-07-14T11:54:11.682 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 2151, 2023-07-14T11:54:11.682 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 13, 2023-07-14T11:54:11.683 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 478, 2023-07-14T11:54:11.683 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2023-07-14T11:54:11.683 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 2794, 2023-07-14T11:54:11.683 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2023-07-14T11:54:11.683 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 755, 2023-07-14T11:54:11.684 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 229819610, 2023-07-14T11:54:11.684 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2023-07-14T11:54:11.684 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 2794, 2023-07-14T11:54:11.684 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2023-07-14T11:54:11.684 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T11:54:11.685 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2023-07-14T11:54:11.685 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 609, 2023-07-14T11:54:11.685 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 17, 2023-07-14T11:54:11.685 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 592, 2023-07-14T11:54:11.685 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 2185, 2023-07-14T11:54:11.685 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 13, 2023-07-14T11:54:11.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 39163, 2023-07-14T11:54:11.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 156559, 2023-07-14T11:54:11.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T11:54:11.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 1800, 2023-07-14T11:54:11.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 91836, 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 40101, 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 7277, 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 117371 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.688 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2023-07-14T11:54:11.688 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 10, 2023-07-14T11:54:11.688 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 12, 2023-07-14T11:54:11.688 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 8, 2023-07-14T11:54:11.688 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 30, 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 210, 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 37533, 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 37533, 2023-07-14T11:54:11.690 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2023-07-14T11:54:11.690 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2023-07-14T11:54:11.690 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2023-07-14T11:54:11.690 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2023-07-14T11:54:11.690 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 17, 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 39, 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 43, 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 211717, 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 130260, 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 130260, 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 81457, 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 732, 2023-07-14T11:54:11.694 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 346, 2023-07-14T11:54:11.694 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 218, 2023-07-14T11:54:11.694 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 218, 2023-07-14T11:54:11.694 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 129, 2023-07-14T11:54:11.694 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 1, 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 780154585, 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 1287512249, 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2023-07-14T11:54:11.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 93538, 2023-07-14T11:54:11.696 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 575.818964723, 2023-07-14T11:54:11.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006155989 2023-07-14T11:54:11.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.696 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 819, 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 40041, 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 39222, 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 149, 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 408, 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 259, 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 2643, 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 70411, 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 67768, 2023-07-14T11:54:11.699 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 225, 2023-07-14T11:54:11.699 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 39170, 2023-07-14T11:54:11.699 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 38945, 2023-07-14T11:54:11.699 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 81144, 2023-07-14T11:54:11.699 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 207156 2023-07-14T11:54:11.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.700 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2023-07-14T11:54:11.700 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 339342, 2023-07-14T11:54:11.700 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 34468, 2023-07-14T11:54:11.700 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 246083, 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 7837, 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 69001, 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 38243, 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 131.024482427, 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003426103 2023-07-14T11:54:11.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.702 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2023-07-14T11:54:11.702 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 35035, 2023-07-14T11:54:11.702 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 315.654849537, 2023-07-14T11:54:11.702 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.009009700 2023-07-14T11:54:11.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.703 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2023-07-14T11:54:11.703 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.703 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.703 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.704 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2023-07-14T11:54:11.704 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.704 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.704 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.705 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2023-07-14T11:54:11.705 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 44348, 2023-07-14T11:54:11.705 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 135.732960497, 2023-07-14T11:54:11.705 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003060633 2023-07-14T11:54:11.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.706 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2023-07-14T11:54:11.706 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.706 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.706 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000328322, 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000328322 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.709 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2023-07-14T11:54:11.709 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.709 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.709 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.710 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2023-07-14T11:54:11.710 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.710 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.710 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.711 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2023-07-14T11:54:11.711 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 32, 2023-07-14T11:54:11.711 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.014252689, 2023-07-14T11:54:11.711 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000445396 2023-07-14T11:54:11.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.712 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2023-07-14T11:54:11.712 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 119, 2023-07-14T11:54:11.712 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.735564038, 2023-07-14T11:54:11.712 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006181210 2023-07-14T11:54:11.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.712 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2023-07-14T11:54:11.713 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.713 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.713 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.713 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2023-07-14T11:54:11.714 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 32, 2023-07-14T11:54:11.714 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 11.896282193, 2023-07-14T11:54:11.714 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.371758818 2023-07-14T11:54:11.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.714 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2023-07-14T11:54:11.715 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 9, 2023-07-14T11:54:11.715 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.011819160, 2023-07-14T11:54:11.715 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001313240 2023-07-14T11:54:11.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.715 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2023-07-14T11:54:11.716 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 56954, 2023-07-14T11:54:11.716 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 552.939740985, 2023-07-14T11:54:11.716 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.009708532 2023-07-14T11:54:11.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.716 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2023-07-14T11:54:11.716 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 9780, 2023-07-14T11:54:11.717 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 32.905071968, 2023-07-14T11:54:11.717 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003364526 2023-07-14T11:54:11.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.717 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2023-07-14T11:54:11.717 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.718 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.718 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.718 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2023-07-14T11:54:11.718 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.719 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.719 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.719 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2023-07-14T11:54:11.719 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2023-07-14T11:54:11.720 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000761059, 2023-07-14T11:54:11.720 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000761059 2023-07-14T11:54:11.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.720 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2023-07-14T11:54:11.720 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 26, 2023-07-14T11:54:11.721 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 5.250324566, 2023-07-14T11:54:11.721 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.201935560 2023-07-14T11:54:11.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.721 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2023-07-14T11:54:11.721 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.721 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.722 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.722 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2023-07-14T11:54:11.722 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 58, 2023-07-14T11:54:11.722 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.712765445, 2023-07-14T11:54:11.723 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.029530438 2023-07-14T11:54:11.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.723 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2023-07-14T11:54:11.723 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.723 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.724 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.724 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2023-07-14T11:54:11.724 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2000, 2023-07-14T11:54:11.724 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2.757193016, 2023-07-14T11:54:11.725 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001378596 2023-07-14T11:54:11.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.725 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2023-07-14T11:54:11.725 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.725 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.726 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.726 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2023-07-14T11:54:11.726 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7, 2023-07-14T11:54:11.726 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.018110675, 2023-07-14T11:54:11.727 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002587239 2023-07-14T11:54:11.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.727 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2023-07-14T11:54:11.727 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.727 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.727 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.728 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2023-07-14T11:54:11.728 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 37533, 2023-07-14T11:54:11.728 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 137.564462814, 2023-07-14T11:54:11.728 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003665160 2023-07-14T11:54:11.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.729 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2023-07-14T11:54:11.729 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2023-07-14T11:54:11.729 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2023-07-14T11:54:11.729 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.730 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.730 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.730 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2023-07-14T11:54:11.731 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 2, 2023-07-14T11:54:11.731 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 5, 2023-07-14T11:54:11.731 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 3, 2023-07-14T11:54:11.731 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 2, 2023-07-14T11:54:11.731 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2023-07-14T11:54:11.731 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 29, 2023-07-14T11:54:11.732 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 14, 2023-07-14T11:54:11.732 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1902 2023-07-14T11:54:11.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.732 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2023-07-14T11:54:11.732 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:11.733 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:11.733 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:11.733 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:11.733 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:11.733 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:11.734 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:11.734 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:11.734 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:11.734 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:11.734 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:11.734 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:11.735 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:11.735 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:11.735 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:11.735 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:11.735 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:11.736 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:11.736 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:11.736 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:11.736 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:11.736 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:11.737 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:11.737 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:11.737 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:11.737 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:11.737 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:11.738 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:11.738 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:11.738 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:11.738 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:11.738 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2023-07-14T11:54:11.738 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:11.739 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:11.739 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:11.739 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:11.739 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 405541, 2023-07-14T11:54:11.739 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 565, 2023-07-14T11:54:11.740 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 880, 2023-07-14T11:54:11.740 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 10, 2023-07-14T11:54:11.740 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2023-07-14T11:54:11.740 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2023-07-14T11:54:11.740 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:11.741 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:11.741 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:11.741 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:11.741 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 9512, 2023-07-14T11:54:11.741 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 182, 2023-07-14T11:54:11.742 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:11.742 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:11.742 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:11.742 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2023-07-14T11:54:11.742 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 4722790, 2023-07-14T11:54:11.742 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 49619, 2023-07-14T11:54:11.743 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:11.743 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2023-07-14T11:54:11.743 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:11.743 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2023-07-14T11:54:11.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.744 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2023-07-14T11:54:11.744 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2023-07-14T11:54:11.744 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2023-07-14T11:54:11.744 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 176135, 2023-07-14T11:54:11.744 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 1292874991, 2023-07-14T11:54:11.745 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 37, 2023-07-14T11:54:11.745 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 176135, 2023-07-14T11:54:11.745 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2023-07-14T11:54:11.745 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 176135, 2023-07-14T11:54:11.745 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1584.199632415, 2023-07-14T11:54:11.746 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.008994235 2023-07-14T11:54:11.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.746 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2023-07-14T11:54:11.746 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2023-07-14T11:54:11.746 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 176135, 2023-07-14T11:54:11.746 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 177721 2023-07-14T11:54:11.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.747 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 176135, 2023-07-14T11:54:11.747 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 3802, 2023-07-14T11:54:11.747 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 172333, 2023-07-14T11:54:11.747 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2023-07-14T11:54:11.748 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2023-07-14T11:54:11.748 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 1, 2023-07-14T11:54:11.748 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 546, 2023-07-14T11:54:11.748 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 984, 2023-07-14T11:54:11.748 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 94505, 2023-07-14T11:54:11.749 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 593, 2023-07-14T11:54:11.749 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2023-07-14T11:54:11.749 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2023-07-14T11:54:11.749 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2023-07-14T11:54:11.749 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 18, 2023-07-14T11:54:11.749 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 76196, 2023-07-14T11:54:11.750 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2023-07-14T11:54:11.750 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2023-07-14T11:54:11.750 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2023-07-14T11:54:11.750 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 2818, 2023-07-14T11:54:11.750 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 870, 2023-07-14T11:54:11.751 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2023-07-14T11:54:11.751 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2023-07-14T11:54:11.751 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2023-07-14T11:54:11.751 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2023-07-14T11:54:11.751 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2023-07-14T11:54:11.752 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2023-07-14T11:54:11.752 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T11:54:11.752 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2023-07-14T11:54:11.752 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2023-07-14T11:54:11.752 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2023-07-14T11:54:11.752 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2023-07-14T11:54:11.753 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2023-07-14T11:54:11.753 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2023-07-14T11:54:11.753 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2023-07-14T11:54:11.753 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2023-07-14T11:54:11.753 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2023-07-14T11:54:11.754 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2023-07-14T11:54:11.754 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2023-07-14T11:54:11.754 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2023-07-14T11:54:11.754 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2023-07-14T11:54:11.754 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2023-07-14T11:54:11.755 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2023-07-14T11:54:11.755 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2023-07-14T11:54:11.755 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2023-07-14T11:54:11.755 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2023-07-14T11:54:11.755 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2023-07-14T11:54:11.755 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 143, 2023-07-14T11:54:11.756 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2023-07-14T11:54:11.756 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 60, 2023-07-14T11:54:11.756 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 12, 2023-07-14T11:54:11.756 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2023-07-14T11:54:11.756 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2023-07-14T11:54:11.757 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2023-07-14T11:54:11.757 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 905, 2023-07-14T11:54:11.757 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 14, 2023-07-14T11:54:11.757 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 271 2023-07-14T11:54:11.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.758 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2023-07-14T11:54:11.758 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2023-07-14T11:54:11.758 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2023-07-14T11:54:11.758 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2023-07-14T11:54:11.758 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2023-07-14T11:54:11.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.759 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2023-07-14T11:54:11.759 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 280733 2023-07-14T11:54:11.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.759 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2023-07-14T11:54:11.759 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2023-07-14T11:54:11.760 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 16, 2023-07-14T11:54:11.760 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2023-07-14T11:54:11.760 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 8, 2023-07-14T11:54:11.760 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 37533, 2023-07-14T11:54:11.760 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2023-07-14T11:54:11.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.761 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:11.761 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:11.761 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2023-07-14T11:54:11.761 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:11.762 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 3759611, 2023-07-14T11:54:11.762 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 366238214, 2023-07-14T11:54:11.762 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:11.762 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 3759611, 2023-07-14T11:54:11.762 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:11.762 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:11.763 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 3759611, 2023-07-14T11:54:11.763 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 366238214, 2023-07-14T11:54:11.763 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:11.763 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.763 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.764 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.764 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2023-07-14T11:54:11.764 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:11.765 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2023-07-14T11:54:11.765 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:11.765 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:11.765 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:11.765 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:11.766 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:11.766 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 176135, 2023-07-14T11:54:11.766 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 1296970264, 2023-07-14T11:54:11.766 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 176135, 2023-07-14T11:54:11.766 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1296970264, 2023-07-14T11:54:11.767 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:11.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.767 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.767 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.768 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2023-07-14T11:54:11.768 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:11.768 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2023-07-14T11:54:11.768 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:11.768 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:11.769 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:11.769 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:11.769 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:11.769 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 176135, 2023-07-14T11:54:11.769 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 176135, 2023-07-14T11:54:11.770 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 176135, 2023-07-14T11:54:11.770 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 176135, 2023-07-14T11:54:11.770 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:11.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.770 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2023-07-14T11:54:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2023-07-14T11:54:11.772 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:11.772 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 37533, 2023-07-14T11:54:11.772 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 4095361, 2023-07-14T11:54:11.772 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:11.772 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 37533, 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 967, 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 4095361, 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:11.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.774 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.774 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:11.774 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x558e7b46c0a0": { 2023-07-14T11:54:11.775 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:11.775 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2023-07-14T11:54:11.775 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:11.775 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 211717, 2023-07-14T11:54:11.775 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1283317945, 2023-07-14T11:54:11.776 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:11.776 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 211717, 2023-07-14T11:54:11.776 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:11.776 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:11.776 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 93538, 2023-07-14T11:54:11.777 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1283317945, 2023-07-14T11:54:11.777 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:11.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:11.777 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:11.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:11.777 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.778 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:11.778 INFO:teuthology.orchestra.run.smithi175.stdout:} 2023-07-14T11:54:12.053 INFO:tasks.check_counter:Daemon mds.b mds.exported=29 2023-07-14T11:54:12.053 INFO:tasks.check_counter:Daemon mds.b mds.imported=13 2023-07-14T11:54:12.054 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-07-14T11:54:12.054 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=32 2023-07-14T11:54:12.054 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=26 2023-07-14T11:54:12.054 DEBUG:tasks.check_counter:Getting stats from e 2023-07-14T11:54:12.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-14T11:54:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:54:13.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:12 smithi175 ceph-mon[129081]: pgmap v1011: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:54:13.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:12 smithi175 ceph-mon[129081]: osdmap e144: 12 total, 12 up, 12 in 2023-07-14T11:54:13.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:12 smithi182 ceph-mon[129366]: pgmap v1011: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:54:13.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:12 smithi182 ceph-mon[129366]: osdmap e144: 12 total, 12 up, 12 in 2023-07-14T11:54:13.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:12 smithi161 ceph-mon[95526]: pgmap v1011: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-07-14T11:54:13.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:12 smithi161 ceph-mon[95526]: osdmap e144: 12 total, 12 up, 12 in 2023-07-14T11:54:13.974 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2023-07-14T11:54:13.974 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:13.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 2724, 2023-07-14T11:54:13.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2724, 2023-07-14T11:54:13.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 224147717, 2023-07-14T11:54:13.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 846432, 2023-07-14T11:54:13.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2023-07-14T11:54:13.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2023-07-14T11:54:13.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.266728392, 2023-07-14T11:54:13.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.042537941, 2023-07-14T11:54:13.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.146642417, 2023-07-14T11:54:13.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.064702785, 2023-07-14T11:54:13.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:13.977 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2723, 2023-07-14T11:54:13.977 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.149374006, 2023-07-14T11:54:13.977 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000054856 2023-07-14T11:54:13.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.977 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:13.977 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:13.978 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:13.978 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:13.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-14T11:54:13.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T11:54:13.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.979 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:13.979 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1637, 2023-07-14T11:54:13.979 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1987, 2023-07-14T11:54:13.979 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 186540735, 2023-07-14T11:54:13.980 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1468827, 2023-07-14T11:54:13.980 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2023-07-14T11:54:13.980 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2023-07-14T11:54:13.980 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.262418851, 2023-07-14T11:54:13.980 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.050250379, 2023-07-14T11:54:13.981 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.173986285, 2023-07-14T11:54:13.981 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.036436356, 2023-07-14T11:54:13.981 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:13.981 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1986, 2023-07-14T11:54:13.981 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.142290398, 2023-07-14T11:54:13.982 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000071646 2023-07-14T11:54:13.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.982 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:13.982 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:13.982 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:13.983 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:13.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.983 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1696320, 2023-07-14T11:54:13.983 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 1162176 2023-07-14T11:54:13.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.983 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:13.984 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 3499, 2023-07-14T11:54:13.984 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 3536, 2023-07-14T11:54:13.984 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 290471761, 2023-07-14T11:54:13.984 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1214454, 2023-07-14T11:54:13.984 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2023-07-14T11:54:13.985 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2023-07-14T11:54:13.985 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.428495719, 2023-07-14T11:54:13.985 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.065154772, 2023-07-14T11:54:13.985 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.235442972, 2023-07-14T11:54:13.985 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.104006502, 2023-07-14T11:54:13.986 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:13.986 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3533, 2023-07-14T11:54:13.986 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.225292599, 2023-07-14T11:54:13.986 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000063768 2023-07-14T11:54:13.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.987 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:13.987 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:13.987 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:13.987 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:13.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.987 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-14T11:54:13.988 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 126576 2023-07-14T11:54:13.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.988 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2023-07-14T11:54:13.988 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2023-07-14T11:54:13.988 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2023-07-14T11:54:13.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.989 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2023-07-14T11:54:13.989 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2023-07-14T11:54:13.989 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2023-07-14T11:54:13.989 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 12069, 2023-07-14T11:54:13.990 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.502543718, 2023-07-14T11:54:13.990 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000041639 2023-07-14T11:54:13.990 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:13.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.990 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2023-07-14T11:54:13.991 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2023-07-14T11:54:13.991 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2023-07-14T11:54:13.991 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:13.991 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:13.991 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:13.991 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:13.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.992 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2023-07-14T11:54:13.992 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2023-07-14T11:54:13.992 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2023-07-14T11:54:13.992 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2023-07-14T11:54:13.993 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:13.993 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:13.993 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:13.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:13.993 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2023-07-14T11:54:13.994 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2023-07-14T11:54:13.994 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2023-07-14T11:54:13.994 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2023-07-14T11:54:13.994 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2023-07-14T11:54:13.994 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2023-07-14T11:54:13.994 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2023-07-14T11:54:13.995 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 22751, 2023-07-14T11:54:13.995 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 1945, 2023-07-14T11:54:13.995 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 19733, 2023-07-14T11:54:13.995 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 1073, 2023-07-14T11:54:13.995 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 19972, 2023-07-14T11:54:13.996 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 14882, 2023-07-14T11:54:13.996 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2023-07-14T11:54:13.996 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2023-07-14T11:54:13.996 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 17, 2023-07-14T11:54:13.997 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2023-07-14T11:54:13.997 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2023-07-14T11:54:13.997 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2023-07-14T11:54:13.997 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2023-07-14T11:54:13.997 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2023-07-14T11:54:13.998 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2023-07-14T11:54:13.998 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2023-07-14T11:54:13.998 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2023-07-14T11:54:13.998 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2023-07-14T11:54:13.998 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2023-07-14T11:54:13.999 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2023-07-14T11:54:13.999 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2023-07-14T11:54:13.999 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2023-07-14T11:54:13.999 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2023-07-14T11:54:13.999 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2023-07-14T11:54:14.000 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2023-07-14T11:54:14.000 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2023-07-14T11:54:14.000 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2023-07-14T11:54:14.000 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2023-07-14T11:54:14.000 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T11:54:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2023-07-14T11:54:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T11:54:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2023-07-14T11:54:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2023-07-14T11:54:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T11:54:14.002 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T11:54:14.002 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2023-07-14T11:54:14.002 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2023-07-14T11:54:14.002 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T11:54:14.002 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2023-07-14T11:54:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T11:54:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T11:54:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T11:54:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2023-07-14T11:54:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2023-07-14T11:54:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2023-07-14T11:54:14.004 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2023-07-14T11:54:14.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.004 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2023-07-14T11:54:14.004 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2023-07-14T11:54:14.004 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2023-07-14T11:54:14.005 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2023-07-14T11:54:14.005 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2023-07-14T11:54:14.005 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2023-07-14T11:54:14.005 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2023-07-14T11:54:14.005 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2023-07-14T11:54:14.006 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2023-07-14T11:54:14.006 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2023-07-14T11:54:14.006 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2023-07-14T11:54:14.006 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2023-07-14T11:54:14.006 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2023-07-14T11:54:14.007 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2023-07-14T11:54:14.007 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2023-07-14T11:54:14.007 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2023-07-14T11:54:14.007 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2023-07-14T11:54:14.007 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2023-07-14T11:54:14.007 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T11:54:14.008 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2023-07-14T11:54:14.008 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2023-07-14T11:54:14.008 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2023-07-14T11:54:14.008 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2023-07-14T11:54:14.008 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2023-07-14T11:54:14.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.009 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2023-07-14T11:54:14.009 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2023-07-14T11:54:14.009 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2023-07-14T11:54:14.009 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2023-07-14T11:54:14.010 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 0, 2023-07-14T11:54:14.010 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2023-07-14T11:54:14.010 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2023-07-14T11:54:14.010 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2023-07-14T11:54:14.010 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2023-07-14T11:54:14.011 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2023-07-14T11:54:14.011 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 128, 2023-07-14T11:54:14.011 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2023-07-14T11:54:14.011 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2023-07-14T11:54:14.011 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 184557839, 2023-07-14T11:54:14.011 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 0, 2023-07-14T11:54:14.012 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 701584573, 2023-07-14T11:54:14.012 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2023-07-14T11:54:14.012 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.012 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.012 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.013 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 164645 2023-07-14T11:54:14.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.013 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2023-07-14T11:54:14.013 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 1750, 2023-07-14T11:54:14.014 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 37382, 2023-07-14T11:54:14.014 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 35632, 2023-07-14T11:54:14.014 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 102, 2023-07-14T11:54:14.014 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 119, 2023-07-14T11:54:14.014 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 17, 2023-07-14T11:54:14.015 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 22751, 2023-07-14T11:54:14.015 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 37791, 2023-07-14T11:54:14.015 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 15040, 2023-07-14T11:54:14.015 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2023-07-14T11:54:14.015 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2023-07-14T11:54:14.016 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2023-07-14T11:54:14.016 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 78324, 2023-07-14T11:54:14.016 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 207156 2023-07-14T11:54:14.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.016 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2023-07-14T11:54:14.016 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2023-07-14T11:54:14.017 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2023-07-14T11:54:14.017 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2023-07-14T11:54:14.017 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2023-07-14T11:54:14.017 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2023-07-14T11:54:14.017 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2023-07-14T11:54:14.018 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.018 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.018 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.018 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2023-07-14T11:54:14.019 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.019 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.019 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.019 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2023-07-14T11:54:14.020 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.020 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.020 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.020 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2023-07-14T11:54:14.021 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.021 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.021 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.021 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2023-07-14T11:54:14.021 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.022 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.022 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.022 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2023-07-14T11:54:14.022 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.023 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.023 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.023 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2023-07-14T11:54:14.023 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.024 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.024 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.024 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2023-07-14T11:54:14.024 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.025 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.025 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.025 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2023-07-14T11:54:14.025 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.026 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.026 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.026 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2023-07-14T11:54:14.026 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.026 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.027 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.027 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2023-07-14T11:54:14.027 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.027 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.028 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.028 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2023-07-14T11:54:14.028 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.028 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.029 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.029 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2023-07-14T11:54:14.029 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.029 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.030 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.030 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2023-07-14T11:54:14.030 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.030 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.031 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.031 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2023-07-14T11:54:14.031 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.031 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.032 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.032 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2023-07-14T11:54:14.032 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.032 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.033 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.033 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2023-07-14T11:54:14.033 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.033 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.033 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.034 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2023-07-14T11:54:14.034 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.034 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.034 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.035 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2023-07-14T11:54:14.035 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.035 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.035 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.036 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2023-07-14T11:54:14.036 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.036 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.036 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.037 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2023-07-14T11:54:14.037 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.037 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.037 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.038 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2023-07-14T11:54:14.038 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.038 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.038 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.039 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2023-07-14T11:54:14.039 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.039 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.039 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.040 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2023-07-14T11:54:14.040 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.040 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.040 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.041 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2023-07-14T11:54:14.041 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.041 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.041 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.042 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2023-07-14T11:54:14.042 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.042 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.042 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.043 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2023-07-14T11:54:14.043 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.043 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.043 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.044 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2023-07-14T11:54:14.044 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.044 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.044 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.045 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2023-07-14T11:54:14.045 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.045 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.045 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.045 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2023-07-14T11:54:14.046 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2023-07-14T11:54:14.046 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2023-07-14T11:54:14.046 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.046 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.046 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.047 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:14.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.047 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2023-07-14T11:54:14.047 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 2, 2023-07-14T11:54:14.047 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 4, 2023-07-14T11:54:14.048 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2023-07-14T11:54:14.048 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 4, 2023-07-14T11:54:14.048 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2023-07-14T11:54:14.048 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2023-07-14T11:54:14.048 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2023-07-14T11:54:14.049 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 7956 2023-07-14T11:54:14.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.049 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2023-07-14T11:54:14.049 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:14.049 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:14.049 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:14.050 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:14.050 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:14.050 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:14.050 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:14.050 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:14.051 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:14.051 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:14.051 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:14.051 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:14.051 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:14.052 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:14.052 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:14.052 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:14.052 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:14.052 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:14.053 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:14.053 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:14.053 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:14.053 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:14.053 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:14.053 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:14.054 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:14.054 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:14.054 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:14.054 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:14.054 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:14.055 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:14.055 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:14.055 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2023-07-14T11:54:14.055 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:14.055 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:14.056 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:14.056 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:14.056 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 313721, 2023-07-14T11:54:14.056 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 2200, 2023-07-14T11:54:14.056 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:54:14.057 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2023-07-14T11:54:14.057 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2023-07-14T11:54:14.057 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2023-07-14T11:54:14.057 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:14.057 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:14.057 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:14.058 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:14.058 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 9752, 2023-07-14T11:54:14.058 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2023-07-14T11:54:14.058 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:14.058 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:14.059 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:14.059 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2023-07-14T11:54:14.059 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 20648225, 2023-07-14T11:54:14.059 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 139337, 2023-07-14T11:54:14.059 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:14.060 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2023-07-14T11:54:14.060 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:14.060 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2023-07-14T11:54:14.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.060 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2023-07-14T11:54:14.061 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2023-07-14T11:54:14.061 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2023-07-14T11:54:14.061 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 7204, 2023-07-14T11:54:14.061 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 21, 2023-07-14T11:54:14.061 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2023-07-14T11:54:14.062 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 7204, 2023-07-14T11:54:14.062 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2023-07-14T11:54:14.062 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7204, 2023-07-14T11:54:14.062 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 14.604757944, 2023-07-14T11:54:14.062 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002027312 2023-07-14T11:54:14.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.063 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2023-07-14T11:54:14.063 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2023-07-14T11:54:14.063 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7204, 2023-07-14T11:54:14.063 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 7205 2023-07-14T11:54:14.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.064 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 7204, 2023-07-14T11:54:14.064 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 7204, 2023-07-14T11:54:14.064 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2023-07-14T11:54:14.064 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2023-07-14T11:54:14.064 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2023-07-14T11:54:14.065 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 4080, 2023-07-14T11:54:14.065 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2023-07-14T11:54:14.065 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 3122, 2023-07-14T11:54:14.065 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2023-07-14T11:54:14.065 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2023-07-14T11:54:14.066 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2023-07-14T11:54:14.066 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2023-07-14T11:54:14.066 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2023-07-14T11:54:14.066 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2023-07-14T11:54:14.066 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2023-07-14T11:54:14.066 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2023-07-14T11:54:14.067 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2023-07-14T11:54:14.067 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2023-07-14T11:54:14.067 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1, 2023-07-14T11:54:14.067 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2023-07-14T11:54:14.067 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2023-07-14T11:54:14.068 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2023-07-14T11:54:14.068 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2023-07-14T11:54:14.068 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2023-07-14T11:54:14.068 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2023-07-14T11:54:14.068 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2023-07-14T11:54:14.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T11:54:14.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2023-07-14T11:54:14.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2023-07-14T11:54:14.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2023-07-14T11:54:14.069 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2023-07-14T11:54:14.070 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2023-07-14T11:54:14.070 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2023-07-14T11:54:14.070 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2023-07-14T11:54:14.070 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2023-07-14T11:54:14.070 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2023-07-14T11:54:14.070 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2023-07-14T11:54:14.071 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2023-07-14T11:54:14.071 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2023-07-14T11:54:14.071 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2023-07-14T11:54:14.071 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2023-07-14T11:54:14.071 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2023-07-14T11:54:14.072 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2023-07-14T11:54:14.072 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2023-07-14T11:54:14.072 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2023-07-14T11:54:14.072 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2023-07-14T11:54:14.072 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 144, 2023-07-14T11:54:14.073 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2023-07-14T11:54:14.073 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 56, 2023-07-14T11:54:14.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 11, 2023-07-14T11:54:14.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 11, 2023-07-14T11:54:14.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2023-07-14T11:54:14.074 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2023-07-14T11:54:14.074 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2023-07-14T11:54:14.074 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2023-07-14T11:54:14.074 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2023-07-14T11:54:14.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.074 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2023-07-14T11:54:14.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2023-07-14T11:54:14.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2023-07-14T11:54:14.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2023-07-14T11:54:14.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2023-07-14T11:54:14.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.076 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2023-07-14T11:54:14.076 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2023-07-14T11:54:14.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.076 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2023-07-14T11:54:14.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2023-07-14T11:54:14.077 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T11:54:14.077 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2023-07-14T11:54:14.077 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2023-07-14T11:54:14.077 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2023-07-14T11:54:14.077 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2023-07-14T11:54:14.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.078 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:14.078 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:14.078 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2023-07-14T11:54:14.078 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:14.078 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 7856, 2023-07-14T11:54:14.079 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 700417184, 2023-07-14T11:54:14.079 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:14.079 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 7856, 2023-07-14T11:54:14.079 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:14.079 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:14.080 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 7856, 2023-07-14T11:54:14.080 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 700417184, 2023-07-14T11:54:14.080 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:14.080 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.080 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.081 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.081 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:14.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.081 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2023-07-14T11:54:14.081 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:14.082 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2023-07-14T11:54:14.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:14.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:14.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:14.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:14.083 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:14.083 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 7204, 2023-07-14T11:54:14.083 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 697462813, 2023-07-14T11:54:14.083 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 7204, 2023-07-14T11:54:14.083 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 697462813, 2023-07-14T11:54:14.083 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:14.084 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.084 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.084 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:14.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.085 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2023-07-14T11:54:14.085 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:14.085 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2023-07-14T11:54:14.085 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:14.085 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:14.086 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:14.086 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:14.086 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:14.086 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 7204, 2023-07-14T11:54:14.086 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 7204, 2023-07-14T11:54:14.087 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 7204, 2023-07-14T11:54:14.087 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 7204, 2023-07-14T11:54:14.087 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:14.087 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.087 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.087 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:14.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.088 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2023-07-14T11:54:14.088 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:14.088 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2023-07-14T11:54:14.089 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:14.089 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:14.089 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:14.089 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:14.089 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:14.090 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:14.090 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:14.090 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2023-07-14T11:54:14.090 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2023-07-14T11:54:14.090 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:14.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.091 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:14.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:14.091 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x56362103c460": { 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:14.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:14.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:14.095 INFO:teuthology.orchestra.run.smithi175.stdout:} 2023-07-14T11:54:14.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:14 smithi175 ceph-mon[129081]: pgmap v1013: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-07-14T11:54:14.480 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-14T11:54:14.480 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-07-14T11:54:14.480 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-07-14T11:54:14.481 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-07-14T11:54:14.481 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T11:54:14.481 DEBUG:tasks.check_counter:Getting stats from h 2023-07-14T11:54:14.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-14T11:54:14.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:14 smithi182 ceph-mon[129366]: pgmap v1013: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-07-14T11:54:14.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:14 smithi161 ceph-mon[95526]: pgmap v1013: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-07-14T11:54:15.078 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:54:16.845 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2023-07-14T11:54:16.845 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:16.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 477045, 2023-07-14T11:54:16.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 656506, 2023-07-14T11:54:16.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 159919268, 2023-07-14T11:54:16.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 627932417, 2023-07-14T11:54:16.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 10, 2023-07-14T11:54:16.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2023-07-14T11:54:16.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 29.805355881, 2023-07-14T11:54:16.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 13.539481661, 2023-07-14T11:54:16.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 19.063053860, 2023-07-14T11:54:16.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 1.377762599, 2023-07-14T11:54:16.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:16.848 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 656504, 2023-07-14T11:54:16.848 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 84.913371264, 2023-07-14T11:54:16.848 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000129341 2023-07-14T11:54:16.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.848 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:16.849 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 919909, 2023-07-14T11:54:16.849 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.135884513, 2023-07-14T11:54:16.849 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000147 2023-07-14T11:54:16.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.849 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 89006240, 2023-07-14T11:54:16.849 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 359492160 2023-07-14T11:54:16.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.850 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:16.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 40442, 2023-07-14T11:54:16.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 40525, 2023-07-14T11:54:16.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 11702389, 2023-07-14T11:54:16.851 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 250999172, 2023-07-14T11:54:16.851 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 41, 2023-07-14T11:54:16.851 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 8, 2023-07-14T11:54:16.851 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 3.077265514, 2023-07-14T11:54:16.851 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.904914131, 2023-07-14T11:54:16.852 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.630116172, 2023-07-14T11:54:16.852 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 1.381175315, 2023-07-14T11:54:16.852 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:16.852 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 40525, 2023-07-14T11:54:16.852 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 7.638065928, 2023-07-14T11:54:16.852 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000188477 2023-07-14T11:54:16.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.853 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:16.853 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 184, 2023-07-14T11:54:16.853 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000033159, 2023-07-14T11:54:16.853 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000180 2023-07-14T11:54:16.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.854 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1717568, 2023-07-14T11:54:16.854 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 387056 2023-07-14T11:54:16.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.854 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:16.855 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 6838608, 2023-07-14T11:54:16.855 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 6840852, 2023-07-14T11:54:16.855 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1091185445, 2023-07-14T11:54:16.855 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1362613647, 2023-07-14T11:54:16.855 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 17, 2023-07-14T11:54:16.855 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2023-07-14T11:54:16.856 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 148.833367754, 2023-07-14T11:54:16.856 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 56.244251573, 2023-07-14T11:54:16.856 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 100.727307856, 2023-07-14T11:54:16.856 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 1.413147496, 2023-07-14T11:54:16.856 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:16.857 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6840849, 2023-07-14T11:54:16.857 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 60.070684681, 2023-07-14T11:54:16.857 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000008781 2023-07-14T11:54:16.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.857 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:16.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 13597945, 2023-07-14T11:54:16.858 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.866172429, 2023-07-14T11:54:16.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000063 2023-07-14T11:54:16.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.858 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2288, 2023-07-14T11:54:16.859 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 978608 2023-07-14T11:54:16.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.859 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2023-07-14T11:54:16.859 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2023-07-14T11:54:16.859 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2023-07-14T11:54:16.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.860 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2023-07-14T11:54:16.860 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2023-07-14T11:54:16.860 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2023-07-14T11:54:16.860 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 82670, 2023-07-14T11:54:16.860 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 104.923397896, 2023-07-14T11:54:16.861 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001269183 2023-07-14T11:54:16.861 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.861 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2023-07-14T11:54:16.861 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2023-07-14T11:54:16.862 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2023-07-14T11:54:16.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 58854, 2023-07-14T11:54:16.862 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 9.861435820, 2023-07-14T11:54:16.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000167557 2023-07-14T11:54:16.862 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.863 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2023-07-14T11:54:16.863 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 193703, 2023-07-14T11:54:16.863 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 190066, 2023-07-14T11:54:16.863 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2023-07-14T11:54:16.863 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 190066, 2023-07-14T11:54:16.864 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 677.809386011, 2023-07-14T11:54:16.864 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003566179 2023-07-14T11:54:16.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.864 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2023-07-14T11:54:16.864 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 3637, 2023-07-14T11:54:16.865 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2023-07-14T11:54:16.865 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2023-07-14T11:54:16.865 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 48, 2023-07-14T11:54:16.865 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 4, 2023-07-14T11:54:16.865 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 2, 2023-07-14T11:54:16.865 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 2302, 2023-07-14T11:54:16.866 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 492, 2023-07-14T11:54:16.866 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 1798, 2023-07-14T11:54:16.866 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 12, 2023-07-14T11:54:16.866 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 1472, 2023-07-14T11:54:16.866 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 136, 2023-07-14T11:54:16.867 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 227, 2023-07-14T11:54:16.867 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 228, 2023-07-14T11:54:16.867 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 20, 2023-07-14T11:54:16.867 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 352038, 2023-07-14T11:54:16.867 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 185792, 2023-07-14T11:54:16.868 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 3634, 2023-07-14T11:54:16.868 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 24, 2023-07-14T11:54:16.868 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2023-07-14T11:54:16.868 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2023-07-14T11:54:16.868 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 3974, 2023-07-14T11:54:16.869 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 1000, 2023-07-14T11:54:16.869 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 1, 2023-07-14T11:54:16.869 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 3, 2023-07-14T11:54:16.869 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 180, 2023-07-14T11:54:16.869 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 20, 2023-07-14T11:54:16.869 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 1465, 2023-07-14T11:54:16.870 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2023-07-14T11:54:16.870 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 1783, 2023-07-14T11:54:16.870 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2023-07-14T11:54:16.870 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 755, 2023-07-14T11:54:16.870 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 229819610, 2023-07-14T11:54:16.871 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2023-07-14T11:54:16.871 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 1783, 2023-07-14T11:54:16.871 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2023-07-14T11:54:16.871 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T11:54:16.871 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2023-07-14T11:54:16.872 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 181, 2023-07-14T11:54:16.872 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T11:54:16.872 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 181, 2023-07-14T11:54:16.872 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 1602, 2023-07-14T11:54:16.872 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 9, 2023-07-14T11:54:16.873 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 2092, 2023-07-14T11:54:16.873 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 129588, 2023-07-14T11:54:16.873 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T11:54:16.873 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 1363, 2023-07-14T11:54:16.873 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T11:54:16.873 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 30170, 2023-07-14T11:54:16.874 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 18907, 2023-07-14T11:54:16.874 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 4396, 2023-07-14T11:54:16.874 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 161435 2023-07-14T11:54:16.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.874 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2023-07-14T11:54:16.875 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 68, 2023-07-14T11:54:16.875 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 120, 2023-07-14T11:54:16.875 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 66, 2023-07-14T11:54:16.875 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 164, 2023-07-14T11:54:16.875 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 112, 2023-07-14T11:54:16.876 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2023-07-14T11:54:16.876 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2023-07-14T11:54:16.876 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2023-07-14T11:54:16.876 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 35501, 2023-07-14T11:54:16.876 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 35501, 2023-07-14T11:54:16.876 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2023-07-14T11:54:16.877 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2023-07-14T11:54:16.877 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2023-07-14T11:54:16.877 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2023-07-14T11:54:16.877 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2023-07-14T11:54:16.877 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2023-07-14T11:54:16.878 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2023-07-14T11:54:16.878 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T11:54:16.878 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 3, 2023-07-14T11:54:16.878 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2023-07-14T11:54:16.878 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 6, 2023-07-14T11:54:16.879 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2023-07-14T11:54:16.879 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2023-07-14T11:54:16.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.879 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2023-07-14T11:54:16.879 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 164645, 2023-07-14T11:54:16.879 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 84276, 2023-07-14T11:54:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 84276, 2023-07-14T11:54:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 80369, 2023-07-14T11:54:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2023-07-14T11:54:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 212, 2023-07-14T11:54:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 232, 2023-07-14T11:54:16.881 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 104, 2023-07-14T11:54:16.881 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 104, 2023-07-14T11:54:16.881 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 129, 2023-07-14T11:54:16.881 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2023-07-14T11:54:16.881 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 1, 2023-07-14T11:54:16.882 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 184557839, 2023-07-14T11:54:16.882 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 701585147, 2023-07-14T11:54:16.882 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2023-07-14T11:54:16.882 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2023-07-14T11:54:16.882 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 44022, 2023-07-14T11:54:16.882 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 291.021113254, 2023-07-14T11:54:16.883 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006610810 2023-07-14T11:54:16.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.883 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2023-07-14T11:54:16.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.883 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2023-07-14T11:54:16.884 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 1765, 2023-07-14T11:54:16.884 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 37381, 2023-07-14T11:54:16.884 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 35616, 2023-07-14T11:54:16.884 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 152, 2023-07-14T11:54:16.884 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 222, 2023-07-14T11:54:16.885 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 70, 2023-07-14T11:54:16.885 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 2302, 2023-07-14T11:54:16.885 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 59672, 2023-07-14T11:54:16.885 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 57370, 2023-07-14T11:54:16.885 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 228, 2023-07-14T11:54:16.886 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 36752, 2023-07-14T11:54:16.886 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 36524, 2023-07-14T11:54:16.886 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 74348, 2023-07-14T11:54:16.886 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 198964 2023-07-14T11:54:16.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.886 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2023-07-14T11:54:16.887 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 312392, 2023-07-14T11:54:16.887 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 102, 2023-07-14T11:54:16.887 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 193703, 2023-07-14T11:54:16.887 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 2147, 2023-07-14T11:54:16.887 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 24157, 2023-07-14T11:54:16.888 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2023-07-14T11:54:16.888 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 35518, 2023-07-14T11:54:16.888 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 134.309142817, 2023-07-14T11:54:16.888 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003781438 2023-07-14T11:54:16.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.889 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2023-07-14T11:54:16.889 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2242, 2023-07-14T11:54:16.889 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 25.708672297, 2023-07-14T11:54:16.889 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.011466847 2023-07-14T11:54:16.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.890 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2023-07-14T11:54:16.890 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.890 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.890 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.891 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2023-07-14T11:54:16.891 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.891 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.891 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2023-07-14T11:54:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 45996, 2023-07-14T11:54:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 68.611866918, 2023-07-14T11:54:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001491692 2023-07-14T11:54:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2023-07-14T11:54:16.893 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.893 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.893 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.893 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2023-07-14T11:54:16.894 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2023-07-14T11:54:16.894 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000115507, 2023-07-14T11:54:16.894 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000115507 2023-07-14T11:54:16.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.894 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2023-07-14T11:54:16.895 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.895 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.895 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.895 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2023-07-14T11:54:16.896 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.896 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.896 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.896 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2023-07-14T11:54:16.896 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.897 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.897 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.897 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2023-07-14T11:54:16.898 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.898 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.898 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.898 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2023-07-14T11:54:16.898 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.899 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.899 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.899 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2023-07-14T11:54:16.899 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.900 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.900 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.900 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2023-07-14T11:54:16.900 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.901 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.901 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.901 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2023-07-14T11:54:16.901 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.902 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.902 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.902 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2023-07-14T11:54:16.902 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 63573, 2023-07-14T11:54:16.902 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 308.257214678, 2023-07-14T11:54:16.903 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004848870 2023-07-14T11:54:16.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.903 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2023-07-14T11:54:16.903 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6629, 2023-07-14T11:54:16.903 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 28.117238544, 2023-07-14T11:54:16.904 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004241550 2023-07-14T11:54:16.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.904 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2023-07-14T11:54:16.904 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.904 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.905 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.905 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2023-07-14T11:54:16.905 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.905 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.906 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.906 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2023-07-14T11:54:16.906 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.906 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.906 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.907 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2023-07-14T11:54:16.907 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.907 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.907 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.908 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2023-07-14T11:54:16.908 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.908 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.908 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.909 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2023-07-14T11:54:16.909 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 22, 2023-07-14T11:54:16.909 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.552883602, 2023-07-14T11:54:16.909 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.025131072 2023-07-14T11:54:16.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.910 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2023-07-14T11:54:16.910 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.910 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.910 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.911 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2023-07-14T11:54:16.911 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 584, 2023-07-14T11:54:16.911 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.019177127, 2023-07-14T11:54:16.911 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001745166 2023-07-14T11:54:16.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.912 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2023-07-14T11:54:16.912 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.912 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.912 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.913 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2023-07-14T11:54:16.913 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.913 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.913 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.914 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2023-07-14T11:54:16.914 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.914 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.914 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.915 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2023-07-14T11:54:16.915 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 35501, 2023-07-14T11:54:16.915 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 111.233074521, 2023-07-14T11:54:16.915 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003133237 2023-07-14T11:54:16.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.915 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2023-07-14T11:54:16.916 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2023-07-14T11:54:16.916 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2023-07-14T11:54:16.916 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.916 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.916 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.917 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.917 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2023-07-14T11:54:16.917 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 2, 2023-07-14T11:54:16.917 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 4, 2023-07-14T11:54:16.918 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2023-07-14T11:54:16.918 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 2, 2023-07-14T11:54:16.918 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2023-07-14T11:54:16.918 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 28654, 2023-07-14T11:54:16.918 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 14327, 2023-07-14T11:54:16.919 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1896 2023-07-14T11:54:16.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.919 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2023-07-14T11:54:16.919 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:16.919 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:16.919 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:16.920 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:16.920 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:16.920 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:16.920 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:16.920 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:16.921 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:16.921 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:16.921 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:16.921 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:16.921 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:16.922 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:16.922 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:16.922 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:16.922 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:16.922 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:16.922 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:16.923 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:16.923 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:16.923 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:16.923 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:16.923 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:16.924 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:16.924 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:16.924 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:16.924 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:16.924 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:16.925 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:16.925 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:16.925 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2023-07-14T11:54:16.925 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:16.925 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:16.926 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:16.926 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:16.926 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 356981, 2023-07-14T11:54:16.926 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 1523, 2023-07-14T11:54:16.926 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 440, 2023-07-14T11:54:16.926 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 5, 2023-07-14T11:54:16.927 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2023-07-14T11:54:16.927 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2023-07-14T11:54:16.927 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:16.927 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:16.927 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:16.928 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:16.928 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 9752, 2023-07-14T11:54:16.928 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2023-07-14T11:54:16.928 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:16.928 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:16.929 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:16.929 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2023-07-14T11:54:16.929 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 6431710, 2023-07-14T11:54:16.929 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 59766, 2023-07-14T11:54:16.929 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:16.930 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2023-07-14T11:54:16.930 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:16.930 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2023-07-14T11:54:16.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.930 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2023-07-14T11:54:16.930 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2023-07-14T11:54:16.931 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2023-07-14T11:54:16.931 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 119988, 2023-07-14T11:54:16.931 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 703698092, 2023-07-14T11:54:16.931 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2023-07-14T11:54:16.931 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 119988, 2023-07-14T11:54:16.932 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2023-07-14T11:54:16.932 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 119988, 2023-07-14T11:54:16.932 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1088.353854187, 2023-07-14T11:54:16.932 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.009070522 2023-07-14T11:54:16.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.933 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2023-07-14T11:54:16.933 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2023-07-14T11:54:16.933 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 119988, 2023-07-14T11:54:16.933 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 120480 2023-07-14T11:54:16.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.934 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 119988, 2023-07-14T11:54:16.934 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 2837, 2023-07-14T11:54:16.934 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 117151, 2023-07-14T11:54:16.934 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2023-07-14T11:54:16.934 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2023-07-14T11:54:16.935 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2023-07-14T11:54:16.935 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 21, 2023-07-14T11:54:16.935 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 1054, 2023-07-14T11:54:16.935 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 45076, 2023-07-14T11:54:16.935 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 557, 2023-07-14T11:54:16.935 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2023-07-14T11:54:16.936 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2023-07-14T11:54:16.936 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2023-07-14T11:54:16.936 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 4, 2023-07-14T11:54:16.936 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 71233, 2023-07-14T11:54:16.936 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2023-07-14T11:54:16.937 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2023-07-14T11:54:16.937 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2023-07-14T11:54:16.937 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1783, 2023-07-14T11:54:16.937 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 42, 2023-07-14T11:54:16.937 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2023-07-14T11:54:16.938 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2023-07-14T11:54:16.938 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2023-07-14T11:54:16.938 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2023-07-14T11:54:16.938 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2023-07-14T11:54:16.938 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2023-07-14T11:54:16.939 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T11:54:16.939 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2023-07-14T11:54:16.939 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2023-07-14T11:54:16.939 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2023-07-14T11:54:16.939 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2023-07-14T11:54:16.939 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2023-07-14T11:54:16.940 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2023-07-14T11:54:16.940 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2023-07-14T11:54:16.940 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2023-07-14T11:54:16.940 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2023-07-14T11:54:16.940 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2023-07-14T11:54:16.941 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2023-07-14T11:54:16.941 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2023-07-14T11:54:16.941 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2023-07-14T11:54:16.941 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2023-07-14T11:54:16.941 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2023-07-14T11:54:16.942 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2023-07-14T11:54:16.942 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2023-07-14T11:54:16.942 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2023-07-14T11:54:16.942 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2023-07-14T11:54:16.942 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 144, 2023-07-14T11:54:16.942 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2023-07-14T11:54:16.943 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 56, 2023-07-14T11:54:16.943 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 12, 2023-07-14T11:54:16.943 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2023-07-14T11:54:16.943 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2023-07-14T11:54:16.943 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 498, 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 212 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 193814 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 16, 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 8, 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 35501, 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 7356093, 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 590352467, 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 7356094, 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 7356093, 2023-07-14T11:54:16.950 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 590352399, 2023-07-14T11:54:16.950 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:16.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.950 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2023-07-14T11:54:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2023-07-14T11:54:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 119988, 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 707570717, 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 119988, 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 707570717, 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 119988, 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 119988, 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 119988, 2023-07-14T11:54:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 119988, 2023-07-14T11:54:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2023-07-14T11:54:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2023-07-14T11:54:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 35501, 2023-07-14T11:54:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 3872625, 2023-07-14T11:54:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 35501, 2023-07-14T11:54:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1054, 2023-07-14T11:54:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 3872625, 2023-07-14T11:54:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55595249c0a0": { 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2023-07-14T11:54:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 164645, 2023-07-14T11:54:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 697390843, 2023-07-14T11:54:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 164645, 2023-07-14T11:54:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 44022, 2023-07-14T11:54:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 697390843, 2023-07-14T11:54:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:16.965 INFO:teuthology.orchestra.run.smithi175.stdout:} 2023-07-14T11:54:16.975 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:16 smithi161 ceph-mon[95526]: pgmap v1014: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-07-14T11:54:17.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:16 smithi175 ceph-mon[129081]: pgmap v1014: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-07-14T11:54:17.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:16 smithi182 ceph-mon[129366]: pgmap v1014: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-07-14T11:54:18.850 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2023-07-14T11:54:18.850 INFO:tasks.check_counter:Daemon mds.h mds.imported=20 2023-07-14T11:54:18.850 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-07-14T11:54:18.851 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-07-14T11:54:18.851 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T11:54:18.851 DEBUG:tasks.check_counter:Getting stats from k 2023-07-14T11:54:18.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-14T11:54:18.975 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:18 smithi161 ceph-mon[95526]: pgmap v1015: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:54:18.976 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:18 smithi161 ceph-mon[95526]: osdmap e145: 12 total, 12 up, 12 in 2023-07-14T11:54:19.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:18 smithi175 ceph-mon[129081]: pgmap v1015: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:54:19.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:18 smithi175 ceph-mon[129081]: osdmap e145: 12 total, 12 up, 12 in 2023-07-14T11:54:19.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:18 smithi182 ceph-mon[129366]: pgmap v1015: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-14T11:54:19.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:18 smithi182 ceph-mon[129366]: osdmap e145: 12 total, 12 up, 12 in 2023-07-14T11:54:19.474 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.b/config 2023-07-14T11:54:20.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:20 smithi175 ceph-mon[129081]: pgmap v1017: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2023-07-14T11:54:21.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:20 smithi182 ceph-mon[129366]: pgmap v1017: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2023-07-14T11:54:21.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:20 smithi161 ceph-mon[95526]: pgmap v1017: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2023-07-14T11:54:21.428 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2023-07-14T11:54:21.428 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:21.428 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 0, 2023-07-14T11:54:21.428 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2, 2023-07-14T11:54:21.429 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 0, 2023-07-14T11:54:21.429 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 0, 2023-07-14T11:54:21.429 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2023-07-14T11:54:21.429 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2023-07-14T11:54:21.429 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.001178228, 2023-07-14T11:54:21.430 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T11:54:21.430 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.000205184, 2023-07-14T11:54:21.430 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:21.430 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:21.430 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:21.431 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:21.431 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:21.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.431 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:21.431 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:21.432 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:21.432 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:21.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.432 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T11:54:21.432 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T11:54:21.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.433 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:21.433 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 568, 2023-07-14T11:54:21.433 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 558, 2023-07-14T11:54:21.433 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1521552, 2023-07-14T11:54:21.434 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 320512, 2023-07-14T11:54:21.434 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2023-07-14T11:54:21.434 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 1, 2023-07-14T11:54:21.434 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.070381557, 2023-07-14T11:54:21.435 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.022833589, 2023-07-14T11:54:21.435 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.039659983, 2023-07-14T11:54:21.435 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.012975954, 2023-07-14T11:54:21.436 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:21.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 557, 2023-07-14T11:54:21.436 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.036111635, 2023-07-14T11:54:21.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000064832 2023-07-14T11:54:21.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.437 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:21.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:21.437 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:21.438 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:21.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.438 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1521552, 2023-07-14T11:54:21.438 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 320512 2023-07-14T11:54:21.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.439 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:21.439 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 8, 2023-07-14T11:54:21.439 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 431, 2023-07-14T11:54:21.439 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3584, 2023-07-14T11:54:21.439 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 379104, 2023-07-14T11:54:21.439 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2023-07-14T11:54:21.440 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2023-07-14T11:54:21.440 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.017772135, 2023-07-14T11:54:21.440 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.011935981, 2023-07-14T11:54:21.440 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.002532294, 2023-07-14T11:54:21.440 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:21.441 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:21.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 429, 2023-07-14T11:54:21.441 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.039311367, 2023-07-14T11:54:21.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000091634 2023-07-14T11:54:21.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.442 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:21.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:21.442 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:21.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:21.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.443 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3584, 2023-07-14T11:54:21.443 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 379104 2023-07-14T11:54:21.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.444 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2023-07-14T11:54:21.444 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2023-07-14T11:54:21.444 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2023-07-14T11:54:21.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.444 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2023-07-14T11:54:21.445 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:21.445 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:21.445 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:21.445 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:21.445 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:21.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:21.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:21.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:21.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:21.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:21.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:21.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:21.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:21.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:21.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:21.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:21.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:21.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:21.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:21.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:21.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:21.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:21.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:21.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:21.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:21.450 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:21.450 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:21.450 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:21.451 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:21.451 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:21.451 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:21.451 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2023-07-14T11:54:21.451 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:21.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:21.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:21.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:21.452 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 147908, 2023-07-14T11:54:21.452 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 28, 2023-07-14T11:54:21.453 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:54:21.453 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2023-07-14T11:54:21.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2023-07-14T11:54:21.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2023-07-14T11:54:21.454 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:21.454 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:21.454 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:21.454 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:21.454 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2023-07-14T11:54:21.454 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2023-07-14T11:54:21.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:21.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:21.455 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:21.455 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2023-07-14T11:54:21.455 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 928, 2023-07-14T11:54:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2023-07-14T11:54:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2023-07-14T11:54:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2023-07-14T11:54:21.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2023-07-14T11:54:21.457 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:21.457 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2023-07-14T11:54:21.457 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2023-07-14T11:54:21.457 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2023-07-14T11:54:21.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 572, 2023-07-14T11:54:21.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1441265, 2023-07-14T11:54:21.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:21.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 572, 2023-07-14T11:54:21.458 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2023-07-14T11:54:21.459 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2023-07-14T11:54:21.459 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 572, 2023-07-14T11:54:21.459 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1441265, 2023-07-14T11:54:21.459 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2023-07-14T11:54:21.459 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2023-07-14T11:54:21.460 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2023-07-14T11:54:21.460 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2023-07-14T11:54:21.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:21.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2023-07-14T11:54:21.460 INFO:teuthology.orchestra.run.smithi175.stdout:} 2023-07-14T11:54:21.844 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-14T11:54:21.845 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-14T11:54:21.846 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-07-14T11:54:21.846 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-07-14T11:54:21.847 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-07-14T11:54:21.847 DEBUG:tasks.check_counter:Getting stats from c 2023-07-14T11:54:21.848 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-14T11:54:22.504 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:54:23.061 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:22 smithi182 ceph-mon[129366]: pgmap v1018: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 7 op/s 2023-07-14T11:54:23.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:22 smithi175 ceph-mon[129081]: pgmap v1018: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 7 op/s 2023-07-14T11:54:23.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:22 smithi161 ceph-mon[95526]: pgmap v1018: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 7 op/s 2023-07-14T11:54:23.871 INFO:teuthology.orchestra.run.smithi182.stdout:{ 2023-07-14T11:54:23.872 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:23.872 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 3728, 2023-07-14T11:54:23.872 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 3728, 2023-07-14T11:54:23.873 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 525413205, 2023-07-14T11:54:23.873 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 1158622, 2023-07-14T11:54:23.873 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 8, 2023-07-14T11:54:23.873 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 4, 2023-07-14T11:54:23.873 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.533142162, 2023-07-14T11:54:23.874 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.069303158, 2023-07-14T11:54:23.874 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.335315226, 2023-07-14T11:54:23.874 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.103102840, 2023-07-14T11:54:23.875 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:23.875 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 3727, 2023-07-14T11:54:23.875 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.194622707, 2023-07-14T11:54:23.875 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000052219 2023-07-14T11:54:23.875 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.876 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:23.876 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.876 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.876 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.876 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.877 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-14T11:54:23.877 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T11:54:23.877 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.877 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:23.877 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 2734, 2023-07-14T11:54:23.878 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 2700, 2023-07-14T11:54:23.878 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 455207589, 2023-07-14T11:54:23.878 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 977609, 2023-07-14T11:54:23.878 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 6, 2023-07-14T11:54:23.878 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 5, 2023-07-14T11:54:23.879 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.409420124, 2023-07-14T11:54:23.879 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.062267796, 2023-07-14T11:54:23.879 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.273727492, 2023-07-14T11:54:23.879 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.068043058, 2023-07-14T11:54:23.879 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:23.880 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 2699, 2023-07-14T11:54:23.880 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.104984858, 2023-07-14T11:54:23.880 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000038897 2023-07-14T11:54:23.880 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.880 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:23.881 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.881 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.881 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.881 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.882 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 1697568, 2023-07-14T11:54:23.882 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 330352 2023-07-14T11:54:23.882 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.882 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:23.882 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 2125, 2023-07-14T11:54:23.883 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 2547, 2023-07-14T11:54:23.883 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 306620459, 2023-07-14T11:54:23.883 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 1623143, 2023-07-14T11:54:23.883 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 9, 2023-07-14T11:54:23.883 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 6, 2023-07-14T11:54:23.884 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.277243201, 2023-07-14T11:54:23.884 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.051545808, 2023-07-14T11:54:23.884 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.161128462, 2023-07-14T11:54:23.884 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.048952227, 2023-07-14T11:54:23.884 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:23.885 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 2544, 2023-07-14T11:54:23.885 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.098569328, 2023-07-14T11:54:23.885 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000038745 2023-07-14T11:54:23.885 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.885 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:23.886 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.886 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.886 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.886 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.886 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-14T11:54:23.887 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 963200 2023-07-14T11:54:23.887 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.887 INFO:teuthology.orchestra.run.smithi182.stdout: "cct": { 2023-07-14T11:54:23.887 INFO:teuthology.orchestra.run.smithi182.stdout: "total_workers": 1, 2023-07-14T11:54:23.887 INFO:teuthology.orchestra.run.smithi182.stdout: "unhealthy_workers": 0 2023-07-14T11:54:23.888 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.888 INFO:teuthology.orchestra.run.smithi182.stdout: "finisher-MDSRank": { 2023-07-14T11:54:23.888 INFO:teuthology.orchestra.run.smithi182.stdout: "queue_len": 0, 2023-07-14T11:54:23.888 INFO:teuthology.orchestra.run.smithi182.stdout: "complete_latency": { 2023-07-14T11:54:23.889 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 12914, 2023-07-14T11:54:23.889 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.544720556, 2023-07-14T11:54:23.889 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000042180 2023-07-14T11:54:23.889 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:23.889 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: "finisher-PurgeQueue": { 2023-07-14T11:54:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: "queue_len": 0, 2023-07-14T11:54:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: "complete_latency": { 2023-07-14T11:54:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.891 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.891 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:23.891 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.891 INFO:teuthology.orchestra.run.smithi182.stdout: "mds": { 2023-07-14T11:54:23.892 INFO:teuthology.orchestra.run.smithi182.stdout: "request": 0, 2023-07-14T11:54:23.892 INFO:teuthology.orchestra.run.smithi182.stdout: "reply": 0, 2023-07-14T11:54:23.892 INFO:teuthology.orchestra.run.smithi182.stdout: "reply_latency": { 2023-07-14T11:54:23.892 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.893 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.893 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.893 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.893 INFO:teuthology.orchestra.run.smithi182.stdout: "slow_reply": 0, 2023-07-14T11:54:23.893 INFO:teuthology.orchestra.run.smithi182.stdout: "forward": 0, 2023-07-14T11:54:23.894 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_fetch_complete": 0, 2023-07-14T11:54:23.894 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_fetch_keys": 0, 2023-07-14T11:54:23.894 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_commit": 0, 2023-07-14T11:54:23.894 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_split": 0, 2023-07-14T11:54:23.895 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_merge": 0, 2023-07-14T11:54:23.895 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes": 19536, 2023-07-14T11:54:23.895 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_top": 4, 2023-07-14T11:54:23.895 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_bottom": 19486, 2023-07-14T11:54:23.895 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_pin_tail": 46, 2023-07-14T11:54:23.895 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_pinned": 19529, 2023-07-14T11:54:23.896 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_expired": 18921, 2023-07-14T11:54:23.896 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_with_caps": 0, 2023-07-14T11:54:23.896 INFO:teuthology.orchestra.run.smithi182.stdout: "caps": 0, 2023-07-14T11:54:23.896 INFO:teuthology.orchestra.run.smithi182.stdout: "subtrees": 21, 2023-07-14T11:54:23.897 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse": 0, 2023-07-14T11:54:23.897 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_hit": 0, 2023-07-14T11:54:23.897 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_forward": 0, 2023-07-14T11:54:23.897 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_discover": 0, 2023-07-14T11:54:23.897 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_dir_fetch": 0, 2023-07-14T11:54:23.898 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_remote_ino": 0, 2023-07-14T11:54:23.898 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_lock": 0, 2023-07-14T11:54:23.898 INFO:teuthology.orchestra.run.smithi182.stdout: "load_cent": 0, 2023-07-14T11:54:23.899 INFO:teuthology.orchestra.run.smithi182.stdout: "q": 0, 2023-07-14T11:54:23.899 INFO:teuthology.orchestra.run.smithi182.stdout: "exported": 0, 2023-07-14T11:54:23.900 INFO:teuthology.orchestra.run.smithi182.stdout: "exported_inodes": 0, 2023-07-14T11:54:23.900 INFO:teuthology.orchestra.run.smithi182.stdout: "imported": 0, 2023-07-14T11:54:23.900 INFO:teuthology.orchestra.run.smithi182.stdout: "imported_inodes": 0, 2023-07-14T11:54:23.901 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_dir_fetch": 0, 2023-07-14T11:54:23.901 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_backtrace_fetch": 0, 2023-07-14T11:54:23.901 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_peer_discover": 0, 2023-07-14T11:54:23.901 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rfiles": 755, 2023-07-14T11:54:23.902 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rbytes": 229819610, 2023-07-14T11:54:23.902 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rsnaps": 6, 2023-07-14T11:54:23.902 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T11:54:23.903 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_set_tag": 0, 2023-07-14T11:54:23.903 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T11:54:23.903 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_inotable_repaired": 0, 2023-07-14T11:54:23.903 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dir_inodes": 0, 2023-07-14T11:54:23.904 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T11:54:23.904 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T11:54:23.904 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_file_inodes": 0, 2023-07-14T11:54:23.905 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_inode_file_caps": 0, 2023-07-14T11:54:23.905 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T11:54:23.905 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_grant": 0, 2023-07-14T11:54:23.905 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T11:54:23.906 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T11:54:23.906 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T11:54:23.906 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_caps": 0, 2023-07-14T11:54:23.906 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_caps_dirty": 0, 2023-07-14T11:54:23.907 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_cap_release": 0, 2023-07-14T11:54:23.907 INFO:teuthology.orchestra.run.smithi182.stdout: "process_request_cap_release": 0 2023-07-14T11:54:23.907 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.908 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_cache": { 2023-07-14T11:54:23.908 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_update": 0, 2023-07-14T11:54:23.908 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_update_receipt": 0, 2023-07-14T11:54:23.909 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_try_discover": 0, 2023-07-14T11:54:23.909 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_send_discover": 0, 2023-07-14T11:54:23.909 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_handle_discover": 0, 2023-07-14T11:54:23.909 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays": 0, 2023-07-14T11:54:23.910 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays_delayed": 0, 2023-07-14T11:54:23.910 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays_enqueuing": 0, 2023-07-14T11:54:23.910 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_created": 0, 2023-07-14T11:54:23.910 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_enqueued": 0, 2023-07-14T11:54:23.910 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_reintegrated": 0, 2023-07-14T11:54:23.911 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_migrated": 0, 2023-07-14T11:54:23.911 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_processing": 0, 2023-07-14T11:54:23.911 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_enqueued": 0, 2023-07-14T11:54:23.911 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_prioritized": 0, 2023-07-14T11:54:23.911 INFO:teuthology.orchestra.run.smithi182.stdout: "recovery_started": 0, 2023-07-14T11:54:23.912 INFO:teuthology.orchestra.run.smithi182.stdout: "recovery_completed": 0, 2023-07-14T11:54:23.912 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T11:54:23.912 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_exportdir": 0, 2023-07-14T11:54:23.912 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_flush": 0, 2023-07-14T11:54:23.913 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_fragmentdir": 0, 2023-07-14T11:54:23.913 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_fragstats": 0, 2023-07-14T11:54:23.913 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_inodestats": 0 2023-07-14T11:54:23.913 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.913 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_log": { 2023-07-14T11:54:23.914 INFO:teuthology.orchestra.run.smithi182.stdout: "evadd": 0, 2023-07-14T11:54:23.914 INFO:teuthology.orchestra.run.smithi182.stdout: "evex": 0, 2023-07-14T11:54:23.914 INFO:teuthology.orchestra.run.smithi182.stdout: "evtrm": 0, 2023-07-14T11:54:23.914 INFO:teuthology.orchestra.run.smithi182.stdout: "ev": 0, 2023-07-14T11:54:23.914 INFO:teuthology.orchestra.run.smithi182.stdout: "evexg": 0, 2023-07-14T11:54:23.915 INFO:teuthology.orchestra.run.smithi182.stdout: "evexd": 0, 2023-07-14T11:54:23.915 INFO:teuthology.orchestra.run.smithi182.stdout: "segadd": 0, 2023-07-14T11:54:23.915 INFO:teuthology.orchestra.run.smithi182.stdout: "segex": 0, 2023-07-14T11:54:23.915 INFO:teuthology.orchestra.run.smithi182.stdout: "segtrm": 0, 2023-07-14T11:54:23.915 INFO:teuthology.orchestra.run.smithi182.stdout: "seg": 129, 2023-07-14T11:54:23.915 INFO:teuthology.orchestra.run.smithi182.stdout: "segexg": 0, 2023-07-14T11:54:23.916 INFO:teuthology.orchestra.run.smithi182.stdout: "segexd": 0, 2023-07-14T11:54:23.916 INFO:teuthology.orchestra.run.smithi182.stdout: "expos": 780154585, 2023-07-14T11:54:23.916 INFO:teuthology.orchestra.run.smithi182.stdout: "wrpos": 0, 2023-07-14T11:54:23.916 INFO:teuthology.orchestra.run.smithi182.stdout: "rdpos": 1287512249, 2023-07-14T11:54:23.916 INFO:teuthology.orchestra.run.smithi182.stdout: "jlat": { 2023-07-14T11:54:23.917 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.917 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.917 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.918 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.918 INFO:teuthology.orchestra.run.smithi182.stdout: "replayed": 211718 2023-07-14T11:54:23.918 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.918 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_mem": { 2023-07-14T11:54:23.919 INFO:teuthology.orchestra.run.smithi182.stdout: "ino": 794, 2023-07-14T11:54:23.919 INFO:teuthology.orchestra.run.smithi182.stdout: "ino+": 39932, 2023-07-14T11:54:23.919 INFO:teuthology.orchestra.run.smithi182.stdout: "ino-": 39138, 2023-07-14T11:54:23.919 INFO:teuthology.orchestra.run.smithi182.stdout: "dir": 84, 2023-07-14T11:54:23.919 INFO:teuthology.orchestra.run.smithi182.stdout: "dir+": 334, 2023-07-14T11:54:23.920 INFO:teuthology.orchestra.run.smithi182.stdout: "dir-": 250, 2023-07-14T11:54:23.920 INFO:teuthology.orchestra.run.smithi182.stdout: "dn": 19536, 2023-07-14T11:54:23.920 INFO:teuthology.orchestra.run.smithi182.stdout: "dn+": 40520, 2023-07-14T11:54:23.920 INFO:teuthology.orchestra.run.smithi182.stdout: "dn-": 20984, 2023-07-14T11:54:23.920 INFO:teuthology.orchestra.run.smithi182.stdout: "cap": 0, 2023-07-14T11:54:23.921 INFO:teuthology.orchestra.run.smithi182.stdout: "cap+": 0, 2023-07-14T11:54:23.921 INFO:teuthology.orchestra.run.smithi182.stdout: "cap-": 0, 2023-07-14T11:54:23.921 INFO:teuthology.orchestra.run.smithi182.stdout: "rss": 77816, 2023-07-14T11:54:23.921 INFO:teuthology.orchestra.run.smithi182.stdout: "heap": 207156 2023-07-14T11:54:23.921 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.922 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_server": { 2023-07-14T11:54:23.922 INFO:teuthology.orchestra.run.smithi182.stdout: "dispatch_client_request": 0, 2023-07-14T11:54:23.922 INFO:teuthology.orchestra.run.smithi182.stdout: "dispatch_server_request": 0, 2023-07-14T11:54:23.922 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_request": 0, 2023-07-14T11:54:23.922 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_session": 0, 2023-07-14T11:54:23.923 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_peer_request": 0, 2023-07-14T11:54:23.923 INFO:teuthology.orchestra.run.smithi182.stdout: "req_create_latency": { 2023-07-14T11:54:23.923 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.923 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.923 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.924 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.924 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getattr_latency": { 2023-07-14T11:54:23.924 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.924 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.925 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.925 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.925 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getfilelock_latency": { 2023-07-14T11:54:23.925 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.925 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.926 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.926 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.926 INFO:teuthology.orchestra.run.smithi182.stdout: "req_link_latency": { 2023-07-14T11:54:23.926 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.926 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.927 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.927 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.927 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookup_latency": { 2023-07-14T11:54:23.927 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.927 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.928 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.928 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.928 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookuphash_latency": { 2023-07-14T11:54:23.928 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.928 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.929 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.929 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.929 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupino_latency": { 2023-07-14T11:54:23.929 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.929 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.930 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.930 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.930 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupname_latency": { 2023-07-14T11:54:23.930 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.930 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.931 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.931 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.931 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupparent_latency": { 2023-07-14T11:54:23.931 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.931 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.932 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.932 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.932 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupsnap_latency": { 2023-07-14T11:54:23.932 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.932 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.933 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.933 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.933 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lssnap_latency": { 2023-07-14T11:54:23.933 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.934 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.934 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.934 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.934 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mkdir_latency": { 2023-07-14T11:54:23.934 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.935 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.935 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.935 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.935 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mknod_latency": { 2023-07-14T11:54:23.935 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.936 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.936 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.936 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.936 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mksnap_latency": { 2023-07-14T11:54:23.936 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.937 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.937 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.937 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.937 INFO:teuthology.orchestra.run.smithi182.stdout: "req_open_latency": { 2023-07-14T11:54:23.938 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.938 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.938 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.938 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.939 INFO:teuthology.orchestra.run.smithi182.stdout: "req_readdir_latency": { 2023-07-14T11:54:23.939 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.939 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.940 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.940 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.940 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rename_latency": { 2023-07-14T11:54:23.941 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.941 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.941 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.941 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.942 INFO:teuthology.orchestra.run.smithi182.stdout: "req_renamesnap_latency": { 2023-07-14T11:54:23.942 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.942 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.942 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.942 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.942 INFO:teuthology.orchestra.run.smithi182.stdout: "req_snapdiff_latency": { 2023-07-14T11:54:23.943 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.943 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.943 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.943 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.944 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmdir_latency": { 2023-07-14T11:54:23.944 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.944 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.944 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.945 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.945 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmsnap_latency": { 2023-07-14T11:54:23.945 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.945 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.946 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.946 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.946 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmxattr_latency": { 2023-07-14T11:54:23.946 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.947 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.947 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.947 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.947 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setattr_latency": { 2023-07-14T11:54:23.947 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.948 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.948 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.948 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.948 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setdirlayout_latency": { 2023-07-14T11:54:23.948 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.949 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.949 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.949 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.949 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setfilelock_latency": { 2023-07-14T11:54:23.949 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.950 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.950 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.950 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.950 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setlayout_latency": { 2023-07-14T11:54:23.950 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.951 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.951 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.951 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.951 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setxattr_latency": { 2023-07-14T11:54:23.951 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.951 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.952 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.952 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.952 INFO:teuthology.orchestra.run.smithi182.stdout: "req_symlink_latency": { 2023-07-14T11:54:23.952 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.952 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.953 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.953 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.953 INFO:teuthology.orchestra.run.smithi182.stdout: "req_unlink_latency": { 2023-07-14T11:54:23.953 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.953 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.954 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.954 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.954 INFO:teuthology.orchestra.run.smithi182.stdout: "cap_revoke_eviction": 0, 2023-07-14T11:54:23.954 INFO:teuthology.orchestra.run.smithi182.stdout: "cap_acquisition_throttle": 0, 2023-07-14T11:54:23.955 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getvxattr_latency": { 2023-07-14T11:54:23.955 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.955 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.955 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.955 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:23.956 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.956 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_sessions": { 2023-07-14T11:54:23.956 INFO:teuthology.orchestra.run.smithi182.stdout: "session_count": 2, 2023-07-14T11:54:23.956 INFO:teuthology.orchestra.run.smithi182.stdout: "session_add": 5, 2023-07-14T11:54:23.956 INFO:teuthology.orchestra.run.smithi182.stdout: "session_remove": 3, 2023-07-14T11:54:23.957 INFO:teuthology.orchestra.run.smithi182.stdout: "sessions_open": 3, 2023-07-14T11:54:23.957 INFO:teuthology.orchestra.run.smithi182.stdout: "sessions_stale": 0, 2023-07-14T11:54:23.957 INFO:teuthology.orchestra.run.smithi182.stdout: "total_load": 0, 2023-07-14T11:54:23.957 INFO:teuthology.orchestra.run.smithi182.stdout: "average_load": 0, 2023-07-14T11:54:23.957 INFO:teuthology.orchestra.run.smithi182.stdout: "avg_session_uptime": 5845 2023-07-14T11:54:23.958 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.958 INFO:teuthology.orchestra.run.smithi182.stdout: "mempool": { 2023-07-14T11:54:23.958 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:23.958 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:23.958 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:23.959 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:23.959 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:23.959 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:23.959 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:23.960 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:23.960 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:23.960 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:23.960 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:23.961 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:23.961 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:23.961 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:23.961 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:23.962 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:23.962 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:23.962 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:23.962 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:23.962 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:23.963 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:23.963 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:23.963 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:23.963 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:23.964 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:23.964 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:23.964 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:23.964 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:23.964 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:23.965 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:23.965 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:23.965 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_items": 0, 2023-07-14T11:54:23.965 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:23.965 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:23.966 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:23.966 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:23.966 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_bytes": 277847, 2023-07-14T11:54:23.966 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_items": 1187, 2023-07-14T11:54:23.966 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:54:23.967 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_items": 0, 2023-07-14T11:54:23.967 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_bytes": 0, 2023-07-14T11:54:23.967 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_items": 0, 2023-07-14T11:54:23.967 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:23.967 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:23.968 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:23.968 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:23.968 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_bytes": 9512, 2023-07-14T11:54:23.968 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_items": 182, 2023-07-14T11:54:23.969 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:23.969 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:23.969 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:23.969 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_items": 0, 2023-07-14T11:54:23.969 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_bytes": 16196605, 2023-07-14T11:54:23.970 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_items": 104069, 2023-07-14T11:54:23.970 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:23.970 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_items": 0, 2023-07-14T11:54:23.970 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:23.970 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_items": 0 2023-07-14T11:54:23.971 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.971 INFO:teuthology.orchestra.run.smithi182.stdout: "objecter": { 2023-07-14T11:54:23.971 INFO:teuthology.orchestra.run.smithi182.stdout: "op_active": 0, 2023-07-14T11:54:23.971 INFO:teuthology.orchestra.run.smithi182.stdout: "op_laggy": 0, 2023-07-14T11:54:23.971 INFO:teuthology.orchestra.run.smithi182.stdout: "op_send": 7927, 2023-07-14T11:54:23.972 INFO:teuthology.orchestra.run.smithi182.stdout: "op_send_bytes": 26, 2023-07-14T11:54:23.972 INFO:teuthology.orchestra.run.smithi182.stdout: "op_resend": 4, 2023-07-14T11:54:23.972 INFO:teuthology.orchestra.run.smithi182.stdout: "op_reply": 7927, 2023-07-14T11:54:23.972 INFO:teuthology.orchestra.run.smithi182.stdout: "op_latency": { 2023-07-14T11:54:23.972 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 7927, 2023-07-14T11:54:23.973 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 15.704633317, 2023-07-14T11:54:23.973 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.001981157 2023-07-14T11:54:23.973 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.973 INFO:teuthology.orchestra.run.smithi182.stdout: "op_inflight": 0, 2023-07-14T11:54:23.973 INFO:teuthology.orchestra.run.smithi182.stdout: "oplen_avg": { 2023-07-14T11:54:23.973 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 7927, 2023-07-14T11:54:23.974 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 7928 2023-07-14T11:54:23.974 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.974 INFO:teuthology.orchestra.run.smithi182.stdout: "op": 7927, 2023-07-14T11:54:23.974 INFO:teuthology.orchestra.run.smithi182.stdout: "op_r": 7927, 2023-07-14T11:54:23.974 INFO:teuthology.orchestra.run.smithi182.stdout: "op_w": 0, 2023-07-14T11:54:23.975 INFO:teuthology.orchestra.run.smithi182.stdout: "op_rmw": 0, 2023-07-14T11:54:23.975 INFO:teuthology.orchestra.run.smithi182.stdout: "op_pg": 0, 2023-07-14T11:54:23.975 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_stat": 4533, 2023-07-14T11:54:23.975 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_create": 0, 2023-07-14T11:54:23.975 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_read": 3391, 2023-07-14T11:54:23.976 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_write": 0, 2023-07-14T11:54:23.976 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_writefull": 0, 2023-07-14T11:54:23.976 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_writesame": 0, 2023-07-14T11:54:23.976 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_append": 0, 2023-07-14T11:54:23.976 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_zero": 0, 2023-07-14T11:54:23.977 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_truncate": 0, 2023-07-14T11:54:23.977 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_delete": 0, 2023-07-14T11:54:23.977 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_mapext": 0, 2023-07-14T11:54:23.977 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_sparse_read": 0, 2023-07-14T11:54:23.977 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_clonerange": 0, 2023-07-14T11:54:23.978 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_getxattr": 2, 2023-07-14T11:54:23.978 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_setxattr": 0, 2023-07-14T11:54:23.978 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_cmpxattr": 0, 2023-07-14T11:54:23.978 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_rmxattr": 0, 2023-07-14T11:54:23.978 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_resetxattrs": 0, 2023-07-14T11:54:23.979 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_call": 0, 2023-07-14T11:54:23.979 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_watch": 0, 2023-07-14T11:54:23.979 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_notify": 0, 2023-07-14T11:54:23.979 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T11:54:23.979 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_pgls": 0, 2023-07-14T11:54:23.980 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_pgls_filter": 0, 2023-07-14T11:54:23.980 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_other": 0, 2023-07-14T11:54:23.980 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_active": 0, 2023-07-14T11:54:23.980 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_send": 0, 2023-07-14T11:54:23.980 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_resend": 0, 2023-07-14T11:54:23.981 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_ping": 0, 2023-07-14T11:54:23.981 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_active": 0, 2023-07-14T11:54:23.981 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_send": 0, 2023-07-14T11:54:23.981 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_resend": 0, 2023-07-14T11:54:23.981 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_active": 0, 2023-07-14T11:54:23.981 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_send": 0, 2023-07-14T11:54:23.982 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_resend": 0, 2023-07-14T11:54:23.982 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_active": 0, 2023-07-14T11:54:23.982 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_send": 0, 2023-07-14T11:54:23.982 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_resend": 0, 2023-07-14T11:54:23.982 INFO:teuthology.orchestra.run.smithi182.stdout: "command_active": 0, 2023-07-14T11:54:23.983 INFO:teuthology.orchestra.run.smithi182.stdout: "command_send": 0, 2023-07-14T11:54:23.983 INFO:teuthology.orchestra.run.smithi182.stdout: "command_resend": 0, 2023-07-14T11:54:23.983 INFO:teuthology.orchestra.run.smithi182.stdout: "map_epoch": 145, 2023-07-14T11:54:23.983 INFO:teuthology.orchestra.run.smithi182.stdout: "map_full": 0, 2023-07-14T11:54:23.983 INFO:teuthology.orchestra.run.smithi182.stdout: "map_inc": 57, 2023-07-14T11:54:23.984 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_sessions": 11, 2023-07-14T11:54:23.984 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_session_open": 11, 2023-07-14T11:54:23.984 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_session_close": 0, 2023-07-14T11:54:23.984 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_laggy": 0, 2023-07-14T11:54:23.984 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_wr": 0, 2023-07-14T11:54:23.984 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_rd": 2, 2023-07-14T11:54:23.985 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_del": 0 2023-07-14T11:54:23.985 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.985 INFO:teuthology.orchestra.run.smithi182.stdout: "oft": { 2023-07-14T11:54:23.985 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_objs": 0, 2023-07-14T11:54:23.985 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_kv_pairs": 0, 2023-07-14T11:54:23.986 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_updates": 0, 2023-07-14T11:54:23.986 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_removes": 0 2023-07-14T11:54:23.986 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.986 INFO:teuthology.orchestra.run.smithi182.stdout: "osd-slow-ops": { 2023-07-14T11:54:23.986 INFO:teuthology.orchestra.run.smithi182.stdout: "slow_ops_count": 0 2023-07-14T11:54:23.987 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.987 INFO:teuthology.orchestra.run.smithi182.stdout: "purge_queue": { 2023-07-14T11:54:23.987 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_ops": 0, 2023-07-14T11:54:23.987 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T11:54:23.988 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing": 0, 2023-07-14T11:54:23.988 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_high_water": 0, 2023-07-14T11:54:23.988 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executed": 0, 2023-07-14T11:54:23.988 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_item_in_journal": 0 2023-07-14T11:54:23.988 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.989 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:23.989 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:23.989 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 104857600, 2023-07-14T11:54:23.989 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:23.990 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 8583, 2023-07-14T11:54:23.990 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 1286432594, 2023-07-14T11:54:23.990 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:23.990 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 8583, 2023-07-14T11:54:23.990 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:23.990 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:23.991 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 8583, 2023-07-14T11:54:23.991 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 1286432594, 2023-07-14T11:54:23.991 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:23.991 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.991 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.992 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.992 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:23.992 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.992 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-objecter_bytes": { 2023-07-14T11:54:23.993 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:23.993 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 104857600, 2023-07-14T11:54:23.993 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:23.993 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:23.993 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:23.994 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:23.994 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:23.994 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 7927, 2023-07-14T11:54:23.994 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 1283355129, 2023-07-14T11:54:23.994 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 7927, 2023-07-14T11:54:23.995 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 1283355129, 2023-07-14T11:54:23.995 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:23.995 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:23.995 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:23.996 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:23.996 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:23.996 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:23.996 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-objecter_ops": { 2023-07-14T11:54:23.996 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:23.997 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 1024, 2023-07-14T11:54:23.997 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:23.997 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:23.997 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:23.997 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:23.998 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:23.998 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 7927, 2023-07-14T11:54:23.998 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 7927, 2023-07-14T11:54:23.999 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 7927, 2023-07-14T11:54:23.999 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 7927, 2023-07-14T11:54:23.999 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:24.000 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:24.000 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:24.001 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:24.001 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:24.001 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:24.002 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-write_buf_throttle": { 2023-07-14T11:54:24.002 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:24.002 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 3758096384, 2023-07-14T11:54:24.003 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:24.003 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:24.004 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:24.004 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:24.004 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:24.005 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:24.005 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:24.005 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 0, 2023-07-14T11:54:24.005 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 0, 2023-07-14T11:54:24.006 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:24.006 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:24.006 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:24.006 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:24.006 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:24.007 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:24.007 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-write_buf_throttle-0x5628bbcce460": { 2023-07-14T11:54:24.007 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:24.007 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 3758096384, 2023-07-14T11:54:24.008 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:24.008 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:24.008 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:24.008 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:24.008 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:24.009 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:24.009 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:24.009 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 0, 2023-07-14T11:54:24.009 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 0, 2023-07-14T11:54:24.010 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:24.010 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:24.010 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:24.010 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:24.010 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:24.011 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:24.011 INFO:teuthology.orchestra.run.smithi182.stdout:} 2023-07-14T11:54:24.263 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-14T11:54:24.263 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-14T11:54:24.263 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-07-14T11:54:24.264 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-07-14T11:54:24.264 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T11:54:24.264 DEBUG:tasks.check_counter:Getting stats from f 2023-07-14T11:54:24.265 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-14T11:54:24.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:24 smithi175 ceph-mon[129081]: pgmap v1019: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:54:24.694 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:24 smithi182 ceph-mon[129366]: pgmap v1019: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:54:24.726 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:24 smithi161 ceph-mon[95526]: pgmap v1019: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:54:24.833 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:54:26.156 INFO:teuthology.orchestra.run.smithi182.stdout:{ 2023-07-14T11:54:26.157 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:26.157 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 4326, 2023-07-14T11:54:26.157 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 4327, 2023-07-14T11:54:26.157 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 173250360, 2023-07-14T11:54:26.157 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 1344908, 2023-07-14T11:54:26.158 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 8, 2023-07-14T11:54:26.158 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 4, 2023-07-14T11:54:26.158 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.373717246, 2023-07-14T11:54:26.158 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.081790451, 2023-07-14T11:54:26.158 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.156317517, 2023-07-14T11:54:26.159 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.109900035, 2023-07-14T11:54:26.159 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:26.159 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 4326, 2023-07-14T11:54:26.159 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.143835319, 2023-07-14T11:54:26.159 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000033249 2023-07-14T11:54:26.160 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.160 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:26.160 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.160 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.160 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.161 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.161 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-14T11:54:26.161 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T11:54:26.161 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.161 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:26.161 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 2488, 2023-07-14T11:54:26.162 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 2454, 2023-07-14T11:54:26.162 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 161675043, 2023-07-14T11:54:26.162 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 901531, 2023-07-14T11:54:26.162 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 6, 2023-07-14T11:54:26.162 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 5, 2023-07-14T11:54:26.163 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.275060141, 2023-07-14T11:54:26.163 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.054546006, 2023-07-14T11:54:26.163 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.160969752, 2023-07-14T11:54:26.163 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.054521693, 2023-07-14T11:54:26.163 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:26.164 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 2453, 2023-07-14T11:54:26.164 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.109537557, 2023-07-14T11:54:26.164 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000044654 2023-07-14T11:54:26.164 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.164 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:26.165 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.165 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.165 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.165 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.165 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 1697680, 2023-07-14T11:54:26.166 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 330784 2023-07-14T11:54:26.166 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.166 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:26.166 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 959, 2023-07-14T11:54:26.166 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 1381, 2023-07-14T11:54:26.167 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 189004730, 2023-07-14T11:54:26.167 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 1260381, 2023-07-14T11:54:26.167 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 9, 2023-07-14T11:54:26.167 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 6, 2023-07-14T11:54:26.167 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.186629989, 2023-07-14T11:54:26.167 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.028342558, 2023-07-14T11:54:26.168 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.120924015, 2023-07-14T11:54:26.168 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.028284105, 2023-07-14T11:54:26.168 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:26.168 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 1378, 2023-07-14T11:54:26.168 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.076957960, 2023-07-14T11:54:26.169 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000055847 2023-07-14T11:54:26.169 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.169 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:26.169 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.169 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.170 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.170 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.170 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-14T11:54:26.170 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 963376 2023-07-14T11:54:26.170 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.171 INFO:teuthology.orchestra.run.smithi182.stdout: "cct": { 2023-07-14T11:54:26.171 INFO:teuthology.orchestra.run.smithi182.stdout: "total_workers": 1, 2023-07-14T11:54:26.171 INFO:teuthology.orchestra.run.smithi182.stdout: "unhealthy_workers": 0 2023-07-14T11:54:26.171 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.171 INFO:teuthology.orchestra.run.smithi182.stdout: "finisher-MDSRank": { 2023-07-14T11:54:26.171 INFO:teuthology.orchestra.run.smithi182.stdout: "queue_len": 1, 2023-07-14T11:54:26.172 INFO:teuthology.orchestra.run.smithi182.stdout: "complete_latency": { 2023-07-14T11:54:26.172 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 12008, 2023-07-14T11:54:26.172 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.336913535, 2023-07-14T11:54:26.172 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000028057 2023-07-14T11:54:26.172 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.173 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.173 INFO:teuthology.orchestra.run.smithi182.stdout: "finisher-PurgeQueue": { 2023-07-14T11:54:26.173 INFO:teuthology.orchestra.run.smithi182.stdout: "queue_len": 0, 2023-07-14T11:54:26.173 INFO:teuthology.orchestra.run.smithi182.stdout: "complete_latency": { 2023-07-14T11:54:26.173 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.174 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.174 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.174 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.174 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.174 INFO:teuthology.orchestra.run.smithi182.stdout: "mds": { 2023-07-14T11:54:26.175 INFO:teuthology.orchestra.run.smithi182.stdout: "request": 0, 2023-07-14T11:54:26.175 INFO:teuthology.orchestra.run.smithi182.stdout: "reply": 0, 2023-07-14T11:54:26.175 INFO:teuthology.orchestra.run.smithi182.stdout: "reply_latency": { 2023-07-14T11:54:26.175 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.175 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.175 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.176 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.176 INFO:teuthology.orchestra.run.smithi182.stdout: "slow_reply": 0, 2023-07-14T11:54:26.176 INFO:teuthology.orchestra.run.smithi182.stdout: "forward": 0, 2023-07-14T11:54:26.176 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_fetch_complete": 0, 2023-07-14T11:54:26.176 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_fetch_keys": 0, 2023-07-14T11:54:26.177 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_commit": 0, 2023-07-14T11:54:26.177 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_split": 0, 2023-07-14T11:54:26.177 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_merge": 0, 2023-07-14T11:54:26.177 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes": 18633, 2023-07-14T11:54:26.177 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_top": 144, 2023-07-14T11:54:26.178 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_bottom": 232, 2023-07-14T11:54:26.178 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_pin_tail": 18257, 2023-07-14T11:54:26.178 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_pinned": 18427, 2023-07-14T11:54:26.178 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_expired": 5130, 2023-07-14T11:54:26.178 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_with_caps": 0, 2023-07-14T11:54:26.179 INFO:teuthology.orchestra.run.smithi182.stdout: "caps": 0, 2023-07-14T11:54:26.179 INFO:teuthology.orchestra.run.smithi182.stdout: "subtrees": 22, 2023-07-14T11:54:26.179 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse": 0, 2023-07-14T11:54:26.179 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_hit": 0, 2023-07-14T11:54:26.179 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_forward": 0, 2023-07-14T11:54:26.179 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_discover": 0, 2023-07-14T11:54:26.180 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_dir_fetch": 0, 2023-07-14T11:54:26.180 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_remote_ino": 0, 2023-07-14T11:54:26.180 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_lock": 0, 2023-07-14T11:54:26.180 INFO:teuthology.orchestra.run.smithi182.stdout: "load_cent": 0, 2023-07-14T11:54:26.180 INFO:teuthology.orchestra.run.smithi182.stdout: "q": 0, 2023-07-14T11:54:26.181 INFO:teuthology.orchestra.run.smithi182.stdout: "exported": 0, 2023-07-14T11:54:26.181 INFO:teuthology.orchestra.run.smithi182.stdout: "exported_inodes": 0, 2023-07-14T11:54:26.181 INFO:teuthology.orchestra.run.smithi182.stdout: "imported": 0, 2023-07-14T11:54:26.181 INFO:teuthology.orchestra.run.smithi182.stdout: "imported_inodes": 0, 2023-07-14T11:54:26.181 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_dir_fetch": 0, 2023-07-14T11:54:26.182 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_backtrace_fetch": 0, 2023-07-14T11:54:26.182 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_peer_discover": 0, 2023-07-14T11:54:26.182 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rfiles": 0, 2023-07-14T11:54:26.182 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rbytes": 0, 2023-07-14T11:54:26.182 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rsnaps": 0, 2023-07-14T11:54:26.183 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T11:54:26.183 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_set_tag": 0, 2023-07-14T11:54:26.183 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T11:54:26.183 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_inotable_repaired": 0, 2023-07-14T11:54:26.183 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dir_inodes": 0, 2023-07-14T11:54:26.184 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T11:54:26.184 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T11:54:26.184 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_file_inodes": 0, 2023-07-14T11:54:26.184 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_inode_file_caps": 0, 2023-07-14T11:54:26.184 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T11:54:26.184 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_grant": 0, 2023-07-14T11:54:26.185 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T11:54:26.185 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T11:54:26.185 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T11:54:26.185 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_caps": 0, 2023-07-14T11:54:26.185 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_caps_dirty": 0, 2023-07-14T11:54:26.186 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_cap_release": 0, 2023-07-14T11:54:26.186 INFO:teuthology.orchestra.run.smithi182.stdout: "process_request_cap_release": 0 2023-07-14T11:54:26.186 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.186 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_cache": { 2023-07-14T11:54:26.186 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_update": 0, 2023-07-14T11:54:26.187 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_update_receipt": 0, 2023-07-14T11:54:26.187 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_try_discover": 0, 2023-07-14T11:54:26.187 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_send_discover": 0, 2023-07-14T11:54:26.187 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_handle_discover": 0, 2023-07-14T11:54:26.187 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays": 0, 2023-07-14T11:54:26.188 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays_delayed": 0, 2023-07-14T11:54:26.188 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays_enqueuing": 0, 2023-07-14T11:54:26.188 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_created": 0, 2023-07-14T11:54:26.188 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_enqueued": 0, 2023-07-14T11:54:26.188 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_reintegrated": 0, 2023-07-14T11:54:26.189 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_migrated": 0, 2023-07-14T11:54:26.189 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_processing": 0, 2023-07-14T11:54:26.189 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_enqueued": 0, 2023-07-14T11:54:26.189 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_prioritized": 0, 2023-07-14T11:54:26.189 INFO:teuthology.orchestra.run.smithi182.stdout: "recovery_started": 0, 2023-07-14T11:54:26.189 INFO:teuthology.orchestra.run.smithi182.stdout: "recovery_completed": 0, 2023-07-14T11:54:26.190 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T11:54:26.190 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_exportdir": 0, 2023-07-14T11:54:26.190 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_flush": 0, 2023-07-14T11:54:26.190 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_fragmentdir": 0, 2023-07-14T11:54:26.190 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_fragstats": 0, 2023-07-14T11:54:26.191 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_inodestats": 0 2023-07-14T11:54:26.191 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.191 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_log": { 2023-07-14T11:54:26.191 INFO:teuthology.orchestra.run.smithi182.stdout: "evadd": 0, 2023-07-14T11:54:26.191 INFO:teuthology.orchestra.run.smithi182.stdout: "evex": 0, 2023-07-14T11:54:26.192 INFO:teuthology.orchestra.run.smithi182.stdout: "evtrm": 0, 2023-07-14T11:54:26.192 INFO:teuthology.orchestra.run.smithi182.stdout: "ev": 0, 2023-07-14T11:54:26.192 INFO:teuthology.orchestra.run.smithi182.stdout: "evexg": 0, 2023-07-14T11:54:26.192 INFO:teuthology.orchestra.run.smithi182.stdout: "evexd": 0, 2023-07-14T11:54:26.192 INFO:teuthology.orchestra.run.smithi182.stdout: "segadd": 0, 2023-07-14T11:54:26.192 INFO:teuthology.orchestra.run.smithi182.stdout: "segex": 0, 2023-07-14T11:54:26.193 INFO:teuthology.orchestra.run.smithi182.stdout: "segtrm": 0, 2023-07-14T11:54:26.193 INFO:teuthology.orchestra.run.smithi182.stdout: "seg": 128, 2023-07-14T11:54:26.193 INFO:teuthology.orchestra.run.smithi182.stdout: "segexg": 0, 2023-07-14T11:54:26.193 INFO:teuthology.orchestra.run.smithi182.stdout: "segexd": 0, 2023-07-14T11:54:26.193 INFO:teuthology.orchestra.run.smithi182.stdout: "expos": 110875976, 2023-07-14T11:54:26.194 INFO:teuthology.orchestra.run.smithi182.stdout: "wrpos": 0, 2023-07-14T11:54:26.194 INFO:teuthology.orchestra.run.smithi182.stdout: "rdpos": 524392237, 2023-07-14T11:54:26.194 INFO:teuthology.orchestra.run.smithi182.stdout: "jlat": { 2023-07-14T11:54:26.194 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.194 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.195 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.195 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.195 INFO:teuthology.orchestra.run.smithi182.stdout: "replayed": 156884 2023-07-14T11:54:26.195 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.195 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_mem": { 2023-07-14T11:54:26.196 INFO:teuthology.orchestra.run.smithi182.stdout: "ino": 1303, 2023-07-14T11:54:26.196 INFO:teuthology.orchestra.run.smithi182.stdout: "ino+": 23816, 2023-07-14T11:54:26.196 INFO:teuthology.orchestra.run.smithi182.stdout: "ino-": 22513, 2023-07-14T11:54:26.196 INFO:teuthology.orchestra.run.smithi182.stdout: "dir": 94, 2023-07-14T11:54:26.196 INFO:teuthology.orchestra.run.smithi182.stdout: "dir+": 203, 2023-07-14T11:54:26.197 INFO:teuthology.orchestra.run.smithi182.stdout: "dir-": 109, 2023-07-14T11:54:26.197 INFO:teuthology.orchestra.run.smithi182.stdout: "dn": 18633, 2023-07-14T11:54:26.197 INFO:teuthology.orchestra.run.smithi182.stdout: "dn+": 24059, 2023-07-14T11:54:26.197 INFO:teuthology.orchestra.run.smithi182.stdout: "dn-": 5426, 2023-07-14T11:54:26.197 INFO:teuthology.orchestra.run.smithi182.stdout: "cap": 0, 2023-07-14T11:54:26.197 INFO:teuthology.orchestra.run.smithi182.stdout: "cap+": 0, 2023-07-14T11:54:26.198 INFO:teuthology.orchestra.run.smithi182.stdout: "cap-": 0, 2023-07-14T11:54:26.198 INFO:teuthology.orchestra.run.smithi182.stdout: "rss": 76088, 2023-07-14T11:54:26.198 INFO:teuthology.orchestra.run.smithi182.stdout: "heap": 207156 2023-07-14T11:54:26.198 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.198 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_server": { 2023-07-14T11:54:26.199 INFO:teuthology.orchestra.run.smithi182.stdout: "dispatch_client_request": 0, 2023-07-14T11:54:26.199 INFO:teuthology.orchestra.run.smithi182.stdout: "dispatch_server_request": 0, 2023-07-14T11:54:26.199 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_request": 0, 2023-07-14T11:54:26.199 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_session": 0, 2023-07-14T11:54:26.199 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_peer_request": 0, 2023-07-14T11:54:26.200 INFO:teuthology.orchestra.run.smithi182.stdout: "req_create_latency": { 2023-07-14T11:54:26.200 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.200 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.200 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.200 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.201 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getattr_latency": { 2023-07-14T11:54:26.201 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.201 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.201 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.201 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.202 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getfilelock_latency": { 2023-07-14T11:54:26.202 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.202 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.202 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.202 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.202 INFO:teuthology.orchestra.run.smithi182.stdout: "req_link_latency": { 2023-07-14T11:54:26.203 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.203 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.203 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.203 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.203 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookup_latency": { 2023-07-14T11:54:26.204 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.204 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.204 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.204 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.204 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookuphash_latency": { 2023-07-14T11:54:26.205 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.205 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.205 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.205 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.205 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupino_latency": { 2023-07-14T11:54:26.206 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.206 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.206 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.206 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.206 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupname_latency": { 2023-07-14T11:54:26.206 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.207 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.207 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.207 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.207 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupparent_latency": { 2023-07-14T11:54:26.207 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.208 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.208 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.208 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.208 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupsnap_latency": { 2023-07-14T11:54:26.208 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.209 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.209 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.209 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.209 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lssnap_latency": { 2023-07-14T11:54:26.209 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.210 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.210 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.210 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.210 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mkdir_latency": { 2023-07-14T11:54:26.210 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.211 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.211 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.211 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.211 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mknod_latency": { 2023-07-14T11:54:26.211 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mksnap_latency": { 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi182.stdout: "req_open_latency": { 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi182.stdout: "req_readdir_latency": { 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.215 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.215 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.215 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rename_latency": { 2023-07-14T11:54:26.215 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.215 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi182.stdout: "req_renamesnap_latency": { 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi182.stdout: "req_snapdiff_latency": { 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.218 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.218 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmdir_latency": { 2023-07-14T11:54:26.218 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.218 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.218 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.219 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.219 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmsnap_latency": { 2023-07-14T11:54:26.219 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.219 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.219 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.220 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.220 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmxattr_latency": { 2023-07-14T11:54:26.220 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.220 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.220 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setattr_latency": { 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setdirlayout_latency": { 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setfilelock_latency": { 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setlayout_latency": { 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setxattr_latency": { 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.226 INFO:teuthology.orchestra.run.smithi182.stdout: "req_symlink_latency": { 2023-07-14T11:54:26.226 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.226 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.226 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.226 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.227 INFO:teuthology.orchestra.run.smithi182.stdout: "req_unlink_latency": { 2023-07-14T11:54:26.227 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.227 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.227 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.227 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi182.stdout: "cap_revoke_eviction": 0, 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi182.stdout: "cap_acquisition_throttle": 0, 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getvxattr_latency": { 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.229 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.229 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.229 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.229 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_sessions": { 2023-07-14T11:54:26.229 INFO:teuthology.orchestra.run.smithi182.stdout: "session_count": 2, 2023-07-14T11:54:26.229 INFO:teuthology.orchestra.run.smithi182.stdout: "session_add": 3, 2023-07-14T11:54:26.230 INFO:teuthology.orchestra.run.smithi182.stdout: "session_remove": 1, 2023-07-14T11:54:26.230 INFO:teuthology.orchestra.run.smithi182.stdout: "sessions_open": 3, 2023-07-14T11:54:26.230 INFO:teuthology.orchestra.run.smithi182.stdout: "sessions_stale": 0, 2023-07-14T11:54:26.230 INFO:teuthology.orchestra.run.smithi182.stdout: "total_load": 0, 2023-07-14T11:54:26.230 INFO:teuthology.orchestra.run.smithi182.stdout: "average_load": 0, 2023-07-14T11:54:26.231 INFO:teuthology.orchestra.run.smithi182.stdout: "avg_session_uptime": 5848 2023-07-14T11:54:26.231 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.231 INFO:teuthology.orchestra.run.smithi182.stdout: "mempool": { 2023-07-14T11:54:26.231 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:26.231 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:26.232 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:26.232 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:26.232 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:26.232 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:26.232 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:26.233 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:26.233 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:26.233 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:26.233 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:26.233 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:26.233 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:26.234 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:26.234 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:26.234 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:26.234 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:26.234 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:26.235 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:26.235 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:26.235 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:26.235 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:26.235 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:26.236 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:26.236 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:26.236 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:26.236 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:26.236 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:26.237 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:26.237 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:26.237 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:26.237 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_items": 0, 2023-07-14T11:54:26.237 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:26.238 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:26.238 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:26.238 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:26.238 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_bytes": 293511, 2023-07-14T11:54:26.238 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_items": 1585, 2023-07-14T11:54:26.238 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:54:26.239 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_items": 0, 2023-07-14T11:54:26.239 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_bytes": 0, 2023-07-14T11:54:26.239 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_items": 0, 2023-07-14T11:54:26.239 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:26.239 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:26.240 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:26.240 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:26.240 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_bytes": 9512, 2023-07-14T11:54:26.240 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_items": 182, 2023-07-14T11:54:26.240 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:26.241 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:26.241 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:26.241 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_items": 0, 2023-07-14T11:54:26.241 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_bytes": 16557815, 2023-07-14T11:54:26.241 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_items": 109464, 2023-07-14T11:54:26.242 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:26.242 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_items": 0, 2023-07-14T11:54:26.242 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:26.242 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_items": 0 2023-07-14T11:54:26.242 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.243 INFO:teuthology.orchestra.run.smithi182.stdout: "objecter": { 2023-07-14T11:54:26.243 INFO:teuthology.orchestra.run.smithi182.stdout: "op_active": 2, 2023-07-14T11:54:26.243 INFO:teuthology.orchestra.run.smithi182.stdout: "op_laggy": 0, 2023-07-14T11:54:26.243 INFO:teuthology.orchestra.run.smithi182.stdout: "op_send": 7115, 2023-07-14T11:54:26.243 INFO:teuthology.orchestra.run.smithi182.stdout: "op_send_bytes": 21, 2023-07-14T11:54:26.243 INFO:teuthology.orchestra.run.smithi182.stdout: "op_resend": 3, 2023-07-14T11:54:26.244 INFO:teuthology.orchestra.run.smithi182.stdout: "op_reply": 7113, 2023-07-14T11:54:26.244 INFO:teuthology.orchestra.run.smithi182.stdout: "op_latency": { 2023-07-14T11:54:26.244 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 7113, 2023-07-14T11:54:26.244 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 14.239666053, 2023-07-14T11:54:26.244 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.002001921 2023-07-14T11:54:26.245 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.245 INFO:teuthology.orchestra.run.smithi182.stdout: "op_inflight": 0, 2023-07-14T11:54:26.245 INFO:teuthology.orchestra.run.smithi182.stdout: "oplen_avg": { 2023-07-14T11:54:26.245 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 7115, 2023-07-14T11:54:26.245 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 7116 2023-07-14T11:54:26.246 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.246 INFO:teuthology.orchestra.run.smithi182.stdout: "op": 7115, 2023-07-14T11:54:26.246 INFO:teuthology.orchestra.run.smithi182.stdout: "op_r": 7115, 2023-07-14T11:54:26.246 INFO:teuthology.orchestra.run.smithi182.stdout: "op_w": 0, 2023-07-14T11:54:26.246 INFO:teuthology.orchestra.run.smithi182.stdout: "op_rmw": 0, 2023-07-14T11:54:26.247 INFO:teuthology.orchestra.run.smithi182.stdout: "op_pg": 0, 2023-07-14T11:54:26.247 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_stat": 4052, 2023-07-14T11:54:26.247 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_create": 0, 2023-07-14T11:54:26.247 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_read": 3061, 2023-07-14T11:54:26.247 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_write": 0, 2023-07-14T11:54:26.248 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_writefull": 0, 2023-07-14T11:54:26.248 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_writesame": 0, 2023-07-14T11:54:26.248 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_append": 0, 2023-07-14T11:54:26.248 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_zero": 0, 2023-07-14T11:54:26.248 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_truncate": 0, 2023-07-14T11:54:26.248 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_delete": 0, 2023-07-14T11:54:26.249 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_mapext": 0, 2023-07-14T11:54:26.249 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_sparse_read": 0, 2023-07-14T11:54:26.249 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_clonerange": 0, 2023-07-14T11:54:26.249 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_getxattr": 1, 2023-07-14T11:54:26.249 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_setxattr": 0, 2023-07-14T11:54:26.250 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_cmpxattr": 0, 2023-07-14T11:54:26.250 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_rmxattr": 0, 2023-07-14T11:54:26.250 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_resetxattrs": 0, 2023-07-14T11:54:26.250 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_call": 0, 2023-07-14T11:54:26.250 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_watch": 0, 2023-07-14T11:54:26.251 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_notify": 0, 2023-07-14T11:54:26.251 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T11:54:26.251 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_pgls": 0, 2023-07-14T11:54:26.251 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_pgls_filter": 0, 2023-07-14T11:54:26.251 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_other": 0, 2023-07-14T11:54:26.252 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_active": 0, 2023-07-14T11:54:26.252 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_send": 0, 2023-07-14T11:54:26.252 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_resend": 0, 2023-07-14T11:54:26.252 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_ping": 0, 2023-07-14T11:54:26.252 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_active": 0, 2023-07-14T11:54:26.253 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_send": 0, 2023-07-14T11:54:26.253 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_resend": 0, 2023-07-14T11:54:26.253 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_active": 0, 2023-07-14T11:54:26.253 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_send": 0, 2023-07-14T11:54:26.253 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_resend": 0, 2023-07-14T11:54:26.253 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_active": 0, 2023-07-14T11:54:26.254 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_send": 0, 2023-07-14T11:54:26.254 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_resend": 0, 2023-07-14T11:54:26.254 INFO:teuthology.orchestra.run.smithi182.stdout: "command_active": 0, 2023-07-14T11:54:26.254 INFO:teuthology.orchestra.run.smithi182.stdout: "command_send": 0, 2023-07-14T11:54:26.254 INFO:teuthology.orchestra.run.smithi182.stdout: "command_resend": 0, 2023-07-14T11:54:26.255 INFO:teuthology.orchestra.run.smithi182.stdout: "map_epoch": 145, 2023-07-14T11:54:26.255 INFO:teuthology.orchestra.run.smithi182.stdout: "map_full": 0, 2023-07-14T11:54:26.255 INFO:teuthology.orchestra.run.smithi182.stdout: "map_inc": 57, 2023-07-14T11:54:26.255 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_sessions": 11, 2023-07-14T11:54:26.255 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_session_open": 11, 2023-07-14T11:54:26.256 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_session_close": 0, 2023-07-14T11:54:26.256 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_laggy": 0, 2023-07-14T11:54:26.256 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_wr": 0, 2023-07-14T11:54:26.256 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_rd": 2, 2023-07-14T11:54:26.256 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_del": 0 2023-07-14T11:54:26.257 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.257 INFO:teuthology.orchestra.run.smithi182.stdout: "oft": { 2023-07-14T11:54:26.257 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_objs": 0, 2023-07-14T11:54:26.257 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_kv_pairs": 0, 2023-07-14T11:54:26.257 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_updates": 0, 2023-07-14T11:54:26.257 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_removes": 0 2023-07-14T11:54:26.258 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.258 INFO:teuthology.orchestra.run.smithi182.stdout: "osd-slow-ops": { 2023-07-14T11:54:26.258 INFO:teuthology.orchestra.run.smithi182.stdout: "slow_ops_count": 0 2023-07-14T11:54:26.258 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.258 INFO:teuthology.orchestra.run.smithi182.stdout: "purge_queue": { 2023-07-14T11:54:26.259 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_ops": 0, 2023-07-14T11:54:26.259 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T11:54:26.259 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing": 0, 2023-07-14T11:54:26.259 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_high_water": 0, 2023-07-14T11:54:26.259 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executed": 0, 2023-07-14T11:54:26.260 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_item_in_journal": 0 2023-07-14T11:54:26.260 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.260 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:26.260 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:26.260 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 104857600, 2023-07-14T11:54:26.261 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:26.261 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 7770, 2023-07-14T11:54:26.261 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 523194974, 2023-07-14T11:54:26.261 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:26.261 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 7770, 2023-07-14T11:54:26.261 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:26.262 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:26.262 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 7770, 2023-07-14T11:54:26.262 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 523194974, 2023-07-14T11:54:26.262 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:26.262 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.263 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.263 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.263 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.263 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.263 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-objecter_bytes": { 2023-07-14T11:54:26.264 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:26.264 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 104857600, 2023-07-14T11:54:26.264 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:26.264 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:26.264 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:26.264 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:26.265 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:26.265 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 7115, 2023-07-14T11:54:26.265 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 520241645, 2023-07-14T11:54:26.265 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 7113, 2023-07-14T11:54:26.265 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 520241645, 2023-07-14T11:54:26.266 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:26.266 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.266 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.266 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.266 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.267 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.267 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-objecter_ops": { 2023-07-14T11:54:26.267 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 2, 2023-07-14T11:54:26.267 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 1024, 2023-07-14T11:54:26.267 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:26.268 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:26.268 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:26.268 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:26.268 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:26.268 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 7115, 2023-07-14T11:54:26.269 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 7115, 2023-07-14T11:54:26.269 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 7113, 2023-07-14T11:54:26.269 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 7113, 2023-07-14T11:54:26.269 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:26.269 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.270 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.270 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.270 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.270 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.270 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-write_buf_throttle": { 2023-07-14T11:54:26.270 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:26.271 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 3758096384, 2023-07-14T11:54:26.271 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:26.271 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:26.271 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:26.271 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:26.272 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:26.272 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:26.272 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:26.272 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 0, 2023-07-14T11:54:26.272 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 0, 2023-07-14T11:54:26.273 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:26.273 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.273 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.273 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.273 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.274 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:26.274 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-write_buf_throttle-0x55a13db3c460": { 2023-07-14T11:54:26.274 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:26.274 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 3758096384, 2023-07-14T11:54:26.274 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:26.274 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:26.275 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:26.275 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:26.275 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:26.275 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:26.275 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:26.276 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 0, 2023-07-14T11:54:26.276 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 0, 2023-07-14T11:54:26.276 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:26.276 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:26.276 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:26.277 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:26.277 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.277 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:26.277 INFO:teuthology.orchestra.run.smithi182.stdout:} 2023-07-14T11:54:26.594 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-14T11:54:26.595 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-14T11:54:26.595 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-07-14T11:54:26.595 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-07-14T11:54:26.596 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T11:54:26.596 DEBUG:tasks.check_counter:Getting stats from i 2023-07-14T11:54:26.596 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-14T11:54:26.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:26 smithi182 ceph-mon[129366]: pgmap v1020: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:54:27.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:26 smithi175 ceph-mon[129081]: pgmap v1020: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:54:27.171 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:54:27.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:26 smithi161 ceph-mon[95526]: pgmap v1020: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-14T11:54:28.408 INFO:teuthology.orchestra.run.smithi182.stdout:{ 2023-07-14T11:54:28.408 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:28.409 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 30988, 2023-07-14T11:54:28.409 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 31021, 2023-07-14T11:54:28.409 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 9307135, 2023-07-14T11:54:28.409 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 184999978, 2023-07-14T11:54:28.410 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 21, 2023-07-14T11:54:28.410 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 7, 2023-07-14T11:54:28.410 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 2.378372742, 2023-07-14T11:54:28.410 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.778099042, 2023-07-14T11:54:28.410 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.513308156, 2023-07-14T11:54:28.411 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.949234968, 2023-07-14T11:54:28.411 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:28.411 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 31015, 2023-07-14T11:54:28.411 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 4.168123183, 2023-07-14T11:54:28.411 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000134390 2023-07-14T11:54:28.411 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.412 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:28.412 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 122, 2023-07-14T11:54:28.412 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000020158, 2023-07-14T11:54:28.412 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000165 2023-07-14T11:54:28.413 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.413 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 1716288, 2023-07-14T11:54:28.413 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 369648 2023-07-14T11:54:28.413 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.413 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:28.414 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 3313350, 2023-07-14T11:54:28.414 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 3404966, 2023-07-14T11:54:28.414 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 608990574, 2023-07-14T11:54:28.414 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 972638534, 2023-07-14T11:54:28.415 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 36, 2023-07-14T11:54:28.415 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 7, 2023-07-14T11:54:28.415 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 98.964742554, 2023-07-14T11:54:28.415 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 40.170127053, 2023-07-14T11:54:28.416 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 67.456490081, 2023-07-14T11:54:28.416 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.681417535, 2023-07-14T11:54:28.416 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:28.416 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 3404966, 2023-07-14T11:54:28.417 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 79.649601522, 2023-07-14T11:54:28.417 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000023392 2023-07-14T11:54:28.417 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.417 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:28.417 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 6567231, 2023-07-14T11:54:28.418 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.599517227, 2023-07-14T11:54:28.418 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000091 2023-07-14T11:54:28.418 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.418 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 79125744, 2023-07-14T11:54:28.418 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 352816368 2023-07-14T11:54:28.418 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.419 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:28.419 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 6933745, 2023-07-14T11:54:28.419 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 6931921, 2023-07-14T11:54:28.419 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 1132894363, 2023-07-14T11:54:28.419 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 1318826182, 2023-07-14T11:54:28.420 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 10, 2023-07-14T11:54:28.420 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 5, 2023-07-14T11:54:28.420 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 164.212678464, 2023-07-14T11:54:28.420 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 66.724278737, 2023-07-14T11:54:28.420 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 105.381753831, 2023-07-14T11:54:28.421 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 1.151462456, 2023-07-14T11:54:28.421 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:28.421 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 6931922, 2023-07-14T11:54:28.421 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 59.699353881, 2023-07-14T11:54:28.421 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000008612 2023-07-14T11:54:28.422 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.422 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:28.422 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 13793945, 2023-07-14T11:54:28.423 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.953894336, 2023-07-14T11:54:28.423 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000069 2023-07-14T11:54:28.423 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.423 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-14T11:54:28.424 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T11:54:28.424 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.424 INFO:teuthology.orchestra.run.smithi182.stdout: "cct": { 2023-07-14T11:54:28.424 INFO:teuthology.orchestra.run.smithi182.stdout: "total_workers": 1, 2023-07-14T11:54:28.424 INFO:teuthology.orchestra.run.smithi182.stdout: "unhealthy_workers": 0 2023-07-14T11:54:28.425 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.425 INFO:teuthology.orchestra.run.smithi182.stdout: "finisher-MDSRank": { 2023-07-14T11:54:28.425 INFO:teuthology.orchestra.run.smithi182.stdout: "queue_len": 0, 2023-07-14T11:54:28.425 INFO:teuthology.orchestra.run.smithi182.stdout: "complete_latency": { 2023-07-14T11:54:28.426 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 62936, 2023-07-14T11:54:28.426 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 67.663592104, 2023-07-14T11:54:28.426 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.001075117 2023-07-14T11:54:28.427 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.427 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.427 INFO:teuthology.orchestra.run.smithi182.stdout: "finisher-PurgeQueue": { 2023-07-14T11:54:28.427 INFO:teuthology.orchestra.run.smithi182.stdout: "queue_len": 0, 2023-07-14T11:54:28.427 INFO:teuthology.orchestra.run.smithi182.stdout: "complete_latency": { 2023-07-14T11:54:28.428 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 38009, 2023-07-14T11:54:28.428 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 4.679517407, 2023-07-14T11:54:28.428 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000123116 2023-07-14T11:54:28.428 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.428 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.429 INFO:teuthology.orchestra.run.smithi182.stdout: "mds": { 2023-07-14T11:54:28.429 INFO:teuthology.orchestra.run.smithi182.stdout: "request": 161335, 2023-07-14T11:54:28.429 INFO:teuthology.orchestra.run.smithi182.stdout: "reply": 156742, 2023-07-14T11:54:28.429 INFO:teuthology.orchestra.run.smithi182.stdout: "reply_latency": { 2023-07-14T11:54:28.429 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 156742, 2023-07-14T11:54:28.430 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 432.143648549, 2023-07-14T11:54:28.430 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.002757037 2023-07-14T11:54:28.430 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.430 INFO:teuthology.orchestra.run.smithi182.stdout: "slow_reply": 0, 2023-07-14T11:54:28.430 INFO:teuthology.orchestra.run.smithi182.stdout: "forward": 4593, 2023-07-14T11:54:28.431 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_fetch_complete": 5, 2023-07-14T11:54:28.431 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_fetch_keys": 0, 2023-07-14T11:54:28.431 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_commit": 148, 2023-07-14T11:54:28.431 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_split": 15, 2023-07-14T11:54:28.431 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_merge": 12, 2023-07-14T11:54:28.432 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes": 2839, 2023-07-14T11:54:28.432 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_top": 1280, 2023-07-14T11:54:28.432 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_bottom": 1510, 2023-07-14T11:54:28.432 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_pin_tail": 49, 2023-07-14T11:54:28.432 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_pinned": 1009, 2023-07-14T11:54:28.433 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_expired": 314, 2023-07-14T11:54:28.433 INFO:teuthology.orchestra.run.smithi182.stdout: "inodes_with_caps": 212, 2023-07-14T11:54:28.433 INFO:teuthology.orchestra.run.smithi182.stdout: "caps": 212, 2023-07-14T11:54:28.433 INFO:teuthology.orchestra.run.smithi182.stdout: "subtrees": 27, 2023-07-14T11:54:28.433 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse": 239053, 2023-07-14T11:54:28.434 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_hit": 137396, 2023-07-14T11:54:28.434 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_forward": 4583, 2023-07-14T11:54:28.434 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_discover": 53, 2023-07-14T11:54:28.434 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_dir_fetch": 1, 2023-07-14T11:54:28.434 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_remote_ino": 0, 2023-07-14T11:54:28.435 INFO:teuthology.orchestra.run.smithi182.stdout: "traverse_lock": 876, 2023-07-14T11:54:28.435 INFO:teuthology.orchestra.run.smithi182.stdout: "load_cent": 0, 2023-07-14T11:54:28.435 INFO:teuthology.orchestra.run.smithi182.stdout: "q": 0, 2023-07-14T11:54:28.435 INFO:teuthology.orchestra.run.smithi182.stdout: "exported": 10, 2023-07-14T11:54:28.435 INFO:teuthology.orchestra.run.smithi182.stdout: "exported_inodes": 298, 2023-07-14T11:54:28.436 INFO:teuthology.orchestra.run.smithi182.stdout: "imported": 9, 2023-07-14T11:54:28.436 INFO:teuthology.orchestra.run.smithi182.stdout: "imported_inodes": 686, 2023-07-14T11:54:28.440 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_dir_fetch": 0, 2023-07-14T11:54:28.441 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_backtrace_fetch": 1341, 2023-07-14T11:54:28.441 INFO:teuthology.orchestra.run.smithi182.stdout: "openino_peer_discover": 0, 2023-07-14T11:54:28.441 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rfiles": 755, 2023-07-14T11:54:28.442 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rbytes": 229819610, 2023-07-14T11:54:28.442 INFO:teuthology.orchestra.run.smithi182.stdout: "root_rsnaps": 6, 2023-07-14T11:54:28.442 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_backtrace_fetch": 1341, 2023-07-14T11:54:28.442 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_set_tag": 0, 2023-07-14T11:54:28.442 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T11:54:28.443 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_inotable_repaired": 0, 2023-07-14T11:54:28.443 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dir_inodes": 120, 2023-07-14T11:54:28.443 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T11:54:28.443 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_dirfrag_rstats": 120, 2023-07-14T11:54:28.443 INFO:teuthology.orchestra.run.smithi182.stdout: "scrub_file_inodes": 1221, 2023-07-14T11:54:28.444 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_inode_file_caps": 5, 2023-07-14T11:54:28.444 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_revoke": 4901, 2023-07-14T11:54:28.444 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_grant": 99232, 2023-07-14T11:54:28.444 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T11:54:28.444 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_flushsnap_ack": 1449, 2023-07-14T11:54:28.445 INFO:teuthology.orchestra.run.smithi182.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T11:54:28.445 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_caps": 42391, 2023-07-14T11:54:28.445 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_caps_dirty": 24248, 2023-07-14T11:54:28.445 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_cap_release": 5305, 2023-07-14T11:54:28.445 INFO:teuthology.orchestra.run.smithi182.stdout: "process_request_cap_release": 121731 2023-07-14T11:54:28.446 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.446 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_cache": { 2023-07-14T11:54:28.446 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_update": 51, 2023-07-14T11:54:28.447 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_update_receipt": 155, 2023-07-14T11:54:28.447 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_try_discover": 84, 2023-07-14T11:54:28.447 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_send_discover": 218, 2023-07-14T11:54:28.447 INFO:teuthology.orchestra.run.smithi182.stdout: "dir_handle_discover": 90, 2023-07-14T11:54:28.447 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays": 0, 2023-07-14T11:54:28.447 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays_delayed": 0, 2023-07-14T11:54:28.448 INFO:teuthology.orchestra.run.smithi182.stdout: "num_strays_enqueuing": 0, 2023-07-14T11:54:28.448 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_created": 22239, 2023-07-14T11:54:28.448 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_enqueued": 22239, 2023-07-14T11:54:28.448 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_reintegrated": 0, 2023-07-14T11:54:28.449 INFO:teuthology.orchestra.run.smithi182.stdout: "strays_migrated": 0, 2023-07-14T11:54:28.449 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_processing": 0, 2023-07-14T11:54:28.449 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_enqueued": 0, 2023-07-14T11:54:28.449 INFO:teuthology.orchestra.run.smithi182.stdout: "num_recovering_prioritized": 0, 2023-07-14T11:54:28.449 INFO:teuthology.orchestra.run.smithi182.stdout: "recovery_started": 0, 2023-07-14T11:54:28.449 INFO:teuthology.orchestra.run.smithi182.stdout: "recovery_completed": 0, 2023-07-14T11:54:28.450 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T11:54:28.450 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_exportdir": 19, 2023-07-14T11:54:28.450 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_flush": 0, 2023-07-14T11:54:28.450 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_fragmentdir": 27, 2023-07-14T11:54:28.450 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_fragstats": 0, 2023-07-14T11:54:28.451 INFO:teuthology.orchestra.run.smithi182.stdout: "ireq_inodestats": 0 2023-07-14T11:54:28.451 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.451 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_log": { 2023-07-14T11:54:28.451 INFO:teuthology.orchestra.run.smithi182.stdout: "evadd": 156884, 2023-07-14T11:54:28.451 INFO:teuthology.orchestra.run.smithi182.stdout: "evex": 66588, 2023-07-14T11:54:28.452 INFO:teuthology.orchestra.run.smithi182.stdout: "evtrm": 66588, 2023-07-14T11:54:28.452 INFO:teuthology.orchestra.run.smithi182.stdout: "ev": 90296, 2023-07-14T11:54:28.452 INFO:teuthology.orchestra.run.smithi182.stdout: "evexg": 0, 2023-07-14T11:54:28.452 INFO:teuthology.orchestra.run.smithi182.stdout: "evexd": 1024, 2023-07-14T11:54:28.452 INFO:teuthology.orchestra.run.smithi182.stdout: "segadd": 209, 2023-07-14T11:54:28.453 INFO:teuthology.orchestra.run.smithi182.stdout: "segex": 81, 2023-07-14T11:54:28.453 INFO:teuthology.orchestra.run.smithi182.stdout: "segtrm": 81, 2023-07-14T11:54:28.453 INFO:teuthology.orchestra.run.smithi182.stdout: "seg": 129, 2023-07-14T11:54:28.453 INFO:teuthology.orchestra.run.smithi182.stdout: "segexg": 0, 2023-07-14T11:54:28.453 INFO:teuthology.orchestra.run.smithi182.stdout: "segexd": 1, 2023-07-14T11:54:28.453 INFO:teuthology.orchestra.run.smithi182.stdout: "expos": 110875976, 2023-07-14T11:54:28.454 INFO:teuthology.orchestra.run.smithi182.stdout: "wrpos": 524392811, 2023-07-14T11:54:28.454 INFO:teuthology.orchestra.run.smithi182.stdout: "rdpos": 0, 2023-07-14T11:54:28.454 INFO:teuthology.orchestra.run.smithi182.stdout: "jlat": { 2023-07-14T11:54:28.454 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 33775, 2023-07-14T11:54:28.454 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 256.923344992, 2023-07-14T11:54:28.455 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.007606908 2023-07-14T11:54:28.455 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.455 INFO:teuthology.orchestra.run.smithi182.stdout: "replayed": 0 2023-07-14T11:54:28.455 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.455 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_mem": { 2023-07-14T11:54:28.456 INFO:teuthology.orchestra.run.smithi182.stdout: "ino": 1350, 2023-07-14T11:54:28.456 INFO:teuthology.orchestra.run.smithi182.stdout: "ino+": 23863, 2023-07-14T11:54:28.456 INFO:teuthology.orchestra.run.smithi182.stdout: "ino-": 22513, 2023-07-14T11:54:28.456 INFO:teuthology.orchestra.run.smithi182.stdout: "dir": 154, 2023-07-14T11:54:28.457 INFO:teuthology.orchestra.run.smithi182.stdout: "dir+": 411, 2023-07-14T11:54:28.457 INFO:teuthology.orchestra.run.smithi182.stdout: "dir-": 257, 2023-07-14T11:54:28.457 INFO:teuthology.orchestra.run.smithi182.stdout: "dn": 2839, 2023-07-14T11:54:28.457 INFO:teuthology.orchestra.run.smithi182.stdout: "dn+": 47081, 2023-07-14T11:54:28.457 INFO:teuthology.orchestra.run.smithi182.stdout: "dn-": 44242, 2023-07-14T11:54:28.458 INFO:teuthology.orchestra.run.smithi182.stdout: "cap": 212, 2023-07-14T11:54:28.458 INFO:teuthology.orchestra.run.smithi182.stdout: "cap+": 24672, 2023-07-14T11:54:28.458 INFO:teuthology.orchestra.run.smithi182.stdout: "cap-": 24460, 2023-07-14T11:54:28.458 INFO:teuthology.orchestra.run.smithi182.stdout: "rss": 76656, 2023-07-14T11:54:28.458 INFO:teuthology.orchestra.run.smithi182.stdout: "heap": 198964 2023-07-14T11:54:28.459 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.459 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_server": { 2023-07-14T11:54:28.459 INFO:teuthology.orchestra.run.smithi182.stdout: "dispatch_client_request": 225690, 2023-07-14T11:54:28.459 INFO:teuthology.orchestra.run.smithi182.stdout: "dispatch_server_request": 3, 2023-07-14T11:54:28.459 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_request": 161335, 2023-07-14T11:54:28.460 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_client_session": 2782, 2023-07-14T11:54:28.460 INFO:teuthology.orchestra.run.smithi182.stdout: "handle_peer_request": 10528, 2023-07-14T11:54:28.460 INFO:teuthology.orchestra.run.smithi182.stdout: "req_create_latency": { 2023-07-14T11:54:28.460 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 22267, 2023-07-14T11:54:28.460 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 66.604761262, 2023-07-14T11:54:28.461 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.002991187 2023-07-14T11:54:28.461 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.461 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getattr_latency": { 2023-07-14T11:54:28.461 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 361, 2023-07-14T11:54:28.461 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 3.908194590, 2023-07-14T11:54:28.462 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.010826023 2023-07-14T11:54:28.462 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.462 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getfilelock_latency": { 2023-07-14T11:54:28.462 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.462 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.463 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.463 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.463 INFO:teuthology.orchestra.run.smithi182.stdout: "req_link_latency": { 2023-07-14T11:54:28.463 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.463 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.464 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.464 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.464 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookup_latency": { 2023-07-14T11:54:28.464 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 32964, 2023-07-14T11:54:28.464 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 45.482760022, 2023-07-14T11:54:28.465 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.001379770 2023-07-14T11:54:28.465 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.465 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookuphash_latency": { 2023-07-14T11:54:28.465 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.465 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.466 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.466 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.466 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupino_latency": { 2023-07-14T11:54:28.466 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.466 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.467 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.467 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.467 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupname_latency": { 2023-07-14T11:54:28.467 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.468 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.468 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.468 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.468 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupparent_latency": { 2023-07-14T11:54:28.468 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.469 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.469 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.469 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.469 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lookupsnap_latency": { 2023-07-14T11:54:28.469 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.470 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.470 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.470 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.470 INFO:teuthology.orchestra.run.smithi182.stdout: "req_lssnap_latency": { 2023-07-14T11:54:28.470 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.471 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.471 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.471 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.471 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mkdir_latency": { 2023-07-14T11:54:28.471 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.472 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.472 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.472 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.472 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mknod_latency": { 2023-07-14T11:54:28.472 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.473 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.473 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.473 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.473 INFO:teuthology.orchestra.run.smithi182.stdout: "req_mksnap_latency": { 2023-07-14T11:54:28.473 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.474 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.474 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.474 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.474 INFO:teuthology.orchestra.run.smithi182.stdout: "req_open_latency": { 2023-07-14T11:54:28.474 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.474 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.475 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.475 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.475 INFO:teuthology.orchestra.run.smithi182.stdout: "req_readdir_latency": { 2023-07-14T11:54:28.475 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 74779, 2023-07-14T11:54:28.475 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 242.303528021, 2023-07-14T11:54:28.476 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.003240261 2023-07-14T11:54:28.476 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.476 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rename_latency": { 2023-07-14T11:54:28.476 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 3623, 2023-07-14T11:54:28.477 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 10.625927708, 2023-07-14T11:54:28.477 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.002932908 2023-07-14T11:54:28.477 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.477 INFO:teuthology.orchestra.run.smithi182.stdout: "req_renamesnap_latency": { 2023-07-14T11:54:28.478 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.478 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.478 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.478 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.479 INFO:teuthology.orchestra.run.smithi182.stdout: "req_snapdiff_latency": { 2023-07-14T11:54:28.479 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.479 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.480 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.480 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.480 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmdir_latency": { 2023-07-14T11:54:28.480 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.481 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.481 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.481 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.481 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmsnap_latency": { 2023-07-14T11:54:28.482 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.482 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.482 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.482 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.483 INFO:teuthology.orchestra.run.smithi182.stdout: "req_rmxattr_latency": { 2023-07-14T11:54:28.483 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.483 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.483 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.483 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.484 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setattr_latency": { 2023-07-14T11:54:28.484 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 17, 2023-07-14T11:54:28.484 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.251467199, 2023-07-14T11:54:28.484 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.014792188 2023-07-14T11:54:28.484 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.485 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setdirlayout_latency": { 2023-07-14T11:54:28.485 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.485 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.485 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.485 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.486 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setfilelock_latency": { 2023-07-14T11:54:28.486 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 492, 2023-07-14T11:54:28.486 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.428802208, 2023-07-14T11:54:28.486 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000871549 2023-07-14T11:54:28.486 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.486 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setlayout_latency": { 2023-07-14T11:54:28.487 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.487 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.487 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.487 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.487 INFO:teuthology.orchestra.run.smithi182.stdout: "req_setxattr_latency": { 2023-07-14T11:54:28.488 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.488 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.488 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.488 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.488 INFO:teuthology.orchestra.run.smithi182.stdout: "req_symlink_latency": { 2023-07-14T11:54:28.489 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.489 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.489 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.489 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.489 INFO:teuthology.orchestra.run.smithi182.stdout: "req_unlink_latency": { 2023-07-14T11:54:28.490 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 22239, 2023-07-14T11:54:28.490 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 62.538207539, 2023-07-14T11:54:28.490 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.002812096 2023-07-14T11:54:28.490 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.490 INFO:teuthology.orchestra.run.smithi182.stdout: "cap_revoke_eviction": 0, 2023-07-14T11:54:28.491 INFO:teuthology.orchestra.run.smithi182.stdout: "cap_acquisition_throttle": 0, 2023-07-14T11:54:28.491 INFO:teuthology.orchestra.run.smithi182.stdout: "req_getvxattr_latency": { 2023-07-14T11:54:28.491 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.491 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.491 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.492 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.492 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.492 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_sessions": { 2023-07-14T11:54:28.492 INFO:teuthology.orchestra.run.smithi182.stdout: "session_count": 2, 2023-07-14T11:54:28.492 INFO:teuthology.orchestra.run.smithi182.stdout: "session_add": 3, 2023-07-14T11:54:28.493 INFO:teuthology.orchestra.run.smithi182.stdout: "session_remove": 1, 2023-07-14T11:54:28.493 INFO:teuthology.orchestra.run.smithi182.stdout: "sessions_open": 2, 2023-07-14T11:54:28.493 INFO:teuthology.orchestra.run.smithi182.stdout: "sessions_stale": 0, 2023-07-14T11:54:28.493 INFO:teuthology.orchestra.run.smithi182.stdout: "total_load": 24237, 2023-07-14T11:54:28.493 INFO:teuthology.orchestra.run.smithi182.stdout: "average_load": 12118, 2023-07-14T11:54:28.494 INFO:teuthology.orchestra.run.smithi182.stdout: "avg_session_uptime": 1876 2023-07-14T11:54:28.494 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.494 INFO:teuthology.orchestra.run.smithi182.stdout: "mempool": { 2023-07-14T11:54:28.494 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:28.494 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:28.494 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:28.495 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:28.495 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:28.495 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:28.495 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:28.495 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:28.496 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:28.496 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:28.496 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:28.496 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:28.496 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:28.497 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:28.497 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:28.497 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:28.497 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:28.497 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:28.498 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:28.498 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:28.498 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:28.498 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:28.498 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:28.499 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:28.499 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:28.499 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:28.499 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:28.499 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:28.500 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:28.500 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:28.500 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:28.500 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_items": 0, 2023-07-14T11:54:28.500 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:28.501 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:28.501 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:28.501 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:28.501 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_bytes": 321529, 2023-07-14T11:54:28.501 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_items": 916, 2023-07-14T11:54:28.502 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_bytes": 176, 2023-07-14T11:54:28.502 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_items": 2, 2023-07-14T11:54:28.502 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_bytes": 0, 2023-07-14T11:54:28.502 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_items": 0, 2023-07-14T11:54:28.502 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:28.503 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:28.503 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:28.503 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:28.503 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_bytes": 9512, 2023-07-14T11:54:28.503 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_items": 182, 2023-07-14T11:54:28.503 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:28.504 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:28.504 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:28.504 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_items": 0, 2023-07-14T11:54:28.504 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_bytes": 5799010, 2023-07-14T11:54:28.504 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_items": 54636, 2023-07-14T11:54:28.505 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:28.505 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_items": 0, 2023-07-14T11:54:28.505 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:28.505 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_items": 0 2023-07-14T11:54:28.505 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.506 INFO:teuthology.orchestra.run.smithi182.stdout: "objecter": { 2023-07-14T11:54:28.506 INFO:teuthology.orchestra.run.smithi182.stdout: "op_active": 0, 2023-07-14T11:54:28.506 INFO:teuthology.orchestra.run.smithi182.stdout: "op_laggy": 0, 2023-07-14T11:54:28.506 INFO:teuthology.orchestra.run.smithi182.stdout: "op_send": 82080, 2023-07-14T11:54:28.507 INFO:teuthology.orchestra.run.smithi182.stdout: "op_send_bytes": 525881654, 2023-07-14T11:54:28.507 INFO:teuthology.orchestra.run.smithi182.stdout: "op_resend": 27, 2023-07-14T11:54:28.507 INFO:teuthology.orchestra.run.smithi182.stdout: "op_reply": 82080, 2023-07-14T11:54:28.507 INFO:teuthology.orchestra.run.smithi182.stdout: "op_latency": { 2023-07-14T11:54:28.507 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 82080, 2023-07-14T11:54:28.508 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 953.022618491, 2023-07-14T11:54:28.508 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.011610899 2023-07-14T11:54:28.508 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.508 INFO:teuthology.orchestra.run.smithi182.stdout: "op_inflight": 0, 2023-07-14T11:54:28.508 INFO:teuthology.orchestra.run.smithi182.stdout: "oplen_avg": { 2023-07-14T11:54:28.509 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 82080, 2023-07-14T11:54:28.509 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 82678 2023-07-14T11:54:28.509 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.509 INFO:teuthology.orchestra.run.smithi182.stdout: "op": 82080, 2023-07-14T11:54:28.510 INFO:teuthology.orchestra.run.smithi182.stdout: "op_r": 2053, 2023-07-14T11:54:28.510 INFO:teuthology.orchestra.run.smithi182.stdout: "op_w": 80027, 2023-07-14T11:54:28.510 INFO:teuthology.orchestra.run.smithi182.stdout: "op_rmw": 0, 2023-07-14T11:54:28.510 INFO:teuthology.orchestra.run.smithi182.stdout: "op_pg": 0, 2023-07-14T11:54:28.510 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_stat": 0, 2023-07-14T11:54:28.511 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_create": 21, 2023-07-14T11:54:28.511 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_read": 707, 2023-07-14T11:54:28.511 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_write": 34482, 2023-07-14T11:54:28.511 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_writefull": 431, 2023-07-14T11:54:28.511 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_writesame": 0, 2023-07-14T11:54:28.511 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_append": 0, 2023-07-14T11:54:28.512 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_zero": 0, 2023-07-14T11:54:28.512 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_truncate": 15, 2023-07-14T11:54:28.512 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_delete": 44721, 2023-07-14T11:54:28.512 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_mapext": 0, 2023-07-14T11:54:28.513 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_sparse_read": 0, 2023-07-14T11:54:28.513 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_clonerange": 0, 2023-07-14T11:54:28.513 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_getxattr": 1341, 2023-07-14T11:54:28.513 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_setxattr": 42, 2023-07-14T11:54:28.514 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_cmpxattr": 0, 2023-07-14T11:54:28.514 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_rmxattr": 0, 2023-07-14T11:54:28.514 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_resetxattrs": 0, 2023-07-14T11:54:28.514 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_call": 0, 2023-07-14T11:54:28.514 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_watch": 0, 2023-07-14T11:54:28.515 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_notify": 0, 2023-07-14T11:54:28.515 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T11:54:28.515 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_pgls": 0, 2023-07-14T11:54:28.515 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_pgls_filter": 0, 2023-07-14T11:54:28.515 INFO:teuthology.orchestra.run.smithi182.stdout: "osdop_other": 0, 2023-07-14T11:54:28.516 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_active": 0, 2023-07-14T11:54:28.516 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_send": 0, 2023-07-14T11:54:28.516 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_resend": 0, 2023-07-14T11:54:28.516 INFO:teuthology.orchestra.run.smithi182.stdout: "linger_ping": 0, 2023-07-14T11:54:28.516 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_active": 0, 2023-07-14T11:54:28.517 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_send": 0, 2023-07-14T11:54:28.517 INFO:teuthology.orchestra.run.smithi182.stdout: "poolop_resend": 0, 2023-07-14T11:54:28.517 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_active": 0, 2023-07-14T11:54:28.517 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_send": 0, 2023-07-14T11:54:28.517 INFO:teuthology.orchestra.run.smithi182.stdout: "poolstat_resend": 0, 2023-07-14T11:54:28.518 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_active": 0, 2023-07-14T11:54:28.518 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_send": 0, 2023-07-14T11:54:28.518 INFO:teuthology.orchestra.run.smithi182.stdout: "statfs_resend": 0, 2023-07-14T11:54:28.518 INFO:teuthology.orchestra.run.smithi182.stdout: "command_active": 0, 2023-07-14T11:54:28.518 INFO:teuthology.orchestra.run.smithi182.stdout: "command_send": 0, 2023-07-14T11:54:28.519 INFO:teuthology.orchestra.run.smithi182.stdout: "command_resend": 0, 2023-07-14T11:54:28.519 INFO:teuthology.orchestra.run.smithi182.stdout: "map_epoch": 145, 2023-07-14T11:54:28.519 INFO:teuthology.orchestra.run.smithi182.stdout: "map_full": 0, 2023-07-14T11:54:28.519 INFO:teuthology.orchestra.run.smithi182.stdout: "map_inc": 57, 2023-07-14T11:54:28.519 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_sessions": 12, 2023-07-14T11:54:28.519 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_session_open": 12, 2023-07-14T11:54:28.520 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_session_close": 0, 2023-07-14T11:54:28.520 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_laggy": 0, 2023-07-14T11:54:28.520 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_wr": 694, 2023-07-14T11:54:28.520 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_rd": 10, 2023-07-14T11:54:28.520 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_del": 214 2023-07-14T11:54:28.521 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.521 INFO:teuthology.orchestra.run.smithi182.stdout: "oft": { 2023-07-14T11:54:28.521 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_objs": 1, 2023-07-14T11:54:28.521 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_kv_pairs": 0, 2023-07-14T11:54:28.521 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_updates": 0, 2023-07-14T11:54:28.522 INFO:teuthology.orchestra.run.smithi182.stdout: "omap_total_removes": 0 2023-07-14T11:54:28.522 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.522 INFO:teuthology.orchestra.run.smithi182.stdout: "osd-slow-ops": { 2023-07-14T11:54:28.522 INFO:teuthology.orchestra.run.smithi182.stdout: "slow_ops_count": 161383 2023-07-14T11:54:28.522 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.523 INFO:teuthology.orchestra.run.smithi182.stdout: "purge_queue": { 2023-07-14T11:54:28.523 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_ops": 0, 2023-07-14T11:54:28.523 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_ops_high_water": 16, 2023-07-14T11:54:28.523 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing": 0, 2023-07-14T11:54:28.523 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executing_high_water": 8, 2023-07-14T11:54:28.524 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_executed": 22239, 2023-07-14T11:54:28.524 INFO:teuthology.orchestra.run.smithi182.stdout: "pq_item_in_journal": 0 2023-07-14T11:54:28.524 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.524 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:28.524 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:28.525 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 104857600, 2023-07-14T11:54:28.525 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:28.525 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 10278081, 2023-07-14T11:54:28.525 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 816950642, 2023-07-14T11:54:28.525 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:28.526 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 10278081, 2023-07-14T11:54:28.526 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:28.526 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:28.526 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 10278081, 2023-07-14T11:54:28.526 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 816950642, 2023-07-14T11:54:28.526 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:28.527 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.527 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.527 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.527 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.527 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.528 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-objecter_bytes": { 2023-07-14T11:54:28.528 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:28.528 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 104857600, 2023-07-14T11:54:28.528 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:28.529 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:28.529 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:28.529 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:28.529 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:28.529 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 82080, 2023-07-14T11:54:28.530 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 528309201, 2023-07-14T11:54:28.530 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 82080, 2023-07-14T11:54:28.530 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 528309201, 2023-07-14T11:54:28.530 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:28.530 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.530 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.531 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.531 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.531 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.531 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-objecter_ops": { 2023-07-14T11:54:28.531 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:28.532 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 1024, 2023-07-14T11:54:28.532 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:28.532 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 0, 2023-07-14T11:54:28.532 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 0, 2023-07-14T11:54:28.532 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:28.533 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 0, 2023-07-14T11:54:28.533 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 82080, 2023-07-14T11:54:28.533 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 82080, 2023-07-14T11:54:28.533 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 82080, 2023-07-14T11:54:28.533 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 82080, 2023-07-14T11:54:28.534 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:28.534 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.534 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.534 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.534 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.535 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.535 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-write_buf_throttle": { 2023-07-14T11:54:28.535 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:28.535 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 3758096384, 2023-07-14T11:54:28.535 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:28.536 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 22239, 2023-07-14T11:54:28.536 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 2427627, 2023-07-14T11:54:28.536 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:28.536 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 22239, 2023-07-14T11:54:28.537 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:28.537 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:28.537 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 707, 2023-07-14T11:54:28.537 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 2427627, 2023-07-14T11:54:28.537 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:28.537 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.538 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.538 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.538 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.538 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:28.538 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-write_buf_throttle-0x55b8da7520a0": { 2023-07-14T11:54:28.539 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:28.539 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 3758096384, 2023-07-14T11:54:28.539 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:28.539 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 156884, 2023-07-14T11:54:28.539 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 520198507, 2023-07-14T11:54:28.540 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:28.540 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 156884, 2023-07-14T11:54:28.540 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:28.540 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:28.540 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 33775, 2023-07-14T11:54:28.541 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 520198507, 2023-07-14T11:54:28.541 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:28.541 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:28.541 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:28.541 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:28.542 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.542 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:28.542 INFO:teuthology.orchestra.run.smithi182.stdout:} 2023-07-14T11:54:29.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:28 smithi175 ceph-mon[129081]: pgmap v1021: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:54:29.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:28 smithi182 ceph-mon[129366]: pgmap v1021: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:54:29.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:28 smithi161 ceph-mon[95526]: pgmap v1021: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:54:30.937 INFO:tasks.check_counter:Daemon mds.i mds.exported=10 2023-07-14T11:54:30.937 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2023-07-14T11:54:30.938 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-07-14T11:54:30.938 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-07-14T11:54:30.938 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T11:54:30.938 DEBUG:tasks.check_counter:Getting stats from l 2023-07-14T11:54:30.939 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-14T11:54:31.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:30 smithi175 ceph-mon[129081]: pgmap v1022: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:54:31.194 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:30 smithi182 ceph-mon[129366]: pgmap v1022: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:54:31.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:30 smithi161 ceph-mon[95526]: pgmap v1022: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2023-07-14T11:54:31.503 INFO:teuthology.orchestra.run.smithi182.stderr:Inferring config /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/mon.c/config 2023-07-14T11:54:32.849 INFO:teuthology.orchestra.run.smithi182.stdout:{ 2023-07-14T11:54:32.849 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T11:54:32.850 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 576, 2023-07-14T11:54:32.850 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 564, 2023-07-14T11:54:32.850 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 1525584, 2023-07-14T11:54:32.850 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 323328, 2023-07-14T11:54:32.850 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 4, 2023-07-14T11:54:32.851 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 0, 2023-07-14T11:54:32.851 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.071612026, 2023-07-14T11:54:32.852 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.023995244, 2023-07-14T11:54:32.852 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.037877532, 2023-07-14T11:54:32.852 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.012701208, 2023-07-14T11:54:32.852 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:32.852 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 564, 2023-07-14T11:54:32.853 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.032097239, 2023-07-14T11:54:32.853 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000056909 2023-07-14T11:54:32.853 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.853 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:32.853 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:32.854 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:32.854 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:32.854 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.854 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 1525584, 2023-07-14T11:54:32.854 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 323328 2023-07-14T11:54:32.855 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.855 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T11:54:32.855 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 4, 2023-07-14T11:54:32.855 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 431, 2023-07-14T11:54:32.855 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 640, 2023-07-14T11:54:32.856 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 379008, 2023-07-14T11:54:32.856 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 7, 2023-07-14T11:54:32.856 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 3, 2023-07-14T11:54:32.856 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.017770508, 2023-07-14T11:54:32.857 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.012394485, 2023-07-14T11:54:32.857 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.001809451, 2023-07-14T11:54:32.857 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:32.857 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:32.858 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 428, 2023-07-14T11:54:32.858 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.038144882, 2023-07-14T11:54:32.858 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000089123 2023-07-14T11:54:32.859 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.859 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:32.859 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:32.860 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:32.860 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:32.860 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.860 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-14T11:54:32.860 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 379008 2023-07-14T11:54:32.861 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.861 INFO:teuthology.orchestra.run.smithi182.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T11:54:32.861 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_messages": 0, 2023-07-14T11:54:32.861 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages": 2, 2023-07-14T11:54:32.861 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_bytes": 0, 2023-07-14T11:54:32.862 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_bytes": 0, 2023-07-14T11:54:32.862 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_created_connections": 2, 2023-07-14T11:54:32.862 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_active_connections": 1, 2023-07-14T11:54:32.862 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_total_time": 0.000341597, 2023-07-14T11:54:32.862 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T11:54:32.863 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_recv_time": 0.000250226, 2023-07-14T11:54:32.863 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T11:54:32.863 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T11:54:32.863 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:32.863 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:32.864 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:32.864 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.864 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_handle_ack_lat": { 2023-07-14T11:54:32.864 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:32.864 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:32.865 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:32.865 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.865 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T11:54:32.865 INFO:teuthology.orchestra.run.smithi182.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T11:54:32.866 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.866 INFO:teuthology.orchestra.run.smithi182.stdout: "cct": { 2023-07-14T11:54:32.867 INFO:teuthology.orchestra.run.smithi182.stdout: "total_workers": 0, 2023-07-14T11:54:32.867 INFO:teuthology.orchestra.run.smithi182.stdout: "unhealthy_workers": 0 2023-07-14T11:54:32.867 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.868 INFO:teuthology.orchestra.run.smithi182.stdout: "mempool": { 2023-07-14T11:54:32.868 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_bytes": 0, 2023-07-14T11:54:32.869 INFO:teuthology.orchestra.run.smithi182.stdout: "bloom_filter_items": 0, 2023-07-14T11:54:32.869 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T11:54:32.869 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_alloc_items": 0, 2023-07-14T11:54:32.870 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T11:54:32.870 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_data_items": 0, 2023-07-14T11:54:32.871 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T11:54:32.871 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T11:54:32.871 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T11:54:32.872 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T11:54:32.872 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T11:54:32.873 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_other_items": 0, 2023-07-14T11:54:32.873 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T11:54:32.873 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T11:54:32.873 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_bytes": 0, 2023-07-14T11:54:32.873 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_extent_items": 0, 2023-07-14T11:54:32.874 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_bytes": 0, 2023-07-14T11:54:32.874 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_blob_items": 0, 2023-07-14T11:54:32.874 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T11:54:32.874 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T11:54:32.874 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T11:54:32.875 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T11:54:32.875 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T11:54:32.875 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_fsck_items": 0, 2023-07-14T11:54:32.875 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_bytes": 0, 2023-07-14T11:54:32.875 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_txc_items": 0, 2023-07-14T11:54:32.875 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T11:54:32.876 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T11:54:32.876 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_bytes": 0, 2023-07-14T11:54:32.876 INFO:teuthology.orchestra.run.smithi182.stdout: "bluestore_writing_items": 0, 2023-07-14T11:54:32.876 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_bytes": 0, 2023-07-14T11:54:32.877 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_items": 0, 2023-07-14T11:54:32.877 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T11:54:32.877 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_reader_items": 0, 2023-07-14T11:54:32.877 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T11:54:32.877 INFO:teuthology.orchestra.run.smithi182.stdout: "bluefs_file_writer_items": 0, 2023-07-14T11:54:32.877 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_bytes": 148010, 2023-07-14T11:54:32.878 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_anon_items": 30, 2023-07-14T11:54:32.878 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_bytes": 0, 2023-07-14T11:54:32.878 INFO:teuthology.orchestra.run.smithi182.stdout: "buffer_meta_items": 0, 2023-07-14T11:54:32.878 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_bytes": 0, 2023-07-14T11:54:32.878 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_items": 0, 2023-07-14T11:54:32.879 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_bytes": 0, 2023-07-14T11:54:32.879 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_mapbl_items": 0, 2023-07-14T11:54:32.879 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_bytes": 0, 2023-07-14T11:54:32.879 INFO:teuthology.orchestra.run.smithi182.stdout: "osd_pglog_items": 0, 2023-07-14T11:54:32.879 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_bytes": 0, 2023-07-14T11:54:32.880 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_items": 0, 2023-07-14T11:54:32.880 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T11:54:32.880 INFO:teuthology.orchestra.run.smithi182.stdout: "osdmap_mapping_items": 0, 2023-07-14T11:54:32.880 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_bytes": 0, 2023-07-14T11:54:32.880 INFO:teuthology.orchestra.run.smithi182.stdout: "pgmap_items": 0, 2023-07-14T11:54:32.881 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_bytes": 928, 2023-07-14T11:54:32.881 INFO:teuthology.orchestra.run.smithi182.stdout: "mds_co_items": 2, 2023-07-14T11:54:32.881 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_bytes": 0, 2023-07-14T11:54:32.881 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_1_items": 0, 2023-07-14T11:54:32.881 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_bytes": 0, 2023-07-14T11:54:32.882 INFO:teuthology.orchestra.run.smithi182.stdout: "unittest_2_items": 0 2023-07-14T11:54:32.882 INFO:teuthology.orchestra.run.smithi182.stdout: }, 2023-07-14T11:54:32.882 INFO:teuthology.orchestra.run.smithi182.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T11:54:32.882 INFO:teuthology.orchestra.run.smithi182.stdout: "val": 0, 2023-07-14T11:54:32.882 INFO:teuthology.orchestra.run.smithi182.stdout: "max": 104857600, 2023-07-14T11:54:32.883 INFO:teuthology.orchestra.run.smithi182.stdout: "get_started": 0, 2023-07-14T11:54:32.883 INFO:teuthology.orchestra.run.smithi182.stdout: "get": 576, 2023-07-14T11:54:32.883 INFO:teuthology.orchestra.run.smithi182.stdout: "get_sum": 1441785, 2023-07-14T11:54:32.883 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_fail": 0, 2023-07-14T11:54:32.883 INFO:teuthology.orchestra.run.smithi182.stdout: "get_or_fail_success": 576, 2023-07-14T11:54:32.884 INFO:teuthology.orchestra.run.smithi182.stdout: "take": 0, 2023-07-14T11:54:32.884 INFO:teuthology.orchestra.run.smithi182.stdout: "take_sum": 0, 2023-07-14T11:54:32.884 INFO:teuthology.orchestra.run.smithi182.stdout: "put": 576, 2023-07-14T11:54:32.884 INFO:teuthology.orchestra.run.smithi182.stdout: "put_sum": 1441785, 2023-07-14T11:54:32.884 INFO:teuthology.orchestra.run.smithi182.stdout: "wait": { 2023-07-14T11:54:32.885 INFO:teuthology.orchestra.run.smithi182.stdout: "avgcount": 0, 2023-07-14T11:54:32.885 INFO:teuthology.orchestra.run.smithi182.stdout: "sum": 0.000000000, 2023-07-14T11:54:32.885 INFO:teuthology.orchestra.run.smithi182.stdout: "avgtime": 0.000000000 2023-07-14T11:54:32.885 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:32.885 INFO:teuthology.orchestra.run.smithi182.stdout: } 2023-07-14T11:54:32.886 INFO:teuthology.orchestra.run.smithi182.stdout:} 2023-07-14T11:54:32.886 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:32 smithi182 ceph-mon[129366]: pgmap v1023: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:54:32.886 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:32 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:33.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:32 smithi175 ceph-mon[129081]: pgmap v1023: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:54:33.149 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:32 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:33.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:32 smithi161 ceph-mon[95526]: pgmap v1023: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:54:33.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:32 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:33.572 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-07-14T11:54:33.572 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-07-14T11:54:33.572 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-07-14T11:54:33.573 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-07-14T11:54:33.573 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-07-14T11:54:33.573 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-14T11:54:33.583 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-14T11:54:33.584 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:54:33.584 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:33.630 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:54:33.631 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:33.717 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:33.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.986+0000 7fc300fff700 1 -- 172.21.15.161:0/2753292448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc100480 msgr2=0x7fc2fc10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.986+0000 7fc300fff700 1 --2- 172.21.15.161:0/2753292448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc100480 0x7fc2fc10c910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009f90 tx=0x7fc2f400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:33.969 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.987+0000 7fc300fff700 1 -- 172.21.15.161:0/2753292448 shutdown_connections 2023-07-14T11:54:33.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.987+0000 7fc300fff700 1 --2- 172.21.15.161:0/2753292448 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc100480 0x7fc2fc10c910 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.987+0000 7fc300fff700 1 --2- 172.21.15.161:0/2753292448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc105ae0 0x7fc2fc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.987+0000 7fc300fff700 1 --2- 172.21.15.161:0/2753292448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2fc0ffad0 0x7fc2fc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.987+0000 7fc300fff700 1 -- 172.21.15.161:0/2753292448 >> 172.21.15.161:0/2753292448 conn(0x7fc2fc0fb590 msgr2=0x7fc2fc0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:33.970 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.987+0000 7fc300fff700 1 -- 172.21.15.161:0/2753292448 shutdown_connections 2023-07-14T11:54:33.971 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.987+0000 7fc300fff700 1 -- 172.21.15.161:0/2753292448 wait complete. 2023-07-14T11:54:33.972 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.988+0000 7fc300fff700 1 Processor -- start 2023-07-14T11:54:33.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc300fff700 1 -- start start 2023-07-14T11:54:33.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc300fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc0ffad0 0x7fc2fc195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc300fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2fc100480 0x7fc2fc1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc300fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc105ae0 0x7fc2fc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.973 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc300fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc199ba0 con 0x7fc2fc100480 2023-07-14T11:54:33.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc300fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc199d10 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc300fff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc199ed0 con 0x7fc2fc105ae0 2023-07-14T11:54:33.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc2fa59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc0ffad0 0x7fc2fc195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc2fa59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc0ffad0 0x7fc2fc195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52536/0 (socket says 172.21.15.161:52536) 2023-07-14T11:54:33.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.989+0000 7fc2fa59c700 1 -- 172.21.15.161:0/2150438164 learned_addr learned my addr 172.21.15.161:0/2150438164 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:33.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2fad9d700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc105ae0 0x7fc2fc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2f9d9b700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2fc100480 0x7fc2fc1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2fa59c700 1 -- 172.21.15.161:0/2150438164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc105ae0 msgr2=0x7fc2fc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2fa59c700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc105ae0 0x7fc2fc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2fa59c700 1 -- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2fc100480 msgr2=0x7fc2fc1961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2fa59c700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2fc100480 0x7fc2fc1961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2fa59c700 1 -- 172.21.15.161:0/2150438164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f4009c40 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2f9d9b700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2fc100480 0x7fc2fc1961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:33.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2fa59c700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc0ffad0 0x7fc2fc195cb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00e9c0 tx=0x7fc2ec00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2eb7fe700 1 -- 172.21.15.161:0/2150438164 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00cbd0 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2eb7fe700 1 -- 172.21.15.161:0/2150438164 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2ec00eeb0 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc19a1b0 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc2eb7fe700 1 -- 172.21.15.161:0/2150438164 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec005580 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.990+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc1a2b10 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.992+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc067300 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.993+0000 7fc2eb7fe700 1 -- 172.21.15.161:0/2150438164 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc2ec004830 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.995+0000 7fc2eb7fe700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2e40a1380 0x7fc2e40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.995+0000 7fc2eb7fe700 1 -- 172.21.15.161:0/2150438164 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fc2ec014070 con 0x7fc2fc0ffad0 2023-07-14T11:54:33.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.995+0000 7fc2f9d9b700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2e40a1380 0x7fc2e40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.996+0000 7fc2f9d9b700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2e40a1380 0x7fc2e40a3830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc197280 tx=0x7fc2f0009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:33.998+0000 7fc2eb7fe700 1 -- 172.21.15.161:0/2150438164 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc2ec0963a0 con 0x7fc2fc0ffad0 2023-07-14T11:54:34.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.200+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2fc1a2d60 con 0x7fc2fc0ffad0 2023-07-14T11:54:34.183 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.201+0000 7fc2eb7fe700 1 -- 172.21.15.161:0/2150438164 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+25182 (secure 0 0 0) 0x7fc2ec0999c0 con 0x7fc2fc0ffad0 2023-07-14T11:54:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:54:34.184 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":145,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:54:18.326616+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/142996677":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/3308966068":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/389110447":"2023-07-15T11:22:15.015135+0000","172.21.15.161:6801/2004305703":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/2004305703":"2023-07-15T11:22:15.015135+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/182250477":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]},{"pool":4,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:34.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2e40a1380 msgr2=0x7fc2e40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2e40a1380 0x7fc2e40a3830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc197280 tx=0x7fc2f0009260 comp rx=0 tx=0).stop 2023-07-14T11:54:34.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc0ffad0 msgr2=0x7fc2fc195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc0ffad0 0x7fc2fc195cb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00e9c0 tx=0x7fc2ec00ecd0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.187 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 shutdown_connections 2023-07-14T11:54:34.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fc2e40a1380 0x7fc2e40a3830 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fc2fc105ae0 0x7fc2fc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2fc0ffad0 0x7fc2fc195cb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.188 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 --2- 172.21.15.161:0/2150438164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2fc100480 0x7fc2fc1961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 >> 172.21.15.161:0/2150438164 conn(0x7fc2fc0fb590 msgr2=0x7fc2fc0fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 shutdown_connections 2023-07-14T11:54:34.189 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.204+0000 7fc300fff700 1 -- 172.21.15.161:0/2150438164 wait complete. 2023-07-14T11:54:34.203 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.204 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.204 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:33 smithi161 ceph-mon[95526]: pgmap v1024: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:34.204 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:34.204 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:34.205 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:33 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:15:32.586339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:19:40.041863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:19:41.055318+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 144, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:19:44.554959+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 144, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:34.207 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:34.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:33 smithi175 ceph-mon[129081]: pgmap v1024: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:34.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:34.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:34.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:33 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 -- 172.21.15.161:0/1846195616 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f60105af0 msgr2=0x7f1f60105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 --2- 172.21.15.161:0/1846195616 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f60105af0 0x7f1f60105ec0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009990 tx=0x7f1f5c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 -- 172.21.15.161:0/1846195616 shutdown_connections 2023-07-14T11:54:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 --2- 172.21.15.161:0/1846195616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f60100490 0x7f1f6010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 --2- 172.21.15.161:0/1846195616 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f60105af0 0x7f1f60105ec0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 --2- 172.21.15.161:0/1846195616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f600ffae0 0x7f1f600fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 -- 172.21.15.161:0/1846195616 >> 172.21.15.161:0/1846195616 conn(0x7f1f600fb5e0 msgr2=0x7f1f600fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 -- 172.21.15.161:0/1846195616 shutdown_connections 2023-07-14T11:54:34.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.431+0000 7f1f673ff700 1 -- 172.21.15.161:0/1846195616 wait complete. 2023-07-14T11:54:34.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.432+0000 7f1f673ff700 1 Processor -- start 2023-07-14T11:54:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.432+0000 7f1f673ff700 1 -- start start 2023-07-14T11:54:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f673ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f600ffae0 0x7f1f60072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f673ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f60100490 0x7f1f60072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f673ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60105af0 0x7f1f601a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f673ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60073580 con 0x7f1f60100490 2023-07-14T11:54:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f673ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f601a4310 con 0x7f1f60105af0 2023-07-14T11:54:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f673ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f601a4450 con 0x7f1f600ffae0 2023-07-14T11:54:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f600ffae0 0x7f1f60072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60105af0 0x7f1f601a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6499a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f60100490 0x7f1f60072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60105af0 0x7f1f601a3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52542/0 (socket says 172.21.15.161:52542) 2023-07-14T11:54:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6519b700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f600ffae0 0x7f1f60072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49396/0 (socket says 172.21.15.161:49396) 2023-07-14T11:54:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 -- 172.21.15.161:0/4286209593 learned_addr learned my addr 172.21.15.161:0/4286209593 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 -- 172.21.15.161:0/4286209593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f600ffae0 msgr2=0x7f1f60072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f600ffae0 0x7f1f60072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 -- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f60100490 msgr2=0x7f1f60072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f60100490 0x7f1f60072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6599c700 1 -- 172.21.15.161:0/4286209593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0095f0 con 0x7f1f60105af0 2023-07-14T11:54:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.433+0000 7f1f6519b700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f600ffae0 0x7f1f60072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.434+0000 7f1f6599c700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60105af0 0x7f1f601a3dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800ef30 tx=0x7f1f5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.434+0000 7f1f567fc700 1 -- 172.21.15.161:0/4286209593 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58018070 con 0x7f1f60105af0 2023-07-14T11:54:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.434+0000 7f1f567fc700 1 -- 172.21.15.161:0/4286209593 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f5800f040 con 0x7f1f60105af0 2023-07-14T11:54:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.434+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f601a4650 con 0x7f1f60105af0 2023-07-14T11:54:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.434+0000 7f1f567fc700 1 -- 172.21.15.161:0/4286209593 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800cd00 con 0x7f1f60105af0 2023-07-14T11:54:34.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.434+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f601a4b30 con 0x7f1f60105af0 2023-07-14T11:54:34.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.435+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f60067290 con 0x7f1f60105af0 2023-07-14T11:54:34.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.437+0000 7f1f567fc700 1 -- 172.21.15.161:0/4286209593 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1f58003a00 con 0x7f1f60105af0 2023-07-14T11:54:34.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.438+0000 7f1f567fc700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1f4c0a12c0 0x7f1f4c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.438+0000 7f1f567fc700 1 -- 172.21.15.161:0/4286209593 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f1f580ccb50 con 0x7f1f60105af0 2023-07-14T11:54:34.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.438+0000 7f1f6519b700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1f4c0a12c0 0x7f1f4c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.439+0000 7f1f6519b700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1f4c0a12c0 0x7f1f4c0a3770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b2e0 tx=0x7f1f5c005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.442+0000 7f1f567fc700 1 -- 172.21.15.161:0/4286209593 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1f58095c30 con 0x7f1f60105af0 2023-07-14T11:54:34.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:33 smithi182 ceph-mon[129366]: pgmap v1024: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:34.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:34.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:34.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:33 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' 2023-07-14T11:54:34.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.642+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f601a4d50 con 0x7f1f60105af0 2023-07-14T11:54:34.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.642+0000 7f1f567fc700 1 -- 172.21.15.161:0/4286209593 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f58099250 con 0x7f1f60105af0 2023-07-14T11:54:34.625 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2023-07-14T11:54:34.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1f4c0a12c0 msgr2=0x7f1f4c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1f4c0a12c0 0x7f1f4c0a3770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b2e0 tx=0x7f1f5c005fd0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60105af0 msgr2=0x7f1f601a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60105af0 0x7f1f601a3dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800ef30 tx=0x7f1f5800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 shutdown_connections 2023-07-14T11:54:34.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f1f4c0a12c0 0x7f1f4c0a3770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60105af0 0x7f1f601a3dd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1f600ffae0 0x7f1f60072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 --2- 172.21.15.161:0/4286209593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f60100490 0x7f1f60072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 >> 172.21.15.161:0/4286209593 conn(0x7f1f600fb5e0 msgr2=0x7f1f600fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 shutdown_connections 2023-07-14T11:54:34.629 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.645+0000 7f1f673ff700 1 -- 172.21.15.161:0/4286209593 wait complete. 2023-07-14T11:54:34.646 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:34.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 -- 172.21.15.161:0/405631058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 msgr2=0x7fcfcc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/405631058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc0fff40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009990 tx=0x7fcfc0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 -- 172.21.15.161:0/405631058 shutdown_connections 2023-07-14T11:54:34.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/405631058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfcc100480 0x7fcfcc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/405631058 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcfcc105ae0 0x7fcfcc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/405631058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc0fff40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 -- 172.21.15.161:0/405631058 >> 172.21.15.161:0/405631058 conn(0x7fcfcc0fb5f0 msgr2=0x7fcfcc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 -- 172.21.15.161:0/405631058 shutdown_connections 2023-07-14T11:54:34.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.867+0000 7fcfd15ff700 1 -- 172.21.15.161:0/405631058 wait complete. 2023-07-14T11:54:34.853 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.868+0000 7fcfd15ff700 1 Processor -- start 2023-07-14T11:54:34.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfd15ff700 1 -- start start 2023-07-14T11:54:34.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfd15ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfd15ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcfcc100480 0x7fcfcc198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfd15ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfcc105ae0 0x7fcfcc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.854 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfd15ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc19cde0 con 0x7fcfcc105ae0 2023-07-14T11:54:34.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfd15ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc19cf50 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfd15ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc19d110 con 0x7fcfcc100480 2023-07-14T11:54:34.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfcaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfcaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52558/0 (socket says 172.21.15.161:52558) 2023-07-14T11:54:34.855 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfca7fc700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcfcc100480 0x7fcfcc198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfcaffd700 1 -- 172.21.15.161:0/1547249213 learned_addr learned my addr 172.21.15.161:0/1547249213 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:34.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.869+0000 7fcfcb7fe700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfcc105ae0 0x7fcfcc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfcaffd700 1 -- 172.21.15.161:0/1547249213 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcfcc100480 msgr2=0x7fcfcc198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfcaffd700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcfcc100480 0x7fcfcc198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.856 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfcaffd700 1 -- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfcc105ae0 msgr2=0x7fcfcc19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfcaffd700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfcc105ae0 0x7fcfcc19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfcaffd700 1 -- 172.21.15.161:0/1547249213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00095f0 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfcaffd700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc197e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00e9a0 tx=0x7fcfbc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfb3fff700 1 -- 172.21.15.161:0/1547249213 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc009790 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.857 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfb3fff700 1 -- 172.21.15.161:0/1547249213 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfbc00cbd0 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfb3fff700 1 -- 172.21.15.161:0/1547249213 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc018680 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc19d3f0 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.870+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc1a4c30 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.858 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.871+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc067290 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.873+0000 7fcfb3fff700 1 -- 172.21.15.161:0/1547249213 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcfbc00cd40 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.874+0000 7fcfb3fff700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcfb40a1340 0x7fcfb40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.859 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.874+0000 7fcfb3fff700 1 -- 172.21.15.161:0/1547249213 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fcfbc014070 con 0x7fcfcc0ffad0 2023-07-14T11:54:34.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.875+0000 7fcfca7fc700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcfb40a1340 0x7fcfb40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.875+0000 7fcfca7fc700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcfb40a1340 0x7fcfb40a37f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc199410 tx=0x7fcfc0005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.860 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:34.876+0000 7fcfb3fff700 1 -- 172.21.15.161:0/1547249213 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcfbc096340 con 0x7fcfcc0ffad0 2023-07-14T11:54:35.059 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.077+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfcc1a4ed0 con 0x7fcfcc0ffad0 2023-07-14T11:54:35.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.078+0000 7fcfb3fff700 1 -- 172.21.15.161:0/1547249213 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfbc099960 con 0x7fcfcc0ffad0 2023-07-14T11:54:35.060 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:54:35.062 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcfb40a1340 msgr2=0x7fcfb40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcfb40a1340 0x7fcfb40a37f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc199410 tx=0x7fcfc0005fd0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 msgr2=0x7fcfcc197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.063 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc197e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00e9a0 tx=0x7fcfbc00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 shutdown_connections 2023-07-14T11:54:35.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fcfb40a1340 0x7fcfb40a37f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfcc105ae0 0x7fcfcc19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.064 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc0ffad0 0x7fcfcc197e00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 --2- 172.21.15.161:0/1547249213 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fcfcc100480 0x7fcfcc198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 >> 172.21.15.161:0/1547249213 conn(0x7fcfcc0fb5f0 msgr2=0x7fcfcc10aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 shutdown_connections 2023-07-14T11:54:35.065 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.080+0000 7fcfd15ff700 1 -- 172.21.15.161:0/1547249213 wait complete. 2023-07-14T11:54:35.081 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:35.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:34 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2150438164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:54:35.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:34 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/4286209593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:54:35.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 -- 172.21.15.161:0/3324184453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c100490 msgr2=0x7f639c10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3324184453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c100490 0x7f639c10c920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f639400b0a0 tx=0x7f639400b3b0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 -- 172.21.15.161:0/3324184453 shutdown_connections 2023-07-14T11:54:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3324184453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c100490 0x7f639c10c920 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3324184453 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f639c105af0 0x7f639c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3324184453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f639c0ffae0 0x7f639c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 -- 172.21.15.161:0/3324184453 >> 172.21.15.161:0/3324184453 conn(0x7f639c0fb5e0 msgr2=0x7f639c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 -- 172.21.15.161:0/3324184453 shutdown_connections 2023-07-14T11:54:35.287 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.303+0000 7f63a33ff700 1 -- 172.21.15.161:0/3324184453 wait complete. 2023-07-14T11:54:35.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.304+0000 7f63a33ff700 1 Processor -- start 2023-07-14T11:54:35.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.304+0000 7f63a33ff700 1 -- start start 2023-07-14T11:54:35.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a33ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f639c0ffae0 0x7f639c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.289 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a33ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f639c100490 0x7f639c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a33ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c105af0 0x7f639c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a33ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19ad90 con 0x7f639c0ffae0 2023-07-14T11:54:35.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a33ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19af00 con 0x7f639c105af0 2023-07-14T11:54:35.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a33ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19b0c0 con 0x7f639c100490 2023-07-14T11:54:35.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f639c0ffae0 0x7f639c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a099a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f639c100490 0x7f639c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a199c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c105af0 0x7f639c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a099a700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f639c100490 0x7f639c1962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:49446/0 (socket says 172.21.15.161:49446) 2023-07-14T11:54:35.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a099a700 1 -- 172.21.15.161:0/3387701471 learned_addr learned my addr 172.21.15.161:0/3387701471 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:35.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a199c700 1 -- 172.21.15.161:0/3387701471 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f639c100490 msgr2=0x7f639c1962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a199c700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f639c100490 0x7f639c1962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a199c700 1 -- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f639c0ffae0 msgr2=0x7f639c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a199c700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f639c0ffae0 0x7f639c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a199c700 1 -- 172.21.15.161:0/3387701471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6398009520 con 0x7f639c105af0 2023-07-14T11:54:35.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.305+0000 7f63a199c700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c105af0 0x7f639c19a750 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6394009fd0 tx=0x7f6394007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.306+0000 7f63927fc700 1 -- 172.21.15.161:0/3387701471 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400e040 con 0x7f639c105af0 2023-07-14T11:54:35.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.306+0000 7f63927fc700 1 -- 172.21.15.161:0/3387701471 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f639401d470 con 0x7f639c105af0 2023-07-14T11:54:35.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.306+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6394009d00 con 0x7f639c105af0 2023-07-14T11:54:35.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.306+0000 7f63927fc700 1 -- 172.21.15.161:0/3387701471 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940052a0 con 0x7f639c105af0 2023-07-14T11:54:35.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.306+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c1a2a00 con 0x7f639c105af0 2023-07-14T11:54:35.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.307+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c067290 con 0x7f639c105af0 2023-07-14T11:54:35.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.309+0000 7f63927fc700 1 -- 172.21.15.161:0/3387701471 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f639401d5e0 con 0x7f639c105af0 2023-07-14T11:54:35.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.310+0000 7f63927fc700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f63880a1340 0x7f63880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.310+0000 7f63927fc700 1 -- 172.21.15.161:0/3387701471 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f63940ce850 con 0x7f639c105af0 2023-07-14T11:54:35.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.310+0000 7f63a119b700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f63880a1340 0x7f63880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.311+0000 7f63a119b700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f63880a1340 0x7f63880a37f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6398009d70 tx=0x7f6398009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.313+0000 7f63927fc700 1 -- 172.21.15.161:0/3387701471 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6394097930 con 0x7f639c105af0 2023-07-14T11:54:35.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:34 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2150438164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:54:35.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:34 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/4286209593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:54:35.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:34 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2150438164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:54:35.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:34 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/4286209593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:54:35.493 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.511+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f639c1a2b90 con 0x7f639c105af0 2023-07-14T11:54:35.494 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.512+0000 7f63927fc700 1 -- 172.21.15.161:0/3387701471 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7f639409af50 con 0x7f639c105af0 2023-07-14T11:54:35.494 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2023-07-14T11:54:35.496 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f63880a1340 msgr2=0x7f63880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f63880a1340 0x7f63880a37f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6398009d70 tx=0x7f6398009260 comp rx=0 tx=0).stop 2023-07-14T11:54:35.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c105af0 msgr2=0x7f639c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c105af0 0x7f639c19a750 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6394009fd0 tx=0x7f6394007930 comp rx=0 tx=0).stop 2023-07-14T11:54:35.497 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 shutdown_connections 2023-07-14T11:54:35.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f63880a1340 0x7f63880a37f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f639c0ffae0 0x7f639c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c105af0 0x7f639c19a750 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 --2- 172.21.15.161:0/3387701471 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f639c100490 0x7f639c1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 >> 172.21.15.161:0/3387701471 conn(0x7f639c0fb5e0 msgr2=0x7f639c106940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.514+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 shutdown_connections 2023-07-14T11:54:35.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.515+0000 7f63a33ff700 1 -- 172.21.15.161:0/3387701471 wait complete. 2023-07-14T11:54:35.515 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:35.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 -- 172.21.15.161:0/938939752 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c0ffad0 msgr2=0x7f4d9c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 --2- 172.21.15.161:0/938939752 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c0fff40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009990 tx=0x7f4d8c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 -- 172.21.15.161:0/938939752 shutdown_connections 2023-07-14T11:54:35.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 --2- 172.21.15.161:0/938939752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d9c100480 0x7f4d9c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 --2- 172.21.15.161:0/938939752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c105ae0 0x7f4d9c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 --2- 172.21.15.161:0/938939752 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c0fff40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 -- 172.21.15.161:0/938939752 >> 172.21.15.161:0/938939752 conn(0x7f4d9c0fb5f0 msgr2=0x7f4d9c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 -- 172.21.15.161:0/938939752 shutdown_connections 2023-07-14T11:54:35.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.736+0000 7f4da3bff700 1 -- 172.21.15.161:0/938939752 wait complete. 2023-07-14T11:54:35.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.737+0000 7f4da3bff700 1 Processor -- start 2023-07-14T11:54:35.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.737+0000 7f4da3bff700 1 -- start start 2023-07-14T11:54:35.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.737+0000 7f4da3bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.726 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da3bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c100480 0x7f4d9c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da3bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d9c105ae0 0x7f4d9c195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.727 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da3bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c1960b0 con 0x7f4d9c105ae0 2023-07-14T11:54:35.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da3bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c196220 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.728 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da3bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c1963e0 con 0x7f4d9c100480 2023-07-14T11:54:35.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c19b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52602/0 (socket says 172.21.15.161:52602) 2023-07-14T11:54:35.729 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 -- 172.21.15.161:0/2739397525 learned_addr learned my addr 172.21.15.161:0/2739397525 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:35.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da119a700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c100480 0x7f4d9c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.730 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da219c700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d9c105ae0 0x7f4d9c195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 -- 172.21.15.161:0/2739397525 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c100480 msgr2=0x7f4d9c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c100480 0x7f4d9c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.731 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 -- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d9c105ae0 msgr2=0x7f4d9c195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d9c105ae0 0x7f4d9c195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.732 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 -- 172.21.15.161:0/2739397525 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c0095f0 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.738+0000 7f4da199b700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c19b9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800e9a0 tx=0x7f4d9800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.733 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.739+0000 7f4d92ffd700 1 -- 172.21.15.161:0/2739397525 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98009790 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.739+0000 7f4d92ffd700 1 -- 172.21.15.161:0/2739397525 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d98010430 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.739+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c1966c0 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.734 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.739+0000 7f4d92ffd700 1 -- 172.21.15.161:0/2739397525 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98018680 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.739+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c1a29c0 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.735 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.740+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c067290 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.736 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.742+0000 7f4d92ffd700 1 -- 172.21.15.161:0/2739397525 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4d980105a0 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.743+0000 7f4d92ffd700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4d880a12c0 0x7f4d880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.737 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.743+0000 7f4d92ffd700 1 -- 172.21.15.161:0/2739397525 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f4d98014070 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.743+0000 7f4da119a700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4d880a12c0 0x7f4d880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.738 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.744+0000 7f4da119a700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4d880a12c0 0x7f4d880a3770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c006010 tx=0x7f4d8c0058e0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.739 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.747+0000 7f4d92ffd700 1 -- 172.21.15.161:0/2739397525 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4d98096430 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.948+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4d9c1a2c20 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.948+0000 7f4d92ffd700 1 -- 172.21.15.161:0/2739397525 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v145) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4d98099a50 con 0x7f4d9c0ffad0 2023-07-14T11:54:35.931 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2023-07-14T11:54:35.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.950+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4d880a12c0 msgr2=0x7f4d880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.950+0000 7f4da3bff700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4d880a12c0 0x7f4d880a3770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c006010 tx=0x7f4d8c0058e0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.950+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c0ffad0 msgr2=0x7f4d9c19b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.950+0000 7f4da3bff700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c19b9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800e9a0 tx=0x7f4d9800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.950+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 shutdown_connections 2023-07-14T11:54:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.950+0000 7f4da3bff700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f4d880a12c0 0x7f4d880a3770 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.950+0000 7f4da3bff700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d9c105ae0 0x7f4d9c195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.951+0000 7f4da3bff700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d9c0ffad0 0x7f4d9c19b9d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.951+0000 7f4da3bff700 1 --2- 172.21.15.161:0/2739397525 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f4d9c100480 0x7f4d9c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.951+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 >> 172.21.15.161:0/2739397525 conn(0x7f4d9c0fb5f0 msgr2=0x7f4d9c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.951+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 shutdown_connections 2023-07-14T11:54:35.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:35.951+0000 7f4da3bff700 1 -- 172.21.15.161:0/2739397525 wait complete. 2023-07-14T11:54:35.952 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:36.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 -- 172.21.15.161:0/4235938338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a81059e0 msgr2=0x7f45a8105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 --2- 172.21.15.161:0/4235938338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a81059e0 0x7f45a8105db0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4598009990 tx=0x7f4598009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 -- 172.21.15.161:0/4235938338 shutdown_connections 2023-07-14T11:54:36.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 --2- 172.21.15.161:0/4235938338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a8100380 0x7f45a810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 --2- 172.21.15.161:0/4235938338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a81059e0 0x7f45a8105db0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 --2- 172.21.15.161:0/4235938338 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45a80ff9d0 0x7f45a80ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 -- 172.21.15.161:0/4235938338 >> 172.21.15.161:0/4235938338 conn(0x7f45a80fb4b0 msgr2=0x7f45a80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:36.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 -- 172.21.15.161:0/4235938338 shutdown_connections 2023-07-14T11:54:36.155 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.171+0000 7f45a759e700 1 -- 172.21.15.161:0/4235938338 wait complete. 2023-07-14T11:54:36.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.173+0000 7f45a759e700 1 Processor -- start 2023-07-14T11:54:36.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.173+0000 7f45a759e700 1 -- start start 2023-07-14T11:54:36.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.173+0000 7f45a759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a80ff9d0 0x7f45a819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.157 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.173+0000 7f45a759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a8100380 0x7f45a819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.173+0000 7f45a759e700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45a81059e0 0x7f45a81979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.173+0000 7f45a759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a8198040 con 0x7f45a80ff9d0 2023-07-14T11:54:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.173+0000 7f45a759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81981b0 con 0x7f45a8100380 2023-07-14T11:54:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a759e700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a8198370 con 0x7f45a81059e0 2023-07-14T11:54:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a80ff9d0 0x7f45a819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a8100380 0x7f45a819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a8100380 0x7f45a819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52634/0 (socket says 172.21.15.161:52634) 2023-07-14T11:54:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 -- 172.21.15.161:0/3565600829 learned_addr learned my addr 172.21.15.161:0/3565600829 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a6d9d700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45a81059e0 0x7f45a81979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 -- 172.21.15.161:0/3565600829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45a81059e0 msgr2=0x7f45a81979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45a81059e0 0x7f45a81979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 -- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a80ff9d0 msgr2=0x7f45a819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a80ff9d0 0x7f45a819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 -- 172.21.15.161:0/3565600829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45980095f0 con 0x7f45a8100380 2023-07-14T11:54:36.160 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a659c700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a80ff9d0 0x7f45a819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.174+0000 7f45a5d9b700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a8100380 0x7f45a819dea0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9a0 tx=0x7f459c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.175+0000 7f45977fe700 1 -- 172.21.15.161:0/3565600829 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c009790 con 0x7f45a8100380 2023-07-14T11:54:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.175+0000 7f45977fe700 1 -- 172.21.15.161:0/3565600829 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f459c010430 con 0x7f45a8100380 2023-07-14T11:54:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.175+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a8198650 con 0x7f45a8100380 2023-07-14T11:54:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.175+0000 7f45977fe700 1 -- 172.21.15.161:0/3565600829 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c018680 con 0x7f45a8100380 2023-07-14T11:54:36.162 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.175+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a81a4950 con 0x7f45a8100380 2023-07-14T11:54:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.176+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a8067220 con 0x7f45a8100380 2023-07-14T11:54:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.178+0000 7f45977fe700 1 -- 172.21.15.161:0/3565600829 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f459c0105a0 con 0x7f45a8100380 2023-07-14T11:54:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.179+0000 7f45977fe700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45900a13e0 0x7f45900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.179+0000 7f45977fe700 1 -- 172.21.15.161:0/3565600829 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f459c014070 con 0x7f45a8100380 2023-07-14T11:54:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.179+0000 7f45a659c700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45900a13e0 0x7f45900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.180+0000 7f45a659c700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45900a13e0 0x7f45900a3890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f459800b2e0 tx=0x7f4598005fb0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.164 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.182+0000 7f45977fe700 1 -- 172.21.15.161:0/3565600829 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f459c0964a0 con 0x7f45a8100380 2023-07-14T11:54:36.225 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:35 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1547249213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:35 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3387701471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:35 smithi161 ceph-mon[95526]: pgmap v1025: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:36.226 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:35 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2739397525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.399+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45a8198ee0 con 0x7f45a8100380 2023-07-14T11:54:36.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.399+0000 7f45977fe700 1 -- 172.21.15.161:0/3565600829 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v145) v1 ==== 81+0+956 (secure 0 0 0) 0x7f459c026020 con 0x7f45a8100380 2023-07-14T11:54:36.382 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/142996677 2023-07-15T11:22:15.015135+0000 2023-07-14T11:54:36.382 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/3308966068 2023-07-15T11:22:15.015135+0000 2023-07-14T11:54:36.383 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/389110447 2023-07-15T11:22:15.015135+0000 2023-07-14T11:54:36.383 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6801/2004305703 2023-07-15T11:22:15.015135+0000 2023-07-14T11:54:36.383 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/871206111 2023-07-15T11:13:14.005108+0000 2023-07-14T11:54:36.383 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6800/2004305703 2023-07-15T11:22:15.015135+0000 2023-07-14T11:54:36.383 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6801/3292405426 2023-07-15T11:13:14.005108+0000 2023-07-14T11:54:36.384 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6800/3292405426 2023-07-15T11:13:14.005108+0000 2023-07-14T11:54:36.384 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/629268496 2023-07-15T11:12:47.212143+0000 2023-07-14T11:54:36.384 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/832098793 2023-07-15T11:13:14.005108+0000 2023-07-14T11:54:36.384 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6800/4131434778 2023-07-15T11:12:47.212143+0000 2023-07-14T11:54:36.384 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:6801/4131434778 2023-07-15T11:12:47.212143+0000 2023-07-14T11:54:36.385 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/182250477 2023-07-15T11:22:15.015135+0000 2023-07-14T11:54:36.385 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/3333551885 2023-07-15T11:12:47.212143+0000 2023-07-14T11:54:36.385 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/1691634661 2023-07-15T11:12:47.212143+0000 2023-07-14T11:54:36.385 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161:0/3118869366 2023-07-15T11:13:14.005108+0000 2023-07-14T11:54:36.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45900a13e0 msgr2=0x7f45900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45900a13e0 0x7f45900a3890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f459800b2e0 tx=0x7f4598005fb0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a8100380 msgr2=0x7f45a819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a8100380 0x7f45a819dea0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9a0 tx=0x7f459c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 shutdown_connections 2023-07-14T11:54:36.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f45900a13e0 0x7f45900a3890 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a80ff9d0 0x7f45a819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f45a81059e0 0x7f45a81979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 --2- 172.21.15.161:0/3565600829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45a8100380 0x7f45a819dea0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 >> 172.21.15.161:0/3565600829 conn(0x7f45a80fb4b0 msgr2=0x7f45a8072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:36.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 shutdown_connections 2023-07-14T11:54:36.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:36.402+0000 7f45a759e700 1 -- 172.21.15.161:0/3565600829 wait complete. 2023-07-14T11:54:36.389 INFO:teuthology.orchestra.run.smithi161.stderr:listed 16 entries 2023-07-14T11:54:36.398 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:35 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1547249213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:35 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3387701471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:35 smithi175 ceph-mon[129081]: pgmap v1025: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:36.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:35 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2739397525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.403 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-14T11:54:36.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:36.404 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-14T11:54:36.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:35 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1547249213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:35 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3387701471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:35 smithi182 ceph-mon[129366]: pgmap v1025: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T11:54:36.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:35 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2739397525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T11:54:36.593 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T11:54:36.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:36.594 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:54:36.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:36.746 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-07-14T11:54:36.802 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:54:36.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:36.803 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:54:36.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:36.875 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2023-07-14T11:54:36.980 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-14T11:54:36.980 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T11:54:36.981 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:37.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345616k,nr_inodes=4086404,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275012k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/2cad584fdbc26945d60db2e7f1d2be14e63813b25fbcb9cf081a61d3c8200bcf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/2cad584fdbc26945d60db2e7f1d2be14e63813b25fbcb9cf081a61d3c8200bcf/diff,workdir=/var/lib/containers/storage/overlay/2cad584fdbc26945d60db2e7f1d2be14e63813b25fbcb9cf081a61d3c8200bcf/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cde7770a3bd70d3ef97c95d47164720926f2a052bda061dfd0bfdfb933320b2f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/cde7770a3bd70d3ef97c95d47164720926f2a052bda061dfd0bfdfb933320b2f/diff,workdir=/var/lib/containers/storage/overlay/cde7770a3bd70d3ef97c95d47164720926f2a052bda061dfd0bfdfb933320b2f/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275012k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/8ca03216adfe99560bfcfaaf4ab3c11299f1f8f54f9156e7ff5c72adcbeb7afc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/8ca03216adfe99560bfcfaaf4ab3c11299f1f8f54f9156e7ff5c72adcbeb7afc/diff,workdir=/var/lib/containers/storage/overlay/8ca03216adfe99560bfcfaaf4ab3c11299f1f8f54f9156e7ff5c72adcbeb7afc/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4ff587adec541c3365bc748eb8bf15d7874af46798d8214293adf587ce202a80/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/4ff587adec541c3365bc748eb8bf15d7874af46798d8214293adf587ce202a80/diff,workdir=/var/lib/containers/storage/overlay/4ff587adec541c3365bc748eb8bf15d7874af46798d8214293adf587ce202a80/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/746f2215d46cac15fc979de9218ab0347cf8eecefc7bcf0f5e0bcde7b2dcfa9c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/746f2215d46cac15fc979de9218ab0347cf8eecefc7bcf0f5e0bcde7b2dcfa9c/diff,workdir=/var/lib/containers/storage/overlay/746f2215d46cac15fc979de9218ab0347cf8eecefc7bcf0f5e0bcde7b2dcfa9c/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/21d20b95c56fc9108370ace389ff6da664c868116f6d1820d13c77fc7a14e03e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/21d20b95c56fc9108370ace389ff6da664c868116f6d1820d13c77fc7a14e03e/diff,workdir=/var/lib/containers/storage/overlay/21d20b95c56fc9108370ace389ff6da664c868116f6d1820d13c77fc7a14e03e/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cbbb7ff723f3e6016ceede6d9dff127d4adeb71e9fdac48f3ce2b57d5d8f7580/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/cbbb7ff723f3e6016ceede6d9dff127d4adeb71e9fdac48f3ce2b57d5d8f7580/diff,workdir=/var/lib/containers/storage/overlay/cbbb7ff723f3e6016ceede6d9dff127d4adeb71e9fdac48f3ce2b57d5d8f7580/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6f639e7ef06c90bf9a264a052bf6fc6284945c11d4f726e33ccade783e2cc2ea/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/6f639e7ef06c90bf9a264a052bf6fc6284945c11d4f726e33ccade783e2cc2ea/diff,workdir=/var/lib/containers/storage/overlay/6f639e7ef06c90bf9a264a052bf6fc6284945c11d4f726e33ccade783e2cc2ea/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1b7af08bfcfa4503dda2dfd67c028e68203c5687cba88fa914543caed1315b47/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/1b7af08bfcfa4503dda2dfd67c028e68203c5687cba88fa914543caed1315b47/diff,workdir=/var/lib/containers/storage/overlay/1b7af08bfcfa4503dda2dfd67c028e68203c5687cba88fa914543caed1315b47/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f8a0a19eb97b4452bb092d00d256cfe2b19cb98e02cd83d237dcd65af98e900a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2I7D4EKSOSYT7UQYSWC572N6NT:/var/lib/containers/storage/overlay/l/FNWQRQS77UWZK6BCQ6ZSZ4IQ6G,upperdir=/var/lib/containers/storage/overlay/f8a0a19eb97b4452bb092d00d256cfe2b19cb98e02cd83d237dcd65af98e900a/diff,workdir=/var/lib/containers/storage/overlay/f8a0a19eb97b4452bb092d00d256cfe2b19cb98e02cd83d237dcd65af98e900a/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-14T11:54:37.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:37.028 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-07-14T11:54:37.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:37.113 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2023-07-14T11:54:37.215 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:36 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3565600829' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:54:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2023-07-14T11:54:37.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:54:37.232 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-14T11:54:37.243 INFO:tasks.cephadm:Teardown begin 2023-07-14T11:54:37.244 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T11:54:37.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:54:37.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:36 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3565600829' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:54:37.443 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:36 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3565600829' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T11:54:37.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 -- 172.21.15.161:0/529398630 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f83680ffaf0 msgr2=0x7f83680fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 --2- 172.21.15.161:0/529398630 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f83680ffaf0 0x7f83680fff60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8350009990 tx=0x7f8350009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.446 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 -- 172.21.15.161:0/529398630 shutdown_connections 2023-07-14T11:54:37.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 --2- 172.21.15.161:0/529398630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83681004a0 0x7f836810c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 --2- 172.21.15.161:0/529398630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8368105b00 0x7f8368105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 --2- 172.21.15.161:0/529398630 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f83680ffaf0 0x7f83680fff60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 -- 172.21.15.161:0/529398630 >> 172.21.15.161:0/529398630 conn(0x7f83680fb630 msgr2=0x7f83680fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.447 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 -- 172.21.15.161:0/529398630 shutdown_connections 2023-07-14T11:54:37.448 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.464+0000 7f836cdff700 1 -- 172.21.15.161:0/529398630 wait complete. 2023-07-14T11:54:37.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 Processor -- start 2023-07-14T11:54:37.449 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 -- start start 2023-07-14T11:54:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83680ffaf0 0x7f8368197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83681004a0 0x7f8368198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8368105b00 0x7f836819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836819cde0 con 0x7f83681004a0 2023-07-14T11:54:37.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836819cf50 con 0x7f83680ffaf0 2023-07-14T11:54:37.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.465+0000 7f836cdff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836819d110 con 0x7f8368105b00 2023-07-14T11:54:37.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83680ffaf0 0x7f8368197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83680ffaf0 0x7f8368197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:52646/0 (socket says 172.21.15.161:52646) 2023-07-14T11:54:37.451 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 -- 172.21.15.161:0/3367018132 learned_addr learned my addr 172.21.15.161:0/3367018132 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f8366d9d700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8368105b00 0x7f836819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f835ffff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83681004a0 0x7f8368198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 -- 172.21.15.161:0/3367018132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8368105b00 msgr2=0x7f836819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8368105b00 0x7f836819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 -- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83681004a0 msgr2=0x7f8368198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83681004a0 0x7f8368198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 -- 172.21.15.161:0/3367018132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83500095f0 con 0x7f83680ffaf0 2023-07-14T11:54:37.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f836659c700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83680ffaf0 0x7f8368197e00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9a0 tx=0x7f835800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.466+0000 7f835f7fe700 1 -- 172.21.15.161:0/3367018132 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358009790 con 0x7f83680ffaf0 2023-07-14T11:54:37.453 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.467+0000 7f835f7fe700 1 -- 172.21.15.161:0/3367018132 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8358010430 con 0x7f83680ffaf0 2023-07-14T11:54:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.467+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836819d3f0 con 0x7f83680ffaf0 2023-07-14T11:54:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.467+0000 7f835f7fe700 1 -- 172.21.15.161:0/3367018132 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358018680 con 0x7f83680ffaf0 2023-07-14T11:54:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.467+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83681a4c30 con 0x7f83680ffaf0 2023-07-14T11:54:37.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.468+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8368067300 con 0x7f83680ffaf0 2023-07-14T11:54:37.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.470+0000 7f835f7fe700 1 -- 172.21.15.161:0/3367018132 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f83580105a0 con 0x7f83680ffaf0 2023-07-14T11:54:37.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.471+0000 7f835f7fe700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f83540a1380 0x7f83540a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.455 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.471+0000 7f835f7fe700 1 -- 172.21.15.161:0/3367018132 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f8358014070 con 0x7f83680ffaf0 2023-07-14T11:54:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.472+0000 7f835ffff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f83540a1380 0x7f83540a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.472+0000 7f835ffff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f83540a1380 0x7f83540a3830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8368199410 tx=0x7f8350005fd0 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.474+0000 7f835f7fe700 1 -- 172.21.15.161:0/3367018132 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8358096440 con 0x7f83680ffaf0 2023-07-14T11:54:37.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.714+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8368199150 con 0x7f83680ffaf0 2023-07-14T11:54:37.698 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.716+0000 7f835f7fe700 1 -- 172.21.15.161:0/3367018132 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10864 (secure 0 0 0) 0x7f8358026020 con 0x7f83680ffaf0 2023-07-14T11:54:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:54:37.698 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:43:51.540083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f83540a1380 msgr2=0x7f83540a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f83540a1380 0x7f83540a3830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8368199410 tx=0x7f8350005fd0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83680ffaf0 msgr2=0x7f8368197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83680ffaf0 0x7f8368197e00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9a0 tx=0x7f835800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 shutdown_connections 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f83540a1380 0x7f83540a3830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f8368105b00 0x7f836819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83680ffaf0 0x7f8368197e00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 --2- 172.21.15.161:0/3367018132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83681004a0 0x7f8368198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 >> 172.21.15.161:0/3367018132 conn(0x7f83680fb630 msgr2=0x7f836810ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 shutdown_connections 2023-07-14T11:54:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.718+0000 7f836cdff700 1 -- 172.21.15.161:0/3367018132 wait complete. 2023-07-14T11:54:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 46 2023-07-14T11:54:37.720 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:54:37.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 -- 172.21.15.161:0/4024414312 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc072850 msgr2=0x7fd7bc072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/4024414312 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc072850 0x7fd7bc072c20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009990 tx=0x7fd7ac009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 -- 172.21.15.161:0/4024414312 shutdown_connections 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/4024414312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc10c3b0 0x7fd7bc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/4024414312 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc072850 0x7fd7bc072c20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/4024414312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7bc073160 0x7fd7bc10be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 -- 172.21.15.161:0/4024414312 >> 172.21.15.161:0/4024414312 conn(0x7fd7bc0fb610 msgr2=0x7fd7bc0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 -- 172.21.15.161:0/4024414312 shutdown_connections 2023-07-14T11:54:37.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.937+0000 7fd7c11ff700 1 -- 172.21.15.161:0/4024414312 wait complete. 2023-07-14T11:54:37.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.938+0000 7fd7c11ff700 1 Processor -- start 2023-07-14T11:54:37.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.938+0000 7fd7c11ff700 1 -- start start 2023-07-14T11:54:37.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7c11ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc072850 0x7fd7bc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7c11ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7bc073160 0x7fd7bc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7c11ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc10c3b0 0x7fd7bc19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7c11ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc19cec0 con 0x7fd7bc073160 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7c11ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc19d030 con 0x7fd7bc072850 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7c11ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc19d1f0 con 0x7fd7bc10c3b0 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc072850 0x7fd7bc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc072850 0x7fd7bc197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.161:51640/0 (socket says 172.21.15.161:51640) 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 -- 172.21.15.161:0/2890075009 learned_addr learned my addr 172.21.15.161:0/2890075009 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bb59e700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc10c3b0 0x7fd7bc19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7b259c700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7bc073160 0x7fd7bc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 -- 172.21.15.161:0/2890075009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc10c3b0 msgr2=0x7fd7bc19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc10c3b0 0x7fd7bc19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 -- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7bc073160 msgr2=0x7fd7bc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7bc073160 0x7fd7bc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7bad9d700 1 -- 172.21.15.161:0/2890075009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac0095f0 con 0x7fd7bc072850 2023-07-14T11:54:37.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.939+0000 7fd7b259c700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7bc073160 0x7fd7bc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:37.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.940+0000 7fd7bad9d700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc072850 0x7fd7bc197ee0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac005910 tx=0x7fd7ac004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.940+0000 7fd7b8d99700 1 -- 172.21.15.161:0/2890075009 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac01d070 con 0x7fd7bc072850 2023-07-14T11:54:37.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.940+0000 7fd7b8d99700 1 -- 172.21.15.161:0/2890075009 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7ac022470 con 0x7fd7bc072850 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.940+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc19d470 con 0x7fd7bc072850 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.940+0000 7fd7b8d99700 1 -- 172.21.15.161:0/2890075009 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac017650 con 0x7fd7bc072850 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.940+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc1a4d10 con 0x7fd7bc072850 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.941+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc106cd0 con 0x7fd7bc072850 2023-07-14T11:54:37.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.943+0000 7fd7b8d99700 1 -- 172.21.15.161:0/2890075009 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd7ac0225e0 con 0x7fd7bc072850 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.944+0000 7fd7b8d99700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7a80a1300 0x7fd7a80a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.945+0000 7fd7b8d99700 1 -- 172.21.15.161:0/2890075009 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7fd7ac0ce5c0 con 0x7fd7bc072850 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.945+0000 7fd7b259c700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7a80a1300 0x7fd7a80a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.945+0000 7fd7b259c700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7a80a1300 0x7fd7a80a37b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc1994f0 tx=0x7fd7a400b040 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:37.947+0000 7fd7b8d99700 1 -- 172.21.15.161:0/2890075009 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd7ac0976a0 con 0x7fd7bc072850 2023-07-14T11:54:38.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.194+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7bc1992d0 con 0x7fd7bc072850 2023-07-14T11:54:38.177 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.195+0000 7fd7b8d99700 1 -- 172.21.15.161:0/2890075009 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10864 (secure 0 0 0) 0x7fd7ac09acc0 con 0x7fd7bc072850 2023-07-14T11:54:38.178 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:54:38.178 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2631325720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2631325720},{"type":"v1","addr":"172.21.15.161:6835","nonce":2631325720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1651269993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1651269993},{"type":"v1","addr":"172.21.15.161:6839","nonce":1651269993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6841/208844218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6840","nonce":208844218},{"type":"v1","addr":"172.21.15.161:6841","nonce":208844218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.182:6833/2580899731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6832","nonce":2580899731},{"type":"v1","addr":"172.21.15.182:6833","nonce":2580899731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3899543425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3899543425},{"type":"v1","addr":"172.21.15.175:6833","nonce":3899543425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4200056265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4200056265},{"type":"v1","addr":"172.21.15.161:6837","nonce":4200056265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:19:41.650966+0000","modified":"2023-07-14T11:43:51.540083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24544,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4098726991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4098726991},{"type":"v1","addr":"172.21.15.175:6835","nonce":4098726991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6835/1383844371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6834","nonce":1383844371},{"type":"v1","addr":"172.21.15.182:6835","nonce":1383844371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.182:6837/1326049792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6836","nonce":1326049792},{"type":"v1","addr":"172.21.15.182:6837","nonce":1326049792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.175:6837/4186191407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4186191407},{"type":"v1","addr":"172.21.15.175:6837","nonce":4186191407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.182:6839/392865750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6838","nonce":392865750},{"type":"v1","addr":"172.21.15.182:6839","nonce":392865750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/2144664422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2144664422},{"type":"v1","addr":"172.21.15.175:6839","nonce":2144664422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:54:38.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.197+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7a80a1300 msgr2=0x7fd7a80a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.197+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7a80a1300 0x7fd7a80a37b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc1994f0 tx=0x7fd7a400b040 comp rx=0 tx=0).stop 2023-07-14T11:54:38.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.197+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc072850 msgr2=0x7fd7bc197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.180 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.197+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc072850 0x7fd7bc197ee0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac005910 tx=0x7fd7ac004710 comp rx=0 tx=0).stop 2023-07-14T11:54:38.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 shutdown_connections 2023-07-14T11:54:38.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7a80a1300 0x7fd7a80a37b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7bc10c3b0 0x7fd7bc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7bc072850 0x7fd7bc197ee0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.181 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 --2- 172.21.15.161:0/2890075009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7bc073160 0x7fd7bc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 >> 172.21.15.161:0/2890075009 conn(0x7fd7bc0fb610 msgr2=0x7fd7bc103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 shutdown_connections 2023-07-14T11:54:38.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.198+0000 7fd7c11ff700 1 -- 172.21.15.161:0/2890075009 wait complete. 2023-07-14T11:54:38.182 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 46 2023-07-14T11:54:38.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:38.261 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:37 smithi161 ceph-mon[95526]: pgmap v1026: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:54:38.261 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:37 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3367018132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:54:38.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:37 smithi175 ceph-mon[129081]: pgmap v1026: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:54:38.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:37 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3367018132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:54:38.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 -- 172.21.15.161:0/1199860301 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 msgr2=0x7f9380109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 --2- 172.21.15.161:0/1199860301 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f9380109b50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f936c009990 tx=0x7f936c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 -- 172.21.15.161:0/1199860301 shutdown_connections 2023-07-14T11:54:38.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 --2- 172.21.15.161:0/1199860301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9380100d50 0x7f9380101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 --2- 172.21.15.161:0/1199860301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938010a120 0x7f938010c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 --2- 172.21.15.161:0/1199860301 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f9380109b50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 -- 172.21.15.161:0/1199860301 >> 172.21.15.161:0/1199860301 conn(0x7f9380075c50 msgr2=0x7f9380076050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 -- 172.21.15.161:0/1199860301 shutdown_connections 2023-07-14T11:54:38.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.420+0000 7f9386bff700 1 -- 172.21.15.161:0/1199860301 wait complete. 2023-07-14T11:54:38.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.421+0000 7f9386bff700 1 Processor -- start 2023-07-14T11:54:38.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f9386bff700 1 -- start start 2023-07-14T11:54:38.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f9386bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9380100d50 0x7f938019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f9386bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f938019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f9386bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938010a120 0x7f9380195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f9386bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9380196080 con 0x7f9380100d50 2023-07-14T11:54:38.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f9386bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801961f0 con 0x7f938010a120 2023-07-14T11:54:38.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f9386bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801963b0 con 0x7f9380101660 2023-07-14T11:54:38.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.422+0000 7f937ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f938019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f938019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:35870/0 (socket says 172.21.15.161:35870) 2023-07-14T11:54:38.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 -- 172.21.15.161:0/1080754136 learned_addr learned my addr 172.21.15.161:0/1080754136 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:38.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f938519c700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938010a120 0x7f9380195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f938499b700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9380100d50 0x7f938019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 -- 172.21.15.161:0/1080754136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938010a120 msgr2=0x7f9380195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938010a120 0x7f9380195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 -- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9380100d50 msgr2=0x7f938019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9380100d50 0x7f938019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 -- 172.21.15.161:0/1080754136 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0095f0 con 0x7f9380101660 2023-07-14T11:54:38.410 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937ffff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f938019bee0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937dffb700 1 -- 172.21.15.161:0/1080754136 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c01d070 con 0x7f9380101660 2023-07-14T11:54:38.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937dffb700 1 -- 172.21.15.161:0/1080754136 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f936c022470 con 0x7f9380101660 2023-07-14T11:54:38.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f937dffb700 1 -- 172.21.15.161:0/1080754136 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c017650 con 0x7f9380101660 2023-07-14T11:54:38.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.423+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9380196630 con 0x7f9380101660 2023-07-14T11:54:38.411 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.424+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9380196850 con 0x7f9380101660 2023-07-14T11:54:38.412 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.424+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9380067750 con 0x7f9380101660 2023-07-14T11:54:38.413 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.426+0000 7f937dffb700 1 -- 172.21.15.161:0/1080754136 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f936c00fad0 con 0x7f9380101660 2023-07-14T11:54:38.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.428+0000 7f937dffb700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93700a1300 0x7f93700a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.428+0000 7f937dffb700 1 -- 172.21.15.161:0/1080754136 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f936c0ce720 con 0x7f9380101660 2023-07-14T11:54:38.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.428+0000 7f938499b700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93700a1300 0x7f93700a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.429+0000 7f938499b700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93700a1300 0x7f93700a37b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9374005950 tx=0x7f937400b410 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.414 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.430+0000 7f937dffb700 1 -- 172.21.15.161:0/1080754136 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f936c097800 con 0x7f9380101660 2023-07-14T11:54:38.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:37 smithi182 ceph-mon[129366]: pgmap v1026: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-14T11:54:38.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:37 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3367018132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:54:38.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.630+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9380196f20 con 0x7f9380101660 2023-07-14T11:54:38.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.632+0000 7f937dffb700 1 -- 172.21.15.161:0/1080754136 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+25182 (secure 0 0 0) 0x7f936c09ae20 con 0x7f9380101660 2023-07-14T11:54:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T11:54:38.615 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":145,"fsid":"2bf7fc3c-2237-11ee-9b32-001a4aab830c","created":"2023-07-14T11:12:16.073598+0000","modified":"2023-07-14T11:54:18.326616+0000","last_up_change":"2023-07-14T11:18:53.696286+0000","last_in_change":"2023-07-14T11:18:36.206192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:15:32.586339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:19:40.041863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:19:41.055318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:19:44.554959+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"606d353c-d690-4ba5-b3d5-2283accaf2e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6803","nonce":3629732803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6805","nonce":3629732803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6809","nonce":3629732803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3629732803},{"type":"v1","addr":"172.21.15.161:6807","nonce":3629732803}]},"public_addr":"172.21.15.161:6803/3629732803","cluster_addr":"172.21.15.161:6805/3629732803","heartbeat_back_addr":"172.21.15.161:6809/3629732803","heartbeat_front_addr":"172.21.15.161:6807/3629732803","state":["exists","up"]},{"osd":1,"uuid":"4a2f0b78-0ea5-4a4a-8b08-17e8f5f14660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6801","nonce":3382092746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6803","nonce":3382092746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6807","nonce":3382092746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3382092746},{"type":"v1","addr":"172.21.15.175:6805","nonce":3382092746}]},"public_addr":"172.21.15.175:6801/3382092746","cluster_addr":"172.21.15.175:6803/3382092746","heartbeat_back_addr":"172.21.15.175:6807/3382092746","heartbeat_front_addr":"172.21.15.175:6805/3382092746","state":["exists","up"]},{"osd":2,"uuid":"32cfe220-ef88-4ab3-a40f-57786bafcf56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6801","nonce":868656265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6803","nonce":868656265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6807","nonce":868656265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":868656265},{"type":"v1","addr":"172.21.15.182:6805","nonce":868656265}]},"public_addr":"172.21.15.182:6801/868656265","cluster_addr":"172.21.15.182:6803/868656265","heartbeat_back_addr":"172.21.15.182:6807/868656265","heartbeat_front_addr":"172.21.15.182:6805/868656265","state":["exists","up"]},{"osd":3,"uuid":"24ac073f-086c-4626-8080-a901f73a1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6811","nonce":2962874030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6813","nonce":2962874030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6817","nonce":2962874030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2962874030},{"type":"v1","addr":"172.21.15.161:6815","nonce":2962874030}]},"public_addr":"172.21.15.161:6811/2962874030","cluster_addr":"172.21.15.161:6813/2962874030","heartbeat_back_addr":"172.21.15.161:6817/2962874030","heartbeat_front_addr":"172.21.15.161:6815/2962874030","state":["exists","up"]},{"osd":4,"uuid":"f4e7501f-63ff-4f01-ade2-136ab260607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6809","nonce":1496586319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6811","nonce":1496586319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6815","nonce":1496586319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1496586319},{"type":"v1","addr":"172.21.15.175:6813","nonce":1496586319}]},"public_addr":"172.21.15.175:6809/1496586319","cluster_addr":"172.21.15.175:6811/1496586319","heartbeat_back_addr":"172.21.15.175:6815/1496586319","heartbeat_front_addr":"172.21.15.175:6813/1496586319","state":["exists","up"]},{"osd":5,"uuid":"228b77ca-2772-40c5-8556-030d1b432280","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6809","nonce":4034252274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6811","nonce":4034252274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6815","nonce":4034252274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":4034252274},{"type":"v1","addr":"172.21.15.182:6813","nonce":4034252274}]},"public_addr":"172.21.15.182:6809/4034252274","cluster_addr":"172.21.15.182:6811/4034252274","heartbeat_back_addr":"172.21.15.182:6815/4034252274","heartbeat_front_addr":"172.21.15.182:6813/4034252274","state":["exists","up"]},{"osd":6,"uuid":"bcacbde1-0008-440a-bbe7-07275b13f16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6819","nonce":2715861815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6821","nonce":2715861815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6825","nonce":2715861815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2715861815},{"type":"v1","addr":"172.21.15.161:6823","nonce":2715861815}]},"public_addr":"172.21.15.161:6819/2715861815","cluster_addr":"172.21.15.161:6821/2715861815","heartbeat_back_addr":"172.21.15.161:6825/2715861815","heartbeat_front_addr":"172.21.15.161:6823/2715861815","state":["exists","up"]},{"osd":7,"uuid":"d6669cd4-11cc-4457-8775-f11dbeb79ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6817","nonce":4058697071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6819","nonce":4058697071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6823","nonce":4058697071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":4058697071},{"type":"v1","addr":"172.21.15.175:6821","nonce":4058697071}]},"public_addr":"172.21.15.175:6817/4058697071","cluster_addr":"172.21.15.175:6819/4058697071","heartbeat_back_addr":"172.21.15.175:6823/4058697071","heartbeat_front_addr":"172.21.15.175:6821/4058697071","state":["exists","up"]},{"osd":8,"uuid":"56d1548f-b2d9-44bb-914a-9dbfdcd39b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6817","nonce":3827127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6818","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6819","nonce":3827127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6822","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6823","nonce":3827127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6820","nonce":3827127},{"type":"v1","addr":"172.21.15.182:6821","nonce":3827127}]},"public_addr":"172.21.15.182:6817/3827127","cluster_addr":"172.21.15.182:6819/3827127","heartbeat_back_addr":"172.21.15.182:6823/3827127","heartbeat_front_addr":"172.21.15.182:6821/3827127","state":["exists","up"]},{"osd":9,"uuid":"016c1241-9eaf-48e6-b54a-4cfc973aa73f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6827","nonce":565147230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6829","nonce":565147230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6833","nonce":565147230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":565147230},{"type":"v1","addr":"172.21.15.161:6831","nonce":565147230}]},"public_addr":"172.21.15.161:6827/565147230","cluster_addr":"172.21.15.161:6829/565147230","heartbeat_back_addr":"172.21.15.161:6833/565147230","heartbeat_front_addr":"172.21.15.161:6831/565147230","state":["exists","up"]},{"osd":10,"uuid":"2755e32a-49c5-4aaa-ad39-861f12b47889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6825","nonce":4263786893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6827","nonce":4263786893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6831","nonce":4263786893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":4263786893},{"type":"v1","addr":"172.21.15.175:6829","nonce":4263786893}]},"public_addr":"172.21.15.175:6825/4263786893","cluster_addr":"172.21.15.175:6827/4263786893","heartbeat_back_addr":"172.21.15.175:6831/4263786893","heartbeat_front_addr":"172.21.15.175:6829/4263786893","state":["exists","up"]},{"osd":11,"uuid":"4d1d4d3c-b2d7-41e9-ae67-771e27332ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6824","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6825","nonce":2744013300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6826","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6827","nonce":2744013300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6830","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6831","nonce":2744013300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6828","nonce":2744013300},{"type":"v1","addr":"172.21.15.182:6829","nonce":2744013300}]},"public_addr":"172.21.15.182:6825/2744013300","cluster_addr":"172.21.15.182:6827/2744013300","heartbeat_back_addr":"172.21.15.182:6831/2744013300","heartbeat_front_addr":"172.21.15.182:6829/2744013300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:14:46.180148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:08.076764+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:28.515422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:15:54.210640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:15.580673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:16:36.375654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:01.802106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:23.184756+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:17:43.768446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:09.246601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:30.882995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:18:52.174460+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/142996677":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/3308966068":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/389110447":"2023-07-15T11:22:15.015135+0000","172.21.15.161:6801/2004305703":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/871206111":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/2004305703":"2023-07-15T11:22:15.015135+0000","172.21.15.161:6801/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/3292405426":"2023-07-15T11:13:14.005108+0000","172.21.15.161:0/629268496":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/832098793":"2023-07-15T11:13:14.005108+0000","172.21.15.161:6800/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:6801/4131434778":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/182250477":"2023-07-15T11:22:15.015135+0000","172.21.15.161:0/3333551885":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/1691634661":"2023-07-15T11:12:47.212143+0000","172.21.15.161:0/3118869366":"2023-07-15T11:13:14.005108+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]},{"pool":4,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:38.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93700a1300 msgr2=0x7f93700a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93700a1300 0x7f93700a37b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9374005950 tx=0x7f937400b410 comp rx=0 tx=0).stop 2023-07-14T11:54:38.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 msgr2=0x7f938019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f938019bee0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c005740 comp rx=0 tx=0).stop 2023-07-14T11:54:38.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 shutdown_connections 2023-07-14T11:54:38.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f93700a1300 0x7f93700a37b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9380100d50 0x7f938019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938010a120 0x7f9380195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 --2- 172.21.15.161:0/1080754136 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f9380101660 0x7f938019bee0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 >> 172.21.15.161:0/1080754136 conn(0x7f9380075c50 msgr2=0x7f93800fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 shutdown_connections 2023-07-14T11:54:38.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.634+0000 7f9386bff700 1 -- 172.21.15.161:0/1080754136 wait complete. 2023-07-14T11:54:38.636 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-14T11:54:38.838 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 -- 172.21.15.161:0/603842726 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0105af0 msgr2=0x7f13e0105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 --2- 172.21.15.161:0/603842726 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0105af0 0x7f13e0105ec0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009990 tx=0x7f13d0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 -- 172.21.15.161:0/603842726 shutdown_connections 2023-07-14T11:54:38.839 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 --2- 172.21.15.161:0/603842726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e0100490 0x7f13e010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 --2- 172.21.15.161:0/603842726 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0105af0 0x7f13e0105ec0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 --2- 172.21.15.161:0/603842726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e00ffae0 0x7f13e00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 -- 172.21.15.161:0/603842726 >> 172.21.15.161:0/603842726 conn(0x7f13e00fb5e0 msgr2=0x7f13e00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 -- 172.21.15.161:0/603842726 shutdown_connections 2023-07-14T11:54:38.840 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.856+0000 7f13e65ff700 1 -- 172.21.15.161:0/603842726 wait complete. 2023-07-14T11:54:38.842 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.857+0000 7f13e65ff700 1 Processor -- start 2023-07-14T11:54:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.857+0000 7f13e65ff700 1 -- start start 2023-07-14T11:54:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.857+0000 7f13e65ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e00ffae0 0x7f13e0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.857+0000 7f13e65ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0100490 0x7f13e0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.843 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.857+0000 7f13e65ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0105af0 0x7f13e01a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13e65ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0073580 con 0x7f13e0105af0 2023-07-14T11:54:38.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13e65ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e01a4310 con 0x7f13e00ffae0 2023-07-14T11:54:38.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13e65ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e01a4450 con 0x7f13e0100490 2023-07-14T11:54:38.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0100490 0x7f13e0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.844 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13e4b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0105af0 0x7f13e01a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13dffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e00ffae0 0x7f13e0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0100490 0x7f13e0072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.182:3300/0 says I am v2:172.21.15.161:35876/0 (socket says 172.21.15.161:35876) 2023-07-14T11:54:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13d7fff700 1 -- 172.21.15.161:0/3229197202 learned_addr learned my addr 172.21.15.161:0/3229197202 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:38.845 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13dffff700 1 -- 172.21.15.161:0/3229197202 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0100490 msgr2=0x7f13e0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13dffff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0100490 0x7f13e0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13dffff700 1 -- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0105af0 msgr2=0x7f13e01a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13dffff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0105af0 0x7f13e01a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.846 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13dffff700 1 -- 172.21.15.161:0/3229197202 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d00095f0 con 0x7f13e00ffae0 2023-07-14T11:54:38.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13e4b9c700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0105af0 0x7f13e01a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:38.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13d7fff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0100490 0x7f13e0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:38.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.858+0000 7f13dffff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e00ffae0 0x7f13e0072810 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f13d0000c00 tx=0x7f13d0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.859+0000 7f13ddffb700 1 -- 172.21.15.161:0/3229197202 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d001d070 con 0x7f13e00ffae0 2023-07-14T11:54:38.847 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.859+0000 7f13ddffb700 1 -- 172.21.15.161:0/3229197202 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13d0021410 con 0x7f13e00ffae0 2023-07-14T11:54:38.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.859+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e01a4650 con 0x7f13e00ffae0 2023-07-14T11:54:38.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.859+0000 7f13ddffb700 1 -- 172.21.15.161:0/3229197202 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0017650 con 0x7f13e00ffae0 2023-07-14T11:54:38.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.859+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e01a4a70 con 0x7f13e00ffae0 2023-07-14T11:54:38.848 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.860+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e0067290 con 0x7f13e00ffae0 2023-07-14T11:54:38.849 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.862+0000 7f13ddffb700 1 -- 172.21.15.161:0/3229197202 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f13d00217f0 con 0x7f13e00ffae0 2023-07-14T11:54:38.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.863+0000 7f13ddffb700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13cc0a12c0 0x7f13cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.863+0000 7f13ddffb700 1 -- 172.21.15.161:0/3229197202 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f13d00ce940 con 0x7f13e00ffae0 2023-07-14T11:54:38.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.863+0000 7f13d7fff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13cc0a12c0 0x7f13cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.850 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.864+0000 7f13d7fff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13cc0a12c0 0x7f13cc0a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f13e0074000 tx=0x7f13c8009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.851 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:38.865+0000 7f13ddffb700 1 -- 172.21.15.161:0/3229197202 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f13d0097a20 con 0x7f13e00ffae0 2023-07-14T11:54:39.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.112+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f13e01a4d20 con 0x7f13e00ffae0 2023-07-14T11:54:39.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.124+0000 7f13ddffb700 1 -- 172.21.15.161:0/3229197202 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f13d0021e60 con 0x7f13e00ffae0 2023-07-14T11:54:39.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.126+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13cc0a12c0 msgr2=0x7f13cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.126+0000 7f13e65ff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13cc0a12c0 0x7f13cc0a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f13e0074000 tx=0x7f13c8009260 comp rx=0 tx=0).stop 2023-07-14T11:54:39.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e00ffae0 msgr2=0x7f13e0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e00ffae0 0x7f13e0072810 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f13d0000c00 tx=0x7f13d0005740 comp rx=0 tx=0).stop 2023-07-14T11:54:39.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 shutdown_connections 2023-07-14T11:54:39.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f13cc0a12c0 0x7f13cc0a3770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.112 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0105af0 0x7f13e01a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e00ffae0 0x7f13e0072810 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 --2- 172.21.15.161:0/3229197202 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f13e0100490 0x7f13e0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 >> 172.21.15.161:0/3229197202 conn(0x7f13e00fb5e0 msgr2=0x7f13e00fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 shutdown_connections 2023-07-14T11:54:39.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.127+0000 7f13e65ff700 1 -- 172.21.15.161:0/3229197202 wait complete. 2023-07-14T11:54:39.114 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-14T11:54:39.128 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-14T11:54:39.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.348+0000 7ffa661ff700 1 -- 172.21.15.161:0/622543827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 msgr2=0x7ffa60105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.348+0000 7ffa661ff700 1 --2- 172.21.15.161:0/622543827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 0x7ffa60105ec0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ffa500098f0 tx=0x7ffa50009c00 comp rx=0 tx=0).stop 2023-07-14T11:54:39.331 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.348+0000 7ffa661ff700 1 -- 172.21.15.161:0/622543827 shutdown_connections 2023-07-14T11:54:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.348+0000 7ffa661ff700 1 --2- 172.21.15.161:0/622543827 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ffa60100490 0x7ffa6010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.348+0000 7ffa661ff700 1 --2- 172.21.15.161:0/622543827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 0x7ffa60105ec0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.348+0000 7ffa661ff700 1 --2- 172.21.15.161:0/622543827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 0x7ffa600fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.348+0000 7ffa661ff700 1 -- 172.21.15.161:0/622543827 >> 172.21.15.161:0/622543827 conn(0x7ffa600fb5e0 msgr2=0x7ffa600fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.349+0000 7ffa661ff700 1 -- 172.21.15.161:0/622543827 shutdown_connections 2023-07-14T11:54:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.349+0000 7ffa661ff700 1 -- 172.21.15.161:0/622543827 wait complete. 2023-07-14T11:54:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 Processor -- start 2023-07-14T11:54:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 -- start start 2023-07-14T11:54:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 0x7ffa60197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ffa60100490 0x7ffa60198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 0x7ffa6019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6019ce30 con 0x7ffa600ffae0 2023-07-14T11:54:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6019cfa0 con 0x7ffa60105af0 2023-07-14T11:54:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa661ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6019d160 con 0x7ffa60100490 2023-07-14T11:54:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa5f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 0x7ffa60197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa5f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 0x7ffa60197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46186/0 (socket says 172.21.15.161:46186) 2023-07-14T11:54:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa5f7fe700 1 -- 172.21.15.161:0/339291331 learned_addr learned my addr 172.21.15.161:0/339291331 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.350+0000 7ffa5effd700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ffa60100490 0x7ffa60198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5ffff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 0x7ffa6019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5f7fe700 1 -- 172.21.15.161:0/339291331 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ffa60100490 msgr2=0x7ffa60198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5f7fe700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ffa60100490 0x7ffa60198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5f7fe700 1 -- 172.21.15.161:0/339291331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 msgr2=0x7ffa6019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5f7fe700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 0x7ffa6019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5f7fe700 1 -- 172.21.15.161:0/339291331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58009d00 con 0x7ffa600ffae0 2023-07-14T11:54:39.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5ffff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 0x7ffa6019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:39.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5f7fe700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 0x7ffa60197e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009810 tx=0x7ffa5000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5cff9700 1 -- 172.21.15.161:0/339291331 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5001c070 con 0x7ffa600ffae0 2023-07-14T11:54:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa50009520 con 0x7ffa600ffae0 2023-07-14T11:54:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5cff9700 1 -- 172.21.15.161:0/339291331 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa5000bbd0 con 0x7ffa600ffae0 2023-07-14T11:54:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa5cff9700 1 -- 172.21.15.161:0/339291331 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500049c0 con 0x7ffa600ffae0 2023-07-14T11:54:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.351+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa601a4aa0 con 0x7ffa600ffae0 2023-07-14T11:54:39.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.352+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa60067290 con 0x7ffa600ffae0 2023-07-14T11:54:39.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.354+0000 7ffa5cff9700 1 -- 172.21.15.161:0/339291331 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ffa50020070 con 0x7ffa600ffae0 2023-07-14T11:54:39.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.357+0000 7ffa5cff9700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ffa480a12c0 0x7ffa480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.357+0000 7ffa5cff9700 1 -- 172.21.15.161:0/339291331 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9594+0+0 (secure 0 0 0) 0x7ffa500ce2b0 con 0x7ffa600ffae0 2023-07-14T11:54:39.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.357+0000 7ffa5effd700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ffa480a12c0 0x7ffa480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.358+0000 7ffa5effd700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ffa480a12c0 0x7ffa480a3770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa60199460 tx=0x7ffa54009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.359+0000 7ffa5cff9700 1 -- 172.21.15.161:0/339291331 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffa500972c0 con 0x7ffa600ffae0 2023-07-14T11:54:39.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2890075009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:54:39.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1080754136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:54:39.400 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: ignoring --setuser ceph since I am not root 2023-07-14T11:54:39.400 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: ignoring --setgroup ceph since I am not root 2023-07-14T11:54:39.401 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-mds[154599]: -- 172.21.15.175:0/1746629271 <== mon.0 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56514656d1e0 con 0x565146542c00 2023-07-14T11:54:39.401 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: starting mds.b at 2023-07-14T11:54:39.401 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-h[154234]: ignoring --setuser ceph since I am not root 2023-07-14T11:54:39.402 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-h[154234]: ignoring --setgroup ceph since I am not root 2023-07-14T11:54:39.402 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-mds[154258]: -- 172.21.15.175:0/1148923970 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dcdb6671e0 con 0x55dcdb63cc00 2023-07-14T11:54:39.402 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-h[154234]: starting mds.h at 2023-07-14T11:54:39.402 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-e[153890]: ignoring --setuser ceph since I am not root 2023-07-14T11:54:39.403 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-e[153890]: ignoring --setgroup ceph since I am not root 2023-07-14T11:54:39.403 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:54:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-e[153890]: starting mds.e at 2023-07-14T11:54:39.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2890075009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:54:39.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1080754136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:54:39.445 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-f[155147]: ignoring --setuser ceph since I am not root 2023-07-14T11:54:39.445 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-f[155147]: ignoring --setgroup ceph since I am not root 2023-07-14T11:54:39.445 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-mds[155171]: -- 172.21.15.182:0/1702738585 <== mon.1 v2:172.21.15.182:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1cf91d1e0 con 0x55e1cf8f3000 2023-07-14T11:54:39.446 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-f[155147]: starting mds.f at 2023-07-14T11:54:39.446 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-c[155485]: ignoring --setuser ceph since I am not root 2023-07-14T11:54:39.446 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-c[155485]: ignoring --setgroup ceph since I am not root 2023-07-14T11:54:39.447 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-c[155485]: starting mds.c at 2023-07-14T11:54:39.447 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-i[155827]: ignoring --setuser ceph since I am not root 2023-07-14T11:54:39.447 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-i[155827]: ignoring --setgroup ceph since I am not root 2023-07-14T11:54:39.448 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-mds[155851]: -- 172.21.15.182:0/1237178641 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563bac92f1e0 con 0x563bac904800 2023-07-14T11:54:39.448 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:54:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-i[155827]: starting mds.i at 2023-07-14T11:54:39.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:39 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2890075009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:54:39.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:39 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1080754136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:54:39.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-a[95502]: 2023-07-14T11:54:39.114+0000 7f74b04cd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T11:54:39.586 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:39.604+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffa601a4c30 con 0x7ffa600ffae0 2023-07-14T11:54:40.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.120+0000 7ffa5cff9700 1 -- 172.21.15.161:0/339291331 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffa5009a8e0 con 0x7ffa600ffae0 2023-07-14T11:54:40.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ffa480a12c0 msgr2=0x7ffa480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ffa480a12c0 0x7ffa480a3770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa60199460 tx=0x7ffa54009260 comp rx=0 tx=0).stop 2023-07-14T11:54:40.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 msgr2=0x7ffa60197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 0x7ffa60197e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009810 tx=0x7ffa5000fa00 comp rx=0 tx=0).stop 2023-07-14T11:54:40.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 shutdown_connections 2023-07-14T11:54:40.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7ffa480a12c0 0x7ffa480a3770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa600ffae0 0x7ffa60197e50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa60105af0 0x7ffa6019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 --2- 172.21.15.161:0/339291331 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7ffa60100490 0x7ffa60198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 >> 172.21.15.161:0/339291331 conn(0x7ffa600fb5e0 msgr2=0x7ffa6010adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:40.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 shutdown_connections 2023-07-14T11:54:40.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.123+0000 7ffa661ff700 1 -- 172.21.15.161:0/339291331 wait complete. 2023-07-14T11:54:40.126 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-14T11:54:40.332 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 -- 172.21.15.161:0/125530505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd744100490 msgr2=0x7fd74410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/125530505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd744100490 0x7fd74410c920 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00b0a0 tx=0x7fd73c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 -- 172.21.15.161:0/125530505 shutdown_connections 2023-07-14T11:54:40.333 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/125530505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd744100490 0x7fd74410c920 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/125530505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd744105af0 0x7fd744105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/125530505 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd7440ffae0 0x7fd7440fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 -- 172.21.15.161:0/125530505 >> 172.21.15.161:0/125530505 conn(0x7fd7440fb5e0 msgr2=0x7fd7440fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 -- 172.21.15.161:0/125530505 shutdown_connections 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.350+0000 7fd74b1ff700 1 -- 172.21.15.161:0/125530505 wait complete. 2023-07-14T11:54:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.351+0000 7fd74b1ff700 1 Processor -- start 2023-07-14T11:54:40.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.351+0000 7fd74b1ff700 1 -- start start 2023-07-14T11:54:40.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd74b1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7440ffae0 0x7fd74419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd74b1ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd744100490 0x7fd74419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd74b1ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd744105af0 0x7fd744195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd74b1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441961c0 con 0x7fd7440ffae0 2023-07-14T11:54:40.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd74b1ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd744196330 con 0x7fd744105af0 2023-07-14T11:54:40.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd74b1ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441964f0 con 0x7fd744100490 2023-07-14T11:54:40.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7440ffae0 0x7fd74419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7440ffae0 0x7fd74419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46198/0 (socket says 172.21.15.161:46198) 2023-07-14T11:54:40.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 -- 172.21.15.161:0/2501001520 learned_addr learned my addr 172.21.15.161:0/2501001520 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:40.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd743fff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd744100490 0x7fd74419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd74979c700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd744105af0 0x7fd744195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 -- 172.21.15.161:0/2501001520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd744100490 msgr2=0x7fd74419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd744100490 0x7fd74419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 -- 172.21.15.161:0/2501001520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd744105af0 msgr2=0x7fd744195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd744105af0 0x7fd744195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.352+0000 7fd748f9b700 1 -- 172.21.15.161:0/2501001520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c009d00 con 0x7fd7440ffae0 2023-07-14T11:54:40.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.353+0000 7fd748f9b700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7440ffae0 0x7fd74419bae0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd73800e9a0 tx=0x7fd73800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.353+0000 7fd741ffb700 1 -- 172.21.15.161:0/2501001520 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738009790 con 0x7fd7440ffae0 2023-07-14T11:54:40.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.353+0000 7fd741ffb700 1 -- 172.21.15.161:0/2501001520 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd738010430 con 0x7fd7440ffae0 2023-07-14T11:54:40.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.353+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7441967d0 con 0x7fd7440ffae0 2023-07-14T11:54:40.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.353+0000 7fd741ffb700 1 -- 172.21.15.161:0/2501001520 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738018680 con 0x7fd7440ffae0 2023-07-14T11:54:40.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.353+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7441a2ad0 con 0x7fd7440ffae0 2023-07-14T11:54:40.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.354+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd744067290 con 0x7fd7440ffae0 2023-07-14T11:54:40.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.358+0000 7fd741ffb700 1 -- 172.21.15.161:0/2501001520 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd7380105a0 con 0x7fd7440ffae0 2023-07-14T11:54:40.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.359+0000 7fd741ffb700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7340a12c0 0x7fd7340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.359+0000 7fd741ffb700 1 -- 172.21.15.161:0/2501001520 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9594+0+0 (secure 0 0 0) 0x7fd738014070 con 0x7fd7440ffae0 2023-07-14T11:54:40.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.359+0000 7fd743fff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7340a12c0 0x7fd7340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.343 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.360+0000 7fd743fff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7340a12c0 0x7fd7340a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd7441050d0 tx=0x7fd730009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.344 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.361+0000 7fd741ffb700 1 -- 172.21.15.161:0/2501001520 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd738096430 con 0x7fd7440ffae0 2023-07-14T11:54:40.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/3229197202' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T11:54:40.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T11:54:40.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T11:54:40.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T11:54:40.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: osdmap e146: 12 total, 12 up, 12 in 2023-07-14T11:54:40.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T11:54:40.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T11:54:40.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/339291331' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:54:40.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:40 smithi175 ceph-mon[129081]: pgmap v1028: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:54:40.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/3229197202' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T11:54:40.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T11:54:40.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T11:54:40.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T11:54:40.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: osdmap e146: 12 total, 12 up, 12 in 2023-07-14T11:54:40.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T11:54:40.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T11:54:40.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/339291331' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:54:40.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:40 smithi182 ceph-mon[129366]: pgmap v1028: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:54:40.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/3229197202' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T11:54:40.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T11:54:40.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T11:54:40.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T11:54:40.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: osdmap e146: 12 total, 12 up, 12 in 2023-07-14T11:54:40.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T11:54:40.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T11:54:40.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/339291331' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:54:40.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:40 smithi161 ceph-mon[95526]: pgmap v1028: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-14T11:54:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:40.561+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7441970e0 con 0x7fd7440ffae0 2023-07-14T11:54:41.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.129+0000 7fd741ffb700 1 -- 172.21.15.161:0/2501001520 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd738099a50 con 0x7fd7440ffae0 2023-07-14T11:54:41.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7340a12c0 msgr2=0x7fd7340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7340a12c0 0x7fd7340a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd7441050d0 tx=0x7fd730009260 comp rx=0 tx=0).stop 2023-07-14T11:54:41.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7440ffae0 msgr2=0x7fd74419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7440ffae0 0x7fd74419bae0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd73800e9a0 tx=0x7fd73800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.116 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 shutdown_connections 2023-07-14T11:54:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7fd7340a12c0 0x7fd7340a3770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7440ffae0 0x7fd74419bae0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd744105af0 0x7fd744195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 --2- 172.21.15.161:0/2501001520 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7fd744100490 0x7fd74419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.117 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.133+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 >> 172.21.15.161:0/2501001520 conn(0x7fd7440fb5e0 msgr2=0x7fd7440fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:41.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.134+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 shutdown_connections 2023-07-14T11:54:41.118 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.134+0000 7fd74b1ff700 1 -- 172.21.15.161:0/2501001520 wait complete. 2023-07-14T11:54:41.118 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2023-07-14T11:54:41.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-14T11:54:41.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.386+0000 7f75d3bff700 1 -- 172.21.15.161:0/3134718344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc101470 msgr2=0x7f75cc109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.386+0000 7f75d3bff700 1 --2- 172.21.15.161:0/3134718344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc101470 0x7f75cc109b40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009990 tx=0x7f75bc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.386+0000 7f75d3bff700 1 -- 172.21.15.161:0/3134718344 shutdown_connections 2023-07-14T11:54:41.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.386+0000 7f75d3bff700 1 --2- 172.21.15.161:0/3134718344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc10a080 0x7f75cc10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.386+0000 7f75d3bff700 1 --2- 172.21.15.161:0/3134718344 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75cc100b60 0x7f75cc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.386+0000 7f75d3bff700 1 --2- 172.21.15.161:0/3134718344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc101470 0x7f75cc109b40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.386+0000 7f75d3bff700 1 -- 172.21.15.161:0/3134718344 >> 172.21.15.161:0/3134718344 conn(0x7f75cc0fa4b0 msgr2=0x7f75cc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:41.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.387+0000 7f75d3bff700 1 -- 172.21.15.161:0/3134718344 shutdown_connections 2023-07-14T11:54:41.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.387+0000 7f75d3bff700 1 -- 172.21.15.161:0/3134718344 wait complete. 2023-07-14T11:54:41.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 Processor -- start 2023-07-14T11:54:41.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 -- start start 2023-07-14T11:54:41.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc100b60 0x7f75cc0fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc101470 0x7f75cc0feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75cc10a080 0x7f75cc1a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0ff700 con 0x7f75cc101470 2023-07-14T11:54:41.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1a4320 con 0x7f75cc100b60 2023-07-14T11:54:41.375 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d3bff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1a4460 con 0x7f75cc10a080 2023-07-14T11:54:41.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc101470 0x7f75cc0feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d219c700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75cc10a080 0x7f75cc1a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc101470 0x7f75cc0feed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46214/0 (socket says 172.21.15.161:46214) 2023-07-14T11:54:41.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d119a700 1 -- 172.21.15.161:0/42012835 learned_addr learned my addr 172.21.15.161:0/42012835 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:41.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.388+0000 7f75d199b700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc100b60 0x7f75cc0fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d219c700 1 -- 172.21.15.161:0/42012835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc100b60 msgr2=0x7f75cc0fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d219c700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc100b60 0x7f75cc0fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.382 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d219c700 1 -- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc101470 msgr2=0x7f75cc0feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d219c700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc101470 0x7f75cc0feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d219c700 1 -- 172.21.15.161:0/42012835 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0095f0 con 0x7f75cc10a080 2023-07-14T11:54:41.383 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d219c700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75cc10a080 0x7f75cc1a3de0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f75c4007f10 tx=0x7f75c400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75c2ffd700 1 -- 172.21.15.161:0/42012835 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4009ea0 con 0x7f75cc10a080 2023-07-14T11:54:41.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc1a46c0 con 0x7f75cc10a080 2023-07-14T11:54:41.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75c2ffd700 1 -- 172.21.15.161:0/42012835 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75c400f040 con 0x7f75cc10a080 2023-07-14T11:54:41.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75c2ffd700 1 -- 172.21.15.161:0/42012835 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004400 con 0x7f75cc10a080 2023-07-14T11:54:41.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.389+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1a4ba0 con 0x7f75cc10a080 2023-07-14T11:54:41.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.390+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc067290 con 0x7f75cc10a080 2023-07-14T11:54:41.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.394+0000 7f75c2ffd700 1 -- 172.21.15.161:0/42012835 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f75c4004ad0 con 0x7f75cc10a080 2023-07-14T11:54:41.386 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.396+0000 7f75c2ffd700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f75b80a12c0 0x7f75b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.396+0000 7f75c2ffd700 1 -- 172.21.15.161:0/42012835 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 9098+0+0 (secure 0 0 0) 0x7f75c40cc9c0 con 0x7f75cc10a080 2023-07-14T11:54:41.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.396+0000 7f75d199b700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f75b80a12c0 0x7f75b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.387 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.397+0000 7f75d199b700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f75b80a12c0 0x7f75b80a3770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009c60 tx=0x7f75c8009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.388 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.397+0000 7f75c2ffd700 1 -- 172.21.15.161:0/42012835 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f75c4095b90 con 0x7f75cc10a080 2023-07-14T11:54:41.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T11:54:41.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T11:54:41.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: Cluster is now healthy 2023-07-14T11:54:41.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6836/1859772395,v1:172.21.15.182:6839/1859772395] up:boot 2023-07-14T11:54:41.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6834/975063843,v1:172.21.15.182:6837/975063843] up:boot 2023-07-14T11:54:41.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.182:6835/3435547235,v1:172.21.15.182:6838/3435547235] up:boot 2023-07-14T11:54:41.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6835/4189680865,v1:172.21.15.175:6839/4189680865] up:boot 2023-07-14T11:54:41.402 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6836/1846379415,v1:172.21.15.175:6838/1846379415] up:boot 2023-07-14T11:54:41.402 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: mds.? [v2:172.21.15.175:6834/3226363317,v1:172.21.15.175:6837/3226363317] up:boot 2023-07-14T11:54:41.402 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/339291331' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:54:41.403 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: fsmap 12 up:standby 2023-07-14T11:54:41.403 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:54:41.403 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:54:41.404 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:54:41.404 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:54:41.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:54:41.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:54:41.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:41 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2501001520' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:41.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T11:54:41.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T11:54:41.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: Cluster is now healthy 2023-07-14T11:54:41.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6836/1859772395,v1:172.21.15.182:6839/1859772395] up:boot 2023-07-14T11:54:41.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6834/975063843,v1:172.21.15.182:6837/975063843] up:boot 2023-07-14T11:54:41.446 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.182:6835/3435547235,v1:172.21.15.182:6838/3435547235] up:boot 2023-07-14T11:54:41.447 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6835/4189680865,v1:172.21.15.175:6839/4189680865] up:boot 2023-07-14T11:54:41.447 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6836/1846379415,v1:172.21.15.175:6838/1846379415] up:boot 2023-07-14T11:54:41.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: mds.? [v2:172.21.15.175:6834/3226363317,v1:172.21.15.175:6837/3226363317] up:boot 2023-07-14T11:54:41.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/339291331' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:54:41.448 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: fsmap 12 up:standby 2023-07-14T11:54:41.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:54:41.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:54:41.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:54:41.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:54:41.449 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:54:41.450 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:54:41.451 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:41 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2501001520' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:41.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T11:54:41.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T11:54:41.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: Cluster is now healthy 2023-07-14T11:54:41.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6836/1859772395,v1:172.21.15.182:6839/1859772395] up:boot 2023-07-14T11:54:41.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6834/975063843,v1:172.21.15.182:6837/975063843] up:boot 2023-07-14T11:54:41.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.182:6835/3435547235,v1:172.21.15.182:6838/3435547235] up:boot 2023-07-14T11:54:41.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6835/4189680865,v1:172.21.15.175:6839/4189680865] up:boot 2023-07-14T11:54:41.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6836/1846379415,v1:172.21.15.175:6838/1846379415] up:boot 2023-07-14T11:54:41.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: mds.? [v2:172.21.15.175:6834/3226363317,v1:172.21.15.175:6837/3226363317] up:boot 2023-07-14T11:54:41.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/339291331' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:54:41.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: fsmap 12 up:standby 2023-07-14T11:54:41.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:54:41.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:54:41.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:54:41.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:54:41.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:54:41.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='mgr.15543 172.21.15.161:0/1411508086' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:54:41.479 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:41 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2501001520' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:41.587 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:41.604+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75cc0ffec0 con 0x7f75cc10a080 2023-07-14T11:54:42.123 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.140+0000 7f75c2ffd700 1 -- 172.21.15.161:0/42012835 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7f75c40190d0 con 0x7f75cc10a080 2023-07-14T11:54:42.126 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.144+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f75b80a12c0 msgr2=0x7f75b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.144+0000 7f75d3bff700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f75b80a12c0 0x7f75b80a3770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009c60 tx=0x7f75c8009260 comp rx=0 tx=0).stop 2023-07-14T11:54:42.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.144+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75cc10a080 msgr2=0x7f75cc1a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.127 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.144+0000 7f75d3bff700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75cc10a080 0x7f75cc1a3de0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f75c4007f10 tx=0x7f75c400d3b0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 shutdown_connections 2023-07-14T11:54:42.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f75b80a12c0 0x7f75b80a3770 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.128 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f75cc10a080 0x7f75cc1a3de0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75cc100b60 0x7f75cc0fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 --2- 172.21.15.161:0/42012835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc101470 0x7f75cc0feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 >> 172.21.15.161:0/42012835 conn(0x7f75cc0fa4b0 msgr2=0x7f75cc10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 shutdown_connections 2023-07-14T11:54:42.129 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.145+0000 7f75d3bff700 1 -- 172.21.15.161:0/42012835 wait complete. 2023-07-14T11:54:42.133 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2023-07-14T11:54:42.159 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-14T11:54:42.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:42 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/2501001520' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T11:54:42.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:42 smithi175 ceph-mon[129081]: osdmap e147: 12 total, 12 up, 12 in 2023-07-14T11:54:42.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:42 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/42012835' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:42.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:42 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:42.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:42 smithi175 ceph-mon[129081]: pgmap v1030: 97 pgs: 97 active+clean; 220 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:54:42.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:42 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/2501001520' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T11:54:42.444 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:42 smithi182 ceph-mon[129366]: osdmap e147: 12 total, 12 up, 12 in 2023-07-14T11:54:42.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:42 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/42012835' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:42.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:42 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:42.445 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:42 smithi182 ceph-mon[129366]: pgmap v1030: 97 pgs: 97 active+clean; 220 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:54:42.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.475+0000 7f16359ff700 1 -- 172.21.15.161:0/2194556418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 msgr2=0x7f1630105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.475+0000 7f16359ff700 1 --2- 172.21.15.161:0/2194556418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 0x7f1630105eb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1620009990 tx=0x7f1620009ca0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.459 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.476+0000 7f16359ff700 1 -- 172.21.15.161:0/2194556418 shutdown_connections 2023-07-14T11:54:42.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.476+0000 7f16359ff700 1 --2- 172.21.15.161:0/2194556418 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f1630100480 0x7f163010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.476+0000 7f16359ff700 1 --2- 172.21.15.161:0/2194556418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 0x7f1630105eb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.476+0000 7f16359ff700 1 --2- 172.21.15.161:0/2194556418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16300ffad0 0x7f16300fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.476+0000 7f16359ff700 1 -- 172.21.15.161:0/2194556418 >> 172.21.15.161:0/2194556418 conn(0x7f16300fb5f0 msgr2=0x7f16300fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:42.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.476+0000 7f16359ff700 1 -- 172.21.15.161:0/2194556418 shutdown_connections 2023-07-14T11:54:42.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.476+0000 7f16359ff700 1 -- 172.21.15.161:0/2194556418 wait complete. 2023-07-14T11:54:42.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.477+0000 7f16359ff700 1 Processor -- start 2023-07-14T11:54:42.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.477+0000 7f16359ff700 1 -- start start 2023-07-14T11:54:42.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f16359ff700 1 --2- >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16300ffad0 0x7f163019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f16359ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1630100480 0x7f163019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f16359ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 0x7f1630195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f16359ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16301960a0 con 0x7f1630100480 2023-07-14T11:54:42.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f16359ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630196210 con 0x7f1630105ae0 2023-07-14T11:54:42.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f16359ff700 1 -- --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16301963d0 con 0x7f16300ffad0 2023-07-14T11:54:42.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f162e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1630100480 0x7f163019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f162f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 0x7f1630195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f162e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1630100480 0x7f163019bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46224/0 (socket says 172.21.15.161:46224) 2023-07-14T11:54:42.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f162e7fc700 1 -- 172.21.15.161:0/1545315296 learned_addr learned my addr 172.21.15.161:0/1545315296 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T11:54:42.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.478+0000 7f162effd700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16300ffad0 0x7f163019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f162effd700 1 -- 172.21.15.161:0/1545315296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 msgr2=0x7f1630195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f162effd700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 0x7f1630195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f162effd700 1 -- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1630100480 msgr2=0x7f163019bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f162effd700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1630100480 0x7f163019bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f162effd700 1 -- 172.21.15.161:0/1545315296 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200095f0 con 0x7f16300ffad0 2023-07-14T11:54:42.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f162effd700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16300ffad0 0x7f163019b9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1620000c00 tx=0x7f1620005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f16349fd700 1 -- 172.21.15.161:0/1545315296 <== mon.1 v2:172.21.15.182:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162001d070 con 0x7f16300ffad0 2023-07-14T11:54:42.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f16349fd700 1 -- 172.21.15.161:0/1545315296 <== mon.1 v2:172.21.15.182:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1620021410 con 0x7f16300ffad0 2023-07-14T11:54:42.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.479+0000 7f16349fd700 1 -- 172.21.15.161:0/1545315296 <== mon.1 v2:172.21.15.182:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620017650 con 0x7f16300ffad0 2023-07-14T11:54:42.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.480+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1630196650 con 0x7f16300ffad0 2023-07-14T11:54:42.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.480+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16301a29b0 con 0x7f16300ffad0 2023-07-14T11:54:42.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.481+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1630067290 con 0x7f16300ffad0 2023-07-14T11:54:42.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.482+0000 7f16349fd700 1 -- 172.21.15.161:0/1545315296 <== mon.1 v2:172.21.15.182:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f162000fad0 con 0x7f16300ffad0 2023-07-14T11:54:42.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.484+0000 7f16349fd700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16180a1340 0x7f16180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.484+0000 7f16349fd700 1 -- 172.21.15.161:0/1545315296 <== mon.1 v2:172.21.15.182:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 8510+0+0 (secure 0 0 0) 0x7f16200cdee0 con 0x7f16300ffad0 2023-07-14T11:54:42.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.484+0000 7f162e7fc700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16180a1340 0x7f16180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.485+0000 7f162e7fc700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16180a1340 0x7f16180a37f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16300fdee0 tx=0x7f1624009260 comp rx=0 tx=0).ready entity=mgr.15543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.471 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.487+0000 7f16349fd700 1 -- 172.21.15.161:0/1545315296 <== mon.1 v2:172.21.15.182:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1620097330 con 0x7f16300ffad0 2023-07-14T11:54:42.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:42 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/2501001520' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T11:54:42.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:42 smithi161 ceph-mon[95526]: osdmap e147: 12 total, 12 up, 12 in 2023-07-14T11:54:42.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:42 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/42012835' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:42.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:42 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:42.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:42 smithi161 ceph-mon[95526]: pgmap v1030: 97 pgs: 97 active+clean; 220 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:54:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:42.687+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 --> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1630196f40 con 0x7f16300ffad0 2023-07-14T11:54:43.132 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.149+0000 7f16349fd700 1 -- 172.21.15.161:0/1545315296 <== mon.1 v2:172.21.15.182:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v149) v1 ==== 178+0+0 (secure 0 0 0) 0x7f162009a950 con 0x7f16300ffad0 2023-07-14T11:54:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16180a1340 msgr2=0x7f16180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16180a1340 0x7f16180a37f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16300fdee0 tx=0x7f1624009260 comp rx=0 tx=0).stop 2023-07-14T11:54:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16300ffad0 msgr2=0x7f163019b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16300ffad0 0x7f163019b9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1620000c00 tx=0x7f1620005740 comp rx=0 tx=0).stop 2023-07-14T11:54:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 shutdown_connections 2023-07-14T11:54:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:6800/4046273551,v1:172.21.15.161:6801/4046273551] conn(0x7f16180a1340 0x7f16180a37f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1630105ae0 0x7f1630195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] conn(0x7f16300ffad0 0x7f163019b9c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 --2- 172.21.15.161:0/1545315296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1630100480 0x7f163019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.154+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 >> 172.21.15.161:0/1545315296 conn(0x7f16300fb5f0 msgr2=0x7f16300fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.155+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 shutdown_connections 2023-07-14T11:54:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T11:54:43.155+0000 7f16359ff700 1 -- 172.21.15.161:0/1545315296 wait complete. 2023-07-14T11:54:43.141 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data_ec' removed 2023-07-14T11:54:43.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:54:43.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:54:43.340 DEBUG:teuthology.orchestra.run.smithi182:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:54:43.368 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:43 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T11:54:43.369 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:43 smithi182 ceph-mon[129366]: osdmap e148: 12 total, 12 up, 12 in 2023-07-14T11:54:43.369 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:43 smithi182 ceph-mon[129366]: from='client.? 172.21.15.161:0/1545315296' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:43.369 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:43 smithi182 ceph-mon[129366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:43.375 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-14T11:54:43.376 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T11:54:43.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:43 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T11:54:43.399 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:43 smithi175 ceph-mon[129081]: osdmap e148: 12 total, 12 up, 12 in 2023-07-14T11:54:43.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:43 smithi175 ceph-mon[129081]: from='client.? 172.21.15.161:0/1545315296' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:43.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:43 smithi175 ceph-mon[129081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:43.434 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T11:54:43.460 DEBUG:teuthology.orchestra.run.smithi182:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T11:54:43.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:43 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T11:54:43.475 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:43 smithi161 ceph-mon[95526]: osdmap e148: 12 total, 12 up, 12 in 2023-07-14T11:54:43.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:43 smithi161 ceph-mon[95526]: from='client.? 172.21.15.161:0/1545315296' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:43.476 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:43 smithi161 ceph-mon[95526]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T11:54:43.477 INFO:tasks.cephadm:Stopping all daemons... 2023-07-14T11:54:43.478 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-14T11:54:43.478 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a 2023-07-14T11:54:43.975 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:43 smithi161 systemd[1]: Stopping Ceph mon.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:54:44.477 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-a[95502]: 2023-07-14T11:54:44.029+0000 7f74b5cd8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:54:44.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-a[95502]: 2023-07-14T11:54:44.029+0000 7f74b5cd8700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-14T11:54:44.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Jul 14 11:54:44 smithi161 bash[168871]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-a 2023-07-14T11:54:44.956 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.a.service' 2023-07-14T11:54:45.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:54:45.055 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-14T11:54:45.055 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-14T11:54:45.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.b 2023-07-14T11:54:45.398 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:45 smithi175 systemd[1]: Stopping Ceph mon.b for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:54:45.824 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-b[129057]: 2023-07-14T11:54:45.553+0000 7f3f8cfd0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:54:45.825 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-b[129057]: 2023-07-14T11:54:45.553+0000 7f3f8cfd0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-14T11:54:45.825 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jul 14 11:54:45 smithi175 bash[166079]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-b 2023-07-14T11:54:46.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.b.service' 2023-07-14T11:54:46.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:54:46.189 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-14T11:54:46.190 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-14T11:54:46.190 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.c 2023-07-14T11:54:46.657 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:46 smithi182 systemd[1]: Stopping Ceph mon.c for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:54:46.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-c[129340]: 2023-07-14T11:54:46.674+0000 7f099aab7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:54:46.944 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-c[129340]: 2023-07-14T11:54:46.674+0000 7f099aab7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-14T11:54:47.405 INFO:journalctl@ceph.mon.c.smithi182.stdout:Jul 14 11:54:47 smithi182 bash[168575]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mon-c 2023-07-14T11:54:47.432 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mon.c.service' 2023-07-14T11:54:47.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:54:47.481 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-14T11:54:47.481 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-14T11:54:47.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.x 2023-07-14T11:54:47.975 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:54:47 smithi161 systemd[1]: Stopping Ceph mgr.x for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:54:48.339 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:54:48 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:54:48.103+0000 7f337f0e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:54:48.340 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:54:48 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x[95769]: 2023-07-14T11:54:48.103+0000 7f337f0e2700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-14T11:54:48.725 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jul 14 11:54:48 smithi161 bash[169018]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-x 2023-07-14T11:54:48.800 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.x.service' 2023-07-14T11:54:48.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:54:48.890 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-14T11:54:48.891 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-14T11:54:48.891 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.y 2023-07-14T11:54:49.312 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:54:48 smithi175 systemd[1]: Stopping Ceph mgr.y for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:54:49.603 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:54:49 smithi175 bash[166226]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-y 2023-07-14T11:54:49.604 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jul 14 11:54:49 smithi175 systemd[1]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-14T11:54:49.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.y.service' 2023-07-14T11:54:49.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:54:49.826 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-14T11:54:49.826 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-14T11:54:49.826 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.z 2023-07-14T11:54:50.194 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:54:49 smithi182 systemd[1]: Stopping Ceph mgr.z for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:54:50.533 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:54:50 smithi182 bash[168727]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mgr-z 2023-07-14T11:54:50.713 INFO:journalctl@ceph.mgr.z.smithi182.stdout:Jul 14 11:54:50 smithi182 systemd[1]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-14T11:54:50.716 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mgr.z.service' 2023-07-14T11:54:50.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:54:50.754 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-14T11:54:50.754 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-14T11:54:50.754 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.0 2023-07-14T11:54:51.225 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:54:50 smithi161 systemd[1]: Stopping Ceph osd.0 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:54:52.226 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:54:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-0[108468]: 2023-07-14T11:54:51.871+0000 7f33fbce8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:54:52.226 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:54:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-0[108468]: 2023-07-14T11:54:51.871+0000 7f33fbce8700 -1 osd.0 149 *** Got signal Terminated *** 2023-07-14T11:54:52.227 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:54:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-0[108468]: 2023-07-14T11:54:51.871+0000 7f33fbce8700 -1 osd.0 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:54:58.226 INFO:journalctl@ceph.osd.0.smithi161.stdout:Jul 14 11:54:57 smithi161 bash[169170]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-0 2023-07-14T11:54:59.717 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.0.service' 2023-07-14T11:54:59.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:54:59.809 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-14T11:54:59.810 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-14T11:54:59.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.1 2023-07-14T11:55:00.149 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:54:59 smithi175 systemd[1]: Stopping Ceph osd.1 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:00.649 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:55:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-1[134195]: 2023-07-14T11:55:00.184+0000 7f510a696700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:00.649 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:55:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-1[134195]: 2023-07-14T11:55:00.184+0000 7f510a696700 -1 osd.1 149 *** Got signal Terminated *** 2023-07-14T11:55:00.650 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:55:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-1[134195]: 2023-07-14T11:55:00.184+0000 7f510a696700 -1 osd.1 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:08.648 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jul 14 11:55:08 smithi175 bash[166373]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-1 2023-07-14T11:55:09.718 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.1.service' 2023-07-14T11:55:09.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:55:09.758 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-14T11:55:09.759 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-14T11:55:09.759 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.2 2023-07-14T11:55:10.120 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:55:09 smithi182 systemd[1]: Stopping Ceph osd.2 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:10.443 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:55:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-2[135157]: 2023-07-14T11:55:10.138+0000 7ff083d7b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:10.444 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:55:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-2[135157]: 2023-07-14T11:55:10.138+0000 7ff083d7b700 -1 osd.2 149 *** Got signal Terminated *** 2023-07-14T11:55:10.444 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:55:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-2[135157]: 2023-07-14T11:55:10.138+0000 7ff083d7b700 -1 osd.2 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:15.898 INFO:journalctl@ceph.osd.2.smithi182.stdout:Jul 14 11:55:15 smithi182 bash[168883]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-2 2023-07-14T11:55:17.563 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.2.service' 2023-07-14T11:55:17.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:55:17.607 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-14T11:55:17.608 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-14T11:55:17.608 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.3 2023-07-14T11:55:17.975 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:17 smithi161 systemd[1]: Stopping Ceph osd.3 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:18.475 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:18 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:55:18.141+0000 7fda80010700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:18.475 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:18 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:55:18.141+0000 7fda80010700 -1 osd.3 149 *** Got signal Terminated *** 2023-07-14T11:55:18.476 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:18 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:55:18.141+0000 7fda80010700 -1 osd.3 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:19.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:18 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:18.699+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:19.475 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:19 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:55:19.205+0000 7fda7a602700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.358626+0000 front 2023-07-14T11:54:52.358665+0000 (oldest deadline 2023-07-14T11:55:18.258786+0000) 2023-07-14T11:55:19.944 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:19.595+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:20.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:19 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:19.730+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:20.222 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:19 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:19.949+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:20.475 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:20 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:55:20.241+0000 7fda7a602700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.358626+0000 front 2023-07-14T11:54:52.358665+0000 (oldest deadline 2023-07-14T11:55:18.258786+0000) 2023-07-14T11:55:20.595 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:20.309+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:20.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:20.613+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:21.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:20 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:20.740+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:21.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:20 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:20.940+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:21.611 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:21.322+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:21.725 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:21 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:55:21.253+0000 7fda7a602700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.358626+0000 front 2023-07-14T11:54:52.358665+0000 (oldest deadline 2023-07-14T11:55:18.258786+0000) 2023-07-14T11:55:21.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:21.629+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:22.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:21 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:21.753+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:22.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:21 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:21.975+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:22.634 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:22.345+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:22.725 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:22 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3[114007]: 2023-07-14T11:55:22.271+0000 7fda7a602700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.358626+0000 front 2023-07-14T11:54:52.358665+0000 (oldest deadline 2023-07-14T11:55:18.258786+0000) 2023-07-14T11:55:22.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:22.653+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:23.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:22 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:22.766+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:23.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:22 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:22.929+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:23.605 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:23.365+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:23.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:23.624+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:24.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:23 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:23.768+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:24.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:23 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:23.926+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:24.226 INFO:journalctl@ceph.osd.3.smithi161.stdout:Jul 14 11:55:24 smithi161 bash[169444]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-3 2023-07-14T11:55:24.693 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:24.616+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:24.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:24.405+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:25.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:24 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:24.722+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:25.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:24 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:24.921+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:25.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:25.400+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:25.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:25.643+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:26.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.3.service' 2023-07-14T11:55:26.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:55:26.105 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-14T11:55:26.105 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-14T11:55:26.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.4 2023-07-14T11:55:26.135 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:25 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:25.704+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:26.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:25 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:25.938+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:26.398 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:55:26 smithi175 systemd[1]: Stopping Ceph osd.4 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:26.693 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:26 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:26.649+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:26.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:26 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:26.428+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:26.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:26 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:26.677+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:26.899 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:55:26 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:55:26.555+0000 7fc83bd68700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:26.900 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:55:26 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:55:26.555+0000 7fc83bd68700 -1 osd.4 149 *** Got signal Terminated *** 2023-07-14T11:55:26.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:55:26 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:55:26.555+0000 7fc83bd68700 -1 osd.4 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:27.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:26 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:26.923+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:27.693 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:27 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:27.618+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:27.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:27 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:27.618+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:27.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:27 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:27.426+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:27.868 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:27 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:27.544+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:28.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:27 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:27.703+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:28.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:27 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:27.886+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:28.644 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:28 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:28.400+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:28.919 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:28 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:28.573+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:28.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:28 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:28.662+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:28.944 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:28 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:28.662+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:29.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:28 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:28.753+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:29.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:28 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:28.933+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:29.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:29 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:29.404+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:29.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:29 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:29.637+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:29.695 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:29 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:29.637+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:29.876 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:29 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:29.598+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:30.148 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:55:29 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:55:29.740+0000 7fc83735c700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.092772+0000 front 2023-07-14T11:55:04.092820+0000 (oldest deadline 2023-07-14T11:55:28.793028+0000) 2023-07-14T11:55:30.149 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:29 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:29.742+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:30.149 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:29 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:29.742+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:30.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:29 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:29.893+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:30.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:29 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:29.893+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:30.623 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:30 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:30.368+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:30.649 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:30 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:30.399+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:30.893 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:30.567+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:30.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:30 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:30.641+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:30.944 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:30 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:30.641+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:31.148 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:55:30 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4[139715]: 2023-07-14T11:55:30.722+0000 7fc83735c700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.092772+0000 front 2023-07-14T11:55:04.092820+0000 (oldest deadline 2023-07-14T11:55:28.793028+0000) 2023-07-14T11:55:31.149 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:30 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:30.743+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:31.150 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:30 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:30.743+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:31.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:30.910+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:31.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:30.910+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:31.310 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:31 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:31.051+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:31.648 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:31 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:31.409+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:31.672 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:31 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:31.327+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:31.852 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:31 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:31.540+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:31.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:31 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:31.689+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:31.944 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:31 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:31.689+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:32.119 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:31 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:31.698+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:32.120 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:31 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:31.698+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:32.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:31 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:31.869+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:32.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:31 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:31.869+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:32.314 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:32 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:32.056+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:32.398 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jul 14 11:55:32 smithi175 bash[166647]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-4 2023-07-14T11:55:32.657 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:32 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:32.332+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:32.700 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:32 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:32.453+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:32.857 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:32 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:32.544+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:32.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:32 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:32.675+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:32.952 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:32 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:32.675+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:33.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:32 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:32.719+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:33.149 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:32 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:32.719+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:33.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:32 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:32.875+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:33.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:32 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:32.875+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:33.344 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:33 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:33.014+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:33.419 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.4.service' 2023-07-14T11:55:33.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:55:33.498 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-14T11:55:33.498 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-14T11:55:33.499 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.5 2023-07-14T11:55:33.650 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:33 smithi182 systemd[1]: Stopping Ceph osd.5 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:33.651 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:33 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:33.362+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:33.687 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:33 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:33.460+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:33.877 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:33.541+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:33.943 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:33 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:33.667+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:33.944 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:33 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:33.667+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:33.944 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:33 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:33.874+0000 7fa66d285700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:33.944 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:33 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:33.874+0000 7fa66d285700 -1 osd.5 149 *** Got signal Terminated *** 2023-07-14T11:55:33.945 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:33 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:33.874+0000 7fa66d285700 -1 osd.5 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:34.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:33 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:33.706+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:34.149 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:33 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:33.706+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:34.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:33.895+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:34.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:33.895+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:34.375 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:34 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:34.038+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:34.687 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:34 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:34.446+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:34.693 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:34 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:34.622+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:34.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:34 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:34.622+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:34.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:34 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:34.393+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:34.922 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:34 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:34.590+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:35.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:34 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:34.705+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:35.149 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:34 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:34.705+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:35.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:34 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:34.939+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:35.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:34 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:34.939+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:35.404 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:35 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:35.078+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:35.650 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:35 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:35.419+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:35.693 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:35 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:35.589+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:35.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:35 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:35.589+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:35.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:35 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:35.421+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:35.922 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:35 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:35.627+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:36.148 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:35 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:35.668+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:36.149 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:35 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:35.668+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:36.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:35 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:35.940+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:36.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:35 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:35.940+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:36.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:35 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:35.940+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:36.371 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:36 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:36.125+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:36.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:36 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:36.389+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:36.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:36 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:36.590+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:36.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:36 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:36.590+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:36.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:36 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:36.590+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.391422+0000 front 2023-07-14T11:55:13.391454+0000 (oldest deadline 2023-07-14T11:55:36.291574+0000) 2023-07-14T11:55:36.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:36 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:36.461+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:36.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:36 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:36.686+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:36.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:36 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:36.686+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:36.899 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:36 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:36.587+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:37.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:36 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:36.916+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:37.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:36 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:36.916+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:37.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:36 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:36.916+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:37.402 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:37 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:37.116+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:37.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:37 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:37.420+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:37.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:37 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:37.549+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:37.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:37 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:37.549+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:37.695 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:37 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:37.549+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.391422+0000 front 2023-07-14T11:55:13.391454+0000 (oldest deadline 2023-07-14T11:55:36.291574+0000) 2023-07-14T11:55:37.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:37 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:37.432+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:37.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:37 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:37.432+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:37.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:37 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:37.666+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:37.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:37 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:37.666+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:37.942 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:37 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:37.546+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:37.943 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:37 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:37.546+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:38.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:37 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:37.960+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:38.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:37 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:37.960+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:38.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:37 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:37.960+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:38.353 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:38 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:38.081+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:38.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:38 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:38.371+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:38.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:38 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:38.568+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:55.288493+0000 front 2023-07-14T11:54:55.288603+0000 (oldest deadline 2023-07-14T11:55:19.388577+0000) 2023-07-14T11:55:38.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:38 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:38.568+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.689886+0000 front 2023-07-14T11:55:01.689972+0000 (oldest deadline 2023-07-14T11:55:26.989940+0000) 2023-07-14T11:55:38.694 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:38 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5[140672]: 2023-07-14T11:55:38.568+0000 7fa66907a700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.391422+0000 front 2023-07-14T11:55:13.391454+0000 (oldest deadline 2023-07-14T11:55:36.291574+0000) 2023-07-14T11:55:38.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:38 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:38.424+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:38.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:38 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:38.424+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:38.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:38 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:38.648+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:38.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:38 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:38.648+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:38.973 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:38 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:38.526+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:38.973 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:38 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:38.526+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:39.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:38 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:38.990+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:39.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:38 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:38.990+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:39.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:38 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:38.990+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:39.335 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:39.061+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:39.335 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:39.061+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:39.695 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:39 smithi182 bash[169148]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-5 2023-07-14T11:55:39.696 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:39 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:39.352+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:39.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:39.438+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:39.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:39.438+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:39.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:39.682+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:39.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:39.682+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:39.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:39 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:39.682+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:39.974 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:39.531+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:39.974 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:39.531+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:40.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:39.992+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:40.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:39.992+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:40.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:39.992+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:40.366 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:40 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:40.025+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:40.366 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:40 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:40.025+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:40.647 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:40 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:40.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:40.889 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.5.service' 2023-07-14T11:55:40.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:40 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:40.477+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:40.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:40 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:40.477+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:40.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:40 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:40.686+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:40.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:40 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:40.686+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:40.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:40 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:40.686+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:40.915 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:40 smithi182 systemd[1]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.5.service: Succeeded. 2023-07-14T11:55:40.915 INFO:journalctl@ceph.osd.5.smithi182.stdout:Jul 14 11:55:40 smithi182 systemd[1]: Stopped Ceph osd.5 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:55:40.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:55:40.929 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-14T11:55:40.929 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-14T11:55:40.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.6 2023-07-14T11:55:40.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:40 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:40.554+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:40.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:40 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:40.554+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:41.193 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:41 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:41.046+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:41.194 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:41 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:41.046+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:41.437 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:41.007+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:41.437 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:41.007+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:41.437 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:41.007+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:41.438 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:41 smithi161 systemd[1]: Stopping Ceph osd.6 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:41.438 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:41.453+0000 7fa4dc612700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:41.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:41 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:41.381+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:41.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:41 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:41.381+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:41.725 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:41.453+0000 7fa4dc612700 -1 osd.6 149 *** Got signal Terminated *** 2023-07-14T11:55:41.725 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:41.453+0000 7fa4dc612700 -1 osd.6 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:41.725 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:41.562+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:41.726 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:41 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:41.562+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:41.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:41 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:41.509+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:41.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:41 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:41.509+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:41.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:41 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:41.677+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:41.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:41 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:41.677+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:41.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:41 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:41.677+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:42.390 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:42 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:42.061+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:42.391 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:42 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:42.062+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:42.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:42 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:42.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:42.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:42 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:42.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:42.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:42 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:42.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:42.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:42 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:42.408+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:42.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:42 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:42.408+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:42.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:42 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:42.535+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:42.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:42 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:42.535+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:42.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:42 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:42.634+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:42.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:42 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:42.634+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:42.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:42 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:42.634+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:42.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:42 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:42.604+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:42.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:42 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:42.604+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:43.429 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:43 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:43.086+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:43.429 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:43 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:43.086+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:43.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:43.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:43.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:43.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:43.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:43.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:43.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:43 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:43.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:43.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:43 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:43.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:43.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:43 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:43.495+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:43.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:43 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:43.495+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:43.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:43 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:43.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:43.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:43 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:43.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:43.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:43 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:43.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:43.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:43.568+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:43.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:43 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:43.568+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:44.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:44 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:44.098+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:44.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:44 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:44.098+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:44.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:44.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:44.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:44.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:44.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:44.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:44.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:44 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:44.521+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:44.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:44 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:44.521+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:44.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:44 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:44.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:44.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:44 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:44.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:44.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:44 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:44.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:44.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:44 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:44.473+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:44.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:44 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:44.473+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:44.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:44.548+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:44.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:44.548+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:44.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:44 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:44.548+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.246009+0000 front 2023-07-14T11:55:21.246249+0000 (oldest deadline 2023-07-14T11:55:44.146048+0000) 2023-07-14T11:55:45.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:45 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:45.116+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:45.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:45 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:45.116+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:45.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:45 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:45.116+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:45.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:45.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:45.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:45.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:45.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:45.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:45.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:45.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:45.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:45.516+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:45.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:45.516+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:45.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:45.516+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:45.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:45.661+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:45.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:45.661+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:45.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:45 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:45.661+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:45.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:45 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:45.490+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:45.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:45 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:45.490+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:45.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:45.586+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:02.841786+0000 front 2023-07-14T11:55:02.841462+0000 (oldest deadline 2023-07-14T11:55:27.541585+0000) 2023-07-14T11:55:45.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:45.586+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.444051+0000 front 2023-07-14T11:55:14.444008+0000 (oldest deadline 2023-07-14T11:55:36.743855+0000) 2023-07-14T11:55:45.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:45 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6[119560]: 2023-07-14T11:55:45.586+0000 7fa4d4c00700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.246009+0000 front 2023-07-14T11:55:21.246249+0000 (oldest deadline 2023-07-14T11:55:44.146048+0000) 2023-07-14T11:55:46.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:46.098+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:46.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:46.098+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:46.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:46.098+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:46.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:46 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:46.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:46.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:46 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:46.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:46.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:46 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:46.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:46.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:46 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:46.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:46.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:46 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:46.477+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:46.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:46 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:46.477+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:46.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:46 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:46.477+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:46.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:46 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:46.613+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:46.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:46 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:46.613+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:46.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:46 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:46.613+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:46.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:46.522+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:46.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:46.522+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:46.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:46 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:46.522+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:47.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:47 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:47.105+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:47.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:47 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:47.105+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:47.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:47 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:47.105+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:47.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:47 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:47.070+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:47.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:47 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:47.070+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:47.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:47 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:47.070+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:47.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:47 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:47.070+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:47.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:47.512+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:47.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:47.512+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:47.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:47.512+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:47.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:47.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:47.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:47.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:47.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:47.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:47.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:47 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:47.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:47.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:47 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:47.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:47.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:47 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:47.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:47.975 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jul 14 11:55:47 smithi161 bash[169719]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-6 2023-07-14T11:55:48.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:48 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:48.059+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:48.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:48 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:48.059+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:48.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:48 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:48.059+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:48.464 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:48 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:48.037+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:48.464 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:48 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:48.037+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:48.465 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:48 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:48.037+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:48.465 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:48 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:48.037+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:48.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:48 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:48.464+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:48.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:48 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:48.464+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:48.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:48 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:48.464+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:48.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:48 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:48.607+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:48.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:48 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:48.607+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:48.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:48 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:48.607+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:48.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:48 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:48.607+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:48.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:48 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:48.530+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:48.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:48 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:48.530+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:48.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:48 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:48.530+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:49.225 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:49 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:49.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:49.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:49 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:49.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:49.226 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:49 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:49.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:49.227 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:49 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:49.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:49.372 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.6.service' 2023-07-14T11:55:49.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:49 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:49.066+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:49.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:49 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:49.066+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:49.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:49 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:49.066+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:49.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:55:49.502 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-14T11:55:49.502 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-14T11:55:49.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.7 2023-07-14T11:55:49.849 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:49.648+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:49.849 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:49.648+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:49.850 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:49.648+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:49.850 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:49.648+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:49.851 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:49.494+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:49.851 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:49.494+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:49.852 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:49.494+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:49.852 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:49 smithi175 systemd[1]: Stopping Ceph osd.7 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:49.852 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:49.866+0000 7fb48ee48700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:49.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:49 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:49.550+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:49.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:49 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:49.550+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:49.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:49 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:49.550+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:50.148 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:49.866+0000 7fb48ee48700 -1 osd.7 149 *** Got signal Terminated *** 2023-07-14T11:55:50.148 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:49 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:49.866+0000 7fb48ee48700 -1 osd.7 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:50.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:50 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:50.063+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:50.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:50 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:50.063+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:50.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:50 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:50.063+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:50.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:50 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:50.046+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:50.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:50 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:50.046+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:50.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:50 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:50.046+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:50.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:50 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:50.046+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:50.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:50 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:50.506+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:50.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:50 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:50.506+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:50.900 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:50 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:50.506+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:50.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:50 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:50.682+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:50.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:50 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:50.682+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:50.901 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:50 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:50.682+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:50.901 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:50 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:50.682+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:50.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:50 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:50.506+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:50.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:50 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:50.506+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:50.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:50 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:50.506+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:51.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:51 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:51.099+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:51.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:51 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:51.099+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:51.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:51 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:51.099+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:51.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.018+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:51.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.018+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:51.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.018+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:51.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:51 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.018+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:51.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:51 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:51.499+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:51.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:51 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:51.499+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:51.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:51 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:51.499+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:51.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:51 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:51.645+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:51.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:51 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:51.645+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:51.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:51 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:51.645+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:51.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:51 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:51.645+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:51.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:51 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:51.547+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:51.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:51 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:51.547+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:51.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:51 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:51.547+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:52.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:52.074+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:52.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:52.074+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:52.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:52.074+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:52.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.999+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:52.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.999+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:52.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.999+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:52.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:51.999+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:52.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:52.548+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:52.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:52.548+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:52.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:52.548+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:52.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:52.619+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:52.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:52.619+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:52.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:52.619+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:52.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:52.619+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:52.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:52.576+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:52.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:52.576+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:52.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:52 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:52.576+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:53.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:53 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:53.060+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:53.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:53 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:53.060+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:53.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:53 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:53.060+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:53.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:52.996+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:53.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:52.996+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:53.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:52.996+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:53.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:52 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:52.996+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:53.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:53 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:53.565+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:53.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:53 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:53.565+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:53.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:53 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:53.565+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:53.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:53 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:53.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:53.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:53 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:53.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:53.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:53 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:53.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:53.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:53 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:53.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:53.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:53 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:53.589+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:53.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:53 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:53.589+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:53.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:53 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:53.589+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:54.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:54 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:54.013+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:54.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:54 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:54.013+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:54.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:54 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:54.013+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:54.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:54 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:54.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:54.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:54 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:54.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:54.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:54 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:54.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:54.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:54 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:54.011+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:54.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:54.562+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.201881+0000 front 2023-07-14T11:55:04.201906+0000 (oldest deadline 2023-07-14T11:55:29.502067+0000) 2023-07-14T11:55:54.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:54.562+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:12.403382+0000 front 2023-07-14T11:55:12.403221+0000 (oldest deadline 2023-07-14T11:55:37.103271+0000) 2023-07-14T11:55:54.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:54.562+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:22.404312+0000 front 2023-07-14T11:55:22.404258+0000 (oldest deadline 2023-07-14T11:55:45.204909+0000) 2023-07-14T11:55:54.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7[145143]: 2023-07-14T11:55:54.562+0000 7fb48943a700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:29.206349+0000 front 2023-07-14T11:55:29.206437+0000 (oldest deadline 2023-07-14T11:55:54.506476+0000) 2023-07-14T11:55:54.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:54.584+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:54.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:54.584+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:54.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:54.584+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:54.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:54.584+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:54.901 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:54 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:54.584+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:55:54.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:54 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:54.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:54.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:54 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:54.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:54.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:54 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:54.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:55.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:55 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:55.039+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:55.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:55 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:55.039+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:55.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:55 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:55.039+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:55.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:55 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:55.051+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:55.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:55 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:55.051+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:55.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:55 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:55.051+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:55.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:55 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:55.051+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:55.899 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jul 14 11:55:55 smithi175 bash[166917]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-7 2023-07-14T11:55:55.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:55 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:55.598+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:55.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:55 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:55.598+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:55.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:55 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:55.598+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:55.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:55 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:55.598+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:55.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:55 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:55.598+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:55:55.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:55 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:55.635+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:55.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:55 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:55.635+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:55.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:55 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:55.635+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:56.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:56 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:56.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:56.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:56 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:56.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:56.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:56 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:56.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:56.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:56 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:56.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.811992+0000 front 2023-07-14T11:55:30.811980+0000 (oldest deadline 2023-07-14T11:55:55.511811+0000) 2023-07-14T11:55:56.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:56 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:56.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:56.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:56 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:56.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:56.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:56 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:56.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:56.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:56 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:56.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:56.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:56 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:56.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:56.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:56 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:56.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:56.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:56 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:56.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:56.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:56 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:56.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:56.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:56 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:56.629+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:55:56.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:56 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:56.606+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:56.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:56 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:56.606+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:56.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:56 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:56.606+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:56.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.7.service' 2023-07-14T11:55:57.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:55:57.012 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-14T11:55:57.013 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-14T11:55:57.013 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.8 2023-07-14T11:55:57.379 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:57.041+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:57.380 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:57.041+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:57.380 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:57.041+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:57.380 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:57.041+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.811992+0000 front 2023-07-14T11:55:30.811980+0000 (oldest deadline 2023-07-14T11:55:55.511811+0000) 2023-07-14T11:55:57.380 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 systemd[1]: Stopping Ceph osd.8 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:55:57.380 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:57.397+0000 7fd4e6907700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:55:57.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:57 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:57.108+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:57.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:57 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:57.108+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:57.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:57 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:57.108+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:57.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:57 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:57.108+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:57.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:57 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:57.108+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:55:57.693 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:57.397+0000 7fd4e6907700 -1 osd.8 149 *** Got signal Terminated *** 2023-07-14T11:55:57.693 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:57.397+0000 7fd4e6907700 -1 osd.8 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:55:57.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:57.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:57.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:57.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:57.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:57.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:57.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:57 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:57.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:55:57.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:57 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:57.593+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:57.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:57 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:57.593+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:57.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:57 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:57.593+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:57.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:57 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:57.593+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:57.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:57 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:57.593+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:55:58.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:58.085+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:58.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:58.085+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:58.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:58.085+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:58.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:58.085+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.811992+0000 front 2023-07-14T11:55:30.811980+0000 (oldest deadline 2023-07-14T11:55:55.511811+0000) 2023-07-14T11:55:58.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:58 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:58.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:58.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:58 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:58.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:58.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:58 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:58.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:58.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:58 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:58.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:58.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:58 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:58.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:55:58.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:58 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:58.592+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:58.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:58 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:58.592+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:58.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:58 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:58.592+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:58.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:58 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:58.592+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:58.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:58 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:58.592+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:55:58.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:58.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:58.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:58.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:58.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:58.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:58.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:58 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:58.569+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:55:59.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:55:59.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:55:59.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:55:59.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.811992+0000 front 2023-07-14T11:55:30.811980+0000 (oldest deadline 2023-07-14T11:55:55.511811+0000) 2023-07-14T11:55:59.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:59 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:59.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:55:59.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:59 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:59.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:55:59.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:59 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:59.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:55:59.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:59 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:59.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:55:59.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:55:59 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:55:59.050+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:55:59.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:59 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:59.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:55:59.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:59 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:59.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:55:59.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:59 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:59.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:55:59.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:59 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:59.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:55:59.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:55:59 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:55:59.583+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:55:59.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:59.598+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:55:59.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:59.598+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:55:59.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:59.598+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:55:59.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:55:59.598+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:00.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.997+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:56:00.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.997+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:56:00.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.997+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:56:00.445 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:55:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:55:59.997+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.811992+0000 front 2023-07-14T11:55:30.811980+0000 (oldest deadline 2023-07-14T11:55:55.511811+0000) 2023-07-14T11:56:00.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:00.019+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:00.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:00.019+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:00.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:00.019+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:00.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:00.019+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:00.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:00 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:00.019+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:00.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:00.535+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:00.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:00.535+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:00.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:00.535+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:00.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:00.535+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:00.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:00 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:00.535+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:00.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:00 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:00.559+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:00.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:00 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:00.559+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:00.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:00 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:00.559+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:00.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:00 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:00.559+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:01.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:01.033+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:56:01.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:01.033+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:56:01.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:01.033+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:56:01.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:01.033+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.811992+0000 front 2023-07-14T11:55:30.811980+0000 (oldest deadline 2023-07-14T11:55:55.511811+0000) 2023-07-14T11:56:01.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:01 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:01.054+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:01.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:01 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:01.054+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:01.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:01 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:01.054+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:01.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:01 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:01.054+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:01.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:01 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:01.054+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:01.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:01 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:01.549+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:01.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:01 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:01.549+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:01.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:01 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:01.549+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:01.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:01 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:01.549+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:01.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:01 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:01.549+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:01.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:01.567+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:01.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:01.567+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:01.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:01.567+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:01.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:01 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:01.567+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:02.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:02.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:04.508548+0000 front 2023-07-14T11:55:04.508409+0000 (oldest deadline 2023-07-14T11:55:30.408439+0000) 2023-07-14T11:56:02.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:02.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.509173+0000 front 2023-07-14T11:55:14.509145+0000 (oldest deadline 2023-07-14T11:55:38.609343+0000) 2023-07-14T11:56:02.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:02.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:21.510337+0000 front 2023-07-14T11:55:21.510459+0000 (oldest deadline 2023-07-14T11:55:44.410546+0000) 2023-07-14T11:56:02.444 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8[146215]: 2023-07-14T11:56:02.040+0000 7fd4e0ef9700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.811992+0000 front 2023-07-14T11:55:30.811980+0000 (oldest deadline 2023-07-14T11:55:55.511811+0000) 2023-07-14T11:56:02.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:02.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:02.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:02.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:02.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:02.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:02.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:02.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:02.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:02.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:02.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:02 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:02.032+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:02.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:02 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:02.546+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:02.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:02 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:02.546+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:02.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:02 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:02.546+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:02.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:02 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:02.546+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:02.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:02 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:02.546+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:02.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:02 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:02.546+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:02.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:02.568+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:02.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:02.568+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:02.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:02.568+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:02.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:02 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:02.568+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:03.443 INFO:journalctl@ceph.osd.8.smithi182.stdout:Jul 14 11:56:03 smithi182 bash[169424]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-8 2023-07-14T11:56:03.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:03 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:03.047+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:03.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:03 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:03.047+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:03.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:03 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:03.047+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:03.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:03 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:03.047+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:03.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:03 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:03.047+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:03.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:03 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:03.047+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:03.833 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:03 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:03.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:03.834 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:03 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:03.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:03.834 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:03 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:03.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:03.835 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:03 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:03.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:03.835 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:03 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:03.590+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:03.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:03 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:03.542+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:03.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:03 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:03.542+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:03.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:03 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:03.542+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:03.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:03 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:03.542+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:03.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:03 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:03.542+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:03.901 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:03 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:03.542+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:04.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:04 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:04.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:04.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:04 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:04.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:04.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:04 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:04.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:04.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:04 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:04.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:04.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:04 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:04.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:04.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:04 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:04.060+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:04.541 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.8.service' 2023-07-14T11:56:04.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:04.582 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-14T11:56:04.582 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-14T11:56:04.583 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.9 2023-07-14T11:56:04.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:04 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:04.573+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:04.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:04 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:04.573+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:04.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:04 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:04.573+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:04.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:04 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:04.573+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:04.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:04 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:04.573+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:04.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:04 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:04.573+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:04.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:04 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:04.565+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:04.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:04 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:04.565+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:04.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:04 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:04.565+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:04.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:04 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:04.565+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:04.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:04 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:04.565+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:04.975 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:04 smithi161 systemd[1]: Stopping Ceph osd.9 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:05.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.086+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:05.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.086+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:05.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.086+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:05.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.086+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:05.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.086+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:05.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.086+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:05.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.108+0000 7f20707c7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:05.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.108+0000 7f20707c7700 -1 osd.9 149 *** Got signal Terminated *** 2023-07-14T11:56:05.477 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:05 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:05.108+0000 7f20707c7700 -1 osd.9 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:56:05.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:05 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:05.582+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:05.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:05 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:05.582+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:05.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:05 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:05.582+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:05.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:05 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:05.582+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:05.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:05 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:05.582+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:05.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:05 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:05.582+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:05.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:05 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:05.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:05.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:05 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:05.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:05.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:05 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:05.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:05.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:05 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:05.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:05.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:05 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:05.544+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:06.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:06.131+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:06.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:06.131+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:06.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:06.131+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:06.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:06.131+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:06.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:06.131+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:06.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:06 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:06.131+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:06.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:06.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:06.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:06.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:06.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:06.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:06.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:06.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:06.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:06.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:06.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:06 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:06.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:06.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:06.524+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:07.109 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:06.524+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:07.110 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:06.524+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:07.110 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:06.524+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:07.110 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:06 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:06.524+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:07.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:07.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:07.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:07.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:07.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:07.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:07.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:07.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:07.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:07.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:07.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:07 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:07.085+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:07.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:07.552+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:07.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:07.552+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:07.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:07.552+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:07.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:07.552+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:07.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:07.552+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:07.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:07.552+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:07.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:07 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:07.552+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:07.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:07.482+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:07.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:07.482+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:07.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:07.482+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:07.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:07.482+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:07.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:07 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:07.482+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:08.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:08.067+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:08.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:08.067+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:08.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:08.067+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:08.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:08.067+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:08.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:08.067+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:08.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:08 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:08.067+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:08.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:08.436+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:08.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:08.436+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:08.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:08.436+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:08.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:08.436+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:08.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:08.436+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:08.696 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:08.436+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:08.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:08.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:08.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:08.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:08.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:08.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:08.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:08.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:08.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:08.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:08.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:08.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:08.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:08 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:08.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:09.474 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:09.044+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:09.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:09.044+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:09.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:09.044+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:09.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:09.044+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:09.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:09.044+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:09.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:09 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:09.044+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:09.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:09.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:09.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:09.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:09.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:09.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:09.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:09.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:09.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:09.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:09.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:09 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:09.446+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:09.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:09.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:09.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:09.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:09.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:09.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:09.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:09.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:09.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:09.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:09.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:09.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:09.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:09 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:09.567+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:10.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:10.020+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:53.500154+0000 front 2023-07-14T11:54:53.500005+0000 (oldest deadline 2023-07-14T11:55:19.400190+0000) 2023-07-14T11:56:10.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:10.020+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:03.402181+0000 front 2023-07-14T11:55:03.402118+0000 (oldest deadline 2023-07-14T11:55:29.302140+0000) 2023-07-14T11:56:10.475 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:10.020+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:13.904108+0000 front 2023-07-14T11:55:13.903993+0000 (oldest deadline 2023-07-14T11:55:35.604162+0000) 2023-07-14T11:56:10.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:10.020+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.705325+0000 front 2023-07-14T11:55:19.705261+0000 (oldest deadline 2023-07-14T11:55:44.405231+0000) 2023-07-14T11:56:10.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:10.020+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:30.306315+0000 front 2023-07-14T11:55:30.306474+0000 (oldest deadline 2023-07-14T11:55:56.206395+0000) 2023-07-14T11:56:10.476 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:10 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9[125112]: 2023-07-14T11:56:10.020+0000 7f20695b6700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:38.507734+0000 front 2023-07-14T11:55:38.507935+0000 (oldest deadline 2023-07-14T11:56:01.407564+0000) 2023-07-14T11:56:10.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:10.563+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:10.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:10.563+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:10.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:10.563+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:10.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:10.563+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:10.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:10.563+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:10.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:10.563+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:10.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:10 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:10.563+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:10.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:10.495+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:10.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:10.495+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:10.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:10.495+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:10.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:10.495+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:10.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:10.495+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:10.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:10 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:10.495+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:11.474 INFO:journalctl@ceph.osd.9.smithi161.stdout:Jul 14 11:56:11 smithi161 bash[169986]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-9 2023-07-14T11:56:11.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:11.459+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:11.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:11.459+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:11.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:11.459+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:11.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:11.459+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:11.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:11.459+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:11.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:11 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:11.459+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:11.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:11.548+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:11.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:11.548+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:11.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:11.548+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:11.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:11.548+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:11.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:11.548+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:11.901 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:11.548+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:11.901 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:11 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:11.548+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:12.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:12.544+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:12.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:12.544+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:12.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:12.544+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:12.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:12.544+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:12.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:12.544+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:12.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:12.544+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:12.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:12 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:12.544+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:12.915 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.9.service' 2023-07-14T11:56:12.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:12.506+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:12.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:12.506+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:12.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:12.506+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:12.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:12.506+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:12.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:12.507+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:12.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:12.507+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:13.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:13.006 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-14T11:56:13.007 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-14T11:56:13.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.10 2023-07-14T11:56:13.370 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 systemd[1]: Stopping Ceph osd.10 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:13.648 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.387+0000 7f3000c79700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:13.648 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.387+0000 7f3000c79700 -1 osd.10 149 *** Got signal Terminated *** 2023-07-14T11:56:13.649 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.387+0000 7f3000c79700 -1 osd.10 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:56:13.649 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.551+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:13.649 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.551+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:13.649 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.551+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:13.649 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.551+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:13.650 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.551+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:13.650 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.551+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:13.650 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:13 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:13.551+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:13.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:13 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:13.519+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:13.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:13 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:13.519+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:13.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:13 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:13.519+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:13.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:13 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:13.519+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:13.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:13 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:13.519+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:13.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:13 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:13.519+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:14.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:14.561+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:14.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:14.561+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:14.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:14.561+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:14.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:14.561+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:14.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:14.561+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:14.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:14.561+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:14.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:14 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:14.561+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:14.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:14 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:14.514+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:14.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:14 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:14.514+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:14.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:14 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:14.514+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:14.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:14 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:14.514+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:14.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:14 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:14.514+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:14.946 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:14 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:14.514+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:15.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:15.600+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:15.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:15.600+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:15.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:15.600+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:15.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:15.600+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:15.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:15.600+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:15.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:15.600+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:15.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:15 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:15.600+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:15.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:15 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:15.526+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:15.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:15 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:15.526+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:15.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:15 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:15.526+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:15.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:15 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:15.526+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:15.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:15 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:15.526+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:15.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:15 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:15.526+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:16.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:16.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:16.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:16.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:16.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:16.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:16.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:16.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:16.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:16.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:16.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:16.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:16.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:16 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:16.601+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:16.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:16 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:16.539+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:16.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:16 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:16.539+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:16.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:16 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:16.539+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:16.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:16 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:16.539+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:16.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:16 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:16.539+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:16.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:16 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:16.539+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:17.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:17 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:17.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:52.576524+0000 front 2023-07-14T11:54:52.576633+0000 (oldest deadline 2023-07-14T11:55:18.476519+0000) 2023-07-14T11:56:17.898 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:17 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:17.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2023-07-14T11:55:01.377797+0000 front 2023-07-14T11:55:01.377817+0000 (oldest deadline 2023-07-14T11:55:29.578873+0000) 2023-07-14T11:56:17.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:17 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:17.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:14.780217+0000 front 2023-07-14T11:55:14.780058+0000 (oldest deadline 2023-07-14T11:55:38.880117+0000) 2023-07-14T11:56:17.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:17 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:17.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:19.381362+0000 front 2023-07-14T11:55:19.381410+0000 (oldest deadline 2023-07-14T11:55:48.181750+0000) 2023-07-14T11:56:17.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:17 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:17.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:28.682785+0000 front 2023-07-14T11:55:28.682806+0000 (oldest deadline 2023-07-14T11:55:54.582857+0000) 2023-07-14T11:56:17.899 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:17 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:17.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.483787+0000 front 2023-07-14T11:55:37.483923+0000 (oldest deadline 2023-07-14T11:56:02.084180+0000) 2023-07-14T11:56:17.900 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:17 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10[150722]: 2023-07-14T11:56:17.594+0000 7f2ffca6e700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:44.285901+0000 front 2023-07-14T11:55:44.285837+0000 (oldest deadline 2023-07-14T11:56:07.185671+0000) 2023-07-14T11:56:17.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:17 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:17.496+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:17.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:17 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:17.496+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:17.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:17 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:17.496+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:17.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:17 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:17.496+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:17.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:17 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:17.496+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:17.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:17 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:17.496+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:18.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:18.468+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:18.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:18.468+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:18.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:18.468+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:18.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:18.468+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:18.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:18.468+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:18.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:18.468+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:18.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:18 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:18.468+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:19.398 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jul 14 11:56:19 smithi175 bash[167197]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-10 2023-07-14T11:56:19.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:19.450+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:19.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:19.450+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:19.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:19.450+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:19.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:19.450+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:19.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:19.450+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:19.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:19.450+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:19.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:19 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:19.450+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:20.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.10.service' 2023-07-14T11:56:20.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:20.468 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-14T11:56:20.468 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-14T11:56:20.469 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.11 2023-07-14T11:56:20.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.423+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:20.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.423+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:20.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.423+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:20.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.423+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:20.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.423+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:20.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.423+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:20.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.423+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:20.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 systemd[1]: Stopping Ceph osd.11 for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:21.193 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.842+0000 7fe7bcdff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:21.193 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.842+0000 7fe7bcdff700 -1 osd.11 149 *** Got signal Terminated *** 2023-07-14T11:56:21.193 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:20 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:20.842+0000 7fe7bcdff700 -1 osd.11 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T11:56:21.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:21.433+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:21.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:21.433+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:21.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:21.433+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:21.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:21.433+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:21.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:21.433+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:21.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:21.433+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:21.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:21 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:21.433+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:22.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:22.401+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:22.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:22.401+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:22.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:22.401+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:22.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:22.401+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:22.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:22.401+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:22.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:22.401+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:22.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:22 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:22.401+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:23.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:23.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:23.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:23.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:23.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:23.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:23.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:23.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:23.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:23.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:23.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:23.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:23.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:23 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:23.382+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:24.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:24.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:24.693 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:24.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:24.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:24.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:24.694 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:24.695 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:24 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:24.419+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6820 osd.8 since back 2023-07-14T11:55:57.617578+0000 front 2023-07-14T11:55:57.617537+0000 (oldest deadline 2023-07-14T11:56:23.517598+0000) 2023-07-14T11:56:25.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6806 osd.0 since back 2023-07-14T11:54:54.409955+0000 front 2023-07-14T11:54:54.409920+0000 (oldest deadline 2023-07-14T11:55:20.210257+0000) 2023-07-14T11:56:25.944 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6804 osd.2 since back 2023-07-14T11:55:11.812808+0000 front 2023-07-14T11:55:11.812850+0000 (oldest deadline 2023-07-14T11:55:40.613518+0000) 2023-07-14T11:56:25.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6814 osd.3 since back 2023-07-14T11:55:20.614086+0000 front 2023-07-14T11:55:20.614065+0000 (oldest deadline 2023-07-14T11:55:46.513973+0000) 2023-07-14T11:56:25.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2023-07-14T11:55:31.214912+0000 front 2023-07-14T11:55:31.214963+0000 (oldest deadline 2023-07-14T11:55:57.114883+0000) 2023-07-14T11:56:25.945 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6812 osd.5 since back 2023-07-14T11:55:37.115424+0000 front 2023-07-14T11:55:37.115328+0000 (oldest deadline 2023-07-14T11:56:03.015393+0000) 2023-07-14T11:56:25.946 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.161:6822 osd.6 since back 2023-07-14T11:55:43.016028+0000 front 2023-07-14T11:55:43.016093+0000 (oldest deadline 2023-07-14T11:56:08.315856+0000) 2023-07-14T11:56:25.946 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2023-07-14T11:55:52.917214+0000 front 2023-07-14T11:55:52.917189+0000 (oldest deadline 2023-07-14T11:56:17.617182+0000) 2023-07-14T11:56:25.947 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:25 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11[151785]: 2023-07-14T11:56:25.467+0000 7fe7b6bf0700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.182:6820 osd.8 since back 2023-07-14T11:55:57.617578+0000 front 2023-07-14T11:55:57.617537+0000 (oldest deadline 2023-07-14T11:56:23.517598+0000) 2023-07-14T11:56:26.943 INFO:journalctl@ceph.osd.11.smithi182.stdout:Jul 14 11:56:26 smithi182 bash[169697]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-osd-11 2023-07-14T11:56:28.063 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@osd.11.service' 2023-07-14T11:56:28.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:28.103 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-14T11:56:28.104 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-14T11:56:28.104 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.a 2023-07-14T11:56:28.475 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:56:28 smithi161 systemd[1]: Stopping Ceph mds.a for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:28.975 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:56:28 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-a[130438]: 2023-07-14T11:56:28.713+0000 7f2046b18700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:28.975 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:56:28 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-a[130438]: 2023-07-14T11:56:28.713+0000 7f2046b18700 -1 mds.a *** got signal Terminated *** 2023-07-14T11:56:30.224 INFO:journalctl@ceph.mds.a.smithi161.stdout:Jul 14 11:56:29 smithi161 bash[170252]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-a 2023-07-14T11:56:30.337 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.a.service' 2023-07-14T11:56:30.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:30.424 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-14T11:56:30.424 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-14T11:56:30.425 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.d 2023-07-14T11:56:30.724 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:56:30 smithi161 systemd[1]: Stopping Ceph mds.d for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:31.225 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:56:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-d[129430]: 2023-07-14T11:56:30.929+0000 7f13a085a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:31.225 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:56:30 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-d[129430]: 2023-07-14T11:56:30.929+0000 7f13a085a700 -1 mds.d *** got signal Terminated *** 2023-07-14T11:56:32.974 INFO:journalctl@ceph.mds.d.smithi161.stdout:Jul 14 11:56:32 smithi161 bash[170398]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-d 2023-07-14T11:56:33.280 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.d.service' 2023-07-14T11:56:33.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:33.365 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-14T11:56:33.366 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-14T11:56:33.367 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.g 2023-07-14T11:56:33.729 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:56:33 smithi161 systemd[1]: Stopping Ceph mds.g for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:34.224 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:56:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-g[130049]: 2023-07-14T11:56:33.879+0000 7fa16b253700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:34.225 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:56:33 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-g[130049]: 2023-07-14T11:56:33.879+0000 7fa16b253700 -1 mds.g *** got signal Terminated *** 2023-07-14T11:56:38.669 INFO:journalctl@ceph.mds.g.smithi161.stdout:Jul 14 11:56:38 smithi161 bash[170539]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-g 2023-07-14T11:56:38.690 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.g.service' 2023-07-14T11:56:38.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:38.774 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-14T11:56:38.774 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-14T11:56:38.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.j 2023-07-14T11:56:38.975 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:56:38 smithi161 systemd[1]: Stopping Ceph mds.j for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:39.724 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:56:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-j[129047]: 2023-07-14T11:56:39.336+0000 7f6be2a65700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:39.725 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:56:39 smithi161 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-j[129047]: 2023-07-14T11:56:39.336+0000 7f6be2a65700 -1 mds.j *** got signal Terminated *** 2023-07-14T11:56:42.087 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:56:41 smithi161 bash[170684]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-j 2023-07-14T11:56:42.281 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.j.service' 2023-07-14T11:56:42.341 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:56:42 smithi161 systemd[1]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.j.service: Succeeded. 2023-07-14T11:56:42.341 INFO:journalctl@ceph.mds.j.smithi161.stdout:Jul 14 11:56:42 smithi161 systemd[1]: Stopped Ceph mds.j for 2bf7fc3c-2237-11ee-9b32-001a4aab830c. 2023-07-14T11:56:42.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:42.363 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-14T11:56:42.363 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-14T11:56:42.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.b 2023-07-14T11:56:42.648 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:56:42 smithi175 systemd[1]: Stopping Ceph mds.b for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:43.148 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:56:42 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: 2023-07-14T11:56:42.799+0000 7f7dc0c61700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:43.148 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:56:42 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b[154575]: 2023-07-14T11:56:42.799+0000 7f7dc0c61700 -1 mds.b *** got signal Terminated *** 2023-07-14T11:56:47.398 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jul 14 11:56:47 smithi175 bash[167469]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-b 2023-07-14T11:56:47.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.b.service' 2023-07-14T11:56:47.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:47.464 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-14T11:56:47.465 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-14T11:56:47.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.e 2023-07-14T11:56:47.845 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:56:47 smithi175 systemd[1]: Stopping Ceph mds.e for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:48.148 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:56:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-e[153890]: 2023-07-14T11:56:47.864+0000 7fc1e6185700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:48.148 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:56:47 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-e[153890]: 2023-07-14T11:56:47.864+0000 7fc1e6185700 -1 mds.e *** got signal Terminated *** 2023-07-14T11:56:51.790 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jul 14 11:56:51 smithi175 bash[167626]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-e 2023-07-14T11:56:51.795 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.e.service' 2023-07-14T11:56:51.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:51.829 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-14T11:56:51.830 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-14T11:56:51.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.h 2023-07-14T11:56:52.148 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:56:51 smithi175 systemd[1]: Stopping Ceph mds.h for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:52.647 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:56:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-h[154234]: 2023-07-14T11:56:52.221+0000 7f914268e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:52.648 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:56:52 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-h[154234]: 2023-07-14T11:56:52.221+0000 7f914268e700 -1 mds.h *** got signal Terminated *** 2023-07-14T11:56:55.648 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jul 14 11:56:55 smithi175 bash[167780]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-h 2023-07-14T11:56:55.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.h.service' 2023-07-14T11:56:55.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:55.692 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-14T11:56:55.693 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-14T11:56:55.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.k 2023-07-14T11:56:56.068 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:56:55 smithi175 systemd[1]: Stopping Ceph mds.k for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:56.397 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:56:56 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-k[153550]: 2023-07-14T11:56:56.086+0000 7f9832c87700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:56.398 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:56:56 smithi175 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-k[153550]: 2023-07-14T11:56:56.086+0000 7f9832c87700 -1 mds.k *** got signal Terminated *** 2023-07-14T11:56:59.126 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jul 14 11:56:58 smithi175 bash[167923]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-k 2023-07-14T11:56:59.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.k.service' 2023-07-14T11:56:59.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:56:59.164 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-14T11:56:59.165 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-14T11:56:59.165 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.c 2023-07-14T11:56:59.526 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:56:59 smithi182 systemd[1]: Stopping Ceph mds.c for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:56:59.942 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:56:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-c[155485]: 2023-07-14T11:56:59.545+0000 7fba411f8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:56:59.943 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:56:59 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-c[155485]: 2023-07-14T11:56:59.545+0000 7fba411f8700 -1 mds.c *** got signal Terminated *** 2023-07-14T11:57:03.693 INFO:journalctl@ceph.mds.c.smithi182.stdout:Jul 14 11:57:03 smithi182 bash[169971]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-c 2023-07-14T11:57:03.756 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.c.service' 2023-07-14T11:57:03.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:57:03.794 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-14T11:57:03.794 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-14T11:57:03.794 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.f 2023-07-14T11:57:04.175 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:57:03 smithi182 systemd[1]: Stopping Ceph mds.f for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:57:04.442 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:57:04 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-f[155147]: 2023-07-14T11:57:04.193+0000 7f9522460700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:57:04.443 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:57:04 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-f[155147]: 2023-07-14T11:57:04.193+0000 7f9522460700 -1 mds.f *** got signal Terminated *** 2023-07-14T11:57:07.540 INFO:journalctl@ceph.mds.f.smithi182.stdout:Jul 14 11:57:07 smithi182 bash[170115]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-f 2023-07-14T11:57:07.724 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.f.service' 2023-07-14T11:57:07.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:57:07.759 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-14T11:57:07.759 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-14T11:57:07.760 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.i 2023-07-14T11:57:08.165 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:57:07 smithi182 systemd[1]: Stopping Ceph mds.i for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:57:08.442 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:57:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-i[155827]: 2023-07-14T11:57:08.183+0000 7f5861b31700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:57:08.443 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:57:08 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-i[155827]: 2023-07-14T11:57:08.183+0000 7f5861b31700 -1 mds.i *** got signal Terminated *** 2023-07-14T11:57:11.506 INFO:journalctl@ceph.mds.i.smithi182.stdout:Jul 14 11:57:11 smithi182 bash[170255]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-i 2023-07-14T11:57:11.692 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.i.service' 2023-07-14T11:57:11.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:57:11.727 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-14T11:57:11.727 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-14T11:57:11.728 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.l 2023-07-14T11:57:11.943 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:57:11 smithi182 systemd[1]: Stopping Ceph mds.l for 2bf7fc3c-2237-11ee-9b32-001a4aab830c... 2023-07-14T11:57:12.443 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:57:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-l[154808]: 2023-07-14T11:57:12.115+0000 7f2f0a58a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:57:12.443 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:57:12 smithi182 ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-l[154808]: 2023-07-14T11:57:12.115+0000 7f2f0a58a700 -1 mds.l *** got signal Terminated *** 2023-07-14T11:57:16.638 INFO:journalctl@ceph.mds.l.smithi182.stdout:Jul 14 11:57:16 smithi182 bash[170395]: ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c-mds-l 2023-07-14T11:57:16.802 DEBUG:teuthology.orchestra.run.smithi182:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2bf7fc3c-2237-11ee-9b32-001a4aab830c@mds.l.service' 2023-07-14T11:57:16.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T11:57:16.836 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-14T11:57:16.837 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c --force --keep-logs 2023-07-14T11:57:20.507 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c --force --keep-logs 2023-07-14T11:57:22.568 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c --force --keep-logs 2023-07-14T11:57:24.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:57:24.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:57:24.851 DEBUG:teuthology.orchestra.run.smithi182:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:57:24.881 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-14T11:57:24.883 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi161/crash 2023-07-14T11:57:24.884 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash -- . 2023-07-14T11:57:25.030 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T11:57:25.031 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2023-07-14T11:57:25.034 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi175/crash 2023-07-14T11:57:25.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash -- . 2023-07-14T11:57:25.169 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T11:57:25.169 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2023-07-14T11:57:25.171 DEBUG:teuthology.misc:Transferring archived files from smithi182:/var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi182/crash 2023-07-14T11:57:25.172 DEBUG:teuthology.orchestra.run.smithi182:> sudo tar c -f - -C /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash -- . 2023-07-14T11:57:25.238 INFO:teuthology.orchestra.run.smithi182.stderr:tar: /var/lib/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T11:57:25.239 INFO:teuthology.orchestra.run.smithi182.stderr:tar: Error is not recoverable: exiting now 2023-07-14T11:57:25.240 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-14T11:57:25.241 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-14T11:57:25.323 INFO:teuthology.orchestra.run.smithi161.stderr:grep: /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph.log: No such file or directory 2023-07-14T11:57:25.329 INFO:tasks.cephadm:Compressing logs... 2023-07-14T11:57:25.329 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T11:57:25.334 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T11:57:25.337 DEBUG:teuthology.orchestra.run.smithi182:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T11:57:25.381 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose : No such file or directory-- 2023-07-14T11:57:25.382 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log 2023-07-14T11:57:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.10.log 2023-07-14T11:57:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.h.log 2023-07-14T11:57:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.e.log 2023-07-14T11:57:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.4.log 2023-07-14T11:57:25.391 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.1.log 2023-07-14T11:57:25.391 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.k.log 2023-07-14T11:57:25.392 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log 2023-07-14T11:57:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.7.log 2023-07-14T11:57:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.b.log 2023-07-14T11:57:25.396 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.b.log 2023-07-14T11:57:25.396 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.y.log 2023-07-14T11:57:25.397 INFO:teuthology.orchestra.run.smithi182.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-14T11:57:25.398 INFO:teuthology.orchestra.run.smithi182.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T11:57:25.398 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.b.log: /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.k.log.gz 2023-07-14T11:57:25.399 INFO:teuthology.orchestra.run.smithi175.stderr: 92.8% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.y.log.gz 2023-07-14T11:57:25.400 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2023-07-14T11:57:25.400 INFO:teuthology.orchestra.run.smithi161.stderr:-5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T11:57:25.401 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.0.log 2023-07-14T11:57:25.402 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.g.log 2023-07-14T11:57:25.403 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.a.log 2023-07-14T11:57:25.405 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.6.log 2023-07-14T11:57:25.408 INFO:teuthology.orchestra.run.smithi182.stderr:gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.5.log 2023-07-14T11:57:25.409 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.2.log 2023-07-14T11:57:25.409 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.11.log 2023-07-14T11:57:25.409 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.z.log 2023-07-14T11:57:25.410 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.8.log 2023-07-14T11:57:25.410 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.f.log 2023-07-14T11:57:25.410 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.c.log 2023-07-14T11:57:25.411 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.i.log 2023-07-14T11:57:25.411 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.c.log 2023-07-14T11:57:25.411 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log 2023-07-14T11:57:25.412 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.l.log 2023-07-14T11:57:25.413 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.3.log 2023-07-14T11:57:25.414 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.d.log 2023-07-14T11:57:25.414 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.3.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T11:57:25.415 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.x.log 2023-07-14T11:57:25.415 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.9.log 2023-07-14T11:57:25.416 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.x.log: 94.5% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.g.log.gz 2023-07-14T11:57:25.416 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log 2023-07-14T11:57:25.417 INFO:teuthology.orchestra.run.smithi161.stderr: 94.5% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.a.log.gz 2023-07-14T11:57:25.418 INFO:teuthology.orchestra.run.smithi182.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log: /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.l.log.gz 2023-07-14T11:57:25.419 INFO:teuthology.orchestra.run.smithi182.stderr: 92.9% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.z.log.gz 2023-07-14T11:57:25.420 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.9.log: /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log: 94.3% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.d.log.gz 2023-07-14T11:57:25.420 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.j.log 2023-07-14T11:57:25.421 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.a.log 2023-07-14T11:57:25.430 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.j.log: /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.a.log: 91.6% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log.gz 2023-07-14T11:57:25.431 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-07-14T11:57:25.432 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.log: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-07-14T11:57:25.432 INFO:teuthology.orchestra.run.smithi161.stderr: 94.5% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.j.log.gz 2023-07-14T11:57:25.465 INFO:teuthology.orchestra.run.smithi175.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T11:57:25.844 INFO:teuthology.orchestra.run.smithi175.stderr: 91.6% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log.gz 2023-07-14T11:57:26.017 INFO:teuthology.orchestra.run.smithi161.stderr: 89.8% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mgr.x.log.gz 2023-07-14T11:57:26.230 INFO:teuthology.orchestra.run.smithi182.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T11:57:26.739 INFO:teuthology.orchestra.run.smithi182.stderr: 91.6% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-volume.log.gz 2023-07-14T11:57:27.727 INFO:teuthology.orchestra.run.smithi182.stderr: 91.9% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.c.log.gz 2023-07-14T11:57:29.479 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.a.log.gz 2023-07-14T11:57:29.949 INFO:teuthology.orchestra.run.smithi175.stderr: 92.0% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mon.b.log.gz 2023-07-14T11:58:04.808 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.0.log.gz 2023-07-14T11:58:07.650 INFO:teuthology.orchestra.run.smithi182.stderr: 95.2% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.c.log.gz 2023-07-14T11:58:08.207 INFO:teuthology.orchestra.run.smithi182.stderr: 95.4% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.f.log.gz 2023-07-14T11:58:13.024 INFO:teuthology.orchestra.run.smithi161.stderr: 91.2% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.6.log.gz 2023-07-14T11:58:16.032 INFO:teuthology.orchestra.run.smithi161.stderr: 91.2% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.3.log.gz 2023-07-14T11:58:17.304 INFO:teuthology.orchestra.run.smithi175.stderr: 95.3% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.e.log.gz 2023-07-14T11:58:20.744 INFO:teuthology.orchestra.run.smithi161.stderr: 91.1% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.9.log.gz 2023-07-14T11:58:20.748 INFO:teuthology.orchestra.run.smithi161.stderr: 2023-07-14T11:58:20.748 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m55.377s 2023-07-14T11:58:20.749 INFO:teuthology.orchestra.run.smithi161.stderr:user 3m2.849s 2023-07-14T11:58:20.749 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m14.707s 2023-07-14T11:59:06.498 INFO:teuthology.orchestra.run.smithi182.stderr: 91.3% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.11.log.gz 2023-07-14T11:59:23.811 INFO:teuthology.orchestra.run.smithi182.stderr: 91.0% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.8.log.gz 2023-07-14T11:59:39.666 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.10.log.gz 2023-07-14T11:59:39.811 INFO:teuthology.orchestra.run.smithi182.stderr: 91.3% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.2.log.gz 2023-07-14T11:59:39.874 INFO:teuthology.orchestra.run.smithi175.stderr: 91.5% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.1.log.gz 2023-07-14T11:59:56.686 INFO:teuthology.orchestra.run.smithi182.stderr: 91.0% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.5.log.gz 2023-07-14T12:00:09.649 INFO:teuthology.orchestra.run.smithi175.stderr: 91.3% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.4.log.gz 2023-07-14T12:00:19.768 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-osd.7.log.gz 2023-07-14T12:02:07.127 INFO:teuthology.orchestra.run.smithi182.stderr: 95.4% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.i.log.gz 2023-07-14T12:02:07.407 INFO:teuthology.orchestra.run.smithi182.stderr: 2023-07-14T12:02:07.407 INFO:teuthology.orchestra.run.smithi182.stderr:real 4m42.055s 2023-07-14T12:02:07.407 INFO:teuthology.orchestra.run.smithi182.stderr:user 6m9.713s 2023-07-14T12:02:07.407 INFO:teuthology.orchestra.run.smithi182.stderr:sys 0m15.405s 2023-07-14T12:05:34.381 INFO:teuthology.orchestra.run.smithi175.stderr: 96.1% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.h.log.gz 2023-07-14T12:06:44.356 INFO:teuthology.orchestra.run.smithi175.stderr: 94.3% -- replaced with /var/log/ceph/2bf7fc3c-2237-11ee-9b32-001a4aab830c/ceph-mds.b.log.gz 2023-07-14T12:06:44.377 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-07-14T12:06:44.377 INFO:teuthology.orchestra.run.smithi175.stderr:real 9m19.026s 2023-07-14T12:06:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:user 10m47.302s 2023-07-14T12:06:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m28.735s 2023-07-14T12:06:44.378 INFO:tasks.cephadm:Archiving logs... 2023-07-14T12:06:44.381 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi161/log 2023-07-14T12:06:44.382 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T12:07:09.574 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi175/log 2023-07-14T12:07:09.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T12:08:21.708 DEBUG:teuthology.misc:Transferring archived files from smithi182:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi182/log 2023-07-14T12:08:21.710 DEBUG:teuthology.orchestra.run.smithi182:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T12:09:06.960 INFO:tasks.cephadm:Removing cluster... 2023-07-14T12:09:06.961 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c --force 2023-07-14T12:09:08.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c --force 2023-07-14T12:09:10.680 DEBUG:teuthology.orchestra.run.smithi182:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2bf7fc3c-2237-11ee-9b32-001a4aab830c --force 2023-07-14T12:09:12.581 INFO:tasks.cephadm:Removing cephadm ... 2023-07-14T12:09:12.582 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T12:09:12.632 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T12:09:12.760 DEBUG:teuthology.orchestra.run.smithi182:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T12:09:12.795 INFO:tasks.cephadm:Teardown complete 2023-07-14T12:09:12.796 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-14T12:09:12.807 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-14T12:09:12.808 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T12:09:12.813 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T12:09:12.815 DEBUG:teuthology.orchestra.run.smithi182:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T12:09:12.971 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-14T12:09:12.981 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-14T12:09:12.981 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T12:09:12.986 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T12:09:12.989 DEBUG:teuthology.orchestra.run.smithi182:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T12:09:13.005 INFO:teuthology.orchestra.run.smithi182.stderr:bash: ntpq: command not found 2023-07-14T12:09:13.006 INFO:teuthology.orchestra.run.smithi175.stderr:bash: ntpq: command not found 2023-07-14T12:09:13.024 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T12:09:13.025 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-07-14T12:09:13.025 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +178us[ +178us] +/- 84ms 2023-07-14T12:09:13.025 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -1666us[-1666us] +/- 82ms 2023-07-14T12:09:13.025 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv03.front.sepia.ceph.com 3 6 377 20 +1260us[+1284us] +/- 68ms 2023-07-14T12:09:13.026 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T12:09:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-07-14T12:09:13.027 INFO:teuthology.orchestra.run.smithi182.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T12:09:13.027 INFO:teuthology.orchestra.run.smithi182.stdout:=============================================================================== 2023-07-14T12:09:13.028 INFO:teuthology.orchestra.run.smithi182.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +181us[ +181us] +/- 84ms 2023-07-14T12:09:13.028 INFO:teuthology.orchestra.run.smithi182.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1630us[-1630us] +/- 82ms 2023-07-14T12:09:13.028 INFO:teuthology.orchestra.run.smithi182.stdout:^* hv03.front.sepia.ceph.com 3 6 377 22 +1297us[+1318us] +/- 68ms 2023-07-14T12:09:13.028 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T12:09:13.032 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T12:09:13.032 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-07-14T12:09:13.032 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +187us[ +187us] +/- 84ms 2023-07-14T12:09:13.032 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 -1638us[-1638us] +/- 82ms 2023-07-14T12:09:13.033 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv03.front.sepia.ceph.com 3 6 377 19 +1290us[+1339us] +/- 68ms 2023-07-14T12:09:13.033 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T12:09:13.034 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-14T12:09:13.044 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-14T12:09:13.045 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-14T12:09:13.114 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T12:09:13.295 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T12:09:13.546 DEBUG:teuthology.orchestra.run.smithi182:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T12:09:13.707 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T12:09:13.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:09:13.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T12:09:13.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:09:13.825 DEBUG:teuthology.orchestra.run.smithi182:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T12:09:13.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:09:13.859 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-14T12:09:13.874 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-14T12:09:13.904 INFO:teuthology.task.internal:Duration was 4785.191423 seconds 2023-07-14T12:09:13.905 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-14T12:09:13.930 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-14T12:09:13.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T12:09:13.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T12:09:13.938 DEBUG:teuthology.orchestra.run.smithi182:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T12:09:14.011 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T12:09:14.019 INFO:teuthology.orchestra.run.smithi182.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T12:09:14.049 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T12:09:14.500 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-14T12:09:14.500 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T12:09:14.501 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T12:09:14.561 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2023-07-14T12:09:14.562 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T12:09:14.606 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi182.front.sepia.ceph.com 2023-07-14T12:09:14.607 DEBUG:teuthology.orchestra.run.smithi182:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T12:09:14.647 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-14T12:09:14.648 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T12:09:14.653 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T12:09:14.655 DEBUG:teuthology.orchestra.run.smithi182:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T12:09:14.756 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-14T12:09:14.757 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T12:09:14.762 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T12:09:14.764 DEBUG:teuthology.orchestra.run.smithi182:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T12:09:14.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-14T12:09:14.940 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-14T12:09:14.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T12:09:15.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T12:09:15.050 DEBUG:teuthology.orchestra.run.smithi182:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T12:09:15.079 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-14T12:09:15.090 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:09:15.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:09:15.097 DEBUG:teuthology.orchestra.run.smithi182:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:09:15.135 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2023-07-14T12:09:15.158 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2023-07-14T12:09:15.164 INFO:teuthology.orchestra.run.smithi182.stdout:kernel.core_pattern = core 2023-07-14T12:09:15.195 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:09:15.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:09:15.237 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:09:15.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:09:15.254 DEBUG:teuthology.orchestra.run.smithi182:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:09:15.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:09:15.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-14T12:09:15.282 INFO:teuthology.task.internal:Transferring archived files... 2023-07-14T12:09:15.284 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi161 2023-07-14T12:09:15.284 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T12:09:15.366 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi175 2023-07-14T12:09:15.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T12:09:15.404 DEBUG:teuthology.misc:Transferring archived files from smithi182:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411/remote/smithi182 2023-07-14T12:09:15.405 DEBUG:teuthology.orchestra.run.smithi182:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T12:09:15.442 INFO:teuthology.task.internal:Removing archive directory... 2023-07-14T12:09:15.443 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T12:09:15.448 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T12:09:15.450 DEBUG:teuthology.orchestra.run.smithi182:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T12:09:15.494 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-14T12:09:15.513 INFO:teuthology.task.internal:Not uploading archives. 2023-07-14T12:09:15.514 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-14T12:09:15.529 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-14T12:09:15.530 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T12:09:15.534 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T12:09:15.536 DEBUG:teuthology.orchestra.run.smithi182:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T12:09:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 12:09 /home/ubuntu/cephtest 2023-07-14T12:09:15.552 INFO:teuthology.orchestra.run.smithi182.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 12:09 /home/ubuntu/cephtest 2023-07-14T12:09:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 12:09 /home/ubuntu/cephtest 2023-07-14T12:09:15.576 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-14T12:09:15.588 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-14T12:09:15.655 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T12:09:15.687 DEBUG:teuthology.nuke:shortname: smithi161 2023-07-14T12:09:15.687 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T12:09:15.713 DEBUG:teuthology.nuke:shortname: smithi175 2023-07-14T12:09:15.713 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T12:09:15.717 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 10:27:35.757815', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T12:09:15.740 DEBUG:teuthology.nuke:shortname: smithi182 2023-07-14T12:09:15.741 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T12:09:15.746 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 10:27:35.756098', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T12:09:15.780 INFO:teuthology.orchestra.console:Power off smithi161 2023-07-14T12:09:15.781 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T12:09:15.797 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi182.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337411', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 10:27:35.759162', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T12:09:15.806 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:15.837 INFO:teuthology.orchestra.console:Power off smithi175 2023-07-14T12:09:15.838 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T12:09:15.861 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:15.882 INFO:teuthology.orchestra.console:Power off smithi182 2023-07-14T12:09:15.883 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T12:09:15.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:19.851 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:19.896 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:19.934 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:23.964 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:24.012 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:24.048 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:24.178 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2023-07-14T12:09:24.226 INFO:teuthology.orchestra.console:Power off for smithi175 completed 2023-07-14T12:09:28.166 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:09:28.379 INFO:teuthology.orchestra.console:Power off for smithi182 completed 2023-07-14T12:09:28.488 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 4785.1914229393005 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi161 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3fc0089cc1082313a98853d48b892acdc1385b8b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3745c23d90bb46c39b9ea03f6dc52fe6 status: fail success: false 2023-07-14T12:09:28.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T12:09:28.582 INFO:teuthology.run:FAIL